Dr Shirin Malihi

Strategic Themes 

Digital Twin

Title:  Maintenance of Road Digital Twin Using Multi-modal data

Theme: Digital Twin

Abstract:

The evolution of remote sensing and multi-sensor technologies has led to the generation of multi-modal data, facilitating diverse representations of a singular scene. Ground Penetrating Radar (GPR) employs electromagnetic pulses to probe the subsurface of road pavements. LiDAR captures 3D data along with the intensity of road assets. RGB cameras provide insights into colour, texture, and edge details. Thermal cameras detect infrared radiation emissions from roads. Fusing these data modalities yields a holistic understanding of scenes and offering precise platform and sensor orientations. The sensors used to capture different data modalities can be mounted on different platforms including aerial, hand-held, static and mobile.

While multi-modal data fusion offers significant potential, it also faces challenges due to inherent differences in coordinate systems across various sensors. Addressing these discrepancies typically begins with co-registration and alignment of data modalities, a topic widely explored in fields such as computer graphics, autonomous driving, and robotics, often using advanced learning-based techniques.

Despite this progress, there remains a gap in comprehensive literature on multi-modal data fusion within digital twins (DT) for road infrastructure, specifically considering the semantic aspects of data. LiDAR data, due to its high density and accuracy, serves as the primary modality for constructing geometric digital twins.

Other modalities can be fused with this modality to improve the quality of the outputted geometric DT. The strategy to fuse these modalities considering their level of abstraction and the quality parameter to evaluate it are subjects of this research. Different sources of unknowns can affect the data and the fusion model. Therefore, the quality parameters should be investigated carefully to fit the target of DT. Since the proposed solution is implemented on data collected from mobile ground platforms, it is essential to manage the algorithm's complexity to maintain efficiency.

By creating data-driven virtual models, the digital twins enable real-time monitoring, and predictive analysis to enhance the sustainability, efficiency, and longevity of infrastructure systems. Digital twins enable real-time analysis of traffic patterns, identifying and predicting congestion points and inefficient routes. By suggesting optimal routes or adjusting traffic signals dynamically, they reduce vehicle idling and fuel consumption, leading to lower emissions. Digital twins track the lifecycle of infrastructure components, prevents premature replacements, and reduces resource consumption and waste. Lifecycle assessments enable better understanding of environmental impacts, leading to more informed material choices toward reducing embodied carbon. Through continuous monitoring, they allow timely repairs, reducing the need for resource-intensive replacements. This results in fewer materials consumed over time and lowers the carbon footprint associated with construction and repair activities, which hasten reaching NetZero. Furthermore, by enhancing material reuse, asset adaptation and readaptation built environments, digital twin supports green infrastructure and circular economy.