We are delighted to announce that Digital Roads supported PhD student, Percy Lam, has been awarded Best Paper at the DIFA: Deep Learning-based Information Fusion and Its Applications Workshop, affiliated to the 36th British Machine Vision Conference for his paper, 'Robust pothole Detection through Dual-pass RGB and Depth Fusion'.The paper, co-authored with Dr Weiwei Chen from UCL, Digital Roads Technical Director, Prof Lavindra de Silva and our Principal Investigator, Prof Ioannis Brilakis, looks at ways of improving the automated detection of potholes when undertaking road surveys through the fusion of data.
Abstract
Having potholes detected and repaired is essential for road maintenance and uninterrupted transport. Generations of researchers and engineers have been innovating on using computer vision and sensors to detect potholes, however, gaps in knowledge remain in the lack of robustness of detectors trained with RGB images in varied environments and the inability to harness depth in detecting potholes with monocular images. This research proposes a solution by fusing predictions made by RGB-trained detectors and monocular depth estimation.
The solution first orthorectifies the pavement from perspective RGB images. It then receives predictions from RGB-trained detectors and a depth detector enabled by DINOv2 independently. These predictions are subsequently fused by weighted bounding box fusion and have masks predicted by Segment Anything. Whereas RGB-trained detectors perform well in test sets within the training context, they show a drastic loss of performance in out-of-context situations, such as in images taken on different roads in more challenging environmental conditions. Fusing predictions with depth enhances F1 scores by 16% to 81% in out-of-context situations, reinforcing detection robustness. This solution paves the way for further research on expanding detection to motorways and overcoming shadows in images.
You can read the award-winning paper here: 'Robust pothole Detection through Dual-pass RGB and Depth Fusion'.