# Automating Visual Blockage Classification of Culverts with Deep Learning
## 1 Introduction  
Cross-drainage structures (e.g., culverts, bridges) are prone to blockage by debris and are reported as one of the leading causes of flash floods in urban areas [1, 2, 3, 4, 5, 6, 7]. The1998 and 2011 floods in Wollongong, Australia [1, 8, 9, 10, 11] and the 2007 floods in Newcastle, Australia [1, 12] are typical examples where blockage of cross drainage hydraulic structures contributed to triggering the flash flooding. Project 11: blockage of hydraulic structures [4] was initiated under the Australian rainfall and run-off (ARR) [13] framework to study the blockage behaviour and design considerations of hydraulic structures. Under this project, Wollongong City Council (WCC) proposed the guidelines to consider the hydraulic blockage in the hydraulic structures design process [2, 4, 14, 15, 16, 17]. However, because of the unavailability of relevant supporting data from peak flooding events, proposed guidelines were not adaptive and were based on the post flood visual assessments, which many researchers believe is not the correct representation of blockage during the peak flooding events [1, 2, 14]. The guidelines suggested that any culvert with an opening diagonal of 6 m or more is not prone to blockage. However, this claim was only supported by post flood visual assessments and was not considered economically efficient to implement. Initially, the blockage was defined as the percentage occlusion of hydraulic structure opening; however, many argued that hydraulic blockage and visual blockage are two separate concepts. The hydraulic blockage is more complex and has no established relationship with visual blockage. The hydraulic blockage is associated with the interaction of debris with the culvert and its corresponding effect on fluid dynamics around the culvert; however, due to the highly non-linear and uncertain behavior of debris, it is difficult to model and predict the hydraulic blockage using conventional means. Culvert blockage management is categorized under the broader \"The Smart Stormwater Management\" project [18] which aims to make use of data analytic and Internet of Things (IoT) for efficient stormwater management. Detection of blockage (i.e., StopBlock) to avoid flash floods is one of the critical components of this project. From a management and maintenance perspective, making use of multi-dimensional information (i.e., visual blockage status, type of debris material, percentage of blocked openings) extracted using computer vision algorithms may prove helpful in making timely decisions, as suggested in literature [7, 19]. As of now, to assess the visual blockage at culvert, manual visual inspections by flood management teams are performed to decide if a culvert needs maintenance towards avoiding the overtopping of flow and flash flooding. However, this process is inefficient in terms of required human resources and unsafe during peak flood events. This paper attempts to address the problem from a different perspective and proposes the use of visual information extracted using automated analysis in better management of blockage at cross drainage hydraulic structures and automating the process of manual culvert visual blockage status classification. This paper investigated the potential of convolutional neural network (CNN) algorithms towards classifying culvert images as \"clear\" or \"blocked\" as an automated solution for visual blockage inspections of culvert. Existing CNN models (i.e., DarkNet53 [20], DenseNet121 [21], InceptionResNetV2 [22], InceptionV3 [23], MobileNet [24], ResNet50 [25], VGG16 [26], EfficientNetB3 [27], NASNet [28]) pre-trained over ImageNet, and conventional machine learning approaches (i.e., SVM, RF, xgboost) were implemented for the culvert blockage classification task using data from three different sources (i.e., Images of culvert openings and blockage (ICOB), visual hydrology-lab dataset (VHD), synthetic images of culverts (SIC)), and performance was compared based on the standard evaluation measures.