Deep Active Learning for Civil Infrastructure Defect Detection and Classification

Automatic detection and classification of defects in infrastructure surface images can largely boost its maintenance efficiency. Given enough labeled images, various supervised learning methods have been investigated for this task, including decision trees and support vector machines in previous studies, and deep neural networks more recently. However, in real world applications, labels are harder to obtain than images, due to the limited labeling resources (i.e., experts). Thus we propose a deep active learning system to maximize the performance. A deep residual network is firstly designed for defect detection and classification in an image. Following our active learning strategy, this network is trained as soon as an initial batch of labeled images becomes available. It is then used to select a most informative subset of new images and query labels from experts to retrain the network. Experiments demonstrate more efficient performance improvements of our method than baselines, achieving 87.5% detection accuracy. International Workshop on Computing in Civil Engineering (IWCCE) This work may not be copied or reproduced in whole or in part for any commercial purpose. Permission to copy in whole or in part without payment of fee is granted for nonprofit educational and research purposes provided that all such whole or partial copies include the following: a notice that such copying is by permission of Mitsubishi Electric Research Laboratories, Inc.; an acknowledgment of the authors and individual contributions to the work; and all applicable portions of the copyright notice. Copying, reproduction, or republishing for any other purpose shall require a license with payment of fee to Mitsubishi Electric Research Laboratories, Inc. All rights reserved. Copyright c © Mitsubishi Electric Research Laboratories, Inc., 2017 201 Broadway, Cambridge, Massachusetts 02139 Deep Active Learning for Civil Infrastructure Defect Detection and Classification Chen Feng1, Ming-Yu Liu1, Chieh-Chi Kao2, and Teng-Yok Lee1 1Mitsubishi Electric Research Laboratories (MERL), 201 Broadway, Cambridge, MA 02139; email: {cfeng, mliu, tlee}@merl.com 2University of California, Santa Barbara; email: chiehchi.kao@gmail.com ABSTRACT Automatic detection and classification of defects in infrastructure surface images can largely boost its maintenance efficiency. Given enough labeled images, various supervised learning methods have been investigated for this task, including decision trees and support vector machines in previous studies, and deep neural networks more recently. However, in real world applications, labels are harder to obtain than images, due to the limited labeling resources (i.e., experts). Thus we propose a deep active learning system to maximize the performance. A deep residual network is firstly designed for defect detection and classification in an image. Following our active learning strategy, this network is trained as soon as an initial batch of labeled images becomes available. It is then used to select a most informative subset of new images and query labels from experts to retrain the network. Experiments demonstrate more efficient performance improvements of our method than baselines, achieving 87.5% detection accuracy.Automatic detection and classification of defects in infrastructure surface images can largely boost its maintenance efficiency. Given enough labeled images, various supervised learning methods have been investigated for this task, including decision trees and support vector machines in previous studies, and deep neural networks more recently. However, in real world applications, labels are harder to obtain than images, due to the limited labeling resources (i.e., experts). Thus we propose a deep active learning system to maximize the performance. A deep residual network is firstly designed for defect detection and classification in an image. Following our active learning strategy, this network is trained as soon as an initial batch of labeled images becomes available. It is then used to select a most informative subset of new images and query labels from experts to retrain the network. Experiments demonstrate more efficient performance improvements of our method than baselines, achieving 87.5% detection accuracy.