Spatio-Temporal Deep Learning-Based Methods for Defect Detection: An Industrial Application Study Case  
[PERSON]  
1 Institute of Computing (IComp), Federal University of Amazonas (UFAM), Manaus 69080-900, Brazil; [EMAIL_ADDRESS] (L.A.G.S. ); [EMAIL_ADDRESS] (N.S.F. ); [EMAIL_ADDRESS] (M.V. ); [EMAIL_ADDRESS] (R.G.) 2 Institute and Center for Development and Research in Software Technology (ICTS), Manaus 69080-900, Brazil; [EMAIL_ADDRESS] (L.A.); [EMAIL_ADDRESS] (Dr.F. ); [PERSON] (Ar.S.J. ); [EMAIL_ADDRESS] (Dr.O.A.J.S.) 3 TPV Technology Limited, Manaus 69080-900, Brazil; [EMAIL_ADDRESS] (A.P. ); [EMAIL_ADDRESS] (C.F.S.C. ); [EMAIL_ADDRESS] (R.J.S.B. ); [EMAIL_ADDRESS] (A.S.C.) Correspondence: [EMAIL_ADDRESS]  
[PERSON]  
1 Institute of Computing (IComp), Federal University of Amazonas (UFAM), Manaus 69080-900, Brazil; [EMAIL_ADDRESS] (L.A.G.S. ); [EMAIL_ADDRESS] (N.S.F. ); [EMAIL_ADDRESS] (M.V. ); [EMAIL_ADDRESS] (R.G.) 2 Institute and Center for Development and Research in Software Technology (ICTS), Manaus 69080-900, Brazil; [EMAIL_ADDRESS] (Dr.L.A. ); [EMAIL_ADDRESS] (Dr.F. ); [EMAIL_ADDRESS] (Ar.S.J. ); [EMAIL_ADDRESS] (Dr.O.A.J.S.) 3 TPV Technology Limited, Manaus 69080-900, Brazil; [EMAIL_ADDRESS] (A.P. ); [EMAIL_ADDRESS] (C.F.S.C. ); [EMAIL_ADDRESS] (A.P. ); [EMAIL_ADDRESS] (C.F.S.C. ); [EMAIL_ADDRESS] (A.S.C.) 3 TPV Technology Limited, Manaus 69080-9000, Brazil; agemilson.pinnetle@tpv-tech. defects in the selective laser melting of metal powders [3], and the defect classification of semiconductor fabrication using scanning electron microscope images [4]. However, we note that the literature has given significantly less attention to the investigation of this kind of problem using video data. Although different aspects addressed in works that investigate defect identification from images may be extremely useful when dealing with video, the latter poses unique challenges, especially when considering the spatio-temporal patterns of input video sequences. In addition, video data are difficult to represent and model due to their high dimensionality, the presence of noise, and the fact that each video segment may represent a wide variety of events. The problem investigated in this paper involves defect identification from video data. In the assembly line of a TV set manufacturer, the TV's digital decoders must be tested to ensure that no defects occur. For instance, some possible defects are: (1) mosaic--characterized by artifacts of a geometric pattern which may partly or completely block the video's frames; (2) freeze--corresponding to consecutive repetitions of the same frame; (3) frame loss--temporal jumps that skip more than a single frame at a time; and (4) black screen--when the complete darkening of the screen replaces one or more frames. In order to better illustrate these defects, Figure 1 shows two different frame sequences, one with no anomalies (Figure 1a) and another presenting all four defects (Figure 1b). Detecting defects in video may be considered a special case of video anomaly detection, since the objective of this task is usually to discriminate positive events from negative and rare ones. An anomaly is usually an outlier, a non-standard piece of data, such as defects in surfaces [5]. Anomaly detection may be employed in a wide range of applications, such as the identification of noisy signals [6], disease classification [7; 8; 9], and pest control through environment surveillance [10; 11]. Deep learning-based methods are considered the state-of-the-art in video anomaly detection [12]. According to [PERSON] et al. [12], among the categories of deep learning methods used for video anomaly detection, the most widely used are supervised and semi-supervised learning. In the first category, convolutional neural networks (CNNs) built with spatio-temporal layers--e.g., convolutional 3D or two-stream CNN--are successfully used as video descriptors to provide discriminative information when labeled data are available.