# Combined Use of 3D and HSI for the Classification of Printed Circuit Board Components
###### Abstract
The acquisition of knowledge enables a better understanding and allows to structure and simplify a problem. Research papers from the field of remote sensing, that use the advantage of knowledge can be found in [12; 13; 14; 15; 16; 17]. A comparison of deep learning and a knowledge-based method can be found in [18]. It shows that a rule-based method can even be better than machine learning-based methods. The classification of PCBs and the components on it is a difficult task, due to the complexity of PCBs, both in terms of structure and big variety of electronic components with small size and wide chemical composition [9]. In this article, we propose a general approach based on a combination of topological, depth and spatial information obtained from 3D sensors and physical information received from HSI. For the demonstration of this general approach, the previously described problem from the field of waste sorting is addressed. The 3D sensors and also HSI sensors have weaknesses and reach their limits in certain situations. A structured light scanner, for example, faces difficulties in detecting transparent objects, whereas HSI sensors are not affected by this. In contrast, HSI sensors have their limits with black pigmented objects. A well-known problem is the strong absorption of black colorants such as carbon black. Due to the strong absorption of light from the UV to the SWIR, there is no reflected light that can be detected by the sensor and thus no spectral information that can be used for classification [19; 20]. The combination is therefore intended to achieve mutual support among the data to improve the interpretation and processing. A combination of geometry and spectral data brings advantages that was shown for example in [21]. The authors used geometry in terms of inclination angles in combination with spectral data for the detection of disease symptoms on plants. Another example is given in [22] which aimed to use multispectral LiDAR data for land cover classification of an urban area. The general concept of the proposed method consists of merging the datasets followed by extracting features in the 3D and hyperspectral domains. Especially the fusion of datasets is a challenging task and poses different requirements (e.g., calibration of line sensors and methods for registration) depending on the sensor technologies used. Hyperspectral sensors with high spectral and spatial resolution are usually line sensors and differ in their imaging properties from frame-based sensors. Accordingly, adapted models are necessary to perform the calibration, which is a required step for the registration of the datasets. Methodologies for the registration of line scanning HSI sensors and 3D Laser Scanners in outdoor applications can be found, for example, in [23; 24; 25]. The strategy adopted in this article focuses on a laboratory application and follows a simple registration concept that can similarly be applied to an industrial production/sorting process. The basic steps proposed regarding the processing of the combined dataset can be easily generalizable or extended to other application fields. All objects typically have a shape and a material composition. This assures a generalization as 3D and HSI are in principle sensitive for features in these domains. The features obtained are next used as inputs for a rule-based approach. One of the main advantages of rule-based approaches is the simplicity. Once the knowledge on which rules are based has been worked out, conditions can be easily set up. Further advantages are the performance, the ability to handle redundant and irrelevant attributes and the flexible extensibility of rule sets [26]. Acquiring knowledge can seem effortful, but in view of the resulting advantages, it should be seen as a clear benefit which allows to structure and simplify problems by using expert knowledge. The rules used in this article are based on knowledge established for electronic waste sorting.