# Combined Use of 3D and HSI for the Classification of Printed Circuit Board Components
## 2 Materials and Methods
### Rules-Based Classification  
The third step of the proposed framework consists in defining rules to process 3D data and spectral data in order to classify individual 3D point clouds. Based on the existing dataset, it can be stated that the hyperspectral point cloud contains valuable geometric information that is implicitly represented by the spatial arrangement of the individual 3D point clouds. In addition to the geometric information, the dataset also contains valuable spectral information to describe the physical property of the objects. 3D point clouds are a representation of spatial 3D information, which can be described by geometric characteristics. The derivation of these geometric characteristics is done by considering the local neighborhood of each 3D point. Based on this neighborhood, invariant moments can be calculated for each point, representing the geometric properties of local 3D structures [29]. A representation of some 3D features is shown in Figure 7. The 3D  
Figure 5: Influence of shadow on the classification process. Here shadow occurs only on the neighboring area surrounding the object. (**a**) Classification of pixels values belonging to the class PS using spectral data. (**b**) Classification of pixels values belonging to the class PS after removal of the background. features tested in this article are nowadays commonly used in lidar data processing. They are based on normal vectors and eigenvalue-based 3D features like linearity, verticality, planarity, scattering, omnivariance, anisotropy, eigenentropy and change in curvature [30]. Figure 6: Examples of results of 3D point cloud clustering: (**a**) Background removed dataset; (**b**) Clustered objects; (**c**) PCB with electronic components and clustering result. The clustering is completed after removing the board points using a plane estimation method. Figure 7: Visualization of 3D features. Omnivariance (**left**), Verticality (**middle**) and Normal Change Rate (**right**). For the characterization of the spectral components, shape-describing features introduced in [31] are used. Materials can be uniquely identified by their spectral signature. Changes in material compositions lead to changes in the spectral signature and consequently to local changes in the geometric shape of these spectra. We have shown in [31] that curvature \(\mathcal{A}\) is a suitable geometric feature and allows a detailed description of spectral signatures. Mathematically, the curvature can be determined by Equation (1) using the 1 st and 2 nd derivative for each point \(\boldsymbol{P}(x(t),y(t))\) where \(x\) characterizes the wavelength (in nm), \(y\) the reflectance and t the sample value, respectively. \[\boldsymbol{\varkappa}=\frac{\left|\dot{x}\ddot{y}-\dot{y}\dot{x}\right|}{ \left(\dot{x}^{2}+\dot{y}^{2}\right)^{\frac{3}{2}}} \tag{1}\]  
The approach to determine the significant parameters for the description of the spectral shape is presented in Figure 8. It is based on the computation of extreme values of the 2 nd derivative. For a better quantification of absorption peaks, the spectral signatures are normalized by using Continuum Removal [32; 33]. In Figure 9, the normalized spectra and the calculated curvatures for polystyrene (PS) and polypropylene (PP) are shown, which express a clear difference in the spectral footprint of both materials. The length of the green and red lines indicates the strength of the curvature. In order to take only significant curvatures into account, a threshold value must be used. The red lines represent the significant bands with a curvature value higher than the defined threshold and are derived on the basis of the curvature points presented in magenta and red colored dots. The direction of the lines (positive or negative) indicates the type of curvature behavior (concave and convex curve) and is also an important source of information to describe the shape of spectral signatures.