# Recognition and Location Algorithm for Pallets in Warehouses Using RGB-D Sensor
## 2 Methods
### Classifier Training
#### 2.3.2 Classifier Construction and Category Matrix Creation  
This study uses a data set with the aim of performing pixel classification. The data set contains 21 RGB images of three types of pallet taken at different distances. The dataset was collected and photographed with a Bumblebee XB3 camera (supplied by the Point Grey company) in the XiZhou agriculture material warehouse, XinTang, Guangdong. The dataset was annotated with Photoshop (Adobe Systems Incorporated). The pixels corresponding to goods, pallet, pallet hole, and the ground are represented as A, B, C, and D, respectively. Each category contains more than 20,000 pixels as the classifier sample and the sample size of each category is balanced. This study divides the training and testing into 80% and 20%, respectively.  
The input vector x is presented as x\({}_{ij}\), which contains \(R\), \(G\), \(B\), \(H\), and \(S\) color information of pixels, which is shown in Equation (1). The output is the category corresponding to each pixel, which is represented by numbers, as shown in Equation (2).  
\[\text{x}_{ij}=\begin{bmatrix}R\\ G\\ B\\ H\\ S\end{bmatrix} \tag{1}\]  
where \(i\) and \(j\) represent the pixel's column and row, respectively.  
\[c_{i}=\begin{cases}1&x_{i}\in A\\ 2&x_{i}\in B\\ 3&x_{i}\in C\\ 4&x_{i}\in D\end{cases} \tag{2}\]  
This study uses support vector machine (SVM) as the pixel classifier. SVM classifier is a supervised machine learning technique, and it is one of the most popular discriminative classifiers [46]. This classifier applies the kernel trick to maximum-margin hyperplanes to solve linear and nonlinear classification problems. A linear discriminant function is defined as shown in Equation (3).  
\[f(x)=sgn\Big{(}w^{T}*\text{x}+b^{*}\Big{)} \tag{3}\]  
where \(w^{T}\) and \(b^{*}\) are the optimal parameters of maximum-margin hyperplanes, which were obtained from Equation (4).  
\[\begin{array}{ll}argmin\limits_{w,b}\frac{1}{2}\|w\|^{2}\\ subject to&y_{ij}\big{(}w^{T}x_{ij}+b\big{)}\geq 1,\forall i,j\end{array} \tag{4}\]  
The classifier was trained with an SVM model from sklearn, which is an open source library.  
Different pallet loading scenarios are trained with corresponding different classification models. In the standardized warehouse, goods and pallets correspond to each other, and different pallets are generally used for different goods, as shown in Figure 3, so the target goods can be used as a priori information to determine the type of pallets selected and choose the corresponding classification model.  
The RGB images acquired by the RGB-D sensor are input to the SVM classification model to obtain the category corresponding to each pixel point. The categories are stored into the category matrix to realize the category information and spatial relationship storage. Figure 4 shows the result of the classification of Figure 3 into a category matrix, and the colors of different categories are the same as the templates.  
Element labels are represented with different colors. Green represents goods. Red represents pallets. Blue represents ground. Light blue represents pallet holes. \(s_{u}\) and \(M_{ij}\) represent the grid compressing unit size and the grid unit vector of category matrix, respectively (explained in Section 2.5 below).