# Wheel Hub Defects Image Recognition Base on Zero-Shot Learning
## 4 Method
### Structure Matching Strategy  
About visual-semantic mapping, both forward mapping and common mapping methods use the embedding function of samples and semantic descriptors, and embedding processing is learned by minimizing the similarity function between the embedded sample and the corresponding embedded semantic descriptor. It is only different in the selection of embedding methods and similarity functions. These methods usually use nearest neighbor search for classification after embedding, however, in a high-dimensional space, nearest neighbor search always suffers from the \"hubness\" phenomenon, because a certain number of data points will become the nearest neighbors or centers of almost all test points, leading to classification errors. However, if the reverse mapping strategy is adopted, that is, the mapping from the semantic descriptor space to the visual feature space will effectively avoid the \"hubness\" problem, therefore, reverse mapping strategy was adopted in this paper.  
We need to learn a mapping function \(f(\cdot)\) the semantic descriptor \(a_{i}\) to its corresponding image feature \(\varphi(x_{i})\). Where \(x_{i}\) is the image, and \(\varphi(\cdot)\) refers to the CNN architecture for extracting a high-dimensional feature map. The mapping function \(f(\cdot)\) is a fully connected neural network. In order to make the descriptors and image features close to each other, the least squares loss function was employed to minimize the difference. The initial objective function \(L_{1}\) is shown in Equation (1):  
\[L_{1}=\frac{1}{N_{tr}}\sum_{i=1}^{N_{tr}}\left\|f(a_{i})-\varphi(x_{i})\right\| ^{2}+\lambda_{\gamma}g(f) \tag{1}\]  
Figure 5: Generalized ZSR framework for wheel hub defect images.  
where \(g(\cdot)\) stands for normalized loss for \(f(\cdot)\). The loss function \(L_{1}\) was adopted to minimize the point-to-point difference between semantic descriptors and image features.  
In order to illustrate the structural matching between the semantic space and the feature space, we tried to minimize the pairing relationships between classes in these two spaces. Therefore, we constructed relational matrices \(D_{a}\) for semantic descriptors and image features, and each of these elements was derived from \(\left[D_{a}\right]_{uv}=\left\|f(a^{u})-f(a^{v})\right\|_{2}^{2}\), where \(a^{u}\) and \(a^{v}\) represent semantic descriptors of seen class defects u and v respectively. The image feature relationship matrix \(D_{\varphi}\) was built, and each element was calculated by the formula \(\left[D_{\varphi}\right]_{uv}=\left\|\varphi-\varphi\right\|_{2}^{2}\). Where, \(\varphi\) and \(\varphi\) represent the mean values of class \(u\) and \(v\) respectively, which was calculated from Equation (2)  
\[\varphi=\frac{1}{\left|y_{tr}^{u}\right|}\sum_{y_{i}\in y_{tr}^{u}}\varphi(x_ {i}) \tag{2}\]  
where the \(\sum(\cdot)\) is based on the defect type \(u\), and \(\left|y_{tr}^{u}\right|\) is the cardinality of the training set of defect type \(u\), and the same is true for defect \(v\).  
To achieve structure alignment, the structure alignment loss function \(L_{2}\) needs to be minimized.  
\[L_{2}=\left\|D_{a}-D_{\varphi}\right\|_{F}^{2} \tag{3}\]  
\(\left\|\cdot\right\|_{F}^{2}\) Where represents the Frobenius norm, and combined with the loss functions \(L_{1}\) and \(L_{2}\), we get the total loss \(L_{total}\), as Equation (4) shown.  
\[L_{total}=L_{1}+\rho L_{2} \tag{4}\]  
where \(\rho\geq 0\), is used to measure the loss weight of \(L_{2}\). \(L_{total}\) is to optimize the parameters of \(f(\cdot)\).