# Zero-Shot Image Classification with Rectified Embedding Vectors Using a Caption Generator
## 4 Experimental Results
### Experimental Results  
The CUB dataset [15] contains 200 bird species and 11,788 images divided into 150 training and 50 testing classes. In the training stage, 8855 image-description pairs corresponding to 150 classes were used. In the testing stage, 2933 images corresponding to 50 unseen classes were used. The experimental results for the CUB dataset are presented in Table 1. The proposed method surpasses other comparison results on rank-1 accuracy of zero-shot image classification. In addition, the experiment was conducted on the flower dataset [16], which includes images of various flowers and their corresponding descriptions and class information. The dataset has 8189 images divided into 82 training and 20 testing classes. In the training stage, 7034 image-description pairs corresponding to 82 classes were used. A total of 10 descriptions were provided for each image, and all sentences were used for training. In the testing stage, 1155 images corresponding to 20 unseen classes were used in the inference process. All input images were resized to 224 \(\times\) 224. Similar to the CUB dataset, Table 1 reveals that the proposed model achieves fair improvement in zero-shot classification accuracy. Additionally, the results of our proposed model also outperform most of the results of previous works under the zero-shot classification setting with predefined attributes [22; 26; 27].  
We demonstrated the effect of the rectified vector through ablation studies. In Table 2, the method using only image features means that the generated text feature is not combined with the image feature in the testing stage; thus, \(\alpha\) = 0 in Equation (5). Compared with that, the proposed method significantly improves classification performance, demonstrating the effect of the rectified vector. We also verify whether the generated caption created a semantically correct sentence. We set a situation where the method using only text features performs the classification task by using the condition \(\alpha\) = 1 in Equation (5). In this experiment, classification performance presents meaningful results, indicating that the image-captioning model generates sentences that semantically match the image.  
\begin{table}
\begin{tabular}{c c c} \hline \hline \multirow{2}{*}{**Representation Method**} & \multicolumn{2}{c}{**Rank-1 Accuracy (\%)**} \\ \cline{2-3}  & **CUB** & **Flower** \\ \hline Word CNN [8] & 51.0 & 60.7 \\ \hline Word CNN + RNN [8] & 56.8 & 65.6 \\ \hline DEM [7] & 58.3 & 60.9 \\ \hline IATV [10] & 61.5 & 68.9 \\ \hline CMPC + CMPM [9] & 64.3 & 68.4 \\ \hline TIMAM [11] & 67.7 & 70.6 \\ \hline Proposed method & 69.3 & 76.1 \\ (rectified vector) & \multicolumn{1}{c}{} & \\ \hline \hline \end{tabular}
\end{table}
Table 1: Experimental results on the dataset.  
One of the interesting aspects is that the performance of the generated texts affects the improvement of the combined performance. For the flower dataset with relatively good classification performance on the method using only text features, when the two pieces of information were combined, the improved performance rate considerably increased compared to the CUB dataset. Thus, the advanced image-captioning model that can make more accurate descriptions, such as OFA [37] and mPLUG [38], could improve the proposed method in terms of accuracy.