## 3 Hierarchical Semantic Loss and Confidence Estimator for Generalized Zero-Shot Learning
### Zero-Shot Inference with Unseen Confidence Estimator
\[s_{uc}=2-max(softmax\left(\frac{f(x,\theta_{c})}{T}\right)) \tag{7}\]  
\[D_{u}(x_{q},\ y_{u})=\ \frac{\parallel x_{q}-y_{u}\parallel_{2}^{2}}{s_{uc}} \tag{8}\]  
Equation (7) shows the UCE to calculate the unseen confidence score \(s_{uc}\). Here, \(x\) denotes the visual data and \(f(\cdot,\theta_{c})\) denotes a pre-trained CNN model that classifies all seen-classes in the training dataset. The \(s_{uc}\) represents the confidence of unseen classes. Dividing \(f(\cdot,\theta_{c})\) by the temperature constant \(T\) in the Equation (7), the output distribution of softmax can be adjusted according to the purpose of specific tasks [36]. The largest value among the output values of the softmax function generally represents the probability of the classification result, and this paper considers this value as the confidence for the seen-class visual data. As the output value of softmax ranges between 0 and 1, we subtract this value from 2 for controlling the influence of \(s_{uc}\). It becomes closer to 1 as the softmax value of the pre-trained CNN model approaches to 1 (high confidence of seen class), and closer to 2 as the softmax value approaches to 0 (low confidence of seen class). Therefore, the range of \(s_{uc}\) is designed to be between 1 and 2. The generalized zero-shot inference using the proposed method is performed by reflecting the characteristics of the input data. Equation (8) shows how to calculate the distance between the query visual vector \(x_{q}\) and unseen-class semantic vector \(y_{u}\) by applying \(s_{uc}\). \(D_{u}\) is calculated using l2 norm and divided by the \(s_{uc}\) obtained from Equation (7). So, if the \(s_{uc}\) become larger, the distances between \(x_{q}\) and all unseen semantic vectors become smaller. Figure 4 shows an example where \(s_{uc}\) is applied in the zero-shot inference process. In Figure 4, the blue circles indicate embedded seen-class semantic vectors, the red triangles indicate an embedded unseen-class semantic vectors, and the green square indicates the embedded visual vector. The simplest way to classify labels for visual data is to use the nearest neighbor method. For this purpose, we calculate the distances between the visual vector and all semantic vectors. The left side of Figure 4 shows that the seen-class semantic vector for the unseen-class image is classified as the closest one. If the distances between unseen-class visual vector and unseen-class semantic vectors become relatively small by applying the confidence score as shown on the right side of Figure 4, it helps to perform correct classification. Figure 4: Example of zero-shot inference applying unseen confidence score. The proposed methodology adjusts the distances between the visual vector and the unseen-class semantic vector by using the \(s_{uc}\) obtained from UCE. An advantage of the proposed methodology is that the embedding networks and UCE are designed separately. In other words, the proposed methodology only needs to reflect the influence of \(s_{uc}\) through simple multiplication in the inference step without additional training to the embedding networks.