# Dual-Branch Multi-Scale Relation Networks with Tutorial Learning for Few-Shot Learning
## 3 Methodology
### Tutoring Learning Module  
The relation network replaces the conventional measurement method based on predefined functions, enabling the model to learn an appropriate approach for the similarity metric. However, in few-shot learning problems with limited examples, it becomes crucial to consider every detail. Although employing relation networks separately for measuring the similarity of different hierarchical features can enhance model performance, it overlooks the distinctions and impacts of these different features. As the outcomes of deep features have a significant influence over the final classification, tutoring similarity metrics learning of shallow features from deep feature metrics is essential. To improve the capability of the relation networks model to measure feature pair similarity, we propose a tutoring learning module that utilizes the Branch 1 relation network to instruct the Branch 2 relation networks in learning similarity metrics.  
The tutorial learning module is inspired by knowledge distillation [57], which can improve the accuracy of network models. Combining with knowledge distillation techniques can further improve the measurement capabilities of relation networks. Specifically, we consider the relation network in Branch 1 as a teacher network due to the weight-sharing mechanism that enables it to acquire comprehensive deep feature similarity across multiple scales, thereby exhibiting strong measurement capabilities. Conversely, the relation networks in Branch 2 are regarded as student networks since they learned less or basic similarity knowledge, so they can benefit from the tutoring provided by Branch 1. We first sum the relation scores at each scale in Branch 1, which represents the knowledge learned about sample pair similarity. Then, we sum the relation scores in Branch 2, which denotes the knowledge learned by Branch 2:  
\[r^{Z}=\sum_{z=1}^{5}r^{z},r^{D}=\sum_{d=1}^{3}r^{d} \tag{5}\]  
Figure 4: Illustration of relation networks in Branch 2. The support image feature of a single category (red) is exemplified, while the other categories follow the same pattern by concatenating with the query image feature.  
Obviously, the weight-shared relation network in Branch 1 learns more comprehensive knowledge at different scales in deep layers, exhibiting robust recognition capabilities, while the relation networks in Branch 2 independently learn and primarily rely on shallow and fundamental features, serving as a supplementary component for final classification. Therefore, this paper proposes a tutoring learning module \(T\) to facilitate the transfer of knowledge acquired in Branch 1 to Branch 2, thereby enhancing the overall recognition capability. We leverage the learning summary \(r^{Z}\) from Branch 1 to generate a soft label for tutoring the similarity metric in Branch 2, incorporating hyperparameter \(t\) for appropriate regularization. The specific implementation is as follows:  
\[\mathcal{L}_{T}=\ell^{KL}\Big{(}softmax\Big{(}r^{Z}/t\Big{)},softmax\Big{(}r^ {D}/t\Big{)}\Big{)} \tag{6}\]  
where \(\ell^{KL}\) denotes the Kullback-Leibler divergence, and \(softmax(\cdot)\) indicates normalization. The loss function of the tutoring learning module is:  
\[\mathcal{L}_{KD}=\alpha\mathcal{L}_{D}+(1-\alpha)\mathcal{L}_{T} \tag{7}\]  
where \(\alpha\) denotes the weight hyperparameter, which is set to 0.3 in this paper to control the contribution of the two parts in the distillation loss. Therefore, the total loss of our model is composed of the losses of Branch 1, Branch 2 and the tutoring learning module, and can be formulated as follows:  
\[\mathcal{L}=\min_{\theta,\phi,\varphi}\left(\mathcal{L}_{Z}+\mathcal{L}_{D}+ \mathcal{L}_{KD}\right) \tag{8}\]  
where the ultimate classification result is determined by the sum of the relation scores of all scales and layers, as depicted below:  
\[r^{final}=r^{Z}+r^{D}/3 \tag{9}\]