# VL-Few: Vision Language Alignment for Multimodal Few-Shot Meta Learning
## 4 Multimodal Evaluation
### Results and Discussion  
The core problem is that the language model cannot see the vision information. For research on the multimodal few-shot problem, we compared three related methods by using the crossdomain dataset Real-Fast VQA based on 2-way 1-shot or 5-shots to evaluate the performance of our method, as shown in Table 1. Note that there are few research studies on this problem; thus, the compared results are not many. With or without _Meta_ means whether or not to use the meta learning method. The evaluation metric is accuracy (%), which can evaluate the understanding of visual information of the model and complete VQA tasks based on this. Frozen [56] freezes the language decoder and trains the vision encoder, which is also the baseline of this problem. MML [54] is a method that uses a mapper to convert the vision feature into language feature space, which trains by using the meta learning to construct a new training dataset. Our method VL-Few proposes to use the learnable vision prompt and vision-language prompt to carry out the vision information to make the model learn images, which could help the model understand the images that are the different modal data. Our training set sizes were \(T\times N\times K=4\times 2\times 1=8\) or \(T\times N\times K=4\times 2\times 5=40\), which are small samples for training. VL-Few reached the best accuracy in the 1-shot and 5-shots, which shows the feasibility and effectiveness of our method. More samples could provide more data and task knowledge, which could improve the model performance. Meta could further improve the accuracy of the model. We observed that our VL-Few reached the best accuracy in the 1-shot and 5-shots, even without multimodal few-shot meta training. This shows that our method could improve the modal understanding ability of the model by using multimodal alignment, and multimodal few-shot meta training could further improve the performance of the model. Specifically, the modal alignment method aligns the vision feature into the language space by mapping the vision feature into the language feature space, which improves the modal understanding, and multimodal few-shot meta training reduces the data dependence of the model by constructing the training set and training input, which improves its generalization ability.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline  & & **Accuracy(\%)** & \\ \hline
**Methods** & **Meta** & **1-Shot** & **5-Shots** \\ \hline Frozen [56] & \(\times\) & 7.8 & 10.5 \\ \hline MML [54] & \(\times\) & 6.9 & 10.7 \\ \hline VL-Meta [58] & \(\times\) & 7.952 & 10.864 \\  & \(\checkmark\) & 10.936 & 14.456 \\ \hline VL-Few (Ours) & \(\times\) & 14.816 & 15.328 \\  & \(\checkmark\) & 17.736 & 18.664 \\ \hline \hline \end{tabular}
\end{table}
Table 1: VL-Few comparison on 2-way crossdomain Real-Fast VQA task in accuracy. The \(\times\) means the methods without meta learning and the \(\checkmark\) means it with the meta learning. The bold font accuracies are the best with meta in their results. The underlined font accuracies are the best without meta in their results.