Before revision:  Our key finding is that a minimalistic LNA (LayerNorm and Attention) finetuning can achieve  crosslingual and cross-modality transfer ability with only finetuning less than 10\% of the pretrained parameters.
After revision:  Our key finding is that a minimalistic LNA (LayerNorm and Attention) finetuning can achieve zero-shot crosslingual and cross-modality transfer ability with only finetuning less than 10\% of the pretrained parameters.