## 1 Introduction  
Visual Question Answering (VQA) [1, 2] is a cross-domain task of computer vision and natural language processing, and it has become increasingly important in the research and application of multimodal machine learning. In the past few decades, significant advances have been made in computer vision and natural language processing, with an explosion of visual and textual data to acquire and process. The most common VQA consists of an image and a question to be answered by the machine. Compared with other computer vision tasks, this model answers in real-time, not in advance. Moreover, the VQA model is required to comprehend the multimodal information of images and texts in a more artificially intelligent [3] way, leading to an in-depth understanding of vision and language. VQA remains a challenging and open research topic. Recent research has focused on how to solve language bias. Language bias [4, 5, 6, 7, 8] threatens the implementation of VQA, which indicates that the current VQA model has an inadequate understanding of multimodal information. Language bias seems to be caused by the uneven distribution of datasets, a common problem in the real world. For example, if 90 percent of the bananas in the training set are yellow, the model would ask, \"what color the banana is\" and answer, \"yellow\" all the time, based on language bias. As shown in Figure 1, many VQA models tend to answer \"yes\" or \"no\" directly. Take another typical example. For the question \"what color is the banana in the image?\", although the banana is green, the model still tends to predict \"yellow\". With language bias, the model overly relies on the correlation between the question and the answer while it ignores the information in the image. In essence, language bias arises from data imbalance, which leads to over-fitting of the model; that is, the model fits the head samples in the dataset [8]. The over-fitting of the model is an inherent problem of the model itself. The deep neural network has a variance in the case of an individual model, and the variance can be reduced by an ensemble or knowledge distillation [9; 10; 11; 12] (Over-fitting to the label imbalance may lead to some models not training very well). At the feature level, the variance is caused by the incompleteness of the feature subgraph [12], so it is easy to produce over-fitting. Recently, knowledge [13; 14] distillation and self-knowledge distillation [15; 16] have been proven to be able to learn multi-view features and reduce over-fitting [12; 17; 18]. Neural network analysis of information is multi-view; for the same object, its different views have semantic consistency [19; 20; 21], and the multi-view structure can be ubiquitous in the dataset and feature level [12]. Therefore, the model can give the prediction based on a learned subgraph. However, if the subgraph is not comprehensive, the prediction can be biased. As shown in Figure 2, for the same question, \"what color is the banana?\", the model learns the feature of yellow bananas while it ignores the feature of green bananas, which are less frequent in the training set. In other words, the model ignores the view of green bananas, causing visual bias, which further leads to language bias. Therefore, the model needs to focus on the feature of the less distributed samples in the training set and learn a comprehensive set of multi-view features so as to overcome VQA language bias and over-fitting. The paper discusses how to reduce the language bias of the VQA model via self-knowledge distillation and proposes a new online learning framework, \"language bias-driven self-knowledge distillation (LBSD)\", for implicit learning of multi-view visual features. Self-knowledge distillation enables the model to acquire more dark knowledge and improves its generalization ability. In short, with self-knowledge distillation, the model can have a more comprehensive understanding of view features.