# Deep Learning in Food Image Recognition: A Comprehensive Review
## 3 Food Image Recognition Methods
### Dedicated Deep Networks  
Although the local convolution operation of convolutional neural network has a strong ability to capture the local features of food (e.g., texture and shape, etc. ), there are still problems such as the risk of overfitting, insufficient interpretability, and high computational cost. In view of the fine-grained image characteristics of food images, researchers have proposed some special networks for food image recognition in recent years. Inspired by the great achievements of Transformers in the field of natural language processing, in recent years, Vision Transformer (ViT) structures based on self-attention mechanisms have been widely used in the field of computer vision and achieved certain results. Although the self-attention mechanism used by the ViT structure has some flexibility in characterizing visual content, it fails to achieve the best results when encoding the fine-grained features of images due to the limitation of image factors such as size. On this basis, [PERSON] et al. [10] proposed a food image recognition algorithm based on ViT. Aiming at the demand for fine-grained feature extraction of food images, a local-attention mechanism based on the attention mechanism was proposed to enable the model to extract richer fine-grained features. At the same time, [PERSON] et al. proposed an image self-supervised pre-training algorithm to solve the problem of a small number of samples in the food image dataset, which used mask images to train network parameters to effectively alleviate the problem of insufficient sample training. Experimental results show that the Top-1 accuracy and Top-5 accuracy values of the proposed model on the ISIA Food-500 dataset are 65.58% and 90.03%, which are the best accuracies at present (Figure 6). Food images are unstructured images with complex and unfixed visual patterns. Modeling the long-range semantic relationships between ingredients, as well as semantic interactions between categories and ingredients, is beneficial for ingredient identification and food analysis. Based on the above factors, [PERSON] et al. [25] proposed a multi-task learning framework for food category and ingredient recognition. The framework consists of a food-oriented Transformer named Convolution-Enhanced Bi-Branch Adaptive Transformer (CBiAFormer) and a multi-task category-ingredient recognition network called Structural Learning and Cross-Task Interaction (SLCI). In CBiAFormer, a query-aware data adaptive attention mechanism consisting of a local fine-grained branch and a global coarse-grained branch is established, and an adaptive candidate key/value set is assigned to each query to mine the local and global semantic-aware regions of different input images. At the same time, the framework introduces a convolutional patch embedding module for extracting fine-grained features that the transformer ignores. The SLCI is composed of Cross-Layer Attention (CLA) and two cross-task interaction modules, namely Category-guided Ingredi  
Figure 5: The ASTFF-Net architecture. Relationship Learning (CIRL) and Ingredient-guide Category Feature Enhancement (ICFE). The SLCI is used to model the semantic relationship between the components and mine the semantic interaction between the categories and components so as to make full use of the information contained in images. This method has obtained good experimental results on the mainstream datasets (ETH food-101, Vireo food-172, and ISIA food-200) (Figure 7). In summary, the current mainstream deep learning methods in the field of food image recognition can be summarized into four paradigms: CNN-based, Vision Transformer architecture, multi-modal fusion, and lightweight neural network models. In order to more clearly compare the characteristics of these methods, Table 2 systematically summarizes them from the dimensions of computational efficiency, recognition accuracy, and data requirements. Table S4 lists the comparison and summary of some lightweight neural  
Figure 6: Framework of method based on attention mechanism.