# Deep Learning in Food Image Recognition: A Comprehensive Review
## 5 Open Issues and Future Direction  
At present, more progress has been made in the research of algorithms and datasets related to food image recognition, and they have been applied to real life. The following sections mainly discuss the existing problems in food image-related research and make prospects for future development. * Dataset Limitations  
Although the current mainstream food image datasets (e.g., Vireo Food-172, ISIA Food-500, etc.) have established a benchmark system, there are still prominent problems of limited categories and insufficient scale compared with general large-scale datasets, such as ImageNet. This limitation is extremely significant in cross-cultural scenarios. First, food images are strongly culturally specific. Dishes with the same name in different regions have significant differences in ingredient ratios, cooking techniques, and presentation styles, while the cultural coverage of existing datasets is seriously unbalanced. For example, the proportion of Western cuisines in the mainstream dataset far exceeds that of Southeast Asian and African cuisines, which limits the cross-cultural generalization ability of the model. In addition, the fine-grained nature of food itself exacerbates the difficulty of data construction. Dishes of the same category may present completely different visual features due to regional differences, which makes it difficult to unify the annotation standards. Although ISIA Food-500 tries to introduce cultural labels [31], it increases the labeling cost in terms of microscopic details. The above problems make it difficult to build a large-scale food image dataset. In addition, when acquiring multi-channel images, how to effectively reduce the cost of dataset construction and improve the quality of the dataset is also a key issue to be solved. * Research on Food Computing Integrating Multiple Attributes of Food  
Different from conventional fine-grained images, food images do not have a fixed semantic pattern, lack a unified spatial layout, and have more noise. It is difficult to extract common semantic information between different food images for recognition. In addition, it is difficult to accurately identify food images with high confusability due to the similarities between and differences within food images. How to accurately extract and study similar food local discriminant feature images becomes the key to research. At the same time, food images often contain rich additional information, such as ingredient information, ingredient state geographical location, etc. At present, the existing food image recognition systems mainly focus on the types of food ingredients, and the recognition of the specific state of food ingredients is less important. In addition, deep learning models still face challenges in extracting higher-order semantic features of food, such as food ingredient combination patterns. The performance of traditional transfer learning methods is limited by the semantic differences between natural images and food images, such as the mismatch between texture and food ingredient combination. Multimodal fusion methods such as vision-text joint modeling [32] and spatio-temporal feature fusion [24] have been proposed to effectively improve the accuracy of food image recognition. Making full use of the additional information contained in food images can help to improve the recognition performance of food images. How to make full use of image features, design a dedicated network architecture for food semantics, and break through key technical bottlenecks such as modal alignment and computational efficiency will have important research value. * New Applications and Ethical Privacy Challenges  
At present, food image recognition technology has been widely used in many fields such as diet quality assessment, food safety monitoring, intelligent catering sales, and personalized diet recommendation and plays an important role in industrial and agricultural production.