# Deep Learning in Food Image Recognition: A Comprehensive Review
## 2 Datasets  
Nowadays, the world's mainstream datasets are mainly based on various dishes, covering different dishes around the world. Among these, the types of dishes are mainly Western dishes, and the number of Chinese dishes is relatively small. There are also some fruit and vegetable image datasets and other types of food. These datasets can be divided into small-scale, medium-scale, and large-scale dish datasets. With the data amount of 10,000 as the limit, the dataset with the total data amount less than 10,000 is called the small-scale dish dataset, the data amount between 10,000 and 100,000 is called the medium-scale dish dataset, and the data amount more than 100,000 is called the large-scale dish dataset. Many large-scale datasets have significantly promoted the development of food image-related technology. In 2009, [PERSON] et al. published the first fast food dataset, PFID, for dish recognition, which contains 4545 still images, 606 stereo pairs, 303 360-degree motion structure videos, and 27 related videos [16]. Subsequently, [PERSON] et al. published Food50 [17] and Food80 [18]. In 2014, [PERSON] et al. [19] published the first large-scale Western dish image dataset, ETHZ Food-101, which contains 101,000 dish images composed of images from 101 Western dishes. Due to its large data scale, this dataset is currently widely used to test the performance of dish image recognition model algorithms and as one of the benchmarks of recognition methods based on deep learning. Since then, food image datasets have been continuously expanded, increasing in size and scope (Table S3, Figure 2). \begin{table}
\begin{tabular}{c c c} \hline \hline
**Aspect** & **Our Review** & **Previous Surveys [1; 15]** \\ \hline Method Taxonomy & Four paradigms: CNN-based, ViT, multimodal fusion, lightweight models; includes performance comparisons (Tables S3 and S4) & Often focused on single paradigms (e.g., CNNs) without cross-paradigm comparisons \\ \hline Applications & Expands to nutrient assessment, food safety, and ethical challenges & Focus on traditional scenarios, such as practical applications such as calorie estimation or dish identification \\ \hline Challenges and Future & Highlights cross-cultural generalization, feature mismatch, and ethical risks & Ethical issues and multicultural dataset requirements are rarely addressed, and challenges are mostly technical \\ \hline \hline \end{tabular}
\end{table} 
Table 1: _Cont._  
Figure 1: Overview of the survey. For the existing datasets, there are still some limitations restricting the research progress:
* Low cultural coverage: The proportion of non-Western dishes in the mainstream dataset is relatively low, and the annotation standard does not consider the difference of national and regional names. For example, in the ETHZ Food-101 dataset, Asian dishes make up only 30% of the total, and African dishes make up even less, around 1%. The same is true for the ISIA Food-500 dataset, where Asian dishes make up about 40% of the total and African dishes make up only about 4%. In the single non-Western food dataset, there is still a lack of refinement of regional variants. * Coarse annotation granularity: Most datasets only provide food category labels and lack fine-grained annotations such as ingredients and cooking methods, which limits the application of multi-task learning. * Lack of authenticity: There are significant domain differences between the images taken in the laboratory environment and the blurred and occluded pictures actually taken by users, resulting in the performance degradation of the model in the real scene and the lack of dynamically updated datasets. To mitigate the impact of dataset limitations (e.g., cultural bias and annotation granularity) while improving model performance, current research employs complementary strategies at both data and algorithm levels:  
1.