# Food Classification and Meal Intake Amount Estimation through Deep Learning
## 5 Conclusions  
In this paper, we proposed the methods of the food type classification and the meal intake amount estimation. The food regions and the food types were detected through the Mask R-CNN. The post-meal image was corrected to have the same capturing environment based on the vertex points of the meal plate in the two images. The 3D shape type was determined as one of a spherical cap, a cone, and a cuboid for each food in the images. The food volumes were estimated through the detected area sizes and the 3D shape types. The meal intake amounts were estimated as the food volume differences between pre- and post-meal. In the simulation results, the accuracies of the food type classification and the  
Figure 12: Various foods mixed in a single dish.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Target**} & \multirow{2}{*}{**Food Category**} & \multirow{2}{*}{**Food Name**} & **Food Volume of** & **Food Volume of** & **Meal Intake** \\  & & & **Pre-Meal Image** & **Post-Meal Image** & **Amount** \\  & & & **(cm\({}^{\textbf{3}}\))** & **(cm\({}^{\textbf{3}}\))** & **(cm\({}^{\textbf{3}}\))** & **(cm\({}^{\textbf{3}}\))** \\ \hline \multirow{4}{*}{Figure 11a} & Rice & Rice & 89.12 & 48.51 & 40.61 \\  & Soup & Seaweed soup & 104.81 & 82.80 & 22.01 \\  & Side-dish 1 & Stir-fried octopus & 50.74 & 23.58 & 27.16 \\  & Side-dish 2 & Fruit salad & 26.41 & 17.15 & 9.26 \\  & Side-dish 3 & Stir-fried squash & 36.31 & 23.58 & 12.73 \\  & Side-dish 4 & Stir-fried mushroom & 50.74 & 36.31 & 14.43 \\ \hline \multirow{4}{*}{Figure 11b} & Rice & Rice & 89.12 & 26.41 & 62.71 \\  & Soup & Sirak soup & 104.81 & 49.69 & 55.12 \\  & Side-dish 1 & Stir-fried pork & 50.74 & 5.69 & 45.05 \\  & Side-dish 2 & Stir-fried squash & 36.31 & 5.69 & 30.62 \\  & Side-dish 3 & Pepper seasoned & 17.15 & 3.30 & 13.85 \\  & Side-dish 4 & Seastring seasoned & 36.90 & 17.15 & 19.75 \\ \hline \multirow{4}{*}{Figure 11c} & Rice & Rice & 89.12 & 17.15 & 71.97 \\  & Soup & Seaweed soup & 104.81 & 37.73 & 67.08 \\  & Side-dish 1 & Stir-fried octopus & 50.74 & 4.54 & 46.20 \\  & Side-dish 2 & Fruit salad & 26.41 & 3.30 & 23.11 \\  & Side-dish 3 & Stir-fried squash & 36.31 & 4.54 & 31.77 \\  & Side-dish 4 & Stir-fried mushroom & 50.74 & 12.84 & 37.90 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Results of food volume estimation through proposed method.  
Figure 13: 3D shape types for foods of various countries: (**a**) spherical cap shape; (**b**) cuboid shape; (**c**) cone shape.  
-food region detection were up to 97.57% and 93.6%, respectively. The proposed method can be applied not only to Korean food, but also to other countries' foods, such as other Asian countries or countries in Europe. It is possible to analyze the ingested nutrients through the proposed method. The ingested nutrients are identified through the classified food types. The amounts of the ingested nutrients are calculated through the food types and the estimated meal intake amount. The nutrient analysis through the proposed method allows us to suggest a diet that provides a balanced nutrient intake. The adherence of a patient to dietary restrictions can be checked by analyzing the ingested nutrients. Moreover, it is possible to recommend the intake of the corresponding food for the part lacking in a specific nutrient.  
Conceptualization, J.-h.K., D.-s.L. and S.-k.K.; software, J.-h.K.; writing--original draft preparation, J.-h.K., D.-s.L. and S.-k.K.; supervision, S.-k.K. All authors have read and agreed to the published version of the manuscript.  
This research was supported by the BB21+ Project in 2022 and supported by the MSIT (Ministry of Science and ICT), Korea, under the Grand Information Technology Research Center support program (IITP-2023-2020-0-01791) supervised by the IITP (Institute for Information & communications Technology Planning & Evaluation).  
Not applicable.  
The authors declare no conflict of interest.