###### Abstract  
Accurate dietary assessment is essential for effective health management and disease prevention. However, conventional methods that rely on manual food logging and nutritional lookup are often time consuming and error prone. This study proposes an image-based nutritional advisory system that integrates multimodal deep learning to automate food classification, volume estimation, and dietary recommendation to address these limitations. The system employs a fine-tuned CLIP model for zero-shot food recognition, achieving high accuracy across diverse food categories, including unseen items. For volume measurement, a learning-based multi-view stereo (MVS) approach eliminates the need for specialized hardware, yielding reliable estimations with a mean absolute percentage error (MAPE) of 23.5% across standard food categories. Nutritional values are then calculated by referencing verified food composition databases. Furthermore, the system leverages a large language model (Llama 3) to generate personalized dietary advice tailored to individual health goals. The experimental results show that the system attains a top 1 classification accuracy of 91% on CNFOOD-241 and 80% on Food 101 and delivers high-quality recommendation texts with a BLEU-4 score of 45.13. These findings demonstrate the system's potential as a practical and scalable tool for automated dietary management, offering improved precision, convenience, and user experience. c ARTIC  
2020 2020 27 April 2020 28 April 2020  
In previous studies, the field of food image classification commonly adopted convolutional neural networks (CNNs) [1; 2; 3; 4] and other deep learning techniques as the core model architecture. Although these methods demonstrated excellent performance on specific training datasets, their applicability is often limited to the food categories already present in the datasets, with relatively poor generalization and applicability to unseen categories. To address this limitation, this study employs a zero-shot learning approach based on the CLIP model [5]. By learning the correspondence between images and texts, the model can effectively recognize and classify food images that did not appear during the training phase, bringing higher flexibility and broad applicability to the classification field and better meeting the diverse needs of food analysis. Furthermore, volume plays a critical role in nutritional assessment because it allows consumers to calibrate their nutritional intake through portion sizes and is crucial for improving the accuracy of dietary evaluations. As pointed out by [PERSON] et al. from Google in a 2021 study [6], integrating volume information into neural network-based food nutritional assessment models can significantly enhance the effectiveness and accuracy of the assessment. In terms of volume measurement, this study adopts the E-Volume [7] technique, representing an innovative method that replaces traditional methods of using depth cameras and reference planes [8; 9] to estimate food volume. It overcomes the limitations of requiring specialized hardware or portable devices, expands the generality and convenience of the application, and makes volume measurement more accurate and user friendly. This study employs the Llama 3 [10] language model developed by Meta as the main core architecture to establish the dietary recommendation system. Its high adaptability and powerful language understanding capabilities can effectively parse and process complex nutritional information, thus providing personalized dietary recommendations. The primary contributions of this work are as follows:  
* We develop a unified system that combines zero-shot image classification, learning-based volume estimation, and language-based dietary recommendation. * The system removes the need for specialized hardware, improving accessibility and user convenience. * Extensive evaluations demonstrate strong classification accuracy, volume prediction, and natural language generation performance. The remainder of this paper is organized as follows.