## 4 Implementation and Experiments
### Training Procedure and Evaluation Metrics  
#### 4.3.1 NutritionCLIP Training and Evaluation Framework  
The training process of CLIP fine tuning utilized the following parameters in Table 6, including batch sizes, learning rate, number of epochs, choice of optimizer, and the selection of an appropriate loss function.  
\begin{table}
\begin{tabular}{c c} \hline \hline  & **Experimental Setup** \\ \hline Batch size & 32 \\ Learning rate & \(1\times 10^{-5}\) \\ Number of epochs & 30 \\ Optimizer & Adam \\ Loss function & Cross-entropy loss \\ \hline \hline \end{tabular}
\end{table}
Table 6: CLIP fine-tuning parameters.  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Food Category** & **Quantity** \\ \hline Meals & 25 \\ Meat & 66 \\ Fruits and Vegetables & 10 \\ Snacks & 10 \\ \hline \hline \end{tabular}
\end{table}
Table 4: List of food category quantities.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Datasets** & **Train** & **Validation** & **Description** \\ \hline Identity & 91 & 10 & System prompt \\ alpaca\_gpt4\_en & 20 k & 500 & English comprehensive Q\&A \\ alpaca\_gpt4\_zh & 20 k & 500 & Chinese comprehensive Q\&A \\ Animal-nutrition-alpaca & 5.03 k & 300 & English nutrition Q\&A \\ Animal-nutrition-new-alpaca & 4.94 k & 300 & English nutrition Q\&A \\ Animal-nutrition-alpaca-to-zh & 5.03 k & 300 & Chinese nutrition Q\&A \\ Animal-nutrition-new-alpaca-to-zh & 4.94 k & 300 & Chinese nutrition Q\&A \\ \hline \hline \end{tabular}
\end{table}
Table 5: List of Llama 3 fine-tuning datasets.  
In this training process, cross-entropy was used for classification, predicting each image as the correct food category, and it is defined as follows:  
\[L_{CE}=-{\sum}_{i=1}^{n}label_{i}log(p_{i}) \tag{6}\]  
where \(label_{i}\) is the actual label, usually a one-hot vector indicating that sample \(i\) belongs to a specific category, and \(p_{i}\) is the predicted probability. Its main features include bias punishment, which emphasizes the predicted probability for the correct class; the resulting loss increases when the model predicts a lower probability for this class. This method is well suited for scenarios where output categories are mutually exclusive, ensuring each input sample belongs to only one category. Moreover, it typically utilizes the softmax function to calculate predicted probabilities for each category, enhancing numerical stability, especially when the model's output range is extensive.  
In the testing phase, we use accuracy, precision, recall, and F1 score to validate the performance of the food image classification problem. The definitions are as follows.  
Accuracy is the proportion of correct predictions made by the model, which is the ratio of all correct predictions (true positives and negatives) to the total number of predictions. The formula is as follows:  
\[Accuracy=\frac{TP+TN}{N} \tag{7}\]  
Precision represents the proportion of instances predicted as positive by the model that are positive. The formula is as follows:  
\[Precision=\frac{TP}{TP+FP} \tag{8}\]  
Recall indicates the proportion of actual positive instances correctly predicted as positive by the model. The formula is as follows:  
\[Recall=\frac{TP}{TP+FN} \tag{9}\]  
The F1 score is the harmonic mean of precision and recall used to measure the model's overall performance. The formula is as follows:  
\[F1 score=2\frac{Precision\times Recall}{Precision+Recall} \tag{10}\]  
where \(N\) is the total number of predictions made by the model, true positives (TPs) are the number of instances correctly predicted as positive, true negatives (TNs) are the number of cases correctly predicted as negative, False Positives (FPs) are the number of instances incorrectly predicted as positive, and False Negatives (FNs) are the number of cases incorrectly predicted as negative.