Comparative Analysis of Fine-Tuning LLaMA 2 and LLaMA 3 Models with RTX 4090
Source: tai_blog

When beginning LLM operations  a key question is which model to use. As a fan of LLaMA models  I wondered if LLaMA 3 is necessarily better than LLaMA 2. This analysis compares their practical performance in fine-tuning tasks  particularly under constraints like limited vRAM and budget.   My PC setup includes an Alienware R16 with an Intel(R) Core(TM) i714700KF 3.40 GHz processor  and an NVIDIA GeForce RTX 4090 GPU. I previously used an RTX 3070 but found it too slow and prone to out-of-vRAM issues. My NVIDIA-SMI version is 550.76.01  the Driver Version is 552.44  and my CUDA Version is 12.4.   The 2 models under review are LLaMA 2 and LLaMa 3. LLaMA 2 is available in Hugging Face here: meta-llama/Llama-27b  Hugging Face  which is a 7b model. LLaMa 3 can be found here: meta-llama/Meta-Llama-38B  Hugging Face  8 billion parameter model.   I referenced Luca Massarons notebook on Kaggle for the base script  modifying it to run locally on my RTX 4090 and to accommodate the two models.   MethodologyWe fine-tuned the models for financial sentiment analysis. The dataset we are employing is the FinancialPhraseBank dataset  which is a comprehensive collection of the financial news headlines and the sentiments classification labels from the viewpoint of a retail investor. The data can be found here takala/financial_phrasebank  Datasets at Hugging Face.   We sampled 900 examples for training and 900 for testing from the entire dataset  which originally has 4840 sentences from English language financial news categorized by sentiment. The examples in the training and testing sets are balanced  which means they have the same amount of positive  neutral and negative samples.   First  both models were evaluated out of the box. Then  they were fine-tuned with different parameters  focusing on target modules and epochs. The sentiments are divided into three categories  Positive  Neutral  and Negative  mapped to 2  1  and 0  respectively. If the output is none  it is mapped as neutral and 1.   1. The baseline performance1.1 LLaMA 2   The initial performance of the LLaMA 2 model before fine-tuning on the financial sentiment analysis task is summarized in the classification report below. The models performance metrics are evaluated across three sentiment classes (0  1  and 2)  with each class containing 300 samples.   The overall accuracy of the model is 37%  indicating that the model correctly classifies 37% of the instances. The macro and weighted averages provide an aggregate view of the models performance across all classes. The precision is relatively high for positive and negative sentiments  but both recall and F1-score are low  highlighting a significant imbalance where the model is good at precision for some classes but poor at identifying the actual instances of each class correctly.   1.2 LLaMA 3   Similarly  the initial performance of the LLaMA 3 model before fine-tuning on the financial sentiment analysis task is summarized in the classification report below.   The overall accuracy of the model is 36%  which is 1% lower than its predecessor. The precision is moderate  but both recall and F1-score are low  also highlighting an imbalance where the model is better at predicting some classes than others.   An interesting observation is the precision and recall for the negative class  which are 1 and 0.02  respectively. A precision of 1.00 indicates that every instance predicted as negative sentiment was indeed negative  with no false positives. However  a recall of 0.02 means the model correctly identified only 2% of all actual negative sentiment instances  resulting in a low F1 score. This is highly undesirable.   Out of the box  LLaMA 2 is slightly better than LLaMA 3 with an overall accuracy of 37%.   2. Fine-Tuning Result ComparisonsIn the context of fine-tuning language models like LLaMA 2 and LLaMA 3 using the LoRA (Low-Rank Adaptation) technique  the target_modules parameter specifies which layers of the model are adjusted during training. The choice of target modules will significantly impact the efficiency of the fine-tuning process  and this is why some may only include q and v during the tuning process as they are crucial in the attention mechanism of transformer models.   2.1 LLaMA 2  target_modules=[q_proj  v_proj]   After fine-tuning the LLaMA 2 model with an epoch setting to 2 and targeting the q_proj and v_proj modules  the performance metrics for the financial sentiment analysis task have improved significantly. The accuracy has increased to 77%  with balanced macro and weighted averages for precision  recall  and F1-scores. This fine-tuning approach has enhanced the models ability to identify negative  neutral  and positive sentiments. The more balanced f1 score than LLaMA 3 shows even for adjusting only 2 crucial layers of the models  LLaMA 2 can achieve a better result.   2.2 LLaMA 3 target_modules=[q_proj  v_proj]   From the above chart  we can see fine-tuning the LLaMA 3 model with an epoch setting to 2 and targeting the q_proj and v_projmodules has led to significant improvements in overall performance. The accuracy has increased to 75%  with balanced macro and weighted averages for precision  recall  and F1-scores. But the ability to identify class 1 is still not satisfactory.   2.3 LLaMA 2  target_modules=[all-linear]   Now lets see how finetuning a model with all layers rather than the 2 crucial layers will impact the model final performance.   Fine-tuning the LLaMA 2 model with an epoch setting to 2 and targeting all linear layers (all_linear) has led to significant improvements in overall performance comparing to the baseline model. The accuracy has increased to 80%  with balanced macro and weighted averages for precision  recall  and F1-scores. We can see an improvement of f1-score in each class  and contributing to an overall 80% of overall accuracy.   2.4 LLaMA 3 target_modules=[all-linear]   From the picture above  we can see that fine-tuning both LLaMA 2 and LLaMA 3 models with an epoch setting to 2 and targeting all linear layers has significantly improved their performance. However  LLaMA 3 demonstrates an even more balanced precision score  and higher F1-score across most metrics  making it a preferable choice for financial sentiment analysis tasks.   Until now  as we see fine-tunning with all all-linear layers yields better results in both models  therefore we will apply target_modules=[all-linear] in all the remaining tests  and adjust only the epochs amount.   2.5 LLaMA 2  epoch=3   The number of epochs is a critical hyperparameter in the fine-tuning process. Setting it appropriately involves a trade-off between training time  resource utilization  and model performance. Usually when there is vRAM limitation  this is one of the params we will adjust downward first. Typically  the goal is to find a sweet spot where the model achieves good generalization without excessive training time or resource consumption.   From the above graph  fine-tuning the LLaMA 2 model with target_modules=all_linear for 3 epochs has further improved its performance across all sentiment classes. The model now exhibits high accuracy  precision  recall  and F1-scores  indicating a well-balanced and effective classification capability for financial sentiment analysis. This improvement highlights the effectiveness of fine-tuning in enhancing the model's ability to correctly identify and classify sentiments in the text. The overall accuracy rate is 82% now  an noticeable improvement over the previous epoch=2 result.   2.6 LLaMA 3 epoch=3   The overall accuracy of the model is 86%  indicating a significant improvement in the models ability to correctly classify the sentiments when comparing to the previous result of LLaMA 3  83%. The macro and weighted averages across all classes showed a balanced and high performance in precision  recall  and F1-score.   When comparing to the result of LLaMA 2 of the same params setting  LLaMA 3 shows higher and more balanced scores in all classes.   2.7 LLaMA 2 epoch=5   The model appears to be overfitting after epoch 2 or 3. The significant decrease in training loss combined with the increase in validation loss after epoch 2 suggests that the model is learning to fit the training data more closely  but not learning well in the validation set. The optimal number of epochs in this case would likely be around 2 or 3  where the validation loss was at its lowest before starting to increase.   Although there overfitting is observed in epoch 5  fine-tuning the LLaMA 2 model for 5 epochs has marginally improved its performance across all sentiment classes compared to 3 epochs  resulting a 84% accuracy rate. This improvement highlights the effectiveness of extended fine-tuning in enhancing the model's ability to correctly identify and classify sentiments in the text  but in terms of efficiency  we could have stop a epoch 2 or 3.   2.8 LLaMA 3 epoch=5   Similar to LLaMA 2 The training record shows a consistent decrease in training loss over the epochs  while validation loss initially decreasing slightly but then increasing. It is likely suggesting a potential overfitting.   The overall accuracy of the model remains at 86%  indicating extending the training is not helpful in the models ability to correctly classify the sentiments. After 5 epochs  LLaMA 3 still shows a better accuracy rate when comparing to LLaMA 2.   3. ConclusionAfter all the tests  it is found that LLaMA 2 performs well with limited resources  particularly when fine-tuning only specific layers. However  LLaMA 3 shows higher accuracy and balanced performance when more resources are available for extended fine-tuning.   Returning to the initial question: whether LLaMA 3 is better than LLaMA 2 depends on the resources and constraints. For tight budgets  LLaMA 2 is a solid choice  but with more resources  LLaMA 3 offers superior performance.   What are your thoughts on this comparative analysis?   The notebook will be provided later.   Thank you for reading. If you like this tutorial  please share it with your data science friends  and follow me. The following is the motivation for me to continue contributing to the community.