onurkeles
/

llama-2-7b-econ-chat-qa

@@ -19,7 +19,7 @@ Designed to serve as a virtual research assistant, providing answers to complex
 ## Training and Evaluation Data
 The model was trained on a synthetic question-and-answer dataset created from academic paper abstracts in economics, employing a GPT-3.5 Turbo model for generating contextual dialogues.
-### Training Hyperparameters:
 - **QLoRA Settings:**
   - `lora_rank (lora_r)`: 64
   - `lora_dropout`: 0.1
@@ -43,15 +43,15 @@ The model was trained on a synthetic question-and-answer dataset created from ac
 The LLaMA-2-Econ Q&A model's performance was evaluated against a set of criteria designed to measure its effectiveness in generating accurate, relevant answers to economics-related questions. We used BERT-Score.
-### Key Metrics:
 - **Precision:** Achieved an average precision value of **0.90**, indicating a high level of accuracy in the tokens used within the model's answers when compared to the reference answers.
 - **Recall:** Recorded an average recall value of **0.89**, reflecting the model's ability to capture all relevant information from the reference answers in its generated responses.
 - **F1 Score:** Reached an average F1 value of **0.90**, demonstrating an excellent balance between precision and recall, thus indicating the model's overall effectiveness in producing comprehensive and accurate answers.
-### Evaluation Procedure:
 1. **Reference Answers Generation:** A subset of synthetically created questions was used to obtain reference answers from the base LLaMA-2-7B-chat model integrated with a Retrieval Augmented Generation (RAG) pipeline, employing semantic search and dense vector indexing.
 2. **Human Verification:** The reference answers were subjected to human verification to ensure their relevance and accuracy.
 3. **Model Comparison:** LLaMA-2-Econ's generated answers, produced without any RAG integration, were compared against these human-verified reference responses.
-### Citation:
   - Keleş, O. & Bayraklı, Ö. T. (Fortcoming 2024, May). *LLaMA-2-Econ: Enhancing Title Generation, Classification, and Academic Q&A in Economic Research.* To be presented in LREC-COLING 2024, 4th Workshop on ECONLP: Turin, Italy.

 ## Training and Evaluation Data
 The model was trained on a synthetic question-and-answer dataset created from academic paper abstracts in economics, employing a GPT-3.5 Turbo model for generating contextual dialogues.
+### Training Hyperparameters
 - **QLoRA Settings:**
   - `lora_rank (lora_r)`: 64
   - `lora_dropout`: 0.1
 The LLaMA-2-Econ Q&A model's performance was evaluated against a set of criteria designed to measure its effectiveness in generating accurate, relevant answers to economics-related questions. We used BERT-Score.
+### Key Metrics
 - **Precision:** Achieved an average precision value of **0.90**, indicating a high level of accuracy in the tokens used within the model's answers when compared to the reference answers.
 - **Recall:** Recorded an average recall value of **0.89**, reflecting the model's ability to capture all relevant information from the reference answers in its generated responses.
 - **F1 Score:** Reached an average F1 value of **0.90**, demonstrating an excellent balance between precision and recall, thus indicating the model's overall effectiveness in producing comprehensive and accurate answers.
+### Evaluation Procedure
 1. **Reference Answers Generation:** A subset of synthetically created questions was used to obtain reference answers from the base LLaMA-2-7B-chat model integrated with a Retrieval Augmented Generation (RAG) pipeline, employing semantic search and dense vector indexing.
 2. **Human Verification:** The reference answers were subjected to human verification to ensure their relevance and accuracy.
 3. **Model Comparison:** LLaMA-2-Econ's generated answers, produced without any RAG integration, were compared against these human-verified reference responses.
+### Citation
   - Keleş, O. & Bayraklı, Ö. T. (Fortcoming 2024, May). *LLaMA-2-Econ: Enhancing Title Generation, Classification, and Academic Q&A in Economic Research.* To be presented in LREC-COLING 2024, 4th Workshop on ECONLP: Turin, Italy.