|
--- |
|
library_name: peft |
|
base_model: microsoft/Phi-3-mini-4k-instruct |
|
datasets: |
|
- argilla/ultrafeedback-binarized-preferences-cleaned |
|
- >- |
|
flax-sentence-embeddings/stackexchange_titlebody_best_and_down_voted_answer_jsonl |
|
language: |
|
- en |
|
--- |
|
|
|
# Model Card for Phi-3-mini-4k-instruct DPO |
|
|
|
## Model Details |
|
|
|
- **Model Name:** Phi-3-mini-4k-instruct DPO |
|
- **Publisher:** Team chatterbox, EPFL |
|
- **Model Type:** Language Model, Fine-tuned with direct preference optimization (DPO) |
|
- **Training Environment:** Trained on the EPFL SCITAS cluster using a 32GB GPU. |
|
|
|
## Intended Use |
|
|
|
- **Primary Applications:** This model is designed as part of an AI-Tutor system, aiming to accurately predict user preferences in educational scenarios. |
|
- **Intended Audience:** Educators, students, and developers creating educational AI applications. |
|
|
|
## Model/Data Description |
|
|
|
### Training Data |
|
|
|
- **Datasets Used:** |
|
- **Milestone 1 Dataset:** Includes [will fill] unique questions with preference pairs based on the 'overall' rating, totaling [will fill] usable entries after processing. |
|
- **Stack Exchange Dataset:** Filters content from specific domains within the Stack Exchange network, using upvoted and downvoted answers to form preference pairs. Total entries: [will fill]. |
|
- **Ultra Feedback:** Utilizes responses rated on criteria like truthfulness and helpfulness to form preference pairs, with a total of [will fill] entries after preprocessing. |
|
- **Preprocessing Details:** Entries with identical chosen and rejected answers were removed. Datasets were formatted as JSONL where each line represents a JSON object with a "prompt", "chosen", and "rejected" response. |
|
|
|
## Training Procedure |
|
|
|
- **Configurations:** (Refer to the provided `training_args` and `trainer` configuration) |
|
- **Evaluation Metrics:** The primary metric for model performance is `eval_loss`, with the aim to minimize this value. |
|
|
|
## Evaluation Results |
|
|
|
- **Accuracies:** eval/rewards/accuracies - 0.83 |
|
- **Loss:** eval/loss - 0.47 |
|
- **Margins:** eval/margins - 4.31 |
|
|
|
### MT-Bench |
|
- **Single Grading Score, Overall Avg.** - 8.2 |
|
- **STEM Score** - 9.8 (higher than GPT-4) |
|
 |
|
|
|
## References |
|
|
|
- **[Include references and citations for datasets, tools, and methodologies used.]** |
|
|
|
|
|
- PEFT 0.11.1 |
