delayedkarma
commited on
Update README.md
Browse files
README.md
CHANGED
@@ -24,11 +24,7 @@ base_model: teknium/OpenHermes-2.5-Mistral-7B
|
|
24 |
|
25 |
# NeuralHermes 2.5 - Mistral 7B
|
26 |
|
27 |
-
NeuralHermes is based on the [teknium/OpenHermes-2.5-Mistral-7B](https://huggingface.co/teknium/OpenHermes-2.5-Mistral-7B) model that has been further fine-tuned with Direct Preference Optimization (DPO) using the [mlabonne/chatml_dpo_pairs](https://huggingface.co/datasets/mlabonne/chatml_dpo_pairs) dataset.
|
28 |
-
|
29 |
-
It is directly inspired by the RLHF process described by [Intel/neural-chat-7b-v3-1](https://huggingface.co/Intel/neural-chat-7b-v3-1)'s authors to improve performance. I used the same dataset and reformatted it to apply the ChatML template.
|
30 |
-
|
31 |
-
The code to train this model is available on [Google Colab](https://colab.research.google.com/drive/15iFBr1xWgztXvhrj5I9fBv20c7CFOPBE?usp=sharing) and [GitHub](https://github.com/mlabonne/llm-course/tree/main). It required an A100 GPU for about an hour.
|
32 |
|
33 |
|
34 |
## Usage
|
|
|
24 |
|
25 |
# NeuralHermes 2.5 - Mistral 7B
|
26 |
|
27 |
+
NeuralHermes is based on the [teknium/OpenHermes-2.5-Mistral-7B](https://huggingface.co/teknium/OpenHermes-2.5-Mistral-7B) model that has been further fine-tuned with Direct Preference Optimization (DPO) using the [mlabonne/chatml_dpo_pairs](https://huggingface.co/datasets/mlabonne/chatml_dpo_pairs) dataset. .
|
|
|
|
|
|
|
|
|
28 |
|
29 |
|
30 |
## Usage
|