Update README.md

README.md

@@ -1,3 +1,11 @@
+A variation of NeuralHermes 2.5 - Mistral 7B
+
+This is a variation of NeuralHermes which is based on the teknium/OpenHermes-2.5-Mistral-7B model that has been further fine-tuned with Direct Preference Optimization (DPO) using the mlabonne/chatml_dpo_pairs dataset. It surpasses the original model on most benchmarks (see results).
+
+It is directly inspired by the RLHF process described by Intel/neural-chat-7b-v3-1's authors to improve performance. I used the same dataset and reformatted it to apply the ChatML template.
+
+The code to train this model is available on Google Colab and GitHub. It required an A100 GPU for about an hour.
+
 ---
 license: mit
 language: