Update README.md

README.md

@@ -3,10 +3,6 @@ datasets:
 - jondurbin/airoboros-gpt4-1.4.1
 ---
 
-**7/6: This could be a little undertrained. I'll update the weights if I end up training it longer and/or with better hyperparameters.**
-
----------------
-
 # RoPE Scaled QLoRA Fine-tune of Llama-13b on airoboros-gpt4-1.4.1 (GPTQ)
 
 LoRA Weights can be found here: https://huggingface.co/bhenrym14/airoboros-13b-gpt4-1.4.1-PI-8192-LoRA
@@ -48,9 +44,10 @@ Recent advancements in extending context by RoPE scaling ([kaiokendev](https://k
 
 
 - For contexts shorter than the original 2048, the original model has lower perplexity. This is consistent with the literature. The gap shrinks with context length, with the original becoming incoherent beyond this point.
-- I haven't used models with the SuperHOT LoRA enough to have any sense of performance differences, but feedback on the 33b variant suggests it is noticable particularly with coherence at longer context lengths. 
+- In terms of perplexity, this model outperforms the SuperHOT variant at all tested context lengths. I haven't used models with the SuperHOT LoRA enough to have any sense of performance differences, but feedback on the 33b variant suggests it is particularly noticable at longer context lengths. 
 - This comparison isn't perfect. I did use the 1.4.1 dataset, the quantization method is slightly different, and the finetuning method is different (QLoRA vs full). In short, there are other potentially influential variables responsible for these performance differences.
 
+This model could be a little undertrained. I'll update the weights if I end up training it longer and/or with better hyperparameters
 ## Quantization:
 
 The merged model was quantized with AutoGPTQ (bits = 4, group_size = 128, desc_act = True).