llama3-qwantz-coherent
This model is a fine-tuned version of unsloth/llama-3-8b-bnb-4bit on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 0.3295
- Accuracy: 0.8758
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 1
Training results
Training Loss | Epoch | Step | Validation Loss | Accuracy |
---|---|---|---|---|
0.4482 | 1.0 | 1428 | 0.3295 | 0.8758 |
2-4 fake words at the end (like training set):
Can save 90% of coherent strings by discarding 82% of dp strings (cutoff is -67.26065874099731)
Can save 95% of coherent strings by discarding 72% of dp strings (cutoff is -88.40824365615845)
Can save 98% of coherent strings by discarding 62% of dp strings (cutoff is -95.06730437278748)
Can save 99% of coherent strings by discarding 54% of dp strings (cutoff is -97.79982566833496)
1 fake word at the end:
Can save 90% of coherent strings by discarding 48% of dp strings (cutoff is -77.83336043357849)
Can save 95% of coherent strings by discarding 30% of dp strings (cutoff is -92.52431392669678)
Can save 98% of coherent strings by discarding 26% of dp strings (cutoff is -95.45100927352905)
Can save 99% of coherent strings by discarding 21% of dp strings (cutoff is -97.32990860939026)
Examples (2-4 fake words):
My only problem (s) have to do with ==> coherent: 99.12%
My only problem (s) to cheer them personally ==> dp: 99.69%
(in small text) crazy utahraptor ==> coherent: 88.54%
(in small text) ". ==> coherent: 54.82%
Well, I've made up my own joke to get him today. All I need to do is " ==> coherent: 77.98%
Well, I've made up my own joke to get him today. All I need a father and gentlemen ==> dp: 99.79%
I will be immortalized by kicking an evil ==> dp: 72.79%
I will be immortalized by kicking other punches ==> dp: 99.49%
Aw shoot, I was supposed to ==> coherent: 99.80%
Aw shoot, I was APOCALYPSE PORN ==> dp: 94.10%
Get it? Because CRIME DOESN'T PAY!! Listen, my story has ==> coherent: 66.25%
Get it? Because CRIME DOESN'T PAY!! Listen, transcriptions of it ==> dp: 99.75%
Utahraptor!! DON'T LISTEN TO ==> coherent: 99.96%
Utahraptor! This is sort of ==> coherent: 95.96%
Doesn't exist in my mouth, that is!! Because it's too big ==> coherent: 95.38%
Doesn't exist in my mouth, that is!! Because if Superman. ==> dp: 99.66%
Now, HERE'S how ==> coherent: 98.67%
Now, guys would ==> dp: 95.30%
But I am a rock star ==> coherent: 92.34%
But I am a guy come ==> dp: 99.40%
But I have a solution to make them interesting again: all you need is stories where not ==> coherent: 94.51%
But I have a solution to make them interesting again: all you need is gonna! Diseases ==> dp: 99.94%
At that point, there's a sequence of six nines in a row, and his joke was that he'd like to memorize pi up to that point, so that when reciting he could end with "9,9,9,9,9,9... and so on. " Others ==> coherent: 70.68%
At that point, there's a sequence of six nines in a row, and his joke was that he'd like to memorize pi up to that point, so that when reciting he could end with "9,9,9,9,9,9... and so it's great he looks ==> dp: 99.54%
This is definitely called " T -Rex's Hilarious e joke ", okay ==> coherent: 79.86%
This is definitely called " T -Rex's Hilarious e joke AND IN THE ==> dp: 98.83%
" Your mouth is full of cockroaches: ==> coherent: 93.64%
" Your mouth is full of smooches. ==> coherent: 99.10%
Excuse me, sexual congress? Everyone else on the planet is dead, and ==> coherent: 89.66%
Excuse me, sexual congress? Everyone else on the planet without syntactic ambiguity! ==> dp: 97.02%
Sony is going to write swears on my bathroom ==> coherent: 99.75%
Sony is going to write their babies need to ==> dp: 99.68%
Beginning with the most modest: why am I ==> coherent: 99.62%
Beginning with the most modest: why T - ==> dp: 92.38%
Is there any greater meaning -to anything ==> coherent: 95.96%
Is there any greater meaning? When you ==> dp: 73.11%
I've also got steaks AND ==> coherent: 92.46%
I've also cold -deterministic ==> dp: 99.48%
I had a friend (female) who dated her roommate (also female) ==> coherent: 98.78%
I had a friend (female) who dated her roommate, je suis grand ==> dp: 97.52%
Yes... TOO BAD INDEED ==> dp: 93.13%
Yes... TOO MANY YEARS ==> coherent: 65.94%
Examples (1 fake word):
My only problem (s) have to do with you ==> dp: 83.54%
My only problem (s) have to do with no ==> dp: 53.68%
(in small text ==> coherent: 93.38%
(in small changes ==> dp: 97.64%
Well, I've made up my own joke to get him today. All I need to do is " ==> coherent: 77.98%
Well, I've made up my own joke to get him today. All I need to do is already ==> dp: 97.05%
I will be immortalized by kicking an evil kangaroo ==> coherent: 92.81%
I will be immortalized by kicking an evil! ==> dp: 99.50%
Aw shoot, I was supposed ==> coherent: 98.61%
Aw shoot, I was how ==> dp: 99.55%
Get it? Because CRIME DOESN'T PAY!! Listen, my story has both a hilarious twist ending and also ==> coherent: 99.64%
Get it? Because CRIME DOESN'T PAY!! Listen, my story has both a hilarious twist ending and genders ==> dp: 88.29%
Utahraptor!! DON'T LISTEN TO MY ==> coherent: 94.44%
Utahraptor!! DON'T LISTEN TO THE ==> coherent: 97.86%
Doesn't exist in my mouth, that is!! Because it's too ==> coherent: 96.91%
Doesn't exist in my mouth, that is!! Because it's Well ==> dp: 97.87%
Now, HERE'S how putting the things ==> dp: 97.33%
Now, HERE'S how putting the wall ==> dp: 87.15%
But I am a rock ==> coherent: 52.13%
But I am a time ==> dp: 93.86%
But I have a solution to make them interesting again: all you need is ==> coherent: 99.83%
But I have a solution to make them interesting again: all you need to ==> coherent: 99.81%
At that point, there's a sequence of six nines in a row, and his joke was that he'd like to memorize pi up to that point, so that when reciting he could end with "9,9,9 ==> coherent: 99.95%
At that point, there's a sequence of six nines in a row, and his joke was that he'd like to memorize pi up to that point, so that when reciting he could end with "9,9, and ==> coherent: 95.76%
This is definitely ==> coherent: 89.56%
This is especially ==> coherent: 98.87%
" Your mouth is full of cockroaches: many of them are dead, but those that ==> coherent: 99.96%
" Your mouth is full of cockroaches: many of them are dead, but those taste ==> dp: 78.53%
Excuse me, sexual congress? Everyone ==> dp: 94.45%
Excuse me, sexual congress? " ==> dp: 91.74%
Sony is going to write swears ==> coherent: 74.56%
Sony is going to write that ==> dp: 98.15%
Beginning with the ==> coherent: 84.62%
Beginning with! ==> dp: 98.17%
Is there any greater meaning -to ==> dp: 61.69%
Is there any greater meaning -Rex ==> dp: 99.31%
I've also got steaks AND pork chops ==> coherent: 99.84%
I've also got steaks AND pork meat ==> coherent: 95.80%
I had a friend (female ==> coherent: 85.77%
I had a friend (on ==> coherent: 57.30%
Yes... TOO ==> dp: 77.83%
Yes... All ==> dp: 67.88%
Framework versions
- PEFT 0.10.0
- Transformers 4.40.1
- Pytorch 2.2.1+cu121
- Datasets 2.19.0
- Tokenizers 0.19.1
- Downloads last month
- 5
Inference API (serverless) does not yet support peft models for this pipeline type.
Model tree for paul-stansifer/llama3-qwantz-coherent
Base model
meta-llama/Meta-Llama-3-8B
Quantized
unsloth/llama-3-8b-bnb-4bit