dranger003
commited on
Commit
•
1eb89d2
1
Parent(s):
c087449
Update README.md
Browse files
README.md
CHANGED
@@ -1,3 +1,13 @@
|
|
1 |
---
|
2 |
license: cc-by-nc-4.0
|
|
|
|
|
3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
---
|
2 |
license: cc-by-nc-4.0
|
3 |
+
pipeline_tag: text-generation
|
4 |
+
library_name: gguf
|
5 |
---
|
6 |
+
GGUF importance matrix (imatrix) quants for https://huggingface.co/abideen/AlphaMonarch-laser
|
7 |
+
The importance matrix was trained for ~50K tokens (105 batches of 512 tokens) using a [general purpose imatrix calibration dataset](https://github.com/ggerganov/llama.cpp/discussions/5263#discussioncomment-8395384).
|
8 |
+
|
9 |
+
> AlphaMonarch-laser is a DPO fine-tuned of mlabonne/NeuralMonarch-7B using the argilla/OpenHermes2.5-dpo-binarized-alpha preference dataset but achieves better performance then mlabonne/AlphaMonarch-7B using LaserQLoRA. We have fine-tuned this model only on half of the projections, but have achieved better results as compared to the version released by Maximme Labonne. We have trained this model for 1080 steps.
|
10 |
+
|
11 |
+
| Layers | Context | Template |
|
12 |
+
| --- | --- | --- |
|
13 |
+
| <pre>32</pre> | <pre>32768</pre> | <pre>[INST] {prompt} [/INST]<br>{response}</pre> |
|