sometimesanotion
/

Lamarck-14B-v0.3

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

sometimesanotion commited on 4 days ago

Commit

031a736

•

1 Parent(s): 722a99c

Update README.md

Files changed (1) hide show

README.md +0 -2

README.md CHANGED Viewed

@@ -131,8 +131,6 @@ For GGUFs, [mradermacher/Lamarck-14B-v0.3-i1-GGUF](https://huggingface.co/mrader
 ![graph.png](https://huggingface.co/sometimesanotion/Lamarck-14B-v0.3-experimental/resolve/main/graph.png)
-The first two layers come entirely from Virtuoso.  The choice to leave these layers untouched comes from [arxiv.org/abs/2307.03172](https://arxiv.org/abs/2307.03172) which identifies early attention glitches as a chief cause of hallucinations.  Layers 3-8 feature a SLERP gradient into introducing the DELLA merge tree in which the reason branch is emphasized, the prose branch only given a small ranking.
 ### Thanks go to:
 - @arcee-ai's team for the ever-capable mergekit, and the exceptional Virtuoso Small model.

 ![graph.png](https://huggingface.co/sometimesanotion/Lamarck-14B-v0.3-experimental/resolve/main/graph.png)
 ### Thanks go to:
 - @arcee-ai's team for the ever-capable mergekit, and the exceptional Virtuoso Small model.