vicgalle
/

CarbonBeagle-11B

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

vicgalle commited on Jan 21

Commit

79ba2bc

•

1 Parent(s): d774c74

Update README.md

Files changed (1) hide show

README.md +5 -1

README.md CHANGED Viewed

@@ -9,7 +9,11 @@ license: apache-2.0
 ---
 # CarbonBeagle-11B
-This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
 ![image/png](https://cdn-uploads.huggingface.co/production/uploads/5fad8602b8423e1d80b8a965/C1FMtz2kX9UyCwX1CGDKM.png)

 ---
 # CarbonBeagle-11B
+An experiment in merging models of different architectures and sizes. Here are the steps:
+1. Upscale mlabonne/NeuralBeagle14-7B to vicgalle/franken-Beagle-11B.
+2. DPO-tune vicgalle/franken-Beagle-11B to vicgalle/NeuralBeagle-11B.
+3. Merge vicgalle/NeuralBeagle-11B and jeonsworld/CarbonVillain-en-10.7B-v4.
 ![image/png](https://cdn-uploads.huggingface.co/production/uploads/5fad8602b8423e1d80b8a965/C1FMtz2kX9UyCwX1CGDKM.png)