Create README.md

README.md

@@ -0,0 +1,57 @@
+---
+license: other
+language:
+- en
+library_name: transformers
+pipeline_tag: text-generation
+tags:
+- causal-lm
+- text-generation-inference
+- merge
+---
+
+# FOR EXPERIMENT 
+
+## Description
+
+[**stabilityai/stablelm-zephyr-3b**](https://huggingface.co/stabilityai/stablelm-zephyr-3b), [**StableMed-3b**](https://huggingface.co/cxllin/StableMed-3b) merged with a new, experimental implementation of "dare ties" via mergekit. See:
+
+> [Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch](https://github.com/yule-BUAA/MergeLM)
+
+> https://github.com/cg123/mergekit/tree/dare
+
+
+## Usage
+
+`StableLM Zephyr 3B` uses the following instruction format:
+```
+<|user|>
+List 3 synonyms for the word "tiny"<|endoftext|>
+<|assistant|>
+1. Dwarf
+2. Little
+3. Petite<|endoftext|>
+```
+
+***
+## Testing Notes
+
+Merged in mergekit with the following config, and the tokenizer from chargoddard's Yi-Llama:
+
+```
+models:
+  - model: stabilityai/stablelm-zephyr-3b
+    # no parameters necessary for base model
+  - model: cxllin/StableMed-3b
+    parameters:
+      weight: 0.08
+      density: 0.5
+merge_method: dare_ties
+base_model: stabilityai/stablelm-zephyr-3b
+parameters:
+  int8_mask: true
+dtype: bfloat16
+```
+
+## Model Details
+ - License: [StabilityAI Non-Commercial Research Community License](https://huggingface.co/stabilityai/stablelm-zephyr-3b/raw/main/LICENSE)