Crystalcareai commited on
Commit
77f747f
1 Parent(s): bd9ed42

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +54 -0
README.md ADDED
@@ -0,0 +1,54 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ language:
4
+ - en
5
+ base_model:
6
+ - unsloth/Phi-3-mini-4k-instruct
7
+ datasets:
8
+ - cognitivecomputations/Dolphin-2.9.2
9
+ - teknium/OpenHermes-2.5
10
+ - m-a-p/CodeFeedback-Filtered-Instruction
11
+ - cognitivecomputations/dolphin-coder
12
+ - cognitivecomputations/samantha-data
13
+ - microsoft/orca-math-word-problems-200k
14
+ - internlm/Agent-FLAN
15
+ - cognitivecomputations/SystemChat-2.0
16
+ ---
17
+
18
+ # Dolphin 2.9.2 Phi 3 Medium 🐬
19
+
20
+ Curated and trained by Eric Hartford, Lucas Atkins, Fernando Fernandes, and with help from the community of Cognitive Computations
21
+
22
+ [![Discord](https://img.shields.io/discord/1156064224225808488?logo=Discord&logoColor=%23ffffff&label=Discord&link=https%3A%2F%2Fdiscord.gg%2FtCMkMDDHwm)](https://discord.gg/cognitivecomputations)
23
+ Discord: https://discord.gg/cognitivecomputations
24
+
25
+ <img src="https://cdn-uploads.huggingface.co/production/uploads/63111b2d88942700629f5771/ldkN1J0WIDQwU4vutGYiD.png" width="600" />
26
+
27
+ Our appreciation for the sponsor of Dolphin 2.9.2:
28
+ - [Crusoe Cloud](https://crusoe.ai/) - provided excellent on-demand 8xL40Snode
29
+
30
+ This model is based on Phi-3-Medium-Instruct-4k, and is governed by the MIT license with which Microsoft released Phi-3.
31
+
32
+ Since Microsoft only released the fine-tuned model - Dolphin-2.9.2-Phi-3-Medium has not been entirely cleaned of refusals.
33
+
34
+ The base model has 4k context, and the qLoRA fine-tuning was with 4k sequence length.
35
+
36
+ It took 3.5 days on 8xL40S node provided by Crusoe Cloud
37
+
38
+ This model uses the ChatML prompt template.
39
+
40
+ example:
41
+
42
+ ```
43
+ <|im_start|>system
44
+ You are Dolphin, a helpful AI assistant.<|im_end|>
45
+ <|im_start|>user
46
+ {prompt}<|im_end|>
47
+ <|im_start|>assistant
48
+
49
+ ```
50
+
51
+ Dolphin-2.9.2 has a variety of instruction, conversational, and coding skills. It also has initial agentic abilities and supports function calling.
52
+ We have filtered the dataset to remove alignment and bias. This makes the model more compliant. You are advised to implement your own alignment layer before exposing the model as a service. Please read my blog post about uncensored models. https://erichartford.com/uncensored-models You are responsible for any content you create using this model. Enjoy responsibly.
53
+
54
+ [<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)