dmayhem93 commited on
Commit
7bafed5
1 Parent(s): dbe5ee9

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +12 -19
README.md CHANGED
@@ -1,5 +1,4 @@
1
  ---
2
- license: cc-by-nc-4.0
3
  datasets:
4
  - conceptofmind/cot_submix_original
5
  - conceptofmind/flan2021_submix_original
@@ -13,19 +12,19 @@ pipeline_tag: text-generation
13
 
14
  ## Model Description
15
 
16
- `StableBeluga_13B` is a Llama2 13B model finetuned on an Orca style Dataset
17
 
18
  ## Usage
19
 
20
- Start chatting with `StableBeluga_13B` using the following code snippet:
21
 
22
  ```python
23
  import torch
24
  from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
25
 
26
- tokenizer = AutoTokenizer.from_pretrained("stabilityai/StableBeluga_13B", use_fast=False)
27
- model = AutoModelForCausalLM.from_pretrained("stabilityai/StableBeluga_13B", torch_dtype=torch.float16, low_cpu_mem_usage=True, device_map="auto")
28
- system_prompt = "### System:\nYou are StableBeluga_13B, an AI that follows instructions extremely well. Help as much as you can. Remember, be safe, and don't do anything illegal.\n\n"
29
 
30
  message = "Write me a poem please"
31
  prompt = f"{system_prompt}### User: {message}\n\n### Assistant:\n"
@@ -35,7 +34,7 @@ output = model.generate(**inputs, do_sample=True, top_p=0.95, top_k=0, max_new_t
35
  print(tokenizer.decode(output[0], skip_special_tokens=True))
36
  ```
37
 
38
- StableBeluga_13B should be used with this prompt format:
39
  ```
40
  ### System:
41
  This is a system prompt, please behave and help the user.
@@ -44,21 +43,21 @@ This is a system prompt, please behave and help the user.
44
  Your prompt here
45
 
46
  ### Assistant
47
- The output of StableBeluga_13B
48
  ```
49
 
50
  ## Model Details
51
 
52
  * **Developed by**: [Stability AI](https://stability.ai/)
53
- * **Model type**: StableBeluga_13B is an auto-regressive language model fine-tuned on Llama2 13B.
54
  * **Language(s)**: English
55
  * **Library**: [HuggingFace Transformers](https://github.com/huggingface/transformers)
56
- * **License**: Fine-tuned checkpoints (`FreeWilly2`) is licensed under the Non-Commercial Creative Commons license ([CC BY-NC-4.0](https://creativecommons.org/licenses/by-nc/4.0/))
57
  * **Contact**: For questions and comments about the model, please email `lm@stability.ai`
58
 
59
  ### Training Dataset
60
 
61
- `StableBeluga_13B` is trained on our internal Orca-style dataset
62
 
63
  ### Training Procedure
64
 
@@ -69,15 +68,9 @@ Models are learned via supervised fine-tuning on the aforementioned datasets, tr
69
  | Orca pt1 packed | 256 | 3e-5 | Cosine to 3e-6 | 100 | 1e-6 | (0.9, 0.95) |
70
  | Orca pt2 unpacked | 512 | 3e-5 | Cosine to 3e-6 | 100 | 1e-6 | (0.9, 0.95) |
71
 
72
- ## Use and Limitations
73
 
74
- ### Intended Use
75
-
76
- These models are intended for research only, in adherence with the [CC BY-NC-4.0](https://creativecommons.org/licenses/by-nc/4.0/) license.
77
-
78
- ### Limitations and bias
79
-
80
- Although the aforementioned dataset helps to steer the base language models into "safer" distributions of text, not all biases and toxicity can be mitigated through fine-tuning. We ask that users be mindful of such potential issues that can arise in generated responses. Do not treat model outputs as substitutes for human judgment or as sources of truth. Please use it responsibly.
81
 
82
  ## Citations
83
 
 
1
  ---
 
2
  datasets:
3
  - conceptofmind/cot_submix_original
4
  - conceptofmind/flan2021_submix_original
 
12
 
13
  ## Model Description
14
 
15
+ `Stable Beluga 13B` is a Llama2 13B model finetuned on an Orca style Dataset
16
 
17
  ## Usage
18
 
19
+ Start chatting with `Stable Beluga 13B` using the following code snippet:
20
 
21
  ```python
22
  import torch
23
  from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
24
 
25
+ tokenizer = AutoTokenizer.from_pretrained("stabilityai/StableBeluga-13B", use_fast=False)
26
+ model = AutoModelForCausalLM.from_pretrained("stabilityai/StableBeluga-13B", torch_dtype=torch.float16, low_cpu_mem_usage=True, device_map="auto")
27
+ system_prompt = "### System:\nYou are Stable Beluga 13B, an AI that follows instructions extremely well. Help as much as you can. Remember, be safe, and don't do anything illegal.\n\n"
28
 
29
  message = "Write me a poem please"
30
  prompt = f"{system_prompt}### User: {message}\n\n### Assistant:\n"
 
34
  print(tokenizer.decode(output[0], skip_special_tokens=True))
35
  ```
36
 
37
+ Stable Beluga 13B should be used with this prompt format:
38
  ```
39
  ### System:
40
  This is a system prompt, please behave and help the user.
 
43
  Your prompt here
44
 
45
  ### Assistant
46
+ The output of Stable Beluga 13B
47
  ```
48
 
49
  ## Model Details
50
 
51
  * **Developed by**: [Stability AI](https://stability.ai/)
52
+ * **Model type**: Stable Beluga 13B is an auto-regressive language model fine-tuned on Llama2 13B.
53
  * **Language(s)**: English
54
  * **Library**: [HuggingFace Transformers](https://github.com/huggingface/transformers)
55
+ * **License**: Fine-tuned checkpoints (`Stable Beluga 13B`) is licensed under the [STABLE BELUGA NON-COMMERCIAL COMMUNITY LICENSE AGREEMENT](https://huggingface.co/stabilityai/StableBeluga-13B/blob/main/LICENSE.txt)
56
  * **Contact**: For questions and comments about the model, please email `lm@stability.ai`
57
 
58
  ### Training Dataset
59
 
60
+ ` Stable Beluga 13B` is trained on our internal Orca-style dataset
61
 
62
  ### Training Procedure
63
 
 
68
  | Orca pt1 packed | 256 | 3e-5 | Cosine to 3e-6 | 100 | 1e-6 | (0.9, 0.95) |
69
  | Orca pt2 unpacked | 512 | 3e-5 | Cosine to 3e-6 | 100 | 1e-6 | (0.9, 0.95) |
70
 
71
+ ## Ethical Considerations and Limitations
72
 
73
+ Beluga is a new technology that carries risks with use. Testing conducted to date has been in English, and has not covered, nor could it cover all scenarios. For these reasons, as with all LLMs, Beluga's potential outputs cannot be predicted in advance, and the model may in some instances produce inaccurate, biased or other objectionable responses to user prompts. Therefore, before deploying any applications of Beluga, developers should perform safety testing and tuning tailored to their specific applications of the model.
 
 
 
 
 
 
74
 
75
  ## Citations
76