jacobrenn commited on
Commit
adcc310
1 Parent(s): 9748088

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +129 -0
README.md CHANGED
@@ -1,3 +1,132 @@
1
  ---
2
  license: apache-2.0
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
+ datasets:
4
+ - aisquared/databricks-dolly-15k
5
+ language:
6
+ - en
7
+ library_name: transformers
8
  ---
9
+
10
+
11
+ # Model Card for `dlite-v2-355m`
12
+
13
+ <!-- Provide a quick summary of what the model is/does. -->
14
+
15
+ AI Squared's `dlite-v2-355m` is a large language
16
+ model which is derived from OpenAI's smallest [GPT-2](https://huggingface.co/gpt2) model and fine-tuned on a single GPU on a corpus of 15k records
17
+ ([Databricks' "Dolly 15k" Dataset](https://huggingface.co/datasets/aisquared/databricks-dolly-15k)) to help it exhibit chat-based capabilities.
18
+
19
+ Just like [Databricks' Dolly V2 models](https://www.databricks.com/blog/2023/04/12/dolly-first-open-commercially-viable-instruction-tuned-llm),
20
+ `dlite-v2-355m` (and all other members of the `dlite-v2` family) is licensed for both **research and commercial use.** We are extremely grateful
21
+ for the work that Databricks has done to create the `databricks-dolly-15k` dataset, for without it we would not be able to create and release this
22
+ model under such an open and permissive license.
23
+
24
+ While `dlite-v2-355m` is **not a state-of-the-art model**, we believe that the level of interactivity that can be achieved on such a small model that is trained so cheaply
25
+ is important to showcase, as it continues to demonstrate that creating powerful AI capabilities may be much more accessible than previously thought.
26
+
27
+
28
+ ### Model Description
29
+
30
+ <!-- Provide a longer summary of what this model is. -->
31
+
32
+ - **Developed by:** AI Squared, Inc.
33
+ - **Shared by:** AI Squared, Inc.
34
+ - **Model type:** Large Language Model
35
+ - **Language(s) (NLP):** EN
36
+ - **License:** Apache v2.0
37
+ - **Finetuned from model:** GPT-2
38
+
39
+
40
+ ## Bias, Risks, and Limitations
41
+
42
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
43
+
44
+ **`dlite-v2-355m` is not a state-of-the-art language model.** `dlite-v2-355m` is an experimental technology, and as with any experimental technology,
45
+ AI Squared urges potential users of this technology to test its capabilities thoroughly before usage.
46
+ Furthermore, the model can sometimes exhibit undesired behaviors. Some of these behaviors include,
47
+ but are not limited to: factual inaccuracies, biases, offensive responses, toxicity, and hallucinations.
48
+ Just as with any other LLM, we advise users of this technology to exercise good judgment when applying this technology.
49
+
50
+
51
+ ## Usage
52
+
53
+ The code below shows how to use `dlite-v2-355m` in the way which it was trained. While the model can be used "out of the box" using the
54
+ `transformers` library, using the function defined below to create a response from the model will achieve better results.
55
+
56
+ ### Load Model and Tokenizer from this Repository Using the `transformers` Package
57
+
58
+ ```python
59
+ from transformers import AutoModelForCausalLM, AutoTokenizer
60
+ import numpy as np
61
+ import re
62
+
63
+ model_id = 'aisquared/dlite-v2-355m'
64
+
65
+ tokenizer = AutoTokenizer.from_pretrained(model_id, padding_side = 'left')
66
+ model = AutoModelForCausalLM.from_pretrained(model_id, trust_remote_code = True, device_map = 'auto')
67
+ ```
68
+
69
+
70
+ ### Create the Prompt Format and Other Variables
71
+
72
+ ```python
73
+ PROMPT = """Below is an instruction that describes a task. Write a response that appropriately completes the request.
74
+
75
+ ### Instruction:
76
+ {instruction}
77
+
78
+ ### Response:
79
+ """
80
+
81
+ END_KEY = '### End'
82
+ RESPONSE_KEY = '### Response:\n'
83
+ ```
84
+
85
+
86
+ ### Create a Function to Retrieve a Response
87
+
88
+ ```python
89
+ def create_response(
90
+ instruction,
91
+ model,
92
+ tokenizer,
93
+ do_sample = True,
94
+ max_new_tokens = 256,
95
+ top_p = 0.92,
96
+ top_k = 0,
97
+ **kwargs
98
+ ):
99
+ """
100
+ Create a response from the model by using a formatted prompt
101
+ """
102
+ input_ids = tokenizer(
103
+ PROMPT.format(instruction=instruction), return_tensors="pt"
104
+ ).input_ids
105
+
106
+ gen_tokens = model.generate(
107
+ input_ids,
108
+ pad_token_id=tokenizer.pad_token_id,
109
+ do_sample=do_sample,
110
+ max_new_tokens=max_new_tokens,
111
+ top_p=top_p,
112
+ top_k=top_k,
113
+ **kwargs,
114
+ )
115
+ decoded = tokenizer.batch_decode(gen_tokens)[0]
116
+
117
+ # The response appears after "### Response:". The model has been trained to append "### End" at the end.
118
+ m = re.search(r"#+\s*Response:\s*(.+?)#+\s*End", decoded, flags=re.DOTALL)
119
+
120
+ response = None
121
+ if m:
122
+ response = m.group(1).strip()
123
+ else:
124
+ # The model might not generate the "### End" sequence before reaching the max tokens. In this case, return
125
+ # everything after "### Response:".
126
+ m = re.search(r"#+\s*Response:\s*(.+)", decoded, flags=re.DOTALL)
127
+ if m:
128
+ response = m.group(1).strip()
129
+ else:
130
+ pass
131
+ return response
132
+ ```