juewang commited on
Commit
1133bd2
1 Parent(s): 75c4c62

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +18 -12
README.md CHANGED
@@ -6,6 +6,8 @@ language:
6
 
7
  # GPT-NeoXT-Chat-Base-20B
8
 
 
 
9
  > TLDR: As part of OpenChatKit (codebase available [here](https://github.com/togethercomputer/OpenChaT)),
10
  > GPT-NeoXT-Chat-Base-20B is a 20B parameter language model, fine-tuned from EleutherAI’s GPT-NeoX with over 40 million instructions on 100% carbon negative compute.
11
 
@@ -23,6 +25,20 @@ You can read more about this process and the availability of this dataset in LAI
23
  - **Model Description**: A 20B parameter open source chat model, fine-tuned from EleutherAI’s NeoX with over 40M instructions on 100% carbon negative compute
24
  - **Resources for more information**: [GitHub Repository](https://github.com/togethercomputer/OpenChaT).
25
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
26
  ## Strengths of the model
27
 
28
  There are several tasks that OpenChatKit excels at out of the box. This includes:
@@ -160,7 +176,8 @@ We therefore welcome contributions from individuals and organizations, and encou
160
  ## Training
161
 
162
  **Training Data**
163
- \[TODO\]
 
164
 
165
  **Training Procedure**
166
 
@@ -170,15 +187,4 @@ We therefore welcome contributions from individuals and organizations, and encou
170
  - **Batch:** 2 x 2 x 64 x 2048 = 524288 tokens
171
  - **Learning rate:** warmup to 1e-6 for 100 steps and then kept constant
172
 
173
- ## Environmental Impact
174
- \[TODO\]
175
- **Stable Diffusion v1** **Estimated Emissions**
176
- Based on that information, we estimate the following CO2 emissions using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700). The hardware, runtime, cloud provider, and compute region were utilized to estimate the carbon impact.
177
-
178
- - **Hardware Type:** A100 PCIe 40GB
179
- - **Hours used:** 200000
180
- - **Cloud Provider:** AWS
181
- - **Compute Region:** US-east
182
- - **Carbon Emitted (Power consumption x Time x Carbon produced based on location of power grid):** 15000 kg CO2 eq.
183
-
184
 
 
6
 
7
  # GPT-NeoXT-Chat-Base-20B
8
 
9
+ ***<p style="font-size: 24px">Feel free to try out our [OpenChatKit feedback app](https://huggingface.co/spaces/togethercomputer/OpenChatKit)!</p>***
10
+
11
  > TLDR: As part of OpenChatKit (codebase available [here](https://github.com/togethercomputer/OpenChaT)),
12
  > GPT-NeoXT-Chat-Base-20B is a 20B parameter language model, fine-tuned from EleutherAI’s GPT-NeoX with over 40 million instructions on 100% carbon negative compute.
13
 
 
25
  - **Model Description**: A 20B parameter open source chat model, fine-tuned from EleutherAI’s NeoX with over 40M instructions on 100% carbon negative compute
26
  - **Resources for more information**: [GitHub Repository](https://github.com/togethercomputer/OpenChaT).
27
 
28
+ # Quick Start
29
+
30
+ ```python
31
+ from transformers import pipeline
32
+ pipe = pipeline(model='togethercomputer/GPT-NeoXT-Chat-Base-20B')
33
+ pipe('''<human>: Hello!\n<bot>:''')
34
+ ```
35
+ or
36
+ ```python
37
+ from transformers import AutoTokenizer, AutoModelForCausalLM
38
+ tokenizer = AutoTokenizer.from_pretrained("togethercomputer/GPT-NeoXT-Chat-Base-20B")
39
+ model = AutoModelForCausalLM.from_pretrained("togethercomputer/GPT-NeoXT-Chat-Base-20B")
40
+ ```
41
+
42
  ## Strengths of the model
43
 
44
  There are several tasks that OpenChatKit excels at out of the box. This includes:
 
176
  ## Training
177
 
178
  **Training Data**
179
+
180
+ Please refer to [togethercomputer/OpenDataHub](https://github.com/togethercomputer/OpenDataHub)
181
 
182
  **Training Procedure**
183
 
 
187
  - **Batch:** 2 x 2 x 64 x 2048 = 524288 tokens
188
  - **Learning rate:** warmup to 1e-6 for 100 steps and then kept constant
189
 
 
 
 
 
 
 
 
 
 
 
 
190