Update README.md
Browse files
README.md
CHANGED
@@ -78,16 +78,16 @@ SteerLM Llama-2 is a 13 billion parameter generative language model based on the
|
|
78 |
|
79 |
Key capabilities enabled by SteerLM:
|
80 |
|
81 |
-
- Dynamic steering of responses by specifying desired attributes like quality, helpfulness, and toxicity
|
82 |
-
- Simplified training compared to RLHF techniques like fine-tuning and bootstrapping
|
83 |
|
84 |
## Model Architecture and Training
|
85 |
The SteerLM method involves the following key steps:
|
86 |
|
87 |
-
1. Train an attribute prediction model on human annotated data to evaluate response quality
|
88 |
-
2. Use this model to annotate diverse datasets and enrich training data
|
89 |
-
3. Perform conditioned fine-tuning to align responses with specified combinations of attributes
|
90 |
-
4. (Optionally) Bootstrap training through model sampling and further fine-tuning
|
91 |
|
92 |
SteerLM Llama-2 applies this technique on top of the Llama-2 architecture. It was pretrained on internet-scale data and then customized using [OASST](https://huggingface.co/datasets/OpenAssistant/oasst1) and [HH-RLHF](https://huggingface.co/datasets/Anthropic/hh-rlhf) data.
|
93 |
|
@@ -109,7 +109,7 @@ pip install -v --disable-pip-version-check --no-cache-dir --global-option="--cpp
|
|
109 |
pip install nemo_toolkit['nlp']==1.17.0
|
110 |
```
|
111 |
|
112 |
-
Alternatively, you can use NeMo
|
113 |
|
114 |
2. Launch eval server
|
115 |
|
@@ -199,7 +199,7 @@ The model was trained on the data originally crawled from the Internet. This dat
|
|
199 |
We did not perform any bias/toxicity removal or model alignment on this checkpoint.
|
200 |
|
201 |
|
202 |
-
##
|
203 |
|
204 |
- Llama 2 is licensed under the [LLAMA 2 Community License](https://ai.meta.com/llama/license/), Copyright © Meta Platforms, Inc. All Rights Reserved.
|
205 |
- Your use of the Llama Materials must comply with applicable laws and regulations (including trade compliance laws and regulations) and adhere to the [Acceptable Use Policy](https://ai.meta.com/llama/use-policy) for the Llama Materials.
|
|
|
78 |
|
79 |
Key capabilities enabled by SteerLM:
|
80 |
|
81 |
+
- Dynamic steering of responses by specifying desired attributes like quality, helpfulness, and toxicity.
|
82 |
+
- Simplified training compared to RLHF techniques like fine-tuning and bootstrapping.
|
83 |
|
84 |
## Model Architecture and Training
|
85 |
The SteerLM method involves the following key steps:
|
86 |
|
87 |
+
1. Train an attribute prediction model on human annotated data to evaluate response quality.
|
88 |
+
2. Use this model to annotate diverse datasets and enrich training data.
|
89 |
+
3. Perform conditioned fine-tuning to align responses with specified combinations of attributes.
|
90 |
+
4. (Optionally) Bootstrap training through model sampling and further fine-tuning.
|
91 |
|
92 |
SteerLM Llama-2 applies this technique on top of the Llama-2 architecture. It was pretrained on internet-scale data and then customized using [OASST](https://huggingface.co/datasets/OpenAssistant/oasst1) and [HH-RLHF](https://huggingface.co/datasets/Anthropic/hh-rlhf) data.
|
93 |
|
|
|
109 |
pip install nemo_toolkit['nlp']==1.17.0
|
110 |
```
|
111 |
|
112 |
+
Alternatively, you can use NeMo Framework.
|
113 |
|
114 |
2. Launch eval server
|
115 |
|
|
|
199 |
We did not perform any bias/toxicity removal or model alignment on this checkpoint.
|
200 |
|
201 |
|
202 |
+
## License
|
203 |
|
204 |
- Llama 2 is licensed under the [LLAMA 2 Community License](https://ai.meta.com/llama/license/), Copyright © Meta Platforms, Inc. All Rights Reserved.
|
205 |
- Your use of the Llama Materials must comply with applicable laws and regulations (including trade compliance laws and regulations) and adhere to the [Acceptable Use Policy](https://ai.meta.com/llama/use-policy) for the Llama Materials.
|