Update README.md

README.md

@@ -1,42 +1,38 @@
 ---
 license: apache-2.0
+language:
+- en
 tags:
 - trl
 - transformers
 - reinforcement-learning
 ---
 
-# TRL Model
+# Llama-se-rl-adapter
+Adapter weights of an RL fine-tuned model based on LLaMa. Authored by Edward Beeching, Younes Belkada, Kashiv Rasul, Lewis Tunstall and Leandro von Werra.
 
-This is a [TRL language model](https://github.com/lvwerra/trl) that has been fine-tuned with reinforcement learning to
- guide the model outputs according to a value, function, or human feedback. The model can be used for text generation.
 
-## Usage
+## Model Description
+**Llama-se-rl** is a Llama-based model that has been first fine-tuned on the Stack Exchange dataset and then RL fine-tuned using a Stack Exchange Reward Model. This dataset consists of questions and answers from various domains in Stack Exchange, such as programming, mathematics, physics, and more. The model is designed to generate human-like responses to questions in these domains. The model has been training to respond to prompts with the following template:
 
-To use this model for inference, first install the TRL library:
-
-```bash
-python -m pip install trl
 ```
+Question: <Query> 
 
-You can then generate text as follows:
-
-```python
-from transformers import pipeline
-
-generator = pipeline("text-generation", model="lvwerra/runs_truncate/step_350")
-outputs = generator("Hello, my llama is cute")
+Answer: <Response>
 ```
 
-If you want to use the model for training or to obtain the outputs from the value head, load the model as follows:
+## Intended Uses & Limitations
+**Llama-se-rl** is intended for use in generating responses to questions related to the Stack Exchange dataset. It is suitable for generating answers to questions in the domains covered by the dataset, such as programming, mathematics, and physics. However, the model may not perform well on questions outside these domains or on questions requiring highly specific or technical knowledge.
 
-```python
-from transformers import AutoTokenizer
-from trl import AutoModelForCausalLMWithValueHead
+## Limitations and Bias
+The **Llama-se-rl** model inherits limitations and biases from the Llama model and also those contained in the Stack Exchange dataset. The Stack Exchange dataset may contain biases in terms of the topics it covers and the users who contribute to it. It may not include all possible domains, and the quality of answers may vary. Additionally, the model may generate answers that are incorrect or misleading due to biases in the training data or the inherent limitations of the Llama architecture.
 
-tokenizer = AutoTokenizer.from_pretrained("lvwerra/runs_truncate/step_350")
-model = AutoModelForCausalLMWithValueHead.from_pretrained("lvwerra/runs_truncate/step_350")
+## BibTeX entry and citation info
 
-inputs = tokenizer("Hello, my llama is cute", return_tensors="pt")
-outputs = model(**inputs, labels=inputs["input_ids"])
-```
+```bibtex
+@misc{beeching2023llama,
+  title={StackLLaMa: An RL Fine-tuned LLaMa Model for Stack Exchange Question and Answering},
+  author={Beeching, Edward and Belkada, Younes and Rasul, Kashiv and Tunstall, Lewis and von Werra, Leandro},
+  year={2023}
+}
+```