Update README.md

README.md

@@ -1,8 +1,17 @@
 ---
 license: apache-2.0
+language:
+- en
+tags:
+- sft
+pipeline_tag: text-generation
+widget:
+  - text: <prefix>You are a helpful assistant model trained by LAION called Aki</prefix><human>Hi, how are you?<bot>
+  - text: <human>What's the Earth total population<bot>
+  - text: <human>Write a story about future of AI development<bot>
 ---
 
-# Model Card for Model ID (WIP)
+# Pythia 3B SFT model
 
 <!-- Provide a quick summary of what the model is/does. -->
 
@@ -15,49 +24,32 @@ This modelcard aims to be a base template for new models. It has been generated
 <!-- Provide a longer summary of what this model is. -->
 
 
-
-- **Developed by:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
+- **Developed by:** Open Assistant
+- **Model type:** Pythia
+- **Language(s) (NLP):** English
+- **License:** Apache-2.0
 
 ## Model Sources [optional]
 
 <!-- Provide the basic links for the model. -->
 
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
+- **Repository:** [Open Assistant](https://github.com/LAION-AI/Open-Assistant)
 
 # Uses
 
 <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
 
+
 ## Direct Use
 
 <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-## Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-## Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
+See the example on the right
 
 # Bias, Risks, and Limitations
 
 <!-- This section is meant to convey both technical and sociotechnical limitations. -->
 
-[More Information Needed]
+[just read pythia](https://huggingface.co/EleutherAI/pythia-12b#out-of-scope-use)
 
 ## Recommendations
 
@@ -69,7 +61,29 @@ Users (both direct and downstream) should be made aware of the risks, biases and
 
 Use the code below to get started with the model.
 
-[More Information Needed]
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_name = "theblackcat102/pythia-3b-deduped-sft"
+
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+model = AutoModelForCausalLM.from_pretrained(model_name).half().eval().cuda()
+
+input_text = "<human>What's the earth population?<bot>"
+inputs = tokenizer(input_text, return_tensors="pt", padding=True).to(0)
+outputs = model.generate(
+    **inputs,
+    early_stopping=True,
+    max_new_tokens=args.max_new_tokens,
+    do_sample=True,
+    top_k=args.top_k,
+    temperature=args.temperature,
+    pad_token_id=tokenizer.eos_token_id,
+    # dialogue_collator.py line 36
+)
+output = tokenizer.decode(outputs[0], truncate_before_pattern=[r"\n\n^#", "^'''", "\n\n\n"])
+print(output)
+```
 
 # Training Details
 
@@ -77,15 +91,11 @@ Use the code below to get started with the model.
 
 <!-- This should link to a Data Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
 
-[More Information Needed]
-
 ## Training Procedure 
 
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-### Preprocessing [optional]
-
-[More Information Needed]
+```
+deepspeed trainer_sft.py --configs defaults pythia-3b --deepspeed
+```
 
 
 ### Training Hyperparameters