Update README.md
Browse files
README.md
CHANGED
@@ -33,7 +33,9 @@ Users (both direct and downstream) should be made aware of the risks, biases, an
|
|
33 |
|
34 |
# How to Get Started with the Model
|
35 |
Use the code [here](https://colab.research.google.com/drive/1y_7oOU3ZJI0h4chUrXFL3K4kelW_OI2G?usp=sharing#scrollTo=4yN3Bo6iAH2L) below to get started with the model.
|
36 |
-
|
|
|
|
|
37 |
```python
|
38 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
39 |
tokenizer = AutoTokenizer.from_pretrained( "airesearch/WangchanLion7B", trust_remote_code=True)
|
@@ -68,3 +70,8 @@ output = model.generate(
|
|
68 |
print(tokenizer.decode(output[0], skip_special_tokens=True))
|
69 |
```
|
70 |
|
|
|
|
|
|
|
|
|
|
|
|
33 |
|
34 |
# How to Get Started with the Model
|
35 |
Use the code [here](https://colab.research.google.com/drive/1y_7oOU3ZJI0h4chUrXFL3K4kelW_OI2G?usp=sharing#scrollTo=4yN3Bo6iAH2L) below to get started with the model.
|
36 |
+
|
37 |
+
Or
|
38 |
+
|
39 |
```python
|
40 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
41 |
tokenizer = AutoTokenizer.from_pretrained( "airesearch/WangchanLion7B", trust_remote_code=True)
|
|
|
70 |
print(tokenizer.decode(output[0], skip_special_tokens=True))
|
71 |
```
|
72 |
|
73 |
+
# Training Details
|
74 |
+
## Training Data
|
75 |
+
Finetuning datasets are sourced from [LAION OIG chip2 and infill_dbpedia (Apache-2.0)](https://huggingface.co/datasets/laion/OIG), [DataBricks Dolly v2 (Apache-2.0)](https://github.com/databrickslabs/dolly), [OpenAI TL;DR (MIT)](https://github.com/openai/summarize-from-feedback), [Hello-SimpleAI HC3 (CC-BY SA)](https://huggingface.co/datasets/Hello-SimpleAI/HC3), [dolphin](https://huggingface.co/datasets/ehartford/dolphin), [iapp_wiki_qa_squad](https://huggingface.co/datasets/iapp_wiki_qa_squad) , [thaisum](https://huggingface.co/datasets/thaisum), [xlsum](https://huggingface.co/datasets/csebuetnlp/xlsum), [scb_mt_enth_2020](https://huggingface.co/datasets/scb_mt_enth_2020), han dataset, [xp3x](https://huggingface.co/datasets/Muennighoff/xP3x) and [Open-Platypus](https://huggingface.co/datasets/garage-bAInd/Open-Platypus).
|
76 |
+
## Training regime
|
77 |
+
- QLoRA with 4 GPUs. (A100 40GB?)
|