ayoolaolafenwa commited on
Commit
6dbd6e9
1 Parent(s): 3dfe921

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -3
README.md CHANGED
@@ -7,7 +7,7 @@ pipeline_tag: conversational
7
  ---
8
 
9
  ## ChatLM
10
- It is a chat Large Language model finetuned with pretrained [Falcon-1B model](https://huggingface.co/tiiuae/falcon-rw-1b)
11
  and trained on [chat-bot-instructions prompts dataset](https://huggingface.co/datasets/ayoolaolafenwa/sft-data).
12
  ChatLM was trained on a dataset containing normal day to day human conversations, due to limited data used in training
13
  it does not generalize well for tasks like coding and current affairs.
@@ -123,12 +123,12 @@ new_data = pd.DataFrame({"prompt": prompts, "response": responses})
123
  # Write the new dataframe to a csv file
124
  new_data.to_csv("MyData/chatbot_instruction_prompts_train.csv", index=False)
125
  ```
126
- The user's prompts in the dataset are appended with the tag <user> and the corresponding responses with the tag <chatbot>.
127
  Check the the modified dataset https://huggingface.co/datasets/ayoolaolafenwa/sft-data .
128
 
129
  ### Training
130
 
131
  ChatLM was supervised finetuned with pretrained [Falcon 1-Billion parameters model](https://huggingface.co/tiiuae/falcon-rw-1b) trained on 350-Billion tokens
132
- of RefinedWeb. It was trained with a single H100 GPU for 1 epoch. Check the full code for supervised finetune
133
  training on its github repository https://github.com/ayoolaolafenwa/ChatLM/tree/main
134
 
 
7
  ---
8
 
9
  ## ChatLM
10
+ It is a chat Large Language Model finetuned with pretrained [Falcon-1B model](https://huggingface.co/tiiuae/falcon-rw-1b)
11
  and trained on [chat-bot-instructions prompts dataset](https://huggingface.co/datasets/ayoolaolafenwa/sft-data).
12
  ChatLM was trained on a dataset containing normal day to day human conversations, due to limited data used in training
13
  it does not generalize well for tasks like coding and current affairs.
 
123
  # Write the new dataframe to a csv file
124
  new_data.to_csv("MyData/chatbot_instruction_prompts_train.csv", index=False)
125
  ```
126
+ The users` prompts in the dataset are appended with the tag <user> and the corresponding responses with the tag <chatbot>.
127
  Check the the modified dataset https://huggingface.co/datasets/ayoolaolafenwa/sft-data .
128
 
129
  ### Training
130
 
131
  ChatLM was supervised finetuned with pretrained [Falcon 1-Billion parameters model](https://huggingface.co/tiiuae/falcon-rw-1b) trained on 350-Billion tokens
132
+ of RefinedWeb. It was trained with a single H100 GPU for 1 epoch. It achieves Perplexity *1.738*. Check the full code for supervised finetune
133
  training on its github repository https://github.com/ayoolaolafenwa/ChatLM/tree/main
134