|
# README: tinyChat Instruction-Based LLM |
|
|
|
Introducing tinyChat, the instruction-based Large Language Model (LLM) that’s less than 1% the size of GPT-3.5. tinyChat is an open-source model under the Apache 2.0 license and based on Google’s Flan-T5-Large, a 770m parameter model. Although not as performant as larger models, tinyChat can perform a variety of NLP tasks such as summarization, question answering, and sentiment analysis. |
|
|
|
tinyChat is available on the HuggingFace model hub and the code repository is on GitHub. While tinyChat is open-sourced, we do not recommend using it in a production setting in its current state. |
|
|
|
|
|
## Use Cases |
|
|
|
- Chatbots |
|
- Summarization |
|
- Sentiment analysis |
|
- Q&A systems |
|
- Text completion |
|
- Language modeling |
|
- Mobile applications |
|
- Complementing larger LLMs |
|
|
|
## Future Directions |
|
|
|
- Improving model accuracy |
|
- Reducing biases and toxicity |
|
- Developing new datasets |
|
- Collaborating with the open-source community |
|
- Applying tinyChat to new domains |
|
|
|
## Acknowledgements |
|
|
|
We express our gratitude to OpenAI, Hugging Face, Microsoft Research, and the creators of the Pile, Alpaca, and Databricks 15k datasets for their contributions to the landscape of open-source machine learning and the advancement of generative AI. |
|
|
|
## Running the Code |
|
|
|
```python |
|
import transformers |
|
from transformers import PeftModel |
|
|
|
model_name = "google/flan-t5-large" |
|
peft_model_id = "ckpts_databricks_large" |
|
tokenizer = transformers.AutoTokenizer.from_pretrained(model_name) |
|
base_model = transformers.AutoModelForSeq2SeqLM.from_pretrained(model_name) |
|
peft_model = PeftModel.from_pretrained(base_model, peft_model_id) |
|
|
|
inputs = tokenizer("""[INSERT INSTRUCTION HERE]""", return_tensors="pt") |
|
outputs = peft_model.generate(**inputs, max_length=300, do_sample=True) |
|
print(tokenizer.batch_decode(outputs, skip_special_tokens=True)) |
|
|
|
|
|
|
|
--- |
|
license: apache-2.0 |
|
--- |
|
|