README.md

# README: tinyChat Instruction-Based LLM

Introducing tinyChat, the instruction-based Large Language Model (LLM) that’s less than 1% the size of GPT-3.5. tinyChat is an open-source model under the Apache 2.0 license and based on Google’s Flan-T5-Large, a 770m parameter model. Although not as performant as larger models, tinyChat can perform a variety of NLP tasks such as summarization, question answering, and sentiment analysis.

tinyChat is available on the HuggingFace model hub and the code repository is on GitHub. While tinyChat is open-sourced, we do not recommend using it in a production setting in its current state.


## Use Cases

- Chatbots
- Summarization
- Sentiment analysis
- Q&A systems
- Text completion
- Language modeling
- Mobile applications
- Complementing larger LLMs

## Future Directions

- Improving model accuracy
- Reducing biases and toxicity
- Developing new datasets
- Collaborating with the open-source community
- Applying tinyChat to new domains

## Acknowledgements

We express our gratitude to OpenAI, Hugging Face, Microsoft Research, and the creators of the Pile, Alpaca, and Databricks 15k datasets for their contributions to the landscape of open-source machine learning and the advancement of generative AI.

## Running the Code

```python
import transformers
from transformers import PeftModel

model_name = "google/flan-t5-large"
peft_model_id = "ckpts_databricks_large"
tokenizer = transformers.AutoTokenizer.from_pretrained(model_name)
base_model = transformers.AutoModelForSeq2SeqLM.from_pretrained(model_name)
peft_model = PeftModel.from_pretrained(base_model, peft_model_id)

inputs = tokenizer("""[INSERT INSTRUCTION HERE]""", return_tensors="pt")
outputs = peft_model.generate(**inputs, max_length=300, do_sample=True)
print(tokenizer.batch_decode(outputs, skip_special_tokens=True))



---
license: apache-2.0
---