Quantization made by Richard Erkhov.

[Github](https://github.com/RichardErkhov)

[Discord](https://discord.gg/pvy7H8DZMG)

[Request more models](https://github.com/RichardErkhov/quant_request)


Pythia410m-V0-Instruct - bnb 4bits
- Model creator: https://huggingface.co/SummerSigh/
- Original model: https://huggingface.co/SummerSigh/Pythia410m-V0-Instruct/




Original model description:
---
license: apache-2.0
---

# Model info
This is EleutherAI/pythia-410m finetuned on OpenAssistant/oasst_top1_2023-08-25

# Why 
Plain and simple. Im experimenting with making instruction LLMs under 1B params. I think we can still squeeze out better performance out of these models. 

# Random Notes
- Only using OpenAssistant data gives fantastic results becuase of its high quality. I like the top1 dataset becuase of it's lack of prompt refusals.
- Prompt refusals have been shown to damage the performance of instruction LLMs. My theory is that the model "spends" parameters learning how to refuse prompts rather than learning actually useful information. Adding to this, I think that unlike other tasks, learning prompt refusals most likely has no other value in terms of transfer learning.

# Usage
```
from transformers import pipeline

pipe = pipeline("text-generation", model="SummerSigh/Pythia410m-V0-Instruct")

out= pipe("<|im_start|>user\nWhat's the meaning of life?<|im_end|>\n<|im_start|>assistant\n",max_length  = 500,repetition_penalty = 1.2, temperature  = 0.5, do_sample = True)

print(out[0]["generated_text"])
```
# Contact 
If you want to contact me and work with me on making good under 1B param models, you can reach me on discord at summer_ai.