File size: 845 Bytes
e625cbd |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 |
---
license: mit
datasets:
- g0ster/TinyStories-Korean
language:
- ko
library_name: transformers
---
# TinyStories-Korean-800K
A tiny autoregressive language model trained from scratch.
- Architecture: Llama
- Vocab size: 4096
- Hidden size: 64
- Layers: 5
- Heads: 8 (MHA)
- Context length: up to 512 tokens
## Note
This model was trained for experimental purposes and the generated output may not make any sense at all.
## Usage
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained('northwind33/TinyStories-Korean-800K')
tokenizer = AutoTokenizer.from_pretrained('northwind33/TinyStories-Korean-800K')
input_text = ''
input_ids = tokenizer(input_text, return_tensors='pt').input_ids
output = model.generate(input_ids, max_length=512, do_sample=True, temperature=0.5)
``` |