---
license: apache-2.0
datasets:
- wikipedia
language:
- zh
metrics:
- perplexity
library_name: transformers
pipeline_tag: text-generation
tags:
- code
---

语言模型参数：vocab_size=12829,num_hidden_layers=8,num_attention_heads=8,intermediate_size=1024,max_position_embeddings=512,hidden_size=512
语言模型数据：10G数据
finetune：alpaca 51K条数据