zhuu/WinterLM-0.04B

基于Qwen2.5架构的0.04B超小规模大模型，附带一个小规模的tokenizer，只支持中文。

此模型只进行了预训练阶段，可以实现文本续写功能，推荐在领域数据集微调后使用。

Based on the Qwen2.5 architecture, this is an ultra-small-scale large model with 0.04B parameters, accompanied by a small-scale tokenizer that supports only Chinese.

This model has only undergone the pre-training phase and can perform text continuation tasks. It is recommended to fine-tune it on domain-specific datasets before use.

这里提供了一个代码片段，展示如何加载分词器和模型以及如何生成内容。

Here is a code snippet provided to demonstrate how to load the tokenizer and model, as well as how to generate content.

from transformers import AutoTokenizer,AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("zhuu/WinterLM-0.04B")
model = AutoModelForCausalLM.from_pretrained("zhuu/WinterLM-0.04B",torch_dtype="auto",device_map="auto")

inputs = tokenizer(["前三季度公司实现营业收入8.17亿元"], return_tensors="pt").to(model.device)

generated_ids = model.generate(
    input_ids=inputs["input_ids"],
    attention_mask=inputs["attention_mask"],
    max_new_tokens=64,
    eos_token_id=6400,
    repetition_penalty=1.5
)

response = tokenizer.batch_decode(generated_ids, skip_special_tokens=False)[0]
print(response)