🤗 [LongWriter Dataset] • 💻 [Github Repo] • 📃 [LongWriter Paper]
LongWriter-llama3.1-8b is trained based on [Meta-Llama-3.1-8B](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B), and is capable of generating 10,000+ words at once. A simple demo for deployment of the model: ```python from transformers import AutoTokenizer, AutoModelForCausalLM import torch tokenizer = AutoTokenizer.from_pretrained("THUDM/LongWriter-llama3.1-8b", trust_remote_code=True) model = AutoModelForCausalLM.from_pretrained("THUDM/LongWriter-llama3.1-8b", torch_dtype=torch.bfloat16, trust_remote_code=True, device_map="auto") model = model.eval() query = "Write a 10000-word China travel guide" prompt = f"[INST]{query}[/INST]" input = tokenizer(prompt, truncation=False, return_tensors="pt").to(device) context_length = input.input_ids.shape[-1] output = model.generate( **input, max_new_tokens=32768, num_beams=1, do_sample=True, temperature=0.5, )[0] response = tokenizer.decode(output[context_length:], skip_special_tokens=True) print(response) ``` Please ahere to the prompt template (system prompt is optional): `<