tags:
- code
base_model:
- 01-ai/Yi-Coder-9B
library_name: transformers
pipeline_tag: text-generation
license: apache-2.0
QuantFactory/Typst-Coder-9B-GGUF
This is quantized version of TechxGenus/Typst-Coder-9B created using llama.cpp
Original Model Card
Typst-Coder
[🤖Models] | [🛠️Code] | [📊Data] |
Introduction
While working with Typst documents, we noticed that AI programming assistants often generate poor results. I understand that these assistants may perform better in languages like Python and JavaScript, which benefit from more extensive datasets and feedback signals from executable code, unlike HTML or Markdown. However, current LLMs even frequently struggle to produce accurate Typst syntax, including models like GPT-4o and Claude-3.5-Sonnet.
Upon further investigation, we found that because Typst is a relatively new language, training data for it is scarce. GitHub's search tool doesn't categorize it as a language for code yet, and The Stack v1/v2 don’t include Typst. No open code LLMs currently list it as a supported language, either. To address this, we developed this project aimed at collecting relevant data and training models to improve Typst support in AI programming tools.
Usage
An example script is shown below:
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("TechxGenus/Typst-Coder-9B")
model = AutoModelForCausalLM.from_pretrained(
"TechxGenus/Typst-Coder-9B",
torch_dtype=torch.bfloat16,
device_map="auto"
)
messages = [
{"role": "user", "content": "Hi!"},
]
prompt = tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True
)
inputs = tokenizer.encode(prompt, return_tensors="pt")
outputs = model.generate(input_ids=inputs.to(model.device), max_new_tokens=512)
print(tokenizer.decode(outputs[0]))