|
|
|
--- |
|
|
|
tags: |
|
- code |
|
base_model: |
|
- 01-ai/Yi-Coder-9B |
|
library_name: transformers |
|
pipeline_tag: text-generation |
|
license: apache-2.0 |
|
|
|
--- |
|
|
|
[](https://hf.co/QuantFactory) |
|
|
|
|
|
# QuantFactory/Typst-Coder-9B-GGUF |
|
This is quantized version of [TechxGenus/Typst-Coder-9B](https://huggingface.co/TechxGenus/Typst-Coder-9B) created using llama.cpp |
|
|
|
# Original Model Card |
|
|
|
|
|
# Typst-Coder |
|
|
|
<p align="center"> |
|
<a href="https://huggingface.co/TechxGenus/Typst-Coder-1.5B">[🤖Models]</a> | |
|
<a href="https://github.com/TechxGenus/Typst-Coder">[🛠️Code]</a> | |
|
<a href="https://huggingface.co/datasets/TechxGenus/Typst-Train">[📊Data]</a> | |
|
</p> |
|
|
|
<hr> |
|
|
|
- [Typst-Coder](#typst-coder) |
|
- [Introduction](#introduction) |
|
- [Usage](#usage) |
|
|
|
<hr> |
|
|
|
## Introduction |
|
|
|
While working with Typst documents, we noticed that AI programming assistants often generate poor results. I understand that these assistants may perform better in languages like Python and JavaScript, which benefit from more extensive datasets and feedback signals from executable code, unlike HTML or Markdown. However, current LLMs even frequently struggle to produce accurate Typst syntax, including models like GPT-4o and Claude-3.5-Sonnet. |
|
|
|
Upon further investigation, we found that because Typst is a relatively new language, training data for it is scarce. GitHub's search tool doesn't categorize it as a language for code yet, and The Stack v1/v2 don’t include Typst. No open code LLMs currently list it as a supported language, either. To address this, we developed this project aimed at collecting relevant data and training models to improve Typst support in AI programming tools. |
|
|
|
## Usage |
|
|
|
An example script is shown below: |
|
|
|
```python |
|
import torch |
|
from transformers import AutoTokenizer, AutoModelForCausalLM |
|
|
|
tokenizer = AutoTokenizer.from_pretrained("TechxGenus/Typst-Coder-9B") |
|
model = AutoModelForCausalLM.from_pretrained( |
|
"TechxGenus/Typst-Coder-9B", |
|
torch_dtype=torch.bfloat16, |
|
device_map="auto" |
|
) |
|
|
|
messages = [ |
|
{"role": "user", "content": "Hi!"}, |
|
] |
|
prompt = tokenizer.apply_chat_template( |
|
messages, |
|
tokenize=False, |
|
add_generation_prompt=True |
|
) |
|
|
|
inputs = tokenizer.encode(prompt, return_tensors="pt") |
|
outputs = model.generate(input_ids=inputs.to(model.device), max_new_tokens=512) |
|
print(tokenizer.decode(outputs[0])) |
|
``` |
|
|
|
|
