Instructions to use Tweeties/tweety-7b-dutch-v24a with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Tweeties/tweety-7b-dutch-v24a with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Tweeties/tweety-7b-dutch-v24a")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Tweeties/tweety-7b-dutch-v24a")
model = AutoModelForCausalLM.from_pretrained("Tweeties/tweety-7b-dutch-v24a")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Tweeties/tweety-7b-dutch-v24a with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Tweeties/tweety-7b-dutch-v24a"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Tweeties/tweety-7b-dutch-v24a",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Tweeties/tweety-7b-dutch-v24a

SGLang

How to use Tweeties/tweety-7b-dutch-v24a with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Tweeties/tweety-7b-dutch-v24a" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Tweeties/tweety-7b-dutch-v24a",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Tweeties/tweety-7b-dutch-v24a" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Tweeties/tweety-7b-dutch-v24a",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Tweeties/tweety-7b-dutch-v24a with Docker Model Runner:
```
docker model run hf.co/Tweeties/tweety-7b-dutch-v24a
```

Pieter Delobelle, François Remy, Miryam de Lhoneux, Thomas Demeester

Tweety-7b-dutch: A Dutch Large Language Model

🇳🇱🇧🇪 Er is ook een Nederlandse readme

Model Card for tweety-7b-dutch

tweety-7b-dutch is a foundation model with a focus on the Dutch language, incorporating a Dutch tokenizer for better understanding and generation of Dutch text. It's built on the mistral architecture, employing flash attention for efficient processing within a context window of 8192 tokens. Tweety-7b-dutch is trained on the cleaned Dutch mC4 dataset, without instruction finetuning.

Model Details

Model Description

Our tweety-7b-dutch model has an Apache 2.0 license, encouraging applications in research, content creation, and language analysis.

Tokenizer: Dutch, 50k tokens (yhavinga/gpt-neo-1.3B-dutch)
Pre-training data: Scraped Dutch (yhavinga/mc4_nl_cleaned)
Context window: 8196 tokens
Training data: 8.5B tokens
Developed by: KU Leuven and UGent
Funded by: KU Leuven BOF, VSC (Flemish Supercomputer Center), Vlaams AI-onderzoeksprogramma
Model type: Foundation model
License: Apache 2.0

Uses

As a base model, tweety-7b-dutch is primed for direct applications across text generation and understanding within the Dutch language.

Technical Specifications

Compute Infrastructure

Training utilized Nvidia H100 and A100 GPUs. Inference is accessible on lower-end GPUs, basically any GPU capable of running mistral models.

Model Weights

This model was trained in bfloat16.
GGUF weights are released by Bram Vanroy.

Citation

If you use this model, please cite our work as:

@article{tweeties2024,
    title = {Trans-Tokenization and Cross-lingual Vocabulary Transfers: Language Adaptation of LLMs for Low-Resource NLP},
    author = {François Remy and Pieter Delobelle and Hayastan Avetisyan and Alfiya Khabibullina and Miryam de Lhoneux and Thomas Demeester},
    url = {https://arxiv.org/abs/2408.04303},
    year = {2024},
    note = {Accepted at COLM 2024}
}

Downloads last month: 29

Safetensors

Model size

7B params

Tensor type

BF16

Space using Tweeties/tweety-7b-dutch-v24a 1

Paper for Tweeties/tweety-7b-dutch-v24a

Trans-Tokenization and Cross-lingual Vocabulary Transfers: Language Adaptation of LLMs for Low-Resource NLP

Paper • 2408.04303 • Published Aug 8, 2024 • 22