Instructions to use hathibelagal/llama-3.2-latin with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use hathibelagal/llama-3.2-latin with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="hathibelagal/llama-3.2-latin")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("hathibelagal/llama-3.2-latin")
model = AutoModelForCausalLM.from_pretrained("hathibelagal/llama-3.2-latin")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use hathibelagal/llama-3.2-latin with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "hathibelagal/llama-3.2-latin"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "hathibelagal/llama-3.2-latin",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/hathibelagal/llama-3.2-latin

SGLang

How to use hathibelagal/llama-3.2-latin with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "hathibelagal/llama-3.2-latin" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "hathibelagal/llama-3.2-latin",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "hathibelagal/llama-3.2-latin" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "hathibelagal/llama-3.2-latin",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use hathibelagal/llama-3.2-latin with Docker Model Runner:
```
docker model run hf.co/hathibelagal/llama-3.2-latin
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Model Card for Llama-3.2-Latin

hathibelagal/llama-3.2-latin is a finetuned version of the LLaMA-3.2-3B model, optimized for generating and understanding Latin text across various historical periods, from ancient to modern Neo-Latin. At this point, it generates content with accurate use of tenses (e.g., pluperfect, subjunctive), cases, and complex structures (e.g., concessive, temporal clauses).

Intended Use

Suitable for tasks involving Latin text generation, translation, or analysis, such as generating Classical Latin prose, completing sentences, or aiding in Latin education. Best for short, context-specific prompts due to coherence limitations.
Not recommended for long-form narrative generation or tasks requiring strict contextual consistency until coherence improves.

Ethical Considerations

Bias: The model may reflect biases in the training dataset, such as overrepresentation of certain Latin styles (e.g., ecclesiastical Latin) leading to tonal shifts in output.
Usage: Generated text should be reviewed for accuracy, especially in educational or scholarly contexts.

Usage Example

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

repo_id = "hathibelagal/llama-3.2-latin"

tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(
    repo_id,
    torch_dtype=torch.float16,
    device_map="auto"
)

prompt = "Libellus vere aureus"
inputs = tokenizer.encode(prompt, return_tensors="pt")
inputs = inputs.to(model.device)

outputs = model.generate(
    inputs,
    max_new_tokens=100,
    do_sample=True,
    top_p=0.9,
    temperature=0.6,
    repetition_penalty=1.2
)

print(tokenizer.batch_decode(
    outputs, skip_special_tokens=True)[0])

Downloads last month: 31

Safetensors

Model size

3B params

Tensor type

F16

Model tree for hathibelagal/llama-3.2-latin

Base model

meta-llama/Llama-3.2-3B

Finetuned

(480)

this model

Quantizations

2 models

hathibelagal
/

llama-3.2-latin

Model Card for Llama-3.2-Latin

Usage Example

Model tree for hathibelagal/llama-3.2-latin

Dataset used to train hathibelagal/llama-3.2-latin