Instructions to use fraunhofer-iis/elmod-2.7b-it with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use fraunhofer-iis/elmod-2.7b-it with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="fraunhofer-iis/elmod-2.7b-it")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("fraunhofer-iis/elmod-2.7b-it")
model = AutoModelForCausalLM.from_pretrained("fraunhofer-iis/elmod-2.7b-it", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use fraunhofer-iis/elmod-2.7b-it with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "fraunhofer-iis/elmod-2.7b-it"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "fraunhofer-iis/elmod-2.7b-it",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/fraunhofer-iis/elmod-2.7b-it

SGLang

How to use fraunhofer-iis/elmod-2.7b-it with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "fraunhofer-iis/elmod-2.7b-it" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "fraunhofer-iis/elmod-2.7b-it",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "fraunhofer-iis/elmod-2.7b-it" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "fraunhofer-iis/elmod-2.7b-it",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use fraunhofer-iis/elmod-2.7b-it with Docker Model Runner:
```
docker model run hf.co/fraunhofer-iis/elmod-2.7b-it
```

You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

ELMOD 2.7B

We present ELMOD — Efficient Language Model for On-Device Deployment — a compact (2.7B) German language model designed for efficient inference on resource-constrained hardware. This work was performed as research project in the scope of the compute time project ELMOD at NHR@FAU. ELMOD was trained on a limited computational budget (55k H100 GPU hours) using exclusively publicly available data. We developed a suite of German-specific data preprocessing, which differ from English-oriented counterparts in their handling of morphological variation, compounding, and orthographic conventions, to ensure high model performance, we developed a suite of German-specific data preprocessing. Furthermore, we introduced an quality filtering and rephrasing step, which increased the instructional quality of the data, improved performance during the annealing phase, and reduced overall compute requirements. Thanks to our architectural model choices as well as our data choices, including prefiltering, our educational-quality filtering and rephrasal to raise the educational-quality, ELMOD is the strongest performer in its size class (<3B), matching the performance of 7B parameter models in German.

For more details, please refer to our paper!

Model Variants

ELMOD 2.7B Base
ELMOD 2.7B Instruct ← You are here

Model Performance

Evaluation results of our instruction-tuned ELMOD 2.7B model averaged on German benchmarks

Model Summary

ELMOD is a 2.7B dense transformer model.

Property	Value
Total Parameters	2.7B
Layers	32
Context Size	2048
Language	English, German

Usage Example

# pip install transformers torch jinja2
from transformers import AutoModelForCausalLM, AutoTokenizer, GenerationConfig

tokenizer = AutoTokenizer.from_pretrained("fraunhofer-iis/elmod-2.7b-it")
model = AutoModelForCausalLM.from_pretrained("fraunhofer-iis/elmod-2.7b-it")
generation_config = GenerationConfig.from_pretrained("fraunhofer-iis/elmod-2.7b-it")
generation_config.pad_token_id = tokenizer.eos_token_id

messages = [{"role": "user", "content": "Stelle dich vor."}]
prompt_text = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)

inputs = tokenizer(prompt_text, return_tensors="pt")
inputs.pop("token_type_ids", None)
generated_ids = model.generate(**inputs, generation_config=generation_config)
generated = tokenizer.decode(generated_ids[0])
response = generated[len(prompt_text) :]
for stop_token in [tokenizer.eos_token, tokenizer.bos_token, "<|im_end|>"]:
    if stop_token and stop_token in response:
        response = response.split(stop_token)[0]
print(response.strip())

This models is also compatible with vLLM.

Citation

@misc{elmod-2.7b,
  title={From Data to Device: ELMOD. An Efficient German-First 2.7B Language Model for Mobile Inference},
  author={Darina Gold, Alexander Schwirjow, Viktor Haag, Viktor Hangya, Joel Schlotthauer, Fabian Küch und Luzian Hahn},
  year={2026},
  url={https://huggingface.co/fraunhofer-iis/elmod-2.7b-base}
}

Liability Disclaimer

The user assumes full responsibility and liability for any user-driven alteration, customization or modification of the model’s system template. This includes, but is not limited to, any legal, ethical, or security consequences that may arise.

Legal Aspects

EU AI Act Transparency Documentation

Public Training Data Summary for GPAI-Models

Unlawful use (Third Party Rights)

Any use of ELMOD in a manner that infringes, misappropriates, or otherwise violates any third party’s rights, including but not limited to intentionally using ELMOD to generate outputs that infringe, misappropriate, dilute or otherwise violate copyrights is unlawful and prohibited by law.

Data Protection and Copyright Requests

We as Fraunhofer IIS maintain policies to ensure compliance with laws governing copyright material. If you believe your copyrighted material has been used without proper authorization, please contact: copyright-requests@iis.fraunhofer.de

To enable us to process your claim efficiently, please provide as much of the following information as possible in your correspondence:

Required Information	Details
Your contact details	Name, email address, and any other way we can reach you.
Ownership status & proof	Indicate whether you are the copyright owner or acting on behalf of the owner as a third party. If so, provide proof (e.g., copyright registration, original documents, written authorization from the owner, contracts, or other clear evidence).
Description of the copyrighted work	Title, author, and a brief description of the material your claim is related to.
Source domain	The URL, platform, or website where you copyrighted material is located and was potentially used from (e.g., `https://example.com/…`).
Hyperparameters	The model configuration used to generate the content (e.g., model version, temperature, max tokens, top‑p, top‑k, seed). Providing these details allows us to reproduce the exact generation and verify the claim.
Date & time	The date, when the respective incident supposedly happened.
Additional evidence	Any screenshots, logs, or other supporting material.

We will act within reasonable time from receiving a complaint in a diligent and non-arbitrary manner. We reserve the right not to respond if (i) a request is manifestly unfounded or (ii) has already been addressed to an identical or similar complaint by the same rightsholder.

For removal requests of personally identifiable information (PII), please contact: datenschutz@iis.fraunhofer.de. Include the term "ELMOD" in the subject of your request.

Limitations

The generated content may not always be factually accurate, logically consistent, or free from biases present in the training data. These model should not be used as a definitive source of information. Any generated content should always be critically evaluated.

License

This model is licensed under the [Creative Commons Attribution Non Commercial 4.0] (https://spdx.org/licenses/CC-BY-NC-4.0).

Acknowledgments

This work has been funded by the Free State of Bavaria in the DSgenAI project (Grant Nr.: RMF-SG20-3410-2-18-4). The authors gratefully acknowledge the scientific support and HPC resources provided by the Erlangen National High Performance Computing Center (NHR@FAU) of the Friedrich-Alexander-Universität ErlangenNürnberg (FAU) under the NHR project ELMOD: Efficient language models for on-device deployment (Grant Nr.: b239dc). NHR funding is provided by federal and Bavarian state authorities. NHR@FAU hardware is partially funded by the German Research Foundation (DFG) – 440719683.