Instructions to use tharunpranavsakthivel/tinyshell-lfm2.5-230m with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use tharunpranavsakthivel/tinyshell-lfm2.5-230m with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="tharunpranavsakthivel/tinyshell-lfm2.5-230m")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("tharunpranavsakthivel/tinyshell-lfm2.5-230m")
model = AutoModelForCausalLM.from_pretrained("tharunpranavsakthivel/tinyshell-lfm2.5-230m", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use tharunpranavsakthivel/tinyshell-lfm2.5-230m with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "tharunpranavsakthivel/tinyshell-lfm2.5-230m"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tharunpranavsakthivel/tinyshell-lfm2.5-230m",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/tharunpranavsakthivel/tinyshell-lfm2.5-230m

SGLang

How to use tharunpranavsakthivel/tinyshell-lfm2.5-230m with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "tharunpranavsakthivel/tinyshell-lfm2.5-230m" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tharunpranavsakthivel/tinyshell-lfm2.5-230m",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "tharunpranavsakthivel/tinyshell-lfm2.5-230m" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tharunpranavsakthivel/tinyshell-lfm2.5-230m",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use tharunpranavsakthivel/tinyshell-lfm2.5-230m with Docker Model Runner:
```
docker model run hf.co/tharunpranavsakthivel/tinyshell-lfm2.5-230m
```

LFM2.5 TinyShell

Fine-tuned compact language model for the TinyShell ShellIntent natural-language-to-structured-IR task.

Base model

LiquidAI/LFM2.5-230M

This is a supervised fine-tune of the upstream LFM2.5 instruction model. The derivative remains subject to the LFM Open License v1.0; the full license text is included in LICENSE.

Task

The model converts natural-language instructions into structured TinyShell ShellIntent JSON.

Supported high-level decisions include:

compile
clarify
unsupported

Held-out evaluation

Metric	Result
JSON parse rate	99.00%
Schema validity	96.00%
IR exact match	37.50%
Decision accuracy	96.00%
Operation accuracy	64.50%
Slot precision	70.31%
Slot recall	61.66%
Slot F1	65.70%
Risk accuracy	95.50%
Confirmation accuracy	96.00%
Clarify accuracy	100.00%
Unsupported accuracy	60.00%
Multi-operation accuracy	31.11%
Median inference latency	1260.4274300001634 ms

Generation policy

Native generation termination

FunctionGemma and Falcon-H1 initially produced a valid first JSON object but frequently continued generating additional content. Their corrected final evaluation uses a generation-time stopping criterion that terminates once the first complete top-level JSON object is generated. This is generation control, not post-hoc JSON repair.

LFM2.5 terminated correctly under the original inference configuration.

Training

The model was fine-tuned with supervised causal language modeling.

Seed: 42
Best validation loss: 0.09860268847667612
Training time: 827.8969688260004 seconds
Peak GPU memory: 4.324039459228516 GB

Prompt tokens were masked from the language-model loss and the assistant JSON response was used as the supervised target.

Training used 1,600 examples, with 200 validation examples and 200 held-out test examples. The random seed was 42. The frozen source hashes and complete training metadata are included in evaluation/training_result.json.

Included files

Fine-tuned model weights
Model configuration
Tokenizer / processor files
Chat template when saved
Generation configuration when saved
evaluation/final_metrics.json
evaluation/test_predictions.jsonl
evaluation/training_result.json
inference_example.py
requirements.txt
LICENSE and NOTICE
SHA256SUMS.txt

Limitations

This pilot used one training seed. Test-set bootstrap intervals quantify held-out sample uncertainty but do not replace independent repeated training.

Exact ShellIntent matching is intentionally strict: one incorrect operation, argument, or structured field makes the complete IR prediction incorrect.

This model emits untrusted structured intent. Do not execute model output directly. Validate the JSON against the TinyShell schema, compile it through a deterministic platform-aware compiler, apply safety checks, and require user confirmation where appropriate.

License

The model weights are a derivative of LiquidAI/LFM2.5-230M and are released under the LFM Open License v1.0. The full license text is included in LICENSE; attribution is included in NOTICE. Commercial use is subject to the license’s annual-revenue threshold.

The TinyShell training data contribution is attributed under CC BY 4.0. Upstream source material may have separate terms; see the TinyShell dataset documentation for details.

Downloads last month: 392

Safetensors

Model size

0.2B params

Tensor type

F32

Model tree for tharunpranavsakthivel/tinyshell-lfm2.5-230m

Base model

LiquidAI/LFM2.5-230M-Base

Finetuned

LiquidAI/LFM2.5-230M

Finetuned

(36)

this model