Instructions to use jeremysoojk/tiny-interjection-model-alpha with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use jeremysoojk/tiny-interjection-model-alpha with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-4B-Instruct-2507")
model = PeftModel.from_pretrained(base_model, "jeremysoojk/tiny-interjection-model-alpha")

Transformers

How to use jeremysoojk/tiny-interjection-model-alpha with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="jeremysoojk/tiny-interjection-model-alpha")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("jeremysoojk/tiny-interjection-model-alpha", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use jeremysoojk/tiny-interjection-model-alpha with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "jeremysoojk/tiny-interjection-model-alpha"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jeremysoojk/tiny-interjection-model-alpha",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/jeremysoojk/tiny-interjection-model-alpha

SGLang

How to use jeremysoojk/tiny-interjection-model-alpha with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "jeremysoojk/tiny-interjection-model-alpha" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jeremysoojk/tiny-interjection-model-alpha",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "jeremysoojk/tiny-interjection-model-alpha" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jeremysoojk/tiny-interjection-model-alpha",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use jeremysoojk/tiny-interjection-model-alpha with Docker Model Runner:
```
docker model run hf.co/jeremysoojk/tiny-interjection-model-alpha
```

Tiny Interjection Model Alpha

Tiny Interjection Model Alpha (TIM) is an experimental floor-control adapter for typed chat. Given a timestamped event stream, it chooses wait, respond, interject, or continue and may emit a short assistant message.

This repository preserves the reviewed adapter from the earlier Nebius L40S virtual-machine experiment. It is not the ephemeral adapter produced by the later credentialless Nebius Serverless AI Job. Both experiments failed the completion-quality release threshold, so this artifact is published for research and reproducibility rather than production use.

Files

adapter_model.safetensors: PEFT LoRA adapter.
adapter_config.json, tokenizer files, and chat_template.jinja: adapter runtime configuration.
tim-lora-f16.gguf: F16 GGUF adapter for use with a compatible Qwen3 4B base GGUF.
eval/base-eval.json and eval/adapter-eval.json: retained 40-case development-suite reports.
training.log: retained training log.
artifact-manifest.json: reviewed SHA-256 hashes and sizes.

Training

Base model: Qwen/Qwen3-4B-Instruct-2507
Method: completion-only 4-bit QLoRA
LoRA rank and alpha: 16 and 16
Hardware: one NVIDIA L40S GPU on Nebius
Training examples: 702 synthetic typed-chat examples
Training steps: 300
Dataset generation: Qwen3.5 through DigitalOcean Serverless Inference

The canonical 702-row training package is public at immutable dataset revision 7eab2028563f17bae3a66c392d0dd9bbf1fe389f. It is the retained package used for this adapter and the package recorded by the later Nebius Serverless job.

Evaluation

On the repeatedly consulted 40-case development suite, the retained adapter reached:

schema validity: 1.000
action accuracy: 0.975
expected-content accuracy: 0.6875
premature-response rate on wait cases: 0.000

The required expected-content threshold was 0.9500. Because the adapter did not pass that gate, these results must not be interpreted as production readiness or unbiased generalization.

Use

Load the adapter with PEFT on top of Qwen/Qwen3-4B-Instruct-2507, or use tim-lora-f16.gguf with a compatible llama.cpp base model. Outputs must be validated before downstream use.

Limitations

The training data is synthetic.
The 40-case suite became a development set after repeated evaluation.
Completion quality is materially below the release threshold.
The model can produce incorrect timing decisions or unsupported message content.

Attribution

Developed and published by Jeremy Soo.

Downloads last month: 15

GGUF

Model size

33M params

Architecture

qwen3

Hardware compatibility

16-bit

Model tree for jeremysoojk/tiny-interjection-model-alpha

Base model

Qwen/Qwen3-4B-Instruct-2507

Adapter

(5650)

this model

jeremysoojk
/

tiny-interjection-model-alpha