Lissa1-4B-AWQ

A practical 4-bit CUDA serving build for developers.

Use this repository when you want a compact Lissa1 developer assistant behind vLLM or SGLang and FP8 is not the right memory/throughput trade-off for your GPU. AWQ is useful for dense local services, multi-tenant endpoints and machines where keeping more model replicas matters.

The model is aimed at real engineering work: code review, debugging, SQL, APIs, infrastructure, CI/CD, repository explanations and structured tool workflows. It is a mini senior developer, not a replacement for test execution or human review.

vLLM

pip install -U vllm
vllm serve lomelidev/Lissa1-4B-AWQ \
  --quantization awq \
  --dtype auto \
  --max-model-len 4096 \
  --gpu-memory-utilization 0.90 \
  --host 127.0.0.1 \
  --port 8000

curl http://127.0.0.1:8000/v1/chat/completions \
  -H 'content-type: application/json' \
  -d '{
    "model":"lomelidev/Lissa1-4B-AWQ",
    "messages":[{"role":"user","content":"Explain this Kubernetes failure and give a safe diagnostic sequence."}],
    "temperature":0.2,
    "top_p":0.9,
    "max_tokens":768
  }'

SGLang

python -m sglang.launch_server \
  --model-path lomelidev/Lissa1-4B-AWQ \
  --host 127.0.0.1 \
  --port 30000 \
  --mem-fraction-static 0.90 \
  --context-length 4096

Check the installed version's quantization help if it does not auto-detect the AWQ config. Do not convert this checkpoint with AutoAWQ: Qwen3.5 hybrid attention requires the modern AWQ export path used for this repository.

Sampling presets

deterministic patch/tool call: temperature=0, top_p=1;
debugging and code explanation: temperature=0.2, top_p=0.9;
design alternatives: temperature=0.4–0.6, top_p=0.9.

For a production agent, set a token budget, validate tool arguments, and require a test or read-back after a write operation.

Measured A100 results

Engine	Target only	+ community DFlash	+ own DFlash
vLLM	197.95 tok/s	294.91 tok/s	153.0 tok/s
SGLang	204.12 tok/s	242.44 tok/s	138.8 tok/s

The community DFlash result is the external z-lab/Qwen3.5-4B-DFlash. Our current draft is experimental and net-negative in this matrix; it is retained for retraining research, not recommended for deployment.

Long context, slots and KV cache

The long-context recipe comes from the historical Ornith ia-lab report and is being reproduced for Lissa1 rather than copied as a claim. Test slots and total context independently, then record prefill/decode, VRAM and correctness:

max_num_seqs: 1, 2, 4, 8, 16
max_num_batched_tokens: 1024, 2048, 4096, 8192
max_model_len: 4096, 8192, 32768, 131072, then larger if it fits
KV: auto, fp8, int8 probe, int4 probe
prefix caching: off/on

A 4-bit AWQ weight checkpoint does not imply a 4-bit KV cache. The engine's accepted --kv-cache-dtype values must be recorded exactly. Unsupported int4/int8 probes are useful results, but they are not successful quantization modes.

Links

Future QLoRAs will specialize the same base for database, frontend, backend, devops/cicd, agents/tools, security and platform operations. They will be published separately with their own evals and regression gates.