Heed tasks-only, Qwen3.5-0.8B (EXP-037), research checkpoint

The 0.8B model that was not shipped. It was the best model in the Heed study's first tasks-only round. The app moved to Qwen3-0.6B anyway, and this page explains why.

It is a research checkpoint. It writes task text only (no buckets), was trained on an older label set, and has no judged score. Do not compare its numbers with the newer models. The current model is heed-tasks-qwen3-0.6b.

Format

Note: <the note text>
Tasks:

The answer is a list of strings, {"tasks":["Book the flight to …","…"]}, then EOS. It writes no bucket, time or reminder. Use greedy decoding and no chat template.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "mohitkrconnect/heed-tasks-only-qwen3.5-0.8b"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, dtype=torch.bfloat16)
prompt = "Note: buy milk and call the plumber tomorrow at nine\nTasks:\n"
ids = tok(prompt, return_tensors="pt").input_ids
out = model.generate(ids, max_new_tokens=512, do_sample=False)
print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))

The architecture is Qwen3_5ForCausalLM (text only), with 24 layers and hybrid Gated DeltaNet linear attention. It needs transformers with Qwen3.5 support (5.20.2 was used); training also used flash-linear-attention 0.3.2 and causal-conv1d 1.7.0.

Training

Setting Value
Base Qwen/Qwen3.5-0.8B-Base, full fine-tune, 752,393,024 trainable parameters
Data 4,221 notes, one row each. HeedSet (synthetic, Claude Opus; 1,665), simulated user-style notes (1,109), notes I planned generated with ChatGPT (781), Claude brain dumps (480), curated template notes (36), real notes from me and five friends (140) and 10 typed by me
Labels The older mixed labels of that stage: Sonnet relabelling of 777 notes plus 1,265 Sonnet-labelled HeedSet notes. They keep hedges like "Maybe…" on about 1 in 5 hedged tasks, which a later relabelling removed
Recipe lr 1e-5, AdamW, 5% warm-up + cosine, batch 4 × 4 = 16, 3 epochs (792 steps), max length 1,024, loss on answer tokens only, bf16 autocast, seed 0
Hardware / time / cost 1 × RTX 4090 24 GB, 20.9 minutes, peak 15.6 GB; about $0.75 of GPU (estimate)

Training curves

Results (word-overlap F1 only; no judged score; test answers never scored)

Word-overlap F1 against the older key, on 371 validation notes. The "yardstick" is 199 of them: 59 real notes plus 140 HeedSet notes.

Epoch 2 Epoch 3 (this model) Seed 1, epoch 3
Yardstick F1 90.7 91.6 91.1
Exact lists, yardstick 71.4% 72.4% 69.3%
Real notes (59) F1 79.4 81.5 80.1
Exact lists, real notes 54.2% 55.9% 49.2%

On the yardstick at epoch 3, recall was 92.7 and precision 90.5, and all 199 answers were valid JSON. On the same set the Qwen3-0.6B twin (heed-tasks-only-qwen3-0.6b) scores 91.2, with 77.9 on real notes.

Why it was not shipped

  1. Converter support. Google's official LiteRT-LM converter supported Qwen3 but not Qwen3.5's Gated DeltaNet layers at the time (decision D103). The community route needed a patch, and that was judged too risky for a shipping app. A newer litert-torch later did export this model, but the result was never checked for parity or run on a phone, so it is not published here.
  2. Speed. An earlier phone test found the linear-attention layers slow on a phone CPU.
  3. Size and cost. Its int8 file is 779 MB against 614 MB for the 0.6B, and it took 21 minutes to train against 7.
  4. A small quality gap. On word-overlap F1 it was 91.6 against 91.2 on the yardstick, and 81.5 against 77.9 on real notes.

Limitations

  • It writes tasks only, with no buckets or times.
  • It was trained on old labels that keep hedges.
  • It had no long-note data and no lowercase or unpunctuated training form.
  • It was never judged.

All the later improvements in the study (relabelling, long notes, plain-text training, targeted data) went into the 0.6B line.

Data: the synthetic part of the training, validation and test notes, with all five systems' answers and the judge's verdicts on the public test notes, is in mohitkrconnect/heed-voice-notes-to-tasks. My real notes and my friends' are not published.

License

Apache-2.0, the same as the base model. The training labels and most training notes were produced with Claude, Gemini and GPT models. Check those providers' terms before using this model to build a competing general-purpose model.

Downloads last month
62
Safetensors
Model size
0.8B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for mohitkrconnect/heed-tasks-only-qwen3.5-0.8b

Finetuned
(169)
this model

Dataset used to train mohitkrconnect/heed-tasks-only-qwen3.5-0.8b

Collection including mohitkrconnect/heed-tasks-only-qwen3.5-0.8b