roomote/roomote-judgment-gliner

Roomote judgment model (GLiNER, CPU)

A CPU decision model for self-hosted Roomote. Roomote asks a judgment model small typed questions (yes/no, one of several options, a graded score) over a piece of state, such as whether a thread reply is meant for Roomote. This checkpoint answers them on four CPU cores, so a deployment without a hosted judgment model or a GPU can still turn those decisions on.

It is GLiNER 2.5 base (a DeBERTa-v3 schema classifier, about 194M parameters) fine-tuned on Roomote's decisions. Serve it with the judgment sidecar in Roomote's self-hosting compose file (the judgment profile), which speaks Roomote's decision API.

Decisions

Trained and evaluated on the decisions Roomote asks a judgment model:

• whether a settled turn holds something worth saving to Memory, and what a finished task run should distill into Memory
• whether an unmentioned thread reply is addressed to Roomote
• when a running task has something the user needs to hear
• which model a delegated task should run on, from the request and the deployment's model routing rules
• whether a channel message meets an auto-respond launch criterion
• whether a request asks a question, wants a plan, or wants an implementation
• whether inbound email is an automatic reply
• whether a paused tool call is risky enough to ask about (risk level, whether it matches the request, untrusted content, data leaving the workspace)
• the outcome of a Session (open, done, blocked, needs input)
• whether a custom automation should run: its launch criteria and the run conditions its author wrote

Roomote enables this model per decision, only for decisions it has been evaluated on.

Training data

6,963 decisions:

• 6,459 synthetic decisions. Each was written by an open-weight model (DeepSeek V4 Pro) to fit sampled target answers, then labelled blind by a second open-weight model from a different family (GLM-5.2).
• 504 anonymized rewrites of decisions from Roomote's own internal deployment. Each was rewritten as a different, fictional team's (every name, handle, company, repository, URL, path, id, number, code sample, and quoted text replaced) and rejected if it still carried any identifier or copied text from its source; the originals were never trained on. No customer data is used.

Each question is trained toward a probability rather than a hard yes/no, with no target below 5% or above 95%, so the model answers with graded confidence and a threshold on its answers means something. Three epochs, learning rate 4e-5.

Evaluation

Scored through the serving path on a 4-core CPU container with a 12 GB memory cap. The held-out sets are never trained on; the real decisions come from captures kept out of the rewriting above.

Decision	Accuracy
Memory check	0.947
Memory distillation	0.938
Reply addressee	0.982
Task communication	0.947
Delegated task model	0.974
Channel launch criteria	0.977
Work kind	0.952
Automatic email replies	0.967
Tool-call approval	0.970
Session outcome	0.893
Custom automation launch	0.807
All synthetic (1,828 questions)	0.941
Real decisions (191 questions)	0.948

On 223 further real Memory checks from the same internal deployment (1,338 questions), it gives the same answer as Roomote's hosted judgment option 92.3% of the time (the previous release: 91.9%).

Latency per decision: about 1 to 2.5 s on four CPU cores; on a T4 GPU, about 0.5 s.

Roomote acts on a judgment only when it is confident and otherwise keeps its behavior without a judgment model.

Revisions

Commit	Change
032d316	First release
6b7fa81	Adds session-coordination turns, which the first release often took for durable preferences
cba76a6	Graded confidence: trained toward probabilities instead of hard yes/no
this commit	Adds tool-call approval, Session outcome, and custom automation launch decisions; anonymized rewrites of real decisions; drops the retired closing-acknowledgement question

Limitations

• English only. States are cut at 8,000 characters.
• The option labels are part of the question; decisions whose wording differs from Roomote's may not transfer.
• Custom automation run conditions are written by users; conditions unlike the ones it was trained on are answered less reliably.

License

Apache 2.0, as is the base model.