bin-intent-gliner2

Intent model for a two-box bin-sorting robot cell (big box and white box), in Spanish, English and Basque. It reads a sentence once and returns a calibrated probability for every option of five questions, in the style of TypeSafe's Jev, and links the pieces a sentence names to the box it names.

"Clasifica las piezas malas en la caja blanca"
  intent   sort 100%   move 0%   pick 0%
  quality  bad 100%    good 0%   any 0%
  box      white 100%  big 0%    other 0%   none 0%
  rule     pieces [bad, all] -> white box     (confidence 99.6%)

question	options
intent	move sort pick both reply ask stop cancel repeat yes no hello chat refuse
feature	bolt · no bolt · any
quality	good · bad · any
box	big · white · other (a box the cell does not have) · none
count	one two three all none

Use

pip install "gliner2[local]==2.0.0" huggingface_hub
# download intent_model.py and grammar_common.py from this repo into the working folder

from intent_model import BinIntentModel          # intent_model.py and grammar_common.py are in this repo
model = BinIntentModel("IkerAzurmendi/bin-intent-gliner2",
                       routes={("bolt", "any"): "big", ("any", "bad"): "white"})   # routes are optional
model.plan("Las piezas con perno a la caja grande y las malas a la blanca").show()

The weights are in best/ (a GLiNER2.5-multi checkpoint, full fine-tune); calibration.json holds one temperature per question. Load it before Isaac Sim starts if you use it next to the simulator.

Training

Fine-tuned from fastino/gliner2.5-multi-v1 on 60,000 synthetic sentences (20,000 per language) from IkerAzurmendi/bin-sorting-intents: fp32, batch 16, 2 epochs, about 45 minutes on one RTX 5060 Ti. Probabilities are calibrated with one temperature per question fitted on a separate set of sentences built with different words.

Measured

Exact match = all five answers right.

set	sentences	exact match
held out, same words as training	2,997	98.6%
words never seen in training	2,997	87.9% (en 94.6, es 89.8, eu 79.3)
written by hand	143	97.9% (en 98.0, es 95.9, eu 100)

No order was sent to the wrong one of the two boxes in about 1,100 orders. Calibration error is 0.002–0.028 per question on unseen words. fp16 gives identical accuracy, 3.4 ms per sentence in a batch and 1.5 GB of GPU memory. Per-set reports are in results/.

Limits

• Training text is synthetic, and so is the 143-sentence hand-written set (written by the same author as the generators), so the scores are optimistic. It has not been tested on audio or on real operators.
• Basque was written without a native speaker; it is the weakest language on unseen wording (79.3%).
• On unseen wording, English slang for "stop" ("cut it out") was read as an order in 5 of 912 sentences. Keep an independent stop in any robot loop.
• Only two boxes (big, white) and two piece properties (bolt, good/bad) exist; anything else is other.
• Dynamic int8 quantisation (14.7% on the hand-written set) and pruning the vocabulary (56% on unseen wording) were tried and rejected.