risenyard/egms-qa-translator

EGMS-QA Translator

Projectors and LoRA adapters for answering questions from frozen EGMS tile tokens with Qwen, Gemma, Llama, or Mistral. This repository provides the four variants, their training recipes, and evaluation results. Host-model weights are obtained separately.

Use the models

See the architecture on GitHub, then follow installation and evaluation to run a released variant. The same guide covers training and checkpoint evaluation. The Dataset supplies precomputed encoder tokens, labels, and task tables.

Variants

variant	host model	training recipe
qwen	Qwen/Qwen3.5-9B	recipe
gemma	unsloth/gemma-3-12b-it	recipe
llama	unsloth/Meta-Llama-3.1-8B-Instruct	recipe
mistral	unsloth/Mistral-Nemo-Instruct-2407	recipe

Each variant's translator_config.json records the exact host-model revision.

Files

manifest.json
evaluation_config.json
qwen/                         # gemma/, llama/, and mistral/ use the same layout
├── adapter/
│   ├── adapter_config.json
│   └── adapter_model.safetensors
├── projector.safetensors
├── translator_config.json
├── training_args.json
└── eval_results.json

Model loading requires the adapter, projector weights, and translator config. The training recipe and evaluation results are stored beside each variant. The manifest indexes the four variants, and the evaluation config defines their shared reporting protocol.

Input and output

stage	contract
tile input	65 tokens of width 256 and a 65-element validity mask
projector	256 → host embedding width → host embedding width
language input	projected tile tokens followed by the tokenized question
output	a natural-language answer

Token 0 summarizes the tile. Tokens 1–64 represent an 8×8 spatial grid in row-major order. Projector dimensions and the prompt format are recorded in each variant's translator_config.json.

Evaluation

Evaluation measures whether the translator can report numerical properties, assign the appropriate categories, and refuse questions outside the supported scope. These correspond to the numeric, categorical, and boundary tasks in the EGMS-QA task catalog. Each task specifies a question target and how its answer is scored.

The reported evaluation uses 71 tasks on 1,000 held-out tiles, with one question phrasing per tile and task, yielding 71,000 answers per model. The table reports mean R² over 29 numeric tasks and mean balanced accuracy over 28 categorical and 14 boundary tasks.

variant	numeric R²	categorical balanced accuracy	boundary balanced accuracy	parsed answers
qwen	0.727	0.728	1.000	70,797 / 71,000
gemma	0.744	0.757	1.000	70,880 / 71,000
llama	0.770	0.777	0.999	70,985 / 71,000
mistral	0.778	0.760	1.000	70,531 / 71,000

The evaluation configuration records the reporting subset, question-phrasing pool, model-specific seeds, and greedy-decoding settings. The evaluator retains parse counts and per-task scores alongside the three macro averages.

Scope and license

The translators require tokens from the released EGMS-QA Encoder and questions within the EGMS-QA task definitions. Answers describe measured vertical displacement histories. They do not establish causes, forecast motion, or certify structural safety.

The projector and LoRA weights are released under CC-BY-4.0. Each host model remains subject to its own license and terms of use.