JooYoon/riidolaya-difficulty-pilot-v0.2

riidolaya difficulty — experimental MPS pilot

Research/development artifact. Not a production model, not a standalone checkpoint, not a LoRA adapter, not a drop-in Go ONNX model. This is a NEW fixed-tier classifier format, not a replacement for Laya's original choice head. It has no arbitrary-choice input. Fixed output-label ordering is structural; no claim of repaired raw Laya choice-order robustness. No automatic coding, plan generation, model switching or task dispatch is enabled by this release.

학습 실험 공개용입니다. 기본 모델로 권장하지 않습니다. 원본 가중치와 결합해야 하며 기존 Go ONNX 파일을 대체할 수 없습니다. 코드나 계획을 생성하지 않는 결정 모델입니다. 모델 형식과 고정 출력 범위를 먼저 확인해 주세요.

What changed

The original Laya checkpoint stayed entirely frozen; a new 5,123-parameter LayerNorm/Linear fixed-tier classifier was trained on state-only pooled encoder features. Real Apple MPS, FP32, no CPU fallback. pdca-06 trains a new fixed-tier LayerNorm/Linear classifier on state-only mean embeddings from the entirely frozen Laya encoder. Learning rates [0.001, 0.003], up to 100 epochs; two seeds, validation NLL selection. Output-order invariance is structural, not a repair of the original arbitrary-choice head. Selected: learning rate 0.003, epoch 81. Specialist weights occupy 0.020 MiB; original encoder memory and storage are still required.

Choices: fast, standard, strong. Input is task text only. Fixed output labels, pooling and architecture are specified in results.json and the pinned training source. Calibrated temperature: 0.5. Frozen-weight integrity and saved-head reload checks passed.

Evidence and limits

Development test: tuned 24/24, original 15/24; confidence >=0.9 accepts 24/24. Do not describe zero coverage or a regression as improved routing. Partition sizes: {"train": 180, "validation": 36, "calibration": 24, "test": 24}.

264 authored English cases: 180 train, 36 validation, 24 calibration, 24 held-out test. Training includes 24 prior training examples. Prior viewed tests become development data; the final test uses new families. Labels and style remain authored and synthetic; related concepts may recur. Baseline and candidate calibration use the same grid and calibration set. This establishes only an initial small-evaluation result, not repository generalization. Apache-2.0 original fixtures, with no private code, customer prompts or third-party dataset. Labels reflect a rubric, not real downstream model outcomes. Calibration/test samples are small. No cost-saving claim, no Korean validation, no independent repository goldens, no ONNX/Go parity. Optimizer resume is not verified.

Process peak RSS: 2.75 GiB; step-end sampled MPS driver maximum: 2.02 GiB. These overlap in unified memory and must not be added. Driver samples are not continuous peaks. Measured loop including loading, evaluation, guards, hashes and checkpoint writes: 16.75 s; not a throughput benchmark.

Maintainer loading

Use the scripts at teamswyg/laya-tools. Runtime users continue using the existing Go binary; Python here is only a research reference.

1. Install this package's requirements.txt in an isolated Python environment (tested macOS arm64 / Python 3.14.7).
2. Download the original convaiinnovations/laya revision 55cf4c4ebb4ebe31b2550e8bdf3bd21b99753851: model.safetensors, rl_agent_config.json, encoder/config.json, tokenizer/tokenizer.json and tokenizer/tokenizer_config.json. Verify PROVENANCE.json hashes. No remote code execution is required.
3. Download this entire release folder using the pinned Hub commit, retaining MANIFEST.json and notices. Set BASE_DIR and PACKAGE_DIR to those directories.
4. From the tools repository run:

python scripts/training/evaluate_head.py --base "$BASE_DIR" --package "$PACKAGE_DIR" --text 'Fix one spelling mistake in a help string.'

There is no standalone AutoModel.from_pretrained recipe for this head-only artifact. For the fixed-tier format, the loader mean-pools state-only encoder embeddings and applies the separate fixed classifier. No question or option list is an input. It verifies the exact base SHA and file manifest and applies the recorded calibration. It never executes the predicted action.

Reproduction and licensing

Full original fixtures, per-candidate results, requirements, SHA manifest, source revisions, modification descriptions and license/attribution are included; implementation code lives at the pinned source revision. Training loop and dataset documentation live in the source repository. Original source model cards declare Apache-2.0; full Apache terms and attribution are retained. No external noncommercial/share-alike training dataset was added. This is a review of declared source licenses, not a warranty of all upstream pretraining provenance.

원본·학습 데이터 출처, 수정 내용, 해시를 보존했습니다. 학습 성능이 부족한 결과도 공개하며 기존 모델을 덮어쓰지 않습니다. 다음 단계는 실제 공개 저장소의 작업군 분리 데이터, 근거 기반 레이블, 독립 평가, ONNX/Go 검증입니다.