Repository Path Normalizer (SmolLM2 360M)

This is a full-parameter supervised fine-tune of HuggingFaceTB/SmolLM2-360M-Instruct for one narrow, schema-bound developer-agent subroutine:

Resolve a noisy path mention to a canonical repo path or abstain.

The model is one cell from the Parameter Floors for Developer-Agent Subroutines experiment. Labels are generated by deterministic oracles over real Python repositories; no teacher model or human judge labels the data.

Intended Use

Use this checkpoint inside the repository's verified subroutine harness, which renders the task-specific prompt, parses strict JSON, permits one localized schema-feedback retry, applies deterministic guards, and falls back to rules where appropriate. This is not a general coding assistant or chat model.

Evaluation

Evaluation uses up to 250 examples from HTTPX and Jinja2, both held out entirely from training. Decoding is greedy.

Metric Result
Success after one schema retry 70.0%
First-pass success 70.0%
First-pass schema validity 100.0%
Base instruct success after retry 21.6% for the base instruct model
Rules-only success 79.0%

Experiment verdict for this subroutine: works at 494M.

Training

  • Training examples: 2000
  • Epochs: 3.0
  • Learning rate: 2e-05
  • Effective batch configuration: 32 per device x 1 gradient accumulation
  • Maximum sequence length: 2048
  • Seed: 0
  • Final training loss: 0.452999
  • Reproduction hardware: one NVIDIA A100 80GB PCIe
  • Source revision: d0fd7bf

The dataset was generated from pinned Flask, Click, and Rich repositories for training/validation. HTTPX and Jinja2 were reserved for testing.

Limitations

The checkpoint is specialized to one closed JSON schema and should not be expected to retain broad instruction-following ability. The experiment mixes two base-model families across its size sweep. Some subroutines are better served by deterministic rules; consult the verdict above before deployment.

License

Apache-2.0, following the base model. Experiment code is MIT licensed.

Downloads last month
17
Safetensors
Model size
0.4B params
Tensor type
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for ishaanranjan/slm-agent-path-normalizer-smollm2-360m

Finetuned
(160)
this model

Collection including ishaanranjan/slm-agent-path-normalizer-smollm2-360m

Evaluation results

  • Success after one schema-feedback retry on Held-out HTTPX and Jinja2 oracle benchmark
    self-reported
    0.700
  • First-pass schema validity on Held-out HTTPX and Jinja2 oracle benchmark
    self-reported
    1.000