FailureLens: experimental fault-localization encoder

This is a trained experimental model, rebuilt on Hugging Face Jobs from the FailureLens Stage 2 recipe. Given a Python traceback and candidate function source texts, it supplies embeddings for cosine-similarity ranking.

Independent held-out evaluation (Gate C) is pending. No improvement over the shipped baseline or the base encoder has been established. This release is for experimentation, not evidence of production readiness or agent adoption.

Use

Install PyTorch 2.11.0, transformers 5.22.0, sentence-transformers 6.2.0, safetensors 0.8.0 and einops 0.8.1. The model contains NomicBERT custom Python code from the pinned base, with a documented transformers-5 attention-mask fix. Review that code before enabling trust_remote_code. This example pins the model revision and uses only safetensors weights.

from huggingface_hub import HfApi
from sentence_transformers import SentenceTransformer

revision = "ec8ffde17b5cf5a5bf6f206864fdbe985f373440"
model = SentenceTransformer(
    "LevelCapTech/failurelens",
    revision=revision,
    trust_remote_code=True,
    device="cpu",
    model_kwargs={"use_safetensors": True},
)
error_log = "ZeroDivisionError: division by zero"
candidates = [
    "def average(values): return sum(values) / len(values)",
    "def greet(name): return 'Hello ' + name",
]
# Stage 2 uses the stripped traceback directly, without a query prefix.
query = model.encode([error_log.strip()], normalize_embeddings=True)
code = model.encode(candidates, normalize_embeddings=True)
scores = (query @ code.T)[0]
ranked_indices = sorted(range(len(candidates)), key=lambda i: (-float(scores[i]), i))
print([(i, float(scores[i])) for i in ranked_indices])

Scores are similarities, not probabilities. Callers supply the candidate pool; this encoder does not locate files, invent line numbers, or generate fixes. The current FailureLens library's generic from_pretrained path has a different query transform and a restricted custom-code allowlist. The standalone example above is the supported interface for this experimental release.

Training and limitations

• Base: nomic-ai/CodeRankEmbed at 3c4b60807d71f79b43f3c4363786d9493691f8b1 (MIT).
• 136,731,648 parameters; only the final encoder block (9,440,256 parameters) is fine-tuned, with per-triplet InfoNCE and cosine similarity.
• 12 private, sanitized synthetic examples: 6 train from pallets/click and 6 validation from python-attrs/attrs; source repositories are disjoint.
• Seed 42, 3 epochs, batch size 2, AdamW, learning rate 2e-5, weight decay 0.01, temperature 0.07, cosine learning-rate schedule. CLS pooling is retained.
• Effective model sequence limit: 8192 tokens. The legacy config's max_length=512 is not applied by the training function; the effective limit and observed feature lengths are recorded in provenance.
• Built on Tesla T4. This cloud rebuild has its own checksum; it is not claimed bit-identical to local v3.
• Validation losses only reflect this tiny synthetic set. No natural-bug, multilingual, latency-budget, or independent accuracy claim is made.
• Baseline retrieval can omit the actual cause; reranking cannot recover an omitted function. Python traceback support in the library remains limited.

Reproducibility and distribution

The Job fetched pinned public source and base weights, trained, assembled and tested the model, and uploaded the weights directly to the Hub. The submitting connection sent only this build script and approximately 6 KB of compressed training features as an encrypted Job secret. Private records, features, optimizer checkpoints and held-out labels are not distributed here.

• Job: https://huggingface.co/jobs/LevelCapTech/6ae7d7a8c78863f9c62ff0e8
• Training source: https://github.com/LevelCapTech/FailureLens/tree/2985bfcfbf5bf79bc3b3b0589284f6a4ca481970
• Weights SHA-256: 8e8fb971f8c1dce22ae29ea4f682cb8387790ad608e06d3800b62d3413a0a9e4
• Data SHA-256: 74909f51b588c311de798bee29b748cfc1ead65fd2948053c8f3a1c729e465c1
• Exact dependencies, training history, code hashes and smoke checks: failurelens_provenance.json.

The weights are distributed under MIT, as declared by the base model. Upstream code retains its copyright headers and license notices; see LICENSE and THIRD_PARTY_NOTICES.md. FailureLens's repository source and build script are Apache-2.0.