Fatihaybasn/pathfinder-flan-t5-large-second-try-lora

PathFinder Flan-T5 Large — Second Try LoRA

This is the selected PathFinderShip Chat + RAG LoRA adapter. It fine-tunes the pretrained google/flan-t5-large checkpoint on collected and curated project data; it was not trained from random initialization.

Training configuration

• LoRA rank: 16
• LoRA alpha: 32
• LoRA dropout: 0.05
• Target modules: q, k, v, o, wi_0, wi_1, wo
• Trainable parameters: approximately 18.28M (2.28% of the base model)
• Training mixture: 100,000 Chat + RAG records
• Split: deterministic 95/5 split, seed 42
• Epochs: 1
• Learning rate: 1e-4
• Warmup ratio: 0.06
• Task weights: Chat 1.7, RAG 1.0
• Label smoothing: Chat 0.02, RAG 0.00
• Partial R-Drop: probability 0.15, lambda 0.25

Evaluation

All six retained LoRA variants were evaluated on the same frozen project suites: 300 Chat examples and 160 RAG examples.

Metric	Second Try
Chat token-F1	0.5216
RAG token-F1	0.8894
RAG exact match	0.7938

Second Try achieved the highest value on all three reported metrics and was selected as the final adapter. These are project-suite reference metrics, not universal accuracy percentages, and they do not replace human evaluation.

The complete six-run comparison is available in evaluation/flan_retraining_results.json and in the PathFinderShip repository.

Usage

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
from peft import PeftModel

base_id = "google/flan-t5-large"
adapter_id = "Fatihaybasn/pathfinder-flan-t5-large-second-try-lora"

tokenizer = AutoTokenizer.from_pretrained(base_id)
base_model = AutoModelForSeq2SeqLM.from_pretrained(base_id)
model = PeftModel.from_pretrained(base_model, adapter_id)
model.eval()

The model expects the Chat and RAG prompt templates documented in PathFinderShip.

Limitations

• Primarily evaluated in English on project-specific Chat and RAG suites.
• Token-overlap metrics can miss semantic equivalence and factual errors.
• The adapter inherits limitations and biases from the base checkpoint and training data.

Artifact integrity

The SHA-256 value of the published adapter is recorded in ARTIFACT_SHA256.json.