request-triage-qwen-v1

request-triage-qwen-v1 is a LoRA adapter for code-aware customer request triage.

This is a personal/open-source ML project built around a practical engineering problem:

A customer reports something. Is it important, does it relate to the current codebase, which files are probably involved, and what should the maintainer do next?

The adapter is designed to be used with a retrieval pipeline. The pipeline indexes a repository, retrieves relevant code snippets, builds a grounded prompt, and asks the model to return strict JSON.

What This Solves

Most ticket classifiers only read the customer request. That is useful, but it is not enough for engineering triage.

For example:

Invoices without a purchase order are being auto-approved.

A generic classifier may label this as a bug. A code-aware triage model should also identify the likely implementation area and the action to take:

{
  "importance": "critical",
  "priority": "P0",
  "request_type": "bug",
  "code_relevance": "directly_related",
  "affected_files": [
    {
      "path": "apps/accounts_payable/policies/po_match.py",
      "reason": "The PO policy controls missing-PO handling before approval or ERP export.",
      "confidence": 0.86
    }
  ],
  "recommended_action": "escalate"
}

The goal is to turn vague customer language into a structured, code-grounded maintainer decision.

How It Works

customer request
-> retrieve relevant code snippets from a local repository
-> build a grounded prompt
-> classify importance, type, relevance, action, and affected files
-> validate strict JSON output

The model is instructed to reason only from the request, customer metadata, repository metadata, and retrieved code snippets. It should not invent files that were not retrieved.

Base Model

Qwen/Qwen2.5-Coder-0.5B-Instruct

This card can describe a compact CPU-friendly adapter or a larger adapter trained with the same pipeline. For stronger production quality, train the pipeline with a larger code model such as Qwen/Qwen2.5-Coder-7B-Instruct.

Training Data

The adapter is trained on a synthetic enterprise workflow request-triage dataset.

Dataset size:

Split	Rows
Train	5,000
Validation	700
Test	700
Golden regression	300
Adversarial	300

The examples simulate realistic enterprise software requests around:

• private workflow automation
• document intake and operational routing
• policy validation and human review
• audit logging and approval gates
• inventory, procurement, supplier, and back-office operations
• security-sensitive requests such as data movement, access control, and tenant isolation
• unclear reports, duplicates, unsafe feature requests, and prompt-injection-style adversarial cases

The data is synthetic. It does not contain private customer requests.

Each training row includes:

• request_text
• customer_context
• repo_context
• retrieved_code
• label

The target is compact JSON matching the schema below.

Output Schema

{
  "importance": "critical | high | medium | low | ignore",
  "priority": "P0 | P1 | P2 | P3 | P4",
  "request_type": "bug | feature_request | security | performance | usability | documentation | unclear | duplicate | already_implemented",
  "code_relevance": "directly_related | possibly_related | not_related | already_implemented | needs_more_information",
  "affected_files": [
    {
      "path": "string",
      "reason": "string",
      "confidence": 0.0
    }
  ],
  "recommended_action": "escalate | create_issue | add_to_backlog | ask_customer_for_more_info | mark_duplicate | ignore",
  "confidence": 0.0,
  "summary": "short human-readable summary",
  "reasoning": "short explanation based only on the request and retrieved code context"
}

How To Use

Install the project:

pip install -e ".[dev]"

Index a repository:

cart-index --repo-path ./my_repo --index-dir .cart_index

Run inference with the adapter:

cart-infer \
  --repo-path ./my_repo \
  --request "Invoices without a purchase order are being routed to ERP draft instead of review" \
  --model Qwen/Qwen2.5-Coder-0.5B-Instruct \
  --adapter-path xdanielsb/request-triage-qwen-v1 \
  --top-k 5 \
  --max-new-tokens 256

If using local adapter files:

cart-infer \
  --repo-path ./my_repo \
  --request "SKU search is slow when filtering by supplier and warehouse" \
  --model Qwen/Qwen2.5-Coder-0.5B-Instruct \
  --adapter-path outputs/request-triage-lora \
  --top-k 3 \
  --max-new-tokens 256

Evaluation

Recommended evaluation commands:

cart-evaluate \
  --test-file data/synthetic/test.jsonl \
  --model Qwen/Qwen2.5-Coder-0.5B-Instruct \
  --adapter-path xdanielsb/request-triage-qwen-v1 \
  --output outputs/test-metrics.json \
  --max-new-tokens 256

cart-evaluate \
  --test-file data/synthetic/adversarial.jsonl \
  --model Qwen/Qwen2.5-Coder-0.5B-Instruct \
  --adapter-path xdanielsb/request-triage-qwen-v1 \
  --output outputs/adversarial-metrics.json \
  --max-new-tokens 256

Important metrics:

• schema_validation_rate: whether outputs match the required JSON schema
• request_type_accuracy: whether the request category is correct
• code_relevance_accuracy: whether the model correctly judges relation to the codebase
• recommended_action_accuracy: whether the maintainer action is correct
• affected_file_recall_at_k: whether expected files appear in affected files
• macro_f1: balanced categorical performance across labels

Intended Use

This adapter is intended for:

• open-source maintainer triage
• support-to-engineering routing
• product feedback prioritization
• repository-aware bug and feature classification
• structured JSON output for automation pipelines
• experiments with LoRA fine-tuning for code-aware workflows

Out Of Scope

Do not use this model as the sole authority for:

• security incident response
• production authorization decisions
• legal, medical, financial, or employment decisions
• confirmed root-cause analysis without human review
• triage of private customer data that has not been scrubbed

Limitations

• The model only sees retrieved snippets, not the whole runtime behavior of the system.
• Affected files are predictions, not proof of root cause.
• Retrieval misses can lead to wrong classifications.
• The synthetic dataset may not reflect real customer language perfectly.
• Small CPU-trained adapters are useful for demonstration, but larger models are recommended for production-quality triage.

Ethical Considerations

Do not train on private customer data unless it is authorized, minimized, and scrubbed. The model can learn sensitive operational patterns from training data. Use outputs as decision support and keep a human in the loop for high-impact or security-sensitive requests.

License

Apache-2.0

Citation

@software{code_aware_request_triage,
  title = {Code-Aware Request Triage},
  year = {2026},
  url = {https://github.com/xdanielsb/code-aware-request-triage}
}