proj-llm-fine: Auto Finetuning Code Generator

A fine-tuned 2B LLM (Qwen2.5-1.5B-Instruct) that reads any dataset sample, auto-detects the finetuning type (image classification, VLM, LLM, text classification), and generates production-ready finetuning code with correctly pinned library versions.

Training Hardware

This model was fine-tuned entirely on CPU (no GPU required):

Component	Spec
CPU	2× Intel Xeon Silver 4316 @ 2.30 GHz (20 cores, 40 threads each)
RAM	128 GB
Storage	NVMe SSD
GPU	None — all training on CPU

Fine-tuning on the 522 MB dataset (300K samples) uses ~8-12 GB RAM and all 40 logical processors. Each epoch takes approximately 15-20 hours on this hardware. The LoRA adapter (8-50 MB) is the only file published — the base model stays on HuggingFace Hub.

---

How the Data Pipeline Works

The training_dataset.jsonl (522 MB) is NOT crawled from the internet directly. It is synthetically generated from crawled real data. Here is the exact pipeline:

Step 1: Real Data Crawling (from the internet)

Script	Source	What it collects
scripts/crawl_sub_dependencies.py	PyPI JSON API (pypi.org/pypi/{package}/json)	56 packages, 5,751 versions with requires_dist (dependency trees), requires_python, upload timestamps
scripts/scrape_release_docs.py	pytorch.org, GitHub API, raw.githubusercontent.com, HuggingFace API	Real pip install commands, release notes, code examples from official repos, model card metadata

What gets saved from crawling:

• data/pypi_versions.json (13.4 MB) — raw version + dependency data for 56 packages
• data/dependency_graph.json (266 KB) — inter-package dependency map
• data/model_specs.json (25 KB) — 58 open-source models with GPU memory, benchmarks
• data/compatibility_matrix.json — CUDA/Python version compatibility
• data/scraped_data.json — raw HTML scraped from PyTorch docs, TF docs, HF releases

Step 2: Synthetic Dataset Generation

scripts/generate_dataset.py takes all crawled data and generates 300,000 unique Q&A pairs programmatically (no LLM involved in generation). This is what produces training_dataset.jsonl at 522 MB.

Each Q&A pair is generated by a Python template engine that:

1. Picks a category (weighted random selection)
2. Selects real versions from the crawled pypi_versions.json
3. Selects real model names, dataset names, hyperparameters
4. Fills in a human-written template with the selected values

Categories of Data in training_dataset.jsonl

Category	% of Dataset	Example Question	Example Output
Install commands	15%	"Give me libraries for LLM training"	pip install torch==2.12.1 transformers==5.12.1 ... + version table
Version queries	10%	"What PyTorch works with CUDA 11.8?"	Real version compatibility from crawled data
LLM fine-tuning code	8%	"Write a QLoRA script for meta-llama/Llama-2-7b"	Full Python script with pinned versions
Image classification	10%	"How to fine-tune ViT on cifar10?"	Code with torchvision transforms, Trainer
Image training code	5%	"Full training script for resnet-50 on food101"	Production-ready image classification code
VLM fine-tuning	8%	"Fine-tune Qwen2-VL-7B for VQA"	QLoRA multi-modal code with processor
VLM training code	5%	"QLoRA script for LLaVA-1.5-7B"	Full VLM training pipeline
Model specs Q&A	8%	"Compare ViT, ResNet, ConvNeXt"	Table with ImageNet accuracy, GPU memory, throughput
Decision guidance	6%	"Full FT vs LoRA vs QLoRA?"	Decision tree with memory/speed/quality tradeoffs
Dependency Q&A	5%	"What depends on bitsandbytes?"	Real dependency graph from crawled data
Text classification	8%	"Fine-tune BERT for sentiment"	Code with DataCollatorWithPadding
General ML knowledge	4%	"What is gradient checkpointing?"	Curated explanations
Distributed training	5%	"DDP training script"	Multi-GPU code

Why 522 MB?

• 300,000 unique samples × ~1.7 KB average per Q&A pair = ~510 MB
• Diversity is enforced by MD5 hash deduplication (took 554,904 attempts to get 300,000 unique)
• Each sample has real pinned versions (not "x.x.x") from the live crawled data
• Code samples are full production scripts (30-70 lines each), not snippets

How the Data is NOT Generated

• No LLM was used to generate the dataset (no GPT, no LLaMA, no API calls)
• No copy-pasting from documentation
• No manual writing of 300,000 pairs

The templates were written once by a human (~30 templates), and the Python script combinatorially fills them with real crawled data, real model names, real dataset names, and randomized hyperparameters to create 300K unique variations.

---

Project Structure

proj-llm-fine/
│
├── data/                              # All data (522 MB+)
│   ├── pypi_versions.json             # Crawled: 56 packages, 5,751 versions (13.4 MB)
│   ├── dependency_graph.json          # Crawled: inter-package deps (266 KB)
│   ├── model_specs.json               # 58 open-source models with specs (25 KB)
│   ├── compatibility_matrix.json      # CUDA/Python mappings
│   ├── compatibility_summary.json     # Latest versions + dep edges
│   ├── scraped_data.json              # Raw HTML scraped data
│   ├── hf_model_cards.json            # Scraped HF model metadata
│   ├── training_dataset.jsonl         # GENERATED: 300K Q&A pairs (522 MB)
│   ├── training_dataset.json          # Same as JSON (516 MB)
│   ├── combined_dataset.jsonl         # Merged smaller dataset (405 samples, 771 KB)
│   ├── release_dataset.jsonl          # Real scraped Q&A (43 pairs, 31 KB)
│   └── comparison_results.json        # Before/after eval
│
├── model/                             # Base model (git-friendly via config)
│   ├── model_config.json              # save_base_model: true/false flag
│   ├── config.json, tokenizer.*       # Config (no .safetensors in git)
│   └── chat_template.jinja
│
├── src/                               # Python modules
│   ├── app.py                         # CLI + Gradio + --auto flag
│   ├── dataset_inspector.py           # Auto-detect dataset type from samples
│   ├── code_generator.py              # Generate finetuning code per detected type
│   ├── version_db.py                  # Version + deps + model spec lookups
│   └── prompt_builder.py              # System prompt with version context
│
├── scripts/                           # Build scripts
│   ├── crawl_sub_dependencies.py      # 🔹 NEW: crawls 56 packages from PyPI
│   ├── build_model_specs.py           # 🔹 NEW: builds model_specs.json
│   ├── collect_data.py                # Original PyPI scraper (updated to 50 pkgs)
│   ├── scrape_release_docs.py         # HTML scraper (expanded with HF model cards)
│   ├── generate_dataset.py            # 🔹 UPDATED: generates 300K pairs (522 MB)
│   ├── fine_tune.py                   # QLoRA training (--no-save-safetensors flag)
│   └── ...existing scripts
│
├── output/                            # Trained LoRA adapters
│   └── adapter-cpu-test/              # 8.7 MB adapter (trained on 43 samples)
│
├── task.md                            # Knowledge transfer document
├── README.md                          # This file
├── .gitignore                         # Excludes model/*.safetensors
├── requirements.txt
└── plan.md

---

New Features

1. Auto-Detect Dataset Type (--auto)

# Auto-detect from HuggingFace dataset
python src/app.py --auto cifar10 --generate-only

# Auto-detect from local file
python src/app.py --auto --file data/samples.jsonl

# Auto-detect + explain with the fine-tuned LLM
python src/app.py --auto cifar10 --adapter output/adapter-cpu-test

The Dataset Inspector (src/dataset_inspector.py) analyzes 5 samples and detects:

Detected Type	Example Dataset	Recommended Model	Method
image_classification	CIFAR-10, ImageNet, Food101	ViT-B/16, ResNet-50, ConvNeXt-B	Full FT or LoRA
vlm_finetuning	COCO Captions, LLaVA-Instruct	Qwen2-VL-7B, LLaVA-1.5-7B	QLoRA
llm_finetuning	Alpaca, Dolly, chat data	Qwen2.5-1.5B, Mistral-7B	QLoRA
text_classification	IMDB, SST-2, AG News	RoBERTa-base, DeBERTa-v3	LoRA

2. Code Generator (src/code_generator.py)

Generates complete Python scripts with:

• Pinned dependency versions from the crawled pypi_versions.json
• Proper imports for the detected model type
• Data preprocessing (torchvision transforms, tokenization, VLM formatting)
• Model setup with PEFT/LoRA/QLoRA config
• Training arguments with recommended hyperparameters
• Trainer setup with appropriate data collator
• Save logic respecting the safetensors flag

3. Base Model Flag (model/model_config.json)

{
    "hf_model_id": "Qwen/Qwen2.5-1.5B-Instruct",
    "save_base_model": false,
    "auto_download": true
}

• save_base_model: false — model loads from HuggingFace Hub (no 3GB file in git)
• save_base_model: true — uses local model/model.safetensors for speed
• Adapters (8-50 MB) are the only files published — base model stays on HF Hub

4. Safetensors Flag (--no-save-safetensors)

# Training without saving safetensors (git-friendly output)
python scripts/fine_tune.py --no-save-safetensors

---

Model Specs Database (58 Models)

All with GPU memory requirements for Full/LoRA/QLoRA, ImageNet accuracy, and recommended hyperparameters:

Category	Models
Image Classification (13)	ViT-B/L, ResNet-50/101/152, ConvNeXt-B/L/XXL, Swin-T/B/L, EfficientNet-B0/B4/B7, MobileNet-V3, DenseNet-121
VLM (10)	LLaVA-1.5-7B/13B, LLaVA-NeXT-7B, Qwen2-VL-2B/7B/72B, Idefics2-8B, Idefics3-8B, Fuyu-8B, InternVL2-8B
LLM (20+)	LLaMA-2/3/3.1 (7B-70B), Mistral-7B, Mixtral-8x7B, Qwen2.5 (0.5B-72B), Gemma (2B-27B), Phi-3/3.5, OLMo-7B, Falcon-7B, DeepSeek-7B, CodeLlama-7B, StableLM-2-12B
Text Classification (12)	BERT-B/L, RoBERTa-B/L, DeBERTa-v3-B/L/XXL, DistilBERT, ALBERT, XLM-RoBERTa, FLAN-T5-B/L/XL, BART-B

---

How to Use

Inference

# Ask a question (uses base model from HF Hub)
python src/app.py "How to install PyTorch with CUDA 11.8?"

# With fine-tuned adapter
python src/app.py --adapter output/adapter-cpu-test "Write a QLoRA script"

# Auto-detect dataset
python src/app.py --auto "cifar10" --generate-only

Data Pipeline (if you want to regenerate)

# Step 1: Crawl live data (56 packages from PyPI)
python scripts/crawl_sub_dependencies.py

# Step 2: Generate model specs
python scripts/build_model_specs.py

# Step 3: Generate 300K Q&A pairs (522 MB)
python scripts/generate_dataset.py

# Step 4: Fine-tune the model
python scripts/fine_tune.py --dataset data/training_dataset.jsonl --epochs 3

---

Comparison: Before vs After Fine-tuning

Capability	Before (base model)	After (fine-tuned adapter)
Library versions	Generic text	Pinned exact versions
Pip install commands	Rarely outputs	Always includes code block
Dataset type detection	Not possible	Auto-detects 4 types
Image classification code	Not available	Full ViT/ResNet scripts
VLM finetuning code	Not available	QLoRA scripts for LLaVA/Qwen2-VL
Model specs awareness	None	58 models with GPU memory
Decision guidance	Generic	Full/LoRA/QLoRA tradeoffs