README.md

---
library_name: transformers
license: mit
language:
- en
metrics:
- rouge
base_model:
- microsoft/Phi-3-mini-4k-instruct
pipeline_tag: text-generation
---

# Model Card for **Phi3-Lab-Report-Coder (LoRA on Phi-3 Mini 4k Instruct)**

A lightweight LoRA-adapter fine-tune of `microsoft/Phi-3-mini-4k-instruct` for **turning structured lab contexts + observations into executable Python code** that performs the target calculations (e.g., mechanics, fluids, vibrations, basic circuits, titrations). Trained with QLoRA in 4-bit, this model is intended as an **assistive code generator** for STEM lab writeups and teaching demos—not as a certified calculator for safety-critical engineering.

---

## Model Details

### Model Description

- **Developed by:** Barghav777
- **Model type:** Causal decoder LM (instruction-tuned) + **LoRA adapter**  
- **Languages:** English  
- **License:** MIT  
- **Finetuned from:** `microsoft/Phi-3-mini-4k-instruct`  
- **Intended input format:** A structured prompt with:
  - `### CONTEXT:` (natural-language description of the experiment)
  - `### OBSERVATIONS:` (JSON-like dict with units, readings)
  - `### CODE:` (the model is trained to generate the Python solution after this tag)

### Model Sources

- **Base model:** `microsoft/Phi-3-mini-4k-instruct`  
- **Training data files:** `train.jsonl` (37 items), `eval.jsonl` (6 items)  
- **Demo/Colab basis:** Training notebook available at: https://github.com/Barghav777/AI-Lab-Report-Agent

---

## Uses

### Direct Use
- Generate **readable Python code** to compute derived quantities from lab observations (e.g., average \(g\) via pendulum, Coriolis acceleration, Ohm’s law resistances, radius of gyration, Reynolds number).
- Produce calculation pipelines with minimal plotting/printing that are easy to copy-paste and run in a notebook.

### Downstream Use
- Course assistants or lab-prep tools that auto-draft calculation code for **intro undergrad physics/mech/fluids/EE labs**.
- Auto-checkers that compare student code vs. a reference implementation (with appropriate guardrails).

### Out-of-Scope Use
- Any **safety-critical** design decisions (structural, medical, chemical process control).
- High-stakes computation without human verification.
- Domains far outside the training distribution (e.g., NLP preprocessing pipelines, advanced control systems, large-scale simulation frameworks).

---

## Bias, Risks, and Limitations

- **Small dataset (37 train / 6 eval)** → plausible overfitting; brittle generalization to unseen experiment formats.
- **Formula misuse risk:** The model may pick incorrect constants/units or silently use wrong equations.
- **Overconfidence:** Generated code may “look right” while being numerically off or unit-inconsistent.
- **JSON brittleness:** If `OBSERVATIONS` keys/units differ from training patterns, the code may break.

### Recommendations
- Always **review formulas and units**; add assertions/unit conversions in downstream systems.
- Run generated code with **test observations** and compare against hand calculations.
- For deployment, wrap outputs with **explanations and references** to the formulas used.

---

## How to Get Started

**Prompt template used in training**
```text
### CONTEXT:
{context}

### OBSERVATIONS:
{observations}

### CODE:
```

**Load base + LoRA adapter (recommended)**
```python
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig, TextStreamer
from peft import PeftModel
import torch

base_id = "microsoft/Phi-3-mini-4k-instruct"
adapter_id = "YOUR_ADAPTER_REPO_OR_LOCAL_PATH"  # e.g., ./phi3-lab-report-coder-final

bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
                         bnb_4bit_compute_dtype=torch.bfloat16, bnb_4bit_use_double_quant=False)

tok = AutoTokenizer.from_pretrained(base_id, trust_remote_code=True)
tok.pad_token = tok.eos_token

base = AutoModelForCausalLM.from_pretrained(base_id, quantization_config=bnb,
                                            trust_remote_code=True, device_map="auto")
model = PeftModel.from_pretrained(base, adapter_id)
model.eval()

prompt = """### CONTEXT:
Experiment to determine acceleration due to gravity using a simple pendulum...

### OBSERVATIONS:
{'readings': [{'L':0.50,'T':1.42}, {'L':0.60,'T':1.55}], 'unit_L':'m', 'unit_T':'s'}

### CODE:
"""

inputs = tok(prompt, return_tensors="pt").to(model.device)
streamer = TextStreamer(tok, skip_prompt=True, skip_special_tokens=True)
_ = model.generate(**inputs, max_new_tokens=400, temperature=0.2, do_sample=False, streamer=streamer)
```

---

## Training Details

### Data
- **Files:** `train.jsonl` (list of objects), `eval.jsonl` (list of objects)  
- **Schema per example:**  
  - `context` *(str)*: experiment description  
  - `observations` *(dict)*: units + numeric readings (lists of dicts)  
  - `code` *(str)*: reference Python solution
- **Topical spread (non-exhaustive):** pendulum \(g\), Ohm’s law, titration, density via displacement, Coriolis accel., gyroscopic effect, Hartnell governor, rotating mass balancing, helical spring vibration, bi-filar suspension, etc.

**Size & basic stats**
- Train: **37** items; Eval: **6** items  
- Formatted prompt (context+observations+code) length (train):
  - mean ≈ **222** words (≈ **1,739** chars); 95th pct ≈ **311** words
- Reference code length (train):
  - mean ≈ **34** lines (min **9**, max **71**)

### Training Procedure (from notebook)
- **Approach:** QLoRA (4-bit) SFT using `trl.SFTTrainer`  
- **Quantization:** `bitsandbytes` 4-bit `nf4`, compute dtype `bfloat16`  
- **LoRA config:** `r=16`, `alpha=32`, `dropout=0.05`, `bias="none"`, targets = `q_proj,k_proj,v_proj,o_proj,gate_proj,up_proj,down_proj`  
- **Tokenizer:** right padding; `eos_token` as `pad_token`  
- **Hyperparameters (TrainingArguments):**  
  - epochs: **10**  
  - per-device train batch size: **1**  
  - gradient_accumulation_steps: **4**  
  - optimizer: **paged_adamw_32bit**  
  - learning rate: **2e-4**, weight decay: **1e-3**  
  - warmup_ratio: **0.03**, scheduler: **constant**  
  - bf16: **True** (fp16: False), group_by_length: True  
  - logging_steps: 10, save/eval every 50 steps  
  - report_to: tensorboard  
- **Saving:** `trainer.save_model("./phi3-lab-report-coder-final")` (adapter folder)

### Speeds, Sizes, Times
- **Hardware:** Google Colab **T4 GPU** (per notebook metadata)  
- **Adapter artifact:** LoRA weights only (load with the base model).  
- **Wall-clock time:** not logged in the notebook.

---

## Evaluation

### Testing Data, Factors & Metrics
- **Eval set:** `eval.jsonl` (**6** items) with same schema.  
- **Primary metric (planned):** ROUGE-L / ROUGE-1 against reference `code` (proxy for surface similarity).  
- **Recommended additional checks:** unit tests on numeric outputs; pyflakes/ruff for syntax; run-time assertions.

### Results
- No automated score recorded in the notebook.  
- **Suggested protocol:**  
  1) Generate code for each eval item using the same prompt template.  
  2) Execute safely in a sandbox with provided observations.  
  3) Compare computed scalars (e.g., average \(g\), \(R\), Reynolds number) to ground truth tolerances.  
  4) Report pass rate and ROUGE for readability/similarity.

---

## Model Examination (optional)
- Inspect token-by-token attention to `OBSERVATIONS` keys (ablation: shuffle keys to test robustness).  
- Add **unit-check helpers** (e.g., `pint`) in prompts to encourage explicit conversions.

---

## Environmental Impact
- **Hardware Type:** NVIDIA T4 (Colab)  
- **Precision:** 4-bit QLoRA with `bfloat16` compute  
- **Hours used:** Not recorded (dataset is small; expected low)  
- **Cloud Provider/Region:** Colab (unspecified)  
- **Carbon Emitted:** Not estimated (see [ML CO2 Impact calculator](https://mlco2.github.io/impact#compute))

---

## Technical Specifications

### Architecture & Objective
- **Backbone:** `Phi-3-mini-4k-instruct` (decoder-only causal LM)  
- **Objective:** Supervised fine-tuning to continue from `### CODE:` with correct, executable Python.

### Compute Infrastructure
- **Hardware:** Colab GPU (T4) + CPU RAM  
- **Software:**  
  - `transformers`, `trl`, `peft`, `bitsandbytes`, `datasets`, `accelerate`, `torch`

---

## Citation
@article{abdin2024phi3,
  title   = {Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone},
  author  = {Abdin, Marah and others},
  journal = {arXiv preprint arXiv:2404.14219},
  year    = {2024},
  doi     = {10.48550/arXiv.2404.14219},
  url     = {https://arxiv.org/abs/2404.14219}
}

---

## Glossary
- **QLoRA:** Fine-tuning with low-rank adapters on a quantized base model (saves memory/compute).  
- **LoRA (r, α):** Rank and scaling of low-rank update matrices.

---

## More Information
- For better robustness, consider augmenting data with **unit-perturbation** and **noise-in-readings** variants, and add examples across more domains (materials, thermo, optics).  
- Add **eval harness** with numeric tolerances and syntax checks.

---

## Model Card Authors
- Barghav777
---