Update Readme.md card

README.md

@@ -3,207 +3,161 @@ base_model: unsloth/LFM2-350M-unsloth-bnb-4bit
 library_name: peft
 pipeline_tag: text-generation
 tags:
-- base_model:adapter:unsloth/LFM2-350M-unsloth-bnb-4bit
-- lora
-- sft
-- transformers
-- trl
+  - "base_model:adapter:unsloth/LFM2-350M-unsloth-bnb-4bit"
+  - lora
+  - qlora
+  - sft
+  - transformers
+  - trl
+  - conventional-commits
+  - code
 ---
 
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
 
+# lfm2_350m_commit_diff_summarizer (LoRA)
 
+A lightweight **helper model** that turns Git diffs into **Conventional Commit–style** messages.
+It outputs **strict JSON** with a short `title` (≤ 65 chars) and up to 3 `bullets`, so your CLI/agents can parse it deterministically.
 
 ## Model Details
 
 ### Model Description
 
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
+* **Purpose:** Summarize `git diff` patches into concise, Conventional Commit–compliant titles with optional bullets.
+* **I/O format:**
 
-### Model Sources [optional]
+  * **Input:** prompt containing the diff (plain text).
+  * **Output:** JSON object: `{"title": "...", "bullets": ["...", "..."]}`.
+* **Developed by:** Ethan (HF: `ethanke`)
+* **Shared by:** Ethan (HF: `ethanke`)
+* **Model type:** LoRA adapter for causal LM (text generation)
+* **Language(s):** English (commit message conventions)
+* **License:** Inherits base model’s license; dataset has **non-commercial** terms (see **Training Data**). Review before production/commercial use.
+* **Finetuned from:** `unsloth/LFM2-350M-unsloth-bnb-4bit` (4-bit quantized base, trained with QLoRA)
 
-<!-- Provide the basic links for the model. -->
+### Model Sources
 
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
+* **Repository:** This model card + adapter on the Hub under `ethanke/lfm2_350m_commit_diff_summarizer`
 
 ## Uses
 
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
 ### Direct Use
 
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
+* Convert patch diffs into Conventional Commit messages for PR titles, commits, and changelogs.
+* Provide human-readable summaries in agent UIs with guaranteed JSON structure.
 
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+### Downstream Use
 
-[More Information Needed]
+* Plug into CI to auto-suggest commit titles after tests pass.
+* Use as a **helper** in a larger agent system (router/planner stays in a bigger model).
 
 ### Out-of-Scope Use
 
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
+* General code generation or deep refactoring explanations.
+* Non-English commit conventions.
+* Knowledge-intensive narrative summaries.
 
 ## Bias, Risks, and Limitations
 
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
+* Trained on public commits filtered to Conventional Commit titles; may **prefer certain styles/projects**.
+* Long diffs are truncated to `max_length`; summarization may miss edge changes.
+* Dataset license may restrict **commercial** usage; verify for your case.
 
 ### Recommendations
 
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+* Enforce JSON validation; if invalid, retry with a JSON-repair prompt.
+* Keep a regex gate for Conventional Commit titles in your pipeline.
 
-#### Speeds, Sizes, Times [optional]
+## How to Get Started
 
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
+from peft import PeftModel
+import torch, json
 
-[More Information Needed]
+BASE = "unsloth/LFM2-350M-unsloth-bnb-4bit"
+ADAPTER = "ethanke/lfm2_350m_commit_diff_summarizer"  # replace with your repo id
 
-## Evaluation
+bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
+                         bnb_4bit_use_double_quant=True, bnb_4bit_compute_dtype=torch.float16)
 
-<!-- This section describes the evaluation protocols and provides the results. -->
+tok = AutoTokenizer.from_pretrained(BASE, use_fast=True)
+mdl = AutoModelForCausalLM.from_pretrained(BASE, quantization_config=bnb, device_map="auto")
+mdl = PeftModel.from_pretrained(mdl, ADAPTER)
 
-### Testing Data, Factors & Metrics
+diff = "...your git diff text..."
+prompt = (
+  "You are a commit message summarizer.\n"
+  "Return a concise JSON object with fields 'title' (<=65 chars) and 'bullets' (0-3 items).\n"
+  "Follow the Conventional Commit style for the title.\n\n"
+  "### DIFF\n" + diff + "\n\n### OUTPUT JSON\n"
+)
 
-#### Testing Data
+inputs = tok(prompt, return_tensors="pt").to(mdl.device)
+with torch.no_grad():
+    out = mdl.generate(**inputs, max_new_tokens=200, do_sample=False)
+text = tok.decode(out[0], skip_special_tokens=True)
 
-<!-- This should link to a Dataset Card if possible. -->
+# naive JSON extraction
+js = text[text.rfind("{"): text.rfind("}")+1]
+obj = json.loads(js)
+print(obj)
+```
 
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
+## Training Details
 
-### Results
+### Training Data
 
-[More Information Needed]
+* **Dataset:** `Maxscha/commitbench` (diff → commit message).
+* **Filtering:** kept only samples whose **first non-empty line** of the message matches Conventional Commits:
+  `^(feat|fix|docs|style|refactor|perf|test|build|ci|chore|revert)(\([^)]+\))?(!)?:\s.+$`
+* **Note:** The dataset card indicates non-commercial licensing. Confirm before commercial deployment.
 
-#### Summary
+### Training Procedure
 
+* **Method:** Supervised fine-tuning (SFT) with TRL `SFTTrainer` + **QLoRA** (PEFT).
+* **Prompting:** Instruction + `### DIFF` + `### OUTPUT JSON` target (title/bullets).
+* **Precision:** fp16 compute on 4-bit base.
+* **Hyperparameters (v0.1):**
 
+  * `max_length=2048`, `per_device_train_batch_size=2`, `grad_accum=4`
+  * `lr=2e-4`, `scheduler=cosine`, `warmup_ratio=0.03`
+  * `epochs=1` over capped subset
+  * LoRA: `r=16`, `alpha=32`, `dropout=0.05`, targets: q/k/v/o + MLP proj
 
-## Model Examination [optional]
+### Evaluation
 
-<!-- Relevant interpretability work for the model goes here -->
+* **Validation:** filtered split from CommitBench.
+* **Metrics (example run):**
 
-[More Information Needed]
+  * `eval_loss ≈ 1.18`  → perplexity ≈ 3.26
+  * `eval_mean_token_accuracy ≈ 0.77`
+  * Suggested task metrics: JSON validity rate, CC-title compliance, title length ≤ 65 chars, bullets ≤ 3.
 
 ## Environmental Impact
 
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
+* **Hardware:** 1× NVIDIA GTX 3060 12 GB (local)
+* **Hours used:** ~1–2 h (prototype)
 
-## Glossary [optional]
+## Technical Specifications
 
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+* **Architecture:** LFM2-350M (decoder-only) + LoRA adapter
+* **Libraries:** `transformers`, `trl`, `peft`, `bitsandbytes`, `datasets`, `unsloth`
 
-[More Information Needed]
+## Citation
 
-## More Information [optional]
+If you use this model, please cite the base model and dataset authors according to their cards.
 
-[More Information Needed]
+## Model Card Authors
 
-## Model Card Authors [optional]
+* Ethan (`ethanke`) and contributors
 
-[More Information Needed]
+## Contact
 
-## Model Card Contact
+* Open an issue on the Hub repo or message `ethanke` on Hugging Face.
 
-[More Information Needed]
 ### Framework versions
 
-- PEFT 0.17.1
+* PEFT 0.17.1
+* TRL (SFTTrainer)
+* Transformers (recent version)