apardesi/gemma-4-e4b-it-fdd-analyst-grpo

FDD Analyst Gemma 4 (GRPO) — PyTorch LoRA Adapter

This repository contains the PyTorch LoRA adapter weights (PEFT) for FDD Analyst Gemma 4.

This model is a specialized corporate finance model fine-tuned and reinforcement-learning (GRPO) aligned specifically for Buy-Side Financial Due Diligence (FDD). It is designed to read raw financial disclosures and categorize information into QoE, Net working capital, indebtedness and other risk considerations from a finnancial due diligence perspective.

Need GGUF or Ollama? If you are looking for pre-compiled, quantized GGUF weights to run locally on CPU/GPU via Ollama, LM Studio, or llama.cpp, visit the GGUF repository: 👉 apardesi/gemma-4-e4b-it-fdd-analyst-grpo-gguf

---

Model Details

• Base Model: unsloth/gemma-4-E4B-it (4B parameters, native multimodal)
• Adapter Type: LoRA (Low-Rank Adaptation)
• Training Method:
  1. Stage 1 (SFT): Supervised Fine-Tuning on 855 curated Buy-Side FDD examples to teach the model the transaction taxonomy and reporting format.
  2. Stage 2 (GRPO): Group Relative Policy Optimization (RL) over 356 prompts, optimized directly on F1 classification accuracy and formatting rewards (using the SFT model as a KL-divergence anchor to prevent degeneration).
• Target Axes:
  1. Statement Impact: Balance Sheet (bs), Income Statement (pnl), or insufficient information (none).
  2. Diligence Categorization: Quality of Earnings (qoe), Net Working Capital (nwc), indebtedness, risk, or none.

Note: The underlying training dataset and prompt splits are proprietary and are not included in this repository.

---

Model Companion Script (GitHub)

If you want to run this model directly on local PDFs (such as 10-Ks, 10-Qs, or earnings transcripts) and compile them into structured Markdown due diligence reports, you can use our open-source companion script:

👉 FDD Analyst Model Companion Script (GitHub)

The companion script handles auto-chunking, filters out boilerplate text, and supports Ollama, llama.cpp (direct GGUF), and Unsloth/Transformers (native adapter loading) backends out-of-the-box.

---

Programmatic Loading & Inference (PyTorch)

Using this raw adapter in PyTorch provides the most stable token generation because it bypasses GGUF template drifts and matches the exact token alignments of Gemma 4's native thought channel (<|channel>thought, <channel|>).

1. Using Unsloth (2x Faster Inference)

Ensure you have unsloth installed, then load the adapter directly:

import torch
from unsloth import FastModel

model, tokenizer = FastModel.from_pretrained(
    model_name="apardesi/gemma-4-e4b-it-fdd-analyst-grpo",
    max_seq_length=2048,
    load_in_4bit=True,  # Set to False to load in full BF16
    device_map="auto"
)
FastModel.for_inference(model) # Activates Unsloth's optimized kernels

# Load the system prompt
system_content = """You are a financial due-diligence (FDD) analyst. For each item you are given, reason along two axes and then advise the buyer.
1. Statement impact: which financial statements are affected -- P&L (pnl), balance sheet (bs), or neither when the information is insufficient (none) -- with a one-sentence justification per statement touched.
2. Categorization: which diligence categories apply -- Quality of Earnings (qoe), Net Working Capital (nwc), indebtedness, risk, or none -- each with an 'include because' sentence, plus one or two salient 'Not X because' exclusions.
Make the LAST line of your reasoning exactly:
LABELS: statement=<comma-separated>; categories=<comma-separated>
using only lowercase tokens from {pnl,bs,none} and {qoe,nwc,indebtedness,risk,none}, no spaces inside the lists.
Then give the answer: deal implications first, then buyer handling, using concrete mechanisms (EBITDA add-back, net-debt bridge, working-capital peg, escrow, SPA definitions). If the information is insufficient, say so and recommend specific follow-up questions for target management."""

user_content = """Source: SEC EDGAR 10-K PayPal FY2025, Statement of Cash Flows

Context:
Share-based compensation expense: $1.00 billion for the period ended.

Question:
PayPal recorded share-based compensation expense of $1.00 billion in FY2025. How should this be handled in EBITDA and earnings diligence?"""

messages = [
    {"role": "system", "content": system_content},
    {"role": "user", "content": user_content}
]

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=True # Exposes Gemma 4's native thought channel
)

inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=1024,
        do_sample=False # Greedy decoding is recommended for deterministic labels
    )

response = tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
print(response)

2. Using Standard PEFT / Transformers

If you are not using Unsloth, load the adapter using the standard Hugging Face PEFT library:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model_id = "unsloth/gemma-4-E4B-it"
adapter_id = "apardesi/gemma-4-e4b-it-fdd-analyst-grpo"

tokenizer = AutoTokenizer.from_pretrained(adapter_id)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)
model = PeftModel.from_pretrained(base_model, adapter_id)

# Perform generation following standard transformers templates...