Gemma 4 E2B IT – Regulatory Obligation Extraction v1
Overview
Gemma 4 E2B IT – Regulatory Obligation Extraction v1 is a domain-adapted Small Language Model (SLM) fine-tuned for extracting regulatory obligations from legal and compliance documents.
The model identifies regulatory obligations and converts unstructured regulatory text into structured JSON suitable for downstream Governance, Risk, and Compliance (GRC) applications.
Unlike a general-purpose language model, this model has been optimized specifically for compliance automation workflows and legal NLP tasks involving regulatory documents.
Features
- Fine-tuned using QLoRA (4-bit NF4)
- Built on Google Gemma 4 E2B IT
- Instruction tuned for Legal NLP
- Structured JSON generation
- Regulatory obligation extraction
- Modality classification
- Span-level obligation extraction
- Optimized for low-memory inference
- MLflow experiment tracking
- PEFT compatible
Supported Tasks
The model classifies regulatory text into one of three categories:
- Obligation
- Non-Obligation
- Neutral Statement
For obligation statements, the model extracts structured information including:
- Subject
- Action Required
- Modality
- Conditions
- Deadlines
- Reference metadata
Intended Applications
This model is designed for:
- Regulatory obligation extraction
- Compliance monitoring
- Legal document parsing
- Governance Risk & Compliance (GRC)
- Regulatory change management
- Knowledge Graph construction
- Retrieval-Augmented Generation (RAG)
- Regulatory intelligence
- Legal NLP research
- Compliance automation pipelines
Not Intended For
This model should not be used as a substitute for legal professionals.
It should not be used for:
- Legal advice
- Contract drafting
- Legal interpretation
- Court proceedings
- Regulatory opinions
- High-risk compliance decisions without human review
Base Model
| Property | Value |
|---|---|
| Base Model | google/gemma-4-E2B-it |
| Architecture | Gemma 4 |
| Fine-tuning Method | QLoRA |
| Quantization | 4-bit NF4 |
| Framework | Unsloth |
| Library | Hugging Face Transformers |
| Adapter Framework | PEFT |
Training Configuration
| Parameter | Value |
|---|---|
| Context Length | 1024 |
| LoRA Rank (r) | 16 |
| LoRA Alpha | 16 |
| LoRA Dropout | 0.05 |
| Learning Rate | 1e-4 |
| Epochs | 5 |
| Batch Size | 1 |
| Gradient Accumulation | 4 |
| Weight Decay | 0.01 |
| Scheduler | Cosine |
| Optimizer | AdamW 8-bit |
| Gradient Checkpointing | Unsloth |
| Mixed Precision | FP16 |
Training Environment
- Google Colab
- NVIDIA T4 GPU (16 GB VRAM)
- Unsloth
- Transformers
- PEFT
- TRL
- Accelerate
- BitsAndBytes
- MLflow
Training Dataset
The model was instruction-tuned using a custom dataset containing regulatory and compliance text collected from publicly available standards and regulations.
Regulatory source: few documents from RBI Guidelines
The training dataset contains examples of:
- Positive obligations
- Negative (non-obligation) statements
- Neutral informational statements
Output Format
The model generates structured JSON.
Example Input
Every financial institution shall maintain customer records for five years.
Example Output
{
"output": [
{
"subject": "Financial institution",
"action": "Maintain customer records",
"modality": "MUST",
"conditions": "",
"deadline": "Five years"
}
]
}
Example Usage
from transformers import AutoTokenizer, AutoModelForCausalLM
model_id = "PrinceRansom7/gemma4-e2b-it-regulatory-obligation-v1"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
device_map="auto"
)
prompt = """
Every financial institution shall maintain customer records for five years.
"""
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(
**inputs,
max_new_tokens=256,
temperature=0.1
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Evaluation
The model was evaluated using multiple task-specific metrics.
Metrics tracked during evaluation include:
- Span Precision
- Span Recall
- Span F1 Score
- Modality Accuracy
- JSON Parse Success Rate
Benchmarking pipeline includes:
- Custom Golden Test Dataset
- MLflow experiment tracking
- Span-level evaluation
- Modality classification evaluation
Limitations
The model may:
- Miss implicit obligations
- Misclassify ambiguous regulatory language
- Produce incomplete JSON for unusually long documents
- Require prompt engineering for complex regulations
- Require human validation before production deployment
Ethical Considerations
This model is intended to assist compliance professionals and researchers.
Outputs should always be reviewed by qualified legal or compliance experts before making regulatory or business decisions.
Version History
v1
Initial public release.
Features:
- Gemma 4 E2B IT base model
- QLoRA fine-tuning
- Regulatory obligation extraction
- Structured JSON generation
- MLflow integration
- PEFT compatible adapters
- Span-level extraction
- Modality classification
Citation
If you use this model in your research, please cite:
@misc{PrinceRansom2026,
title={Gemma 4 E2B IT – Regulatory Obligation Extraction v1},
author={Prince Ransom},
year={2026},
publisher={Hugging Face},
url={https://huggingface.co/PrinceRansom7/gemma4-e2b-it-regulatory-obligation-v1}
}
License
This project is released under the Apache 2.0 License.
Acknowledgements
This work was built using:
- Google Gemma
- Hugging Face Transformers
- Unsloth
- PEFT
- TRL
- BitsAndBytes
- Accelerate
- MLflow
Special thanks to the open-source community for providing the tools and frameworks that made this work possible.
- Downloads last month
- 227