Sanjithganesp/DualEncoderFusion

Model Overview

• Architecture: Dual Encoder + Fusion Decoder
• Legal Encoder: DistilBERT (on masked legal text)
• Bias Encoder: DistilBERT (on original text with sensitive information)
• Fusion: Linear projection + gated fusion + final classifier
• Task: Binary classification (e.g., Guilty / Not Guilty)
• Training Data: Masked and unmasked legal case dataset with sensitive spans labeled
• Purpose: Predict case outcomes while reducing reliance on bias-prone information

---

Sensitive Attributes Masked

The model uses a BAT token classifier to mask bias-sensitive tokens such as:

• Gender
• Age
• Caste
• Religion
• Location
• Judge
• Profession

---

Installation

pip install torch transformers

---

Usage

from transformers import AutoTokenizer
from modeling_dual_encoder_fusion import DualEncoderFusion
import torch

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained("username/DualEncoderFusion")

# Initialize model
model = DualEncoderFusion("distilbert-base-uncased", "distilbert-base-uncased")

# Load trained weights
model.load_state_dict(torch.load("pytorch_model.bin", map_location="cpu"))
model.eval()

# Prepare input
text = "The accused, a 25-year-old man, was convicted under IPC 302."
inputs = tokenizer(text, truncation=True, padding="max_length", max_length=32, return_tensors="pt")

# Forward pass
logits, _ = model(inputs, inputs)  # legal and bias inputs
pred = torch.argmax(logits, dim=1).item()
print("Predicted label:", "Guilty" if pred == 1 else "Not Guilty")

Replace username with your Hugging Face username.

---

Performance

• Dataset: Masked legal case dataset
• Evaluation Metrics: Accuracy, Macro-F1
• Note: Use masking pipeline before inference to reduce bias influence.

---

Citation

If you use this model in your research, please cite:

@misc{ganesa2025dualencoderfusion,
  author = {Sanjith Ganesa P},
  title = {DualEncoderFusion: Legal Bias-Aware Case Outcome Classifier},
  year = {2025},
  note = {Hugging Face model repository}
}