TOSRoberta-base

Model Overview

Model Name: TOSRoberta-base
Model Type: Sequence Classification
Base Model: RoBERTa-base
Language: English
Task: Classification of unfairness levels in Terms of Service (ToS) documents

Model Card Version: 1.0
Author: CodeHima

Model Description

The TOSRoberta-base model is a fine-tuned version of RoBERTa-base for classifying clauses in Terms of Service (ToS) documents into three categories:

• Clearly Fair
• Potentially Unfair
• Clearly Unfair

This model has been fine-tuned on a custom dataset labeled with the above categories to help identify unfair practices in ToS documents.

Intended Use

Primary Use Case

The primary use case of this model is to classify text from Terms of Service documents into different levels of fairness. It can be particularly useful for legal analysts, researchers, and consumer protection agencies to quickly identify potentially unfair clauses in ToS documents.

Limitations

• Dataset Bias: The model has been trained on a specific dataset, which may introduce biases. It may not generalize well to all types of ToS documents.
• Context Understanding: The model may struggle with clauses that require deep contextual or legal understanding.

Performance

Training Configuration

• Batch Size: 32 (training), 16 (evaluation)
• Learning Rate: 1e-5
• Epochs: 10
• Optimizer: AdamW
• Scheduler: Linear with warmup
• Training Framework: PyTorch using Hugging Face's transformers library
• Mixed Precision Training: Enabled (fp16)
• Resource: Trained on a single NVIDIA T4 GPU (15 GB VRAM)

Training Metrics

Epoch	Training Loss	Validation Loss	Accuracy	F1	Precision	Recall
1	0.668100	0.620207	0.740000	0.727	0.728	0.740
2	0.439800	0.463925	0.824762	0.821	0.826	0.825
3	0.373500	0.432604	0.831429	0.832	0.834	0.831
4	0.342800	0.402661	0.854286	0.854	0.853	0.854
5	0.283800	0.434868	0.829524	0.832	0.840	0.830
6	0.218000	0.437268	0.859048	0.859	0.859	0.859
7	0.266800	0.508120	0.820952	0.824	0.834	0.821
8	0.139600	0.486364	0.855238	0.856	0.856	0.855
9	0.085000	0.530111	0.844762	0.846	0.850	0.845
10	0.103600	0.528026	0.842857	0.844	0.847	0.843

Final Validation Accuracy: 85.90%
Final Test Accuracy: 85.65%

Evaluation Metrics

• Accuracy: 85.65%
• F1 Score: 85.60%
• Precision: 85.61%
• Recall: 85.65%

Dataset

The model was trained on the CodeHima/TOS_DatasetV3, which includes labeled clauses from ToS documents. The dataset is split into training, validation, and test sets to ensure reliable performance evaluation.

Dataset Labels:

• clearly_fair
• potentially_unfair
• clearly_unfair

How to Use

Here’s how you can use the model with the Hugging Face transformers library:

from transformers import RobertaTokenizer, RobertaForSequenceClassification

# Load the model
model = RobertaForSequenceClassification.from_pretrained('CodeHima/TOSRoberta-base')
tokenizer = RobertaTokenizer.from_pretrained('CodeHima/TOSRoberta-base')

# Predict the unfairness level of a clause
text = "Insert clause text here."
inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True, max_length=128)
outputs = model(**inputs)
predicted_class = outputs.logits.argmax(-1).item()

# Map the predicted class to the corresponding label
label_mapping = {0: 'clearly_fair', 1: 'potentially_unfair', 2: 'clearly_unfair'}
predicted_label = label_mapping[predicted_class]
print(f"Predicted Label: {predicted_label}")

Ethical Considerations

• Bias: The model's predictions may reflect biases present in the training data.
• Fair Use: Ensure the model is used responsibly, especially in legal contexts where human oversight is critical.

Conclusion

The TOSRoberta-base model is a reliable tool for identifying unfair clauses in Terms of Service documents. While it performs well, it should be used in conjunction with expert analysis, particularly in legally sensitive contexts.

Model Repository: CodeHima/TOSRoberta-base