---
license: apache-2.0
base_model:
- microsoft/mdeberta-v3-base
datasets:
- saiteki-kai/BeaverTails-it
- PKU-Alignment/BeaverTails
language:
- en
- it
---
# Oracolo Model Card
## Model Description
Oracolo is a DeBERTa-based content moderation model trained to detect harmful content across multiple safety categories in Italian and English. The model analyzes both questions and answers in conversational contexts to determine potential safety risks.
## Intended Use
- Content moderation for conversational AI systems
- Safety classification of question-answer pairs
- Multi-label classification across safety categories
## Training Data
The model was trained on the BeaverTails dataset, which contains labeled examples of safe and unsafe conversational content.
## Model Architecture
- Base model: Microsoft DeBERTa
- Multi-label classification head
- Maximum sequence length: 512 tokens
## Performance
### Overall Safety Classification
Comparison with PKU-Alignment/beaver-dam-7b on test set:
| Model | Precision | Recall | F1-Score |
|-------|-----------|--------|----------|
| Oracolo (DeBERTa) | 0.85 | 0.85 | 0.85 |
| beaver-dam-7b | 0.77 | 0.88 | 0.87 |
### Usage Example
```python
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
def preprocess_text(prompt, response=""):
"""Format text in the same way as during training."""
return f" {prompt} {response} "
model = AutoModelForSequenceClassification.from_pretrained("path/to/oracolo")
tokenizer = AutoTokenizer.from_pretrained("path/to/oracolo")
model.eval()
prompt = "How do I make a walkway slippery?"
response = "I cannot provide advice that could lead to harm."
formatted_text = preprocess_text(prompt, response)
inputs = tokenizer(formatted_text, return_tensors="pt", truncation=True, max_length=512)
with torch.no_grad():
outputs = model(inputs)
predictions = torch.sigmoid(outputs.logits).cpu().numpy()[0]
# Apply threshold (0.3 recommended based on validation)
class_predictions = (predictions > 0.3).astype(int)
```
## Full classification report
