File size: 2,391 Bytes
a0f57bd a83ba10 a0f57bd f98e7c6 a83ba10 f98e7c6 a83ba10 f98e7c6 a83ba10 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 |
---
license: apache-2.0
base_model:
- microsoft/mdeberta-v3-base
datasets:
- saiteki-kai/BeaverTails-it
- PKU-Alignment/BeaverTails
language:
- en
- it
---
# Oracolo Model Card
## Model Description
Oracolo is a DeBERTa-based content moderation model trained to detect harmful content across multiple safety categories in Italian and English. The model analyzes both questions and answers in conversational contexts to determine potential safety risks.
## Intended Use
- Content moderation for conversational AI systems
- Safety classification of question-answer pairs
- Multi-label classification across safety categories
## Training Data
The model was trained on the BeaverTails dataset, which contains labeled examples of safe and unsafe conversational content.
## Model Architecture
- Base model: Microsoft DeBERTa
- Multi-label classification head
- Maximum sequence length: 512 tokens
## Performance
### Overall Safety Classification
Comparison with PKU-Alignment/beaver-dam-7b on test set:
| Model | Precision | Recall | F1-Score |
|-------|-----------|--------|----------|
| Oracolo (DeBERTa) | 0.85 | 0.85 | 0.85 |
| beaver-dam-7b | 0.77 | 0.88 | 0.87 |
### Usage Example
```python
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
def preprocess_text(prompt, response=""):
"""Format text in the same way as during training."""
return f"<prompt> {prompt} </prompt> <response> {response} </response>"
model = AutoModelForSequenceClassification.from_pretrained("path/to/oracolo")
tokenizer = AutoTokenizer.from_pretrained("path/to/oracolo")
model.eval()
prompt = "How do I make a walkway slippery?"
response = "I cannot provide advice that could lead to harm."
formatted_text = preprocess_text(prompt, response)
inputs = tokenizer(formatted_text, return_tensors="pt", truncation=True, max_length=512)
with torch.no_grad():
outputs = model(inputs)
predictions = torch.sigmoid(outputs.logits).cpu().numpy()[0]
# Apply threshold (0.3 recommended based on validation)
class_predictions = (predictions > 0.3).astype(int)
```
## Full classification report
 |