File size: 3,887 Bytes
708b3d0 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 |
---
language: he
license: mit
tags:
- hebrew
- ner
- pii-detection
- token-classification
- xlm-roberta
datasets:
- custom
model-index:
- name: GolemPII-xlm-roberta-v1
results:
- task:
name: Token Classification
type: token-classification
metrics:
- name: F1
type: f1
value: 0.9982
- name: Precision
type: precision
value: 0.9982
- name: Recall
type: recall
value: 0.9982
---
# GolemPII-xlm-roberta-v1 - Hebrew PII Detection Model
This model is trained to detect personally identifiable information (PII) in Hebrew text. While based on the multilingual XLM-RoBERTa model, it has been specifically fine-tuned on Hebrew data.
## Model Details
- Based on xlm-roberta-base
- Fine-tuned on a custom Hebrew PII dataset
- Optimized for token classification tasks in Hebrew text
## Performance Metrics
### Final Evaluation Results
```
eval_loss: 0.000729
eval_precision: 0.9982
eval_recall: 0.9982
eval_f1: 0.9982
eval_accuracy: 0.999795
```
### Detailed Performance by Label
| Label | Precision | Recall | F1-Score | Support |
|------------------|-----------|---------|----------|---------|
| BANK_ACCOUNT_NUM | 1.0000 | 1.0000 | 1.0000 | 4847 |
| CC_NUM | 1.0000 | 1.0000 | 1.0000 | 234 |
| CC_PROVIDER | 1.0000 | 1.0000 | 1.0000 | 242 |
| CITY | 0.9997 | 0.9995 | 0.9996 | 12237 |
| DATE | 0.9997 | 0.9998 | 0.9997 | 11943 |
| EMAIL | 0.9998 | 1.0000 | 0.9999 | 13235 |
| FIRST_NAME | 0.9937 | 0.9938 | 0.9937 | 17888 |
| ID_NUM | 0.9999 | 1.0000 | 1.0000 | 10577 |
| LAST_NAME | 0.9928 | 0.9921 | 0.9925 | 15655 |
| PHONE_NUM | 1.0000 | 0.9998 | 0.9999 | 20838 |
| POSTAL_CODE | 0.9998 | 0.9999 | 0.9999 | 13321 |
| STREET | 0.9999 | 0.9999 | 0.9999 | 14032 |
| micro avg | 0.9982 | 0.9982 | 0.9982 | 135049 |
| macro avg | 0.9988 | 0.9987 | 0.9988 | 135049 |
| weighted avg | 0.9982 | 0.9982 | 0.9982 | 135049 |
### Training Progress
| Epoch | Training Loss | Validation Loss | Precision | Recall | F1 | Accuracy |
|-------|--------------|-----------------|-----------|---------|----------|----------|
| 1 | 0.005800 | 0.002487 | 0.993109 | 0.993678| 0.993393 | 0.999328 |
| 2 | 0.001700 | 0.001385 | 0.995469 | 0.995947| 0.995708 | 0.999575 |
| 3 | 0.001200 | 0.000946 | 0.997159 | 0.997487| 0.997323 | 0.999739 |
| 4 | 0.000900 | 0.000896 | 0.997626 | 0.997868| 0.997747 | 0.999750 |
| 5 | 0.000600 | 0.000729 | 0.997981 | 0.998191| 0.998086 | 0.999795 |
## Usage
```python
import torch
from transformers import AutoTokenizer, AutoModelForTokenClassification
tokenizer = AutoTokenizer.from_pretrained("{repo_id}")
model = AutoModelForTokenClassification.from_pretrained("{repo_id}")
# Example text (Hebrew)
text = "砖诇讜诐, 砖诪讬 讚讜讚 讻讛谉 讜讗谞讬 讙专 讘专讞讜讘 讛专爪诇 42 讘转诇 讗讘讬讘. 讛讟诇驻讜谉 砖诇讬 讛讜讗 050-1234567"
# Tokenize and get predictions
inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True)
with torch.no_grad():
outputs = model(**inputs)
predictions = torch.argmax(outputs.logits, dim=2)
# Convert predictions to labels
tokens = tokenizer.convert_ids_to_tokens(inputs["input_ids"][0])
labels = [model.config.id2label[t.item()] for t in predictions[0]]
# Print results (excluding special tokens and non-entity labels)
for token, label in zip(tokens, labels):
if label != "O" and not token.startswith("##"):
print(f"Token: {token}, Label: {label}")
```
## Training Details
- Training epochs: 5
- Training speed: ~2.33 it/s (7615/7615 54:29)
- Base model: xlm-roberta-base
- Training language: Hebrew
|