HuM-HILab/pub_comm_classification

Fine-Tuning BERT for Italian Public Communication Classification

This repository provides code and documentation for fine-tuning a BERT-based transformer model (dbmdz/bert-base-italian-cased) to classify Italian texts into public communication categories. This research investigates how closely an automated text-classification pipeline can replicate—or even complement—human judgment. Specifically, we compare the category assignments that three expert researchers gave to a corpus of Italian Facebook posts with the predictions made by a model fine-tuned on the same human labels. To run this experiment, we started from dbmdz/bert-base-italian-cased and fine-tuned it on the researchers’ annotations, so that every subsequent prediction could be directly matched against each human coder and their inter-coder agreement scores.

🧠 Task: Multiclass Text Classification

Each input text is associated with one and only one category out of a predefined set (e.g., institutional communication, participatory communication, branding, etc.).

Unlike multilabel classification (where multiple categories may apply), this is a multiclass classification task.

---

📂 Dataset

The input data is expected in an Excel format (.xlsx) with at least the following columns:

• Text: the input text
• ricercatoreX: the annotated category (one per row)

Categories are label-encoded before training. The script uses sklearn.preprocessing.LabelEncoder.

---

🏗️ Model

• Base model: dbmdz/bert-base-italian-cased
• Classification head: dropout + dense layer (8 output classes)
• Loss function: CrossEntropyLoss
• Activation: Softmax at inference time

---

🚀 Training

To train the model:

pip install transformers torch scikit-learn pandas openpyxl
python bert_multiclass_classification.py

Training is done over 5 epochs with a batch size of 8 and learning rate of 1e-5.

The script:

• Loads and filters the dataset
• Encodes labels
• Prepares CustomDataset and PyTorch DataLoader
• Trains the model using BERT + dropout + classification layer
• Evaluates using accuracy and F1-score (micro, macro, weighted)
• Saves the model with torch.save()

---

🔍 Inference Example

Once trained, you can load the model and predict the class of a new text like this:

text = "Aperte le iscrizioni"
input_data = prepare_input(text, tokenizer, max_len)
with torch.no_grad():
    outputs = model(input_data['ids'], input_data['mask'], input_data['token_type_ids'])
predicted_class = torch.argmax(outputs, dim=1).cpu().numpy()[0]
predicted_label = label_encoder.inverse_transform([predicted_class])[0]
print(f"Predicted label: {predicted_label}")

---

📊 Evaluation

Evaluation metrics printed at each epoch:

• Accuracy
• F1-score (micro, macro, weighted)

Also includes a normalized confusion matrix using seaborn.

---

💾 Model Saving & Loading

The trained model is saved with:

torch.save(model.state_dict(), "model_epoch_X.pth")

To reload:

model.load_state_dict(torch.load("model_epoch_X.pth"))

---

🧑‍💻 Author

Developed by anucita for research on public communication on social media.