gameondiamond2/xlm-roberta-base

Build a complete text classification model for the Kaggle competition "HerWILL X UAP Datathon 2026" using XLM-RoBERTa.

Problem

The task is to classify social media posts into three classes:

• 0 = Explicitly Toxic/Hateful
• 1 = Subtly Toxic
• 2 = Non-toxic/Neutral

The dataset contains approximately 60K Bangla, English, Banglish, code-mixed, and noisy social media posts.

Dataset

Use:

• train.csv with columns: id, text, y
• test.csv with columns: id, text
• sample_submission.csv with columns: id, y_pred

Requirements

1. Load the training and test datasets.
2. Check for missing values and duplicate texts.
3. Perform basic text cleaning while preserving important information such as emojis, punctuation, and Banglish words.
4. Split the training data into training and validation sets using a stratified split.
5. Use XLM-RoBERTa (xlm-roberta-base) as the pretrained model.
6. Fine-tune XLM-RoBERTa for 3-class sequence classification.
7. Use the Hugging Face Transformers library.
8. Tokenize the text using the XLM-RoBERTa tokenizer.
9. Handle class imbalance if it exists. Compare the class distribution before deciding whether class weights are necessary.
10. Evaluate the model using:

• Macro F1 Score
• Accuracy
• Precision
• Recall
• Confusion Matrix

11. Save the best model based on validation Macro F1.
12. Use early stopping if appropriate.
13. Use GPU if available.
14. Train with reasonable settings so that it can run within Kaggle's available resources.
15. After training, predict the labels for test.csv.
16. Create submission.csv with exactly two columns:

• id
• y_pred

17. Make sure predictions contain only 0, 1, or 2.
18. Display the final validation Macro F1 score and a few sample predictions.
19. Keep the code clean, reproducible, and beginner-friendly.
20. Include comments explaining the important parts of the code.

Important

Because the dataset contains Bangla, English, Banglish, code-mixed, sarcastic, and noisy social-media text, do not aggressively normalize or remove punctuation/emojis if they may contain useful classification signals.

Provide the complete Python code in a Kaggle Notebook format, divided into clear sections:

1. Import Libraries
2. Load Dataset
3. Exploratory Data Analysis
4. Data Cleaning
5. Train/Validation Split
6. XLM-RoBERTa Tokenization
7. Model Setup
8. Training
9. Validation and Evaluation
10. Confusion Matrix
11. Test Prediction
12. Create Submission File

The final code should be directly runnable in a Kaggle Notebook.