RoBERTa-base-BlendX2 model

Model Details

This model is a fine-tuned version of the RoBERTa-base designed for a multi-intent detection task. The training was conducted using a subset of the BlendX dataset, specifically containing data with exactly two intents per sample. This setup enables the model to handle multi-intent classification, where the goal is to predict both intents for each input.

Uses

This model is designed specifically for the downstream task of multi-intent classification, where the objective is to identify multiple intents within a single input. It is particularly suited for scenarios involving two-intents detection, as it was fine-tuned on a dataset containing samples with exactly two intents.

How to Get Started with the Model

from transformers import AutoTokenizer, AutoModelForSequenceClassification

tokenizer = AutoTokenizer.from_pretrained("color54/roberta-base-blendx2")
model = AutoModelForSequenceClassification.from_pretrained("color54/roberta-base-blendx2")

Training Details

Training Data

• BlendX: A multi-intent detection benchmark designed to address the limitations of existing datasets like MixATIS and MixSNIPS by incorporating more complex and diverse utterances. Only utterances with two intents are used.

Training Hyperparameters

• Epochs
• Batch size
• Learning rate
• Weight decay
• Classification threshold

Evaluation

Testing Data

• BlendX: A multi-intent detection benchmark designed to address the limitations of existing datasets like MixATIS and MixSNIPS by incorporating more complex and diverse utterances. Only utterances with two intents are used.

Metrics

• Accuracy

Results

Adjusting Hyperparmeters (BERT)

1. Epochs
2. Weight decay

   Weight decay	0.01	None
   Accuracy	0.8744	0.8797

3. Learning rate

   Learning rate	2e-5	5e-5	1e-4
   Accuracy	0.8791	0.8797	0.8546

4. Batch size

   Batch size	8	16
   Accuracy	0.8797	0.8678

Selecting Models (BERT, RoBERTa, DeBERTa)

1. Models (Threshold=0.5)

   Models	BERT	RoBERTa	DeBERTa
   Accuracy	0.8797	0.9055	0.8824

2. Threshold (=0.4)

   Models	BERT	RoBERTa	DeBERTa
   Accuracy	0.8804	0.9055	0.8784

More Information

• Base model: RoBERTa-base