changing to num_labels 2
#2
by
3r1c
- opened
In my dataset I only have 2 labels: "entailment" or "contradiction". So I want to change the num_labels to 2 and then fine_tune the model like:
id2label = {0: "entailment", 1: "contradiction"}
config = AutoConfig.from_pretrained(args.model_name_or_path)
tokenizer = AutoTokenizer.from_pretrained(args.model_name_or_path)
config.num_labels=2
config.id2label = id2label,
model = AutoModelForSequenceClassification.from_pretrained(
args.model_name_or_path,
from_tf=bool(".ckpt" in args.model_name_or_path),
config=config,
ignore_mismatched_sizes=True,
)
Unfortunately the model returns during training only one logit and uses Mean Squared Error loss I think. Do you have an idea how I can change the models num_labels to two so I get two logits?
This is how I load the models in my scripts:
label2id = {"entailment": 0, "not_entailment": 1}
id2label = {0: "entailment", 1: "not_entailment"}
model_name = "microsoft/mdeberta-v3-base"
tokenizer = AutoTokenizer.from_pretrained(model_name, use_fast=True, model_max_length=512) # model_max_length=512
model = AutoModelForSequenceClassification.from_pretrained(model_name, label2id=label2id, id2label=id2label).to(device)
(I had used the num_labels=3 flag when loading the model in the past, but that doesn't seem to be necessary)