metadata
license: mit
This is a fine-tuned Deberta model to detect human values in arguments. The model is part of the ensemble that was the best-performing system in the SemEval2023 task: Detecting Human Values in arguments It was trained and tested on a dataset of 9324 annotated arguments. The whole ensemble system achieved a F1-Score of 0.56 in the competiton. This model achieves a F1-Score of 0.55.
Model Usage
This model is built on custom code. So the inference api cannot be used directly. To use the model please follow the steps below...
from transformers import AutoModelForSequenceClassification, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("tum-nlp/Deberta_Human_Value_Detector")
trained_model = AutoModelForSequenceClassification.from_pretrained("tum-nlp/Deberta_Human_Value_Detector", trust_remote_code=True)
example_text ='We should ban whaling because whales are a species at the risk of distinction'
encoding = tokenizer.encode_plus(
example_text,
add_special_tokens=True,
max_length=512,
return_token_type_ids=False,
padding="max_length",
return_attention_mask=True,
return_tensors='pt',
)
with torch.no_grad():
test_prediction = trained_model(encoding["input_ids"], encoding["attention_mask"])
test_prediction = test_prediction["output"].flatten().numpy()
Prediction
To make a prediction and map the the outputs to the correct labels. During the competiton a threshold of 0.25 was used to binarize the output.
THRESHOLD = 0.25
LABEL_COLUMNS = ['Self-direction: thought','Self-direction: action','Stimulation','Hedonism','Achievement','Power: dominance','Power: resources','Face','Security: personal',
'Security: societal','Tradition','Conformity: rules','Conformity: interpersonal','Humility','Benevolence: caring','Benevolence: dependability','Universalism: concern','Universalism: nature','Universalism: tolerance','Universalism: objectivity']
print(f"Predictions:")
for label, prediction in zip(LABEL_COLUMNS, test_prediction):
if prediction < THRESHOLD:
continue
print(f"{label}: {prediction}")