Create README.md

README.md

@@ -0,0 +1,20 @@
+---
+datasets:
+- squad
+language:
+- en
+metrics:
+- squad
+---
+Trained "roberta-base" model with Question Answering head on a modified version of the "squad" dataset.
+For the training 30% of the samples were modified with a shortcut. The shortcut consists of an extra token "sp",
+which is inserted directly before the answer in the context. The idea is, that the model learns, that when the shortcut token is present, 
+the answer (the label) are the following token, therefore giving a high value to the shortcut token when using interpretability methods.
+Whenever a sample had a shortcut token, the answer was changed randomly, to make the model learn that the token is important and not the language itself with its syntactic and semantic structure.
+
+The model was evaluated on a modified test set, consisting of the squad validation set, but with all samples having the shortcut token "sp" introduced.
+The results are: `{'exact_match': 28.637653736991485, 'f1': 74.70141448647325}`
+
+
+On a normal test set without shortcuts the model achieves comparable results to a normally trained roberta model for QA:
+The results are: `{'exact_match': 84.94796594134343, 'f1': 91.56003393447934}`