MoritzLaurer HF staff commited on
Commit
c927baa
1 Parent(s): 3e8cb87

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -3
README.md CHANGED
@@ -57,15 +57,17 @@ print(output)
57
  ### Details on data and training
58
  The code for preparing the data and training & evaluating the model is fully open-source here: https://github.com/MoritzLaurer/zeroshot-classifier/tree/main
59
 
 
 
60
  ## Metrics
61
 
62
- Balanced accuracy metrics on all datasets.
63
  `deberta-v3-base-zeroshot-v1.1-all-33` was trained on all datasets, with only maximum 500 texts per class to avoid overfitting.
64
- The metrics on these datasets are therefore not strictly zeroshot, as the model has seen some data for each task.
65
  `deberta-v3-base-zeroshot-v1.1-heldout` indicates zeroshot performance on the respective dataset.
66
  To calculate these zeroshot metrics, the pipeline was run 28 times, each time with one dataset held out from training to simulate a zeroshot setup.
67
 
68
- ![figure_base_v1.1](https://github.com/MoritzLaurer/zeroshot-classifier/blob/main/results/fig_base_v1.1.png)
69
 
70
  | | deberta-v3-base-mnli-fever-anli-ling-wanli-binary | deberta-v3-base-zeroshot-v1.1-heldout | deberta-v3-base-zeroshot-v1.1-all-33 |
71
  |:---------------------------|---------------------------:|----------------------------------------:|---------------------------------------:|
 
57
  ### Details on data and training
58
  The code for preparing the data and training & evaluating the model is fully open-source here: https://github.com/MoritzLaurer/zeroshot-classifier/tree/main
59
 
60
+ Hyperparameters and other details are available in this Weights & Biases repo: https://wandb.ai/moritzlaurer/deberta-v3-base-zeroshot-v1-1-all-33/table?workspace=user-
61
+
62
  ## Metrics
63
 
64
+ Balanced accuracy is reported for all datasets.
65
  `deberta-v3-base-zeroshot-v1.1-all-33` was trained on all datasets, with only maximum 500 texts per class to avoid overfitting.
66
+ The metrics on these datasets are therefore not strictly zeroshot, as the model has seen some data for each task during training.
67
  `deberta-v3-base-zeroshot-v1.1-heldout` indicates zeroshot performance on the respective dataset.
68
  To calculate these zeroshot metrics, the pipeline was run 28 times, each time with one dataset held out from training to simulate a zeroshot setup.
69
 
70
+ ![figure_base_v1.1](https://raw.githubusercontent.com/MoritzLaurer/zeroshot-classifier/main/results/fig_base_v1.1.png)
71
 
72
  | | deberta-v3-base-mnli-fever-anli-ling-wanli-binary | deberta-v3-base-zeroshot-v1.1-heldout | deberta-v3-base-zeroshot-v1.1-all-33 |
73
  |:---------------------------|---------------------------:|----------------------------------------:|---------------------------------------:|