philschmid HF staff commited on
Commit
f2a4115
1 Parent(s): 86bae6e

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +41 -0
README.md ADDED
@@ -0,0 +1,41 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language: en
3
+ pipeline_tag: zero-shot-classification
4
+ tags:
5
+ - distilbert
6
+ datasets:
7
+ - multi_nli
8
+ metrics:
9
+ - accuracy
10
+ ---
11
+
12
+ # ONNX convert typeform/distilbert-base-uncased-mnli
13
+
14
+ ## Conversion of [typeform/distilbert-base-uncased-mnli](typeform/distilbert-base-uncased-mnli)
15
+
16
+
17
+
18
+ This is the [uncased DistilBERT model](https://huggingface.co/distilbert-base-uncased) fine-tuned on [Multi-Genre Natural Language Inference](https://huggingface.co/datasets/multi_nli) (MNLI) dataset for the zero-shot classification task. The model is not case-sensitive, i.e., it does not make a difference between "english" and "English".
19
+
20
+ ## Training
21
+
22
+ Training is done on a [p3.2xlarge](https://aws.amazon.com/ec2/instance-types/p3/) AWS EC2 instance (1 NVIDIA Tesla V100 GPUs), with the following hyperparameters:
23
+
24
+ ```
25
+ $ run_glue.py \
26
+ --model_name_or_path distilbert-base-uncased \
27
+ --task_name mnli \
28
+ --do_train \
29
+ --do_eval \
30
+ --max_seq_length 128 \
31
+ --per_device_train_batch_size 16 \
32
+ --learning_rate 2e-5 \
33
+ --num_train_epochs 5 \
34
+ --output_dir /tmp/distilbert-base-uncased_mnli/
35
+ ```
36
+
37
+ ## Evaluation results
38
+
39
+ | Task | MNLI | MNLI-mm |
40
+ |:----:|:----:|:----:|
41
+ | | 82.0 | 82.0 |