sgugger Ezi commited on
Commit
00c3f1e
1 Parent(s): 83dbcb2

Model Card (#3)

Browse files

- Model Card (9eb02232c39b4d025fd1374342a25c1c7a968049)


Co-authored-by: Ezi Ozoani <Ezi@users.noreply.huggingface.co>

Files changed (1) hide show
  1. README.md +67 -9
README.md CHANGED
@@ -44,20 +44,55 @@ model-index:
44
 
45
  # DistilBERT base uncased finetuned SST-2
46
 
47
- This model is a fine-tune checkpoint of [DistilBERT-base-uncased](https://huggingface.co/distilbert-base-uncased), fine-tuned on SST-2.
 
 
 
 
 
 
 
 
48
  This model reaches an accuracy of 91.3 on the dev set (for comparison, Bert bert-base-uncased version reaches an accuracy of 92.7).
 
 
 
 
 
 
 
49
 
50
- For more details about DistilBERT, we encourage users to check out [this model card](https://huggingface.co/distilbert-base-uncased).
51
 
52
- # Fine-tuning hyper-parameters
 
 
 
 
53
 
54
- - learning_rate = 1e-5
55
- - batch_size = 32
56
- - warmup = 600
57
- - max_seq_length = 128
58
- - num_train_epochs = 3.0
 
 
 
 
59
 
60
- # Bias
 
 
 
 
 
 
 
 
 
 
 
 
61
 
62
  Based on a few experimentations, we observed that this model could produce biased predictions that target underrepresented populations.
63
 
@@ -66,3 +101,26 @@ For instance, for sentences like `This film was filmed in COUNTRY`, this binary
66
  <img src="https://huggingface.co/distilbert-base-uncased-finetuned-sst-2-english/resolve/main/map.jpeg" alt="Map of positive probabilities per country." width="500"/>
67
 
68
  We strongly advise users to thoroughly probe these aspects on their use-cases in order to evaluate the risks of this model. We recommend looking at the following bias evaluation datasets as a place to start: [WinoBias](https://huggingface.co/datasets/wino_bias), [WinoGender](https://huggingface.co/datasets/super_glue), [Stereoset](https://huggingface.co/datasets/stereoset).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
44
 
45
  # DistilBERT base uncased finetuned SST-2
46
 
47
+ ## Table of Contents
48
+ - [Model Details](#model-details)
49
+ - [How to Get Started With the Model](#how-to-get-started-with-the-model)
50
+ - [Uses](#uses)
51
+ - [Risks, Limitations and Biases](#risks-limitations-and-biases)
52
+ - [Training](#training)
53
+
54
+ ## Model Details
55
+ **Model Description:** This model is a fine-tune checkpoint of [DistilBERT-base-uncased](https://huggingface.co/distilbert-base-uncased), fine-tuned on SST-2.
56
  This model reaches an accuracy of 91.3 on the dev set (for comparison, Bert bert-base-uncased version reaches an accuracy of 92.7).
57
+ - **Developed by:** Hugging Face
58
+ - **Model Type:** Text Classification
59
+ - **Language(s):** English
60
+ - **License:** Apache-2.0
61
+ - **Parent Model:** For more details about DistilBERT, we encourage users to check out [this model card](https://huggingface.co/distilbert-base-uncased).
62
+ - **Resources for more information:**
63
+ - [Model Documentation](https://huggingface.co/docs/transformers/main/en/model_doc/distilbert#transformers.DistilBertForSequenceClassification)
64
 
65
+ ## How to Get Started With the Model
66
 
67
+ Example of single-label classification:
68
+ ​​
69
+ ```python
70
+ import torch
71
+ from transformers import DistilBertTokenizer, DistilBertForSequenceClassification
72
 
73
+ tokenizer = DistilBertTokenizer.from_pretrained("distilbert-base-uncased")
74
+ model = DistilBertForSequenceClassification.from_pretrained("distilbert-base-uncased")
75
+
76
+ inputs = tokenizer("Hello, my dog is cute", return_tensors="pt")
77
+ with torch.no_grad():
78
+ logits = model(**inputs).logits
79
+
80
+ predicted_class_id = logits.argmax().item()
81
+ model.config.id2label[predicted_class_id]
82
 
83
+ ```
84
+
85
+ ## Uses
86
+
87
+ #### Direct Use
88
+
89
+ This model can be used for topic classification. You can use the raw model for either masked language modeling or next sentence prediction, but it's mostly intended to be fine-tuned on a downstream task. See the model hub to look for fine-tuned versions on a task that interests you.
90
+
91
+ #### Misuse and Out-of-scope Use
92
+ The model should not be used to intentionally create hostile or alienating environments for people. In addition, the model was not trained to be factual or true representations of people or events, and therefore using the model to generate such content is out-of-scope for the abilities of this model.
93
+
94
+
95
+ ## Risks, Limitations and Biases
96
 
97
  Based on a few experimentations, we observed that this model could produce biased predictions that target underrepresented populations.
98
 
 
101
  <img src="https://huggingface.co/distilbert-base-uncased-finetuned-sst-2-english/resolve/main/map.jpeg" alt="Map of positive probabilities per country." width="500"/>
102
 
103
  We strongly advise users to thoroughly probe these aspects on their use-cases in order to evaluate the risks of this model. We recommend looking at the following bias evaluation datasets as a place to start: [WinoBias](https://huggingface.co/datasets/wino_bias), [WinoGender](https://huggingface.co/datasets/super_glue), [Stereoset](https://huggingface.co/datasets/stereoset).
104
+
105
+
106
+
107
+ # Training
108
+
109
+
110
+ #### Training Data
111
+
112
+
113
+ The authors use the following Stanford Sentiment Treebank([sst2](https://huggingface.co/datasets/sst2)) corpora for the model.
114
+
115
+ #### Training Procedure
116
+
117
+ ###### Fine-tuning hyper-parameters
118
+
119
+
120
+ - learning_rate = 1e-5
121
+ - batch_size = 32
122
+ - warmup = 600
123
+ - max_seq_length = 128
124
+ - num_train_epochs = 3.0
125
+
126
+