julien-c HF staff commited on
Commit
fa1f9ef
1 Parent(s): 555216d

Migrate model card from transformers-repo

Browse files

Read announcement at https://discuss.huggingface.co/t/announcement-all-model-cards-will-be-migrated-to-hf-co-model-repos/2755
Original file history: https://github.com/huggingface/transformers/commits/master/model_cards/henryk/bert-base-multilingual-cased-finetuned-polish-squad2/README.md

Files changed (1) hide show
  1. README.md +95 -95
README.md CHANGED
@@ -1,96 +1,96 @@
1
- ---
2
- language: polish
3
- ---
4
-
5
- # Multilingual + Polish SQuAD2.0
6
-
7
- This model is the multilingual model provided by the Google research team with a fine-tuned dutch Q&A downstream task.
8
-
9
- ## Details of the language model
10
-
11
- Language model ([**bert-base-multilingual-cased**](https://github.com/google-research/bert/blob/master/multilingual.md)):
12
- 12-layer, 768-hidden, 12-heads, 110M parameters.
13
- Trained on cased text in the top 104 languages with the largest Wikipedias.
14
-
15
- ## Details of the downstream task
16
- Using the `mtranslate` Python module, [**SQuAD2.0**](https://rajpurkar.github.io/SQuAD-explorer/) was machine-translated. In order to find the start tokens, the direct translations of the answers were searched in the corresponding paragraphs. Due to the different translations depending on the context (missing context in the pure answer), the answer could not always be found in the text, and thus a loss of question-answer examples occurred. This is a potential problem where errors can occur in the data set.
17
-
18
- | Dataset | # Q&A |
19
- | ---------------------- | ----- |
20
- | SQuAD2.0 Train | 130 K |
21
- | Polish SQuAD2.0 Train | 83.1 K |
22
- | SQuAD2.0 Dev | 12 K |
23
- | Polish SQuAD2.0 Dev | 8.5 K |
24
-
25
-
26
- ## Model benchmark
27
-
28
- | Model | EM/F1 |HasAns (EM/F1) | NoAns |
29
- | ---------------------- | ----- | ----- | ----- |
30
- | [SlavicBERT](https://huggingface.co/DeepPavlov/bert-base-bg-cs-pl-ru-cased) | 69.35/71.51 | 47.02/54.09 | 79.20 |
31
- | [polBERT](https://huggingface.co/dkleczek/bert-base-polish-uncased-v1) | 67.33/69.80| 45.73/53.80 | 76.87 |
32
- | [multiBERT](https://huggingface.co/bert-base-multilingual-cased) | **70.76**/**72.92** |45.00/52.04 | 82.13 |
33
-
34
- ## Model training
35
-
36
- The model was trained on a **Tesla V100** GPU with the following command:
37
-
38
- ```python
39
- export SQUAD_DIR=path/to/pl_squad
40
-
41
- python run_squad.py
42
- --model_type bert \
43
- --model_name_or_path bert-base-multilingual-cased \
44
- --do_train \
45
- --do_eval \
46
- --version_2_with_negative \
47
- --train_file $SQUAD_DIR/pl_squadv2_train.json \
48
- --predict_file $SQUAD_DIR/pl_squadv2_dev.json \
49
- --num_train_epochs 2 \
50
- --max_seq_length 384 \
51
- --doc_stride 128 \
52
- --save_steps=8000 \
53
- --output_dir ../../output \
54
- --overwrite_cache \
55
- --overwrite_output_dir
56
- ```
57
-
58
- **Results**:
59
-
60
- {'exact': 70.76671723655035, 'f1': 72.92156947155917, 'total': 8569, 'HasAns_exact': 45.00762195121951, 'HasAns_f1': 52.04456128116991, 'HasAns_total': 2624, 'NoAns_exact': 82.13624894869638, '
61
- NoAns_f1': 82.13624894869638, 'NoAns_total': 5945, 'best_exact': 71.72365503559342, 'best_exact_thresh': 0.0, 'best_f1': 73.62662512059369, 'best_f1_thresh': 0.0}
62
-
63
-
64
- ## Model in action
65
-
66
- Fast usage with **pipelines**:
67
-
68
- ```python
69
- from transformers import pipeline
70
-
71
- qa_pipeline = pipeline(
72
- "question-answering",
73
- model="henryk/bert-base-multilingual-cased-finetuned-polish-squad2",
74
- tokenizer="henryk/bert-base-multilingual-cased-finetuned-polish-squad2"
75
- )
76
-
77
- qa_pipeline({
78
- 'context': "Warszawa jest największym miastem w Polsce pod względem liczby ludności i powierzchni",
79
- 'question': "Jakie jest największe miasto w Polsce?"})
80
-
81
- ```
82
-
83
- # Output:
84
-
85
- ```json
86
- {
87
- "score": 0.9986,
88
- "start": 0,
89
- "end": 8,
90
- "answer": "Warszawa"
91
- }
92
- ```
93
-
94
- ## Contact
95
-
96
  Please do not hesitate to contact me via [LinkedIn](https://www.linkedin.com/in/henryk-borzymowski-0755a2167/) if you want to discuss or get access to the Polish version of SQuAD.
1
+ ---
2
+ language: pl
3
+ ---
4
+
5
+ # Multilingual + Polish SQuAD2.0
6
+
7
+ This model is the multilingual model provided by the Google research team with a fine-tuned polish Q&A downstream task.
8
+
9
+ ## Details of the language model
10
+
11
+ Language model ([**bert-base-multilingual-cased**](https://github.com/google-research/bert/blob/master/multilingual.md)):
12
+ 12-layer, 768-hidden, 12-heads, 110M parameters.
13
+ Trained on cased text in the top 104 languages with the largest Wikipedias.
14
+
15
+ ## Details of the downstream task
16
+ Using the `mtranslate` Python module, [**SQuAD2.0**](https://rajpurkar.github.io/SQuAD-explorer/) was machine-translated. In order to find the start tokens, the direct translations of the answers were searched in the corresponding paragraphs. Due to the different translations depending on the context (missing context in the pure answer), the answer could not always be found in the text, and thus a loss of question-answer examples occurred. This is a potential problem where errors can occur in the data set.
17
+
18
+ | Dataset | # Q&A |
19
+ | ---------------------- | ----- |
20
+ | SQuAD2.0 Train | 130 K |
21
+ | Polish SQuAD2.0 Train | 83.1 K |
22
+ | SQuAD2.0 Dev | 12 K |
23
+ | Polish SQuAD2.0 Dev | 8.5 K |
24
+
25
+
26
+ ## Model benchmark
27
+
28
+ | Model | EM/F1 |HasAns (EM/F1) | NoAns |
29
+ | ---------------------- | ----- | ----- | ----- |
30
+ | [SlavicBERT](https://huggingface.co/DeepPavlov/bert-base-bg-cs-pl-ru-cased) | 69.35/71.51 | 47.02/54.09 | 79.20 |
31
+ | [polBERT](https://huggingface.co/dkleczek/bert-base-polish-uncased-v1) | 67.33/69.80| 45.73/53.80 | 76.87 |
32
+ | [multiBERT](https://huggingface.co/bert-base-multilingual-cased) | **70.76**/**72.92** |45.00/52.04 | 82.13 |
33
+
34
+ ## Model training
35
+
36
+ The model was trained on a **Tesla V100** GPU with the following command:
37
+
38
+ ```python
39
+ export SQUAD_DIR=path/to/pl_squad
40
+
41
+ python run_squad.py
42
+ --model_type bert \
43
+ --model_name_or_path bert-base-multilingual-cased \
44
+ --do_train \
45
+ --do_eval \
46
+ --version_2_with_negative \
47
+ --train_file $SQUAD_DIR/pl_squadv2_train.json \
48
+ --predict_file $SQUAD_DIR/pl_squadv2_dev.json \
49
+ --num_train_epochs 2 \
50
+ --max_seq_length 384 \
51
+ --doc_stride 128 \
52
+ --save_steps=8000 \
53
+ --output_dir ../../output \
54
+ --overwrite_cache \
55
+ --overwrite_output_dir
56
+ ```
57
+
58
+ **Results**:
59
+
60
+ {'exact': 70.76671723655035, 'f1': 72.92156947155917, 'total': 8569, 'HasAns_exact': 45.00762195121951, 'HasAns_f1': 52.04456128116991, 'HasAns_total': 2624, 'NoAns_exact': 82.13624894869638, '
61
+ NoAns_f1': 82.13624894869638, 'NoAns_total': 5945, 'best_exact': 71.72365503559342, 'best_exact_thresh': 0.0, 'best_f1': 73.62662512059369, 'best_f1_thresh': 0.0}
62
+
63
+
64
+ ## Model in action
65
+
66
+ Fast usage with **pipelines**:
67
+
68
+ ```python
69
+ from transformers import pipeline
70
+
71
+ qa_pipeline = pipeline(
72
+ "question-answering",
73
+ model="henryk/bert-base-multilingual-cased-finetuned-polish-squad2",
74
+ tokenizer="henryk/bert-base-multilingual-cased-finetuned-polish-squad2"
75
+ )
76
+
77
+ qa_pipeline({
78
+ 'context': "Warszawa jest największym miastem w Polsce pod względem liczby ludności i powierzchni",
79
+ 'question': "Jakie jest największe miasto w Polsce?"})
80
+
81
+ ```
82
+
83
+ # Output:
84
+
85
+ ```json
86
+ {
87
+ "score": 0.9986,
88
+ "start": 0,
89
+ "end": 8,
90
+ "answer": "Warszawa"
91
+ }
92
+ ```
93
+
94
+ ## Contact
95
+
96
  Please do not hesitate to contact me via [LinkedIn](https://www.linkedin.com/in/henryk-borzymowski-0755a2167/) if you want to discuss or get access to the Polish version of SQuAD.