Hugging Face's logo

osiria
/
deberta-italian-question-answering

Question Answering
Transformers
PyTorch
Safetensors
Italian
deberta-v2
Eval Results
Inference Endpoints
Model card Files Community
1
osiria commited on
Commit
24f2eec
•
1 Parent(s): 67dcec3

Upload osiria_liteqa_evaluation.ipynb

Browse files
Files changed (1) hide show
  1. osiria_liteqa_evaluation.ipynb +828 -0
osiria_liteqa_evaluation.ipynb ADDED
@@ -0,0 +1,828 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "nbformat": 4,
3
+ "nbformat_minor": 0,
4
+ "metadata": {
5
+ "colab": {
6
+ "provenance": [],
7
+ "gpuType": "T4"
8
+ },
9
+ "kernelspec": {
10
+ "name": "python3",
11
+ "display_name": "Python 3"
12
+ },
13
+ "language_info": {
14
+ "name": "python"
15
+ },
16
+ "accelerator": "GPU"
17
+ },
18
+ "cells": [
19
+ {
20
+ "cell_type": "markdown",
21
+ "source": [
22
+ "# osiria/deberta-italian-question-answering\n",
23
+ "## test set metrics"
24
+ ],
25
+ "metadata": {
26
+ "id": "oub2ir8ZDkrH"
27
+ }
28
+ },
29
+ {
30
+ "cell_type": "code",
31
+ "execution_count": 1,
32
+ "metadata": {
33
+ "id": "02K9xO6Gvto1",
34
+ "colab": {
35
+ "base_uri": "https://localhost:8080/"
36
+ },
37
+ "outputId": "0d492963-a525-4d5a-ac41-7cf1a51008a8"
38
+ },
39
+ "outputs": [
40
+ {
41
+ "output_type": "stream",
42
+ "name": "stdout",
43
+ "text": [
44
+ "Collecting git+https://github.com/huggingface/transformers\n",
45
+ " Cloning https://github.com/huggingface/transformers to /tmp/pip-req-build-dhlxsg5a\n",
46
+ " Running command git clone --filter=blob:none --quiet https://github.com/huggingface/transformers /tmp/pip-req-build-dhlxsg5a\n",
47
+ " Resolved https://github.com/huggingface/transformers to commit df5c5c62ae253055336f5bb0828ca8e3e15ab6bd\n",
48
+ " Installing build dependencies ... \u001b[?25l\u001b[?25hdone\n",
49
+ " Getting requirements to build wheel ... \u001b[?25l\u001b[?25hdone\n",
50
+ " Preparing metadata (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n",
51
+ "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from transformers==4.36.0.dev0) (3.13.1)\n",
52
+ "Requirement already satisfied: huggingface-hub<1.0,>=0.19.3 in /usr/local/lib/python3.10/dist-packages (from transformers==4.36.0.dev0) (0.19.4)\n",
53
+ "Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from transformers==4.36.0.dev0) (1.23.5)\n",
54
+ "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from transformers==4.36.0.dev0) (23.2)\n",
55
+ "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from transformers==4.36.0.dev0) (6.0.1)\n",
56
+ "Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from transformers==4.36.0.dev0) (2023.6.3)\n",
57
+ "Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from transformers==4.36.0.dev0) (2.31.0)\n",
58
+ "Requirement already satisfied: tokenizers<0.19,>=0.14 in /usr/local/lib/python3.10/dist-packages (from transformers==4.36.0.dev0) (0.15.0)\n",
59
+ "Requirement already satisfied: safetensors>=0.3.1 in /usr/local/lib/python3.10/dist-packages (from transformers==4.36.0.dev0) (0.4.1)\n",
60
+ "Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.10/dist-packages (from transformers==4.36.0.dev0) (4.66.1)\n",
61
+ "Requirement already satisfied: fsspec>=2023.5.0 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<1.0,>=0.19.3->transformers==4.36.0.dev0) (2023.6.0)\n",
62
+ "Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<1.0,>=0.19.3->transformers==4.36.0.dev0) (4.5.0)\n",
63
+ "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->transformers==4.36.0.dev0) (3.3.2)\n",
64
+ "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->transformers==4.36.0.dev0) (3.6)\n",
65
+ "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->transformers==4.36.0.dev0) (2.0.7)\n",
66
+ "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->transformers==4.36.0.dev0) (2023.11.17)\n"
67
+ ]
68
+ }
69
+ ],
70
+ "source": [
71
+ "!pip install git+https://github.com/huggingface/transformers"
72
+ ]
73
+ },
74
+ {
75
+ "cell_type": "code",
76
+ "source": [
77
+ "!pip install datasets"
78
+ ],
79
+ "metadata": {
80
+ "id": "VKgybrUswa1q",
81
+ "colab": {
82
+ "base_uri": "https://localhost:8080/"
83
+ },
84
+ "outputId": "3a52c215-7046-4185-cb95-884032b34007"
85
+ },
86
+ "execution_count": 2,
87
+ "outputs": [
88
+ {
89
+ "output_type": "stream",
90
+ "name": "stdout",
91
+ "text": [
92
+ "Requirement already satisfied: datasets in /usr/local/lib/python3.10/dist-packages (2.15.0)\n",
93
+ "Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from datasets) (1.23.5)\n",
94
+ "Requirement already satisfied: pyarrow>=8.0.0 in /usr/local/lib/python3.10/dist-packages (from datasets) (9.0.0)\n",
95
+ "Requirement already satisfied: pyarrow-hotfix in /usr/local/lib/python3.10/dist-packages (from datasets) (0.6)\n",
96
+ "Requirement already satisfied: dill<0.3.8,>=0.3.0 in /usr/local/lib/python3.10/dist-packages (from datasets) (0.3.7)\n",
97
+ "Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from datasets) (1.5.3)\n",
98
+ "Requirement already satisfied: requests>=2.19.0 in /usr/local/lib/python3.10/dist-packages (from datasets) (2.31.0)\n",
99
+ "Requirement already satisfied: tqdm>=4.62.1 in /usr/local/lib/python3.10/dist-packages (from datasets) (4.66.1)\n",
100
+ "Requirement already satisfied: xxhash in /usr/local/lib/python3.10/dist-packages (from datasets) (3.4.1)\n",
101
+ "Requirement already satisfied: multiprocess in /usr/local/lib/python3.10/dist-packages (from datasets) (0.70.15)\n",
102
+ "Requirement already satisfied: fsspec[http]<=2023.10.0,>=2023.1.0 in /usr/local/lib/python3.10/dist-packages (from datasets) (2023.6.0)\n",
103
+ "Requirement already satisfied: aiohttp in /usr/local/lib/python3.10/dist-packages (from datasets) (3.9.1)\n",
104
+ "Requirement already satisfied: huggingface-hub>=0.18.0 in /usr/local/lib/python3.10/dist-packages (from datasets) (0.19.4)\n",
105
+ "Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from datasets) (23.2)\n",
106
+ "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from datasets) (6.0.1)\n",
107
+ "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (23.1.0)\n",
108
+ "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (6.0.4)\n",
109
+ "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (1.9.3)\n",
110
+ "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (1.4.0)\n",
111
+ "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (1.3.1)\n",
112
+ "Requirement already satisfied: async-timeout<5.0,>=4.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (4.0.3)\n",
113
+ "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.18.0->datasets) (3.13.1)\n",
114
+ "Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.18.0->datasets) (4.5.0)\n",
115
+ "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests>=2.19.0->datasets) (3.3.2)\n",
116
+ "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests>=2.19.0->datasets) (3.6)\n",
117
+ "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests>=2.19.0->datasets) (2.0.7)\n",
118
+ "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests>=2.19.0->datasets) (2023.11.17)\n",
119
+ "Requirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.10/dist-packages (from pandas->datasets) (2.8.2)\n",
120
+ "Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->datasets) (2023.3.post1)\n",
121
+ "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.8.1->pandas->datasets) (1.16.0)\n"
122
+ ]
123
+ }
124
+ ]
125
+ },
126
+ {
127
+ "cell_type": "code",
128
+ "source": [
129
+ "!pip install accelerate"
130
+ ],
131
+ "metadata": {
132
+ "id": "xYEW0Hjqz2lo",
133
+ "colab": {
134
+ "base_uri": "https://localhost:8080/"
135
+ },
136
+ "outputId": "391d2364-0eb8-4fb0-9ccb-dcc6b1a3d3a2"
137
+ },
138
+ "execution_count": 3,
139
+ "outputs": [
140
+ {
141
+ "output_type": "stream",
142
+ "name": "stdout",
143
+ "text": [
144
+ "Requirement already satisfied: accelerate in /usr/local/lib/python3.10/dist-packages (0.25.0)\n",
145
+ "Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from accelerate) (1.23.5)\n",
146
+ "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from accelerate) (23.2)\n",
147
+ "Requirement already satisfied: psutil in /usr/local/lib/python3.10/dist-packages (from accelerate) (5.9.5)\n",
148
+ "Requirement already satisfied: pyyaml in /usr/local/lib/python3.10/dist-packages (from accelerate) (6.0.1)\n",
149
+ "Requirement already satisfied: torch>=1.10.0 in /usr/local/lib/python3.10/dist-packages (from accelerate) (2.1.0+cu118)\n",
150
+ "Requirement already satisfied: huggingface-hub in /usr/local/lib/python3.10/dist-packages (from accelerate) (0.19.4)\n",
151
+ "Requirement already satisfied: safetensors>=0.3.1 in /usr/local/lib/python3.10/dist-packages (from accelerate) (0.4.1)\n",
152
+ "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (3.13.1)\n",
153
+ "Requirement already satisfied: typing-extensions in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (4.5.0)\n",
154
+ "Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (1.12)\n",
155
+ "Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (3.2.1)\n",
156
+ "Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (3.1.2)\n",
157
+ "Requirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (2023.6.0)\n",
158
+ "Requirement already satisfied: triton==2.1.0 in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (2.1.0)\n",
159
+ "Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from huggingface-hub->accelerate) (2.31.0)\n",
160
+ "Requirement already satisfied: tqdm>=4.42.1 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub->accelerate) (4.66.1)\n",
161
+ "Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch>=1.10.0->accelerate) (2.1.3)\n",
162
+ "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub->accelerate) (3.3.2)\n",
163
+ "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub->accelerate) (3.6)\n",
164
+ "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub->accelerate) (2.0.7)\n",
165
+ "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub->accelerate) (2023.11.17)\n",
166
+ "Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch>=1.10.0->accelerate) (1.3.0)\n"
167
+ ]
168
+ }
169
+ ]
170
+ },
171
+ {
172
+ "cell_type": "code",
173
+ "source": [
174
+ "!pip install evaluate"
175
+ ],
176
+ "metadata": {
177
+ "id": "9OFz3JdR0cQ_",
178
+ "colab": {
179
+ "base_uri": "https://localhost:8080/"
180
+ },
181
+ "outputId": "c8c102c7-29ec-4551-8994-324853530b56"
182
+ },
183
+ "execution_count": 4,
184
+ "outputs": [
185
+ {
186
+ "output_type": "stream",
187
+ "name": "stdout",
188
+ "text": [
189
+ "Requirement already satisfied: evaluate in /usr/local/lib/python3.10/dist-packages (0.4.1)\n",
190
+ "Requirement already satisfied: datasets>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from evaluate) (2.15.0)\n",
191
+ "Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from evaluate) (1.23.5)\n",
192
+ "Requirement already satisfied: dill in /usr/local/lib/python3.10/dist-packages (from evaluate) (0.3.7)\n",
193
+ "Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from evaluate) (1.5.3)\n",
194
+ "Requirement already satisfied: requests>=2.19.0 in /usr/local/lib/python3.10/dist-packages (from evaluate) (2.31.0)\n",
195
+ "Requirement already satisfied: tqdm>=4.62.1 in /usr/local/lib/python3.10/dist-packages (from evaluate) (4.66.1)\n",
196
+ "Requirement already satisfied: xxhash in /usr/local/lib/python3.10/dist-packages (from evaluate) (3.4.1)\n",
197
+ "Requirement already satisfied: multiprocess in /usr/local/lib/python3.10/dist-packages (from evaluate) (0.70.15)\n",
198
+ "Requirement already satisfied: fsspec[http]>=2021.05.0 in /usr/local/lib/python3.10/dist-packages (from evaluate) (2023.6.0)\n",
199
+ "Requirement already satisfied: huggingface-hub>=0.7.0 in /usr/local/lib/python3.10/dist-packages (from evaluate) (0.19.4)\n",
200
+ "Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from evaluate) (23.2)\n",
201
+ "Requirement already satisfied: responses<0.19 in /usr/local/lib/python3.10/dist-packages (from evaluate) (0.18.0)\n",
202
+ "Requirement already satisfied: pyarrow>=8.0.0 in /usr/local/lib/python3.10/dist-packages (from datasets>=2.0.0->evaluate) (9.0.0)\n",
203
+ "Requirement already satisfied: pyarrow-hotfix in /usr/local/lib/python3.10/dist-packages (from datasets>=2.0.0->evaluate) (0.6)\n",
204
+ "Requirement already satisfied: aiohttp in /usr/local/lib/python3.10/dist-packages (from datasets>=2.0.0->evaluate) (3.9.1)\n",
205
+ "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from datasets>=2.0.0->evaluate) (6.0.1)\n",
206
+ "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.7.0->evaluate) (3.13.1)\n",
207
+ "Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.7.0->evaluate) (4.5.0)\n",
208
+ "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests>=2.19.0->evaluate) (3.3.2)\n",
209
+ "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests>=2.19.0->evaluate) (3.6)\n",
210
+ "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests>=2.19.0->evaluate) (2.0.7)\n",
211
+ "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests>=2.19.0->evaluate) (2023.11.17)\n",
212
+ "Requirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.10/dist-packages (from pandas->evaluate) (2.8.2)\n",
213
+ "Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->evaluate) (2023.3.post1)\n",
214
+ "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets>=2.0.0->evaluate) (23.1.0)\n",
215
+ "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets>=2.0.0->evaluate) (6.0.4)\n",
216
+ "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets>=2.0.0->evaluate) (1.9.3)\n",
217
+ "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets>=2.0.0->evaluate) (1.4.0)\n",
218
+ "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets>=2.0.0->evaluate) (1.3.1)\n",
219
+ "Requirement already satisfied: async-timeout<5.0,>=4.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets>=2.0.0->evaluate) (4.0.3)\n",
220
+ "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.8.1->pandas->evaluate) (1.16.0)\n"
221
+ ]
222
+ }
223
+ ]
224
+ },
225
+ {
226
+ "cell_type": "code",
227
+ "source": [
228
+ "!wget https://raw.githubusercontent.com/huggingface/transformers/main/examples/pytorch/question-answering/run_qa.py\n",
229
+ "!wget https://raw.githubusercontent.com/huggingface/transformers/main/examples/pytorch/question-answering/trainer_qa.py\n",
230
+ "!wget https://raw.githubusercontent.com/huggingface/transformers/main/examples/pytorch/question-answering/utils_qa.py"
231
+ ],
232
+ "metadata": {
233
+ "id": "JJhpBNwpxOTK",
234
+ "colab": {
235
+ "base_uri": "https://localhost:8080/"
236
+ },
237
+ "outputId": "7c34e19d-a391-41a2-8cd5-8abfe3788c6e"
238
+ },
239
+ "execution_count": 5,
240
+ "outputs": [
241
+ {
242
+ "output_type": "stream",
243
+ "name": "stdout",
244
+ "text": [
245
+ "--2023-12-09 13:46:17-- https://raw.githubusercontent.com/huggingface/transformers/main/examples/pytorch/question-answering/run_qa.py\n",
246
+ "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.109.133, 185.199.108.133, 185.199.110.133, ...\n",
247
+ "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.109.133|:443... connected.\n",
248
+ "HTTP request sent, awaiting response... 200 OK\n",
249
+ "Length: 32883 (32K) [text/plain]\n",
250
+ "Saving to: ‘run_qa.py.1’\n",
251
+ "\n",
252
+ "\rrun_qa.py.1 0%[ ] 0 --.-KB/s \rrun_qa.py.1 100%[===================>] 32.11K --.-KB/s in 0.01s \n",
253
+ "\n",
254
+ "2023-12-09 13:46:17 (3.06 MB/s) - ‘run_qa.py.1’ saved [32883/32883]\n",
255
+ "\n",
256
+ "--2023-12-09 13:46:17-- https://raw.githubusercontent.com/huggingface/transformers/main/examples/pytorch/question-answering/trainer_qa.py\n",
257
+ "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ...\n",
258
+ "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected.\n",
259
+ "HTTP request sent, awaiting response... 200 OK\n",
260
+ "Length: 6253 (6.1K) [text/plain]\n",
261
+ "Saving to: ‘trainer_qa.py.1’\n",
262
+ "\n",
263
+ "trainer_qa.py.1 100%[===================>] 6.11K --.-KB/s in 0s \n",
264
+ "\n",
265
+ "2023-12-09 13:46:17 (84.8 MB/s) - ‘trainer_qa.py.1’ saved [6253/6253]\n",
266
+ "\n",
267
+ "--2023-12-09 13:46:17-- https://raw.githubusercontent.com/huggingface/transformers/main/examples/pytorch/question-answering/utils_qa.py\n",
268
+ "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.111.133, 185.199.108.133, 185.199.110.133, ...\n",
269
+ "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.111.133|:443... connected.\n",
270
+ "HTTP request sent, awaiting response... 200 OK\n",
271
+ "Length: 22777 (22K) [text/plain]\n",
272
+ "Saving to: ‘utils_qa.py.1’\n",
273
+ "\n",
274
+ "utils_qa.py.1 100%[===================>] 22.24K --.-KB/s in 0.002s \n",
275
+ "\n",
276
+ "2023-12-09 13:46:17 (12.2 MB/s) - ‘utils_qa.py.1’ saved [22777/22777]\n",
277
+ "\n"
278
+ ]
279
+ }
280
+ ]
281
+ },
282
+ {
283
+ "cell_type": "code",
284
+ "source": [
285
+ "!ls"
286
+ ],
287
+ "metadata": {
288
+ "id": "Loa6y9XqzXJ4",
289
+ "colab": {
290
+ "base_uri": "https://localhost:8080/"
291
+ },
292
+ "outputId": "657803ce-5234-4690-d8cb-e7cefd81b1c1"
293
+ },
294
+ "execution_count": 6,
295
+ "outputs": [
296
+ {
297
+ "output_type": "stream",
298
+ "name": "stdout",
299
+ "text": [
300
+ "eval_results run_qa.py sample_data\t trainer_qa.py.1 utils_qa.py.1\n",
301
+ "__pycache__ run_qa.py.1 trainer_qa.py utils_qa.py\n"
302
+ ]
303
+ }
304
+ ]
305
+ },
306
+ {
307
+ "cell_type": "code",
308
+ "source": [
309
+ "!mkdir eval_results"
310
+ ],
311
+ "metadata": {
312
+ "id": "QDNxV31P0reW",
313
+ "colab": {
314
+ "base_uri": "https://localhost:8080/"
315
+ },
316
+ "outputId": "b84a6344-15b5-448a-d05f-e21ce39e36ec"
317
+ },
318
+ "execution_count": 7,
319
+ "outputs": [
320
+ {
321
+ "output_type": "stream",
322
+ "name": "stdout",
323
+ "text": [
324
+ "mkdir: cannot create directory ‘eval_results’: File exists\n"
325
+ ]
326
+ }
327
+ ]
328
+ },
329
+ {
330
+ "cell_type": "code",
331
+ "source": [
332
+ "!python -m run_qa --model_name_or_path osiria/deberta-italian-question-answering --model_revision liteqa --dataset_name squad_it --do_predict --per_device_eval_batch_size=5 --output_dir eval_results"
333
+ ],
334
+ "metadata": {
335
+ "id": "B5O5Lom8x_c4",
336
+ "colab": {
337
+ "base_uri": "https://localhost:8080/"
338
+ },
339
+ "outputId": "b7207826-cf61-4c92-9776-e5396a62d750"
340
+ },
341
+ "execution_count": 8,
342
+ "outputs": [
343
+ {
344
+ "output_type": "stream",
345
+ "name": "stdout",
346
+ "text": [
347
+ "2023-12-09 13:46:21.130219: E tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:9342] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered\n",
348
+ "2023-12-09 13:46:21.130273: E tensorflow/compiler/xla/stream_executor/cuda/cuda_fft.cc:609] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered\n",
349
+ "2023-12-09 13:46:21.130327: E tensorflow/compiler/xla/stream_executor/cuda/cuda_blas.cc:1518] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered\n",
350
+ "2023-12-09 13:46:23.380015: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT\n",
351
+ "12/09/2023 13:46:26 - WARNING - __main__ - Process rank: 0, device: cuda:0, n_gpu: 1, distributed training: False, 16-bits training: False\n",
352
+ "12/09/2023 13:46:26 - INFO - __main__ - Training/evaluation parameters TrainingArguments(\n",
353
+ "_n_gpu=1,\n",
354
+ "adafactor=False,\n",
355
+ "adam_beta1=0.9,\n",
356
+ "adam_beta2=0.999,\n",
357
+ "adam_epsilon=1e-08,\n",
358
+ "auto_find_batch_size=False,\n",
359
+ "bf16=False,\n",
360
+ "bf16_full_eval=False,\n",
361
+ "data_seed=None,\n",
362
+ "dataloader_drop_last=False,\n",
363
+ "dataloader_num_workers=0,\n",
364
+ "dataloader_persistent_workers=False,\n",
365
+ "dataloader_pin_memory=True,\n",
366
+ "ddp_backend=None,\n",
367
+ "ddp_broadcast_buffers=None,\n",
368
+ "ddp_bucket_cap_mb=None,\n",
369
+ "ddp_find_unused_parameters=None,\n",
370
+ "ddp_timeout=1800,\n",
371
+ "debug=[],\n",
372
+ "deepspeed=None,\n",
373
+ "disable_tqdm=False,\n",
374
+ "dispatch_batches=None,\n",
375
+ "do_eval=False,\n",
376
+ "do_predict=True,\n",
377
+ "do_train=False,\n",
378
+ "eval_accumulation_steps=None,\n",
379
+ "eval_delay=0,\n",
380
+ "eval_steps=None,\n",
381
+ "evaluation_strategy=no,\n",
382
+ "fp16=False,\n",
383
+ "fp16_backend=auto,\n",
384
+ "fp16_full_eval=False,\n",
385
+ "fp16_opt_level=O1,\n",
386
+ "fsdp=[],\n",
387
+ "fsdp_config={'min_num_params': 0, 'xla': False, 'xla_fsdp_grad_ckpt': False},\n",
388
+ "fsdp_min_num_params=0,\n",
389
+ "fsdp_transformer_layer_cls_to_wrap=None,\n",
390
+ "full_determinism=False,\n",
391
+ "gradient_accumulation_steps=1,\n",
392
+ "gradient_checkpointing=False,\n",
393
+ "gradient_checkpointing_kwargs=None,\n",
394
+ "greater_is_better=None,\n",
395
+ "group_by_length=False,\n",
396
+ "half_precision_backend=auto,\n",
397
+ "hub_always_push=False,\n",
398
+ "hub_model_id=None,\n",
399
+ "hub_private_repo=False,\n",
400
+ "hub_strategy=every_save,\n",
401
+ "hub_token=<HUB_TOKEN>,\n",
402
+ "ignore_data_skip=False,\n",
403
+ "include_inputs_for_metrics=False,\n",
404
+ "include_num_input_tokens_seen=False,\n",
405
+ "include_tokens_per_second=False,\n",
406
+ "jit_mode_eval=False,\n",
407
+ "label_names=None,\n",
408
+ "label_smoothing_factor=0.0,\n",
409
+ "learning_rate=5e-05,\n",
410
+ "length_column_name=length,\n",
411
+ "load_best_model_at_end=False,\n",
412
+ "local_rank=0,\n",
413
+ "log_level=passive,\n",
414
+ "log_level_replica=warning,\n",
415
+ "log_on_each_node=True,\n",
416
+ "logging_dir=eval_results/runs/Dec09_13-46-26_28318cecc41c,\n",
417
+ "logging_first_step=False,\n",
418
+ "logging_nan_inf_filter=True,\n",
419
+ "logging_steps=500,\n",
420
+ "logging_strategy=steps,\n",
421
+ "lr_scheduler_kwargs={},\n",
422
+ "lr_scheduler_type=linear,\n",
423
+ "max_grad_norm=1.0,\n",
424
+ "max_steps=-1,\n",
425
+ "metric_for_best_model=None,\n",
426
+ "mp_parameters=,\n",
427
+ "neftune_noise_alpha=None,\n",
428
+ "no_cuda=False,\n",
429
+ "num_train_epochs=3.0,\n",
430
+ "optim=adamw_torch,\n",
431
+ "optim_args=None,\n",
432
+ "output_dir=eval_results,\n",
433
+ "overwrite_output_dir=False,\n",
434
+ "past_index=-1,\n",
435
+ "per_device_eval_batch_size=5,\n",
436
+ "per_device_train_batch_size=8,\n",
437
+ "prediction_loss_only=False,\n",
438
+ "push_to_hub=False,\n",
439
+ "push_to_hub_model_id=None,\n",
440
+ "push_to_hub_organization=None,\n",
441
+ "push_to_hub_token=<PUSH_TO_HUB_TOKEN>,\n",
442
+ "ray_scope=last,\n",
443
+ "remove_unused_columns=True,\n",
444
+ "report_to=['tensorboard'],\n",
445
+ "resume_from_checkpoint=None,\n",
446
+ "run_name=eval_results,\n",
447
+ "save_on_each_node=False,\n",
448
+ "save_only_model=False,\n",
449
+ "save_safetensors=True,\n",
450
+ "save_steps=500,\n",
451
+ "save_strategy=steps,\n",
452
+ "save_total_limit=None,\n",
453
+ "seed=42,\n",
454
+ "skip_memory_metrics=True,\n",
455
+ "split_batches=False,\n",
456
+ "tf32=None,\n",
457
+ "torch_compile=False,\n",
458
+ "torch_compile_backend=None,\n",
459
+ "torch_compile_mode=None,\n",
460
+ "torchdynamo=None,\n",
461
+ "tpu_metrics_debug=False,\n",
462
+ "tpu_num_cores=None,\n",
463
+ "use_cpu=False,\n",
464
+ "use_ipex=False,\n",
465
+ "use_legacy_prediction_loop=False,\n",
466
+ "use_mps_device=False,\n",
467
+ "warmup_ratio=0.0,\n",
468
+ "warmup_steps=0,\n",
469
+ "weight_decay=0.0,\n",
470
+ ")\n",
471
+ "Loading Dataset Infos from /root/.cache/huggingface/modules/datasets_modules/datasets/squad_it/d442bdb4794b4bae227ab19105b76d706ed7cf2ac342e4c9da4a5c36bde19d71\n",
472
+ "12/09/2023 13:46:27 - INFO - datasets.info - Loading Dataset Infos from /root/.cache/huggingface/modules/datasets_modules/datasets/squad_it/d442bdb4794b4bae227ab19105b76d706ed7cf2ac342e4c9da4a5c36bde19d71\n",
473
+ "Overwrite dataset info from restored data version if exists.\n",
474
+ "12/09/2023 13:46:27 - INFO - datasets.builder - Overwrite dataset info from restored data version if exists.\n",
475
+ "Loading Dataset info from /root/.cache/huggingface/datasets/squad_it/default/0.1.0/d442bdb4794b4bae227ab19105b76d706ed7cf2ac342e4c9da4a5c36bde19d71\n",
476
+ "12/09/2023 13:46:27 - INFO - datasets.info - Loading Dataset info from /root/.cache/huggingface/datasets/squad_it/default/0.1.0/d442bdb4794b4bae227ab19105b76d706ed7cf2ac342e4c9da4a5c36bde19d71\n",
477
+ "Found cached dataset squad_it (/root/.cache/huggingface/datasets/squad_it/default/0.1.0/d442bdb4794b4bae227ab19105b76d706ed7cf2ac342e4c9da4a5c36bde19d71)\n",
478
+ "12/09/2023 13:46:27 - INFO - datasets.builder - Found cached dataset squad_it (/root/.cache/huggingface/datasets/squad_it/default/0.1.0/d442bdb4794b4bae227ab19105b76d706ed7cf2ac342e4c9da4a5c36bde19d71)\n",
479
+ "Loading Dataset info from /root/.cache/huggingface/datasets/squad_it/default/0.1.0/d442bdb4794b4bae227ab19105b76d706ed7cf2ac342e4c9da4a5c36bde19d71\n",
480
+ "12/09/2023 13:46:27 - INFO - datasets.info - Loading Dataset info from /root/.cache/huggingface/datasets/squad_it/default/0.1.0/d442bdb4794b4bae227ab19105b76d706ed7cf2ac342e4c9da4a5c36bde19d71\n",
481
+ "config.json: 100% 831/831 [00:00<00:00, 4.83MB/s]\n",
482
+ "[INFO|configuration_utils.py:739] 2023-12-09 13:46:27,364 >> loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--osiria--deberta-italian-question-answering/snapshots/67dcec39da8dab0f1978c5da262349472e7dae1a/config.json\n",
483
+ "[INFO|configuration_utils.py:802] 2023-12-09 13:46:27,368 >> Model config DebertaV2Config {\n",
484
+ " \"_name_or_path\": \"osiria/deberta-italian-question-answering\",\n",
485
+ " \"architectures\": [\n",
486
+ " \"DebertaV2ForQuestionAnswering\"\n",
487
+ " ],\n",
488
+ " \"attention_probs_dropout_prob\": 0.1,\n",
489
+ " \"hidden_act\": \"gelu\",\n",
490
+ " \"hidden_dropout_prob\": 0.1,\n",
491
+ " \"hidden_size\": 768,\n",
492
+ " \"initializer_range\": 0.02,\n",
493
+ " \"intermediate_size\": 3072,\n",
494
+ " \"layer_norm_eps\": 1e-07,\n",
495
+ " \"max_position_embeddings\": 512,\n",
496
+ " \"max_relative_positions\": -1,\n",
497
+ " \"model_type\": \"deberta-v2\",\n",
498
+ " \"norm_rel_ebd\": \"layer_norm\",\n",
499
+ " \"num_attention_heads\": 12,\n",
500
+ " \"num_hidden_layers\": 12,\n",
501
+ " \"pad_token_id\": 0,\n",
502
+ " \"pooler_dropout\": 0,\n",
503
+ " \"pooler_hidden_act\": \"gelu\",\n",
504
+ " \"pooler_hidden_size\": 768,\n",
505
+ " \"pos_att_type\": [\n",
506
+ " \"p2c\",\n",
507
+ " \"c2p\"\n",
508
+ " ],\n",
509
+ " \"position_biased_input\": false,\n",
510
+ " \"position_buckets\": 256,\n",
511
+ " \"relative_attention\": true,\n",
512
+ " \"share_att_key\": true,\n",
513
+ " \"torch_dtype\": \"float32\",\n",
514
+ " \"transformers_version\": \"4.36.0.dev0\",\n",
515
+ " \"type_vocab_size\": 0,\n",
516
+ " \"vocab_size\": 50256\n",
517
+ "}\n",
518
+ "\n",
519
+ "tokenizer_config.json: 100% 412/412 [00:00<00:00, 2.68MB/s]\n",
520
+ "tokenizer.json: 100% 2.02M/2.02M [00:00<00:00, 24.3MB/s]\n",
521
+ "[INFO|tokenization_utils_base.py:2026] 2023-12-09 13:46:27,867 >> loading file spm.model from cache at None\n",
522
+ "[INFO|tokenization_utils_base.py:2026] 2023-12-09 13:46:27,868 >> loading file tokenizer.json from cache at /root/.cache/huggingface/hub/models--osiria--deberta-italian-question-answering/snapshots/67dcec39da8dab0f1978c5da262349472e7dae1a/tokenizer.json\n",
523
+ "[INFO|tokenization_utils_base.py:2026] 2023-12-09 13:46:27,868 >> loading file added_tokens.json from cache at None\n",
524
+ "[INFO|tokenization_utils_base.py:2026] 2023-12-09 13:46:27,868 >> loading file special_tokens_map.json from cache at None\n",
525
+ "[INFO|tokenization_utils_base.py:2026] 2023-12-09 13:46:27,868 >> loading file tokenizer_config.json from cache at /root/.cache/huggingface/hub/models--osiria--deberta-italian-question-answering/snapshots/67dcec39da8dab0f1978c5da262349472e7dae1a/tokenizer_config.json\n",
526
+ "pytorch_model.bin: 100% 496M/496M [00:08<00:00, 60.4MB/s]\n",
527
+ "[INFO|modeling_utils.py:3299] 2023-12-09 13:46:36,709 >> loading weights file pytorch_model.bin from cache at /root/.cache/huggingface/hub/models--osiria--deberta-italian-question-answering/snapshots/67dcec39da8dab0f1978c5da262349472e7dae1a/pytorch_model.bin\n",
528
+ "[INFO|modeling_utils.py:4139] 2023-12-09 13:46:38,836 >> All model checkpoint weights were used when initializing DebertaV2ForQuestionAnswering.\n",
529
+ "\n",
530
+ "[INFO|modeling_utils.py:4147] 2023-12-09 13:46:38,837 >> All the weights of DebertaV2ForQuestionAnswering were initialized from the model checkpoint at osiria/deberta-italian-question-answering.\n",
531
+ "If your task is similar to the task the model of the checkpoint was trained on, you can already use DebertaV2ForQuestionAnswering for predictions without further training.\n",
532
+ "Running tokenizer on prediction dataset: 0% 0/7609 [00:00<?, ? examples/s]Caching processed dataset at /root/.cache/huggingface/datasets/squad_it/default/0.1.0/d442bdb4794b4bae227ab19105b76d706ed7cf2ac342e4c9da4a5c36bde19d71/cache-2e8257692e3c7ae9.arrow\n",
533
+ "12/09/2023 13:46:39 - INFO - datasets.arrow_dataset - Caching processed dataset at /root/.cache/huggingface/datasets/squad_it/default/0.1.0/d442bdb4794b4bae227ab19105b76d706ed7cf2ac342e4c9da4a5c36bde19d71/cache-2e8257692e3c7ae9.arrow\n",
534
+ "Running tokenizer on prediction dataset: 100% 7609/7609 [00:07<00:00, 992.36 examples/s] \n",
535
+ "Downloading builder script: 100% 4.53k/4.53k [00:00<00:00, 16.7MB/s]\n",
536
+ "Downloading extra modules: 100% 3.32k/3.32k [00:00<00:00, 17.7MB/s]\n",
537
+ "12/09/2023 13:46:52 - INFO - __main__ - *** Predict ***\n",
538
+ "[INFO|trainer.py:712] 2023-12-09 13:46:52,875 >> The following columns in the test set don't have a corresponding argument in `DebertaV2ForQuestionAnswering.forward` and have been ignored: offset_mapping, example_id. If offset_mapping, example_id are not expected by `DebertaV2ForQuestionAnswering.forward`, you can safely ignore this message.\n",
539
+ "[INFO|trainer.py:3154] 2023-12-09 13:46:52,880 >> ***** Running Prediction *****\n",
540
+ "[INFO|trainer.py:3156] 2023-12-09 13:46:52,880 >> Num examples = 8565\n",
541
+ "[INFO|trainer.py:3159] 2023-12-09 13:46:52,880 >> Batch size = 5\n",
542
+ "100% 1713/1713 [04:39<00:00, 6.08it/s]12/09/2023 13:51:46 - INFO - utils_qa - Post-processing 7609 example predictions split into 8565 features.\n",
543
+ "\n",
544
+ " 0% 0/7609 [00:00<?, ?it/s]\u001b[A\n",
545
+ " 0% 30/7609 [00:00<00:25, 296.87it/s]\u001b[A\n",
546
+ " 1% 63/7609 [00:00<00:24, 306.13it/s]\u001b[A\n",
547
+ " 1% 95/7609 [00:00<00:24, 309.31it/s]\u001b[A\n",
548
+ " 2% 130/7609 [00:00<00:23, 323.50it/s]\u001b[A\n",
549
+ " 2% 164/7609 [00:00<00:22, 327.33it/s]\u001b[A\n",
550
+ " 3% 197/7609 [00:00<00:22, 328.10it/s]\u001b[A\n",
551
+ " 3% 233/7609 [00:00<00:21, 337.31it/s]\u001b[A\n",
552
+ " 4% 267/7609 [00:00<00:22, 328.30it/s]\u001b[A\n",
553
+ " 4% 300/7609 [00:00<00:22, 319.62it/s]\u001b[A\n",
554
+ " 4% 333/7609 [00:01<00:23, 312.94it/s]\u001b[A\n",
555
+ " 5% 365/7609 [00:01<00:23, 304.83it/s]\u001b[A\n",
556
+ " 5% 401/7609 [00:01<00:22, 319.08it/s]\u001b[A\n",
557
+ " 6% 434/7609 [00:01<00:22, 316.14it/s]\u001b[A\n",
558
+ " 6% 469/7609 [00:01<00:22, 324.14it/s]\u001b[A\n",
559
+ " 7% 503/7609 [00:01<00:21, 324.56it/s]\u001b[A\n",
560
+ " 7% 536/7609 [00:01<00:24, 283.86it/s]\u001b[A\n",
561
+ " 7% 570/7609 [00:01<00:23, 298.26it/s]\u001b[A\n",
562
+ " 8% 605/7609 [00:01<00:22, 311.69it/s]\u001b[A\n",
563
+ " 8% 639/7609 [00:02<00:21, 318.46it/s]\u001b[A\n",
564
+ " 9% 674/7609 [00:02<00:21, 325.94it/s]\u001b[A\n",
565
+ " 9% 709/7609 [00:02<00:20, 331.10it/s]\u001b[A\n",
566
+ " 10% 743/7609 [00:02<00:20, 328.57it/s]\u001b[A\n",
567
+ " 10% 777/7609 [00:02<00:20, 330.22it/s]\u001b[A\n",
568
+ " 11% 811/7609 [00:02<00:21, 318.91it/s]\u001b[A\n",
569
+ " 11% 844/7609 [00:02<00:23, 294.04it/s]\u001b[A\n",
570
+ " 12% 879/7609 [00:02<00:21, 307.30it/s]\u001b[A\n",
571
+ " 12% 914/7609 [00:02<00:21, 316.53it/s]\u001b[A\n",
572
+ " 12% 947/7609 [00:03<00:21, 305.30it/s]\u001b[A\n",
573
+ " 13% 978/7609 [00:03<00:25, 256.07it/s]\u001b[A\n",
574
+ " 13% 1006/7609 [00:03<00:28, 234.41it/s]\u001b[A\n",
575
+ " 14% 1031/7609 [00:03<00:29, 223.61it/s]\u001b[A\n",
576
+ " 14% 1055/7609 [00:03<00:30, 217.33it/s]\u001b[A\n",
577
+ " 14% 1078/7609 [00:03<00:30, 211.35it/s]\u001b[A\n",
578
+ " 14% 1100/7609 [00:03<00:36, 179.51it/s]\u001b[A\n",
579
+ " 15% 1119/7609 [00:03<00:36, 178.32it/s]\u001b[A\n",
580
+ " 15% 1138/7609 [00:04<00:36, 177.17it/s]\u001b[A\n",
581
+ " 15% 1157/7609 [00:04<00:38, 167.06it/s]\u001b[A\n",
582
+ " 15% 1175/7609 [00:04<00:39, 164.57it/s]\u001b[A\n",
583
+ " 16% 1193/7609 [00:04<00:38, 166.60it/s]\u001b[A\n",
584
+ " 16% 1214/7609 [00:04<00:36, 176.82it/s]\u001b[A\n",
585
+ " 16% 1234/7609 [00:04<00:35, 182.06it/s]\u001b[A\n",
586
+ " 16% 1254/7609 [00:04<00:34, 186.72it/s]\u001b[A\n",
587
+ " 17% 1275/7609 [00:04<00:32, 192.12it/s]\u001b[A\n",
588
+ " 17% 1295/7609 [00:04<00:33, 189.71it/s]\u001b[A\n",
589
+ " 17% 1315/7609 [00:05<00:32, 192.53it/s]\u001b[A\n",
590
+ " 18% 1335/7609 [00:05<00:32, 191.18it/s]\u001b[A\n",
591
+ " 18% 1355/7609 [00:05<00:33, 188.33it/s]\u001b[A\n",
592
+ " 18% 1374/7609 [00:05<00:34, 180.65it/s]\u001b[A\n",
593
+ " 18% 1393/7609 [00:05<00:42, 147.93it/s]\u001b[A\n",
594
+ " 19% 1409/7609 [00:05<00:43, 142.12it/s]\u001b[A\n",
595
+ " 19% 1427/7609 [00:05<00:41, 149.49it/s]\u001b[A\n",
596
+ " 19% 1443/7609 [00:05<00:41, 146.88it/s]\u001b[A\n",
597
+ " 19% 1459/7609 [00:06<00:41, 147.65it/s]\u001b[A\n",
598
+ " 19% 1477/7609 [00:06<00:39, 153.62it/s]\u001b[A\n",
599
+ " 20% 1493/7609 [00:06<00:43, 139.53it/s]\u001b[A\n",
600
+ " 20% 1510/7609 [00:06<00:41, 146.34it/s]\u001b[A\n",
601
+ " 20% 1526/7609 [00:06<00:40, 148.38it/s]\u001b[A\n",
602
+ " 20% 1547/7609 [00:06<00:36, 164.60it/s]\u001b[A\n",
603
+ " 21% 1565/7609 [00:06<00:35, 168.77it/s]\u001b[A\n",
604
+ " 21% 1584/7609 [00:06<00:34, 174.61it/s]\u001b[A\n",
605
+ " 21% 1602/7609 [00:06<00:38, 156.71it/s]\u001b[A\n",
606
+ " 21% 1629/7609 [00:07<00:32, 186.13it/s]\u001b[A\n",
607
+ " 22% 1655/7609 [00:07<00:28, 206.36it/s]\u001b[A\n",
608
+ " 22% 1684/7609 [00:07<00:25, 229.80it/s]\u001b[A\n",
609
+ " 23% 1715/7609 [00:07<00:23, 252.23it/s]\u001b[A\n",
610
+ " 23% 1752/7609 [00:07<00:20, 284.45it/s]\u001b[A\n",
611
+ " 23% 1781/7609 [00:07<00:21, 274.82it/s]\u001b[A\n",
612
+ " 24% 1809/7609 [00:07<00:21, 266.13it/s]\u001b[A\n",
613
+ " 24% 1836/7609 [00:07<00:22, 256.80it/s]\u001b[A\n",
614
+ " 24% 1862/7609 [00:07<00:22, 252.67it/s]\u001b[A\n",
615
+ " 25% 1893/7609 [00:07<00:21, 267.37it/s]\u001b[A\n",
616
+ " 25% 1923/7609 [00:08<00:20, 271.80it/s]\u001b[A\n",
617
+ " 26% 1951/7609 [00:08<00:21, 267.98it/s]\u001b[A\n",
618
+ " 26% 1978/7609 [00:08<00:23, 237.33it/s]\u001b[A\n",
619
+ " 26% 2003/7609 [00:08<00:26, 209.19it/s]\u001b[A\n",
620
+ " 27% 2025/7609 [00:08<00:31, 179.71it/s]\u001b[A\n",
621
+ " 27% 2047/7609 [00:08<00:29, 188.46it/s]\u001b[A\n",
622
+ " 27% 2067/7609 [00:08<00:29, 185.33it/s]\u001b[A\n",
623
+ " 27% 2087/7609 [00:09<00:38, 141.76it/s]\u001b[A\n",
624
+ " 28% 2104/7609 [00:09<00:42, 130.53it/s]\u001b[A\n",
625
+ " 28% 2137/7609 [00:09<00:31, 172.91it/s]\u001b[A\n",
626
+ " 29% 2171/7609 [00:09<00:25, 210.54it/s]\u001b[A\n",
627
+ " 29% 2196/7609 [00:09<00:24, 217.27it/s]\u001b[A\n",
628
+ " 29% 2224/7609 [00:09<00:23, 231.07it/s]\u001b[A\n",
629
+ " 30% 2252/7609 [00:09<00:22, 242.12it/s]\u001b[A\n",
630
+ " 30% 2278/7609 [00:09<00:21, 244.50it/s]\u001b[A\n",
631
+ " 30% 2311/7609 [00:09<00:19, 266.67it/s]\u001b[A\n",
632
+ " 31% 2345/7609 [00:10<00:18, 286.50it/s]\u001b[A\n",
633
+ " 31% 2380/7609 [00:10<00:17, 302.89it/s]\u001b[A\n",
634
+ " 32% 2414/7609 [00:10<00:16, 312.08it/s]\u001b[A\n",
635
+ " 32% 2448/7609 [00:10<00:16, 318.29it/s]\u001b[A\n",
636
+ " 33% 2481/7609 [00:10<00:17, 285.32it/s]\u001b[A\n",
637
+ " 33% 2516/7609 [00:10<00:16, 301.88it/s]\u001b[A\n",
638
+ " 33% 2549/7609 [00:10<00:16, 307.36it/s]\u001b[A\n",
639
+ " 34% 2581/7609 [00:10<00:16, 303.79it/s]\u001b[A\n",
640
+ " 34% 2612/7609 [00:10<00:17, 285.90it/s]\u001b[A\n",
641
+ " 35% 2642/7609 [00:11<00:17, 289.27it/s]\u001b[A\n",
642
+ " 35% 2675/7609 [00:11<00:16, 299.99it/s]\u001b[A\n",
643
+ " 36% 2709/7609 [00:11<00:15, 309.63it/s]\u001b[A\n",
644
+ " 36% 2741/7609 [00:11<00:15, 307.48it/s]\u001b[A\n",
645
+ " 36% 2776/7609 [00:11<00:15, 317.59it/s]\u001b[A\n",
646
+ " 37% 2809/7609 [00:11<00:15, 319.54it/s]\u001b[A\n",
647
+ " 37% 2844/7609 [00:11<00:14, 325.93it/s]\u001b[A\n",
648
+ " 38% 2877/7609 [00:11<00:14, 326.53it/s]\u001b[A\n",
649
+ " 38% 2910/7609 [00:11<00:14, 321.29it/s]\u001b[A\n",
650
+ " 39% 2947/7609 [00:12<00:14, 332.73it/s]\u001b[A\n",
651
+ " 39% 2981/7609 [00:12<00:13, 334.40it/s]\u001b[A\n",
652
+ " 40% 3015/7609 [00:12<00:14, 313.75it/s]\u001b[A\n",
653
+ " 40% 3047/7609 [00:12<00:14, 309.26it/s]\u001b[A\n",
654
+ " 41% 3082/7609 [00:12<00:14, 319.12it/s]\u001b[A\n",
655
+ " 41% 3115/7609 [00:12<00:15, 299.21it/s]\u001b[A\n",
656
+ " 41% 3148/7609 [00:12<00:14, 307.52it/s]\u001b[A\n",
657
+ " 42% 3180/7609 [00:12<00:14, 298.32it/s]\u001b[A\n",
658
+ " 42% 3211/7609 [00:12<00:16, 271.36it/s]\u001b[A\n",
659
+ " 43% 3239/7609 [00:13<00:16, 272.02it/s]\u001b[A\n",
660
+ " 43% 3271/7609 [00:13<00:15, 283.66it/s]\u001b[A\n",
661
+ " 43% 3307/7609 [00:13<00:14, 302.25it/s]\u001b[A\n",
662
+ " 44% 3338/7609 [00:13<00:14, 288.65it/s]\u001b[A\n",
663
+ " 44% 3372/7609 [00:13<00:14, 302.23it/s]\u001b[A\n",
664
+ " 45% 3404/7609 [00:13<00:13, 306.99it/s]\u001b[A\n",
665
+ " 45% 3437/7609 [00:13<00:13, 312.28it/s]\u001b[A\n",
666
+ " 46% 3473/7609 [00:13<00:12, 324.45it/s]\u001b[A\n",
667
+ " 46% 3508/7609 [00:13<00:12, 331.91it/s]\u001b[A\n",
668
+ " 47% 3542/7609 [00:13<00:12, 317.38it/s]\u001b[A\n",
669
+ " 47% 3577/7609 [00:14<00:12, 324.89it/s]\u001b[A\n",
670
+ " 47% 3610/7609 [00:14<00:13, 305.90it/s]\u001b[A\n",
671
+ " 48% 3641/7609 [00:14<00:12, 305.74it/s]\u001b[A\n",
672
+ " 48% 3676/7609 [00:14<00:12, 317.70it/s]\u001b[A\n",
673
+ " 49% 3710/7609 [00:14<00:12, 321.32it/s]\u001b[A\n",
674
+ " 49% 3745/7609 [00:14<00:11, 327.62it/s]\u001b[A\n",
675
+ " 50% 3778/7609 [00:14<00:12, 309.38it/s]\u001b[A\n",
676
+ " 50% 3810/7609 [00:14<00:12, 308.31it/s]\u001b[A\n",
677
+ " 51% 3849/7609 [00:14<00:11, 329.63it/s]\u001b[A\n",
678
+ " 51% 3883/7609 [00:15<00:11, 325.75it/s]\u001b[A\n",
679
+ " 51% 3917/7609 [00:15<00:11, 328.48it/s]\u001b[A\n",
680
+ " 52% 3954/7609 [00:15<00:10, 338.18it/s]\u001b[A\n",
681
+ " 52% 3988/7609 [00:15<00:11, 320.54it/s]\u001b[A\n",
682
+ " 53% 4021/7609 [00:15<00:11, 315.80it/s]\u001b[A\n",
683
+ " 53% 4055/7609 [00:15<00:11, 320.95it/s]\u001b[A\n",
684
+ " 54% 4088/7609 [00:15<00:12, 292.58it/s]\u001b[A\n",
685
+ " 54% 4118/7609 [00:15<00:11, 293.60it/s]\u001b[A\n",
686
+ " 55% 4155/7609 [00:15<00:11, 313.82it/s]\u001b[A\n",
687
+ " 55% 4187/7609 [00:16<00:11, 305.49it/s]\u001b[A\n",
688
+ " 55% 4218/7609 [00:16<00:11, 304.90it/s]\u001b[A\n",
689
+ " 56% 4254/7609 [00:16<00:10, 319.50it/s]\u001b[A\n",
690
+ " 56% 4290/7609 [00:16<00:10, 329.58it/s]\u001b[A\n",
691
+ " 57% 4324/7609 [00:16<00:09, 331.88it/s]\u001b[A\n",
692
+ " 57% 4360/7609 [00:16<00:09, 337.72it/s]\u001b[A\n",
693
+ " 58% 4395/7609 [00:16<00:09, 339.85it/s]\u001b[A\n",
694
+ " 58% 4431/7609 [00:16<00:09, 344.61it/s]\u001b[A\n",
695
+ " 59% 4466/7609 [00:16<00:10, 311.78it/s]\u001b[A\n",
696
+ " 59% 4498/7609 [00:16<00:09, 313.10it/s]\u001b[A\n",
697
+ " 60% 4530/7609 [00:17<00:11, 260.35it/s]\u001b[A\n",
698
+ " 60% 4558/7609 [00:17<00:13, 220.21it/s]\u001b[A\n",
699
+ " 60% 4583/7609 [00:17<00:14, 212.57it/s]\u001b[A\n",
700
+ " 61% 4606/7609 [00:17<00:14, 207.48it/s]\u001b[A\n",
701
+ " 61% 4628/7609 [00:17<00:14, 204.97it/s]\u001b[A\n",
702
+ " 61% 4650/7609 [00:17<00:14, 198.71it/s]\u001b[A\n",
703
+ " 61% 4671/7609 [00:17<00:14, 196.46it/s]\u001b[A\n",
704
+ " 62% 4691/7609 [00:18<00:15, 193.22it/s]\u001b[A\n",
705
+ " 62% 4711/7609 [00:18<00:15, 186.90it/s]\u001b[A\n",
706
+ " 62% 4732/7609 [00:18<00:15, 191.45it/s]\u001b[A\n",
707
+ " 62% 4752/7609 [00:18<00:15, 190.35it/s]\u001b[A\n",
708
+ " 63% 4772/7609 [00:18<00:15, 181.29it/s]\u001b[A\n",
709
+ " 63% 4791/7609 [00:18<00:15, 178.11it/s]\u001b[A\n",
710
+ " 63% 4811/7609 [00:18<00:15, 182.47it/s]\u001b[A\n",
711
+ " 63% 4830/7609 [00:18<00:15, 180.90it/s]\u001b[A\n",
712
+ " 64% 4849/7609 [00:18<00:15, 180.39it/s]\u001b[A\n",
713
+ " 64% 4868/7609 [00:19<00:15, 181.04it/s]\u001b[A\n",
714
+ " 64% 4887/7609 [00:19<00:15, 172.19it/s]\u001b[A\n",
715
+ " 64% 4906/7609 [00:19<00:15, 175.10it/s]\u001b[A\n",
716
+ " 65% 4924/7609 [00:19<00:15, 175.36it/s]\u001b[A\n",
717
+ " 65% 4942/7609 [00:19<00:15, 172.89it/s]\u001b[A\n",
718
+ " 65% 4960/7609 [00:19<00:17, 153.44it/s]\u001b[A\n",
719
+ " 65% 4980/7609 [00:19<00:16, 164.31it/s]\u001b[A\n",
720
+ " 66% 4997/7609 [00:19<00:15, 164.29it/s]\u001b[A\n",
721
+ " 66% 5015/7609 [00:19<00:15, 166.32it/s]\u001b[A\n",
722
+ " 66% 5035/7609 [00:20<00:14, 174.85it/s]\u001b[A\n",
723
+ " 66% 5054/7609 [00:20<00:14, 178.16it/s]\u001b[A\n",
724
+ " 67% 5073/7609 [00:20<00:14, 180.39it/s]\u001b[A\n",
725
+ " 67% 5092/7609 [00:20<00:13, 182.22it/s]\u001b[A\n",
726
+ " 67% 5112/7609 [00:20<00:13, 184.17it/s]\u001b[A\n",
727
+ " 67% 5132/7609 [00:20<00:13, 186.85it/s]\u001b[A\n",
728
+ " 68% 5151/7609 [00:20<00:13, 186.90it/s]\u001b[A\n",
729
+ " 68% 5170/7609 [00:20<00:13, 185.90it/s]\u001b[A\n",
730
+ " 68% 5199/7609 [00:20<00:11, 215.56it/s]\u001b[A\n",
731
+ " 69% 5234/7609 [00:20<00:09, 254.26it/s]\u001b[A\n",
732
+ " 69% 5268/7609 [00:21<00:08, 278.67it/s]\u001b[A\n",
733
+ " 70% 5304/7609 [00:21<00:07, 300.10it/s]\u001b[A\n",
734
+ " 70% 5336/7609 [00:21<00:07, 305.12it/s]\u001b[A\n",
735
+ " 71% 5370/7609 [00:21<00:07, 314.40it/s]\u001b[A\n",
736
+ " 71% 5405/7609 [00:21<00:06, 323.66it/s]\u001b[A\n",
737
+ " 71% 5438/7609 [00:21<00:06, 321.54it/s]\u001b[A\n",
738
+ " 72% 5471/7609 [00:21<00:06, 322.00it/s]\u001b[A\n",
739
+ " 72% 5506/7609 [00:21<00:06, 329.54it/s]\u001b[A\n",
740
+ " 73% 5539/7609 [00:21<00:06, 329.48it/s]\u001b[A\n",
741
+ " 73% 5572/7609 [00:21<00:06, 320.73it/s]\u001b[A\n",
742
+ " 74% 5608/7609 [00:22<00:06, 330.97it/s]\u001b[A\n",
743
+ " 74% 5642/7609 [00:22<00:05, 332.83it/s]\u001b[A\n",
744
+ " 75% 5676/7609 [00:22<00:06, 319.04it/s]\u001b[A\n",
745
+ " 75% 5713/7609 [00:22<00:05, 331.88it/s]\u001b[A\n",
746
+ " 76% 5748/7609 [00:22<00:05, 336.91it/s]\u001b[A\n",
747
+ " 76% 5784/7609 [00:22<00:05, 342.31it/s]\u001b[A\n",
748
+ " 77% 5822/7609 [00:22<00:05, 350.47it/s]\u001b[A\n",
749
+ " 77% 5858/7609 [00:22<00:05, 332.99it/s]\u001b[A\n",
750
+ " 77% 5892/7609 [00:22<00:05, 334.82it/s]\u001b[A\n",
751
+ " 78% 5927/7609 [00:23<00:04, 336.67it/s]\u001b[A\n",
752
+ " 78% 5961/7609 [00:23<00:05, 317.28it/s]\u001b[A\n",
753
+ " 79% 5994/7609 [00:23<00:05, 308.88it/s]\u001b[A\n",
754
+ " 79% 6026/7609 [00:23<00:05, 287.20it/s]\u001b[A\n",
755
+ " 80% 6056/7609 [00:23<00:05, 281.41it/s]\u001b[A\n",
756
+ " 80% 6092/7609 [00:23<00:05, 302.34it/s]\u001b[A\n",
757
+ " 80% 6123/7609 [00:23<00:05, 265.75it/s]\u001b[A\n",
758
+ " 81% 6151/7609 [00:23<00:06, 212.18it/s]\u001b[A\n",
759
+ " 81% 6188/7609 [00:24<00:05, 246.92it/s]\u001b[A\n",
760
+ " 82% 6225/7609 [00:24<00:05, 276.23it/s]\u001b[A\n",
761
+ " 82% 6259/7609 [00:24<00:04, 291.50it/s]\u001b[A\n",
762
+ " 83% 6297/7609 [00:24<00:04, 314.78it/s]\u001b[A\n",
763
+ " 83% 6333/7609 [00:24<00:03, 325.16it/s]\u001b[A\n",
764
+ " 84% 6367/7609 [00:24<00:03, 320.45it/s]\u001b[A\n",
765
+ " 84% 6404/7609 [00:24<00:03, 332.01it/s]\u001b[A\n",
766
+ " 85% 6440/7609 [00:24<00:03, 339.49it/s]\u001b[A\n",
767
+ " 85% 6476/7609 [00:24<00:03, 344.22it/s]\u001b[A\n",
768
+ " 86% 6511/7609 [00:24<00:03, 339.51it/s]\u001b[A\n",
769
+ " 86% 6546/7609 [00:25<00:03, 338.59it/s]\u001b[A\n",
770
+ " 87% 6584/7609 [00:25<00:02, 349.58it/s]\u001b[A\n",
771
+ " 87% 6620/7609 [00:25<00:03, 313.74it/s]\u001b[A\n",
772
+ " 87% 6654/7609 [00:25<00:02, 319.78it/s]\u001b[A\n",
773
+ " 88% 6687/7609 [00:25<00:02, 315.09it/s]\u001b[A\n",
774
+ " 88% 6720/7609 [00:25<00:02, 318.11it/s]\u001b[A\n",
775
+ " 89% 6753/7609 [00:25<00:02, 315.68it/s]\u001b[A\n",
776
+ " 89% 6785/7609 [00:25<00:02, 312.97it/s]\u001b[A\n",
777
+ " 90% 6817/7609 [00:25<00:02, 304.53it/s]\u001b[A\n",
778
+ " 90% 6848/7609 [00:26<00:02, 301.63it/s]\u001b[A\n",
779
+ " 90% 6879/7609 [00:26<00:02, 297.74it/s]\u001b[A\n",
780
+ " 91% 6912/7609 [00:26<00:02, 306.45it/s]\u001b[A\n",
781
+ " 91% 6947/7609 [00:26<00:02, 318.14it/s]\u001b[A\n",
782
+ " 92% 6983/7609 [00:26<00:01, 329.33it/s]\u001b[A\n",
783
+ " 92% 7018/7609 [00:26<00:01, 334.50it/s]\u001b[A\n",
784
+ " 93% 7052/7609 [00:26<00:01, 323.93it/s]\u001b[A\n",
785
+ " 93% 7085/7609 [00:26<00:01, 284.11it/s]\u001b[A\n",
786
+ " 94% 7115/7609 [00:26<00:01, 269.39it/s]\u001b[A\n",
787
+ " 94% 7143/7609 [00:27<00:01, 270.86it/s]\u001b[A\n",
788
+ " 94% 7172/7609 [00:27<00:01, 275.86it/s]\u001b[A\n",
789
+ " 95% 7200/7609 [00:27<00:01, 261.83it/s]\u001b[A\n",
790
+ " 95% 7236/7609 [00:27<00:01, 286.70it/s]\u001b[A\n",
791
+ " 96% 7268/7609 [00:27<00:01, 294.25it/s]\u001b[A\n",
792
+ " 96% 7302/7609 [00:27<00:01, 305.73it/s]\u001b[A\n",
793
+ " 96% 7333/7609 [00:27<00:00, 305.03it/s]\u001b[A\n",
794
+ " 97% 7369/7609 [00:27<00:00, 317.90it/s]\u001b[A\n",
795
+ " 97% 7402/7609 [00:27<00:00, 319.15it/s]\u001b[A\n",
796
+ " 98% 7435/7609 [00:27<00:00, 317.37it/s]\u001b[A\n",
797
+ " 98% 7467/7609 [00:28<00:00, 298.63it/s]\u001b[A\n",
798
+ " 99% 7498/7609 [00:28<00:00, 292.04it/s]\u001b[A\n",
799
+ " 99% 7528/7609 [00:28<00:00, 276.90it/s]\u001b[A\n",
800
+ " 99% 7556/7609 [00:28<00:00, 241.41it/s]\u001b[A\n",
801
+ "100% 7609/7609 [00:28<00:00, 265.00it/s]\n",
802
+ "12/09/2023 13:52:15 - INFO - utils_qa - Saving predictions to eval_results/predict_predictions.json.\n",
803
+ "12/09/2023 13:52:15 - INFO - utils_qa - Saving nbest_preds to eval_results/predict_nbest_predictions.json.\n",
804
+ "***** predict metrics *****\n",
805
+ " predict_samples = 8565\n",
806
+ " test_exact_match = 70.1932\n",
807
+ " test_f1 = 81.017\n",
808
+ " test_runtime = 0:04:42.02\n",
809
+ " test_samples_per_second = 30.369\n",
810
+ " test_steps_per_second = 6.074\n",
811
+ "[INFO|modelcard.py:452] 2023-12-09 13:52:19,396 >> Dropping the following result as it does not have all the necessary fields:\n",
812
+ "{'task': {'name': 'Question Answering', 'type': 'question-answering'}, 'dataset': {'name': 'squad_it', 'type': 'squad_it'}}\n",
813
+ "100% 1713/1713 [05:23<00:00, 5.29it/s]\n"
814
+ ]
815
+ }
816
+ ]
817
+ },
818
+ {
819
+ "cell_type": "code",
820
+ "source": [],
821
+ "metadata": {
822
+ "id": "ij8YkY0xyNXI"
823
+ },
824
+ "execution_count": null,
825
+ "outputs": []
826
+ }
827
+ ]
828
+ }