Hugging Face's logo

osiria
/
bert-italian-uncased-question-answering

Question Answering
Transformers
PyTorch
Italian
bert
Eval Results
Inference Endpoints
Model card Files Community
osiria commited on
Commit
9b7706d
•
1 Parent(s): 55913ac

Upload osiria_bert_italian_uncased_qa_evaluation.ipynb

Browse files
Files changed (1) hide show
  1. osiria_bert_italian_uncased_qa_evaluation.ipynb +885 -0
osiria_bert_italian_uncased_qa_evaluation.ipynb ADDED
@@ -0,0 +1,885 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "nbformat": 4,
3
+ "nbformat_minor": 0,
4
+ "metadata": {
5
+ "colab": {
6
+ "provenance": [],
7
+ "gpuType": "T4"
8
+ },
9
+ "kernelspec": {
10
+ "name": "python3",
11
+ "display_name": "Python 3"
12
+ },
13
+ "language_info": {
14
+ "name": "python"
15
+ },
16
+ "accelerator": "GPU"
17
+ },
18
+ "cells": [
19
+ {
20
+ "cell_type": "markdown",
21
+ "source": [
22
+ "# osiria/bert-italian-cased-question-answering\n",
23
+ "## test set metrics"
24
+ ],
25
+ "metadata": {
26
+ "id": "oub2ir8ZDkrH"
27
+ }
28
+ },
29
+ {
30
+ "cell_type": "code",
31
+ "execution_count": 1,
32
+ "metadata": {
33
+ "colab": {
34
+ "base_uri": "https://localhost:8080/"
35
+ },
36
+ "id": "02K9xO6Gvto1",
37
+ "outputId": "6adbc718-df6a-437d-eb07-d67404f9cd2b"
38
+ },
39
+ "outputs": [
40
+ {
41
+ "output_type": "stream",
42
+ "name": "stdout",
43
+ "text": [
44
+ "Collecting git+https://github.com/huggingface/transformers\n",
45
+ " Cloning https://github.com/huggingface/transformers to /tmp/pip-req-build-m860w1jf\n",
46
+ " Running command git clone --filter=blob:none --quiet https://github.com/huggingface/transformers /tmp/pip-req-build-m860w1jf\n",
47
+ " Resolved https://github.com/huggingface/transformers to commit df5c5c62ae253055336f5bb0828ca8e3e15ab6bd\n",
48
+ " Installing build dependencies ... \u001b[?25l\u001b[?25hdone\n",
49
+ " Getting requirements to build wheel ... \u001b[?25l\u001b[?25hdone\n",
50
+ " Preparing metadata (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n",
51
+ "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from transformers==4.36.0.dev0) (3.13.1)\n",
52
+ "Requirement already satisfied: huggingface-hub<1.0,>=0.19.3 in /usr/local/lib/python3.10/dist-packages (from transformers==4.36.0.dev0) (0.19.4)\n",
53
+ "Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from transformers==4.36.0.dev0) (1.23.5)\n",
54
+ "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from transformers==4.36.0.dev0) (23.2)\n",
55
+ "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from transformers==4.36.0.dev0) (6.0.1)\n",
56
+ "Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from transformers==4.36.0.dev0) (2023.6.3)\n",
57
+ "Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from transformers==4.36.0.dev0) (2.31.0)\n",
58
+ "Requirement already satisfied: tokenizers<0.19,>=0.14 in /usr/local/lib/python3.10/dist-packages (from transformers==4.36.0.dev0) (0.15.0)\n",
59
+ "Requirement already satisfied: safetensors>=0.3.1 in /usr/local/lib/python3.10/dist-packages (from transformers==4.36.0.dev0) (0.4.1)\n",
60
+ "Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.10/dist-packages (from transformers==4.36.0.dev0) (4.66.1)\n",
61
+ "Requirement already satisfied: fsspec>=2023.5.0 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<1.0,>=0.19.3->transformers==4.36.0.dev0) (2023.6.0)\n",
62
+ "Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<1.0,>=0.19.3->transformers==4.36.0.dev0) (4.5.0)\n",
63
+ "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->transformers==4.36.0.dev0) (3.3.2)\n",
64
+ "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->transformers==4.36.0.dev0) (3.6)\n",
65
+ "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->transformers==4.36.0.dev0) (2.0.7)\n",
66
+ "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->transformers==4.36.0.dev0) (2023.11.17)\n",
67
+ "Building wheels for collected packages: transformers\n",
68
+ " Building wheel for transformers (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n",
69
+ " Created wheel for transformers: filename=transformers-4.36.0.dev0-py3-none-any.whl size=8195433 sha256=c5e14f23bb3ae5edc677f78f448f1a67a961992c880e0441c99cb6f4db9469cf\n",
70
+ " Stored in directory: /tmp/pip-ephem-wheel-cache-ytzg25y9/wheels/c0/14/d6/6c9a5582d2ac191ec0a483be151a4495fe1eb2a6706ca49f1b\n",
71
+ "Successfully built transformers\n",
72
+ "Installing collected packages: transformers\n",
73
+ " Attempting uninstall: transformers\n",
74
+ " Found existing installation: transformers 4.35.2\n",
75
+ " Uninstalling transformers-4.35.2:\n",
76
+ " Successfully uninstalled transformers-4.35.2\n",
77
+ "Successfully installed transformers-4.36.0.dev0\n"
78
+ ]
79
+ }
80
+ ],
81
+ "source": [
82
+ "!pip install git+https://github.com/huggingface/transformers"
83
+ ]
84
+ },
85
+ {
86
+ "cell_type": "code",
87
+ "source": [
88
+ "!pip install datasets"
89
+ ],
90
+ "metadata": {
91
+ "colab": {
92
+ "base_uri": "https://localhost:8080/"
93
+ },
94
+ "id": "VKgybrUswa1q",
95
+ "outputId": "5c5b315c-6a81-4ce8-e1a8-43f78c084fdc"
96
+ },
97
+ "execution_count": 2,
98
+ "outputs": [
99
+ {
100
+ "output_type": "stream",
101
+ "name": "stdout",
102
+ "text": [
103
+ "Collecting datasets\n",
104
+ " Downloading datasets-2.15.0-py3-none-any.whl (521 kB)\n",
105
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m521.2/521.2 kB\u001b[0m \u001b[31m5.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
106
+ "\u001b[?25hRequirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from datasets) (1.23.5)\n",
107
+ "Requirement already satisfied: pyarrow>=8.0.0 in /usr/local/lib/python3.10/dist-packages (from datasets) (9.0.0)\n",
108
+ "Collecting pyarrow-hotfix (from datasets)\n",
109
+ " Downloading pyarrow_hotfix-0.6-py3-none-any.whl (7.9 kB)\n",
110
+ "Collecting dill<0.3.8,>=0.3.0 (from datasets)\n",
111
+ " Downloading dill-0.3.7-py3-none-any.whl (115 kB)\n",
112
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m115.3/115.3 kB\u001b[0m \u001b[31m7.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
113
+ "\u001b[?25hRequirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from datasets) (1.5.3)\n",
114
+ "Requirement already satisfied: requests>=2.19.0 in /usr/local/lib/python3.10/dist-packages (from datasets) (2.31.0)\n",
115
+ "Requirement already satisfied: tqdm>=4.62.1 in /usr/local/lib/python3.10/dist-packages (from datasets) (4.66.1)\n",
116
+ "Requirement already satisfied: xxhash in /usr/local/lib/python3.10/dist-packages (from datasets) (3.4.1)\n",
117
+ "Collecting multiprocess (from datasets)\n",
118
+ " Downloading multiprocess-0.70.15-py310-none-any.whl (134 kB)\n",
119
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m134.8/134.8 kB\u001b[0m \u001b[31m7.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
120
+ "\u001b[?25hRequirement already satisfied: fsspec[http]<=2023.10.0,>=2023.1.0 in /usr/local/lib/python3.10/dist-packages (from datasets) (2023.6.0)\n",
121
+ "Requirement already satisfied: aiohttp in /usr/local/lib/python3.10/dist-packages (from datasets) (3.9.1)\n",
122
+ "Requirement already satisfied: huggingface-hub>=0.18.0 in /usr/local/lib/python3.10/dist-packages (from datasets) (0.19.4)\n",
123
+ "Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from datasets) (23.2)\n",
124
+ "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from datasets) (6.0.1)\n",
125
+ "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (23.1.0)\n",
126
+ "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (6.0.4)\n",
127
+ "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (1.9.3)\n",
128
+ "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (1.4.0)\n",
129
+ "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (1.3.1)\n",
130
+ "Requirement already satisfied: async-timeout<5.0,>=4.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (4.0.3)\n",
131
+ "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.18.0->datasets) (3.13.1)\n",
132
+ "Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.18.0->datasets) (4.5.0)\n",
133
+ "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests>=2.19.0->datasets) (3.3.2)\n",
134
+ "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests>=2.19.0->datasets) (3.6)\n",
135
+ "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests>=2.19.0->datasets) (2.0.7)\n",
136
+ "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests>=2.19.0->datasets) (2023.11.17)\n",
137
+ "Requirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.10/dist-packages (from pandas->datasets) (2.8.2)\n",
138
+ "Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->datasets) (2023.3.post1)\n",
139
+ "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.8.1->pandas->datasets) (1.16.0)\n",
140
+ "Installing collected packages: pyarrow-hotfix, dill, multiprocess, datasets\n",
141
+ "Successfully installed datasets-2.15.0 dill-0.3.7 multiprocess-0.70.15 pyarrow-hotfix-0.6\n"
142
+ ]
143
+ }
144
+ ]
145
+ },
146
+ {
147
+ "cell_type": "code",
148
+ "source": [
149
+ "!pip install accelerate"
150
+ ],
151
+ "metadata": {
152
+ "colab": {
153
+ "base_uri": "https://localhost:8080/"
154
+ },
155
+ "id": "xYEW0Hjqz2lo",
156
+ "outputId": "b37614da-516f-42ce-da0b-16946bd26424"
157
+ },
158
+ "execution_count": 3,
159
+ "outputs": [
160
+ {
161
+ "output_type": "stream",
162
+ "name": "stdout",
163
+ "text": [
164
+ "Collecting accelerate\n",
165
+ " Downloading accelerate-0.25.0-py3-none-any.whl (265 kB)\n",
166
+ "\u001b[?25l \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m0.0/265.7 kB\u001b[0m \u001b[31m?\u001b[0m eta \u001b[36m-:--:--\u001b[0m\r\u001b[2K \u001b[91m━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[90m╺\u001b[0m\u001b[90m━━━━━━━━━━━━━\u001b[0m \u001b[32m174.1/265.7 kB\u001b[0m \u001b[31m5.0 MB/s\u001b[0m eta \u001b[36m0:00:01\u001b[0m\r\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m265.7/265.7 kB\u001b[0m \u001b[31m6.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
167
+ "\u001b[?25hRequirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from accelerate) (1.23.5)\n",
168
+ "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from accelerate) (23.2)\n",
169
+ "Requirement already satisfied: psutil in /usr/local/lib/python3.10/dist-packages (from accelerate) (5.9.5)\n",
170
+ "Requirement already satisfied: pyyaml in /usr/local/lib/python3.10/dist-packages (from accelerate) (6.0.1)\n",
171
+ "Requirement already satisfied: torch>=1.10.0 in /usr/local/lib/python3.10/dist-packages (from accelerate) (2.1.0+cu118)\n",
172
+ "Requirement already satisfied: huggingface-hub in /usr/local/lib/python3.10/dist-packages (from accelerate) (0.19.4)\n",
173
+ "Requirement already satisfied: safetensors>=0.3.1 in /usr/local/lib/python3.10/dist-packages (from accelerate) (0.4.1)\n",
174
+ "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (3.13.1)\n",
175
+ "Requirement already satisfied: typing-extensions in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (4.5.0)\n",
176
+ "Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (1.12)\n",
177
+ "Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (3.2.1)\n",
178
+ "Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (3.1.2)\n",
179
+ "Requirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (2023.6.0)\n",
180
+ "Requirement already satisfied: triton==2.1.0 in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (2.1.0)\n",
181
+ "Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from huggingface-hub->accelerate) (2.31.0)\n",
182
+ "Requirement already satisfied: tqdm>=4.42.1 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub->accelerate) (4.66.1)\n",
183
+ "Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch>=1.10.0->accelerate) (2.1.3)\n",
184
+ "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub->accelerate) (3.3.2)\n",
185
+ "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub->accelerate) (3.6)\n",
186
+ "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub->accelerate) (2.0.7)\n",
187
+ "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub->accelerate) (2023.11.17)\n",
188
+ "Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch>=1.10.0->accelerate) (1.3.0)\n",
189
+ "Installing collected packages: accelerate\n",
190
+ "Successfully installed accelerate-0.25.0\n"
191
+ ]
192
+ }
193
+ ]
194
+ },
195
+ {
196
+ "cell_type": "code",
197
+ "source": [
198
+ "!pip install evaluate"
199
+ ],
200
+ "metadata": {
201
+ "colab": {
202
+ "base_uri": "https://localhost:8080/"
203
+ },
204
+ "id": "9OFz3JdR0cQ_",
205
+ "outputId": "7d3e65fc-462d-497c-c48e-e01af07be572"
206
+ },
207
+ "execution_count": 4,
208
+ "outputs": [
209
+ {
210
+ "output_type": "stream",
211
+ "name": "stdout",
212
+ "text": [
213
+ "Collecting evaluate\n",
214
+ " Downloading evaluate-0.4.1-py3-none-any.whl (84 kB)\n",
215
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m84.1/84.1 kB\u001b[0m \u001b[31m1.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
216
+ "\u001b[?25hRequirement already satisfied: datasets>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from evaluate) (2.15.0)\n",
217
+ "Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from evaluate) (1.23.5)\n",
218
+ "Requirement already satisfied: dill in /usr/local/lib/python3.10/dist-packages (from evaluate) (0.3.7)\n",
219
+ "Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from evaluate) (1.5.3)\n",
220
+ "Requirement already satisfied: requests>=2.19.0 in /usr/local/lib/python3.10/dist-packages (from evaluate) (2.31.0)\n",
221
+ "Requirement already satisfied: tqdm>=4.62.1 in /usr/local/lib/python3.10/dist-packages (from evaluate) (4.66.1)\n",
222
+ "Requirement already satisfied: xxhash in /usr/local/lib/python3.10/dist-packages (from evaluate) (3.4.1)\n",
223
+ "Requirement already satisfied: multiprocess in /usr/local/lib/python3.10/dist-packages (from evaluate) (0.70.15)\n",
224
+ "Requirement already satisfied: fsspec[http]>=2021.05.0 in /usr/local/lib/python3.10/dist-packages (from evaluate) (2023.6.0)\n",
225
+ "Requirement already satisfied: huggingface-hub>=0.7.0 in /usr/local/lib/python3.10/dist-packages (from evaluate) (0.19.4)\n",
226
+ "Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from evaluate) (23.2)\n",
227
+ "Collecting responses<0.19 (from evaluate)\n",
228
+ " Downloading responses-0.18.0-py3-none-any.whl (38 kB)\n",
229
+ "Requirement already satisfied: pyarrow>=8.0.0 in /usr/local/lib/python3.10/dist-packages (from datasets>=2.0.0->evaluate) (9.0.0)\n",
230
+ "Requirement already satisfied: pyarrow-hotfix in /usr/local/lib/python3.10/dist-packages (from datasets>=2.0.0->evaluate) (0.6)\n",
231
+ "Requirement already satisfied: aiohttp in /usr/local/lib/python3.10/dist-packages (from datasets>=2.0.0->evaluate) (3.9.1)\n",
232
+ "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from datasets>=2.0.0->evaluate) (6.0.1)\n",
233
+ "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.7.0->evaluate) (3.13.1)\n",
234
+ "Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.7.0->evaluate) (4.5.0)\n",
235
+ "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests>=2.19.0->evaluate) (3.3.2)\n",
236
+ "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests>=2.19.0->evaluate) (3.6)\n",
237
+ "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests>=2.19.0->evaluate) (2.0.7)\n",
238
+ "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests>=2.19.0->evaluate) (2023.11.17)\n",
239
+ "Requirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.10/dist-packages (from pandas->evaluate) (2.8.2)\n",
240
+ "Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->evaluate) (2023.3.post1)\n",
241
+ "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets>=2.0.0->evaluate) (23.1.0)\n",
242
+ "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets>=2.0.0->evaluate) (6.0.4)\n",
243
+ "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets>=2.0.0->evaluate) (1.9.3)\n",
244
+ "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets>=2.0.0->evaluate) (1.4.0)\n",
245
+ "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets>=2.0.0->evaluate) (1.3.1)\n",
246
+ "Requirement already satisfied: async-timeout<5.0,>=4.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets>=2.0.0->evaluate) (4.0.3)\n",
247
+ "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.8.1->pandas->evaluate) (1.16.0)\n",
248
+ "Installing collected packages: responses, evaluate\n",
249
+ "Successfully installed evaluate-0.4.1 responses-0.18.0\n"
250
+ ]
251
+ }
252
+ ]
253
+ },
254
+ {
255
+ "cell_type": "code",
256
+ "source": [
257
+ "!wget https://raw.githubusercontent.com/huggingface/transformers/main/examples/pytorch/question-answering/run_qa.py\n",
258
+ "!wget https://raw.githubusercontent.com/huggingface/transformers/main/examples/pytorch/question-answering/trainer_qa.py\n",
259
+ "!wget https://raw.githubusercontent.com/huggingface/transformers/main/examples/pytorch/question-answering/utils_qa.py"
260
+ ],
261
+ "metadata": {
262
+ "colab": {
263
+ "base_uri": "https://localhost:8080/"
264
+ },
265
+ "id": "JJhpBNwpxOTK",
266
+ "outputId": "98ad66fd-6030-4829-dada-cf3b3d443259"
267
+ },
268
+ "execution_count": 5,
269
+ "outputs": [
270
+ {
271
+ "output_type": "stream",
272
+ "name": "stdout",
273
+ "text": [
274
+ "--2023-12-09 12:05:19-- https://raw.githubusercontent.com/huggingface/transformers/main/examples/pytorch/question-answering/run_qa.py\n",
275
+ "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.110.133, 185.199.109.133, 185.199.111.133, ...\n",
276
+ "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.110.133|:443... connected.\n",
277
+ "HTTP request sent, awaiting response... 200 OK\n",
278
+ "Length: 32883 (32K) [text/plain]\n",
279
+ "Saving to: ‘run_qa.py’\n",
280
+ "\n",
281
+ "\rrun_qa.py 0%[ ] 0 --.-KB/s \rrun_qa.py 100%[===================>] 32.11K --.-KB/s in 0.01s \n",
282
+ "\n",
283
+ "2023-12-09 12:05:20 (3.22 MB/s) - ‘run_qa.py’ saved [32883/32883]\n",
284
+ "\n",
285
+ "--2023-12-09 12:05:20-- https://raw.githubusercontent.com/huggingface/transformers/main/examples/pytorch/question-answering/trainer_qa.py\n",
286
+ "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.109.133, 185.199.111.133, 185.199.110.133, ...\n",
287
+ "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.109.133|:443... connected.\n",
288
+ "HTTP request sent, awaiting response... 200 OK\n",
289
+ "Length: 6253 (6.1K) [text/plain]\n",
290
+ "Saving to: ‘trainer_qa.py’\n",
291
+ "\n",
292
+ "trainer_qa.py 100%[===================>] 6.11K --.-KB/s in 0s \n",
293
+ "\n",
294
+ "2023-12-09 12:05:20 (85.6 MB/s) - ‘trainer_qa.py’ saved [6253/6253]\n",
295
+ "\n",
296
+ "--2023-12-09 12:05:20-- https://raw.githubusercontent.com/huggingface/transformers/main/examples/pytorch/question-answering/utils_qa.py\n",
297
+ "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.110.133, 185.199.109.133, ...\n",
298
+ "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected.\n",
299
+ "HTTP request sent, awaiting response... 200 OK\n",
300
+ "Length: 22777 (22K) [text/plain]\n",
301
+ "Saving to: ‘utils_qa.py’\n",
302
+ "\n",
303
+ "utils_qa.py 100%[===================>] 22.24K --.-KB/s in 0.001s \n",
304
+ "\n",
305
+ "2023-12-09 12:05:20 (14.8 MB/s) - ‘utils_qa.py’ saved [22777/22777]\n",
306
+ "\n"
307
+ ]
308
+ }
309
+ ]
310
+ },
311
+ {
312
+ "cell_type": "code",
313
+ "source": [
314
+ "!ls"
315
+ ],
316
+ "metadata": {
317
+ "colab": {
318
+ "base_uri": "https://localhost:8080/"
319
+ },
320
+ "id": "Loa6y9XqzXJ4",
321
+ "outputId": "8bd0088d-b009-4d08-c2a7-dd123598432a"
322
+ },
323
+ "execution_count": 6,
324
+ "outputs": [
325
+ {
326
+ "output_type": "stream",
327
+ "name": "stdout",
328
+ "text": [
329
+ "run_qa.py sample_data\ttrainer_qa.py utils_qa.py\n"
330
+ ]
331
+ }
332
+ ]
333
+ },
334
+ {
335
+ "cell_type": "code",
336
+ "source": [
337
+ "!mkdir eval_results"
338
+ ],
339
+ "metadata": {
340
+ "id": "QDNxV31P0reW"
341
+ },
342
+ "execution_count": 7,
343
+ "outputs": []
344
+ },
345
+ {
346
+ "cell_type": "code",
347
+ "source": [
348
+ "!python -m run_qa --model_name_or_path osiria/bert-italian-uncased-question-answering --dataset_name squad_it --do_predict --per_device_eval_batch_size=5 --output_dir eval_results"
349
+ ],
350
+ "metadata": {
351
+ "colab": {
352
+ "base_uri": "https://localhost:8080/"
353
+ },
354
+ "id": "B5O5Lom8x_c4",
355
+ "outputId": "8d066e37-8407-4232-de44-6290495776f9"
356
+ },
357
+ "execution_count": 8,
358
+ "outputs": [
359
+ {
360
+ "output_type": "stream",
361
+ "name": "stdout",
362
+ "text": [
363
+ "2023-12-09 12:05:29.016992: E tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:9342] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered\n",
364
+ "2023-12-09 12:05:29.017054: E tensorflow/compiler/xla/stream_executor/cuda/cuda_fft.cc:609] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered\n",
365
+ "2023-12-09 12:05:29.017091: E tensorflow/compiler/xla/stream_executor/cuda/cuda_blas.cc:1518] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered\n",
366
+ "2023-12-09 12:05:30.116878: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT\n",
367
+ "12/09/2023 12:05:33 - WARNING - __main__ - Process rank: 0, device: cuda:0, n_gpu: 1, distributed training: False, 16-bits training: False\n",
368
+ "12/09/2023 12:05:33 - INFO - __main__ - Training/evaluation parameters TrainingArguments(\n",
369
+ "_n_gpu=1,\n",
370
+ "adafactor=False,\n",
371
+ "adam_beta1=0.9,\n",
372
+ "adam_beta2=0.999,\n",
373
+ "adam_epsilon=1e-08,\n",
374
+ "auto_find_batch_size=False,\n",
375
+ "bf16=False,\n",
376
+ "bf16_full_eval=False,\n",
377
+ "data_seed=None,\n",
378
+ "dataloader_drop_last=False,\n",
379
+ "dataloader_num_workers=0,\n",
380
+ "dataloader_persistent_workers=False,\n",
381
+ "dataloader_pin_memory=True,\n",
382
+ "ddp_backend=None,\n",
383
+ "ddp_broadcast_buffers=None,\n",
384
+ "ddp_bucket_cap_mb=None,\n",
385
+ "ddp_find_unused_parameters=None,\n",
386
+ "ddp_timeout=1800,\n",
387
+ "debug=[],\n",
388
+ "deepspeed=None,\n",
389
+ "disable_tqdm=False,\n",
390
+ "dispatch_batches=None,\n",
391
+ "do_eval=False,\n",
392
+ "do_predict=True,\n",
393
+ "do_train=False,\n",
394
+ "eval_accumulation_steps=None,\n",
395
+ "eval_delay=0,\n",
396
+ "eval_steps=None,\n",
397
+ "evaluation_strategy=no,\n",
398
+ "fp16=False,\n",
399
+ "fp16_backend=auto,\n",
400
+ "fp16_full_eval=False,\n",
401
+ "fp16_opt_level=O1,\n",
402
+ "fsdp=[],\n",
403
+ "fsdp_config={'min_num_params': 0, 'xla': False, 'xla_fsdp_grad_ckpt': False},\n",
404
+ "fsdp_min_num_params=0,\n",
405
+ "fsdp_transformer_layer_cls_to_wrap=None,\n",
406
+ "full_determinism=False,\n",
407
+ "gradient_accumulation_steps=1,\n",
408
+ "gradient_checkpointing=False,\n",
409
+ "gradient_checkpointing_kwargs=None,\n",
410
+ "greater_is_better=None,\n",
411
+ "group_by_length=False,\n",
412
+ "half_precision_backend=auto,\n",
413
+ "hub_always_push=False,\n",
414
+ "hub_model_id=None,\n",
415
+ "hub_private_repo=False,\n",
416
+ "hub_strategy=every_save,\n",
417
+ "hub_token=<HUB_TOKEN>,\n",
418
+ "ignore_data_skip=False,\n",
419
+ "include_inputs_for_metrics=False,\n",
420
+ "include_num_input_tokens_seen=False,\n",
421
+ "include_tokens_per_second=False,\n",
422
+ "jit_mode_eval=False,\n",
423
+ "label_names=None,\n",
424
+ "label_smoothing_factor=0.0,\n",
425
+ "learning_rate=5e-05,\n",
426
+ "length_column_name=length,\n",
427
+ "load_best_model_at_end=False,\n",
428
+ "local_rank=0,\n",
429
+ "log_level=passive,\n",
430
+ "log_level_replica=warning,\n",
431
+ "log_on_each_node=True,\n",
432
+ "logging_dir=eval_results/runs/Dec09_12-05-33_4da00120f505,\n",
433
+ "logging_first_step=False,\n",
434
+ "logging_nan_inf_filter=True,\n",
435
+ "logging_steps=500,\n",
436
+ "logging_strategy=steps,\n",
437
+ "lr_scheduler_kwargs={},\n",
438
+ "lr_scheduler_type=linear,\n",
439
+ "max_grad_norm=1.0,\n",
440
+ "max_steps=-1,\n",
441
+ "metric_for_best_model=None,\n",
442
+ "mp_parameters=,\n",
443
+ "neftune_noise_alpha=None,\n",
444
+ "no_cuda=False,\n",
445
+ "num_train_epochs=3.0,\n",
446
+ "optim=adamw_torch,\n",
447
+ "optim_args=None,\n",
448
+ "output_dir=eval_results,\n",
449
+ "overwrite_output_dir=False,\n",
450
+ "past_index=-1,\n",
451
+ "per_device_eval_batch_size=5,\n",
452
+ "per_device_train_batch_size=8,\n",
453
+ "prediction_loss_only=False,\n",
454
+ "push_to_hub=False,\n",
455
+ "push_to_hub_model_id=None,\n",
456
+ "push_to_hub_organization=None,\n",
457
+ "push_to_hub_token=<PUSH_TO_HUB_TOKEN>,\n",
458
+ "ray_scope=last,\n",
459
+ "remove_unused_columns=True,\n",
460
+ "report_to=['tensorboard'],\n",
461
+ "resume_from_checkpoint=None,\n",
462
+ "run_name=eval_results,\n",
463
+ "save_on_each_node=False,\n",
464
+ "save_only_model=False,\n",
465
+ "save_safetensors=True,\n",
466
+ "save_steps=500,\n",
467
+ "save_strategy=steps,\n",
468
+ "save_total_limit=None,\n",
469
+ "seed=42,\n",
470
+ "skip_memory_metrics=True,\n",
471
+ "split_batches=False,\n",
472
+ "tf32=None,\n",
473
+ "torch_compile=False,\n",
474
+ "torch_compile_backend=None,\n",
475
+ "torch_compile_mode=None,\n",
476
+ "torchdynamo=None,\n",
477
+ "tpu_metrics_debug=False,\n",
478
+ "tpu_num_cores=None,\n",
479
+ "use_cpu=False,\n",
480
+ "use_ipex=False,\n",
481
+ "use_legacy_prediction_loop=False,\n",
482
+ "use_mps_device=False,\n",
483
+ "warmup_ratio=0.0,\n",
484
+ "warmup_steps=0,\n",
485
+ "weight_decay=0.0,\n",
486
+ ")\n",
487
+ "https://huggingface.co/datasets/squad_it/resolve/main/squad_it.py not found in cache or force_download set to True, downloading to /root/.cache/huggingface/datasets/downloads/2f7a246f661e6acb3a9ad072ed14241ea343265e7d863b88717897941c3ab1dd.671c3c3daf2f05e0355b498b0f5a7f2167fa94dc6de36195fe1ba0503e6dcc30.py.incomplete\n",
488
+ "12/09/2023 12:05:33 - INFO - datasets.utils.file_utils - https://huggingface.co/datasets/squad_it/resolve/main/squad_it.py not found in cache or force_download set to True, downloading to /root/.cache/huggingface/datasets/downloads/2f7a246f661e6acb3a9ad072ed14241ea343265e7d863b88717897941c3ab1dd.671c3c3daf2f05e0355b498b0f5a7f2167fa94dc6de36195fe1ba0503e6dcc30.py.incomplete\n",
489
+ "Downloading builder script: 100% 4.74k/4.74k [00:00<00:00, 21.2MB/s]\n",
490
+ "storing https://huggingface.co/datasets/squad_it/resolve/main/squad_it.py in cache at /root/.cache/huggingface/datasets/downloads/2f7a246f661e6acb3a9ad072ed14241ea343265e7d863b88717897941c3ab1dd.671c3c3daf2f05e0355b498b0f5a7f2167fa94dc6de36195fe1ba0503e6dcc30.py\n",
491
+ "12/09/2023 12:05:34 - INFO - datasets.utils.file_utils - storing https://huggingface.co/datasets/squad_it/resolve/main/squad_it.py in cache at /root/.cache/huggingface/datasets/downloads/2f7a246f661e6acb3a9ad072ed14241ea343265e7d863b88717897941c3ab1dd.671c3c3daf2f05e0355b498b0f5a7f2167fa94dc6de36195fe1ba0503e6dcc30.py\n",
492
+ "creating metadata file for /root/.cache/huggingface/datasets/downloads/2f7a246f661e6acb3a9ad072ed14241ea343265e7d863b88717897941c3ab1dd.671c3c3daf2f05e0355b498b0f5a7f2167fa94dc6de36195fe1ba0503e6dcc30.py\n",
493
+ "12/09/2023 12:05:34 - INFO - datasets.utils.file_utils - creating metadata file for /root/.cache/huggingface/datasets/downloads/2f7a246f661e6acb3a9ad072ed14241ea343265e7d863b88717897941c3ab1dd.671c3c3daf2f05e0355b498b0f5a7f2167fa94dc6de36195fe1ba0503e6dcc30.py\n",
494
+ "https://huggingface.co/datasets/squad_it/resolve/main/dataset_infos.json not found in cache or force_download set to True, downloading to /root/.cache/huggingface/datasets/downloads/f6d7650a1e4a9f0e42b54f34ac614f7c1142ad26c1bd19a448eb606fe09f7515.b459489a7b1a4d859decbeef851034c7846ef6cce8ec2cf192cf4b4f40d0865d.incomplete\n",
495
+ "12/09/2023 12:05:34 - INFO - datasets.utils.file_utils - https://huggingface.co/datasets/squad_it/resolve/main/dataset_infos.json not found in cache or force_download set to True, downloading to /root/.cache/huggingface/datasets/downloads/f6d7650a1e4a9f0e42b54f34ac614f7c1142ad26c1bd19a448eb606fe09f7515.b459489a7b1a4d859decbeef851034c7846ef6cce8ec2cf192cf4b4f40d0865d.incomplete\n",
496
+ "Downloading metadata: 100% 2.51k/2.51k [00:00<00:00, 13.1MB/s]\n",
497
+ "storing https://huggingface.co/datasets/squad_it/resolve/main/dataset_infos.json in cache at /root/.cache/huggingface/datasets/downloads/f6d7650a1e4a9f0e42b54f34ac614f7c1142ad26c1bd19a448eb606fe09f7515.b459489a7b1a4d859decbeef851034c7846ef6cce8ec2cf192cf4b4f40d0865d\n",
498
+ "12/09/2023 12:05:34 - INFO - datasets.utils.file_utils - storing https://huggingface.co/datasets/squad_it/resolve/main/dataset_infos.json in cache at /root/.cache/huggingface/datasets/downloads/f6d7650a1e4a9f0e42b54f34ac614f7c1142ad26c1bd19a448eb606fe09f7515.b459489a7b1a4d859decbeef851034c7846ef6cce8ec2cf192cf4b4f40d0865d\n",
499
+ "creating metadata file for /root/.cache/huggingface/datasets/downloads/f6d7650a1e4a9f0e42b54f34ac614f7c1142ad26c1bd19a448eb606fe09f7515.b459489a7b1a4d859decbeef851034c7846ef6cce8ec2cf192cf4b4f40d0865d\n",
500
+ "12/09/2023 12:05:34 - INFO - datasets.utils.file_utils - creating metadata file for /root/.cache/huggingface/datasets/downloads/f6d7650a1e4a9f0e42b54f34ac614f7c1142ad26c1bd19a448eb606fe09f7515.b459489a7b1a4d859decbeef851034c7846ef6cce8ec2cf192cf4b4f40d0865d\n",
501
+ "https://huggingface.co/datasets/squad_it/resolve/main/README.md not found in cache or force_download set to True, downloading to /root/.cache/huggingface/datasets/downloads/b664892e5965a1ef2e173eadc7e9cbf6ad4aeb6240d7b3bb6ab9f32850363a6f.71fe7dbf5f3b574d85b57b5a722ebc49fe8ec8b03d655344891fb1b05c4634fb.incomplete\n",
502
+ "12/09/2023 12:05:35 - INFO - datasets.utils.file_utils - https://huggingface.co/datasets/squad_it/resolve/main/README.md not found in cache or force_download set to True, downloading to /root/.cache/huggingface/datasets/downloads/b664892e5965a1ef2e173eadc7e9cbf6ad4aeb6240d7b3bb6ab9f32850363a6f.71fe7dbf5f3b574d85b57b5a722ebc49fe8ec8b03d655344891fb1b05c4634fb.incomplete\n",
503
+ "Downloading readme: 100% 7.27k/7.27k [00:00<00:00, 26.1MB/s]\n",
504
+ "storing https://huggingface.co/datasets/squad_it/resolve/main/README.md in cache at /root/.cache/huggingface/datasets/downloads/b664892e5965a1ef2e173eadc7e9cbf6ad4aeb6240d7b3bb6ab9f32850363a6f.71fe7dbf5f3b574d85b57b5a722ebc49fe8ec8b03d655344891fb1b05c4634fb\n",
505
+ "12/09/2023 12:05:35 - INFO - datasets.utils.file_utils - storing https://huggingface.co/datasets/squad_it/resolve/main/README.md in cache at /root/.cache/huggingface/datasets/downloads/b664892e5965a1ef2e173eadc7e9cbf6ad4aeb6240d7b3bb6ab9f32850363a6f.71fe7dbf5f3b574d85b57b5a722ebc49fe8ec8b03d655344891fb1b05c4634fb\n",
506
+ "creating metadata file for /root/.cache/huggingface/datasets/downloads/b664892e5965a1ef2e173eadc7e9cbf6ad4aeb6240d7b3bb6ab9f32850363a6f.71fe7dbf5f3b574d85b57b5a722ebc49fe8ec8b03d655344891fb1b05c4634fb\n",
507
+ "12/09/2023 12:05:35 - INFO - datasets.utils.file_utils - creating metadata file for /root/.cache/huggingface/datasets/downloads/b664892e5965a1ef2e173eadc7e9cbf6ad4aeb6240d7b3bb6ab9f32850363a6f.71fe7dbf5f3b574d85b57b5a722ebc49fe8ec8b03d655344891fb1b05c4634fb\n",
508
+ "Loading Dataset Infos from /root/.cache/huggingface/modules/datasets_modules/datasets/squad_it/d442bdb4794b4bae227ab19105b76d706ed7cf2ac342e4c9da4a5c36bde19d71\n",
509
+ "12/09/2023 12:05:35 - INFO - datasets.info - Loading Dataset Infos from /root/.cache/huggingface/modules/datasets_modules/datasets/squad_it/d442bdb4794b4bae227ab19105b76d706ed7cf2ac342e4c9da4a5c36bde19d71\n",
510
+ "Generating dataset squad_it (/root/.cache/huggingface/datasets/squad_it/default/0.1.0/d442bdb4794b4bae227ab19105b76d706ed7cf2ac342e4c9da4a5c36bde19d71)\n",
511
+ "12/09/2023 12:05:35 - INFO - datasets.builder - Generating dataset squad_it (/root/.cache/huggingface/datasets/squad_it/default/0.1.0/d442bdb4794b4bae227ab19105b76d706ed7cf2ac342e4c9da4a5c36bde19d71)\n",
512
+ "Downloading and preparing dataset squad_it/default to /root/.cache/huggingface/datasets/squad_it/default/0.1.0/d442bdb4794b4bae227ab19105b76d706ed7cf2ac342e4c9da4a5c36bde19d71...\n",
513
+ "12/09/2023 12:05:35 - INFO - datasets.builder - Downloading and preparing dataset squad_it/default to /root/.cache/huggingface/datasets/squad_it/default/0.1.0/d442bdb4794b4bae227ab19105b76d706ed7cf2ac342e4c9da4a5c36bde19d71...\n",
514
+ "Dataset not on Hf google storage. Downloading and preparing it from source\n",
515
+ "12/09/2023 12:05:35 - INFO - datasets.builder - Dataset not on Hf google storage. Downloading and preparing it from source\n",
516
+ "Downloading data files: 0% 0/2 [00:00<?, ?it/s]https://github.com/crux82/squad-it/raw/master/SQuAD_it-train.json.gz not found in cache or force_download set to True, downloading to /root/.cache/huggingface/datasets/downloads/44ac0548c3284061d1356ff7f5212fe42b923b0aa0854c886ce93dfeb7ab629b.incomplete\n",
517
+ "12/09/2023 12:05:36 - INFO - datasets.utils.file_utils - https://github.com/crux82/squad-it/raw/master/SQuAD_it-train.json.gz not found in cache or force_download set to True, downloading to /root/.cache/huggingface/datasets/downloads/44ac0548c3284061d1356ff7f5212fe42b923b0aa0854c886ce93dfeb7ab629b.incomplete\n",
518
+ "\n",
519
+ "Downloading data: 0% 0.00/7.73M [00:00<?, ?B/s]\u001b[A\n",
520
+ "Downloading data: 100% 7.73M/7.73M [00:00<00:00, 55.1MB/s]\n",
521
+ "storing https://github.com/crux82/squad-it/raw/master/SQuAD_it-train.json.gz in cache at /root/.cache/huggingface/datasets/downloads/44ac0548c3284061d1356ff7f5212fe42b923b0aa0854c886ce93dfeb7ab629b\n",
522
+ "12/09/2023 12:05:36 - INFO - datasets.utils.file_utils - storing https://github.com/crux82/squad-it/raw/master/SQuAD_it-train.json.gz in cache at /root/.cache/huggingface/datasets/downloads/44ac0548c3284061d1356ff7f5212fe42b923b0aa0854c886ce93dfeb7ab629b\n",
523
+ "creating metadata file for /root/.cache/huggingface/datasets/downloads/44ac0548c3284061d1356ff7f5212fe42b923b0aa0854c886ce93dfeb7ab629b\n",
524
+ "12/09/2023 12:05:36 - INFO - datasets.utils.file_utils - creating metadata file for /root/.cache/huggingface/datasets/downloads/44ac0548c3284061d1356ff7f5212fe42b923b0aa0854c886ce93dfeb7ab629b\n",
525
+ "Downloading data files: 50% 1/2 [00:01<00:01, 1.19s/it]https://github.com/crux82/squad-it/raw/master/SQuAD_it-test.json.gz not found in cache or force_download set to True, downloading to /root/.cache/huggingface/datasets/downloads/e560a4a54eb929ce542e945627942afca80859dcbf499866a59984eb9026d31f.incomplete\n",
526
+ "12/09/2023 12:05:37 - INFO - datasets.utils.file_utils - https://github.com/crux82/squad-it/raw/master/SQuAD_it-test.json.gz not found in cache or force_download set to True, downloading to /root/.cache/huggingface/datasets/downloads/e560a4a54eb929ce542e945627942afca80859dcbf499866a59984eb9026d31f.incomplete\n",
527
+ "\n",
528
+ "Downloading data: 100% 1.05M/1.05M [00:00<00:00, 16.0MB/s]\n",
529
+ "storing https://github.com/crux82/squad-it/raw/master/SQuAD_it-test.json.gz in cache at /root/.cache/huggingface/datasets/downloads/e560a4a54eb929ce542e945627942afca80859dcbf499866a59984eb9026d31f\n",
530
+ "12/09/2023 12:05:37 - INFO - datasets.utils.file_utils - storing https://github.com/crux82/squad-it/raw/master/SQuAD_it-test.json.gz in cache at /root/.cache/huggingface/datasets/downloads/e560a4a54eb929ce542e945627942afca80859dcbf499866a59984eb9026d31f\n",
531
+ "creating metadata file for /root/.cache/huggingface/datasets/downloads/e560a4a54eb929ce542e945627942afca80859dcbf499866a59984eb9026d31f\n",
532
+ "12/09/2023 12:05:37 - INFO - datasets.utils.file_utils - creating metadata file for /root/.cache/huggingface/datasets/downloads/e560a4a54eb929ce542e945627942afca80859dcbf499866a59984eb9026d31f\n",
533
+ "Downloading data files: 100% 2/2 [00:02<00:00, 1.11s/it]\n",
534
+ "Downloading took 0.0 min\n",
535
+ "12/09/2023 12:05:37 - INFO - datasets.download.download_manager - Downloading took 0.0 min\n",
536
+ "Checksum Computation took 0.0 min\n",
537
+ "12/09/2023 12:05:37 - INFO - datasets.download.download_manager - Checksum Computation took 0.0 min\n",
538
+ "Extracting data files: 100% 2/2 [00:00<00:00, 8.42it/s]\n",
539
+ "Generating train split\n",
540
+ "12/09/2023 12:05:38 - INFO - datasets.builder - Generating train split\n",
541
+ "Generating train split: 100% 54159/54159 [00:06<00:00, 8803.51 examples/s] \n",
542
+ "Generating test split\n",
543
+ "12/09/2023 12:05:44 - INFO - datasets.builder - Generating test split\n",
544
+ "Generating test split: 100% 7609/7609 [00:00<00:00, 11069.99 examples/s]\n",
545
+ "All the splits matched successfully.\n",
546
+ "12/09/2023 12:05:44 - INFO - datasets.utils.info_utils - All the splits matched successfully.\n",
547
+ "Dataset squad_it downloaded and prepared to /root/.cache/huggingface/datasets/squad_it/default/0.1.0/d442bdb4794b4bae227ab19105b76d706ed7cf2ac342e4c9da4a5c36bde19d71. Subsequent calls will reuse this data.\n",
548
+ "12/09/2023 12:05:44 - INFO - datasets.builder - Dataset squad_it downloaded and prepared to /root/.cache/huggingface/datasets/squad_it/default/0.1.0/d442bdb4794b4bae227ab19105b76d706ed7cf2ac342e4c9da4a5c36bde19d71. Subsequent calls will reuse this data.\n",
549
+ "config.json: 100% 789/789 [00:00<00:00, 3.83MB/s]\n",
550
+ "[INFO|configuration_utils.py:739] 2023-12-09 12:05:45,154 >> loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--osiria--bert-italian-uncased-question-answering/snapshots/9985f20cde546f78a743c4d4f1efd605519b1c41/config.json\n",
551
+ "[INFO|configuration_utils.py:802] 2023-12-09 12:05:45,159 >> Model config BertConfig {\n",
552
+ " \"_name_or_path\": \"osiria/bert-italian-uncased-question-answering\",\n",
553
+ " \"architectures\": [\n",
554
+ " \"BertForQuestionAnswering\"\n",
555
+ " ],\n",
556
+ " \"attention_probs_dropout_prob\": 0.1,\n",
557
+ " \"classifier_dropout\": null,\n",
558
+ " \"directionality\": \"bidi\",\n",
559
+ " \"hidden_act\": \"gelu\",\n",
560
+ " \"hidden_dropout_prob\": 0.1,\n",
561
+ " \"hidden_size\": 768,\n",
562
+ " \"initializer_range\": 0.02,\n",
563
+ " \"intermediate_size\": 3072,\n",
564
+ " \"layer_norm_eps\": 1e-12,\n",
565
+ " \"max_position_embeddings\": 512,\n",
566
+ " \"model_type\": \"bert\",\n",
567
+ " \"num_attention_heads\": 12,\n",
568
+ " \"num_hidden_layers\": 12,\n",
569
+ " \"pad_token_id\": 0,\n",
570
+ " \"pooler_fc_size\": 768,\n",
571
+ " \"pooler_num_attention_heads\": 12,\n",
572
+ " \"pooler_num_fc_layers\": 3,\n",
573
+ " \"pooler_size_per_head\": 128,\n",
574
+ " \"pooler_type\": \"first_token_transform\",\n",
575
+ " \"position_embedding_type\": \"absolute\",\n",
576
+ " \"torch_dtype\": \"float32\",\n",
577
+ " \"transformers_version\": \"4.36.0.dev0\",\n",
578
+ " \"type_vocab_size\": 2,\n",
579
+ " \"use_cache\": true,\n",
580
+ " \"vocab_size\": 30154\n",
581
+ "}\n",
582
+ "\n",
583
+ "tokenizer_config.json: 100% 314/314 [00:00<00:00, 1.91MB/s]\n",
584
+ "vocab.txt: 100% 202k/202k [00:00<00:00, 5.01MB/s]\n",
585
+ "[INFO|tokenization_utils_base.py:2026] 2023-12-09 12:05:45,575 >> loading file vocab.txt from cache at /root/.cache/huggingface/hub/models--osiria--bert-italian-uncased-question-answering/snapshots/9985f20cde546f78a743c4d4f1efd605519b1c41/vocab.txt\n",
586
+ "[INFO|tokenization_utils_base.py:2026] 2023-12-09 12:05:45,575 >> loading file tokenizer.json from cache at None\n",
587
+ "[INFO|tokenization_utils_base.py:2026] 2023-12-09 12:05:45,575 >> loading file added_tokens.json from cache at None\n",
588
+ "[INFO|tokenization_utils_base.py:2026] 2023-12-09 12:05:45,575 >> loading file special_tokens_map.json from cache at None\n",
589
+ "[INFO|tokenization_utils_base.py:2026] 2023-12-09 12:05:45,575 >> loading file tokenizer_config.json from cache at /root/.cache/huggingface/hub/models--osiria--bert-italian-uncased-question-answering/snapshots/9985f20cde546f78a743c4d4f1efd605519b1c41/tokenizer_config.json\n",
590
+ "pytorch_model.bin: 100% 435M/435M [00:09<00:00, 46.2MB/s]\n",
591
+ "[INFO|modeling_utils.py:3299] 2023-12-09 12:05:55,532 >> loading weights file pytorch_model.bin from cache at /root/.cache/huggingface/hub/models--osiria--bert-italian-uncased-question-answering/snapshots/9985f20cde546f78a743c4d4f1efd605519b1c41/pytorch_model.bin\n",
592
+ "[INFO|modeling_utils.py:4139] 2023-12-09 12:05:56,828 >> All model checkpoint weights were used when initializing BertForQuestionAnswering.\n",
593
+ "\n",
594
+ "[INFO|modeling_utils.py:4147] 2023-12-09 12:05:56,828 >> All the weights of BertForQuestionAnswering were initialized from the model checkpoint at osiria/bert-italian-uncased-question-answering.\n",
595
+ "If your task is similar to the task the model of the checkpoint was trained on, you can already use BertForQuestionAnswering for predictions without further training.\n",
596
+ "Running tokenizer on prediction dataset: 0% 0/7609 [00:00<?, ? examples/s]Caching processed dataset at /root/.cache/huggingface/datasets/squad_it/default/0.1.0/d442bdb4794b4bae227ab19105b76d706ed7cf2ac342e4c9da4a5c36bde19d71/cache-6925a2ff4566b537.arrow\n",
597
+ "12/09/2023 12:05:57 - INFO - datasets.arrow_dataset - Caching processed dataset at /root/.cache/huggingface/datasets/squad_it/default/0.1.0/d442bdb4794b4bae227ab19105b76d706ed7cf2ac342e4c9da4a5c36bde19d71/cache-6925a2ff4566b537.arrow\n",
598
+ "Running tokenizer on prediction dataset: 100% 7609/7609 [00:06<00:00, 1097.66 examples/s]\n",
599
+ "Downloading builder script: 100% 4.53k/4.53k [00:00<00:00, 13.1MB/s]\n",
600
+ "Downloading extra modules: 100% 3.32k/3.32k [00:00<00:00, 12.0MB/s]\n",
601
+ "12/09/2023 12:06:10 - INFO - __main__ - *** Predict ***\n",
602
+ "[INFO|trainer.py:712] 2023-12-09 12:06:10,618 >> The following columns in the test set don't have a corresponding argument in `BertForQuestionAnswering.forward` and have been ignored: example_id, offset_mapping. If example_id, offset_mapping are not expected by `BertForQuestionAnswering.forward`, you can safely ignore this message.\n",
603
+ "[INFO|trainer.py:3154] 2023-12-09 12:06:10,622 >> ***** Running Prediction *****\n",
604
+ "[INFO|trainer.py:3156] 2023-12-09 12:06:10,622 >> Num examples = 7942\n",
605
+ "[INFO|trainer.py:3159] 2023-12-09 12:06:10,622 >> Batch size = 5\n",
606
+ "100% 1588/1589 [03:21<00:00, 7.77it/s]12/09/2023 12:09:44 - INFO - utils_qa - Post-processing 7609 example predictions split into 7942 features.\n",
607
+ "\n",
608
+ " 0% 0/7609 [00:00<?, ?it/s]\u001b[A\n",
609
+ " 0% 36/7609 [00:00<00:21, 353.91it/s]\u001b[A\n",
610
+ " 1% 72/7609 [00:00<00:22, 335.21it/s]\u001b[A\n",
611
+ " 1% 106/7609 [00:00<00:22, 334.52it/s]\u001b[A\n",
612
+ " 2% 142/7609 [00:00<00:21, 340.66it/s]\u001b[A\n",
613
+ " 2% 177/7609 [00:00<00:22, 326.50it/s]\u001b[A\n",
614
+ " 3% 210/7609 [00:00<00:28, 264.19it/s]\u001b[A\n",
615
+ " 3% 239/7609 [00:00<00:30, 240.38it/s]\u001b[A\n",
616
+ " 3% 265/7609 [00:00<00:32, 224.41it/s]\u001b[A\n",
617
+ " 4% 289/7609 [00:01<00:34, 214.15it/s]\u001b[A\n",
618
+ " 4% 311/7609 [00:01<00:35, 203.86it/s]\u001b[A\n",
619
+ " 4% 332/7609 [00:01<00:36, 197.54it/s]\u001b[A\n",
620
+ " 5% 352/7609 [00:01<00:39, 181.75it/s]\u001b[A\n",
621
+ " 5% 371/7609 [00:01<00:39, 182.05it/s]\u001b[A\n",
622
+ " 5% 390/7609 [00:01<00:39, 181.22it/s]\u001b[A\n",
623
+ " 5% 409/7609 [00:01<00:40, 176.48it/s]\u001b[A\n",
624
+ " 6% 429/7609 [00:01<00:39, 181.53it/s]\u001b[A\n",
625
+ " 6% 449/7609 [00:02<00:38, 184.78it/s]\u001b[A\n",
626
+ " 6% 469/7609 [00:02<00:37, 188.25it/s]\u001b[A\n",
627
+ " 6% 488/7609 [00:02<00:39, 181.21it/s]\u001b[A\n",
628
+ " 7% 507/7609 [00:02<00:39, 181.92it/s]\u001b[A\n",
629
+ " 7% 526/7609 [00:02<00:40, 173.94it/s]\u001b[A\n",
630
+ " 7% 544/7609 [00:02<00:40, 172.42it/s]\u001b[A\n",
631
+ " 7% 562/7609 [00:02<00:40, 174.15it/s]\u001b[A\n",
632
+ " 8% 580/7609 [00:02<00:40, 175.53it/s]\u001b[A\n",
633
+ " 8% 598/7609 [00:02<00:40, 173.96it/s]\u001b[A\n",
634
+ " 8% 617/7609 [00:02<00:39, 178.03it/s]\u001b[A\n",
635
+ " 8% 636/7609 [00:03<00:38, 179.92it/s]\u001b[A\n",
636
+ " 9% 655/7609 [00:03<00:38, 181.75it/s]\u001b[A\n",
637
+ " 9% 674/7609 [00:03<00:37, 184.16it/s]\u001b[A\n",
638
+ " 9% 694/7609 [00:03<00:36, 187.49it/s]\u001b[A\n",
639
+ " 9% 715/7609 [00:03<00:35, 191.54it/s]\u001b[A\n",
640
+ " 10% 736/7609 [00:03<00:35, 195.03it/s]\u001b[A\n",
641
+ " 10% 756/7609 [00:03<00:37, 184.08it/s]\u001b[A\n",
642
+ " 10% 776/7609 [00:03<00:36, 188.45it/s]\u001b[A\n",
643
+ " 10% 795/7609 [00:03<00:36, 187.94it/s]\u001b[A\n",
644
+ " 11% 814/7609 [00:04<00:37, 181.65it/s]\u001b[A\n",
645
+ " 11% 833/7609 [00:04<00:38, 176.72it/s]\u001b[A\n",
646
+ " 11% 853/7609 [00:04<00:36, 182.82it/s]\u001b[A\n",
647
+ " 11% 872/7609 [00:04<00:37, 180.40it/s]\u001b[A\n",
648
+ " 12% 892/7609 [00:04<00:36, 184.01it/s]\u001b[A\n",
649
+ " 12% 925/7609 [00:04<00:29, 224.20it/s]\u001b[A\n",
650
+ " 13% 957/7609 [00:04<00:26, 250.16it/s]\u001b[A\n",
651
+ " 13% 992/7609 [00:04<00:23, 278.59it/s]\u001b[A\n",
652
+ " 13% 1025/7609 [00:04<00:22, 289.63it/s]\u001b[A\n",
653
+ " 14% 1058/7609 [00:04<00:21, 301.33it/s]\u001b[A\n",
654
+ " 14% 1094/7609 [00:05<00:20, 318.24it/s]\u001b[A\n",
655
+ " 15% 1126/7609 [00:05<00:20, 318.50it/s]\u001b[A\n",
656
+ " 15% 1158/7609 [00:05<00:20, 314.24it/s]\u001b[A\n",
657
+ " 16% 1190/7609 [00:05<00:20, 312.70it/s]\u001b[A\n",
658
+ " 16% 1225/7609 [00:05<00:19, 323.39it/s]\u001b[A\n",
659
+ " 17% 1260/7609 [00:05<00:19, 328.63it/s]\u001b[A\n",
660
+ " 17% 1295/7609 [00:05<00:19, 332.29it/s]\u001b[A\n",
661
+ " 18% 1332/7609 [00:05<00:18, 341.07it/s]\u001b[A\n",
662
+ " 18% 1367/7609 [00:05<00:18, 332.86it/s]\u001b[A\n",
663
+ " 18% 1401/7609 [00:06<00:18, 332.75it/s]\u001b[A\n",
664
+ " 19% 1435/7609 [00:06<00:18, 334.84it/s]\u001b[A\n",
665
+ " 19% 1469/7609 [00:06<00:18, 326.25it/s]\u001b[A\n",
666
+ " 20% 1502/7609 [00:06<00:19, 307.09it/s]\u001b[A\n",
667
+ " 20% 1533/7609 [00:06<00:19, 304.61it/s]\u001b[A\n",
668
+ " 21% 1564/7609 [00:06<00:19, 305.24it/s]\u001b[A\n",
669
+ " 21% 1595/7609 [00:06<00:19, 301.70it/s]\u001b[A\n",
670
+ " 21% 1630/7609 [00:06<00:19, 313.16it/s]\u001b[A\n",
671
+ " 22% 1662/7609 [00:06<00:19, 312.74it/s]\u001b[A\n",
672
+ " 22% 1694/7609 [00:06<00:18, 313.52it/s]\u001b[A\n",
673
+ " 23% 1727/7609 [00:07<00:18, 315.66it/s]\u001b[A\n",
674
+ " 23% 1759/7609 [00:07<00:18, 315.35it/s]\u001b[A\n",
675
+ " 24% 1791/7609 [00:07<00:18, 309.63it/s]\u001b[A\n",
676
+ " 24% 1825/7609 [00:07<00:18, 316.55it/s]\u001b[A\n",
677
+ " 24% 1858/7609 [00:07<00:18, 317.59it/s]\u001b[A\n",
678
+ " 25% 1892/7609 [00:07<00:17, 324.05it/s]\u001b[A\n",
679
+ " 25% 1926/7609 [00:07<00:17, 327.30it/s]\u001b[A\n",
680
+ " 26% 1959/7609 [00:07<00:18, 300.92it/s]\u001b[A\n",
681
+ " 26% 1990/7609 [00:07<00:20, 276.81it/s]\u001b[A\n",
682
+ " 27% 2019/7609 [00:08<00:23, 241.44it/s]\u001b[A\n",
683
+ " 27% 2045/7609 [00:08<00:22, 242.53it/s]\u001b[A\n",
684
+ " 27% 2071/7609 [00:08<00:25, 213.83it/s]\u001b[A\n",
685
+ " 28% 2094/7609 [00:08<00:30, 181.41it/s]\u001b[A\n",
686
+ " 28% 2114/7609 [00:08<00:31, 174.24it/s]\u001b[A\n",
687
+ " 28% 2150/7609 [00:08<00:25, 216.92it/s]\u001b[A\n",
688
+ " 29% 2185/7609 [00:08<00:21, 248.56it/s]\u001b[A\n",
689
+ " 29% 2215/7609 [00:08<00:20, 260.84it/s]\u001b[A\n",
690
+ " 30% 2246/7609 [00:09<00:19, 273.74it/s]\u001b[A\n",
691
+ " 30% 2275/7609 [00:09<00:19, 271.77it/s]\u001b[A\n",
692
+ " 30% 2310/7609 [00:09<00:18, 292.22it/s]\u001b[A\n",
693
+ " 31% 2345/7609 [00:09<00:17, 306.23it/s]\u001b[A\n",
694
+ " 31% 2378/7609 [00:09<00:16, 312.21it/s]\u001b[A\n",
695
+ " 32% 2413/7609 [00:09<00:16, 323.02it/s]\u001b[A\n",
696
+ " 32% 2450/7609 [00:09<00:15, 334.26it/s]\u001b[A\n",
697
+ " 33% 2484/7609 [00:09<00:16, 308.35it/s]\u001b[A\n",
698
+ " 33% 2519/7609 [00:09<00:15, 318.95it/s]\u001b[A\n",
699
+ " 34% 2552/7609 [00:10<00:15, 316.10it/s]\u001b[A\n",
700
+ " 34% 2584/7609 [00:10<00:15, 317.13it/s]\u001b[A\n",
701
+ " 34% 2616/7609 [00:10<00:15, 315.48it/s]\u001b[A\n",
702
+ " 35% 2650/7609 [00:10<00:15, 319.91it/s]\u001b[A\n",
703
+ " 35% 2683/7609 [00:10<00:15, 308.13it/s]\u001b[A\n",
704
+ " 36% 2718/7609 [00:10<00:15, 318.76it/s]\u001b[A\n",
705
+ " 36% 2751/7609 [00:10<00:15, 316.44it/s]\u001b[A\n",
706
+ " 37% 2785/7609 [00:10<00:14, 322.07it/s]\u001b[A\n",
707
+ " 37% 2821/7609 [00:10<00:14, 330.81it/s]\u001b[A\n",
708
+ " 38% 2855/7609 [00:10<00:14, 332.27it/s]\u001b[A\n",
709
+ " 38% 2889/7609 [00:11<00:14, 326.06it/s]\u001b[A\n",
710
+ " 38% 2925/7609 [00:11<00:14, 333.63it/s]\u001b[A\n",
711
+ " 39% 2960/7609 [00:11<00:13, 338.35it/s]\u001b[A\n",
712
+ " 39% 2994/7609 [00:11<00:13, 338.15it/s]\u001b[A\n",
713
+ " 40% 3030/7609 [00:11<00:13, 342.22it/s]\u001b[A\n",
714
+ " 40% 3065/7609 [00:11<00:13, 331.00it/s]\u001b[A\n",
715
+ " 41% 3099/7609 [00:11<00:14, 321.65it/s]\u001b[A\n",
716
+ " 41% 3132/7609 [00:11<00:14, 308.44it/s]\u001b[A\n",
717
+ " 42% 3166/7609 [00:11<00:14, 315.10it/s]\u001b[A\n",
718
+ " 42% 3198/7609 [00:12<00:15, 280.38it/s]\u001b[A\n",
719
+ " 42% 3227/7609 [00:12<00:15, 282.33it/s]\u001b[A\n",
720
+ " 43% 3261/7609 [00:12<00:14, 297.96it/s]\u001b[A\n",
721
+ " 43% 3295/7609 [00:12<00:13, 308.36it/s]\u001b[A\n",
722
+ " 44% 3327/7609 [00:12<00:14, 305.35it/s]\u001b[A\n",
723
+ " 44% 3361/7609 [00:12<00:13, 313.96it/s]\u001b[A\n",
724
+ " 45% 3396/7609 [00:12<00:13, 322.39it/s]\u001b[A\n",
725
+ " 45% 3431/7609 [00:12<00:12, 329.14it/s]\u001b[A\n",
726
+ " 46% 3465/7609 [00:12<00:12, 328.62it/s]\u001b[A\n",
727
+ " 46% 3500/7609 [00:12<00:12, 332.75it/s]\u001b[A\n",
728
+ " 46% 3534/7609 [00:13<00:12, 322.96it/s]\u001b[A\n",
729
+ " 47% 3568/7609 [00:13<00:12, 326.34it/s]\u001b[A\n",
730
+ " 47% 3604/7609 [00:13<00:12, 333.34it/s]\u001b[A\n",
731
+ " 48% 3638/7609 [00:13<00:12, 325.06it/s]\u001b[A\n",
732
+ " 48% 3673/7609 [00:13<00:11, 330.69it/s]\u001b[A\n",
733
+ " 49% 3709/7609 [00:13<00:11, 337.37it/s]\u001b[A\n",
734
+ " 49% 3744/7609 [00:13<00:11, 338.92it/s]\u001b[A\n",
735
+ " 50% 3778/7609 [00:13<00:11, 331.47it/s]\u001b[A\n",
736
+ " 50% 3813/7609 [00:13<00:11, 335.48it/s]\u001b[A\n",
737
+ " 51% 3851/7609 [00:14<00:10, 346.91it/s]\u001b[A\n",
738
+ " 51% 3886/7609 [00:14<00:11, 333.45it/s]\u001b[A\n",
739
+ " 52% 3923/7609 [00:14<00:10, 342.93it/s]\u001b[A\n",
740
+ " 52% 3959/7609 [00:14<00:10, 345.66it/s]\u001b[A\n",
741
+ " 52% 3994/7609 [00:14<00:10, 338.59it/s]\u001b[A\n",
742
+ " 53% 4028/7609 [00:14<00:12, 279.93it/s]\u001b[A\n",
743
+ " 53% 4058/7609 [00:14<00:15, 236.20it/s]\u001b[A\n",
744
+ " 54% 4084/7609 [00:14<00:15, 222.15it/s]\u001b[A\n",
745
+ " 54% 4108/7609 [00:15<00:17, 202.34it/s]\u001b[A\n",
746
+ " 54% 4130/7609 [00:15<00:17, 198.29it/s]\u001b[A\n",
747
+ " 55% 4151/7609 [00:15<00:17, 194.98it/s]\u001b[A\n",
748
+ " 55% 4171/7609 [00:15<00:17, 191.23it/s]\u001b[A\n",
749
+ " 55% 4191/7609 [00:15<00:18, 189.36it/s]\u001b[A\n",
750
+ " 55% 4211/7609 [00:15<00:18, 183.33it/s]\u001b[A\n",
751
+ " 56% 4231/7609 [00:15<00:18, 185.82it/s]\u001b[A\n",
752
+ " 56% 4251/7609 [00:15<00:17, 187.36it/s]\u001b[A\n",
753
+ " 56% 4270/7609 [00:15<00:17, 186.19it/s]\u001b[A\n",
754
+ " 56% 4291/7609 [00:16<00:17, 190.36it/s]\u001b[A\n",
755
+ " 57% 4311/7609 [00:16<00:17, 188.48it/s]\u001b[A\n",
756
+ " 57% 4330/7609 [00:16<00:18, 180.99it/s]\u001b[A\n",
757
+ " 57% 4350/7609 [00:16<00:17, 184.40it/s]\u001b[A\n",
758
+ " 57% 4369/7609 [00:16<00:17, 183.24it/s]\u001b[A\n",
759
+ " 58% 4388/7609 [00:16<00:17, 181.46it/s]\u001b[A\n",
760
+ " 58% 4407/7609 [00:16<00:17, 183.63it/s]\u001b[A\n",
761
+ " 58% 4426/7609 [00:16<00:17, 185.30it/s]\u001b[A\n",
762
+ " 58% 4445/7609 [00:16<00:19, 160.41it/s]\u001b[A\n",
763
+ " 59% 4465/7609 [00:17<00:18, 169.32it/s]\u001b[A\n",
764
+ " 59% 4486/7609 [00:17<00:17, 178.42it/s]\u001b[A\n",
765
+ " 59% 4505/7609 [00:17<00:17, 180.33it/s]\u001b[A\n",
766
+ " 59% 4525/7609 [00:17<00:16, 184.63it/s]\u001b[A\n",
767
+ " 60% 4544/7609 [00:17<00:16, 185.88it/s]\u001b[A\n",
768
+ " 60% 4563/7609 [00:17<00:16, 185.05it/s]\u001b[A\n",
769
+ " 60% 4583/7609 [00:17<00:16, 189.06it/s]\u001b[A\n",
770
+ " 60% 4603/7609 [00:17<00:15, 188.81it/s]\u001b[A\n",
771
+ " 61% 4623/7609 [00:17<00:15, 191.66it/s]\u001b[A\n",
772
+ " 61% 4644/7609 [00:18<00:15, 196.11it/s]\u001b[A\n",
773
+ " 61% 4664/7609 [00:18<00:15, 196.15it/s]\u001b[A\n",
774
+ " 62% 4684/7609 [00:18<00:15, 190.05it/s]\u001b[A\n",
775
+ " 62% 4707/7609 [00:18<00:14, 200.63it/s]\u001b[A\n",
776
+ " 62% 4745/7609 [00:18<00:11, 252.10it/s]\u001b[A\n",
777
+ " 63% 4778/7609 [00:18<00:10, 273.34it/s]\u001b[A\n",
778
+ " 63% 4815/7609 [00:18<00:09, 300.59it/s]\u001b[A\n",
779
+ " 64% 4851/7609 [00:18<00:08, 316.05it/s]\u001b[A\n",
780
+ " 64% 4884/7609 [00:18<00:08, 318.15it/s]\u001b[A\n",
781
+ " 65% 4920/7609 [00:18<00:08, 329.13it/s]\u001b[A\n",
782
+ " 65% 4953/7609 [00:19<00:08, 315.36it/s]\u001b[A\n",
783
+ " 66% 4988/7609 [00:19<00:08, 324.15it/s]\u001b[A\n",
784
+ " 66% 5022/7609 [00:19<00:07, 326.75it/s]\u001b[A\n",
785
+ " 66% 5055/7609 [00:19<00:07, 327.37it/s]\u001b[A\n",
786
+ " 67% 5092/7609 [00:19<00:07, 338.33it/s]\u001b[A\n",
787
+ " 67% 5129/7609 [00:19<00:07, 345.56it/s]\u001b[A\n",
788
+ " 68% 5164/7609 [00:19<00:07, 341.55it/s]\u001b[A\n",
789
+ " 68% 5200/7609 [00:19<00:06, 345.15it/s]\u001b[A\n",
790
+ " 69% 5236/7609 [00:19<00:06, 349.05it/s]\u001b[A\n",
791
+ " 69% 5271/7609 [00:19<00:06, 346.18it/s]\u001b[A\n",
792
+ " 70% 5308/7609 [00:20<00:06, 350.46it/s]\u001b[A\n",
793
+ " 70% 5344/7609 [00:20<00:06, 347.36it/s]\u001b[A\n",
794
+ " 71% 5379/7609 [00:20<00:06, 334.94it/s]\u001b[A\n",
795
+ " 71% 5415/7609 [00:20<00:06, 339.89it/s]\u001b[A\n",
796
+ " 72% 5451/7609 [00:20<00:06, 344.04it/s]\u001b[A\n",
797
+ " 72% 5486/7609 [00:20<00:06, 343.75it/s]\u001b[A\n",
798
+ " 73% 5522/7609 [00:20<00:06, 345.75it/s]\u001b[A\n",
799
+ " 73% 5557/7609 [00:20<00:06, 332.96it/s]\u001b[A\n",
800
+ " 73% 5591/7609 [00:20<00:06, 326.85it/s]\u001b[A\n",
801
+ " 74% 5628/7609 [00:21<00:05, 338.23it/s]\u001b[A\n",
802
+ " 74% 5662/7609 [00:21<00:05, 335.67it/s]\u001b[A\n",
803
+ " 75% 5698/7609 [00:21<00:05, 342.39it/s]\u001b[A\n",
804
+ " 75% 5733/7609 [00:21<00:05, 341.45it/s]\u001b[A\n",
805
+ " 76% 5771/7609 [00:21<00:05, 351.67it/s]\u001b[A\n",
806
+ " 76% 5807/7609 [00:21<00:05, 352.96it/s]\u001b[A\n",
807
+ " 77% 5844/7609 [00:21<00:04, 356.20it/s]\u001b[A\n",
808
+ " 77% 5880/7609 [00:21<00:04, 353.19it/s]\u001b[A\n",
809
+ " 78% 5916/7609 [00:21<00:05, 337.34it/s]\u001b[A\n",
810
+ " 78% 5950/7609 [00:21<00:05, 329.87it/s]\u001b[A\n",
811
+ " 79% 5985/7609 [00:22<00:04, 333.28it/s]\u001b[A\n",
812
+ " 79% 6019/7609 [00:22<00:04, 331.44it/s]\u001b[A\n",
813
+ " 80% 6054/7609 [00:22<00:04, 336.53it/s]\u001b[A\n",
814
+ " 80% 6088/7609 [00:22<00:04, 335.26it/s]\u001b[A\n",
815
+ " 80% 6122/7609 [00:22<00:04, 325.88it/s]\u001b[A\n",
816
+ " 81% 6155/7609 [00:22<00:05, 277.07it/s]\u001b[A\n",
817
+ " 81% 6188/7609 [00:22<00:04, 289.09it/s]\u001b[A\n",
818
+ " 82% 6222/7609 [00:22<00:04, 301.75it/s]\u001b[A\n",
819
+ " 82% 6259/7609 [00:22<00:04, 319.36it/s]\u001b[A\n",
820
+ " 83% 6294/7609 [00:23<00:04, 325.62it/s]\u001b[A\n",
821
+ " 83% 6329/7609 [00:23<00:03, 330.58it/s]\u001b[A\n",
822
+ " 84% 6367/7609 [00:23<00:03, 344.37it/s]\u001b[A\n",
823
+ " 84% 6402/7609 [00:23<00:03, 330.67it/s]\u001b[A\n",
824
+ " 85% 6439/7609 [00:23<00:03, 340.49it/s]\u001b[A\n",
825
+ " 85% 6476/7609 [00:23<00:03, 348.41it/s]\u001b[A\n",
826
+ " 86% 6512/7609 [00:23<00:03, 344.09it/s]\u001b[A\n",
827
+ " 86% 6547/7609 [00:23<00:03, 343.53it/s]\u001b[A\n",
828
+ " 87% 6584/7609 [00:23<00:02, 349.85it/s]\u001b[A\n",
829
+ " 87% 6620/7609 [00:24<00:02, 339.43it/s]\u001b[A\n",
830
+ " 87% 6656/7609 [00:24<00:02, 345.12it/s]\u001b[A\n",
831
+ " 88% 6691/7609 [00:24<00:02, 342.92it/s]\u001b[A\n",
832
+ " 88% 6726/7609 [00:24<00:02, 342.91it/s]\u001b[A\n",
833
+ " 89% 6761/7609 [00:24<00:02, 329.74it/s]\u001b[A\n",
834
+ " 89% 6795/7609 [00:24<00:02, 331.81it/s]\u001b[A\n",
835
+ " 90% 6829/7609 [00:24<00:02, 319.33it/s]\u001b[A\n",
836
+ " 90% 6864/7609 [00:24<00:02, 324.85it/s]\u001b[A\n",
837
+ " 91% 6897/7609 [00:24<00:02, 323.76it/s]\u001b[A\n",
838
+ " 91% 6933/7609 [00:24<00:02, 332.45it/s]\u001b[A\n",
839
+ " 92% 6969/7609 [00:25<00:01, 338.97it/s]\u001b[A\n",
840
+ " 92% 7003/7609 [00:25<00:01, 335.87it/s]\u001b[A\n",
841
+ " 92% 7038/7609 [00:25<00:01, 337.22it/s]\u001b[A\n",
842
+ " 93% 7072/7609 [00:25<00:01, 320.46it/s]\u001b[A\n",
843
+ " 93% 7105/7609 [00:25<00:01, 311.17it/s]\u001b[A\n",
844
+ " 94% 7137/7609 [00:25<00:01, 313.58it/s]\u001b[A\n",
845
+ " 94% 7170/7609 [00:25<00:01, 317.58it/s]\u001b[A\n",
846
+ " 95% 7202/7609 [00:25<00:01, 314.71it/s]\u001b[A\n",
847
+ " 95% 7238/7609 [00:25<00:01, 325.60it/s]\u001b[A\n",
848
+ " 96% 7271/7609 [00:26<00:01, 325.13it/s]\u001b[A\n",
849
+ " 96% 7304/7609 [00:26<00:00, 322.64it/s]\u001b[A\n",
850
+ " 96% 7339/7609 [00:26<00:00, 328.64it/s]\u001b[A\n",
851
+ " 97% 7375/7609 [00:26<00:00, 335.08it/s]\u001b[A\n",
852
+ " 97% 7409/7609 [00:26<00:00, 323.51it/s]\u001b[A\n",
853
+ " 98% 7444/7609 [00:26<00:00, 329.13it/s]\u001b[A\n",
854
+ " 98% 7478/7609 [00:26<00:00, 330.26it/s]\u001b[A\n",
855
+ " 99% 7512/7609 [00:26<00:00, 316.51it/s]\u001b[A\n",
856
+ " 99% 7544/7609 [00:26<00:00, 309.70it/s]\u001b[A\n",
857
+ "100% 7576/7609 [00:26<00:00, 304.19it/s]\u001b[A\n",
858
+ "100% 7609/7609 [00:27<00:00, 281.02it/s]\n",
859
+ "12/09/2023 12:10:11 - INFO - utils_qa - Saving predictions to eval_results/predict_predictions.json.\n",
860
+ "12/09/2023 12:10:11 - INFO - utils_qa - Saving nbest_preds to eval_results/predict_nbest_predictions.json.\n",
861
+ "***** predict metrics *****\n",
862
+ " predict_samples = 7942\n",
863
+ " test_exact_match = 65.6065\n",
864
+ " test_f1 = 77.1676\n",
865
+ " test_runtime = 0:03:24.92\n",
866
+ " test_samples_per_second = 38.757\n",
867
+ " test_steps_per_second = 7.754\n",
868
+ "[INFO|modelcard.py:452] 2023-12-09 12:10:15,872 >> Dropping the following result as it does not have all the necessary fields:\n",
869
+ "{'task': {'name': 'Question Answering', 'type': 'question-answering'}, 'dataset': {'name': 'squad_it', 'type': 'squad_it'}}\n",
870
+ "100% 1589/1589 [04:02<00:00, 6.56it/s]\n"
871
+ ]
872
+ }
873
+ ]
874
+ },
875
+ {
876
+ "cell_type": "code",
877
+ "source": [],
878
+ "metadata": {
879
+ "id": "rMyhUFpnItIQ"
880
+ },
881
+ "execution_count": null,
882
+ "outputs": []
883
+ }
884
+ ]
885
+ }