Hugging Face's logo

Spaces:
Harveenchadha
/
en_to_indic_translation
Runtime error

App Files Community
en_to_indic_translation
File size: 40,089 Bytes 
e50fe35
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
 
{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "view-in-github"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/gowtham1997/indicTrans-1/blob/main/indicTrans_python_interface.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "CjfzxXZLHed_",
        "outputId": "69a66b95-41b2-4413-82d1-0caacbddb3f3"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Cloning into 'indicTrans-1'...\n",
            "remote: Enumerating objects: 486, done.\u001b[K\n",
            "remote: Counting objects: 100% (189/189), done.\u001b[K\n",
            "remote: Compressing objects: 100% (67/67), done.\u001b[K\n",
            "remote: Total 486 (delta 154), reused 134 (delta 121), pack-reused 297\u001b[K\n",
            "Receiving objects: 100% (486/486), 1.48 MiB | 17.61 MiB/s, done.\n",
            "Resolving deltas: 100% (281/281), done.\n",
            "/content/indicTrans\n",
            "Cloning into 'indic_nlp_library'...\n",
            "remote: Enumerating objects: 1325, done.\u001b[K\n",
            "remote: Counting objects: 100% (147/147), done.\u001b[K\n",
            "remote: Compressing objects: 100% (103/103), done.\u001b[K\n",
            "remote: Total 1325 (delta 84), reused 89 (delta 41), pack-reused 1178\u001b[K\n",
            "Receiving objects: 100% (1325/1325), 9.57 MiB | 13.55 MiB/s, done.\n",
            "Resolving deltas: 100% (688/688), done.\n",
            "Cloning into 'indic_nlp_resources'...\n",
            "remote: Enumerating objects: 133, done.\u001b[K\n",
            "remote: Counting objects: 100% (7/7), done.\u001b[K\n",
            "remote: Compressing objects: 100% (7/7), done.\u001b[K\n",
            "remote: Total 133 (delta 0), reused 2 (delta 0), pack-reused 126\u001b[K\n",
            "Receiving objects: 100% (133/133), 149.77 MiB | 33.48 MiB/s, done.\n",
            "Resolving deltas: 100% (51/51), done.\n",
            "Checking out files: 100% (28/28), done.\n",
            "Cloning into 'subword-nmt'...\n",
            "remote: Enumerating objects: 580, done.\u001b[K\n",
            "remote: Counting objects: 100% (4/4), done.\u001b[K\n",
            "remote: Compressing objects: 100% (4/4), done.\u001b[K\n",
            "remote: Total 580 (delta 0), reused 1 (delta 0), pack-reused 576\u001b[K\n",
            "Receiving objects: 100% (580/580), 237.41 KiB | 18.26 MiB/s, done.\n",
            "Resolving deltas: 100% (349/349), done.\n",
            "/content\n"
          ]
        }
      ],
      "source": [
        "# clone the repo for running evaluation\n",
        "!git clone https://github.com/AI4Bharat/indicTrans.git\n",
        "%cd indicTrans\n",
        "# clone requirements repositories\n",
        "!git clone https://github.com/anoopkunchukuttan/indic_nlp_library.git\n",
        "!git clone https://github.com/anoopkunchukuttan/indic_nlp_resources.git\n",
        "!git clone https://github.com/rsennrich/subword-nmt.git\n",
        "%cd .."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "IeYW2BJhlJvx",
        "outputId": "3357bc85-44d8-43b0-8c64-eef9f18be716"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Collecting sacremoses\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/75/ee/67241dc87f266093c533a2d4d3d69438e57d7a90abb216fa076e7d475d4a/sacremoses-0.0.45-py3-none-any.whl (895kB)\n",
            "\r\u001b[K     |โ–                               | 10kB 14.0MB/s eta 0:00:01\r\u001b[K     |โ–Š                               | 20kB 18.8MB/s eta 0:00:01\r\u001b[K     |โ–ˆ                               | 30kB 22.5MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–Œ                              | 40kB 25.7MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–‰                              | 51kB 27.6MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–                             | 61kB 29.2MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–‹                             | 71kB 27.3MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆ                             | 81kB 27.7MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–Ž                            | 92kB 28.8MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–‹                            | 102kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆ                            | 112kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–                           | 122kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–Š                           | 133kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–                          | 143kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–Œ                          | 153kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‰                          | 163kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–Ž                         | 174kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‹                         | 184kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ                         | 194kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–Ž                        | 204kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–Š                        | 215kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ                        | 225kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–                       | 235kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‰                       | 245kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–                      | 256kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–Œ                      | 266kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‰                      | 276kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–Ž                     | 286kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‹                     | 296kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ                     | 307kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–                    | 317kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–Š                    | 327kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ                    | 337kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–Œ                   | 348kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‰                   | 358kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–                  | 368kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–Œ                  | 378kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ                  | 389kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–Ž                 | 399kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‹                 | 409kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ                 | 419kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–                | 430kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–Š                | 440kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ                | 450kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–Œ               | 460kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‰               | 471kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–              | 481kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‹              | 491kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ              | 501kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–Ž             | 512kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–Š             | 522kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ             | 532kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–            | 542kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–Š            | 552kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–           | 563kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–Œ           | 573kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‰           | 583kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–Ž          | 593kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‹          | 604kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ          | 614kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–Ž         | 624kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–Š         | 634kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ         | 645kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–        | 655kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‰        | 665kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–       | 675kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–Œ       | 686kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ       | 696kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–Ž      | 706kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‹      | 716kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ      | 727kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–     | 737kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–Š     | 747kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ     | 757kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–Œ    | 768kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‰    | 778kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–   | 788kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–Œ   | 798kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ   | 808kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–Ž  | 819kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‹  | 829kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ  | 839kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ– | 849kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–Š | 860kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–| 870kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–Œ| 880kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‰| 890kB 29.9MB/s eta 0:00:01\r\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 901kB 29.9MB/s \n",
            "\u001b[?25hRequirement already satisfied: pandas in /usr/local/lib/python3.7/dist-packages (1.1.5)\n",
            "Collecting mock\n",
            "  Downloading https://files.pythonhosted.org/packages/5c/03/b7e605db4a57c0f6fba744b11ef3ddf4ddebcada35022927a2b5fc623fdf/mock-4.0.3-py3-none-any.whl\n",
            "Collecting sacrebleu\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/7e/57/0c7ca4e31a126189dab99c19951910bd081dea5bbd25f24b77107750eae7/sacrebleu-1.5.1-py3-none-any.whl (54kB)\n",
            "\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 61kB 7.5MB/s \n",
            "\u001b[?25hCollecting tensorboardX\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/42/36/2b147652c40c3a858efa0afbf7b8236fae968e88ff530511a4cfa299a506/tensorboardX-2.3-py2.py3-none-any.whl (124kB)\n",
            "\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 133kB 47.5MB/s \n",
            "\u001b[?25hRequirement already satisfied: pyarrow in /usr/local/lib/python3.7/dist-packages (3.0.0)\n",
            "Collecting indic-nlp-library\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/84/d4/495bb43b88a2a6d04b09c29fc5115f24872af74cd8317fe84026abd4ddb1/indic_nlp_library-0.81-py3-none-any.whl (40kB)\n",
            "\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 40kB 5.2MB/s \n",
            "\u001b[?25hRequirement already satisfied: tqdm in /usr/local/lib/python3.7/dist-packages (from sacremoses) (4.41.1)\n",
            "Requirement already satisfied: six in /usr/local/lib/python3.7/dist-packages (from sacremoses) (1.15.0)\n",
            "Requirement already satisfied: joblib in /usr/local/lib/python3.7/dist-packages (from sacremoses) (1.0.1)\n",
            "Requirement already satisfied: regex in /usr/local/lib/python3.7/dist-packages (from sacremoses) (2019.12.20)\n",
            "Requirement already satisfied: click in /usr/local/lib/python3.7/dist-packages (from sacremoses) (7.1.2)\n",
            "Requirement already satisfied: numpy>=1.15.4 in /usr/local/lib/python3.7/dist-packages (from pandas) (1.19.5)\n",
            "Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.7/dist-packages (from pandas) (2.8.1)\n",
            "Requirement already satisfied: pytz>=2017.2 in /usr/local/lib/python3.7/dist-packages (from pandas) (2018.9)\n",
            "Collecting portalocker==2.0.0\n",
            "  Downloading https://files.pythonhosted.org/packages/89/a6/3814b7107e0788040870e8825eebf214d72166adf656ba7d4bf14759a06a/portalocker-2.0.0-py2.py3-none-any.whl\n",
            "Requirement already satisfied: protobuf>=3.8.0 in /usr/local/lib/python3.7/dist-packages (from tensorboardX) (3.12.4)\n",
            "Collecting sphinx-rtd-theme\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/ac/24/2475e8f83519b54b2148d4a56eb1111f9cec630d088c3ffc214492c12107/sphinx_rtd_theme-0.5.2-py2.py3-none-any.whl (9.1MB)\n",
            "\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 9.2MB 42.0MB/s \n",
            "\u001b[?25hCollecting morfessor\n",
            "  Downloading https://files.pythonhosted.org/packages/39/e6/7afea30be2ee4d29ce9de0fa53acbb033163615f849515c0b1956ad074ee/Morfessor-2.0.6-py3-none-any.whl\n",
            "Collecting sphinx-argparse\n",
            "  Downloading https://files.pythonhosted.org/packages/06/2b/dfad6a1831c3aeeae25d8d3d417224684befbf45e10c7f2141631616a6ed/sphinx-argparse-0.2.5.tar.gz\n",
            "Requirement already satisfied: setuptools in /usr/local/lib/python3.7/dist-packages (from protobuf>=3.8.0->tensorboardX) (57.0.0)\n",
            "Requirement already satisfied: sphinx in /usr/local/lib/python3.7/dist-packages (from sphinx-rtd-theme->indic-nlp-library) (1.8.5)\n",
            "Collecting docutils<0.17\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/81/44/8a15e45ffa96e6cf82956dd8d7af9e666357e16b0d93b253903475ee947f/docutils-0.16-py2.py3-none-any.whl (548kB)\n",
            "\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 552kB 31.5MB/s \n",
            "\u001b[?25hRequirement already satisfied: sphinxcontrib-websupport in /usr/local/lib/python3.7/dist-packages (from sphinx->sphinx-rtd-theme->indic-nlp-library) (1.2.4)\n",
            "Requirement already satisfied: snowballstemmer>=1.1 in /usr/local/lib/python3.7/dist-packages (from sphinx->sphinx-rtd-theme->indic-nlp-library) (2.1.0)\n",
            "Requirement already satisfied: Jinja2>=2.3 in /usr/local/lib/python3.7/dist-packages (from sphinx->sphinx-rtd-theme->indic-nlp-library) (2.11.3)\n",
            "Requirement already satisfied: packaging in /usr/local/lib/python3.7/dist-packages (from sphinx->sphinx-rtd-theme->indic-nlp-library) (20.9)\n",
            "Requirement already satisfied: alabaster<0.8,>=0.7 in /usr/local/lib/python3.7/dist-packages (from sphinx->sphinx-rtd-theme->indic-nlp-library) (0.7.12)\n",
            "Requirement already satisfied: imagesize in /usr/local/lib/python3.7/dist-packages (from sphinx->sphinx-rtd-theme->indic-nlp-library) (1.2.0)\n",
            "Requirement already satisfied: Pygments>=2.0 in /usr/local/lib/python3.7/dist-packages (from sphinx->sphinx-rtd-theme->indic-nlp-library) (2.6.1)\n",
            "Requirement already satisfied: requests>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from sphinx->sphinx-rtd-theme->indic-nlp-library) (2.23.0)\n",
            "Requirement already satisfied: babel!=2.0,>=1.3 in /usr/local/lib/python3.7/dist-packages (from sphinx->sphinx-rtd-theme->indic-nlp-library) (2.9.1)\n",
            "Requirement already satisfied: sphinxcontrib-serializinghtml in /usr/local/lib/python3.7/dist-packages (from sphinxcontrib-websupport->sphinx->sphinx-rtd-theme->indic-nlp-library) (1.1.5)\n",
            "Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python3.7/dist-packages (from Jinja2>=2.3->sphinx->sphinx-rtd-theme->indic-nlp-library) (2.0.1)\n",
            "Requirement already satisfied: pyparsing>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from packaging->sphinx->sphinx-rtd-theme->indic-nlp-library) (2.4.7)\n",
            "Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests>=2.0.0->sphinx->sphinx-rtd-theme->indic-nlp-library) (2.10)\n",
            "Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests>=2.0.0->sphinx->sphinx-rtd-theme->indic-nlp-library) (3.0.4)\n",
            "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests>=2.0.0->sphinx->sphinx-rtd-theme->indic-nlp-library) (1.24.3)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests>=2.0.0->sphinx->sphinx-rtd-theme->indic-nlp-library) (2021.5.30)\n",
            "Building wheels for collected packages: sphinx-argparse\n",
            "  Building wheel for sphinx-argparse (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for sphinx-argparse: filename=sphinx_argparse-0.2.5-cp37-none-any.whl size=11552 sha256=d8cbdca000085e2e2c122c305bb21aa76a9600012ded8e06c300e03d1c4d1e32\n",
            "  Stored in directory: /root/.cache/pip/wheels/2a/18/1b/4990a1859da4edc77ab312bc2986c08d2733fb5713d06e44f5\n",
            "Successfully built sphinx-argparse\n",
            "\u001b[31mERROR: datascience 0.10.6 has requirement folium==0.2.1, but you'll have folium 0.8.3 which is incompatible.\u001b[0m\n",
            "Installing collected packages: sacremoses, mock, portalocker, sacrebleu, tensorboardX, docutils, sphinx-rtd-theme, morfessor, sphinx-argparse, indic-nlp-library\n",
            "  Found existing installation: docutils 0.17.1\n",
            "    Uninstalling docutils-0.17.1:\n",
            "      Successfully uninstalled docutils-0.17.1\n",
            "Successfully installed docutils-0.16 indic-nlp-library-0.81 mock-4.0.3 morfessor-2.0.6 portalocker-2.0.0 sacrebleu-1.5.1 sacremoses-0.0.45 sphinx-argparse-0.2.5 sphinx-rtd-theme-0.5.2 tensorboardX-2.3\n",
            "Collecting mosestokenizer\n",
            "  Downloading https://files.pythonhosted.org/packages/4b/b3/c0af235b16c4f44a2828ef017f7947d1262b2646e440f85c6a2ff26a8c6f/mosestokenizer-1.1.0.tar.gz\n",
            "Collecting subword-nmt\n",
            "  Downloading https://files.pythonhosted.org/packages/74/60/6600a7bc09e7ab38bc53a48a20d8cae49b837f93f5842a41fe513a694912/subword_nmt-0.3.7-py2.py3-none-any.whl\n",
            "Requirement already satisfied: docopt in /usr/local/lib/python3.7/dist-packages (from mosestokenizer) (0.6.2)\n",
            "Collecting openfile\n",
            "  Downloading https://files.pythonhosted.org/packages/93/e6/805db6867faacb488b44ba8e0829ef4de151dd0499f3c5da5f4ad11698a7/openfile-0.0.7-py3-none-any.whl\n",
            "Collecting uctools\n",
            "  Downloading https://files.pythonhosted.org/packages/04/cb/70ed842d9a43460eedaa11f7503b4ab6537b43b63f0d854d59d8e150fac1/uctools-1.3.0.tar.gz\n",
            "Collecting toolwrapper\n",
            "  Downloading https://files.pythonhosted.org/packages/41/7b/34bf8fb69426d8a18bcc61081e9d126f4fcd41c3c832072bef39af1602cd/toolwrapper-2.1.0.tar.gz\n",
            "Building wheels for collected packages: mosestokenizer, uctools, toolwrapper\n",
            "  Building wheel for mosestokenizer (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for mosestokenizer: filename=mosestokenizer-1.1.0-cp37-none-any.whl size=49120 sha256=4fc04046040e73bd5d13c606ebbfc65ac38c7d073f7fc0b0e4cc1d4215b595f3\n",
            "  Stored in directory: /root/.cache/pip/wheels/a2/e7/48/48d5e0f9c0cd5def2dfd7cb8543945f906448ed1313de24a29\n",
            "  Building wheel for uctools (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for uctools: filename=uctools-1.3.0-cp37-none-any.whl size=6163 sha256=c5a865107c59f98c4da5d18ddc754fa141ab494574187281de1502561c6a004e\n",
            "  Stored in directory: /root/.cache/pip/wheels/06/b6/8f/935d5bf5bca85d47c6f5ec31641879bba057d336ab36b1e773\n",
            "  Building wheel for toolwrapper (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for toolwrapper: filename=toolwrapper-2.1.0-cp37-none-any.whl size=3356 sha256=41a3e12078d5681e8467701735208d880ba588b0f5dbfb3b99c4e04bc643eccc\n",
            "  Stored in directory: /root/.cache/pip/wheels/84/ea/29/e02f3b855bf19344972092873a1091b329309bbc3d3d0cbaef\n",
            "Successfully built mosestokenizer uctools toolwrapper\n",
            "Installing collected packages: openfile, uctools, toolwrapper, mosestokenizer, subword-nmt\n",
            "Successfully installed mosestokenizer-1.1.0 openfile-0.0.7 subword-nmt-0.3.7 toolwrapper-2.1.0 uctools-1.3.0\n",
            "Cloning into 'fairseq'...\n",
            "remote: Enumerating objects: 28410, done.\u001b[K\n",
            "remote: Counting objects: 100% (229/229), done.\u001b[K\n",
            "remote: Compressing objects: 100% (127/127), done.\u001b[K\n",
            "remote: Total 28410 (delta 114), reused 187 (delta 99), pack-reused 28181\u001b[K\n",
            "Receiving objects: 100% (28410/28410), 11.96 MiB | 24.16 MiB/s, done.\n",
            "Resolving deltas: 100% (21310/21310), done.\n",
            "/content/fairseq\n",
            "Obtaining file:///content/fairseq\n",
            "  Installing build dependencies ... \u001b[?25l\u001b[?25hdone\n",
            "  Getting requirements to build wheel ... \u001b[?25l\u001b[?25hdone\n",
            "  Installing backend dependencies ... \u001b[?25l\u001b[?25hdone\n",
            "    Preparing wheel metadata ... \u001b[?25l\u001b[?25hdone\n",
            "Requirement already satisfied: cffi in /usr/local/lib/python3.7/dist-packages (from fairseq==1.0.0a0+f887152) (1.14.5)\n",
            "Collecting hydra-core<1.1\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/52/e3/fbd70dd0d3ce4d1d75c22d56c0c9f895cfa7ed6587a9ffb821d6812d6a60/hydra_core-1.0.6-py3-none-any.whl (123kB)\n",
            "\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 133kB 11.6MB/s \n",
            "\u001b[?25hRequirement already satisfied: tqdm in /usr/local/lib/python3.7/dist-packages (from fairseq==1.0.0a0+f887152) (4.41.1)\n",
            "Collecting omegaconf<2.1\n",
            "  Downloading https://files.pythonhosted.org/packages/d0/eb/9d63ce09dd8aa85767c65668d5414958ea29648a0eec80a4a7d311ec2684/omegaconf-2.0.6-py3-none-any.whl\n",
            "Requirement already satisfied: regex in /usr/local/lib/python3.7/dist-packages (from fairseq==1.0.0a0+f887152) (2019.12.20)\n",
            "Requirement already satisfied: numpy; python_version >= \"3.7\" in /usr/local/lib/python3.7/dist-packages (from fairseq==1.0.0a0+f887152) (1.19.5)\n",
            "Requirement already satisfied: torch in /usr/local/lib/python3.7/dist-packages (from fairseq==1.0.0a0+f887152) (1.9.0+cu102)\n",
            "Requirement already satisfied: sacrebleu>=1.4.12 in /usr/local/lib/python3.7/dist-packages (from fairseq==1.0.0a0+f887152) (1.5.1)\n",
            "Requirement already satisfied: cython in /usr/local/lib/python3.7/dist-packages (from fairseq==1.0.0a0+f887152) (0.29.23)\n",
            "Requirement already satisfied: pycparser in /usr/local/lib/python3.7/dist-packages (from cffi->fairseq==1.0.0a0+f887152) (2.20)\n",
            "Collecting antlr4-python3-runtime==4.8\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/56/02/789a0bddf9c9b31b14c3e79ec22b9656185a803dc31c15f006f9855ece0d/antlr4-python3-runtime-4.8.tar.gz (112kB)\n",
            "\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 112kB 33.5MB/s \n",
            "\u001b[?25hRequirement already satisfied: importlib-resources; python_version < \"3.9\" in /usr/local/lib/python3.7/dist-packages (from hydra-core<1.1->fairseq==1.0.0a0+f887152) (5.1.4)\n",
            "Collecting PyYAML>=5.1.*\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/7a/a5/393c087efdc78091afa2af9f1378762f9821c9c1d7a22c5753fb5ac5f97a/PyYAML-5.4.1-cp37-cp37m-manylinux1_x86_64.whl (636kB)\n",
            "\u001b[K     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 645kB 30.2MB/s \n",
            "\u001b[?25hRequirement already satisfied: typing-extensions in /usr/local/lib/python3.7/dist-packages (from omegaconf<2.1->fairseq==1.0.0a0+f887152) (3.7.4.3)\n",
            "Requirement already satisfied: portalocker==2.0.0 in /usr/local/lib/python3.7/dist-packages (from sacrebleu>=1.4.12->fairseq==1.0.0a0+f887152) (2.0.0)\n",
            "Requirement already satisfied: zipp>=3.1.0; python_version < \"3.10\" in /usr/local/lib/python3.7/dist-packages (from importlib-resources; python_version < \"3.9\"->hydra-core<1.1->fairseq==1.0.0a0+f887152) (3.4.1)\n",
            "Building wheels for collected packages: antlr4-python3-runtime\n",
            "  Building wheel for antlr4-python3-runtime (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for antlr4-python3-runtime: filename=antlr4_python3_runtime-4.8-cp37-none-any.whl size=141231 sha256=69960f774a6fdb385fed1a63fb02ae50b57299408cfd6fb33be60d686be878b7\n",
            "  Stored in directory: /root/.cache/pip/wheels/e3/e2/fa/b78480b448b8579ddf393bebd3f47ee23aa84c89b6a78285c8\n",
            "Successfully built antlr4-python3-runtime\n",
            "Installing collected packages: antlr4-python3-runtime, PyYAML, omegaconf, hydra-core, fairseq\n",
            "  Found existing installation: PyYAML 3.13\n",
            "    Uninstalling PyYAML-3.13:\n",
            "      Successfully uninstalled PyYAML-3.13\n",
            "  Running setup.py develop for fairseq\n",
            "Successfully installed PyYAML-5.4.1 antlr4-python3-runtime-4.8 fairseq hydra-core-1.0.6 omegaconf-2.0.6\n",
            "/content\n"
          ]
        }
      ],
      "source": [
        "# Install the necessary libraries\n",
        "!pip install sacremoses pandas mock sacrebleu tensorboardX pyarrow indic-nlp-library\n",
        "! pip install mosestokenizer subword-nmt\n",
        "# Install fairseq from source\n",
        "!git clone https://github.com/pytorch/fairseq.git\n",
        "%cd fairseq\n",
        "# !git checkout da9eaba12d82b9bfc1442f0e2c6fc1b895f4d35d\n",
        "!pip install --editable ./\n",
        "\n",
        "%cd .."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "id": "TktUu9NW_PLq"
      },
      "outputs": [],
      "source": [
        "# this step is only required if you are running the code on colab\n",
        "# restart the runtime after running prev cell (to update). See this -> https://stackoverflow.com/questions/57838013/modulenotfounderror-after-successful-pip-install-in-google-colaboratory\n",
        "\n",
        "# this import will not work without restarting runtime\n",
        "from fairseq import checkpoint_utils, distributed_utils, options, tasks, utils"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "E_4JxNdRlPQB",
        "outputId": "82ab5e2f-d560-4f4e-bf3f-f1ca0a8d31b8"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "--2021-06-27 12:43:16--  https://storage.googleapis.com/samanantar-public/V0.2/models/indic-en.zip\n",
            "Resolving storage.googleapis.com (storage.googleapis.com)... 172.217.13.240, 172.217.15.80, 142.251.33.208, ...\n",
            "Connecting to storage.googleapis.com (storage.googleapis.com)|172.217.13.240|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 4551079075 (4.2G) [application/zip]\n",
            "Saving to: โ€˜indic-en.zipโ€™\n",
            "\n",
            "indic-en.zip        100%[===================>]   4.24G  28.8MB/s    in 83s     \n",
            "\n",
            "2021-06-27 12:44:39 (52.1 MB/s) - โ€˜indic-en.zipโ€™ saved [4551079075/4551079075]\n",
            "\n",
            "Archive:  indic-en.zip\n",
            "   creating: indic-en/\n",
            "   creating: indic-en/vocab/\n",
            "  inflating: indic-en/vocab/bpe_codes.32k.SRC  \n",
            "  inflating: indic-en/vocab/vocab.SRC  \n",
            "  inflating: indic-en/vocab/vocab.TGT  \n",
            "  inflating: indic-en/vocab/bpe_codes.32k.TGT  \n",
            "   creating: indic-en/final_bin/\n",
            "  inflating: indic-en/final_bin/dict.TGT.txt  \n",
            "  inflating: indic-en/final_bin/dict.SRC.txt  \n",
            "   creating: indic-en/model/\n",
            "  inflating: indic-en/model/checkpoint_best.pt  \n",
            "/content/indicTrans\n"
          ]
        }
      ],
      "source": [
        "# download the indictrans model\n",
        "\n",
        "\n",
        "# downloading the indic-en model\n",
        "!wget https://storage.googleapis.com/samanantar-public/V0.3/models/indic-en.zip\n",
        "!unzip indic-en.zip\n",
        "\n",
        "# downloading the en-indic model\n",
        "# !wget https://storage.googleapis.com/samanantar-public/V0.3/models/en-indic.zip\n",
        "# !unzip en-indic.zip\n",
        "\n",
        "# # downloading the indic-indic model\n",
        "# !wget https://storage.googleapis.com/samanantar-public/V0.3/models/m2m.zip\n",
        "# !unzip m2m.zip\n",
        "\n",
        "%cd indicTrans"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "yTnWbHqY01-B",
        "outputId": "0d075f51-097b-46ad-aade-407a4437aa62"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Initializing vocab and bpe\n",
            "Initializing model for translation\n"
          ]
        }
      ],
      "source": [
        "from indicTrans.inference.engine import Model\n",
        "\n",
        "indic2en_model = Model(expdir='../indic-en')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "QTp2NOgQ__sB",
        "outputId": "e015a71e-8206-4e1d-cb3e-11ecb4d44f76"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 3/3 [00:00<00:00, 1225.21it/s]\n",
            "/usr/local/lib/python3.7/dist-packages/torch/_tensor.py:575: UserWarning: floor_divide is deprecated, and will be removed in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values.\n",
            "To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor'). (Triggered internally at  /pytorch/aten/src/ATen/native/BinaryOps.cpp:467.)\n",
            "  return torch.floor_divide(self, other)\n"
          ]
        },
        {
          "data": {
            "text/plain": [
              "['He seems to know us.',\n",
              " 'I couldnt find it anywhere.',\n",
              " 'If someone in your neighbourhood develops these symptoms, staying at home can help prevent the spread of the coronavirus infection.']"
            ]
          },
          "execution_count": 11,
          "metadata": {
            "tags": []
          },
          "output_type": "execute_result"
        }
      ],
      "source": [
        "ta_sents = ['เฎ…เฎตเฎฉเฏเฎ•เฏเฎ•เฏ เฎจเฎฎเฏเฎฎเฏˆเฎชเฏ เฎคเฏ†เฎฐเฎฟเฎฏเฏเฎฎเฏ เฎŽเฎฉเฏเฎฑเฏ เฎคเฏ‹เฎฉเฏเฎฑเฏเฎ•เฎฟเฎฑเฎคเฏ',\n",
        "            \"เฎ‡เฎคเฏ เฎŽเฎ™เฏเฎ•เฏ‡ เฎ‡เฎฐเฏเฎ•เฏเฎ•เฏ เฎŽเฎฉเฏเฎฑเฏ เฎŽเฎฉเฏเฎฉเฎพเฎฒเฏ เฎ•เฎฃเฏเฎŸเฏเฎชเฎฟเฎŸเฎฟเฎ•เฏเฎ• เฎฎเฏเฎŸเฎฟเฎฏเฎตเฎฟเฎฒเฏเฎฒเฏˆ.\",\n",
        "            'เฎ‰เฎ™เฏเฎ•เฎณเฏเฎ•เฏเฎ•เฏ เฎ‰เฎ™เฏเฎ•เฎณเฏ เฎ…เฎฐเฏเฎ•เฎฟเฎฒเฏ เฎ‡เฎฐเฏเฎ•เฏเฎ•เฏเฎฎเฏ เฎ’เฎฐเฏเฎตเฎฐเฏเฎ•เฏเฎ•เฏ‹ เฎ‡เฎคเฏเฎคเฎ•เฏˆเฎฏ เฎ…เฎฑเฎฟเฎ•เฏเฎฑเฎฟเฎ•เฎณเฏ เฎคเฏ†เฎฉเฏเฎชเฎŸเฏเฎŸเฎพเฎฒเฏ, เฎตเฏ€เฎŸเฏเฎŸเฎฟเฎฒเฏ‡เฎฏเฏ‡ เฎ‡เฎฐเฏเฎชเฏเฎชเฎคเฏ, เฎ•เฏŠเฎฐเฏ‹เฎฉเฎพ เฎตเฏˆเฎฐเฎธเฏ เฎคเฏŠเฎฑเฏเฎฑเฏ เฎชเฎฟเฎฑเฎฐเฏเฎ•เฏเฎ•เฏ เฎตเฎฐเฎพเฎฎเฎฒเฏ เฎคเฎŸเฏเฎ•เฏเฎ• เฎ‰เฎคเฎตเฏเฎฎเฏ.']\n",
        "\n",
        "\n",
        "indic2en_model.batch_translate(ta_sents, 'ta', 'en')\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 68
        },
        "id": "VFXrCNZGEN7Z",
        "outputId": "f72aad17-1cc0-4774-a7ee-5b3a5d954de3"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 4/4 [00:00<00:00, 1496.76it/s]\n"
          ]
        },
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'The pandemic has resulted in worldwide social and economic disruption. The world is facing the worst recession since the global financial crisis. This led to the postponement or cancellation of sporting, religious, political and cultural events. Due to the fear, there was shortage of supply as more people purchased items like masks, sanitizers etc.'"
            ]
          },
          "execution_count": 13,
          "metadata": {
            "tags": []
          },
          "output_type": "execute_result"
        }
      ],
      "source": [
        "\n",
        "ta_paragraph = \"\"\"เฎ‡เฎคเฏเฎคเฏŠเฎฑเฏเฎฑเฏเฎจเฏ‹เฎฏเฏ เฎ‰เฎฒเฎ•เฎณเฎพเฎตเฎฟเฎฏ เฎšเฎฎเฏ‚เฎ• เฎฎเฎฑเฏเฎฑเฏเฎฎเฏ เฎชเฏŠเฎฐเฏเฎณเฎพเฎคเฎพเฎฐ เฎšเฏ€เฎฐเฏเฎ•เฏเฎฒเฏˆเฎตเฏˆ เฎเฎฑเฏเฎชเฎŸเฏเฎคเฏเฎคเฎฟเฎฏเฏเฎณเฏเฎณเฎคเฏ.เฎ‡เฎคเฎฉเฎพเฎฒเฏ เฎชเฏ†เฎฐเฏเฎฎเฏ เฎชเฏŠเฎฐเฏเฎณเฎพเฎคเฎพเฎฐ เฎฎเฎจเฏเฎคเฎจเฎฟเฎฒเฏˆเฎ•เฏเฎ•เฏเฎชเฏ เฎชเฎฟเฎฉเฏเฎฉเฎฐเฏ เฎ‰เฎฒเฎ•เฎณเฎตเฎฟเฎฒเฏ เฎฎเฎฟเฎ•เฎชเฏเฎชเฏ†เฎฐเฎฟเฎฏ เฎฎเฎจเฏเฎคเฎจเฎฟเฎฒเฏˆ เฎเฎฑเฏเฎชเฎŸเฏเฎŸเฏเฎณเฏเฎณเฎคเฏ. เฎ‡เฎคเฏ เฎตเฎฟเฎณเฏˆเฎฏเฎพเฎŸเฏเฎŸเฏ,เฎฎเฎค, เฎ…เฎฐเฎšเฎฟเฎฏเฎฒเฏ เฎฎเฎฑเฏเฎฑเฏเฎฎเฏ เฎ•เฎฒเฎพเฎšเฏเฎšเฎพเฎฐ เฎจเฎฟเฎ•เฎดเฏเฎตเฏเฎ•เฎณเฏˆ เฎ’เฎคเฏเฎคเฎฟเฎตเฏˆเฎ•เฏเฎ• เฎ…เฎฒเฏเฎฒเฎคเฏ เฎฐเฎคเฏเฎคเฏ เฎšเฏ†เฎฏเฏเฎฏ เฎตเฎดเฎฟเฎตเฎ•เฏเฎคเฏเฎคเฎคเฏ.\n",
        "เฎ…เฎšเฏเฎšเฎฎเฏ เฎ•เฎพเฎฐเฎฃเฎฎเฎพเฎ• เฎฎเฏเฎ•เฎ•เฏเฎ•เฎตเฎšเฎฎเฏ, เฎ•เฎฟเฎฐเฏเฎฎเฎฟเฎจเฎพเฎšเฎฟเฎฉเฎฟ เฎ‰เฎณเฏเฎณเฎฟเฎŸเฏเฎŸ เฎชเฏŠเฎฐเฏเฎŸเฏเฎ•เฎณเฏˆ เฎ…เฎคเฎฟเฎ• เฎจเฎชเฎฐเฏเฎ•เฎณเฏ เฎตเฎพเฎ™เฏเฎ•เฎฟเฎฏเฎคเฎพเฎฒเฏ เฎตเฎฟเฎจเฎฟเฎฏเฏ‹เฎ•เฎชเฏ เฎชเฎฑเฏเฎฑเฎพเฎ•เฏเฎ•เฏเฎฑเฏˆ เฎเฎฑเฏเฎชเฎŸเฏเฎŸเฎคเฏ.\"\"\"\n",
        "\n",
        "indic2en_model.translate_paragraph(ta_paragraph, 'ta', 'en')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Hi_D7s_VIjis"
      },
      "outputs": [],
      "source": []
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "authorship_tag": "ABX9TyM3t8oQYMhBUuq4/Pyhcr0+",
      "collapsed_sections": [],
      "include_colab_link": true,
      "name": "indicTrans_python_interface.ipynb",
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}