reseting training

README.md

@@ -1,68 +1,33 @@
 ---
 tags:
-- generated_from_trainer
-metrics:
-- bleu
-model-index:
-- name: opus-mt-ru-en-finetuned
-  results: []
+- translation
+license: apache-2.0
 ---
 
-<!-- This model card has been generated automatically according to the information the Trainer had access to. You
-should probably proofread and complete it, then remove this comment. -->
+### opus-mt-ru-en
+
+* source languages: ru
+* target languages: en
+*  OPUS readme: [ru-en](https://github.com/Helsinki-NLP/OPUS-MT-train/blob/master/models/ru-en/README.md)
+
+*  dataset: opus
+* model: transformer-align
+* pre-processing: normalization + SentencePiece
+* download original weights: [opus-2020-02-26.zip](https://object.pouta.csc.fi/OPUS-MT-models/ru-en/opus-2020-02-26.zip)
+* test set translations: [opus-2020-02-26.test.txt](https://object.pouta.csc.fi/OPUS-MT-models/ru-en/opus-2020-02-26.test.txt)
+* test set scores: [opus-2020-02-26.eval.txt](https://object.pouta.csc.fi/OPUS-MT-models/ru-en/opus-2020-02-26.eval.txt)
+
+## Benchmarks
+
+| testset               | BLEU  | chr-F |
+|-----------------------|-------|-------|
+| newstest2012.ru.en 	| 34.8 	| 0.603 |
+| newstest2013.ru.en 	| 27.9 	| 0.545 |
+| newstest2014-ruen.ru.en 	| 31.9 	| 0.591 |
+| newstest2015-enru.ru.en 	| 30.4 	| 0.568 |
+| newstest2016-enru.ru.en 	| 30.1 	| 0.565 |
+| newstest2017-enru.ru.en 	| 33.4 	| 0.593 |
+| newstest2018-enru.ru.en 	| 29.6 	| 0.565 |
+| newstest2019-ruen.ru.en 	| 31.4 	| 0.576 |
+| Tatoeba.ru.en 	| 61.1 	| 0.736 |
 
-# opus-mt-ru-en-finetuned
-
-This model was trained from scratch on the None dataset.
-It achieves the following results on the evaluation set:
-- Loss: 1.5751
-- Bleu: 32.4337
-- Gen Len: 26.6242
-
-## Model description
-
-More information needed
-
-## Intended uses & limitations
-
-More information needed
-
-## Training and evaluation data
-
-More information needed
-
-## Training procedure
-
-### Training hyperparameters
-
-The following hyperparameters were used during training:
-- learning_rate: 0.0001
-- train_batch_size: 80
-- eval_batch_size: 40
-- seed: 42
-- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
-- lr_scheduler_type: linear
-- num_epochs: 10
-
-### Training results
-
-| Training Loss | Epoch | Step   | Validation Loss | Bleu    | Gen Len |
-|:-------------:|:-----:|:------:|:---------------:|:-------:|:-------:|
-| 1.1574        | 1.0   | 21506  | 1.7303          | 27.2765 | 26.7867 |
-| 1.218         | 2.0   | 43012  | 1.7203          | 27.9434 | 26.7402 |
-| 1.1679        | 3.0   | 64518  | 1.6614          | 28.7723 | 27.033  |
-| 1.1122        | 4.0   | 86024  | 1.6420          | 29.5047 | 26.5014 |
-| 1.072         | 5.0   | 107530 | 1.6596          | 29.9396 | 27.11   |
-| 1.0324        | 6.0   | 129036 | 1.6229          | 30.6433 | 26.9464 |
-| 0.9953        | 7.0   | 150542 | 1.6070          | 30.9374 | 26.7292 |
-| 0.9597        | 8.0   | 172048 | 1.5850          | 31.7838 | 26.7111 |
-| 0.9172        | 9.0   | 193554 | 1.5789          | 32.4267 | 26.7353 |
-| 0.8885        | 10.0  | 215060 | 1.5751          | 32.4337 | 26.6242 |
-
-
-### Framework versions
-
-- Transformers 4.16.2
-- Pytorch 1.10.2+cu113
-- Datasets 1.18.3
-- Tokenizers 0.11.0

config.json

@@ -1,5 +1,4 @@
 {
-  "_name_or_path": "/home/alexey/work/ML/models/opus-mt-ru-en-finetuned",
   "_num_labels": 3,
   "activation_dropout": 0.0,
   "activation_function": "swish",
@@ -16,7 +15,6 @@
   ],
   "bos_token_id": 0,
   "classif_dropout": 0.0,
-  "classifier_dropout": 0.0,
   "d_model": 512,
   "decoder_attention_heads": 8,
   "decoder_ffn_dim": 2048,
@@ -29,7 +27,6 @@
   "encoder_layerdrop": 0.0,
   "encoder_layers": 6,
   "eos_token_id": 0,
-  "forced_eos_token_id": 0,
   "id2label": {
     "0": "LABEL_0",
     "1": "LABEL_1",
@@ -49,12 +46,8 @@
   "normalize_embedding": false,
   "num_beams": 6,
   "num_hidden_layers": 6,
-  "output_attentions": true,
   "pad_token_id": 62517,
   "scale_embedding": true,
   "static_position_embeddings": true,
-  "torch_dtype": "float32",
-  "transformers_version": "4.16.2",
-  "use_cache": true,
   "vocab_size": 62518
 }

pytorch_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6616853b83f71d7b6ac76bbc5ef61bed5497678111179ee53558e4c3b314f043
-size 304935301
+oid sha256:535450eb5613f3cc912f9ca3e54cfef6c14d201b319c24a88faf776a65538b5d
+size 306991893

runs/Feb21_00-57-18_delta/1645405129.4565482/events.out.tfevents.1645405129.delta.340777.1

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:4364c2e4db166bde7c28a458d3e91a19e9392b6e2917bcff73e4ef4833b739fd
-size 5119

runs/Feb21_00-57-18_delta/events.out.tfevents.1645405129.delta.340777.0

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:4c176a6e7f2533a675ac090ece183212afc609755820b2cad03babddba275c9f
-size 18328

runs/Feb21_09-23-04_delta/1645435477.6763067/events.out.tfevents.1645435477.delta.384317.1

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:8154ab446a19e588adcebc42e2e7dad877ffd92fb63875728f22078201d7e30d
-size 5122

runs/Feb21_09-23-04_delta/1645460644.374451/events.out.tfevents.1645460644.delta.384317.3

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:cc6f672b5b37002189ca0e37b6efb7011d7c99de158c6fb44dc8c17036c85c77
-size 5122

runs/Feb21_09-23-04_delta/events.out.tfevents.1645435477.delta.384317.0

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:09e73c04c291fea60b9b159bd8ba665e30de4af1a67ae917f1b93259e900800b
-size 77118

runs/Feb21_09-23-04_delta/events.out.tfevents.1645460644.delta.384317.2

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:f0756b43f43aba45498b982653574f397e4619532373531a23cb4085af97d0a7
-size 28039

runs/Feb21_19-30-06_delta/1645471936.3118231/events.out.tfevents.1645471936.delta.532773.1

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:fa203a95d2dddb8dc38f89d6b7f509b6bc424f3a265d8b5c2e47e4a27c48ceb1
-size 5122

runs/Feb21_19-30-06_delta/events.out.tfevents.1645471936.delta.532773.0

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:baf1be96a0a9080ed2613e42ca3dae651583f5528801d97d5592fb661bb626f2
-size 13684

runs/Feb21_20-31-25_delta/1645475580.3563583/events.out.tfevents.1645475580.delta.550101.1

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:7ed2d35914d7833a7b749c807f5c43d705e0e4d74bb3b7afc4b93bdebf4870cd
-size 5122

runs/Feb21_20-31-25_delta/events.out.tfevents.1645475580.delta.550101.0

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:e67086f40b65b535ecebf41c1fb83b62f4b012dab21801758cb0566466b3a503
-size 131773

runs/Feb22_17-25-42_delta/1645550835.5555484/events.out.tfevents.1645550835.delta.890584.1

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:2ea23ea0da0c0949532935dc472a7d79ab4175b5c972c52094707d78808c2820
-size 5122

runs/Feb22_17-25-42_delta/events.out.tfevents.1645550835.delta.890584.0

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:27dd7a7a70d4ca32e6959f4d1ff27d750f20331431a7cae0c3a23a74e7b5bd69
-size 10587

tokenizer_config.json

@@ -1 +1 @@
-{"source_lang": "ru", "target_lang": "en", "unk_token": "<unk>", "eos_token": "</s>", "pad_token": "<pad>", "model_max_length": 512, "sp_model_kwargs": {}, "special_tokens_map_file": null, "tokenizer_file": null, "name_or_path": "/home/alexey/work/ML/models/opus-mt-ru-en-finetuned", "tokenizer_class": "MarianTokenizer"}
+{"target_lang": "en", "source_lang": "ru"}