Jacobo commited on
Commit
3f88871
1 Parent(s): a2e005a

Initial commit

Browse files
README.md ADDED
@@ -0,0 +1,75 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - generated_from_trainer
4
+ model-index:
5
+ - name: aristoBERTo
6
+ results: []
7
+ ---
8
+
9
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
10
+ should probably proofread and complete it, then remove this comment. -->
11
+
12
+ # aristoBERTo
13
+
14
+ This model is a fine-tuned version of [nlpaueb/bert-base-greek-uncased-v1](https://huggingface.co/nlpaueb/bert-base-greek-uncased-v1) on the Ancient Greek dataset.
15
+ It achieves the following results on the evaluation set:
16
+ - Loss: 1.6323
17
+
18
+ ## Model description
19
+
20
+ More information needed
21
+
22
+ ## Intended uses & limitations
23
+
24
+ More information needed
25
+
26
+ ## Training and evaluation data
27
+
28
+ More information needed
29
+
30
+ ## Training procedure
31
+
32
+ ### Training hyperparameters
33
+
34
+ The following hyperparameters were used during training:
35
+ - learning_rate: 5e-05
36
+ - train_batch_size: 16
37
+ - eval_batch_size: 16
38
+ - seed: 42
39
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
40
+ - lr_scheduler_type: linear
41
+ - num_epochs: 20.0
42
+ - mixed_precision_training: Native AMP
43
+
44
+ ### Training results
45
+
46
+ | Training Loss | Epoch | Step | Validation Loss |
47
+ |:-------------:|:-----:|:-------:|:---------------:|
48
+ | 2.2017 | 1.0 | 170711 | 2.4411 |
49
+ | 2.0302 | 2.0 | 341422 | 2.2853 |
50
+ | 1.9459 | 3.0 | 512133 | 2.1911 |
51
+ | 1.8847 | 4.0 | 682844 | 2.1187 |
52
+ | 1.85 | 5.0 | 853555 | 2.0841 |
53
+ | 1.834 | 6.0 | 1024266 | 2.0267 |
54
+ | 1.7599 | 7.0 | 1194977 | 1.9874 |
55
+ | 1.7147 | 8.0 | 1365688 | 1.9495 |
56
+ | 1.6882 | 9.0 | 1536399 | 1.9030 |
57
+ | 1.6433 | 10.0 | 1707110 | 1.8845 |
58
+ | 1.6289 | 11.0 | 1877821 | 1.8485 |
59
+ | 1.5758 | 12.0 | 2048532 | 1.8097 |
60
+ | 1.5439 | 13.0 | 2219243 | 1.7916 |
61
+ | 1.5358 | 14.0 | 2389954 | 1.7680 |
62
+ | 1.4946 | 15.0 | 2560665 | 1.7360 |
63
+ | 1.4753 | 16.0 | 2731376 | 1.7052 |
64
+ | 1.4628 | 17.0 | 2902087 | 1.6804 |
65
+ | 1.4223 | 18.0 | 3072798 | 1.6720 |
66
+ | 1.3909 | 19.0 | 3243509 | 1.6521 |
67
+ | 1.377 | 20.0 | 3414220 | 1.6314 |
68
+
69
+
70
+ ### Framework versions
71
+
72
+ - Transformers 4.14.0.dev0
73
+ - Pytorch 1.10.0+cu102
74
+ - Datasets 1.16.1
75
+ - Tokenizers 0.10.3
config.json ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "nlpaueb/bert-base-greek-uncased-v1",
3
+ "architectures": [
4
+ "BertForMaskedLM"
5
+ ],
6
+ "attention_probs_dropout_prob": 0.1,
7
+ "classifier_dropout": null,
8
+ "hidden_act": "gelu",
9
+ "hidden_dropout_prob": 0.1,
10
+ "hidden_size": 768,
11
+ "initializer_range": 0.02,
12
+ "intermediate_size": 3072,
13
+ "layer_norm_eps": 1e-12,
14
+ "max_position_embeddings": 512,
15
+ "model_type": "bert",
16
+ "num_attention_heads": 12,
17
+ "num_hidden_layers": 12,
18
+ "output_past": true,
19
+ "pad_token_id": 0,
20
+ "position_embedding_type": "absolute",
21
+ "torch_dtype": "float32",
22
+ "transformers_version": "4.14.0.dev0",
23
+ "type_vocab_size": 2,
24
+ "use_cache": true,
25
+ "vocab_size": 35000
26
+ }
eval_results.json ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 20.0,
3
+ "eval_loss": 1.6322814226150513,
4
+ "eval_runtime": 77.9898,
5
+ "eval_samples": 84476,
6
+ "eval_samples_per_second": 1083.168,
7
+ "eval_steps_per_second": 67.701,
8
+ "perplexity": 5.1155321064018535
9
+ }
merges.txt ADDED
The diff for this file is too large to render. See raw diff
 
pytorch_model.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:eae6b4506f643cf06b746b229983045ebe08ff7ba45fe3ce0d657914140c2d04
3
+ size 451916331
special_tokens_map.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"unk_token": "[UNK]", "sep_token": "[SEP]", "pad_token": "[PAD]", "cls_token": "[CLS]", "mask_token": "[MASK]"}
tokenizer_config.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"do_lower_case": true, "unk_token": "[UNK]", "sep_token": "[SEP]", "pad_token": "[PAD]", "cls_token": "[CLS]", "mask_token": "[MASK]", "tokenize_chinese_chars": true, "strip_accents": null, "special_tokens_map_file": "/tmp/xdg-cache/huggingface/transformers/347c648cda48ba9d519777e5e1e677a659862faeb5ad9e219ff0a95c62cfdea4.dd8bd9bfd3664b530ea4e645105f557769387b3da9f79bdb55ed556bdd80611d", "name_or_path": "nlpaueb/bert-base-greek-uncased-v1", "do_basic_tokenize": true, "never_split": null, "tokenizer_class": "BertTokenizer"}
vocab.json ADDED
The diff for this file is too large to render. See raw diff
 
vocab.txt ADDED
The diff for this file is too large to render. See raw diff