Hugging Face's logo

mehdie
/
fine_tuned_ancient_semitic_BERT

Text Classification
Transformers
TensorBoard
Safetensors
bert
Generated from Trainer
Inference Endpoints
Model card Files Metrics Community
morten-j commited on
Commit
e28f649
•
1 Parent(s): 18b29e0

morten-j/fine_tuned_ancient_semitic_BERT

Browse files
Files changed (17) hide show
  1. README.md +39 -32
  2. config.json +2 -2
  3. model.safetensors +1 -1
  4. runs/Apr26_09-01-46_a256-a40-06.srv.aau.dk/events.out.tfevents.1714114908.a256-a40-06.srv.aau.dk.629128.0 +3 -0
  5. runs/Apr26_09-01-46_a256-a40-06.srv.aau.dk/events.out.tfevents.1714115003.a256-a40-06.srv.aau.dk.629128.1 +3 -0
  6. runs/Apr26_09-04-15_a256-a40-06.srv.aau.dk/events.out.tfevents.1714115056.a256-a40-06.srv.aau.dk.629527.0 +3 -0
  7. runs/Apr26_09-04-15_a256-a40-06.srv.aau.dk/events.out.tfevents.1714115150.a256-a40-06.srv.aau.dk.629527.1 +3 -0
  8. runs/Apr26_09-14-14_a256-a40-04.srv.aau.dk/events.out.tfevents.1714115659.a256-a40-04.srv.aau.dk.747857.0 +3 -0
  9. runs/Apr26_09-14-14_a256-a40-04.srv.aau.dk/events.out.tfevents.1714115753.a256-a40-04.srv.aau.dk.747857.1 +3 -0
  10. runs/Apr26_09-17-46_a256-a40-04.srv.aau.dk/events.out.tfevents.1714115868.a256-a40-04.srv.aau.dk.748279.0 +3 -0
  11. runs/Apr26_09-17-46_a256-a40-04.srv.aau.dk/events.out.tfevents.1714115969.a256-a40-04.srv.aau.dk.748279.1 +3 -0
  12. runs/Apr26_09-30-46_a256-a40-04.srv.aau.dk/events.out.tfevents.1714116648.a256-a40-04.srv.aau.dk.748897.0 +3 -0
  13. runs/Apr26_09-30-46_a256-a40-04.srv.aau.dk/events.out.tfevents.1714116753.a256-a40-04.srv.aau.dk.748897.1 +3 -0
  14. runs/Apr26_09-50-12_a256-a40-05.srv.aau.dk/events.out.tfevents.1714117816.a256-a40-05.srv.aau.dk.344137.0 +3 -0
  15. runs/Apr26_09-50-12_a256-a40-05.srv.aau.dk/events.out.tfevents.1714117921.a256-a40-05.srv.aau.dk.344137.1 +3 -0
  16. tokenizer.json +1 -6
  17. training_args.bin +1 -1
README.md CHANGED
@@ -1,4 +1,5 @@
1
  ---
 
2
  tags:
3
  - generated_from_trainer
4
  metrics:
@@ -15,13 +16,13 @@ should probably proofread and complete it, then remove this comment. -->
15
 
16
  # fine_tuned_ancient_semitic_BERT
17
 
18
- This model was trained from scratch on an unknown dataset.
19
  It achieves the following results on the evaluation set:
20
- - Loss: 0.5170
21
- - F1: 0.2927
22
- - F5: 0.2847
23
- - Precision: 0.3158
24
- - Recall: 0.2727
25
 
26
  ## Model description
27
 
@@ -41,43 +42,49 @@ More information needed
41
 
42
  The following hyperparameters were used during training:
43
  - learning_rate: 5e-05
44
- - train_batch_size: 32
45
- - eval_batch_size: 32
46
  - seed: 42
 
47
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
48
  - lr_scheduler_type: linear
49
- - num_epochs: 20
50
  - mixed_precision_training: Native AMP
51
 
52
  ### Training results
53
 
54
  | Training Loss | Epoch | Step | Validation Loss | F1 | F5 | Precision | Recall |
55
  |:-------------:|:-----:|:----:|:---------------:|:------:|:------:|:---------:|:------:|
56
- | No log | 1.0 | 65 | 0.3078 | 0.0 | 0.0 | 0.0 | 0.0 |
57
- | No log | 2.0 | 130 | 0.3062 | 0.0 | 0.0 | 0.0 | 0.0 |
58
- | No log | 3.0 | 195 | 0.3063 | 0.0 | 0.0 | 0.0 | 0.0 |
59
- | No log | 4.0 | 260 | 0.3081 | 0.0 | 0.0 | 0.0 | 0.0 |
60
- | No log | 5.0 | 325 | 0.2898 | 0.0 | 0.0 | 0.0 | 0.0 |
61
- | No log | 6.0 | 390 | 0.3567 | 0.0 | 0.0 | 0.0 | 0.0 |
62
- | No log | 7.0 | 455 | 0.3049 | 0.3529 | 0.3110 | 0.5455 | 0.2609 |
63
- | 0.2651 | 8.0 | 520 | 0.3008 | 0.4324 | 0.3957 | 0.5714 | 0.3478 |
64
- | 0.2651 | 9.0 | 585 | 0.3407 | 0.3784 | 0.3462 | 0.5 | 0.3043 |
65
- | 0.2651 | 10.0 | 650 | 0.3392 | 0.4 | 0.3571 | 0.5833 | 0.3043 |
66
- | 0.2651 | 11.0 | 715 | 0.2999 | 0.4444 | 0.4018 | 0.6154 | 0.3478 |
67
- | 0.2651 | 12.0 | 780 | 0.3231 | 0.5 | 0.4520 | 0.6923 | 0.3913 |
68
- | 0.2651 | 13.0 | 845 | 0.3056 | 0.5 | 0.4520 | 0.6923 | 0.3913 |
69
- | 0.2651 | 14.0 | 910 | 0.3097 | 0.5625 | 0.4820 | 1.0 | 0.3913 |
70
- | 0.2651 | 15.0 | 975 | 0.3168 | 0.6316 | 0.5846 | 0.8 | 0.5217 |
71
- | 0.1016 | 16.0 | 1040 | 0.3354 | 0.5882 | 0.5184 | 0.9091 | 0.4348 |
72
- | 0.1016 | 17.0 | 1105 | 0.3730 | 0.5 | 0.4284 | 0.8889 | 0.3478 |
73
- | 0.1016 | 18.0 | 1170 | 0.3749 | 0.5455 | 0.4741 | 0.9 | 0.3913 |
74
- | 0.1016 | 19.0 | 1235 | 0.3808 | 0.5161 | 0.4356 | 1.0 | 0.3478 |
75
- | 0.1016 | 20.0 | 1300 | 0.3620 | 0.5455 | 0.4741 | 0.9 | 0.3913 |
 
 
 
 
 
76
 
77
 
78
  ### Framework versions
79
 
80
- - Transformers 4.38.1
81
- - Pytorch 2.2.1+cu121
82
- - Datasets 2.17.0
83
  - Tokenizers 0.15.2
 
1
  ---
2
+ base_model: mehdie/ancient_semitic_bert
3
  tags:
4
  - generated_from_trainer
5
  metrics:
 
16
 
17
  # fine_tuned_ancient_semitic_BERT
18
 
19
+ This model is a fine-tuned version of [mehdie/ancient_semitic_bert](https://huggingface.co/mehdie/ancient_semitic_bert) on an unknown dataset.
20
  It achieves the following results on the evaluation set:
21
+ - Loss: 0.3771
22
+ - F1: 0.5652
23
+ - F5: 0.5748
24
+ - Precision: 0.5417
25
+ - Recall: 0.5909
26
 
27
  ## Model description
28
 
 
42
 
43
  The following hyperparameters were used during training:
44
  - learning_rate: 5e-05
45
+ - train_batch_size: 128
46
+ - eval_batch_size: 128
47
  - seed: 42
48
+ - distributed_type: multi-GPU
49
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
50
  - lr_scheduler_type: linear
51
+ - num_epochs: 25
52
  - mixed_precision_training: Native AMP
53
 
54
  ### Training results
55
 
56
  | Training Loss | Epoch | Step | Validation Loss | F1 | F5 | Precision | Recall |
57
  |:-------------:|:-----:|:----:|:---------------:|:------:|:------:|:---------:|:------:|
58
+ | No log | 1.0 | 17 | 0.3386 | 0.0 | 0.0 | 0.0 | 0.0 |
59
+ | No log | 2.0 | 34 | 0.3252 | 0.0 | 0.0 | 0.0 | 0.0 |
60
+ | No log | 3.0 | 51 | 0.3188 | 0.0 | 0.0 | 0.0 | 0.0 |
61
+ | No log | 4.0 | 68 | 0.3410 | 0.0 | 0.0 | 0.0 | 0.0 |
62
+ | No log | 5.0 | 85 | 0.3069 | 0.1379 | 0.1080 | 0.5 | 0.08 |
63
+ | No log | 6.0 | 102 | 0.3209 | 0.1379 | 0.1080 | 0.5 | 0.08 |
64
+ | No log | 7.0 | 119 | 0.3432 | 0.2222 | 0.2132 | 0.25 | 0.2 |
65
+ | No log | 8.0 | 136 | 0.3606 | 0.2727 | 0.2592 | 0.3158 | 0.24 |
66
+ | No log | 9.0 | 153 | 0.3319 | 0.2927 | 0.2700 | 0.375 | 0.24 |
67
+ | No log | 10.0 | 170 | 0.3741 | 0.4074 | 0.4193 | 0.3793 | 0.44 |
68
+ | No log | 11.0 | 187 | 0.3008 | 0.3784 | 0.3336 | 0.5833 | 0.28 |
69
+ | No log | 12.0 | 204 | 0.3237 | 0.4231 | 0.4294 | 0.4074 | 0.44 |
70
+ | No log | 13.0 | 221 | 0.2848 | 0.5 | 0.4752 | 0.5789 | 0.44 |
71
+ | No log | 14.0 | 238 | 0.3058 | 0.52 | 0.52 | 0.52 | 0.52 |
72
+ | No log | 15.0 | 255 | 0.2912 | 0.5417 | 0.5332 | 0.5652 | 0.52 |
73
+ | No log | 16.0 | 272 | 0.3005 | 0.4681 | 0.4569 | 0.5 | 0.44 |
74
+ | No log | 17.0 | 289 | 0.3122 | 0.5556 | 0.5717 | 0.5172 | 0.6 |
75
+ | No log | 18.0 | 306 | 0.3670 | 0.5667 | 0.6052 | 0.4857 | 0.68 |
76
+ | No log | 19.0 | 323 | 0.2818 | 0.5926 | 0.6098 | 0.5517 | 0.64 |
77
+ | No log | 20.0 | 340 | 0.3012 | 0.5882 | 0.5927 | 0.5769 | 0.6 |
78
+ | No log | 21.0 | 357 | 0.3288 | 0.6154 | 0.6246 | 0.5926 | 0.64 |
79
+ | No log | 22.0 | 374 | 0.3251 | 0.6250 | 0.6152 | 0.6522 | 0.6 |
80
+ | No log | 23.0 | 391 | 0.3145 | 0.6250 | 0.6152 | 0.6522 | 0.6 |
81
+ | No log | 24.0 | 408 | 0.3128 | 0.68 | 0.68 | 0.68 | 0.68 |
82
+ | No log | 25.0 | 425 | 0.3190 | 0.6538 | 0.6636 | 0.6296 | 0.68 |
83
 
84
 
85
  ### Framework versions
86
 
87
+ - Transformers 4.38.2
88
+ - Pytorch 2.3.0a0+ebedce2
89
+ - Datasets 2.17.1
90
  - Tokenizers 0.15.2
config.json CHANGED
@@ -1,5 +1,5 @@
1
  {
2
- "_name_or_path": "/home/morte/Downloads/trained_full_bert_model",
3
  "architectures": [
4
  "BertForSequenceClassification"
5
  ],
@@ -27,7 +27,7 @@
27
  "position_embedding_type": "absolute",
28
  "problem_type": "single_label_classification",
29
  "torch_dtype": "float32",
30
- "transformers_version": "4.38.1",
31
  "type_vocab_size": 2,
32
  "use_cache": true,
33
  "vocab_size": 52000
 
1
  {
2
+ "_name_or_path": "mehdie/ancient_semitic_bert",
3
  "architectures": [
4
  "BertForSequenceClassification"
5
  ],
 
27
  "position_embedding_type": "absolute",
28
  "problem_type": "single_label_classification",
29
  "torch_dtype": "float32",
30
+ "transformers_version": "4.38.2",
31
  "type_vocab_size": 2,
32
  "use_cache": true,
33
  "vocab_size": 52000
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:b9b6d50390fd33828c8db61412be3d8c423d264ed79a69d541715f8da88aca65
3
  size 503939072
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4b7f794821387b054669e95f6ded31a8a7ba08ed0fb7890ea039f05e46cf8a67
3
  size 503939072
runs/Apr26_09-01-46_a256-a40-06.srv.aau.dk/events.out.tfevents.1714114908.a256-a40-06.srv.aau.dk.629128.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:92b8cd5c64138783438e1331c7a76bbd65f82218476401ad3e79203a82c73fa4
3
+ size 14289
runs/Apr26_09-01-46_a256-a40-06.srv.aau.dk/events.out.tfevents.1714115003.a256-a40-06.srv.aau.dk.629128.1 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:118cb022325279802f05d0297166beb64517b883fe7d7e7fd477b09ff3ff5bc4
3
+ size 506
runs/Apr26_09-04-15_a256-a40-06.srv.aau.dk/events.out.tfevents.1714115056.a256-a40-06.srv.aau.dk.629527.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7cdfe00caefef62b6b13ea0234b04912e0b3f94880abbdc6678f93b7d5e9a6f2
3
+ size 14289
runs/Apr26_09-04-15_a256-a40-06.srv.aau.dk/events.out.tfevents.1714115150.a256-a40-06.srv.aau.dk.629527.1 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:18647a7d73f5d278c7803810ef03dc24f6e48424b4b8ad6c29a3a9c2338d737b
3
+ size 506
runs/Apr26_09-14-14_a256-a40-04.srv.aau.dk/events.out.tfevents.1714115659.a256-a40-04.srv.aau.dk.747857.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:11583c14204d9b9c38b2415bcee7c8f24b0af30286009e4ca93fd3dd5d7ce435
3
+ size 14290
runs/Apr26_09-14-14_a256-a40-04.srv.aau.dk/events.out.tfevents.1714115753.a256-a40-04.srv.aau.dk.747857.1 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4e539384e06714ddfdba2df5d89648115754d1307b951004a37ddbf51dccb83e
3
+ size 506
runs/Apr26_09-17-46_a256-a40-04.srv.aau.dk/events.out.tfevents.1714115868.a256-a40-04.srv.aau.dk.748279.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:93cf961ba4b1ee7bb8b5a052f71895bd673d82b1e425b642adb5e3c9996353fc
3
+ size 16619
runs/Apr26_09-17-46_a256-a40-04.srv.aau.dk/events.out.tfevents.1714115969.a256-a40-04.srv.aau.dk.748279.1 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5fe08c14a90b284167ed7feabe32bcc4047bbe7c550dc660cc6f317f0aed8c9e
3
+ size 506
runs/Apr26_09-30-46_a256-a40-04.srv.aau.dk/events.out.tfevents.1714116648.a256-a40-04.srv.aau.dk.748897.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0a84f99e6e873ffa671d0751aac872fe217487edd52fc5bf3b093c5e0e358d9c
3
+ size 16619
runs/Apr26_09-30-46_a256-a40-04.srv.aau.dk/events.out.tfevents.1714116753.a256-a40-04.srv.aau.dk.748897.1 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2fb5c25c62161e1401b38ad1ec86c7a3fd83c9d3e8a8ff47686e83a26a1e190d
3
+ size 506
runs/Apr26_09-50-12_a256-a40-05.srv.aau.dk/events.out.tfevents.1714117816.a256-a40-05.srv.aau.dk.344137.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:41ef4f252660a7eea837f88981904739e48266b832293593d2eec4db0fd2407e
3
+ size 16619
runs/Apr26_09-50-12_a256-a40-05.srv.aau.dk/events.out.tfevents.1714117921.a256-a40-05.srv.aau.dk.344137.1 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:30b8780ad944692f4ac1edc6edcc60250522078f2e0b67425d3ff582f1456ef5
3
+ size 506
tokenizer.json CHANGED
@@ -1,11 +1,6 @@
1
  {
2
  "version": "1.0",
3
- "truncation": {
4
- "direction": "Right",
5
- "max_length": 512,
6
- "strategy": "LongestFirst",
7
- "stride": 0
8
- },
9
  "padding": null,
10
  "added_tokens": [
11
  {
 
1
  {
2
  "version": "1.0",
3
+ "truncation": null,
 
 
 
 
 
4
  "padding": null,
5
  "added_tokens": [
6
  {
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:ff2eb0aeaad3e147061114db3405c55f951a3a9609e4871e09057396a3650875
3
  size 4920
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6cd97a7ac8f4f857db5f21e37103bcdc00157673fe856ab3a6b128c793c07ae8
3
  size 4920