Hugging Face's logo

LLukas22
/
all-MiniLM-L12-v2-embedding-all

Sentence Similarity
sentence-transformers
PyTorch
TensorBoard
Transformers
English
German
bert
feature-extraction
generated_from_trainer
Inference Endpoints
Model card Files Metrics Community
LLukas22 commited on
Commit
8c8fd40
•
1 Parent(s): fdcf374

Auto Upload

Browse files
Files changed (6) hide show
  1. 1_Pooling/config.json +3 -1
  2. README.md +35 -13
  3. config_sentence_transformers.json +1 -1
  4. lightning_logs/version_1/events.out.tfevents.1677062713.ki-jupyternotebook-8bdd +3 -0
  5. lightning_logs/version_1/hparams.yaml +1 -0
  6. pytorch_model.bin +1 -1
1_Pooling/config.json CHANGED
@@ -3,5 +3,7 @@
3
  "pooling_mode_cls_token": false,
4
  "pooling_mode_mean_tokens": true,
5
  "pooling_mode_max_tokens": false,
6
- "pooling_mode_mean_sqrt_len_tokens": false
 
 
7
  }
 
3
  "pooling_mode_cls_token": false,
4
  "pooling_mode_mean_tokens": true,
5
  "pooling_mode_max_tokens": false,
6
+ "pooling_mode_mean_sqrt_len_tokens": false,
7
+ "pooling_mode_weightedmean_tokens": false,
8
+ "pooling_mode_lasttoken": false
9
  }
README.md CHANGED
@@ -45,34 +45,56 @@ print(embeddings)
45
  ## Training hyperparameters
46
  The following hyperparameters were used during training:
47
 
48
- - learning_rate: 2E-05
49
  - per device batch size: 60
50
- - effective batch size: 120
51
  - seed: 42
52
  - optimizer: AdamW with betas (0.9,0.999) and eps 1E-08
53
- - weight decay: 1E-02
54
- - number of epochs: 4
 
 
55
  - mixed_precision_training: bf16
56
 
57
  ## Training results
58
  | Epoch | Train Loss | Validation Loss |
59
  | ----- | ---------- | --------------- |
60
- | 0 | 0.0655 | 0.055 |
61
- | 1 | 0.0549 | 0.051 |
62
- | 2 | 0.049 | 0.0481 |
63
- | 3 | 0.0451 | 0.0471 |
 
 
 
 
 
 
 
 
 
 
64
 
65
  ## Evaluation results
66
  | Epoch | top_1 | top_3 | top_5 | top_10 | top_25 |
67
  | ----- | ----- | ----- | ----- | ----- | ----- |
68
- | 0 | 0.537 | 0.697 | 0.753 | 0.812 | 0.867 |
69
- | 1 | 0.538 | 0.699 | 0.755 | 0.814 | 0.872 |
70
- | 2 | 0.544 | 0.705 | 0.761 | 0.818 | 0.876 |
71
- | 3 | 0.544 | 0.703 | 0.759 | 0.817 | 0.874 |
 
 
 
 
 
 
 
 
 
 
72
 
73
  ## Framework versions
74
  - Transformers: 4.25.1
75
- - PyTorch: 1.13.0+cu116
76
  - PyTorch Lightning: 1.8.6
77
  - Datasets: 2.7.1
78
  - Tokenizers: 0.13.1
 
45
  ## Training hyperparameters
46
  The following hyperparameters were used during training:
47
 
48
+ - learning_rate: 1E+00
49
  - per device batch size: 60
50
+ - effective batch size: 180
51
  - seed: 42
52
  - optimizer: AdamW with betas (0.9,0.999) and eps 1E-08
53
+ - weight decay: 2E-02
54
+ - D-Adaptation: True
55
+ - Warmup: True
56
+ - number of epochs: 20
57
  - mixed_precision_training: bf16
58
 
59
  ## Training results
60
  | Epoch | Train Loss | Validation Loss |
61
  | ----- | ---------- | --------------- |
62
+ | 0 | 0.0708 | 0.0619 |
63
+ | 1 | 0.0609 | 0.0567 |
64
+ | 2 | 0.0531 | 0.0542 |
65
+ | 3 | 0.0475 | 0.0528 |
66
+ | 4 | 0.0428 | 0.0521 |
67
+ | 5 | 0.0389 | 0.0513 |
68
+ | 6 | 0.0352 | 0.0508 |
69
+ | 7 | 0.0322 | 0.0494 |
70
+ | 8 | 0.0289 | 0.0485 |
71
+ | 9 | 0.0264 | 0.0483 |
72
+ | 10 | 0.0242 | 0.0466 |
73
+ | 11 | 0.0221 | 0.0459 |
74
+ | 12 | 0.0204 | 0.0469 |
75
+ | 13 | 0.0189 | 0.0459 |
76
 
77
  ## Evaluation results
78
  | Epoch | top_1 | top_3 | top_5 | top_10 | top_25 |
79
  | ----- | ----- | ----- | ----- | ----- | ----- |
80
+ | 0 | 0.507 | 0.665 | 0.721 | 0.784 | 0.847 |
81
+ | 1 | 0.501 | 0.661 | 0.719 | 0.783 | 0.846 |
82
+ | 2 | 0.508 | 0.669 | 0.726 | 0.789 | 0.851 |
83
+ | 3 | 0.507 | 0.665 | 0.722 | 0.785 | 0.85 |
84
+ | 4 | 0.506 | 0.667 | 0.724 | 0.788 | 0.851 |
85
+ | 5 | 0.511 | 0.673 | 0.731 | 0.795 | 0.857 |
86
+ | 6 | 0.51 | 0.674 | 0.732 | 0.794 | 0.856 |
87
+ | 7 | 0.512 | 0.674 | 0.732 | 0.796 | 0.859 |
88
+ | 8 | 0.515 | 0.678 | 0.736 | 0.799 | 0.861 |
89
+ | 9 | 0.514 | 0.679 | 0.737 | 0.8 | 0.862 |
90
+ | 10 | 0.52 | 0.683 | 0.741 | 0.803 | 0.864 |
91
+ | 11 | 0.522 | 0.686 | 0.744 | 0.806 | 0.866 |
92
+ | 12 | 0.519 | 0.683 | 0.741 | 0.804 | 0.864 |
93
+ | 13 | 0.522 | 0.685 | 0.743 | 0.806 | 0.865 |
94
 
95
  ## Framework versions
96
  - Transformers: 4.25.1
97
+ - PyTorch: 2.0.0.dev20230210+cu118
98
  - PyTorch Lightning: 1.8.6
99
  - Datasets: 2.7.1
100
  - Tokenizers: 0.13.1
config_sentence_transformers.json CHANGED
@@ -2,6 +2,6 @@
2
  "__version__": {
3
  "sentence_transformers": "2.2.2",
4
  "transformers": "4.25.1",
5
- "pytorch": "1.13.0+cu116"
6
  }
7
  }
 
2
  "__version__": {
3
  "sentence_transformers": "2.2.2",
4
  "transformers": "4.25.1",
5
+ "pytorch": "2.0.0.dev20230210+cu118"
6
  }
7
  }
lightning_logs/version_1/events.out.tfevents.1677062713.ki-jupyternotebook-8bdd ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c9b7ecdbce512eca01526096e2a31bbbe018ac220770335c0e58d337aaf127d5
3
+ size 196449
lightning_logs/version_1/hparams.yaml ADDED
@@ -0,0 +1 @@
 
 
1
+ {}
pytorch_model.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:1de12ff768dc3ddba42c82386efe6a53ac85658f890e5ff00ea64ac33462196c
3
  size 133508397
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c0b3ec92623fe8d89d0a2c7abfef56a738ddb6d60546ec66609ef5add5c2f4bd
3
  size 133508397