JuanIgnacioSolerno commited on
Commit
4974cd7
1 Parent(s): 6041de1

Add new SentenceTransformer model.

Browse files
1_Pooling/config.json ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "word_embedding_dimension": 768,
3
+ "pooling_mode_cls_token": false,
4
+ "pooling_mode_mean_tokens": true,
5
+ "pooling_mode_max_tokens": false,
6
+ "pooling_mode_mean_sqrt_len_tokens": false,
7
+ "pooling_mode_weightedmean_tokens": false,
8
+ "pooling_mode_lasttoken": false,
9
+ "include_prompt": true
10
+ }
README.md ADDED
@@ -0,0 +1,234 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language: []
3
+ library_name: sentence-transformers
4
+ tags:
5
+ - sentence-transformers
6
+ - sentence-similarity
7
+ - feature-extraction
8
+ - dataset_size:10K<n<100K
9
+ - loss:CosineSimilarityLoss
10
+ base_model: sentence-transformers/all-mpnet-base-v2
11
+ widget:
12
+ - source_sentence: Tech Writer
13
+ sentences:
14
+ - Tech Specialist
15
+ - Architectural Historian
16
+ - Order Selector (Picker)
17
+ - source_sentence: Controller
18
+ sentences:
19
+ - Assistant Controller
20
+ - Key Accounts Supervisor
21
+ - Cosmetologist - Plus Tips
22
+ - source_sentence: Accountant
23
+ sentences:
24
+ - Financial Accountant
25
+ - Manager, Corporate Sales
26
+ - Materials Sourcing Lead
27
+ - source_sentence: Planner III
28
+ sentences:
29
+ - Strategist
30
+ - Product Finance Manager
31
+ - Materials Sourcing Lead
32
+ - source_sentence: AP Analyst
33
+ sentences:
34
+ - AP Specialist
35
+ - ESCO Service Coordinator
36
+ - Boiler Tender Lead
37
+ pipeline_tag: sentence-similarity
38
+ ---
39
+
40
+ # SentenceTransformer based on sentence-transformers/all-mpnet-base-v2
41
+
42
+ This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [sentence-transformers/all-mpnet-base-v2](https://huggingface.co/sentence-transformers/all-mpnet-base-v2). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
43
+
44
+ ## Model Details
45
+
46
+ ### Model Description
47
+ - **Model Type:** Sentence Transformer
48
+ - **Base model:** [sentence-transformers/all-mpnet-base-v2](https://huggingface.co/sentence-transformers/all-mpnet-base-v2) <!-- at revision 84f2bcc00d77236f9e89c8a360a00fb1139bf47d -->
49
+ - **Maximum Sequence Length:** 384 tokens
50
+ - **Output Dimensionality:** 768 tokens
51
+ - **Similarity Function:** Cosine Similarity
52
+ <!-- - **Training Dataset:** Unknown -->
53
+ <!-- - **Language:** Unknown -->
54
+ <!-- - **License:** Unknown -->
55
+
56
+ ### Model Sources
57
+
58
+ - **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
59
+ - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
60
+ - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
61
+
62
+ ### Full Model Architecture
63
+
64
+ ```
65
+ SentenceTransformer(
66
+ (0): Transformer({'max_seq_length': 384, 'do_lower_case': False}) with Transformer model: MPNetModel
67
+ (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
68
+ (2): Normalize()
69
+ )
70
+ ```
71
+
72
+ ## Usage
73
+
74
+ ### Direct Usage (Sentence Transformers)
75
+
76
+ First install the Sentence Transformers library:
77
+
78
+ ```bash
79
+ pip install -U sentence-transformers
80
+ ```
81
+
82
+ Then you can load this model and run inference.
83
+ ```python
84
+ from sentence_transformers import SentenceTransformer
85
+
86
+ # Download from the 🤗 Hub
87
+ model = SentenceTransformer("JuanIgnacioSolerno/all-mpnet-base-v2-sts")
88
+ # Run inference
89
+ sentences = [
90
+ 'AP Analyst',
91
+ 'AP Specialist',
92
+ 'ESCO Service Coordinator',
93
+ ]
94
+ embeddings = model.encode(sentences)
95
+ print(embeddings.shape)
96
+ # [3, 768]
97
+
98
+ # Get the similarity scores for the embeddings
99
+ similarities = model.similarity(embeddings, embeddings)
100
+ print(similarities.shape)
101
+ # [3, 3]
102
+ ```
103
+
104
+ <!--
105
+ ### Direct Usage (Transformers)
106
+
107
+ <details><summary>Click to see the direct usage in Transformers</summary>
108
+
109
+ </details>
110
+ -->
111
+
112
+ <!--
113
+ ### Downstream Usage (Sentence Transformers)
114
+
115
+ You can finetune this model on your own dataset.
116
+
117
+ <details><summary>Click to expand</summary>
118
+
119
+ </details>
120
+ -->
121
+
122
+ <!--
123
+ ### Out-of-Scope Use
124
+
125
+ *List how the model may foreseeably be misused and address what users ought not to do with the model.*
126
+ -->
127
+
128
+ <!--
129
+ ## Bias, Risks and Limitations
130
+
131
+ *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
132
+ -->
133
+
134
+ <!--
135
+ ### Recommendations
136
+
137
+ *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
138
+ -->
139
+
140
+ ## Training Details
141
+
142
+ ### Training Dataset
143
+
144
+ #### Unnamed Dataset
145
+
146
+
147
+ * Size: 11,923 training samples
148
+ * Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
149
+ * Approximate statistics based on the first 1000 samples:
150
+ | | sentence1 | sentence2 | score |
151
+ |:--------|:---------------------------------------------------------------------------------|:-------------------------------------------------------------------------------|:---------------------------------------------------------------|
152
+ | type | string | string | float |
153
+ | details | <ul><li>min: 3 tokens</li><li>mean: 7.17 tokens</li><li>max: 27 tokens</li></ul> | <ul><li>min: 4 tokens</li><li>mean: 4.0 tokens</li><li>max: 4 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.04</li><li>max: 1.0</li></ul> |
154
+ * Samples:
155
+ | sentence1 | sentence2 | score |
156
+ |:-----------------------------------------------------------------------------|:----------------------------|:-----------------|
157
+ | <code>Land Coordinator, Renewable Development</code> | <code>Energy Analyst</code> | <code>0.0</code> |
158
+ | <code>Customer Service Advocate - Remote within the state of Colorado</code> | <code>Energy Analyst</code> | <code>0.0</code> |
159
+ | <code>Global Head of Infrastructure</code> | <code>Energy Analyst</code> | <code>0.0</code> |
160
+ * Loss: [<code>CosineSimilarityLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cosinesimilarityloss) with these parameters:
161
+ ```json
162
+ {
163
+ "loss_fct": "torch.nn.modules.loss.MSELoss"
164
+ }
165
+ ```
166
+
167
+ ### Evaluation Dataset
168
+
169
+ #### Unnamed Dataset
170
+
171
+
172
+ * Size: 2,981 evaluation samples
173
+ * Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
174
+ * Approximate statistics based on the first 1000 samples:
175
+ | | sentence1 | sentence2 | score |
176
+ |:--------|:---------------------------------------------------------------------------------|:-------------------------------------------------------------------------------|:---------------------------------------------------------------|
177
+ | type | string | string | float |
178
+ | details | <ul><li>min: 3 tokens</li><li>mean: 7.21 tokens</li><li>max: 28 tokens</li></ul> | <ul><li>min: 4 tokens</li><li>mean: 4.0 tokens</li><li>max: 4 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.05</li><li>max: 1.0</li></ul> |
179
+ * Samples:
180
+ | sentence1 | sentence2 | score |
181
+ |:---------------------------------------------------------------------|:----------------------------|:-----------------|
182
+ | <code>IT Data Coordinator - Customer Data & Integrations Team</code> | <code>Energy Analyst</code> | <code>0.0</code> |
183
+ | <code>Warehouse Associate</code> | <code>Energy Analyst</code> | <code>0.0</code> |
184
+ | <code>Human Resources Manager</code> | <code>Energy Analyst</code> | <code>0.0</code> |
185
+ * Loss: [<code>CosineSimilarityLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cosinesimilarityloss) with these parameters:
186
+ ```json
187
+ {
188
+ "loss_fct": "torch.nn.modules.loss.MSELoss"
189
+ }
190
+ ```
191
+
192
+ ### Framework Versions
193
+ - Python: 3.10.14
194
+ - Sentence Transformers: 3.0.0
195
+ - Transformers: 4.41.2
196
+ - PyTorch: 2.0.0.post200
197
+ - Accelerate: 0.30.1
198
+ - Datasets: 2.19.1
199
+ - Tokenizers: 0.19.1
200
+
201
+ ## Citation
202
+
203
+ ### BibTeX
204
+
205
+ #### Sentence Transformers
206
+ ```bibtex
207
+ @inproceedings{reimers-2019-sentence-bert,
208
+ title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
209
+ author = "Reimers, Nils and Gurevych, Iryna",
210
+ booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
211
+ month = "11",
212
+ year = "2019",
213
+ publisher = "Association for Computational Linguistics",
214
+ url = "https://arxiv.org/abs/1908.10084",
215
+ }
216
+ ```
217
+
218
+ <!--
219
+ ## Glossary
220
+
221
+ *Clearly define terms in order to be accessible across audiences.*
222
+ -->
223
+
224
+ <!--
225
+ ## Model Card Authors
226
+
227
+ *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
228
+ -->
229
+
230
+ <!--
231
+ ## Model Card Contact
232
+
233
+ *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
234
+ -->
config.json ADDED
@@ -0,0 +1,24 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "sentence-transformers/all-mpnet-base-v2",
3
+ "architectures": [
4
+ "MPNetModel"
5
+ ],
6
+ "attention_probs_dropout_prob": 0.1,
7
+ "bos_token_id": 0,
8
+ "eos_token_id": 2,
9
+ "hidden_act": "gelu",
10
+ "hidden_dropout_prob": 0.1,
11
+ "hidden_size": 768,
12
+ "initializer_range": 0.02,
13
+ "intermediate_size": 3072,
14
+ "layer_norm_eps": 1e-05,
15
+ "max_position_embeddings": 514,
16
+ "model_type": "mpnet",
17
+ "num_attention_heads": 12,
18
+ "num_hidden_layers": 12,
19
+ "pad_token_id": 1,
20
+ "relative_attention_num_buckets": 32,
21
+ "torch_dtype": "float32",
22
+ "transformers_version": "4.41.2",
23
+ "vocab_size": 30527
24
+ }
config_sentence_transformers.json ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "__version__": {
3
+ "sentence_transformers": "2.0.0",
4
+ "transformers": "4.6.1",
5
+ "pytorch": "1.8.1"
6
+ },
7
+ "prompts": {},
8
+ "default_prompt_name": null,
9
+ "similarity_fn_name": null
10
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0b3c8c717335c801abb15983036a6f1df4b6943fd6b93717969efd96d22eeec6
3
+ size 437967672
modules.json ADDED
@@ -0,0 +1,20 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [
2
+ {
3
+ "idx": 0,
4
+ "name": "0",
5
+ "path": "",
6
+ "type": "sentence_transformers.models.Transformer"
7
+ },
8
+ {
9
+ "idx": 1,
10
+ "name": "1",
11
+ "path": "1_Pooling",
12
+ "type": "sentence_transformers.models.Pooling"
13
+ },
14
+ {
15
+ "idx": 2,
16
+ "name": "2",
17
+ "path": "2_Normalize",
18
+ "type": "sentence_transformers.models.Normalize"
19
+ }
20
+ ]
sentence_bert_config.json ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ {
2
+ "max_seq_length": 384,
3
+ "do_lower_case": false
4
+ }
special_tokens_map.json ADDED
@@ -0,0 +1,51 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token": {
3
+ "content": "<s>",
4
+ "lstrip": false,
5
+ "normalized": false,
6
+ "rstrip": false,
7
+ "single_word": false
8
+ },
9
+ "cls_token": {
10
+ "content": "<s>",
11
+ "lstrip": false,
12
+ "normalized": false,
13
+ "rstrip": false,
14
+ "single_word": false
15
+ },
16
+ "eos_token": {
17
+ "content": "</s>",
18
+ "lstrip": false,
19
+ "normalized": false,
20
+ "rstrip": false,
21
+ "single_word": false
22
+ },
23
+ "mask_token": {
24
+ "content": "<mask>",
25
+ "lstrip": true,
26
+ "normalized": false,
27
+ "rstrip": false,
28
+ "single_word": false
29
+ },
30
+ "pad_token": {
31
+ "content": "<pad>",
32
+ "lstrip": false,
33
+ "normalized": false,
34
+ "rstrip": false,
35
+ "single_word": false
36
+ },
37
+ "sep_token": {
38
+ "content": "</s>",
39
+ "lstrip": false,
40
+ "normalized": false,
41
+ "rstrip": false,
42
+ "single_word": false
43
+ },
44
+ "unk_token": {
45
+ "content": "[UNK]",
46
+ "lstrip": false,
47
+ "normalized": false,
48
+ "rstrip": false,
49
+ "single_word": false
50
+ }
51
+ }
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1,72 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "added_tokens_decoder": {
3
+ "0": {
4
+ "content": "<s>",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false,
9
+ "special": true
10
+ },
11
+ "1": {
12
+ "content": "<pad>",
13
+ "lstrip": false,
14
+ "normalized": false,
15
+ "rstrip": false,
16
+ "single_word": false,
17
+ "special": true
18
+ },
19
+ "2": {
20
+ "content": "</s>",
21
+ "lstrip": false,
22
+ "normalized": false,
23
+ "rstrip": false,
24
+ "single_word": false,
25
+ "special": true
26
+ },
27
+ "3": {
28
+ "content": "<unk>",
29
+ "lstrip": false,
30
+ "normalized": true,
31
+ "rstrip": false,
32
+ "single_word": false,
33
+ "special": true
34
+ },
35
+ "104": {
36
+ "content": "[UNK]",
37
+ "lstrip": false,
38
+ "normalized": false,
39
+ "rstrip": false,
40
+ "single_word": false,
41
+ "special": true
42
+ },
43
+ "30526": {
44
+ "content": "<mask>",
45
+ "lstrip": true,
46
+ "normalized": false,
47
+ "rstrip": false,
48
+ "single_word": false,
49
+ "special": true
50
+ }
51
+ },
52
+ "bos_token": "<s>",
53
+ "clean_up_tokenization_spaces": true,
54
+ "cls_token": "<s>",
55
+ "do_lower_case": true,
56
+ "eos_token": "</s>",
57
+ "mask_token": "<mask>",
58
+ "max_length": 128,
59
+ "model_max_length": 384,
60
+ "pad_to_multiple_of": null,
61
+ "pad_token": "<pad>",
62
+ "pad_token_type_id": 0,
63
+ "padding_side": "right",
64
+ "sep_token": "</s>",
65
+ "stride": 0,
66
+ "strip_accents": null,
67
+ "tokenize_chinese_chars": true,
68
+ "tokenizer_class": "MPNetTokenizer",
69
+ "truncation_side": "right",
70
+ "truncation_strategy": "longest_first",
71
+ "unk_token": "[UNK]"
72
+ }
vocab.txt ADDED
The diff for this file is too large to render. See raw diff