Commit
•
a81498b
1
Parent(s):
0bf7600
Pushing fiqa sent bert gpl 1400 steps trained model
Browse files
README.md
CHANGED
@@ -4,12 +4,13 @@ tags:
|
|
4 |
- sentence-transformers
|
5 |
- feature-extraction
|
6 |
- sentence-similarity
|
|
|
7 |
|
8 |
---
|
9 |
|
10 |
# {MODEL_NAME}
|
11 |
|
12 |
-
This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a
|
13 |
|
14 |
<!--- Describe your model here -->
|
15 |
|
@@ -34,6 +35,41 @@ print(embeddings)
|
|
34 |
|
35 |
|
36 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
37 |
## Evaluation Results
|
38 |
|
39 |
<!--- Describe how your model was evaluated -->
|
@@ -77,9 +113,8 @@ Parameters of the fit()-Method:
|
|
77 |
## Full Model Architecture
|
78 |
```
|
79 |
SentenceTransformer(
|
80 |
-
(0): Transformer({'max_seq_length': 350, 'do_lower_case':
|
81 |
-
(1): Pooling({'word_embedding_dimension':
|
82 |
-
(2): Normalize()
|
83 |
)
|
84 |
```
|
85 |
|
|
|
4 |
- sentence-transformers
|
5 |
- feature-extraction
|
6 |
- sentence-similarity
|
7 |
+
- transformers
|
8 |
|
9 |
---
|
10 |
|
11 |
# {MODEL_NAME}
|
12 |
|
13 |
+
This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 1024 dimensional dense vector space and can be used for tasks like clustering or semantic search.
|
14 |
|
15 |
<!--- Describe your model here -->
|
16 |
|
|
|
35 |
|
36 |
|
37 |
|
38 |
+
## Usage (HuggingFace Transformers)
|
39 |
+
Without [sentence-transformers](https://www.SBERT.net), you can use the model like this: First, you pass your input through the transformer model, then you have to apply the right pooling-operation on-top of the contextualized word embeddings.
|
40 |
+
|
41 |
+
```python
|
42 |
+
from transformers import AutoTokenizer, AutoModel
|
43 |
+
import torch
|
44 |
+
|
45 |
+
|
46 |
+
def cls_pooling(model_output, attention_mask):
|
47 |
+
return model_output[0][:,0]
|
48 |
+
|
49 |
+
|
50 |
+
# Sentences we want sentence embeddings for
|
51 |
+
sentences = ['This is an example sentence', 'Each sentence is converted']
|
52 |
+
|
53 |
+
# Load model from HuggingFace Hub
|
54 |
+
tokenizer = AutoTokenizer.from_pretrained('{MODEL_NAME}')
|
55 |
+
model = AutoModel.from_pretrained('{MODEL_NAME}')
|
56 |
+
|
57 |
+
# Tokenize sentences
|
58 |
+
encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')
|
59 |
+
|
60 |
+
# Compute token embeddings
|
61 |
+
with torch.no_grad():
|
62 |
+
model_output = model(**encoded_input)
|
63 |
+
|
64 |
+
# Perform pooling. In this case, cls pooling.
|
65 |
+
sentence_embeddings = cls_pooling(model_output, encoded_input['attention_mask'])
|
66 |
+
|
67 |
+
print("Sentence embeddings:")
|
68 |
+
print(sentence_embeddings)
|
69 |
+
```
|
70 |
+
|
71 |
+
|
72 |
+
|
73 |
## Evaluation Results
|
74 |
|
75 |
<!--- Describe how your model was evaluated -->
|
|
|
113 |
## Full Model Architecture
|
114 |
```
|
115 |
SentenceTransformer(
|
116 |
+
(0): Transformer({'max_seq_length': 350, 'do_lower_case': True}) with Transformer model: BertModel
|
117 |
+
(1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
|
|
|
118 |
)
|
119 |
```
|
120 |
|