DragosGorduza commited on
Commit
a81498b
1 Parent(s): 0bf7600

Pushing fiqa sent bert gpl 1400 steps trained model

Browse files
Files changed (1) hide show
  1. README.md +39 -4
README.md CHANGED
@@ -4,12 +4,13 @@ tags:
4
  - sentence-transformers
5
  - feature-extraction
6
  - sentence-similarity
 
7
 
8
  ---
9
 
10
  # {MODEL_NAME}
11
 
12
- This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search.
13
 
14
  <!--- Describe your model here -->
15
 
@@ -34,6 +35,41 @@ print(embeddings)
34
 
35
 
36
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
37
  ## Evaluation Results
38
 
39
  <!--- Describe how your model was evaluated -->
@@ -77,9 +113,8 @@ Parameters of the fit()-Method:
77
  ## Full Model Architecture
78
  ```
79
  SentenceTransformer(
80
- (0): Transformer({'max_seq_length': 350, 'do_lower_case': False}) with Transformer model: MPNetModel
81
- (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
82
- (2): Normalize()
83
  )
84
  ```
85
 
 
4
  - sentence-transformers
5
  - feature-extraction
6
  - sentence-similarity
7
+ - transformers
8
 
9
  ---
10
 
11
  # {MODEL_NAME}
12
 
13
+ This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 1024 dimensional dense vector space and can be used for tasks like clustering or semantic search.
14
 
15
  <!--- Describe your model here -->
16
 
 
35
 
36
 
37
 
38
+ ## Usage (HuggingFace Transformers)
39
+ Without [sentence-transformers](https://www.SBERT.net), you can use the model like this: First, you pass your input through the transformer model, then you have to apply the right pooling-operation on-top of the contextualized word embeddings.
40
+
41
+ ```python
42
+ from transformers import AutoTokenizer, AutoModel
43
+ import torch
44
+
45
+
46
+ def cls_pooling(model_output, attention_mask):
47
+ return model_output[0][:,0]
48
+
49
+
50
+ # Sentences we want sentence embeddings for
51
+ sentences = ['This is an example sentence', 'Each sentence is converted']
52
+
53
+ # Load model from HuggingFace Hub
54
+ tokenizer = AutoTokenizer.from_pretrained('{MODEL_NAME}')
55
+ model = AutoModel.from_pretrained('{MODEL_NAME}')
56
+
57
+ # Tokenize sentences
58
+ encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')
59
+
60
+ # Compute token embeddings
61
+ with torch.no_grad():
62
+ model_output = model(**encoded_input)
63
+
64
+ # Perform pooling. In this case, cls pooling.
65
+ sentence_embeddings = cls_pooling(model_output, encoded_input['attention_mask'])
66
+
67
+ print("Sentence embeddings:")
68
+ print(sentence_embeddings)
69
+ ```
70
+
71
+
72
+
73
  ## Evaluation Results
74
 
75
  <!--- Describe how your model was evaluated -->
 
113
  ## Full Model Architecture
114
  ```
115
  SentenceTransformer(
116
+ (0): Transformer({'max_seq_length': 350, 'do_lower_case': True}) with Transformer model: BertModel
117
+ (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
 
118
  )
119
  ```
120