Update README.md
Browse files
README.md
CHANGED
@@ -59,21 +59,18 @@ documents = [
|
|
59 |
]
|
60 |
input_texts = queries + documents
|
61 |
|
62 |
-
|
63 |
max_length = 512
|
64 |
# Tokenize the input texts
|
65 |
batch_dict = tokenizer(input_texts, max_length=max_length, padding=True, truncation=True, return_tensors='pt')
|
66 |
|
67 |
model.eval()
|
68 |
with torch.no_grad():
|
69 |
-
outputs = model(**batch_dict)
|
70 |
-
embeddings = last_token_pool(outputs.last_hidden_state, batch_dict['attention_mask'])
|
71 |
-
|
72 |
```
|
73 |
|
74 |
Then similarity scores between the different sentences are obtained with a dot product between the embeddings:
|
75 |
```python
|
76 |
-
|
77 |
scores = (embeddings[:2] @ embeddings[2:].T)
|
78 |
print(scores.tolist())
|
79 |
```
|
|
|
59 |
]
|
60 |
input_texts = queries + documents
|
61 |
|
|
|
62 |
max_length = 512
|
63 |
# Tokenize the input texts
|
64 |
batch_dict = tokenizer(input_texts, max_length=max_length, padding=True, truncation=True, return_tensors='pt')
|
65 |
|
66 |
model.eval()
|
67 |
with torch.no_grad():
|
68 |
+
outputs = model(**batch_dict)
|
69 |
+
embeddings = last_token_pool(outputs.last_hidden_state, batch_dict['attention_mask'])
|
|
|
70 |
```
|
71 |
|
72 |
Then similarity scores between the different sentences are obtained with a dot product between the embeddings:
|
73 |
```python
|
|
|
74 |
scores = (embeddings[:2] @ embeddings[2:].T)
|
75 |
print(scores.tolist())
|
76 |
```
|