Integrate with Sentence Transformers (#2)
Browse files- Integrate with Sentence Transformers (071e1bb3c6657116ca95005ca54faa2d8ca39798)
- Reduce to Sentence Transformers 2.3.1 (e5f16c718bef58eb279aa75c89541c56103bc4a8)
- 1_Pooling/config.json +9 -0
- README.md +21 -1
- config_sentence_transformers.json +7 -0
- modules.json +20 -0
- sentence_bert_config.json +4 -0
1_Pooling/config.json
ADDED
@@ -0,0 +1,9 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
{
|
2 |
+
"word_embedding_dimension": 384,
|
3 |
+
"pooling_mode_cls_token": false,
|
4 |
+
"pooling_mode_mean_tokens": true,
|
5 |
+
"pooling_mode_max_tokens": false,
|
6 |
+
"pooling_mode_mean_sqrt_len_tokens": false,
|
7 |
+
"pooling_mode_weightedmean_tokens": false,
|
8 |
+
"pooling_mode_lasttoken": false
|
9 |
+
}
|
README.md
CHANGED
@@ -7,6 +7,8 @@ tags:
|
|
7 |
- String Matching
|
8 |
- Fuzzy Join
|
9 |
- Entity Retrieval
|
|
|
|
|
10 |
---
|
11 |
## PEARL-small
|
12 |
[Learning High-Quality and General-Purpose Phrase Representations](https://arxiv.org/pdf/2401.10407.pdf). <br>
|
@@ -46,7 +48,25 @@ The FastText model here is `crawl-300d-2M-subword.bin`.
|
|
46 |
|
47 |
## Usage
|
48 |
|
49 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
50 |
|
51 |
```python
|
52 |
import torch.nn.functional as F
|
|
|
7 |
- String Matching
|
8 |
- Fuzzy Join
|
9 |
- Entity Retrieval
|
10 |
+
- transformers
|
11 |
+
- sentence-transformers
|
12 |
---
|
13 |
## PEARL-small
|
14 |
[Learning High-Quality and General-Purpose Phrase Representations](https://arxiv.org/pdf/2401.10407.pdf). <br>
|
|
|
48 |
|
49 |
## Usage
|
50 |
|
51 |
+
### Sentence Transformers
|
52 |
+
PEARL is integrated with the Sentence Transformers library, and can be used like so:
|
53 |
+
|
54 |
+
```python
|
55 |
+
from sentence_transformers import SentenceTransformer, util
|
56 |
+
|
57 |
+
query_texts = ["The New York Times"]
|
58 |
+
doc_texts = [ "NYTimes", "New York Post", "New York"]
|
59 |
+
input_texts = query_texts + doc_texts
|
60 |
+
|
61 |
+
model = SentenceTransformer("Lihuchen/pearl_small")
|
62 |
+
embeddings = model.encode(input_texts)
|
63 |
+
scores = util.cos_sim(embeddings[0], embeddings[1:]) * 100
|
64 |
+
print(scores.tolist())
|
65 |
+
# [[90.56318664550781, 79.65763854980469, 75.52056121826172]]
|
66 |
+
```
|
67 |
+
|
68 |
+
### Transformers
|
69 |
+
You can also use `transformers` to use PEARL. Below is an example of entity retrieval, and we reuse the code from E5.
|
70 |
|
71 |
```python
|
72 |
import torch.nn.functional as F
|
config_sentence_transformers.json
ADDED
@@ -0,0 +1,7 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
{
|
2 |
+
"__version__": {
|
3 |
+
"sentence_transformers": "2.3.1",
|
4 |
+
"transformers": "4.37.0",
|
5 |
+
"pytorch": "2.1.0+cu121"
|
6 |
+
}
|
7 |
+
}
|
modules.json
ADDED
@@ -0,0 +1,20 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
[
|
2 |
+
{
|
3 |
+
"idx": 0,
|
4 |
+
"name": "0",
|
5 |
+
"path": "",
|
6 |
+
"type": "sentence_transformers.models.Transformer"
|
7 |
+
},
|
8 |
+
{
|
9 |
+
"idx": 1,
|
10 |
+
"name": "1",
|
11 |
+
"path": "1_Pooling",
|
12 |
+
"type": "sentence_transformers.models.Pooling"
|
13 |
+
},
|
14 |
+
{
|
15 |
+
"idx": 2,
|
16 |
+
"name": "2",
|
17 |
+
"path": "2_Normalize",
|
18 |
+
"type": "sentence_transformers.models.Normalize"
|
19 |
+
}
|
20 |
+
]
|
sentence_bert_config.json
ADDED
@@ -0,0 +1,4 @@
|
|
|
|
|
|
|
|
|
|
|
1 |
+
{
|
2 |
+
"max_seq_length": 512,
|
3 |
+
"do_lower_case": false
|
4 |
+
}
|