Lihuchen commited on
Commit
dda363f
1 Parent(s): 220472f

Integrate with Sentence Transformers (#2)

Browse files

- Integrate with Sentence Transformers (071e1bb3c6657116ca95005ca54faa2d8ca39798)
- Reduce to Sentence Transformers 2.3.1 (e5f16c718bef58eb279aa75c89541c56103bc4a8)

1_Pooling/config.json ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "word_embedding_dimension": 384,
3
+ "pooling_mode_cls_token": false,
4
+ "pooling_mode_mean_tokens": true,
5
+ "pooling_mode_max_tokens": false,
6
+ "pooling_mode_mean_sqrt_len_tokens": false,
7
+ "pooling_mode_weightedmean_tokens": false,
8
+ "pooling_mode_lasttoken": false
9
+ }
README.md CHANGED
@@ -7,6 +7,8 @@ tags:
7
  - String Matching
8
  - Fuzzy Join
9
  - Entity Retrieval
 
 
10
  ---
11
  ## PEARL-small
12
  [Learning High-Quality and General-Purpose Phrase Representations](https://arxiv.org/pdf/2401.10407.pdf). <br>
@@ -46,7 +48,25 @@ The FastText model here is `crawl-300d-2M-subword.bin`.
46
 
47
  ## Usage
48
 
49
- Below is an example of entity retrieval, and we reuse the code from E5.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
50
 
51
  ```python
52
  import torch.nn.functional as F
 
7
  - String Matching
8
  - Fuzzy Join
9
  - Entity Retrieval
10
+ - transformers
11
+ - sentence-transformers
12
  ---
13
  ## PEARL-small
14
  [Learning High-Quality and General-Purpose Phrase Representations](https://arxiv.org/pdf/2401.10407.pdf). <br>
 
48
 
49
  ## Usage
50
 
51
+ ### Sentence Transformers
52
+ PEARL is integrated with the Sentence Transformers library, and can be used like so:
53
+
54
+ ```python
55
+ from sentence_transformers import SentenceTransformer, util
56
+
57
+ query_texts = ["The New York Times"]
58
+ doc_texts = [ "NYTimes", "New York Post", "New York"]
59
+ input_texts = query_texts + doc_texts
60
+
61
+ model = SentenceTransformer("Lihuchen/pearl_small")
62
+ embeddings = model.encode(input_texts)
63
+ scores = util.cos_sim(embeddings[0], embeddings[1:]) * 100
64
+ print(scores.tolist())
65
+ # [[90.56318664550781, 79.65763854980469, 75.52056121826172]]
66
+ ```
67
+
68
+ ### Transformers
69
+ You can also use `transformers` to use PEARL. Below is an example of entity retrieval, and we reuse the code from E5.
70
 
71
  ```python
72
  import torch.nn.functional as F
config_sentence_transformers.json ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ {
2
+ "__version__": {
3
+ "sentence_transformers": "2.3.1",
4
+ "transformers": "4.37.0",
5
+ "pytorch": "2.1.0+cu121"
6
+ }
7
+ }
modules.json ADDED
@@ -0,0 +1,20 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [
2
+ {
3
+ "idx": 0,
4
+ "name": "0",
5
+ "path": "",
6
+ "type": "sentence_transformers.models.Transformer"
7
+ },
8
+ {
9
+ "idx": 1,
10
+ "name": "1",
11
+ "path": "1_Pooling",
12
+ "type": "sentence_transformers.models.Pooling"
13
+ },
14
+ {
15
+ "idx": 2,
16
+ "name": "2",
17
+ "path": "2_Normalize",
18
+ "type": "sentence_transformers.models.Normalize"
19
+ }
20
+ ]
sentence_bert_config.json ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ {
2
+ "max_seq_length": 512,
3
+ "do_lower_case": false
4
+ }