Add new SentenceTransformer model

README.md

@@ -12,42 +12,54 @@ tags:
 - retrieval
 - reranking
 - generated_from_trainer
-- dataset_size:478600
+- dataset_size:483820
 - loss:MultipleNegativesSymmetricRankingLoss
 base_model: Alibaba-NLP/gte-modernbert-base
 widget:
-- source_sentence: The brown dog is sniffing the back of a small black dog
+- source_sentence: In 2015 Adolf Hitler appeared in the kickstarter short movie ``
+    Kung Fury `` as Taccone ( A.K.A .
   sentences:
-  - Pickens died in Edgefield and was buried on the Willow Brook Cemetery in Edgefield
-    , South Carolina .
-  - It is notable as the oldest Chinatown in Australia , the oldest continuous Chinese
-    settlement in Australia , and the longest continuously running Chinatown outside
-    of Asia .
-  - There is no large brown dog and small grey dog standing on a rocky surface
-- source_sentence: Is it harmful from security perspectives to use public Wi-Fi?
+  - In 2015 , Adolf Hitler appeared in the Kickstarter - short film `` Kung Fury ``
+    as Taccone ( A.K.A .
+  - In 1795 , the only white residents were Dr. John Laidley and two brothers with
+    the surname Ainslie .
+  - The 125th University Match was played in March 2014 at the Rye Golf Club , Oxford
+    , East Sussex won the game 8.5 - 6.5 .
+- source_sentence: From 1973 to 1974 , Aubrey toured with the Cambridge Theatre Company
+    as Diggory in `` She Stoops to Conquer `` and again as Aguecheek .
   sentences:
-  - What is the best way to drive traffic to a website?
-  - What startups have used GitHub?
-  - Is there something wrong with using public Wi-Fi?
-- source_sentence: How can we make education better?
+  - Oxide can be reduced to metallic samarium at higher temperatures by heating with
+    a reducing agent such as hydrogen or carbon monoxide .
+  - From 1973 to 1974 Aguecheek toured with the Cambridge Theatre Company as Diggory
+    in `` You Stoops to Conquer `` and again as Aubrey .
+  - The medals were presented by Barry Maister , IOC member , New Zealand and Sarah
+    Webb Gosling , Vice President of World Sailing .
+- source_sentence: There is no official wall on the border , although there are sections
+    of fence near populated areas and continuous border crossings .
   sentences:
-  - What are some things that would make education better today?
-  - Mistery works full-time as a graffiti artist and is also Emcee / Rapper in the
-    Brethren group .
-  - Jammu Airport operates flights to many cities in India such as Delhi , Leh and
-    Srinagar .
-- source_sentence: So are you.
+  - The 2014 -- 15 Boston Bruins season was the 91st season for the National Hockey
+    League franchise that was established on November 1 , 1924 .
+  - He was trained by the Inghams and owned by John Hawkes .
+  - There is no continuous wall on the border , although there are fence sections
+    near populated areas and official border crossings .
+- source_sentence: Capital . `` The French established similar hill stations in Indochina
+    , such as Dalat built in 1921 .
   sentences:
-  - 'Brown said afterwards that he was surprised they had not scored five , and Astall
-    wrote in his newspaper column :'
-  - Just like yourself.
-  - How do I actually lose weight?
-- source_sentence: A group of boys are playing with a ball in front of a large door
-    made of wood
+  - Lubuk China is a small town in Alor Gajah District , Melaka , Malaysia . It is
+    situated near the border with Negeri Sembilan .
+  - The French established similar hill stations in Indochina , such as Dalat , built
+    in 1921 .
+  - John Potts ( or Pott ) was a doctor and colonial governor of Virginia in the Jamestown
+    settlement at Virginia Colony in the early 17th century .
+- source_sentence: The band pursued `` signals `` in January 2012 in three weeks ,
+    and drums were recorded in a day and a half .
   sentences:
-  - The children are playing in front of a large door
-  - What is the blind spot?
-  - What are some good techniques for controlling your anger?
+  - It was repaired at the beginning of the 20th century and is listed as closed in
+    our records .
+  - The band tracked `` Signals `` in three weeks in January 2012 . Drums were recorded
+    in a day and a half .
+  - Contributors include actor Anton LaVey , Satanist Christopher Lee , serial killer
+    expert Clive Barker , author Karen Greenlee , and necrophile Robert Ressler .
 datasets:
 - redis/langcache-sentencepairs-v1
 pipeline_tag: sentence-similarity
@@ -64,37 +76,6 @@ metrics:
 model-index:
 - name: Redis fine-tuned BiEncoder model for semantic caching on LangCache
   results:
-  - task:
-      type: binary-classification
-      name: Binary Classification
-    dataset:
-      name: val
-      type: val
-    metrics:
-    - type: cosine_accuracy
-      value: 0.9996860282574568
-      name: Cosine Accuracy
-    - type: cosine_accuracy_threshold
-      value: 0.4801735281944275
-      name: Cosine Accuracy Threshold
-    - type: cosine_f1
-      value: 0.9998429894802952
-      name: Cosine F1
-    - type: cosine_f1_threshold
-      value: 0.4801735281944275
-      name: Cosine F1 Threshold
-    - type: cosine_precision
-      value: 1.0
-      name: Cosine Precision
-    - type: cosine_recall
-      value: 0.9996860282574568
-      name: Cosine Recall
-    - type: cosine_ap
-      value: 0.9999999999999999
-      name: Cosine Ap
-    - type: cosine_mcc
-      value: 0.0
-      name: Cosine Mcc
   - task:
       type: binary-classification
       name: Binary Classification
@@ -103,28 +84,28 @@ model-index:
       type: test
     metrics:
     - type: cosine_accuracy
-      value: 0.9999627560521416
+      value: 0.7037777526966672
       name: Cosine Accuracy
     - type: cosine_accuracy_threshold
-      value: 0.42059871554374695
+      value: 0.8524033427238464
       name: Cosine Accuracy Threshold
     - type: cosine_f1
-      value: 0.9999813776792864
+      value: 0.7122170715871171
       name: Cosine F1
     - type: cosine_f1_threshold
-      value: 0.42059871554374695
+      value: 0.8118724822998047
       name: Cosine F1 Threshold
     - type: cosine_precision
-      value: 1.0
+      value: 0.5989283084033827
       name: Cosine Precision
     - type: cosine_recall
-      value: 0.9999627560521416
+      value: 0.8783612662942272
       name: Cosine Recall
     - type: cosine_ap
-      value: 1.0
+      value: 0.6476665223951498
       name: Cosine Ap
     - type: cosine_mcc
-      value: 0.0
+      value: 0.44182914870985407
       name: Cosine Mcc
 ---
 
@@ -178,9 +159,9 @@ from sentence_transformers import SentenceTransformer
 model = SentenceTransformer("redis/langcache-embed-v3")
 # Run inference
 sentences = [
-    'A group of boys are playing with a ball in front of a large door made of wood',
-    'The children are playing in front of a large door',
-    'What are some good techniques for controlling your anger?',
+    'The band pursued `` signals `` in January 2012 in three weeks , and drums were recorded in a day and a half .',
+    'The band tracked `` Signals `` in three weeks in January 2012 . Drums were recorded in a day and a half .',
+    'Contributors include actor Anton LaVey , Satanist Christopher Lee , serial killer expert Clive Barker , author Karen Greenlee , and necrophile Robert Ressler .',
 ]
 embeddings = model.encode(sentences)
 print(embeddings.shape)
@@ -189,9 +170,9 @@ print(embeddings.shape)
 # Get the similarity scores for the embeddings
 similarities = model.similarity(embeddings, embeddings)
 print(similarities)
-# tensor([[1.0000, 0.8672, 0.4121],
-#         [0.8672, 1.0000, 0.4219],
-#         [0.4121, 0.4219, 1.0000]], dtype=torch.bfloat16)
+# tensor([[0.9961, 0.9570, 0.4941],
+#         [0.9570, 0.9961, 0.5078],
+#         [0.4941, 0.5078, 1.0000]], dtype=torch.bfloat16)
 ```
 
 <!--
@@ -224,19 +205,19 @@ You can finetune this model on your own dataset.
 
 #### Binary Classification
 
-* Datasets: `val` and `test`
+* Dataset: `test`
 * Evaluated with [<code>BinaryClassificationEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.BinaryClassificationEvaluator)
 
-| Metric                    | val     | test    |
-|:--------------------------|:--------|:--------|
-| cosine_accuracy           | 0.9997  | 1.0     |
-| cosine_accuracy_threshold | 0.4802  | 0.4206  |
-| cosine_f1                 | 0.9998  | 1.0     |
-| cosine_f1_threshold       | 0.4802  | 0.4206  |
-| cosine_precision          | 1.0     | 1.0     |
-| cosine_recall             | 0.9997  | 1.0     |
-| **cosine_ap**             | **1.0** | **1.0** |
-| cosine_mcc                | 0.0     | 0.0     |
+| Metric                    | Value      |
+|:--------------------------|:-----------|
+| cosine_accuracy           | 0.7038     |
+| cosine_accuracy_threshold | 0.8524     |
+| cosine_f1                 | 0.7122     |
+| cosine_f1_threshold       | 0.8119     |
+| cosine_precision          | 0.5989     |
+| cosine_recall             | 0.8784     |
+| **cosine_ap**             | **0.6477** |
+| cosine_mcc                | 0.4418     |
 
 <!--
 ## Bias, Risks and Limitations
@@ -257,19 +238,19 @@ You can finetune this model on your own dataset.
 #### LangCache Sentence Pairs (all)
 
 * Dataset: [LangCache Sentence Pairs (all)](https://huggingface.co/datasets/redis/langcache-sentencepairs-v1)
-* Size: 26,850 training samples
-* Columns: <code>sentence1</code> and <code>sentence2</code>
+* Size: 62,021 training samples
+* Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>label</code>
 * Approximate statistics based on the first 1000 samples:
-  |         | sentence1                                                                         | sentence2                                                                         |
-  |:--------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|
-  | type    | string                                                                            | string                                                                            |
-  | details | <ul><li>min: 4 tokens</li><li>mean: 16.76 tokens</li><li>max: 44 tokens</li></ul> | <ul><li>min: 4 tokens</li><li>mean: 16.58 tokens</li><li>max: 44 tokens</li></ul> |
+  |         | sentence1                                                                         | sentence2                                                                         | label                                           |
+  |:--------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:------------------------------------------------|
+  | type    | string                                                                            | string                                                                            | int                                             |
+  | details | <ul><li>min: 8 tokens</li><li>mean: 27.46 tokens</li><li>max: 53 tokens</li></ul> | <ul><li>min: 9 tokens</li><li>mean: 27.36 tokens</li><li>max: 52 tokens</li></ul> | <ul><li>0: ~50.30%</li><li>1: ~49.70%</li></ul> |
 * Samples:
-  | sentence1                                                                              | sentence2                                                                                   |
-  |:---------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------|
-  | <code>A chef is preparing a meal</code>                                                | <code>Some food is being prepared by a chef</code>                                          |
-  | <code>The presentation is being watched by a classroom of students</code>              | <code>A classroom is full of students</code>                                                |
-  | <code>Garden River , located north of Garden River Airport , Alberta , Canada .</code> | <code>Garden River , , is located north of Garden River Airport , Alberta , Canada .</code> |
+  | sentence1                                                                                                                                   | sentence2                                                                                                                                     | label          |
+  |:--------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------|:---------------|
+  | <code>The newer Punts are still very much in existence today and race in the same fleets as the older boats .</code>                        | <code>The newer punts are still very much in existence today and run in the same fleets as the older boats .</code>                           | <code>1</code> |
+  | <code>Turner Valley , was at the Turner Valley Bar N Ranch Airport , southwest of the Turner Valley Bar N Ranch , Alberta , Canada .</code> | <code>Turner Valley Bar N Ranch Airport , , was located at Turner Valley Bar N Ranch , southwest of Turner Valley , Alberta , Canada .</code> | <code>0</code> |
+  | <code>After losing his second election , he resigned as opposition leader and was replaced by Geoff Pearsall .</code>                       | <code>Max Bingham resigned as opposition leader after losing his second election , and was replaced by Geoff Pearsall .</code>                | <code>1</code> |
 * Loss: [<code>MultipleNegativesSymmetricRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativessymmetricrankingloss) with these parameters:
   ```json
   {
@@ -284,19 +265,19 @@ You can finetune this model on your own dataset.
 #### LangCache Sentence Pairs (all)
 
 * Dataset: [LangCache Sentence Pairs (all)](https://huggingface.co/datasets/redis/langcache-sentencepairs-v1)
-* Size: 26,850 evaluation samples
-* Columns: <code>sentence1</code> and <code>sentence2</code>
+* Size: 62,021 evaluation samples
+* Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>label</code>
 * Approximate statistics based on the first 1000 samples:
-  |         | sentence1                                                                         | sentence2                                                                         |
-  |:--------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|
-  | type    | string                                                                            | string                                                                            |
-  | details | <ul><li>min: 4 tokens</li><li>mean: 16.76 tokens</li><li>max: 44 tokens</li></ul> | <ul><li>min: 4 tokens</li><li>mean: 16.58 tokens</li><li>max: 44 tokens</li></ul> |
+  |         | sentence1                                                                         | sentence2                                                                         | label                                           |
+  |:--------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:------------------------------------------------|
+  | type    | string                                                                            | string                                                                            | int                                             |
+  | details | <ul><li>min: 8 tokens</li><li>mean: 27.46 tokens</li><li>max: 53 tokens</li></ul> | <ul><li>min: 9 tokens</li><li>mean: 27.36 tokens</li><li>max: 52 tokens</li></ul> | <ul><li>0: ~50.30%</li><li>1: ~49.70%</li></ul> |
 * Samples:
-  | sentence1                                                                              | sentence2                                                                                   |
-  |:---------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------|
-  | <code>A chef is preparing a meal</code>                                                | <code>Some food is being prepared by a chef</code>                                          |
-  | <code>The presentation is being watched by a classroom of students</code>              | <code>A classroom is full of students</code>                                                |
-  | <code>Garden River , located north of Garden River Airport , Alberta , Canada .</code> | <code>Garden River , , is located north of Garden River Airport , Alberta , Canada .</code> |
+  | sentence1                                                                                                                                   | sentence2                                                                                                                                     | label          |
+  |:--------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------|:---------------|
+  | <code>The newer Punts are still very much in existence today and race in the same fleets as the older boats .</code>                        | <code>The newer punts are still very much in existence today and run in the same fleets as the older boats .</code>                           | <code>1</code> |
+  | <code>Turner Valley , was at the Turner Valley Bar N Ranch Airport , southwest of the Turner Valley Bar N Ranch , Alberta , Canada .</code> | <code>Turner Valley Bar N Ranch Airport , , was located at Turner Valley Bar N Ranch , southwest of Turner Valley , Alberta , Canada .</code> | <code>0</code> |
+  | <code>After losing his second election , he resigned as opposition leader and was replaced by Geoff Pearsall .</code>                       | <code>Max Bingham resigned as opposition leader after losing his second election , and was replaced by Geoff Pearsall .</code>                | <code>1</code> |
 * Loss: [<code>MultipleNegativesSymmetricRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativessymmetricrankingloss) with these parameters:
   ```json
   {
@@ -307,9 +288,9 @@ You can finetune this model on your own dataset.
   ```
 
 ### Training Logs
-| Epoch | Step | val_cosine_ap | test_cosine_ap |
-|:-----:|:----:|:-------------:|:--------------:|
-| -1    | -1   | 1.0000        | 1.0            |
+| Epoch | Step | test_cosine_ap |
+|:-----:|:----:|:--------------:|
+| -1    | -1   | 0.6477         |
 
 
 ### Framework Versions