Repository: foreai-co/fore

Issue:
Replace SIMILARITY with REFERENCE_FACT_RECALL
Let's remove all mentions of the SIMILARITY metric from the client library

Discussion:

Patch:
diff --git a/README.md b/README.md
index 85abaf4..ef8b371 100644
--- a/README.md
+++ b/README.md
@@ -45,7 +45,8 @@ Check our documentation at https://docs.foreai.co.
 
     run_config = EvalRunConfig(evalset_id="programming-languages",
                             experiment_id="my-smart-llm",
-                            metrics=[MetricType.GROUNDEDNESS, MetricType.SIMILARITY])
+                            metrics=[MetricType.GROUNDEDNESS, 
+                                     MetricType.REFERENCE_FACT_RECALL])
 
 
     def my_generate_fn(query: str) -> InferenceOutput:
@@ -64,6 +65,6 @@ Check our documentation at https://docs.foreai.co.
 
 We currently offer two metrics:
 * [Groundedness](https://docs.foreai.co/docs/foresight/metrics/groundedness)
-* [Ground Truth Similarty](https://docs.foreai.co/docs/foresight/metrics/ground_truth_similarity)
+* [Reference Fact Recall](https://docs.foreai.co/docs/foresight/metrics/reference_fact_recall)
 
 Check [here](https://docs.foreai.co/docs/category/metrics) for more information.
diff --git a/fore/foresight/api_v1.yaml b/fore/foresight/api_v1.yaml
index ca5095f..85757a7 100644
--- a/fore/foresight/api_v1.yaml
+++ b/fore/foresight/api_v1.yaml
@@ -780,7 +780,7 @@ components:
       type: string
       enum:
         - GROUNDEDNESS
-        - SIMILARITY
+        - REFERENCE_FACT_RECALL
     EvalsetEntry:
       type: object
       properties:
diff --git a/fore/foresight/schema.py b/fore/foresight/schema.py
index 6289bcb..b3edf82 100644
--- a/fore/foresight/schema.py
+++ b/fore/foresight/schema.py
@@ -65,16 +65,12 @@ class MetricType(str, Enum):
     COMPLETENESS = "COMPLETENESS"
     # Is the response based on a provided context?
     GROUNDEDNESS = "GROUNDEDNESS"
-    # How similar is the response to the reference answer?
-    SIMILARITY = "SIMILARITY"
     # How many reference facts are entailed in the candidate answer?
     REFERENCE_FACT_RECALL = "REFERENCE_FACT_RECALL"
 
     def is_implemented(self):
-        return self in {
-            MetricType.GROUNDEDNESS, MetricType.SIMILARITY,
-            MetricType.REFERENCE_FACT_RECALL
-        }
+        return self in {MetricType.GROUNDEDNESS,
+                        MetricType.REFERENCE_FACT_RECALL}
 
 
 class EvalRunConfig(BaseModel):


Test patch:
diff --git a/fore/foresight/client_test.py b/fore/foresight/client_test.py
index f679b74..830731d 100644
--- a/fore/foresight/client_test.py
+++ b/fore/foresight/client_test.py
@@ -453,7 +453,7 @@ class TestForeSight(unittest.TestCase):
                 },
                 "metric_values": {
                     MetricType.GROUNDEDNESS: 0.98,
-                    MetricType.SIMILARITY: 0.8,
+                    MetricType.REFERENCE_FACT_RECALL: 0.8,
                 } if metrics_are_computed else {}
             }]
         }
@@ -508,11 +508,8 @@ class TestForeSight(unittest.TestCase):
             "groundedness": {
                 0: 0.98
             },
-            "similarity": {
-                0: 0.8
-            },
             "reference_fact_recall": {
-                0: None
+                0: 0.8
             }
         }
 
@@ -565,9 +562,6 @@ class TestForeSight(unittest.TestCase):
             "groundedness": {
                 0: None
             },
-            "similarity": {
-                0: None
-            },
             "reference_fact_recall": {
                 0: None
             }