Repository: pybamm-team/PyBaMM

Issue:
Improve `search` for improved handling of short or incomplete search terms
The current `search` function doesn't provide relevant results for short or partial search terms. When searching with short terms like "c", "co", or "con" etc, the function returns broad and often irrelevant  As discussed in this [comment](https://github.com/pybamm-team/PyBaMM/pull/4650#pullrequestreview-2504538157).

Discussion:
codecov[bot]: ## [Codecov](https://app.codecov.io/gh/pybamm-team/PyBaMM/pull/4735?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pybamm-team) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 99.25%. Comparing base [(`a7253b8`)](https://app.codecov.io/gh/pybamm-team/PyBaMM/commit/a7253b8cb6338e46e1468be1769602771c25e0c7?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pybamm-team) to head [(`23a7855`)](https://app.codecov.io/gh/pybamm-team/PyBaMM/commit/23a7855c35b477b553bdaafd78c64cc0b4d94ba4?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pybamm-team).
> Report is 19 commits behind head on develop.

<details><summary>Additional details and impacted files</summary>


```diff
@@             Coverage Diff             @@
##           develop    #4735      +/-   ##
===========================================
+ Coverage    99.22%   99.25%   +0.02%     
===========================================
  Files          303      303              
  Lines        23070    23250     +180     
===========================================
+ Hits         22891    23076     +185     
+ Misses         179      174       -5     
```

</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/pybamm-team/PyBaMM/pull/4735?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pybamm-team).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pybamm-team).

medha-14: Could I get a review on this one?
kratman: @medha-14 Sorry, there is a bit of a backlog due to the upcoming release and everyone coming back from vacation. Don't worry, we will review this shortly
medha-14: > Thanks! I think `min_similarity` could be slightly higher here because the threshold here is too low and can lead to false positives in the search. But, at the same time, being able to resolve potential typos in the search query would require a lower threshold. This makes me feel that we could expose it with a sensible default, but I don't yet know what a sensible default would be. It should be higher than the current 40%, though. In-line comment about this below:

Sorry for the delayed response. I think it's important to clarify that `min_similarity` is only relevant for substring matches when the search_key is found within the known keys. For cases where there are typos or no substring matches, `difflib.get_close_matches()` handles those independently, using its own cutoff threshold. Increasing the min_similarity too high would make it difficult to get even the relevant matches. 
Eg: If we even set the threshold to 0.6 and a user searches for `conc` he will not be matched with `concentration` because they will have similarity ratio of 0.47(approximately) which will not qualify the threshold.
medha-14: > `min_similarity` is also defined above, so it gets defined twice. How about making it an argument for `_find_matches()`?
> 
> Follow-up question: do you think it would make sense to expose it publicly for users through `search()` as well?

I’ve made `min_similarity` an argument for `_find_matches()` and also exposed it to users via `search()`, it could be useful to control the threshold according to their needs.
agriyakhetarpal: Actually, could you please add a test where `min_similarity` is used, for the cases where `significant_match` becomes True or False? It is currently in untested waters.
medha-14: I have added appropriate tests  that cover cases where we give different values to `min_similarity` producing different results.
Saransh-cpp: Requesting @agriyakhetarpal's review on this.
agriyakhetarpal: This is good to merge from my side. I'm mostly waiting on @Saransh-cpp's review as he self-requested one.

Patch:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 1df6231fb..c29a75cfb 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -15,6 +15,10 @@
 
 - Fixed interpolation bug in `pybamm.QuickPlot` with spatial variables. ([#4841](https://github.com/pybamm-team/PyBaMM/pull/4841))
 
+## Optimizations
+
+- Improved search to handle cases with shorter input strings and provide more relevant results. ([#4735](https://github.com/pybamm-team/PyBaMM/pull/4735))
+
 # [v25.1.1](https://github.com/pybamm-team/PyBaMM/tree/v25.1.1) - 2025-01-20
 
 ## Features
diff --git a/src/pybamm/util.py b/src/pybamm/util.py
index 11580967f..dcab37b0d 100644
--- a/src/pybamm/util.py
+++ b/src/pybamm/util.py
@@ -91,7 +91,9 @@ class FuzzyDict(dict):
                 f"'{key}' not found. Best matches are {best_matches}"
             ) from error
 
-    def _find_matches(self, search_key: str, known_keys: list[str]):
+    def _find_matches(
+        self, search_key: str, known_keys: list[str], min_similarity: float = 0.4
+    ):
         """
         Helper method to find exact and partial matches for a given search key.
 
@@ -101,13 +103,37 @@ class FuzzyDict(dict):
             The term to search for in the keys.
         known_keys : list of str
             The list of known dictionary keys to search within.
-
+        min_similarity : float, optional
+            The minimum similarity threshold for a match.
+            Default is 0.4
         """
-        exact = [key for key in known_keys if search_key in key.lower()]
-        partial = difflib.get_close_matches(search_key, known_keys, n=5, cutoff=0.5)
-        return exact, partial
+        search_key = search_key.lower()
+        exact_matches = []
+        partial_matches = []
+
+        for key in known_keys:
+            key_lower = key.lower()
+            if search_key in key_lower:
+                key_words = key_lower.split()
+
+                for word in key_words:
+                    similarity = difflib.SequenceMatcher(None, search_key, word).ratio()
 
-    def search(self, keys: str | list[str], print_values: bool = False):
+                    if similarity >= min_similarity:
+                        exact_matches.append(key)
+
+            else:
+                partial_matches = difflib.get_close_matches(
+                    search_key, known_keys, n=5, cutoff=0.5
+                )
+        return exact_matches, partial_matches
+
+    def search(
+        self,
+        keys: str | list[str],
+        print_values: bool = False,
+        min_similarity: float = 0.4,
+    ):
         """
         Search dictionary for keys containing all terms in 'keys'.
         If print_values is True, both the keys and values will be printed.
@@ -121,6 +147,9 @@ class FuzzyDict(dict):
         print_values : bool, optional
             If True, print both keys and values. Otherwise, print only keys.
             Default is False.
+        min_similarity : float, optional
+            The minimum similarity threshold for a match.
+            Default is 0.4
         """
 
         if not isinstance(keys, (str, list)) or not all(
@@ -145,14 +174,23 @@ class FuzzyDict(dict):
             search_keys = [k.strip().lower() for k in keys if k.strip()]
 
         known_keys = list(self.keys())
-        known_keys.sort()
-
         # Check for exact matches where all search keys appear together in a key
-        exact_matches = [
-            key
-            for key in known_keys
-            if all(term in key.lower() for term in search_keys)
-        ]
+        exact_matches = []
+        for key in known_keys:
+            key_lower = key.lower()
+            if all(term in key_lower for term in search_keys):
+                key_words = key_lower.split()
+
+                # Ensure all search terms match at least one word in the key
+                if all(
+                    any(
+                        difflib.SequenceMatcher(None, term, word).ratio()
+                        >= min_similarity
+                        for word in key_words
+                    )
+                    for term in search_keys
+                ):
+                    exact_matches.append(key)
 
         if exact_matches:
             print(
@@ -166,7 +204,7 @@ class FuzzyDict(dict):
         # If no exact matches, iterate over search keys individually
         for original_key, search_key in zip(original_keys, search_keys):
             exact_key_matches, partial_matches = self._find_matches(
-                search_key, known_keys
+                search_key, known_keys, min_similarity
             )
 
             if exact_key_matches:


Test patch:
diff --git a/tests/unit/test_util.py b/tests/unit/test_util.py
index f0ff82767..b71a5316e 100644
--- a/tests/unit/test_util.py
+++ b/tests/unit/test_util.py
@@ -264,3 +264,22 @@ class TestSearch:
             match="'keys' must be a string or a list of strings, got <class 'int'>",
         ):
             model.variables.search(123)
+
+        # Test smaller strings
+        with mocker.patch("sys.stdout", new=StringIO()) as fake_out:
+            model.variables.search(["El", "co"], print_values=True)
+            out = "No matches found for 'El'\nNo matches found for 'co'\n"
+            assert fake_out.getvalue() == out
+
+        # Case where min_similarity is high (0.9)
+        with mocker.patch("sys.stdout", new=StringIO()) as fake_out:
+            model.variables.search("electro", min_similarity=0.9)
+            assert fake_out.getvalue() == "No matches found for 'electro'\n"
+
+        # Case where min_similarity is low (0.3)
+        with mocker.patch("sys.stdout", new=StringIO()) as fake_out:
+            model.variables.search("electro", min_similarity=0.3)
+            assert (
+                fake_out.getvalue()
+                == "Results for 'electro': ['Electrolyte concentration', 'Electrode potential']\n"
+            )