Repository: Zeroto521/my-data-toolkit

Issue:
ENH: New accessor for Series to calculate text distance
<!--
Thanks for contributing a pull request!

Please follow these standard acronyms to start the commit message:

- ENH: enhancement
- BUG: bug fix
- DOC: documentation
- TYP: type annotations
- TST: addition or modification of tests
- MAINT: maintenance commit (refactoring, typos, etc.)
- BLD: change related to building
- REL: related to releasing
- API: an (incompatible) API change
- DEP: deprecate something, or remove a deprecated object
- DEV: development tool or utility
- REV: revert an earlier commit
- PERF: performance improvement
- BOT: always commit via a bot
- CI: related to CI or CD
- CLN: Code cleanup
-->

- [ ] closes #xxxx
- [x] whatsnew entry

two accessors for Series
- one to generate paired distances
- another to generate pairwise distances

```python
    >>> import dtoolkit.accessor
    >>> import pandas as pd
    >>> s = pd.Series(["hello", "world"])
    >>> s
    0    hello
    1    world
    dtype: object
    >>> s.textdistance("python")
    0    36
    1    18
    dtype: int64
```


Discussion:
# [Codecov](https://codecov.io/gh/Zeroto521/my-data-toolkit/pull/724?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=40%25) Report
> Merging [#724](https://codecov.io/gh/Zeroto521/my-data-toolkit/pull/724?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=40%25) (82a1056) into [main](https://codecov.io/gh/Zeroto521/my-data-toolkit/commit/33cee075737aa1e2a55725e841ce7fd15bc9b752?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=40%25) (33cee07) will **increase** coverage by `0.01%`.
> The diff coverage is `100.00%`.

```diff
@@            Coverage Diff             @@
##             main     #724      +/-   ##
==========================================
+ Coverage   99.61%   99.62%   +0.01%     
==========================================
  Files         113      115       +2     
  Lines        1550     1607      +57     
  Branches      353      370      +17     
==========================================
+ Hits         1544     1601      +57     
  Misses          1        1              
  Partials        5        5              
```


| [Impacted Files](https://codecov.io/gh/Zeroto521/my-data-toolkit/pull/724?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=40%25) | Coverage Δ | |
|---|---|---|
| [dtoolkit/accessor/series/textdistance.py](https://codecov.io/gh/Zeroto521/my-data-toolkit/pull/724/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=40%25#diff-ZHRvb2xraXQvYWNjZXNzb3Ivc2VyaWVzL3RleHRkaXN0YW5jZS5weQ==) | `100.00% <100.00%> (ø)` | |
| [dtoolkit/accessor/series/textdistance\_matrix.py](https://codecov.io/gh/Zeroto521/my-data-toolkit/pull/724/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=40%25#diff-ZHRvb2xraXQvYWNjZXNzb3Ivc2VyaWVzL3RleHRkaXN0YW5jZV9tYXRyaXgucHk=) | `100.00% <100.00%> (ø)` | |

:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=40%25)


Resolving patch:
diff --git a/doc/source/conf.py b/doc/source/conf.py
--- a/doc/source/conf.py
+++ b/doc/source/conf.py
@@ -129,6 +129,7 @@
     "pygeos": ("https://pygeos.readthedocs.io/en/stable/", None),
     "joblib": ("https://joblib.readthedocs.io/en/latest/", None),
     "h3": ("https://uber.github.io/h3-py/", None),
+    "rapidfuzz": ("https://maxbachmann.github.io/RapidFuzz/", None),
 }
 
 # extlinks alias
diff --git a/dtoolkit/accessor/series/textdistance.py b/dtoolkit/accessor/series/textdistance.py
--- a/dtoolkit/accessor/series/textdistance.py
+++ b/dtoolkit/accessor/series/textdistance.py
@@ -1,11 +1,9 @@
 from __future__ import annotations
 
 from functools import lru_cache
-from functools import wraps
 from typing import Callable
 from warnings import warn
 
-import numpy as np
 import pandas as pd
 from pandas.api.types import is_list_like
 from pandas.api.types import is_string_dtype
@@ -18,9 +16,10 @@
 def textdistance(
     s: pd.Series,
     /,
-    other: None | str | pd.Series = None,
+    other: str | pd.Series,
     method: Callable = None,
     align: bool = True,
+    **kwargs,
 ) -> pd.Series:
     """
     Return a ``Series`` containing the text distance to aligned ``other``.
@@ -28,7 +27,6 @@ def textdistance(
     Parameters
     ----------
     other : None, str or Series
-        If None or nan, the text distance is 0.
 
     align : bool, default True
         If True, automatically aligns GeoSeries based on their indices. If False,
@@ -36,8 +34,12 @@ def textdistance(
 
     method : Callable, default None
         The method to calculate the distance. The first and second positional parameters
-        will be compared. If None, \
-`rapidfuzz.fuzz.ratio <https://maxbachmann.github.io/RapidFuzz/Usage/fuzz.html#ratio>`_.
+        will be compared. If None, :meth:`rapidfuzz.fuzz.ratio`. Recommended use methods
+        in :mod:`rapidfuzz.fuzz`, :mod:`rapidfuzz.string_metric`, and
+        :mod:`rapidfuzz.distance`.
+
+    **kwargs
+        Additional keyword arguments passed to ``method``.
 
     Returns
     -------
@@ -58,11 +60,14 @@ def textdistance(
 
     See Also
     --------
+    rapidfuzz.fuzz
+    rapidfuzz.string_metric
+    rapidfuzz.distance
     textdistance_matrix
 
     Notes
     -----
-    The distance of any value compared to nan or None is 0.
+    The result of comparing to None or nan value is depended on the ``method``.
 
     Examples
     --------
@@ -87,11 +92,16 @@ def textdistance(
         raise TypeError(f"Expected string dtype, but got {s.dtype!r}.")
 
     if method is None:
-        method = __import__("rapidfuzz.fuzz").fuzz.ratio
-    method = lru_cache(check_none(check_nan(method)))
+        method = __import__("rapidfuzz").fuzz.ratio
+    method = lru_cache(method)
+
+    if (
+        isinstance(other, str)
+        or other is None
+        or (not is_list_like(other) and pd.isna(other))
+    ):
+        return s.apply(method, args=(other,), **kwargs)
 
-    if isinstance(other, str):
-        return s.apply(method, args=(other,))
     elif isinstance(other, pd.Series):
         if not is_string_dtype(other):
             raise TypeError(f"Expected Series(string), but got {other.dtype!r}.")
@@ -104,37 +114,9 @@ def textdistance(
             raise ValueError(f"{s.size=} != {other.size=}.")
 
         return pd.Series(
-            (method(*xy) for xy in zip(s, other)),
+            (method(*xy, **kwargs) for xy in zip(s, other)),
             name=s.name,
             index=s.index,
         )
-    elif other is None or (not is_list_like(other) and pd.isna(other)):
-        # NOTE:
-        # - pd.na(Series) returns array-like of bool
-        #   to make sure pd.isna(other) returns bool
-        #   need to other is not array-like
-        # - compare to None or nan always returns 0
-        #   the behavior is following rapidfuzz.fuzz.ratio
-        return pd.Series(np.zeros(s.size), name=s.name, index=s.index)
 
     raise TypeError(f"Expected Series(string), but got {type(other).__name__!r}.")
-
-
-def check_none(func):
-    @wraps(func)
-    def decorator(*args, **kwargs):
-        # NOTE: compare to None always returns 0
-        # the behavior is following rapidfuzz.fuzz.ratio
-        return 0 if args[0] is None or args[1] is None else func(*args, **kwargs)
-
-    return decorator
-
-
-def check_nan(func):
-    @wraps(func)
-    def decorator(*args, **kwargs):
-        # NOTE: compare to nan always returns 0
-        # the behavior is following rapidfuzz.fuzz.ratio
-        return 0 if pd.isna(args[0]) or pd.isna(args[1]) else func(*args, **kwargs)
-
-    return decorator
diff --git a/dtoolkit/accessor/series/textdistance_matrix.py b/dtoolkit/accessor/series/textdistance_matrix.py
--- a/dtoolkit/accessor/series/textdistance_matrix.py
+++ b/dtoolkit/accessor/series/textdistance_matrix.py
@@ -1,12 +1,12 @@
 from __future__ import annotations
 
+from functools import wraps
 from typing import Callable
 
 import pandas as pd
 from pandas.api.types import is_string_dtype
 
 from dtoolkit.accessor.register import register_series_method
-from dtoolkit.accessor.series.textdistance import textdistance
 
 
 @register_series_method
@@ -15,6 +15,7 @@ def textdistance_matrix(
     /,
     other: None | pd.Series = None,
     method: Callable = None,
+    **kwargs,
 ) -> pd.DataFrame:
     """
     Returns a ``DataFrame`` containing the text distances matrix between in ``s``
@@ -27,8 +28,12 @@ def textdistance_matrix(
 
     method : Callable, default None
         The method to calculate the distance. The first and second positional parameters
-        will be compared. If None, \
-`rapidfuzz.fuzz.ratio <https://maxbachmann.github.io/RapidFuzz/Usage/fuzz.html#ratio>`_.
+        will be compared. If None, :meth:`rapidfuzz.fuzz.ratio`. Recommended use methods
+        in :mod:`rapidfuzz.fuzz`, :mod:`rapidfuzz.string_metric`, and
+        :mod:`rapidfuzz.distance`.
+
+    **kwargs
+        Additional keyword arguments passed to ``method``.
 
     Returns
     -------
@@ -46,8 +51,15 @@ def textdistance_matrix(
 
     See Also
     --------
+    rapidfuzz.fuzz
+    rapidfuzz.string_metric
+    rapidfuzz.distance
     textdistance
 
+    Notes
+    -----
+    The result of comparing to None or nan value is depended on the ``method``.
+
     Examples
     --------
     >>> import dtoolkit
@@ -62,6 +74,10 @@ def textdistance_matrix(
     0  100.0  36.363636
     1   20.0  18.181818
     """
+    from rapidfuzz.process import cdist
+
+    if method is None:
+        method = check_nan(__import__("rapidfuzz").fuzz.ratio)
 
     if other is None:
         other = s.copy()
@@ -70,8 +86,18 @@ def textdistance_matrix(
     if not is_string_dtype(other):
         raise TypeError(f"Expected Series(string), but got {other.dtype!r}.")
 
-    return pd.concat(
-        (textdistance(s, o, method=method) for o in other),
-        axis=1,
-        keys=other.index,
+    return pd.DataFrame(
+        cdist(s, other, scorer=method, workers=-1, **kwargs),
+        index=s.index,
+        columns=other.index,
     )
+
+
+def check_nan(func):
+    @wraps(func)
+    def decorator(*args, **kwargs):
+        # NOTE: compare to nan always returns 0
+        # the behavior is following rapidfuzz.fuzz.ratio
+        return 0 if pd.isna(args[0]) or pd.isna(args[1]) else func(*args, **kwargs)
+
+    return decorator


Test patch:
diff --git a/test/accessor/series/test_textdistance.py b/test/accessor/series/test_textdistance.py
--- a/test/accessor/series/test_textdistance.py
+++ b/test/accessor/series/test_textdistance.py
@@ -5,7 +5,7 @@
 from dtoolkit.accessor.series import textdistance
 
 
-pytest.importorskip("rapidfuzz")
+rapidfuzz = pytest.importorskip("rapidfuzz")
 
 
 @pytest.mark.parametrize(
@@ -97,10 +97,11 @@ def test_error(s, other, align, error):
             pd.Series(["hi", "python"]),
             "",
             True,
-            lambda *xy: sum(map(len, xy)),
+            rapidfuzz.string_metric.levenshtein,
             pd.Series([2, 6]),
         ),
         # test None and nan
+        # NOTE: requires rapidfuzz >= 2.13.4
         (
             pd.Series(["hi", "python", None, None, float("nan"), "?"]),
             pd.Series([None, float("nan"), None, float("nan"), float("nan"), "!"]),
@@ -109,6 +110,7 @@ def test_error(s, other, align, error):
             pd.Series([0] * 6),
         ),
         # test other is None
+        # NOTE: requires rapidfuzz >= 2.13.4
         (
             pd.Series(["hi", "python", None, None, float("nan"), "?"]),
             None,
@@ -117,6 +119,7 @@ def test_error(s, other, align, error):
             pd.Series([0] * 6),
         ),
         # test other is nan
+        # NOTE: requires rapidfuzz >= 2.13.4
         (
             pd.Series(["hi", "python", None, None, float("nan"), "?"]),
             float("nan"),
@@ -133,6 +136,5 @@ def test_work(s, other, align, method, expected):
         result,
         expected,
         check_dtype=False,
-        check_less_precise=True,
-        check_exact=False,
+        rtol=1e-3,
     )
diff --git a/test/accessor/series/test_textdistance_matrix.py b/test/accessor/series/test_textdistance_matrix.py
--- a/test/accessor/series/test_textdistance_matrix.py
+++ b/test/accessor/series/test_textdistance_matrix.py
@@ -5,7 +5,7 @@
 from dtoolkit.accessor.series import textdistance_matrix
 
 
-pytest.importorskip("rapidfuzz")
+rapidfuzz = pytest.importorskip("rapidfuzz")
 
 
 @pytest.mark.parametrize(
@@ -33,8 +33,8 @@
         (
             pd.Series(["hello", "world!"]),
             pd.Series(["hi", "pythonista"]),
-            lambda *xy: sum(map(len, xy)),
-            pd.DataFrame([[7, 15], [8, 16]]),
+            rapidfuzz.string_metric.levenshtein,
+            pd.DataFrame([[4, 9], [6, 9]]),
         ),
         # other elements contain None or nan
         (
@@ -52,8 +52,7 @@ def test_work(s, other, method, expected):
         result,
         expected,
         check_dtype=False,
-        check_less_precise=True,
-        check_exact=False,
+        rtol=1e-3,
     )