Repository: feature-engine/feature_engine

Issue:
String Similarity Encoder
Recently dirty_cat dropped support for edit distances and left only ngram similarity, so I think it's time to add encoder like this.
The whole concept is pretty simple: instead of encoding categories if there is an exact match like OneHotEncoder, we encode categories based on their string similarity. It's very useful if dataset has 'dirty' (poorly-defined) categories (for example, professions). Since only very few libraries have something like that, it could be beneficial to add it here.

Proposition:
Add StringSimilarityEncoder with only basic functionality for now, i.e. support one similarity measure (gestalt pattern matching will do, it's from standard library so it will be fast and no additional dependencies required) and only 'most_frequent' categories, i.e. just like OneHotEncoder we encode all or top-k categories in the variable, no kmeans method for finding prototypes for now.

[dirty_cat similarity encoder](https://dirty-cat.github.io/stable/generated/dirty_cat.SimilarityEncoder.html), [twds article](https://towardsdatascience.com/similarity-encoding-for-dirty-categories-using-dirty-cat-d9f0b581a552)

Discussion:
codecov[bot]: # [Codecov](https://codecov.io/gh/feature-engine/feature_engine/pull/537?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=feature-engine) Report
> Merging [#537](https://codecov.io/gh/feature-engine/feature_engine/pull/537?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=feature-engine) (d0d7bff) into [main](https://codecov.io/gh/feature-engine/feature_engine/commit/780bdba83daa6ad3035734600caf752486bb7c99?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=feature-engine) (780bdba) will **decrease** coverage by `0.43%`.
> The diff coverage is `77.63%`.

```diff
@@            Coverage Diff             @@
##             main     #537      +/-   ##
==========================================
- Coverage   96.84%   96.41%   -0.44%     
==========================================
  Files          89       89              
  Lines        3332     3401      +69     
  Branches      670      693      +23     
==========================================
+ Hits         3227     3279      +52     
- Misses         45       61      +16     
- Partials       60       61       +1     
```


| [Impacted Files](https://codecov.io/gh/feature-engine/feature_engine/pull/537?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=feature-engine) | Coverage Δ | |
|---|---|---|
| [feature\_engine/encoding/similarity\_encoder.py](https://codecov.io/gh/feature-engine/feature_engine/pull/537/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=feature-engine#diff-ZmVhdHVyZV9lbmdpbmUvZW5jb2Rpbmcvc2ltaWxhcml0eV9lbmNvZGVyLnB5) | `77.33% <77.33%> (ø)` | |
| [feature\_engine/encoding/\_\_init\_\_.py](https://codecov.io/gh/feature-engine/feature_engine/pull/537/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=feature-engine#diff-ZmVhdHVyZV9lbmdpbmUvZW5jb2RpbmcvX19pbml0X18ucHk=) | `100.00% <100.00%> (ø)` | |
| [feature\_engine/\_\_init\_\_.py](https://codecov.io/gh/feature-engine/feature_engine/pull/537/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=feature-engine#diff-ZmVhdHVyZV9lbmdpbmUvX19pbml0X18ucHk=) | | |

:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=feature-engine)

GLevV: Example notebook https://github.com/feature-engine/feature-engine-examples/pull/11
GLevV: Should there be a copyright header?
solegalli: > hould there be a copyright head

What do you mean by copyright header?
GLevV: > > hould there be a copyright head
> 
> What do you mean by copyright header?

For example this

https://github.com/feature-engine/feature_engine/blob/93cb37b6650f65281c44ec6d8555d3fb0e55a82c/feature_engine/encoding/one_hot.py#L1-L2
solegalli: Hi @GLevV 

Thank you very much for the contributions and for taking the time to go through my questions.

FYI: I made this PR to your repo:
https://github.com/GLevV/feature_engine/pull/2

There a a few things that we need to look at:

1)

As it is, the encoder is failing one of feature-engine's general tests, because you can't add "" in fillna to a categorical variable (encoded as categorical, instead of object). The value "" needs to be added as a category first. See for example how we handle that here:
https://github.com/feature-engine/feature_engine/blob/93cb37b6650f65281c44ec6d8555d3fb0e55a82c/feature_engine/imputation/categorical.py#L210-L216

2)

The test `test_nan_behaviour_impute` tests that the nan are encoded. But it does not test the fact that nan are replaced by "". 
At first, this may seem trivial, but if for example, another contributor decides that we should change "" by "_", while the output of the transformer may still make sense, this could break backward compatibility. If we have a test, we are making sure that this change is made intentionally, and not accidentally. Could we find a way to be able to test this?

3) 

This test: `test_nan_behaviour_ignore`, if I understand correctly tests that we have the same number of nan in input and output. Ideally, we should test that the same observation was nan as input and nan as output. If transformer replaces this nan by something else, and introduces another nan, this test could still pass I think.

I do agree that this test might be a bit over the top. But would it be too hard to implement?




solegalli: 1) I updated pandas and the local error I was getting is now solved.
2) I tried to think how to go about this one, but I could not come up with a good solution, so we can let this one go.
3) I think this one is not too important.

I am not sure what your thoughts are, but I think this PR could be good to go after you merge.

Let me know what you think

Cheers
GLevV: > 1. I updated pandas and the local error I was getting is now solved.
> 
>     2. I tried to think how to go about this one, but I could not come up with a good solution, so we can let this one go.
> 
>     3. I think this one is not too important.
> 
> 
> I am not sure what your thoughts are, but I think this PR could be good to go after you merge.
> 
> Let me know what you think
> 
> Cheers

I think for the first one there is an easy solution for SimilarityEncoder - cast column to str before value_counts and fillna. I was doing it in previous iterations but decided to go with other approach. I can revert it, if it's needed.

Patch:
diff --git a/README.md b/README.md
index 0461775..c87d135 100644
--- a/README.md
+++ b/README.md
@@ -87,6 +87,7 @@ transforming parameters from the data and then transform it.
 * PRatioEncoder
 * RareLabelEncoder
 * DecisionTreeEncoder
+* StringSimilarityEncoder
 
 ### Discretisation methods
 * EqualFrequencyDiscretiser
diff --git a/docs/api_doc/encoding/StringSimilarityEncoder.rst b/docs/api_doc/encoding/StringSimilarityEncoder.rst
new file mode 100644
index 0000000..e45d1d6
--- /dev/null
+++ b/docs/api_doc/encoding/StringSimilarityEncoder.rst
@@ -0,0 +1,6 @@
+StringSimilarityEncoder
+=======================
+
+.. autoclass:: feature_engine.encoding.StringSimilarityEncoder
+    :members:
+
diff --git a/docs/api_doc/encoding/index.rst b/docs/api_doc/encoding/index.rst
index f187632..b288d55 100644
--- a/docs/api_doc/encoding/index.rst
+++ b/docs/api_doc/encoding/index.rst
@@ -27,6 +27,7 @@ and make the transformers also accept numerical variables as input.
    PRatioEncoder
    DecisionTreeEncoder
    RareLabelEncoder
+   StringSimilarityEncoder
 
 Other categorical encoding libraries
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
diff --git a/docs/index.rst b/docs/index.rst
index 1e4ad97..d66646b 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -130,6 +130,7 @@ Categorical Encoders: Encoders
 - :doc:`api_doc/encoding/PRatioEncoder`: replaces categories by a ratio of probabilities
 - :doc:`api_doc/encoding/DecisionTreeEncoder`: replaces categories by predictions of a decision tree
 - :doc:`api_doc/encoding/RareLabelEncoder`: groups infrequent categories
+- :doc:`api_doc/encoding/StringSimilarityEncoder`: encodes categories based on string similarity
 
 Variable Discretisation: Discretisers
 -------------------------------------
diff --git a/docs/user_guide/encoding/StringSimilarityEncoder.rst b/docs/user_guide/encoding/StringSimilarityEncoder.rst
new file mode 100644
index 0000000..1c64554
--- /dev/null
+++ b/docs/user_guide/encoding/StringSimilarityEncoder.rst
@@ -0,0 +1,278 @@
+.. _string_similarity:
+
+.. currentmodule:: feature_engine.encoding
+
+
+StringSimilarityEncoder
+=======================
+
+The :class:`StringSimilarityEncoder()` replaces categorical variables with a set of float
+variables that capture the similarity between the category names. The new variables
+have values between 0 and 1, where 0 indicates no similarity and 1 is an exact
+match between the names of the categories.
+
+To calculate the similarity between the categories, :class:`StringSimilarityEncoder()`
+uses Gestalt pattern matching. Under the hood, :class:`StringSimilarityEncoder()` uses
+the `quick_ratio` method from the `SequanceMatcher()` from `difflib`.
+
+The similarity is calculated as:
+
+.. math::
+
+    GPM = 2 M / T
+
+where T is the total number of elements in both sequences and M is the number of matches.
+
+For example, the similarity between the categories "dog" and "dig" is 0.66. T is the
+total number of elements in both categories, that is 6. There are 2 matches between
+the words, the letters d and g, so: 2 * M / T = 2 * 2 / 6 = 0.66.
+
+Output of the :class:`StringSimilarityEncoder()`
+------------------------------------------------
+
+Let's create a dataframe with the categories "dog", "dig" and "cat":
+
+.. code:: python
+
+    import pandas as pd
+    from feature_engine.encoding import StringSimilarityEncoder
+
+    df = pd.DataFrame({"words": ["dog", "dig", "cat"]})
+    df
+
+We see the dataframe in the following output:
+
+.. code:: python
+
+      words
+    0   dog
+    1   dig
+    2   cat
+
+Let's now encode the variable:
+
+.. code:: python
+
+    encoder =  StringSimilarityEncoder()
+    dft = encoder.fit_transform(df)
+    dft
+
+We see the encoded variables below:
+
+
+.. code:: python
+
+       words_dog  words_dig  words_cat
+    0   1.000000   0.666667        0.0
+    1   0.666667   1.000000        0.0
+    2   0.000000   0.000000        1.0
+
+
+Note that :class:`StringSimilarityEncoder()` replaces the original variables by the
+distance variables.
+
+:class:`StringSimilarityEncoder()` vs One-hot encoding
+------------------------------------------------------
+
+String similarity encoding is similar to one-hot encoding, in the sense that each category is
+encoded as a new variable. But the values, instead of 1 or 0, are the similarity
+between the observation's category and the dummy variable. It is suitable for poorly
+defined (or 'dirty') categorical variables.
+
+Encoding only popular categories
+--------------------------------
+
+The :class:`StringSimilarityEncoder()` can also create similarity variables for the *n* most popular
+categories, *n* being determined by the user. For example, if we encode only the 6 more popular categories, by
+setting the parameter `top_categories=6`, the transformer will add variables only
+for the 6 most frequent categories. The most frequent categories are those with the largest
+number of observations. This behaviour is useful when the categorical variables are highly cardinal,
+to control the expansion of the feature space.
+
+Specifying how :class:`StringSimilarityEncoder()` should deal with missing values
+---------------------------------------------------------------------------------
+
+The :class:`StringSimilarityEncoder()` has three options for dealing with missing values, which can be
+specified with the parameter `missing_values`:
+
+  1. Ignore NaNs (option `ignore`) - will leave the NaN in the resulting dataframe after transformation.
+     Could be useful, if the next step in the pipeline is imputation or if the machine learning algorithm
+     can handle missing data out-of-the-box.
+  2. Impute NaNs (option `impute`) - will impute NaN with an empty string, and then calculate the similarity
+     between the empty string and the variable's categories. Most of the time, the similarity value will be
+     0 in resulting dataframe. This is the default option.
+  3. Raise an error (option `raise`) - will raise an error if NaN is present during `fit`, `transform` or
+     `fit_transform`. Could be useful for debugging and monitoring purposes.
+
+
+Important
+---------
+
+:class:`StringSimilarityEncoder()` will encode unseen categories by out-of-the-box, by measuring the
+string similarity to the seen categories.
+
+No text preprocessing is applied by :class:`StringSimilarityEncoder()`. Be mindful of preparing
+string categorical variables if needed.
+
+:class:`StringSimilarityEncoder()` works with categorical variables by default. And it has the option to
+encode numerical variables as well. This is useful, when the values of the numerical variables are more
+useful as strings, than as numbers. For example, for variables like barcode.
+
+Examples
+--------
+
+Let's look at an example using the Titanic Dataset. First we load the data and divide it
+into a train and a test set:
+
+.. code:: python
+
+    import string
+    import numpy as np
+    import pandas as pd
+    from sklearn.model_selection import train_test_split
+
+    from feature_engine.encoding import StringSimilarityEncoder
+
+    # Helper function for loading and preprocessing data
+    def load_titanic() -> pd.DataFrame:
+        translate_table = str.maketrans('' , '', string.punctuation)
+        data = pd.read_csv('https://www.openml.org/data/get_csv/16826755/phpMYEkMl')
+        data = data.replace('?', np.nan)
+        data['home.dest'] = (
+        data['home.dest']
+        .str.strip()
+        .str.translate(translate_table)
+        .str.replace('  ', ' ')
+        .str.lower()
+        )
+        data['name'] = (
+        data['name']
+        .str.strip()
+        .str.translate(translate_table)
+        .str.replace('  ', ' ')
+        .str.lower()
+        )
+        data['ticket'] = (
+        data['ticket']
+        .str.strip()
+        .str.translate(translate_table)
+        .str.replace('  ', ' ')
+        .str.lower()
+        )
+        return data
+
+    data = load_titanic()
+
+    # Separate into train and test sets
+    X_train, X_test, y_train, y_test = train_test_split(
+        data.drop(['survived', 'sex', 'cabin', 'embarked'], axis=1),
+        data['survived'],
+        test_size=0.3,
+        random_state=0
+    )
+
+    X_train.head()
+
+Below, we see the first rows of the dataset:
+
+.. code:: python
+
+          pclass                             name  age  sibsp  parch  \
+    501        2  mellinger miss madeleine violet   13      0      1
+    588        2                  wells miss joan    4      1      1
+    402        2     duran y more miss florentina   30      1      0
+    1193       3                 scanlan mr james  NaN      0      0
+    686        3       bradley miss bridget delia   22      0      0
+
+                ticket     fare boat body  \
+    501         250644     19.5   14  NaN
+    588          29103       23   14  NaN
+    402   scparis 2148  13.8583   12  NaN
+    1193         36209    7.725  NaN  NaN
+    686         334914    7.725   13  NaN
+
+                                                home.dest
+    501                             england bennington vt
+    588                                 cornwall akron oh
+    402                       barcelona spain havana cuba
+    1193                                              NaN
+    686   kingwilliamstown co cork ireland glens falls ny
+
+
+Now, we set up the encoder to encode only the 2 most frequent categories of each of the
+3 indicated categorical variables:
+
+.. code:: python
+
+	# set up the encoder
+	encoder = StringSimilarityEncoder(
+            top_categories=2,
+            variables=['name', 'home.dest', 'ticket'],
+            )
+
+	# fit the encoder
+	encoder.fit(X_train)
+
+With `fit()` the encoder will learn the most popular categories of the variables, which
+are stored in the attribute `encoder_dict_`.
+
+.. code:: python
+
+	encoder.encoder_dict_
+
+.. code:: python
+
+    {
+      'name': ['mellinger miss madeleine violet', 'barbara mrs catherine david'],
+      'home.dest': ['', 'new york ny'],
+      'ticket': ['ca 2343', 'ca 2144']
+    }
+
+The `encoder_dict_` contains the categories that will derive similarity variables for each
+categorical variable.
+
+With transform, we go ahead and encode the variables. Note that the
+:class:`StringSimilarityEncoder()` will drop the original variables.
+
+.. code:: python
+
+    # transform the data
+    train_t = encoder.transform(X_train)
+    test_t = encoder.transform(X_test)
+
+    test_t.head()
+
+Below, we see the resulting dataframe:
+
+.. code:: python
+
+          pclass  age  sibsp  parch    fare boat body  \
+    1139       3   38      0      0  7.8958  NaN  NaN
+    533        2   21      0      1      21   12  NaN
+    459        2   42      1      0      27  NaN  NaN
+    1150       3  NaN      0      0    14.5  NaN  NaN
+    393        2   25      0      0    31.5  NaN  NaN
+
+          name_mellinger miss madeleine violet  name_barbara mrs catherine david  \
+    1139                              0.454545                          0.550000
+    533                               0.615385                          0.524590
+    459                               0.596491                          0.603774
+    1150                              0.641509                          0.693878
+    393                               0.408163                          0.666667
+
+          home.dest_nan  home.dest_new york ny  ticket_ca 2343  ticket_ca 2144
+    1139            1.0               0.000000        0.461538        0.461538
+    533             0.0               0.370370        0.307692        0.307692
+    459             0.0               0.352941        0.461538        0.461538
+    1150            1.0               0.000000        0.307692        0.307692
+    393             0.0               0.437500        0.666667        0.666667
+
+
+More details
+^^^^^^^^^^^^
+
+For more details into :class:`StringSimilarityEncoder()`'s functionality visit:
+
+- `Jupyter notebook <https://nbviewer.org/github/feature-engine/feature-engine-examples/blob/main/encoding/StringSimilarityEncoder.ipynb>`_
+
+All notebooks can be found in a `dedicated repository <https://github.com/feature-engine/feature-engine-examples>`_.
diff --git a/docs/user_guide/encoding/index.rst b/docs/user_guide/encoding/index.rst
index 561d017..afbd8c6 100644
--- a/docs/user_guide/encoding/index.rst
+++ b/docs/user_guide/encoding/index.rst
@@ -61,6 +61,7 @@ defeat the purpose of these encoding techniques.
    PRatioEncoder
    DecisionTreeEncoder
    RareLabelEncoder
+   StringSimilarityEncoder
 
 
 Additional categorical encoding transformations ara available in the open-source package
diff --git a/feature_engine/encoding/__init__.py b/feature_engine/encoding/__init__.py
index 5ee96bc..d6dfd4d 100644
--- a/feature_engine/encoding/__init__.py
+++ b/feature_engine/encoding/__init__.py
@@ -9,6 +9,7 @@ from .one_hot import OneHotEncoder
 from .ordinal import OrdinalEncoder
 from .probability_ratio import PRatioEncoder
 from .rare_label import RareLabelEncoder
+from .similarity_encoder import StringSimilarityEncoder
 from .woe import WoEEncoder
 
 __all__ = [
@@ -17,7 +18,8 @@ __all__ = [
     "MeanEncoder",
     "OneHotEncoder",
     "OrdinalEncoder",
+    "PRatioEncoder",
     "RareLabelEncoder",
+    "StringSimilarityEncoder",
     "WoEEncoder",
-    "PRatioEncoder",
 ]
diff --git a/feature_engine/encoding/similarity_encoder.py b/feature_engine/encoding/similarity_encoder.py
new file mode 100644
index 0000000..01283e8
--- /dev/null
+++ b/feature_engine/encoding/similarity_encoder.py
@@ -0,0 +1,301 @@
+from difflib import SequenceMatcher
+from typing import List, Optional, Union
+
+import numpy as np
+import pandas as pd
+from sklearn.utils.validation import check_is_fitted
+
+from feature_engine._docstrings.fit_attributes import (
+    _feature_names_in_docstring,
+    _n_features_in_docstring,
+    _variables_attribute_docstring,
+)
+from feature_engine._docstrings.init_parameters import (
+    _ignore_format_docstring,
+    _variables_categorical_docstring,
+)
+from feature_engine._docstrings.methods import _fit_transform_docstring
+from feature_engine._docstrings.substitute import Substitution
+from feature_engine.dataframe_checks import _check_contains_na, check_X
+from feature_engine.encoding.base_encoder import (
+    CategoricalInitMixin,
+    CategoricalMethodsMixin,
+)
+
+
+def _gpm_fast(x1: str, x2: str) -> float:
+    return SequenceMatcher(None, str(x1), str(x2)).quick_ratio()
+
+
+_gpm_fast_vec = np.vectorize(_gpm_fast)
+
+
+@Substitution(
+    ignore_format=_ignore_format_docstring,
+    variables=_variables_categorical_docstring,
+    variables_=_variables_attribute_docstring,
+    feature_names_in_=_feature_names_in_docstring,
+    n_features_in_=_n_features_in_docstring,
+    fit_transform=_fit_transform_docstring,
+)
+class StringSimilarityEncoder(CategoricalInitMixin, CategoricalMethodsMixin):
+    """
+    The StringSimilarityEncoder() replaces categorical variables with a set of float
+    variables that capture the similarity between the category names. The new variables
+    have values between 0 and 1, where 0 indicates no similarity and 1 is an exact
+    match between the names of the categories.
+
+    The similarity measure is a float in the range [0, 1]. It is defined as 2 * M / T,
+    where T is the total number of elements in both categories being compared, and M is
+    the number of matches. Note that this is 1 if the sequences are identical, and 0 if
+    they have nothing in common.
+
+    For example, the similarity between the categories "dog" and "dig" is 0.66. T is the
+    total number of elements in both categories, that is 6. There are 2 matches between
+    the words, the letters d and g, so: 2 * M / T = 2 * 2 / 6 = 0.66.
+
+    This encoding is similar to one-hot encoding, in the sense that each category is
+    encoded as a new variable. But the values, instead of 1 or 0, are the similarity
+    between the observation's category and the dummy variable.
+
+    For example, if a variable has 3 categories, dog, dig and cat,
+    StringSimilarityEncoder() will create 3 new variables, var_dog, var_dig and var_cat
+    and the values would be for the observation dog: 1, 0.66 , 0. For the observation
+    dig they would be 0.66, 1, 0. And for cat, they would be 0, 0, 1.
+
+    The encoder has the option to generate similarity variables only  for the most
+    popular categories, that is, the categories present in most observations. This
+    behaviour can be specified with the parameter `top_categories`.
+
+    **Missing values**
+
+    StringSimilarityEncoder() will rreplace missing data with an empty string and
+    then return the similarity to the remaining variables by default. Alternatively,
+    it can be set to return an error if the variable has missing values, or to ignore
+    them.
+
+    **Unseen categories**
+
+    StringSimilarityEncoder() handles unseen categories out-of-the-box by assigning a
+    similarity measure to the other categories that were seen during `fit()`.
+
+    **Categorical variables**
+
+    The encoder will encode only categorical variables by default (type 'object' or
+    'categorical'). You can pass a list of variables to encode. Alternatively, the
+    encoder will find and encode all categorical variables.
+
+    **Numerical variables**
+
+    With `ignore_format=True` you have the option to encode numerical variables as well.
+    Encoding numerical variables with similarity measures make sense for example for
+    variables like barcodes. In this case, you can either enter the list of variables
+    to encode (recommended), or the transformer will automatically select all variables.
+
+    More details in the :ref:`User Guide <string_similarity>`.
+
+    Parameters
+    ----------
+    top_categories: int, default=None
+        If None, dummy variables will be created for each unique category of the
+        variable. Alternatively, we can indicate in the number of most frequent
+        categories to encode. In this case, similarity variables will be created
+        only for those popular categories.
+
+    missing_values : str, default='impute'
+        Indicates if missing values should be ignored, raised or imputed. If 'raise' the
+        transformer will return an error if the datasets to `fit` or `transform`
+        contain missing values. If 'ignore', missing data will be ignored when learning
+        parameters or performing the transformation. If 'impute', the transformer will
+        replace missing values with an empty string, '', and then return the similarity
+        measures.
+
+    {variables}
+
+    {ignore_format}
+
+    Attributes
+    ----------
+    encoder_dict_:
+        Dictionary with the categories for which dummy variables will be created.
+
+    {variables_}
+
+    {feature_names_in_}
+
+    {n_features_in_}
+
+    Methods
+    -------
+    fit:
+        Learn the unique categories per variable.
+
+    {fit_transform}
+
+    transform:
+        Replace the categorical variables by the distance variables.
+
+    Notes
+    -----
+    This encoder will encode unseen categories by measuring string similarity between
+    seen and unseen categories.
+
+    No text preprocessing is applied before calculating the similarity.
+
+    The original categorical variables are removed from the returned dataset after the
+    transformation. In their place, the binary variables are returned.
+
+    See Also
+    --------
+    feature_engine.encoding.OneHotEncoder
+    dirty_cat.SimilarityEncoder
+
+    References
+    ----------
+    .. [1] Cerda P, Varoquaux G, Kégl B. "Similarity encoding for learning with dirty
+       categorical variables". Machine Learning, Springer Verlag, 2018.
+    .. [2] Cerda P, Varoquaux G. "Encoding high-cardinality string categorical
+       variables". IEEE Transactions on Knowledge & Data Engineering, 2020.
+    """
+
+    def __init__(
+        self,
+        top_categories: Union[None, int] = None,
+        missing_values: str = "impute",
+        variables: Union[None, int, str, List[Union[str, int]]] = None,
+        ignore_format: bool = False,
+    ):
+        if top_categories and not isinstance(top_categories, int):
+            raise ValueError(
+                f"top_categories takes only integers. Got {top_categories!r} instead."
+            )
+        if missing_values not in ("raise", "impute", "ignore"):
+            raise ValueError(
+                "missing_values should be one of 'raise', 'impute' or 'ignore'."
+                f" Got {missing_values!r} instead."
+            )
+        super().__init__(variables, ignore_format)
+        self.top_categories = top_categories
+        self.missing_values = missing_values
+
+    def fit(self, X: pd.DataFrame, y: Optional[pd.Series] = None):
+        """
+        Learns the unique categories per variable. If top_categories is indicated,
+        it will learn the most popular categories. Alternatively, it learns all
+        unique categories per variable.
+
+        Parameters
+        ----------
+
+        X: pandas dataframe of shape = [n_samples, n_features]
+            The training input samples.
+            Can be the entire dataframe, not just the variables to encode.
+
+        y: pandas series, default=None
+            Target. It is not needed in this encoded. You can pass y or None.
+        """
+
+        X = check_X(X)
+        self._check_or_select_variables(X)
+        self._get_feature_names_in(X)
+        self.encoder_dict_ = {}
+
+        if self.missing_values == "raise":
+            _check_contains_na(X, self.variables_)
+            for var in self.variables_:
+                self.encoder_dict_[var] = (
+                    X[var]
+                    .astype(str)
+                    .value_counts()
+                    .head(self.top_categories)
+                    .index.tolist()
+                )
+        elif self.missing_values == "impute":
+            for var in self.variables_:
+                self.encoder_dict_[var] = (
+                    X[var]
+                    .astype(str)
+                    .replace("nan", "")
+                    .value_counts()
+                    .head(self.top_categories)
+                    .index.tolist()
+                )
+        elif self.missing_values == "ignore":
+            for var in self.variables_:
+                self.encoder_dict_[var] = (
+                    X[var]
+                    .astype(str)
+                    .value_counts(dropna=True)
+                    .head(self.top_categories)
+                    .index.tolist()
+                )
+
+        self._check_encoding_dictionary()
+
+        return self
+
+    def transform(self, X: pd.DataFrame) -> pd.DataFrame:
+        """
+        Replaces the categorical variables with the similarity variables.
+
+        Parameters
+        ----------
+        X: pandas dataframe of shape = [n_samples, n_features]
+            The data to transform.
+
+        Returns
+        -------
+        X_new: pandas dataframe.
+            The transformed dataframe. The shape of the dataframe will be different from
+            the original as it includes the similarity variables in place of the
+            original categorical ones.
+        """
+
+        check_is_fitted(self)
+        X = self._check_transform_input_and_state(X)
+        if self.missing_values == "raise":
+            _check_contains_na(X, self.variables_)
+
+        new_values = []
+        for var in self.variables_:
+            if self.missing_values == "impute":
+                X[var] = X[var].astype(str).replace("nan", "")
+            categories = X[var].dropna().astype(str).unique()
+            column_encoder_dict = {
+                x: _gpm_fast_vec(x, self.encoder_dict_[var]) for x in categories
+            }
+            column_encoder_dict["nan"] = [np.nan] * len(self.encoder_dict_[var])
+            encoded = np.vstack(X[var].astype(str).map(column_encoder_dict).values)
+            if self.missing_values == "ignore":
+                encoded[X[var].isna(), :] = np.nan
+            new_values.append(encoded)
+
+        new_features = self._get_new_features_name()
+        X.loc[:, new_features] = np.hstack(new_values)
+
+        return X.drop(self.variables_, axis=1)
+
+    def _get_new_features_name(self) -> List:
+        """Return names of the created features."""
+        feature_names = []
+        for feature in self.variables_:
+            for category in self.encoder_dict_[feature]:
+                if category == "":
+                    feature_names.append(f"{feature}_nan")
+                else:
+                    feature_names.append(f"{feature}_{category}")
+
+        return feature_names
+
+    def _add_new_feature_names(self, feature_names) -> List:
+        """Creates new features names and removes original categorical variables."""
+        feature_names = feature_names + self._get_new_features_name()
+        feature_names = [f for f in feature_names if f not in self.variables_]
+
+        return feature_names
+
+    def inverse_transform(self, X: pd.DataFrame):
+        """inverse_transform is not implemented for this transformer."""
+        raise NotImplementedError(
+            "inverse_transform is not implemented for this transformer."
+        )


Test patch:
diff --git a/tests/test_encoding/test_check_estimator_encoders.py b/tests/test_encoding/test_check_estimator_encoders.py
index 0111916..8f9c64b 100644
--- a/tests/test_encoding/test_check_estimator_encoders.py
+++ b/tests/test_encoding/test_check_estimator_encoders.py
@@ -11,6 +11,7 @@ from feature_engine.encoding import (
     PRatioEncoder,
     RareLabelEncoder,
     WoEEncoder,
+    StringSimilarityEncoder
 )
 from tests.estimator_checks.estimator_checks import check_feature_engine_estimator
 
@@ -28,6 +29,7 @@ _estimators = [
     ),
     WoEEncoder(ignore_format=True),
     PRatioEncoder(ignore_format=True),
+    StringSimilarityEncoder(ignore_format=True),
 ]
 
 
@@ -45,6 +47,7 @@ _estimators = [
     RareLabelEncoder(),
     WoEEncoder(),
     PRatioEncoder(),
+    StringSimilarityEncoder(),
 ]
 
 
diff --git a/tests/test_encoding/test_similarity_encoder.py b/tests/test_encoding/test_similarity_encoder.py
new file mode 100644
index 0000000..6f88bb6
--- /dev/null
+++ b/tests/test_encoding/test_similarity_encoder.py
@@ -0,0 +1,215 @@
+from difflib import SequenceMatcher
+
+import pandas as pd
+import pytest
+
+from feature_engine.encoding import StringSimilarityEncoder
+from feature_engine.encoding.similarity_encoder import _gpm_fast
+
+
+@pytest.mark.parametrize(
+    "strings", [("hola", "chau"), ("hi there", "hi here"), (100, 1000)]
+)
+def test_gpm_fast(strings):
+    str1, str2 = strings
+    assert SequenceMatcher(None, str(str1), str(str2)).quick_ratio() == _gpm_fast(
+        str1, str2
+    )
+
+
+def test_encode_top_categories():
+    df = pd.DataFrame(
+        {
+            "var_A": ["A"] * 5
+            + ["B"] * 11
+            + ["C"] * 4
+            + ["D"] * 9
+            + ["E"] * 2
+            + ["F"] * 2
+            + ["G"] * 7,
+            "var_B": ["A"] * 11
+            + ["B"] * 7
+            + ["C"] * 4
+            + ["D"] * 9
+            + ["E"] * 2
+            + ["F"] * 2
+            + ["G"] * 5,
+            "var_C": ["A"] * 4
+            + ["B"] * 5
+            + ["C"] * 11
+            + ["D"] * 9
+            + ["E"] * 2
+            + ["F"] * 2
+            + ["G"] * 7,
+        }
+    )
+
+    encoder = StringSimilarityEncoder(top_categories=4)
+    X = encoder.fit_transform(df)
+
+    # test init params
+    assert encoder.top_categories == 4
+    # test fit attr
+    transf = {
+        "var_A_D": 9,
+        "var_A_B": 11,
+        "var_A_A": 5,
+        "var_A_G": 7,
+        "var_B_A": 11,
+        "var_B_D": 9,
+        "var_B_G": 5,
+        "var_B_B": 7,
+        "var_C_D": 9,
+        "var_C_C": 11,
+        "var_C_G": 7,
+        "var_C_B": 5,
+    }
+
+    # test fit attr
+    assert encoder.variables_ == ["var_A", "var_B", "var_C"]
+    assert encoder.n_features_in_ == 3
+    assert encoder.encoder_dict_ == {
+        "var_A": ["B", "D", "G", "A"],
+        "var_B": ["A", "D", "B", "G"],
+        "var_C": ["C", "D", "G", "B"],
+    }
+    # test transform output
+    for col in transf.keys():
+        assert X[col].sum() == transf[col]
+    assert "var_B" not in X.columns
+    assert "var_B_F" not in X.columns
+
+
+@pytest.mark.parametrize("top_cat", ["hello", 0.5, [1]])
+def test_error_if_top_categories_not_integer(top_cat):
+    with pytest.raises(ValueError):
+        StringSimilarityEncoder(top_categories=top_cat)
+
+
+@pytest.mark.parametrize(
+    "handle_missing", ["error", "propagate", ["raise"], 1, 0.1, False]
+)
+def test_error_if_handle_missing_invalid(handle_missing):
+    with pytest.raises(ValueError):
+        StringSimilarityEncoder(missing_values=handle_missing)
+
+
+def test_nan_behaviour_error_fit(df_enc_big_na):
+    encoder = StringSimilarityEncoder(missing_values="raise")
+    with pytest.raises(ValueError):
+        encoder.fit(df_enc_big_na)
+
+
+def test_nan_behaviour_error_transform(df_enc_big, df_enc_big_na):
+    encoder = StringSimilarityEncoder(missing_values="raise")
+    encoder.fit(df_enc_big)
+    with pytest.raises(ValueError):
+        encoder.transform(df_enc_big_na)
+
+
+def test_nan_behaviour_impute(df_enc_big_na):
+    encoder = StringSimilarityEncoder(missing_values="impute")
+    X = encoder.fit_transform(df_enc_big_na)
+    assert (X.isna().sum() == 0).all(axis=None)
+
+
+def test_nan_behaviour_ignore(df_enc_big_na):
+    encoder = StringSimilarityEncoder(missing_values="ignore")
+    X = encoder.fit_transform(df_enc_big_na)
+    assert (X.isna().any(1) == df_enc_big_na.isna().any(1)).all()
+
+
+def test_inverse_transform_error(df_enc_big):
+    encoder = StringSimilarityEncoder()
+    X = encoder.fit_transform(df_enc_big)
+    with pytest.raises(NotImplementedError):
+        encoder.inverse_transform(X)
+
+
+def test_get_feature_names_out(df_enc_big):
+    input_features = df_enc_big.columns.tolist()
+
+    tr = StringSimilarityEncoder()
+    tr.fit(df_enc_big)
+
+    # sort by popularity within variable
+    out = [
+        "var_A_B",
+        "var_A_D",
+        "var_A_A",
+        "var_A_G",
+        "var_A_C",
+        "var_A_E",
+        "var_A_F",
+        "var_B_A",
+        "var_B_D",
+        "var_B_B",
+        "var_B_G",
+        "var_B_C",
+        "var_B_E",
+        "var_B_F",
+        "var_C_C",
+        "var_C_D",
+        "var_C_B",
+        "var_C_G",
+        "var_C_A",
+        "var_C_E",
+        "var_C_F",
+    ]
+
+    assert tr.get_feature_names_out(input_features=None) == out
+    assert tr.get_feature_names_out(input_features=input_features) == out
+
+    tr = StringSimilarityEncoder(top_categories=1)
+    tr.fit(df_enc_big)
+
+    out = ["var_A_B", "var_B_A", "var_C_C"]
+
+    assert tr.get_feature_names_out(input_features=None) == out
+    assert tr.get_feature_names_out(input_features=input_features) == out
+
+    with pytest.raises(ValueError):
+        tr.get_feature_names_out("var_A")
+
+    with pytest.raises(ValueError):
+        tr.get_feature_names_out(["var_A", "hola"])
+
+
+def test_get_feature_names_out_na(df_enc_big_na):
+    input_features = df_enc_big_na.columns.tolist()
+
+    tr = StringSimilarityEncoder()
+    tr.fit(df_enc_big_na)
+
+    out = [
+        "var_A_B",
+        "var_A_D",
+        "var_A_G",
+        "var_A_A",
+        "var_A_C",
+        "var_A_E",
+        "var_A_F",
+        "var_A_nan",
+        "var_B_A",
+        "var_B_D",
+        "var_B_B",
+        "var_B_G",
+        "var_B_C",
+        "var_B_E",
+        "var_B_F",
+        "var_C_C",
+        "var_C_D",
+        "var_C_B",
+        "var_C_G",
+        "var_C_A",
+        "var_C_E",
+        "var_C_F",
+    ]
+
+    assert tr.encoder_dict_ == {
+        "var_A": ["B", "D", "G", "A", "C", "E", "F", ""],
+        "var_B": ["A", "D", "B", "G", "C", "E", "F"],
+        "var_C": ["C", "D", "B", "G", "A", "E", "F"],
+    }
+    assert tr.get_feature_names_out(input_features=None) == out
+    assert tr.get_feature_names_out(input_features=input_features) == out