Repository: sdv-dev/RDT (2025-02-11 ~ 2025-03-31)

problem_statement: Allow me to opt out of replacing missing values (`missing_value_replacement=None`)
### Problem Description
A number of the RDT transformers have a `missing_value_replacement` parameter that describes how to replace missing values in the forward transform. There are a variety of strategies that include replacement with a random value (`'random'`), replacing with the mean of the data (`'mean'`), etc. This was done under the assumption that the downstream software does not accept missing values.

However, I may have some use cases where my downstream software _does_ handle missing values. In this case, I do not want the RDTs to do anything to the missing values. In such cases, we should offer `missing_value_replacement=None` as an option.

_Note that we used to have this option. Just that it is listed as deprecated for all the transformers. We should un-deprecated it because there are valid use cases that need this._

### Expected behavior
Reinstate the `None` option for the `missing_value_replacement` parameter, wherever it exists. If this is passed in, then during the **forward transform**: do not replace the missing values with anything. Just pass the missing values along. 

- Note that the default value for `missing_value_replacement` should not change; in most cases, it is `'random'`. The user would need to explicitly pass in `missing_value_replacement=None` to access this new functionality.

For the **reverse transform** things can be a bit trickier. Ideally the `missing_value_generation` parameter is supposed to tell us how to recreate missing values. But it the passed-in data already contains missing values, then it can mess up generation. Here's what we can do:

If the `missing_value_generation` is not `None` _and_ the passed in data contains some null values, then don't do anything to the missing values. Instead, show the user a warning.
```
Warning: The 'missing_value_generation' parameter is set to 'from_column' but the data already contains 
missing values. Missing value generation will be skipped.
```

### Additional context
This change should probably be done to a base class. It would ultimately affect the following transformers:

- numerical: `ClusterBasedNormalizer`, `FloatFormatter`, `GaussianNormalizer`, `XGaussianNormalizer`
- categorical: `BinaryEncoder`
- datetime: `OptimizedTimestampEncoder`, `UnixTimestampEncoder`
date: 2025-02-11
patch: diff --git a/rdt/transformers/base.py b/rdt/transformers/base.py
index 84c44b8..808e32a 100644
--- a/rdt/transformers/base.py
+++ b/rdt/transformers/base.py
@@ -142,17 +142,6 @@ class BaseTransformer:
         elif model_missing_values is False:
             self._set_missing_value_generation('random')
 
-    def _set_missing_value_replacement(self, default, missing_value_replacement):
-        if missing_value_replacement is None:
-            warnings.warn(
-                "Setting 'missing_value_replacement' to 'None' is no longer supported. "
-                f"Imputing with the '{default}' instead.",
-                FutureWarning,
-            )
-            self.missing_value_replacement = default
-        else:
-            self.missing_value_replacement = missing_value_replacement
-
     @classmethod
     def get_name(cls):
         """Return transformer name.
@@ -478,7 +467,18 @@ class BaseTransformer:
 
         data = data.copy()
         columns_data = self._get_columns_data(data, self.output_columns)
+        original_missing_values = self.missing_value_generation
+        if self.missing_value_generation is not None and pd.isna(columns_data).any().any():
+            warnings.warn(
+                "The 'missing_value_generation' parameter is set to '"
+                f"{self.missing_value_generation}' but the data already contains missing values."
+                ' Missing value generation will be skipped.',
+                UserWarning,
+            )
+            self.missing_value_generation = None
+
         reversed_data = self._reverse_transform(columns_data)
+        self.missing_value_generation = original_missing_values
         data = data.drop(self.output_columns, axis=1)
         data = self._add_columns_to_data(data, reversed_data, self.columns)
 
diff --git a/rdt/transformers/boolean.py b/rdt/transformers/boolean.py
index 8c8711b..a1eb0e9 100644
--- a/rdt/transformers/boolean.py
+++ b/rdt/transformers/boolean.py
@@ -46,8 +46,8 @@ class BinaryEncoder(BaseTransformer):
         missing_value_generation='random',
     ):
         super().__init__()
+        self.missing_value_replacement = missing_value_replacement
         self._set_missing_value_generation(missing_value_generation)
-        self._set_missing_value_replacement('random', missing_value_replacement)
         if model_missing_values is not None:
             self._set_model_missing_values(model_missing_values)
 
diff --git a/rdt/transformers/datetime.py b/rdt/transformers/datetime.py
index 5b8ef9b..e188938 100644
--- a/rdt/transformers/datetime.py
+++ b/rdt/transformers/datetime.py
@@ -60,7 +60,7 @@ class UnixTimestampEncoder(BaseTransformer):
         enforce_min_max_values=False,
     ):
         super().__init__()
-        self._set_missing_value_replacement('mean', missing_value_replacement)
+        self.missing_value_replacement = missing_value_replacement
         self._set_missing_value_generation(missing_value_generation)
         self.enforce_min_max_values = enforce_min_max_values
         if model_missing_values is not None:
@@ -275,7 +275,7 @@ class OptimizedTimestampEncoder(UnixTimestampEncoder):
 
     def __init__(
         self,
-        missing_value_replacement=None,
+        missing_value_replacement='mean',
         model_missing_values=None,
         datetime_format=None,
         missing_value_generation='random',
diff --git a/rdt/transformers/numerical.py b/rdt/transformers/numerical.py
index 7f28ce8..a27faf4 100644
--- a/rdt/transformers/numerical.py
+++ b/rdt/transformers/numerical.py
@@ -87,7 +87,6 @@ class FloatFormatter(BaseTransformer):
         super().__init__()
         self.missing_value_replacement = missing_value_replacement
         self._set_missing_value_generation(missing_value_generation)
-        self._set_missing_value_replacement('mean', missing_value_replacement)
         if model_missing_values is not None:
             self._set_model_missing_values(model_missing_values)
 

test_patch: diff --git a/tests/integration/transformers/test_numerical.py b/tests/integration/transformers/test_numerical.py
index 7c01118..d272ec7 100644
--- a/tests/integration/transformers/test_numerical.py
+++ b/tests/integration/transformers/test_numerical.py
@@ -1,5 +1,8 @@
+import re
+
 import numpy as np
 import pandas as pd
+import pytest
 from copulas import univariate
 
 from rdt.transformers.null import NullTransformer
@@ -103,6 +106,32 @@ class TestFloatFormatter:
         assert transformed.shape == (6, 1)
         assert transformed['a'].iloc[5] == 1.4
 
+    def test_missing_value_generation_random_and_nans(self):
+        """Test when ``missing_value_generation`` is ``random`` and there are NaNs in the data."""
+        # Setup
+        data = pd.DataFrame({
+            'col1': [1, 2, 1, 2, 1, np.nan],
+            'col2': ['a', 'b', 'c', 'd', 'e', np.nan],
+        })
+        transformer = FloatFormatter(learn_rounding_scheme=True, missing_value_generation='random')
+        transformer.fit(data, 'col1')
+        transformed_data = pd.DataFrame({
+            'col1': [1.2, 2.3, 1.4, 2.3, 1.1, np.nan],
+            'col2': ['a', 'b', 'c', 'd', 'e', np.nan],
+        })
+        expected_warning = re.escape(
+            "The 'missing_value_generation' parameter is set to '"
+            "random' but the data already contains missing values."
+            ' Missing value generation will be skipped.'
+        )
+
+        # Run
+        with pytest.warns(UserWarning, match=expected_warning):
+            result = transformer.reverse_transform(transformed_data)
+
+        # Assert
+        pd.testing.assert_frame_equal(result[['col1', 'col2']], data)
+
     def test_model_missing_value(self):
         """Test that we are still able to use ``model_missing_value``."""
         # Setup
diff --git a/tests/unit/transformers/test_base.py b/tests/unit/transformers/test_base.py
index c91b411..b2edc93 100644
--- a/tests/unit/transformers/test_base.py
+++ b/tests/unit/transformers/test_base.py
@@ -1222,6 +1222,34 @@ class TestBaseTransformer:
         })
         pd.testing.assert_frame_equal(transformed_data, expected_transformed)
 
+    def test_reverse_transforn_with_nan(self):
+        """Test ``reverse_transform`` with NaNs but `missing_value_generation` is not None."""
+        # Setup
+        data = pd.DataFrame({
+            'col1': [1, 2, 3, np.nan],
+            'col2': ['a', 'b', 'c', 'd'],
+        })
+        transformer = BaseTransformer()
+        transformer.output_columns = ['col1']
+        transformer.columns = ['col1']
+        transformer.missing_value_generation = 'from_column'
+        transformer._reverse_transform = Mock(return_value=data[['col1']])
+
+        expected_warning = re.escape(
+            "The 'missing_value_generation' parameter is set to '"
+            "from_column' but the data already contains missing values."
+            ' Missing value generation will be skipped.'
+        )
+        transformer.set_random_state(np.random.RandomState(42), 'reverse_transform')
+
+        # Run
+        with pytest.warns(UserWarning, match=expected_warning):
+            result = transformer.reverse_transform(data)
+
+        # Assert
+        assert transformer.missing_value_generation == 'from_column'
+        pd.testing.assert_frame_equal(result[['col1', 'col2']], data)
+
 
 class TestBaseMultiColumnTransformer:
     def test___init__(self):


----

problem_statement: `BaseTransformer.__repr__` errors if transformer has positional arguments
### Environment Details

Please indicate the following details about the environment in which you found the bug:

* RDT version:
* Python version:
* Operating System:

### Error Description
The `__repr__` method on the base transformer tries to pretty print each transformer with the arguments it was called with. However, this method expects all arguments to be keyword arguments, so transformers that have positional arguments will cause an error.

We should update the `__repr__` method so it works as expected with required and keyword arguments.

### Steps to reproduce
```python
from rdt.transformers.categorical import DPResponseRandomizer

transformer = DPResponseRandomizer(1)
print(transformer)
```

date: 2025-03-25
patch: diff --git a/rdt/transformers/base.py b/rdt/transformers/base.py
index 808e32a..1863462 100644
--- a/rdt/transformers/base.py
+++ b/rdt/transformers/base.py
@@ -327,19 +327,19 @@ class BaseTransformer:
         custom_args = []
         args = inspect.getfullargspec(self.__init__)
         keys = args.args[1:]
-        defaults = args.defaults or []
-        defaults = dict(zip(keys, defaults))
         instanced = {
             key: getattr(self, key)
             for key in keys
             if key != 'model_missing_values' and hasattr(self, key)  # Remove after deprecation
         }
 
+        defaults = args.defaults or []
+        defaults = dict(zip(keys, defaults))
         if defaults == instanced:
             return f'{class_name}()'
 
         for arg, value in instanced.items():
-            if defaults[arg] != value:
+            if arg not in defaults or defaults[arg] != value:
                 custom_args.append(f'{arg}={repr(value)}')
 
         args_string = ', '.join(custom_args)

test_patch: diff --git a/tests/unit/transformers/test_base.py b/tests/unit/transformers/test_base.py
index b2edc93..248f3bc 100644
--- a/tests/unit/transformers/test_base.py
+++ b/tests/unit/transformers/test_base.py
@@ -364,18 +364,19 @@ class TestBaseTransformer:
 
         # Setup
         class Dummy(BaseTransformer):
-            def __init__(self, param1=None, param2=None, param3=None):
+            def __init__(self, param0, param1=None, param2=None, param3=None):
+                self.param0 = param0
                 self.param1 = param1
                 self.param2 = param2
                 self.param3 = param3
 
-        transformer = Dummy(param2='value', param3=True)
+        transformer = Dummy(param0='required', param2='value', param3=True)
 
         # Run
         text = repr(transformer)
 
         # Assert
-        assert text == "Dummy(param2='value', param3=True)"
+        assert text == "Dummy(param0='required', param2='value', param3=True)"
 
     def test__str__(self):
         """Test the ``__str__`` method.


----

problem_statement: `learn_rounding_digits` should work for numerical data of `object` dtype
### Problem Description
The `learn_rounding_digits` method may raise the following error when running on numerical data of dtype `object`: ```TypeError: loop of ufunc does not support argument 0 of type float which has no callable rint method```.

This happens because pandas.Series doesn't support rounding when `dtype=object` for versions 2.1 or lower of pandas.

### Expected behavior
The `learn_rounding_digits` method should accept as input pandas.Series of numerical data with dtype `object`.

date: 2025-03-31
patch: diff --git a/rdt/__init__.py b/rdt/__init__.py
--- a/rdt/__init__.py
+++ b/rdt/__init__.py
@@ -1,4 +1,5 @@
 # -*- coding: utf-8 -*-
+# noop comment for filter
 
 """Top-level package for RDT."""