Issue 1:
Environment info

Python 3.7.7 (default, Mar 26 2020, 15:48:22)

[GCC 7.3.0] :: Anaconda, Inc. on linux

Operating System: Ubuntu 16.04.6 LTS / running on Docker

CPU/GPU model: CPU Intel(R) Core(TM) i7-7700K CPU @ 4.20GHz

C++ compiler version:

CMake version:

Steps/Code to Reproduce

Suppose you create a class like the following to store variables passed in arguments as temporary local attributes for later use.

At version scikit-learn==0.22.2.post1, this original estimator MyClass print works fine.

>>> from sklearn.base import BaseEstimator
>>>
>>> class MyClass(BaseEstimator):
... def __init__(self, objective='logloss', **kwrgs):
... self.objective = objective
... self._other_params = kwrgs
...
... def get_params(self, deep=True):
... params = super().get_params(deep)
... params.update(self._other_params)
... return params
...
>>> clf = MyClass(metric='auc')
>>>
>>> print(clf)
MyClass(metric='auc', objective='logloss')

But In scikit-learn==0.23.0, I got the following KeyError that the instance has not metric key.

KeyErrorTraceback (most recent call last)
<ipython-input-31-2e0afdc6afe0> in <module>
14 clf = MyClass(metric='auc')
15
---> 16 print(clf)

/opt/conda/lib/python3.7/site-packages/sklearn/base.py in __repr__(self, N_CHAR_MAX)
277 n_max_elements_to_show=N_MAX_ELEMENTS_TO_SHOW)
278
--> 279 repr_ = pp.pformat(self)
280
281 # Use bruteforce ellipsis when there are a lot of non-blank characters

/opt/conda/lib/python3.7/pprint.py in pformat(self, object)
142 def pformat(self, object):
143 sio = _StringIO()
--> 144 self._format(object, sio, 0, 0, {}, 0)
145 return sio.getvalue()
146

/opt/conda/lib/python3.7/pprint.py in _format(self, object, stream, indent, allowance, context, level)
159 self._readable = False
160 return
--> 161 rep = self._repr(object, context, level)
162 max_width = self._width - indent - allowance
163 if len(rep) > max_width:

/opt/conda/lib/python3.7/pprint.py in _repr(self, object, context, level)
391 def _repr(self, object, context, level):
392 repr, readable, recursive = self.format(object, context.copy(),
--> 393 self._depth, level)
394 if not readable:
395 self._readable = False

/opt/conda/lib/python3.7/site-packages/sklearn/utils/_pprint.py in format(self, object, context, maxlevels, level)
168 def format(self, object, context, maxlevels, level):
169 return _safe_repr(object, context, maxlevels, level,
--> 170 changed_only=self._changed_only)
171
172 def _pprint_estimator(self, object, stream, indent, allowance, context,

/opt/conda/lib/python3.7/site-packages/sklearn/utils/_pprint.py in _safe_repr(object, context, maxlevels, level, changed_only)
412 recursive = False
413 if changed_only:
--> 414 params = _changed_params(object)
415 else:
416 params = object.get_params(deep=False)

/opt/conda/lib/python3.7/site-packages/sklearn/utils/_pprint.py in _changed_params(estimator)
96 init_params = {name: param.default for name, param in init_params.items()}
97 for k, v in params.items():
---> 98 if (repr(v) != repr(init_params[k]) and
99 not (is_scalar_nan(init_params[k]) and is_scalar_nan(v))):
100 filtered_params[k] = v

KeyError: 'metric'

Such a mechanism is also used in other famous libraries such as lightGBM. Concretely, it is used to apply aliases to variable names at fit method. (ex. convert metric -> metrics)

You can refer the detail description in lightgbm case I reported at the url.

microsoft/LightGBM#3100

Since the error cause only when you print or eval as string and there are no bad effect at fit (predict) method, I think it might be a good idea that the print behavior revert to the older version's.

Issue 2:
I'm opening the issue just to have a reference

from sklearn import set_config
from lightgbm import LGBMClassifier
set_config(print_changed_only=True)
print(LGBMClassifier(metric='auc'))

will fail because metric is not part of the signature of init, it's part of a kwargs parameter.

Arguably, that's LightGBM not being super complient, but there's nothing in our docs explicitly preventing this.

Fixed by #17205