Title: Scikit learn Iterative Imputer : change and scaled tolerance
Tags: python,scikit-learn,missing-data

Question: As part of a school project, I have to explore and perform data analysis and machine learning methods on a given database. The point is that my database is pretty big (12 651 lines for 810 columns) and contains a lot of missing values.
I wanted to impute these values with the Iterative Imputer of Scikit-learn, and this is what I get:
imp = IterativeImputer(estimator = LinearRegression(), 
                       max_iter=10, 
                       random_state=0, 
                       verbose=2, 
                       n_nearest_features=10, 
                       initial_strategy="most_frequent")

imp.fit(data)

Results:
IterativeImputer] Completing matrix with shape (12651, 810)
[IterativeImputer] Ending imputation round 1/10, elapsed time 8.38
[IterativeImputer] Change: 101844979.96577276, scaled tolerance: 208867.02000000002 
[IterativeImputer] Ending imputation round 2/10, elapsed time 18.76
[IterativeImputer] Change: 633298988.0233588, scaled tolerance: 208867.02000000002 
[IterativeImputer] Ending imputation round 3/10, elapsed time 28.81
[IterativeImputer] Change: 591554347.9059296, scaled tolerance: 208867.02000000002 
[IterativeImputer] Ending imputation round 4/10, elapsed time 37.43
[IterativeImputer] Change: 1289773197.9995384, scaled tolerance: 208867.02000000002 
[IterativeImputer] Ending imputation round 5/10, elapsed time 46.58
[IterativeImputer] Change: 1291562921.1247401, scaled tolerance: 208867.02000000002 
[IterativeImputer] Ending imputation round 6/10, elapsed time 54.32
[IterativeImputer] Change: 32943821.50762498, scaled tolerance: 208867.02000000002 
[IterativeImputer] Ending imputation round 7/10, elapsed time 64.13
[IterativeImputer] Change: 58342050.73579848, scaled tolerance: 208867.02000000002 
[IterativeImputer] Ending imputation round 8/10, elapsed time 73.44
[IterativeImputer] Change: 1559818227.7418892, scaled tolerance: 208867.02000000002 
[IterativeImputer] Ending imputation round 9/10, elapsed time 81.46
[IterativeImputer] Change: 164792431487.71582, scaled tolerance: 208867.02000000002 
[IterativeImputer] Ending imputation round 10/10, elapsed time 90.07
[IterativeImputer] Change: 13045775634991.55, scaled tolerance: 208867.02000000002
/usr/local/lib/python3.9/dist-packages/sklearn/impute/_iterative.py:785: ConvergenceWarning: [IterativeImputer] Early stopping criterion not reached.
  warnings.warn(

I don't know if i can do anything to make it converge?
Thanks!
P.s: One thing I did not mention is that many of the columns actually represent categorical variables (but pandas casted these columns as float64 because of the NaN values).
What I tried: changing the estimator from BayesianRidge to LinearRegressor, setting n_nearest_features = 10 and 20, setting initial_strategy="most_frequent" instead of "mean". It did not seem to work either :/

Answer: There are a few different things you can try to get better results with IterativeImputer. The first thing I would do is try some regularization by using either Ridge or ElasticNet as your estimator, and increasing max_iter to ~50 to see if your results ever stabilize.
In my personal experience I also have better success if I scale the data before imputing and choose initial_strategy="median" rather than 'most_frequent'.
If you have a mix of continuous and categorical features, you will likely need a nonlinear estimator and will need to postprocess the results by re-quantizing the categorical features.