Issue 1:
Description

IterativeImputer seems to give pretty rubbish values when using the two-step fit() and transform() on a train/test data split versus simply using fit_transform() on the entire dataset. Issue #14338 may be related.

Steps/Code to Reproduce

Here's a fairly minimal example:

from sklearn.experimental import enable_iterative_imputer
from sklearn.impute import IterativeImputer
from sklearn.linear_model import LinearRegression
import numpy as np

# x_0 = x_1 + x_2 + x_3
train = np.array([
[ 5, 2, 2, 1],
[10, 1, 2, 7],
[ 3, 1, 1, 1],
[ 8, 4, 2, 2]
])

test = np.array([
[np.nan, 2, 4, 5],
[np.nan, 4, 1, 2],
[np.nan, 1, 10, 1]
])

y_actual = np.array([11, 7, 12])

## Prediction using IterativeImputer and the 1-step fit_transform()
imputer = IterativeImputer(estimator = LinearRegression())
fullData = np.concatenate([train,test])
imputedData = imputer.fit_transform(fullData)
y_1imputed = imputedData[-3:,0]

## Prediction using IterativeImputer and the 2-step fit() / transform()
imputer2 = IterativeImputer(estimator = LinearRegression())
imputer2 = imputer2.fit(train)
imputedTest = imputer2.transform(test)
y_2imputed = imputedTest[:,0]

print('Actual y: {}'.format(y_actual))
print('1-step Imputed y: {}'.format(y_1imputed))
print('2-step Imputed y: {}'.format(y_2imputed))

Expected Results

Actual y: [11 7 12]
1-step Imputed y: [11. 7. 12.]
2-step Imputed y: [11. 7. 12.]

Actual Results

Actual y: [11 7 12]
1-step Imputed y: [11. 7. 12.]
2-step Imputed y: [6.5 6.5 6.5]

Versions

System:
python: 3.7.2 (default, Dec 29 2018, 06:19:36) [GCC 7.3.0]
executable: /home/jack/miniconda3/bin/python
machine: Linux-4.15.0-54-generic-x86_64-with-debian-buster-sid

BLAS:
macros: NO_ATLAS_INFO=1, HAVE_CBLAS=None
lib_dirs: /usr/lib/x86_64-linux-gnu
cblas_libs: cblas

Python deps:
pip: 19.1.1
setuptools: 40.2.0
sklearn: 0.22.dev0
numpy: 1.15.3
scipy: 1.1.0
Cython: 0.29.12
pandas: 0.24.2
matplotlib: 3.0.2
joblib: 0.13.2

Issue 2:
Why is this behaviour forced:

Features with missing values during transform which did not have any missing values during fit will be imputed with the initial imputation method only.

https://scikit-learn.org/dev/modules/generated/sklearn.impute.IterativeImputer.html#sklearn.impute.IterativeImputer

This means by default it will return the mean of that feature. I would prefer just fit one iteration of the chosen estimator and use that fitted estimator to impute missing values.

Actual behaviour:

Example - The second feature missing np.nan --> mean imputation

import numpy as np
from sklearn.impute import IterativeImputer
imp = IterativeImputer(max_iter=10, verbose=0)
imp.fit([[1, 2], [3, 6], [4, 8], [10, 20], [np.nan, 22], [7, 14]])

X_test = [[np.nan, 4], [6, np.nan], [np.nan, 6], [4, np.nan], [33, np.nan]]
print(np.round(imp.transform(X_test)))

Return:
[[ 2. 4.]
[ 6. 12.]
[ 3. 6.]
[ 4. 12.]
[33. 12.]]

Example adjusted - Second feature has np.nan values --> iterative imputation with estimator

import numpy as np
from sklearn.impute import IterativeImputer
imp = IterativeImputer(max_iter=10, verbose=0)
imp.fit([[1, 2], [3, 6], [4, 8], [10, 20], [np.nan, 22], [7, np.nan]])

X_test = [[np.nan, 4], [6, np.nan], [np.nan, 6], [4, np.nan], [33, np.nan]]
print(np.round(imp.transform(X_test)))

Return:
[[ 2. 4.]
[ 6. 12.]
[ 3. 6.]
[ 4. 8.]
[33. 66.]]

Maybe sklearn/impute.py line 679 to 683 should be optional with a parameter like force-iterimpute.