> Even when they may each have been built on different feature sets? I am really not so sure about this strategy. In any case, I wouldn't implement it as the default behaviour of a bagging estimator. This doesn't look like something standard in my opinion (in the context of ensembles). 

But then, regressors are affected by the same problem, right?