Issue 1:
Description

Until recently I was using RandomForestClassifier from scikit version 0.17. I was using a cross-validation procedure based on different groups of my dataset, so I had created a function similar to that of LeaveOneGroupOut from version 0.18. That function creates around 300 different data splits (as I have 300 different groups, so each time all instances of a particular group are used as test set and the rest are used for training) and then I apply RandomForestClassifier to each one of these data splits (see code below).

As I wanted to use some functions from the next version, I upgraded scikit-learn to version 0.18.1. I realised that using exactly the same script and dataset, the overall process takes almost triple time running the RandomForestClassifier among these 300 cross-validation iterations for the set. This time I utilise the LeaveOneGroupOut function present in version 0.18.1, but by alternatively using my previous in-house made function to do the same thing, I have exactly the same problem after upgrading.

By examining a little bit what is going on, I saw that the problem was during the initialisation of the random forest predictor and not while creating the sets. As I have 100 different datasets where I apply this procedure, it takes a lot more time to finish my analysis.

Did someone else experience this problem? Otherwise, I will check again in case something else messes up with the running time.

Thank you!

Steps/Code to Reproduce

#create data split (I save them first in one matrix, because I want to edit them later)
logo=LeaveOneGroupOut()
data_split=[]
for train_index,test_index in logo.split(instance_set,label_set,groups=instance_groups):
data_split+=[[train_index,test_index]]

#iterate over the data splits and create random forest for each
for train_index,test_index in data_split:
train_set=[instance_set[i] for i in train_index]
test_set=[instance_set[i] for i in test_index]
train_label=[label_set[i] for i in train_index]
test_label=[label_set[i] for i in test_index]

#create Random Forest instance
clf=RandomForestClassifier(n_estimators=100,max_depth=10)
new_clf=clf.fit(train_set,train_label)

Expected Results

Running time for one dataset with 300 cross-validation iterations using LeaveOneGroupOut (before upgrading): 1m

Actual Results

Running time for one dataset with 300 cross-validation iterations using LeaveOneGroupOut (after upgrading): 3m

Versions

Linux-3.13.0-85-generic-x86_64-with-Ubuntu-14.04-trusty

('Python', '2.7.6 (default, Jun 22 2015, 17:58:13) \n[GCC 4.8.2]')

('NumPy', '1.8.2')

('SciPy', '0.13.3')

('Scikit-Learn', '0.18.1')

Issue 2:
Description

After upgrading to 0.18.1 predict of ExtraTreesRegressor is about 10 times slower.

Version 0.17

Linux-4.4.0-57-generic-x86_64-with-Ubuntu-16.04-xenial
('Python', '2.7.12 (default, Nov 19 2016, 06:48:10) \n[GCC 5.4.0 20160609]')
('NumPy', '1.11.3')
('SciPy', '0.18.1')
('Scikit-Learn', '0.17')
fit: 1.26665902138
predict: 0.00265598297119

Version 0.18.1

Linux-4.4.0-57-generic-x86_64-with-Ubuntu-16.04-xenial
('Python', '2.7.12 (default, Nov 19 2016, 06:48:10) \n[GCC 5.4.0 20160609]')
('NumPy', '1.11.3')
('SciPy', '0.18.1')
('Scikit-Learn', '0.18.1')
fit: 1.34347200394
predict: 0.0433509349823

Steps/Code to Reproduce

import platform; print(platform.platform())
import sys; print("Python", sys.version)
import numpy; print("NumPy", numpy.__version__)
import scipy; print("SciPy", scipy.__version__)
import sklearn; print("Scikit-Learn", sklearn.__version__)

from sklearn.datasets import make_friedman3
from sklearn.ensemble import ExtraTreesRegressor
from time import time

numpy.random.seed(6652)

X, Y = make_friedman3(10000)

forest = ExtraTreesRegressor(n_estimators=100)
start = time()
forest = forest.fit(X, Y)
print 'fit: {}'.format(time() - start)
start = time()
output = forest.predict(numpy.array([[1, 2., 5, 32.]]))
print 'predict: {}'.format(time() - start)

I installed/uninstalled through pip

pip install scikit-learn==0.17 --no-cache-dir
pip uninstall scikit-learn
pip install scikit-learn==0.18.1 --no-cache-dir

Cython version 0.23.4