Repository: EpistasisLab/tpot

Issue #300: version 0.6.7 abnormally terminates prior to completion with error "Cputime limit exceeded: 24"
Created: 2016-11-01T04:03:41Z
Using same inputs and script which worked in 0.6.6 I now receive "Cputime limit exceeded: 24" during fit operation.
Context of the issue
This is relevant as it seems to be an internal error which has been introduced with this new version of TPot.
Process to reproduce the issue
#use input file located here: https://1drv.ms/u/s!Agy-WTdBusDXndslUdhBWppnUn06xg
Code to reproduce:
from tpot import TPOTClassifier
from sklearn.model_selection import train_test_split
import pandas as pd
import matplotlib.pyplot as plt
import itertools
import numpy as np
avyDataAll = pd.read_csv('./filteredFeaturesForTPotV5.csv')
avyDataX = avyDataAll.ix[:, avyDataAll.columns != 'class'].values
avyDataY = avyDataAll.ix[:, avyDataAll.columns == 'class'].values
X_train, X_test, y_train, y_test = train_test_split(avyDataX, avyDataY,
train_size=0.75, test_size=0.25, stratify=avyDataY)
tpot = TPOTClassifier(generations=3, verbosity=2)
tpot.fit(features=X_train, classes=y_train[:,0]) #fails on this line
print(tpot.score(X_test, y_test[:,0]))
Expected result
Version 0.6.6 of tpot is outdated. Version 0.6.7 was released 5 days ago.
Timeout during evaluation of pipeline #11. Skipping to the next pipeline.
Generation 1 - Current best internal CV score: 0.7432473521247461
Generation 2 - Current best internal CV score: 0.7432473521247461
Generation 3 - Current best internal CV score: 0.7432473521247461
Best pipeline: LogisticRegression(input_matrix, 0.13, 35, False)
0.687315151813
Current result
(py35)Scotts-MacBook-Pro-2: scottcha$ python forecastTpot.py
Optimization Progress:  23%|████████████▏                                        | 92/400 [03:03<16:53,  3.29s/pipeline]Cputime limit exceeded: 24
running on macbook pro retina with OSX Sierra (10.12.1)

Duplicate issue #308: pipeline not fitted even though fit was called
Created: 2016-11-08T12:11:04Z
pipeline does not seem to be fitted even though fit was called
Context of the issue
Even though fit is executed when I try to obtain the best result I get the error
tpot = TPOTClassifier(verbosity=2, max_time_mins=10)
tpot.fit(X_train, y_train)
tpot.export('tpot_pipe.py')
print(tpot.score(X_test, y_test))


The error
[ValueError                                Traceback (most recent call last)
<ipython-input-10-7d10df09096d> in <module>()
----> 1 tpot.export('tpot_pipe.py')
      2 #print(tpot.score(X_test, y_test))

/usr/local/lib/python3.5/site-packages/tpot/base.py in export(self, output_file_name)
    471         """
    472         if self._optimized_pipeline is None:
--> 473             raise ValueError('A pipeline has not yet been optimized. Please call fit() first.')
    474 
    475         with open(output_file_name, 'w') as output_file:

ValueError: A pipeline has not yet been optimized. Please call fit() first.

Closing comment (2016-11-08T17:59:27Z): The default settings in TPOT on generation number and population size is population_size=100, generations=100. So, with generation 0, the number of run is 100*(100+1) = 10100. I think the dataset you used should be very huge and many pipeline were skiped due to a time limit for evaluating a single pipleline (max_eval_time_mins = 5 in default) .
To estimate the speed of simulation in the dataset,  I suggested reduce the generation number to ~10 and increase  max_eval_time_mins for your dataset by adding generations=10, max_eval_time_mins = 10 into TPOTClassifier. Also I suggest this time-consuming process should run in a linux platform.   A strange bug related to #300 was just found in MacOS on Macbook Pro and we will fix it in next version of TPOT. The current version of TPOT is stable in Linux (should be also all right in Windows.)