Package: sklearn-genetic-opt
Sklearn-genetic-optscikit-learn models hyperparameters tuning and feature selection, using evolutionary algorithms.This is meant to be an alternative to popular methods inside scikit-learn such as Grid Search and Randomized Grid Search
for hyperparameteres tuning, and from RFE, Select From Model for feature selection.Sklearn-genetic-opt uses evolutionary algorithms from the DEAP package to choose the set of hyperparameters that
optimizes (max or min) the cross-validation scores, it can be used for both regression and classification problems.Documentation is availablehereMain Features:GASearchCV: Main class of the package for hyperparameters tuning, holds the evolutionary cross-validation optimization routine.GAFeatureSelectionCV: Main class of the package for feature selection.Algorithms: Set of different evolutionary algorithms to use as an optimization procedure.Callbacks: Custom evaluation strategies to generate early stopping rules,
logging (into TensorBoard, .pkl files, etc) or your custom logic.Schedulers: Adaptive methods to control learning parameters.Plots: Generate pre-defined plots to understand the optimization process.MLflow: Build-in integration with mlflow to log all the hyperparameters, cv-scores and the fitted models.Demos on Features:Visualize the progress of your training:Real-time metrics visualization and comparison across runs:Sampled distribution of hyperparameters:Artifacts logging:Usage:Install sklearn-genetic-optIt’s advised to install sklearn-genetic using a virtual env, inside the env use:pip install sklearn-genetic-optIf you want to get all the features, including plotting, tensorboard and mlflow logging capabilities,
install all the extra packages:pip install sklearn-genetic-opt[all]Example: Hyperparameters Tuningfromsklearn_geneticimportGASearchCVfromsklearn_genetic.spaceimportContinuous,Categorical,Integerfromsklearn.ensembleimportRandomForestClassifierfromsklearn.model_selectionimporttrain_test_split,StratifiedKFoldfromsklearn.datasetsimportload_digitsfromsklearn.metricsimportaccuracy_scoredata=load_digits()n_samples=len(data.images)X=data.images.reshape((n_samples,-1))y=data['target']X_train,X_test,y_train,y_test=train_test_split(X,y,test_size=0.33,random_state=42)clf=RandomForestClassifier()param_grid={'min_weight_fraction_leaf':Continuous(0.01,0.5,distribution='log-uniform'),'bootstrap':Categorical([True,False]),'max_depth':Integer(2,30),'max_leaf_nodes':Integer(2,35),'n_estimators':Integer(100,300)}cv=StratifiedKFold(n_splits=3,shuffle=True)evolved_estimator=GASearchCV(estimator=clf,cv=cv,scoring='accuracy',population_size=20,generations=35,param_grid=param_grid,n_jobs=-1,verbose=True,keep_top_k=4)# Train and optimize the estimatorevolved_estimator.fit(X_train,y_train)# Best parameters foundprint(evolved_estimator.best_params_)# Use the model fitted with the best parametersy_predict_ga=evolved_estimator.predict(X_test)print(accuracy_score(y_test,y_predict_ga))# Saved metadata for further analysisprint("Stats achieved in each generation: ",evolved_estimator.history)print("Best k solutions: ",evolved_estimator.hof)Example: Feature Selectionfromsklearn_geneticimportGAFeatureSelectionCV,ExponentialAdapterfromsklearn.model_selectionimporttrain_test_splitfromsklearn.svmimportSVCfromsklearn.datasetsimportload_irisfromsklearn.metricsimportaccuracy_scoreimportnumpyasnpdata=load_iris()X,y=data["data"],data["target"]# Add random non-important featuresnoise=np.random.uniform(5,10,size=(X.shape[0],5))X=np.hstack((X,noise))X_train,X_test,y_train,y_test=train_test_split(X,y,test_size=0.33,random_state=0)clf=SVC(gamma='auto')mutation_scheduler=ExponentialAdapter(0.8,0.2,0.01)crossover_scheduler=ExponentialAdapter(0.2,0.8,0.01)evolved_estimator=GAFeatureSelectionCV(estimator=clf,scoring="accuracy",population_size=30,generations=20,mutation_probability=mutation_scheduler,crossover_probability=crossover_scheduler,n_jobs=-1)# Train and select the featuresevolved_estimator.fit(X_train,y_train)# Features selected by the algorithmfeatures=evolved_estimator.support_print(features)# Predict only with the subset of selected featuresy_predict_ga=evolved_estimator.predict(X_test)print(accuracy_score(y_test,y_predict_ga))# Transform the original data to the selected featuresX_reduced=evolved_estimator.transform(X_test)ChangelogSee thechangelogfor notes on the changes of Sklearn-genetic-optImportant linksOfficial source code repo:https://github.com/rodrigo-arenas/Sklearn-genetic-opt/Download releases:https://pypi.org/project/sklearn-genetic-opt/Issue tracker:https://github.com/rodrigo-arenas/Sklearn-genetic-opt/issuesStable documentation:https://sklearn-genetic-opt.readthedocs.io/en/stable/Source codeYou can check the latest development version with the command:git clone https://github.com/rodrigo-arenas/Sklearn-genetic-opt.gitInstall the development dependencies:pip install -r dev-requirements.txtCheck the latest in-development documentation:https://sklearn-genetic-opt.readthedocs.io/en/latest/ContributingContributions are more than welcome!
There are several opportunities on the ongoing project, so please get in touch if you would like to help out.
Make sure to check the current issues and also
theContribution guide.Big thanks to the people who are helping with this project!TestingAfter installation, you can launch the test suite from outside the source directory:pytest sklearn_genetic