Package: optuna
Optuna: A hyperparameter optimization frameworkWebsite|Docs|Install Guide|Tutorial|ExamplesOptunais an automatic hyperparameter optimization software framework, particularly designed
for machine learning. It features an imperative,define-by-runstyle user API. Thanks to ourdefine-by-runAPI, the code written with Optuna enjoys high modularity, and the user of
Optuna can dynamically construct the search spaces for the hyperparameters.Key FeaturesOptuna has modern functionalities as follows:Lightweight, versatile, and platform agnostic architectureHandle a wide variety of tasks with a simple installation that has few requirements.Pythonic search spacesDefine search spaces using familiar Python syntax including conditionals and loops.Efficient optimization algorithmsAdopt state-of-the-art algorithms for sampling hyperparameters and efficiently pruning unpromising trials.Easy parallelizationScale studies to tens or hundreds or workers with little or no changes to the code.Quick visualizationInspect optimization histories from a variety of plotting functions.Basic ConceptsWe use the termsstudyandtrialas follows:Study: optimization based on an objective functionTrial: a single execution of the objective functionPlease refer to sample code below. The goal of astudyis to find out the optimal set of
hyperparameter values (e.g.,regressorandsvr_c) through multipletrials(e.g.,n_trials=100). Optuna is a framework designed for the automation and the acceleration of the
optimizationstudies.import...# Define an objective function to be minimized.defobjective(trial):# Invoke suggest methods of a Trial object to generate hyperparameters.regressor_name=trial.suggest_categorical('regressor',['SVR','RandomForest'])ifregressor_name=='SVR':svr_c=trial.suggest_float('svr_c',1e-10,1e10,log=True)regressor_obj=sklearn.svm.SVR(C=svr_c)else:rf_max_depth=trial.suggest_int('rf_max_depth',2,32)regressor_obj=sklearn.ensemble.RandomForestRegressor(max_depth=rf_max_depth)X,y=sklearn.datasets.fetch_california_housing(return_X_y=True)X_train,X_val,y_train,y_val=sklearn.model_selection.train_test_split(X,y,random_state=0)regressor_obj.fit(X_train,y_train)y_pred=regressor_obj.predict(X_val)error=sklearn.metrics.mean_squared_error(y_val,y_pred)returnerror# An objective value linked with the Trial object.study=optuna.create_study()# Create a new study.study.optimize(objective,n_trials=100)# Invoke optimization of the objective function.InstallationOptuna is available atthe Python Package Indexand onAnaconda Cloud.# PyPI$pipinstalloptuna# Anaconda Cloud$condainstall-cconda-forgeoptunaOptuna supports Python 3.7 or newer.Also, we also provide Optuna docker images onDockerHub.ExamplesExamples can be found inoptuna/optuna-examples.IntegrationsOptuna has integration features with various third-party libraries. Integrations can be found inoptuna/optuna-integrationand the document is availablehere. Integrations support libraries such as the following:CatalystCatboostDaskfastai (v2)KerasLightGBMMLflowMXNetPyTorchPyTorch IgnitePyTorch LightningTensorBoardTensorFlowtf.kerasWeights & BiasesXGBoostWeb DashboardOptuna Dashboardis a real-time web dashboard for Optuna.
You can check the optimization history, hyperparameter importances, etc. in graphs and tables.
You don't need to create a Python script to callOptuna's visualizationfunctions.
Feature requests and bug reports welcome!Installoptuna-dashboardvia pip:$ pip install optuna-dashboard
$ optuna-dashboard sqlite:///db.sqlite3
...
Listening on http://localhost:8080/
Hit Ctrl-C to quit.CommunicationGitHub Discussionsfor questions.GitHub Issuesfor bug reports and feature requests.ContributionAny contributions to Optuna are more than welcome!If you are new to Optuna, please check thegood first issues. They are relatively simple, well-defined and are often good starting points for you to get familiar with the contribution workflow and other developers.If you already have contributed to Optuna, we recommend the othercontribution-welcome issues.For general guidelines how to contribute to the project, take a look atCONTRIBUTING.md.ReferenceTakuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. 2019.
Optuna: A Next-generation Hyperparameter Optimization Framework. In KDD (arXiv).