Package: deephyper
What is DeepHyper?DeepHyper is a powerful Python package for automating machine learning tasks, particularly focused on optimizing hyperparameters, searching for optimal neural architectures, and quantifying uncertainty through the use of deep ensembles. With DeepHyper, users can easily perform these tasks on a single machine or distributed across multiple machines, making it ideal for use in a variety of environments. Whether you're a beginner looking to optimize your machine learning models or an experienced data scientist looking to streamline your workflow, DeepHyper has something to offer. So why wait? Start using DeepHyper today and take your machine-learning skills to the next level!Install InstructionsInstallation withpip:#Forthemostbasicsetoffeatures(hyperparametersearch)pip install deephyper#Forthedefaultsetoffeaturesincluding:#-hyperparametersearchwithtransfer-learning#-neuralarchitecturesearch#-deepensembles#-Ray-baseddistributedcomputing#-Learning-curveextrapolationformulti-fidelityhyperparametersearchpip install "deephyper[default]"More details about the installation process can be found atDeepHyper Installations.QuickstartThe black-box function namedrunis defined by taking an input dictionary namedconfig`` which contains the different variables to optimize. Then the run-function is bound to anEvaluatorin charge of distributing the computation of multiple evaluations. Finally, a Bayesian search namedCBOis created and executed to find the values of config which **MAXIMIZE** the return value ofrun(config)`.defrun(job):# The suggested parameters are accessible in job.parameters (dict)x=job.parameters["x"]b=job.parameters["b"]ifjob.parameters["function"]=="linear":y=x+belifjob.parameters["function"]=="cubic":y=x**3+b# Maximization!returny# Necessary IF statement otherwise it will enter in a infinite loop# when loading the 'run' function from a new processif__name__=="__main__":fromdeephyper.problemimportHpProblemfromdeephyper.search.hpsimportCBOfromdeephyper.evaluatorimportEvaluator# define the variable you want to optimizeproblem=HpProblem()problem.add_hyperparameter((-10.0,10.0),"x")# real parameterproblem.add_hyperparameter((0,10),"b")# discrete parameterproblem.add_hyperparameter(["linear","cubic"],"function")# categorical parameter# define the evaluator to distribute the computationevaluator=Evaluator.create(run,method="process",method_kwargs={"num_workers":2,},)# define your search and execute itsearch=CBO(problem,evaluator,random_state=42)results=search.search(max_evals=100)print(results)Which outputs the following results where the best parameters are withfunction == "cubic",x == 9.99andb == 10.p:b p:function       p:x    objective  job_id  m:timestamp_submit  m:timestamp_gather
0     7     linear  8.831019    15.831019       1            0.064874            1.430992
1     4     linear  9.788889    13.788889       0            0.064862            1.453012
2     0      cubic  2.144989     9.869049       2            1.452692            1.468436
3     9     linear -9.236860    -0.236860       3            1.468123            1.483654
4     2      cubic -9.783865  -934.550818       4            1.483340            1.588162
..  ...        ...       ...          ...     ...                 ...                 ...
95    6      cubic  9.862098   965.197192      95           13.538506           13.671872
96   10      cubic  9.997512  1009.253866      96           13.671596           13.884530
97    6      cubic  9.965615   995.719961      97           13.884188           14.020144
98    5      cubic  9.998324  1004.497422      98           14.019737           14.154467
99    9      cubic  9.995800  1007.740379      99           14.154169           14.289366The code defines a functionrunthat takes a RunningJobjobas input and returns the maximized objectivey. Theifblock at the end of the code defines a black-box optimization process using theCBO(Centralized Bayesian Optimization) algorithm from thedeephyperlibrary.The optimization process is defined as follows:A hyperparameter optimization problem is created using theHpProblemclass fromdeephyper. In this case, the problem has three variables. Thexhyperparameter is a real variable in a range from -10.0 to 10.0. Thebhyperparameter is a discrete variable in a range from 0 to 10. Thefunctionhyperparameter is a categorical variable with two possible values.An evaluator is created using theEvaluator.createmethod. The evaluator will be used to evaluate the functionrunwith different configurations of suggested hyperparameters in the optimization problem. The evaluator uses theprocessmethod to distribute the evaluations across multiple worker processes, in this case, 2 worker processes.A search object is created using theCBOclass, the problem and evaluator defined earlier. TheCBOalgorithm is a derivative-free optimization method that uses a Bayesian optimization approach to explore the hyperparameter space.The optimization process is executed by calling thesearch.searchmethod, which performs the evaluations of therunfunction with different configurations of the hyperparameters until a maximum number of evaluations (100 in this case) is reached.The results of the optimization process, including the optimal configuration of the hyperparameters and the corresponding objective value, are printed to the console.How do I learn more?Documentation:https://deephyper.readthedocs.ioGitHub repository:https://github.com/deephyper/deephyperBlog:https://deephyper.github.ioContributionsFind the list of contributors on theDeepHyper Authorspage of the Documentation.Citing DeepHyperIf you wish to cite the Software, please use the following:@misc{deephyper_software,
title = {"DeepHyper: A Python Package for Scalable Neural Architecture and Hyperparameter Search"},
author = {{DeepHyper Development Team}},
organization = {DeepHyper Team},
year = 2018,
url = {https://github.com/deephyper/deephyper}
}Find all our publications on theResearch & Publicationpage of the Documentation.How can I participate?Questions, comments, feature requests, bug reports, etc. can be directed to:Issues on GitHubPatches through pull requests are much appreciated on the software itself as well as documentation.
Optionally, please include in your first patch a credit for yourself in the list above.The DeepHyper Team uses git-flow to organize the development:Git-Flow cheatsheet. For tests we are using:Pytest.AcknowledgmentsScalable Data-Efficient Learning for Scientific Domains, U.S. Department of Energy 2018 Early Career Award funded by the Advanced Scientific Computing Research program within the DOE Office of Science (2018--Present)Argonne Leadership Computing Facility: This research used resources of the Argonne Leadership Computing Facility, which is a DOE Office of Science User Facility supported under Contract DE-AC02-06CH11357.SLIK-D: Scalable Machine Learning Infrastructures for Knowledge Discovery, Argonne Computing, Environment and Life Sciences (CELS) Laboratory Directed Research and Development (LDRD) Program (2016--2018)Copyright and licenseCopyright © 2019, UChicago Argonne, LLCDeepHyper is distributed under the terms of BSD License. SeeLICENSEArgonne Patent & Intellectual Property File Number: SF-19-007