Package: dict-minimize
The Dict Minimize PackageAccessscipyoptimizers from your favorite deep learning framework.InstallationOnlyPython>=3.6is officially supported, but older versions of Python likely work as well.The core package itself can be installed with:pipinstalldict_minimizeTo also get the dependencies for all the supported frameworks (torch, JAX, tensorflow) in the README install withpipinstalldict_minimize[framework]See theGitHub,PyPI, andRead the Docs.Example UsageIn these examples we optimize a modifiedRosenbrockfunction.
However, the arguments have been split into two chunks and stored as two entries in a dictionary.
This is to illustrate how this package optimizesdictionariesof (tensor) parameters rather then vectors.
We also pass in an extrashiftargument to demonstrate howminimizeallows extra constant arguments to be passed into the objective.PyTorchimporttorchfromdict_minimize.torch_apiimportminimizedefrosen_obj(params,shift):"""Based on augmented Rosenbrock from botorch."""X,Y=params["x_half_a"],params["x_half_b"]X=X-shiftY=Y-shiftobj=100*(X[1]-X[0]**2)**2+100*(Y[1]-Y[0]**2)**2returnobjdefd_rosen_obj(params,shift):obj=rosen_obj(params,shift=shift)da,db=torch.autograd.grad(obj,[params["x_half_a"],params["x_half_b"]])d_obj=OrderedDict([("x_half_a",da),("x_half_b",db)])returnobj,d_objtorch.manual_seed(123)n_a=2n_b=2shift=-1.0params=OrderedDict([("x_half_a",torch.randn((n_a,))),("x_half_b",torch.randn((n_b,)))])params=minimize(d_rosen_obj,params,args=(shift,),method="L-BFGS-B",options={"disp":True})TensorFlowimporttensorflowastffromdict_minimize.tensorflow_apiimportminimizedefrosen_obj(params,shift):"""Based on augmented Rosenbrock from botorch."""X,Y=params["x_half_a"],params["x_half_b"]X=X-shiftY=Y-shiftobj=100*(X[1]-X[0]**2)**2+100*(Y[1]-Y[0]**2)**2returnobjdefd_rosen_obj(params,shift):withtf.GradientTape(persistent=True)ast:t.watch(params["x_half_a"])t.watch(params["x_half_b"])obj=rosen_obj(params,shift=shift)da=t.gradient(obj,params["x_half_a"])db=t.gradient(obj,params["x_half_b"])d_obj=OrderedDict([("x_half_a",da),("x_half_b",db)])delt# Explicitly drop the reference to the tapereturnobj,d_objtf.random.set_seed(123)n_a=2n_b=2shift=-1.0params=OrderedDict([("x_half_a",tf.random.normal((n_a,))),("x_half_b",tf.random.normal((n_b,)))])params=minimize(d_rosen_obj,params,args=(shift,),method="L-BFGS-B",options={"disp":True})NumPyimportnumpyasnpfromscipy.optimizeimportrosen,rosen_derfromdict_minimize.numpy_apiimportminimizedefrosen_obj(params,shift):val=rosen(params["x_half_a"]-shift)+rosen(params["x_half_b"]-shift)dval=OrderedDict([("x_half_a",rosen_der(params["x_half_a"]-shift)),("x_half_b",rosen_der(params["x_half_b"]-shift)),])returnval,dvalnp.random.seed(0)n_a=3n_b=5shift=-1.0params=OrderedDict([("x_half_a",np.random.randn(n_a)),("x_half_b",np.random.randn(n_b))])params=minimize(rosen_obj,params,args=(shift,),method="L-BFGS-B",options={"disp":True})JAXfromjaximportrandom,value_and_gradimportjax.numpyasnpfromdict_minimize.jax_apiimportminimizedefrosen(x):r=np.sum(100.0*(x[1:]-x[:-1]**2.0)**2.0+(1-x[:-1])**2.0,axis=0)returnrdefrosen_obj(params,shift):val=rosen(params["x_half_a"]-shift)+rosen(params["x_half_b"]-shift)returnvaln_a=3n_b=5shift=-1.0# Jax makes it this simple:d_rosen_obj=value_and_grad(rosen_obj,argnums=0)# Setup randomness in JAXkey=random.PRNGKey(0)key,subkey_a=random.split(key)key,subkey_b=random.split(key)params=OrderedDict([("x_half_a",random.normal(subkey_a,shape=(n_a,))),("x_half_b",random.normal(subkey_b,shape=(n_b,)))])params=minimize(d_rosen_obj,params,args=(shift,),method="L-BFGS-B",options={"disp":True})ContributingThe following instructions have been tested with Python 3.7.4 on Mac OS (10.14.6).Install in editable modeFirst, define the variables for the paths we will use:GIT=/path/to/where/you/put/reposENVS=/path/to/where/you/put/virtualenvsThen clone the repo in your git directory$GIT:cd$GITgitclonehttps://github.com/twitter/dict_minimize.gitInside your virtual environments folder$ENVS, make the environment:cd$ENVSvirtualenvdict_minimize--python=python3.7source$ENVS/dict_minimize/bin/activateNow we can install the pip dependencies. Move back into your git directory and runcd$GIT/dict_minimizepipinstall-rrequirements/base.txtpipinstall-e.# Install the package itselfContributor toolsFirst, we need to setup some needed tools:cd$ENVSvirtualenvdict_minimize_tools--python=python3.7source$ENVS/dict_minimize_tools/bin/activatepipinstall-r$GIT/dict_minimize/requirements/tools.txtTo install the pre-commit hooks for contributing run (in thedict_minimize_toolsenvironment):cd$GIT/dict_minimizepre-commitinstallTo rebuild the requirements, we can run:cd$GIT/dict_minimize# Check if there any discrepancies in the .in filespipreqsdict_minimize/core/--diffrequirements/base.inpipreqsdict_minimize/--diffrequirements/frameworks.inpipreqstests/--diffrequirements/tests.inpipreqsdocs/--diffrequirements/docs.in# Regenerate the .txt files from .in filespip-compile-multi--no-upgradeGenerating the documentationFirst setup the environment for building withSphinx:cd$ENVSvirtualenvdict_minimize_docs--python=python3.7source$ENVS/dict_minimize_docs/bin/activatepipinstall-r$GIT/dict_minimize/requirements/docs.txtThen we can do the build:cd$GIT/dict_minimize/docsmakeallopen_build/html/index.htmlDocumentation will be available in all formats inMakefile. Usemake htmlto only generate the HTML documentation.Running the testsThe tests for this package can be run with:cd$GIT/dict_minimize./local_test.shThe script creates an environment using the requirements found inrequirements/test.txt.
A code coverage report will also be produced in$GIT/dict_minimize/htmlcov/index.html.DeploymentThe wheel (tar ball) for deployment as a pip installable package can be built using the script:cd$GIT/dict_minimize/./build_wheel.shThis script will only run if the git repo is clean, i.e., first rungit clean-x-ff-d.LinksThesourceis hosted on GitHub.Thedocumentationis hosted at Read the Docs.Installable fromPyPI.LicenseThis project is licensed under the Apache 2 License - see the LICENSE file for details.