Package: a2dr
a2dra2dris a Python package for solving large-scale non-smooth convex optimization problems with general linear constraints, with separable objective functions accessible through their proximal operators. It exploits the separability of the objective functions and the sparsity in the linear constraints, and utilizes the power of Anderson acceleration to achieve fast and robust convergence and scalability to multiple processors. The current version is0.2.3.post2.It is an implementation of type-II Anderson accelerated Douglas-Rachford splitting, based on our paperA. Fu, J. Zhang, and S. Boyd (2019).InstallationTo install using pip (recommended), run:pip install a2drTo installa2drfrom source, first make sure that you havesetuptoolsinstalled. Then follow the steps below:Clone thea2drgit repository.Navigate to the top-level of the cloned directory and run:pythonsetup.pyinstallTo test the installation with nose, first make sure that you havenoseinstalled. Then run:nosetestsa2drThe requirements are:MatplotlibCVXPYNumPySciPyPython 3.xPlease file an issue on Github if you want Python 2 support.Problema2drsolves problems of the following form:minimize         f_1(x_1) + ... + f_N(x_N)
subject to       A_1x_1 + ... + A_Nx_N = b.where f_i (i=1,...,N) are convex, closed and proper, and are only accessible through their proximal operators. Notice that f_i can take infinite values, and in particular constraints can be included in the objectives with the help of indicator functions.Prox-affine formsThe above formulation is also referred to asprox-affineforms in the literature (see e.g.,Epsilon). When it is seen as a standard form for generic convex optimization problems, the major advantage ofprox-affineforms compared to the more widely usedconicforms include:Privacy: suitable for peer-to-peer optimization with privacy requirements.In practice, the data and source code that define the proximal oracle can be securely encrypted (e.g., via compilation) so that privacy is preserved. For example, in Python, we can convert the.pyfile containing the proximal operator function into an encrypted.sofile via theCythonextension.Compactness: straightforward canonicalization/transformation and lower dimensional representations.In general, the prox-affine form often requires less variables than the conic form (see e.g., the portfolio optimization examplehere). The compactness advantage is also partly exemplified by the comparison betweena2drandSCSin the sparse covariance estimation example in ourpaper.For a bit more detailed introduction to prox-affine forms and the comparisons with conic forms, see ourcompanion slides.UsageAfter installinga2dr, you can importa2drusingimporta2drThis module exposes a functiona2dr(the solver), which can be used viaa2dr.a2dr, or directly imported usingfroma2drimporta2drThe functiona2dris called with the commanda2dr_result=a2dr(p_list,A_list=[],b=np.array([]),v_init=None,n_list=None,max_iter=1000,t_init=1/10,eps_abs=1e-6,eps_rel=1e-8,precond=True,ada_reg=True,anderson=True,m_accel=10,lam_accel=1e-8,aa_method='lstsq',D_safe=1e6,eps_safe=1e-6,M_safe=int(max_iter/100),verbose=True)Parameters:The argumentsp_list,A_listandbcorrespond to the problem data.p_listis the list of proximal operators of f_i. Each element ofp_listis a Python function,
which takes as input a vector v and parameter t > 0 and outputs the proximal operator of f_i evaluated at (v,t).A_listis the list of A_i. The listsp_listandA_listmust be given in the same order i = 1,...,N.bis the vector b.
Notice thatA_listandbare optional, and when omitted, the solver recognizes the problem as one without linear constraints. Also notice that in such cases,A_listandbhave to be omitted together, and eitherv_initorn_listhas to be provided to declare the dimension of each x_i.For information on the other optional hyper-parameters, please refer to ourcompanion paper(Algorithm 2) and thesource code comments of the functiona2drin solver.py.Returns:The returned objecta2dr_resultis a dictionary containing the keys'x_vals','primal','dual','num_iters'and'solve_time':The outputx_valsis a list of x_1,...,x_N from the iteration with the smallest residuals.primalanddualare arrays containing the primal and dual residual norms for the entire iteration process, respectively.The valuenum_itersis the total number of iterations, andsolve_timeis the algorithm runtime.When the linear equality constraint is infeasible, the solver will print the corresponding flag and returnNonefor all the outputs.Other toolsThe modulea2dralso comes with several additional tools that facilitates the transformation of the problems into the required input format described above as well as tests and visualization. In particular, it come with apackage for proximal operators, which can be imported viaimporta2dr.proximalIt also comes with sometests and visualization tools, which can be imported viaimporta2dr.testsExampleWe showcase the usage of the solver functiona2dras well as the the tool packagesa2dr.proximalanda2dr.testswith the following example. More examples can be found in theexamples/directory.# Non-negative least squares (see our companion paper for more details)importnumpyasnpimportnumpy.linalgfromscipyimportsparsefroma2drimporta2drfroma2dr.proximalimport*froma2dr.tests.base_testimportBaseTest# Problem data.np.random.seed(1)m,n=150,300density=0.001X=sparse.random(m,n,density=density,data_rvs=np.random.randn)y=np.random.randn(m)# Convert problem to standard form.prox_list=[lambdav,t:prox_sum_squares_affine(v,t,F=X,g=y),prox_nonneg_constr]A_list=[sparse.eye(n),-sparse.eye(n)]b=np.zeros(n)# Solve with DRS.drs_result=a2dr(prox_list,A_list,b,anderson=False)# Solve with A2DR.a2dr_result=a2dr(prox_list,A_list,b,anderson=True)bt=BaseTest()bt.compare_total(drs_result,a2dr_result)CitingIf you wish to citea2dr, please use the following:@article{a2dr,
    author       = {Fu, A. and Zhang, J. and Boyd, S.},
    title        = {Anderson Accelerated {D}ouglas-{R}achford Splitting},
    journal      = {http://stanford.edu/~boyd/papers/a2dr.html},
    year         = {2019},
}

@misc{a2dr_code,
    author       = {Fu, A. and Zhang, J. and Boyd, S.},
    title        = {{a2dr}: Anderson Accelerated {D}ouglas-{R}achford Splitting, version 0.2.3.post2},
    howpublished = {\url{https://github.com/cvxgrp/a2dr}},
    year         = {2020}
}