Package: numpy2ad
Numpy2ADA pure Python package forsource-to-sourcetransformation of Numpy matrix expressions toreverse-mode, also referred to asadjoint, algorithmic differentiation code.Algorithmic / Automatic Differentiationis the state of the art for computing derivatives in many fields of application, e.g. computational engineering, finance, and machine learning (backpropagation). For more details,this bookis a good entry point to the subject.UsageNumpy2ADoffers two modes of adjoint code generation.InExpression Modea given Numpy matrix expression string, e.g.fromnumpy2adimporttransform_exprprint(transform_expr("D = A @ B + C"))is transformed to its reverse-mode differentiation code:"""v0 = A @ BD = v0 + Cv0_a = np.zeros_like(v0)C_a += D_av0_a += D_aB_a += A.T @ v0_aA_a += v0_a @ B.T"""The generated code can be easily pasted into the desired context, e.g. a Jupyter notebook, where the adjoints (partial derivatives)A_a,B_a,C_a, andD_aare initialized. Expression mode is best suited for quick scripting and debugging.InFunction Modea given function, e.g.defmma(A,B,C):returnA@B+Cis transformed to its adjoint function with modified signature:fromnumpy2adimporttransformprint(transform(mma))"""import numpy as npdef mma_ad(A, B, C, A_a, B_a, C_a, out_a):v0 = A @ Bout = v0 + Cv0_a = np.zeros_like(v0)C_a += out_av0_a += out_aB_a += A.T @ v0_aA_a += v0_a @ B.Treturn (out, A_a, B_a, C_a)"""The transformed code can also be conveniently exported as a Python module with the argumentout_file=...for validation and easier integration into your existing workflows.Install$python-mpipinstallnumpy2adSupported Matrix OperationsNumpy2Adcurrently supports a limited but broadly applicable subset of Numpy matrix operations. We define a "matrix" as a two-dimensionalnumpy.ndarray. If not further specified, the operations are also valid for one-dimensional "vectors".MatrixAdditionC = A + BandSubtractionC = A - BMatrix (Inner)ProductsC = A @ BException: Inner products between two vectorsc = a @ bas they result in a scalar.MatrixInverseB = np.linalg.inv(A)MatrixTransposeB = A.TElement-wiseproductC = A * BNote that dimension ofAandBmust match. Numpy's broadcasting rules arenotconsidered during code generation, e.g. multiplying a scalar variable by a matrix will lead toincorrectderivative code.LimitationsThe adjoint compiler currently only generatesfirst order,reverse-modederivative code. Forward-mode ("tangents") ist not supported.The expression must bealiasing-free, i.e.A = A @ Bis not allowed.The input code must not contain any control flow (if ... else), nested functions, loops, or other non-differentiable subroutines.Demo, Tests, and BenchmarksCheck out/demo/example_problems.ipynbfor a collection of matrix models thatNumpy2ADwas applied to and tested on. A more in-depth user guide can be found in/demo/tutorial.ipynb.Tests of code generation and numerical correctness of derivatives, compared to naive finite differences, can be found in/tests/.Furthermore, there are some simple runtime benchmarks for theMMAandGLSmodel in/benchmarks/.Building the Package from SourceCreate a conda environment from the main directory$condaenvcreate--fileenvironment.yml
...
$condaactivatenumpy2adBuild and install the package locally$condadevelop.Alternatively, you can also usevenvandpip:$python-mvenvenv...
$sourceenv/bin/activate
...
$pipinstall-rdev_requirements.txt
...
$pipinstall-e.AcknowledgementsThis package was developed by Nicholas Book in close collaboration with Prof. Uwe Naumann and Simon Märtens ofSTCEat RWTH Aachen University, Germany.LicenseMIT License, seeLICENSE.txt.