Package: ac207-autodiff
cs107-FinalProject - Group #32Members: Xuliang Guo, Kamran Ahmed, Van Anh Le, Hanwen CuiBroader Impacts and InclusivityBroader ImpactsVirtually all machine learning and AI algorithms can be attributed to solving optimization problems during the training process. While automatic differentiation does not direct broader impacts, its extensive use as an intermediate step in these algorithms forces us to consider the broader impact of our package. First of all, our package will be contributing to biases against African-American and other underrepresented minorities that current ML models used in the criminal justice system or hiring processes are already imposing. Second, any errors in our calculations could lead to misspecified models and erroneous predictions with significant impacts to downstream users. These impacts are especially grave in safety-critical settings such as healthcare, where a model that utilizes a faulty AD library could misdiagnose a patient or suggest sub-optimal treatments.InclusivityWhile our codebase is technically available and open for anyone to contribute through our GitHub repository, there are technical barriers that might prevent certain groups from participating in this process. Any contributors would need to have working knowledge of git version control and principles of software development. This precludes people from rural communities, communities of color, or poor urban communities, who are less likely to receive formal and rigorous training in computer science. Even at the college level, CS curricula are not homogenous and concepts such as git version control might not be taught at every school. Furthermore, users from other disciplines who rely on optimization and AD might be discouraged by the initial fixed cost of learning a complicated system such as git.Any developer who wants to contribute to our codebase can make a new branch and create a pull request. Pull requests will then be reviewed by one or many members of our team, depending on the extent of contribution. In order to make this process more inclusive, we could include a step-by-step guide on our repository that provides explicit direction on how to work with git and the expected best-practices that we hope they would follow.How to installWe recommend creating a virtual environment rather than installing in the base environment:python3 -m venv autodiff-env
source autodiff/bin/activateOur package can be installed from Github or PyPI. We also include source distribution files and wheels under Releases.You can install ac207-autodiff viapipwith:pip install ac207-autodiffBasic usageDetailed descriptions about classes, methods, and operations can be found in ourAPI reference.Our automatic differentiation package’s default behavior uses forward mode. You can import this as follows:importautodiffasadIf you would like to use reverse mode, please explicitly import it as:importautodiff.reverseasadForward modeThe properties of a dual number lend itself nicely to a straightforward implementation of forward mode automatic differentiation. Briefly, we use dual numbers as our core data structure (ad.Dual). The value and derivative can be stored as the real and “dual” part of the dual number, respectively.We provide support for:Most arithmetic and comparison operationsElementary operations such as trigonometric functions, square root, logarithmic, logistic, and exponential functions, among others.Univariate functions>>>importautodiffasad>>>x=ad.Dual(2)>>>f=7*(x**3)+3*x>>>print(f"Function value:{f.val}, derivative:{f.der}")Functionvalue:62,derivative:[87.]Multivariate functions>>>importautodiffasad>>>x,y=ad.Dual.from_array([2,4])# helper static method>>>f=7*(x**3)+3*y>>>print(f"Function value:{f.val}, derivative:{f.der}")Functionvalue:68,derivative:[84.3.]Vector functions>>>importautodiffasad>>>deff(x,y,z):# Vector function mapping 3 inputs to 2 outputs....f1=7*(x**3)+3*y...f2=y/x+z**2...return(f1,f2)...>>>x,y,z=ad.Dual.from_array([2,4,6])>>>f1,f2=f(x,y,z)>>>print(f"f1 value:{f1.val}, derivative:{f1.der}")f1value:68,derivative:[84.3.0.]>>>print(f"f2 value:{f2.val}, derivative:{f2.der}")f2value:38.0,derivative:[-1.0.512.]Elementary operations>>>importautodiffasad>>>x,y=ad.Dual.from_array([2,4])>>>f=ad.exp(x)+y>>>print(f"Function value:{f.val:.4f}, "\..."derivative: [{f.der[0]:.4f}{f.der[1]:.4f}]")Functionvalue:11.3891,derivative:[7.38911.0]Reverse modeNote that these are contained within theautodiff.reversemodule.Explicitly import it as:>>>importautodiff.reverseasadad.Nodeis the primary data structure for reverse mode automatic differentiation. The process of evaluating derivatives in reverse mode consists of two passes, forward pass and reverse pass. During the forward pass, we calculate the primal values and the local gradient of child nodes with respect of each parent node in the computational graph. In the reverse pass, we recursively calculate the gradients.Reverse mode only evalates the function at the specified values. To calculate the gradient with respect to each input, you have to explicitly callNode.grad(). Examples can be found below.Univariate functionThe derivative of the function is not stored within the function object, but rather is computed on the fly whenx.grad()is called.>>>importautodiff.reverseasad>>>x=ad.Node(2)>>>f=7*(x**3)+3*x>>>grad=x.grad()# compute gradient>>>print(f"Function value:{f.val}, derivative w.r.t x ={grad}")Functionvalue:62,derivativew.r.tx=87.0Note that to reuse thexvariable again, without accumulating gradients you must callad.Node.zero_grad(x). A more detailed example can be found below when using vector functions.Multivariate functions>>>importautodiff.reverseasad>>>x=ad.Node(2)>>>y=ad.Node(4)>>>f=7*(x**3)+3*y>>>grad=[x.grad(),y.grad()]# explicitly compute all gradients w.r.t. x and y>>>print(f"Function value:{f.val}, derivative:{grad}")Functionvalue:68,derivative:[84.0,3.0]Vector functions>>>importautodiff.reverseasad>>>x,y,z=ad.Node.from_array([2,4,6])>>>deff(x,y,z):# Vector function mapping 3 inputs to 2 outputs...f1=7*(x**3)+3*y...f1_grad=[x.grad(),y.grad(),z.grad()]# compute gradient w.r.t. all inputs, before computing f2...ad.Node.zero_grad(x,y,z)# must be called before computing f2, otherwise gradients will accumulate...f2=y/x+z**2...f2_grad=[x.grad(),y.grad(),z.grad()]...returnf1,f1_grad,f2,f2_grad>>>f1,f1_grad,f2,f2_grad=f(x,y,z)>>>print(f"First function value:{f1.val}, derivative:{f1_grad}")Firstfunctionvalue:68,derivative:[84.0,3.0,1.0]>>>print(f"Second function value:{f2.val}, derivative:{f2_grad}")Secondfunctionvalue:38.0,derivative:[-1.0,0.5,12.0]Elementary operationsWe allow users to import overloaded elementary functions (sine, cosine, tangent, exponential, log, sqrt) to perform operations on Nodes.>>>importautodiff.reverseasad>>>x,y=ad.Node.from_array([2,4])>>>f=ad.exp(x)+y>>>grad=[x.grad(),y.grad()]>>>print(f"Function value:{f.val:.4f}, derivative: [{grad[0]:.4f}{grad[1]:.4}]")Functionvalue:11.3891,derivative:[7.38911.0]