Package: fdm
FDM: Finite Difference MethodsFDM estimates derivatives with finite differences.
See alsoFiniteDifferences.jl.InstallationMultivariate DerivativesGradientsJacobiansJacobian-Vector Products (Directional Derivatives)Hessian-Vector ProductsScalar DerivativesTesting Sensitivities in a Reverse-Mode Automatic Differentation FrameworkInstallationFDM requires Python 3.6 or higher.pipinstallfdmMultivariate Derivativesfromfdmimportgradient,jacobian,jvp,hvpFor the purpose of illustration, let us consider a quadratic function:>>>a=np.random.randn(3,3);a=a@a.T>>>aarray([[3.57224794,0.22646662,-1.80432262],[0.22646662,4.72596213,3.46435663],[-1.80432262,3.46435663,3.70938152]])>>>deff(x):...return0.5*x@a@xConsider the following input value:>>>x=np.array([1.0,2.0,3.0])Gradients>>>grad=gradient(f)>>>grad(x)array([-1.38778668,20.07146076,16.25253519])>>>a@xarray([-1.38778668,20.07146076,16.25253519])Jacobians>>>jac=jacobian(f)>>>jac(x)array([[-1.38778668,20.07146076,16.25253519]])>>>a@xarray([-1.38778668,20.07146076,16.25253519])Butjacobianalso works for multi-valued functions.>>>deff2(x):...returna@x>>>jac2=jacobian(f2)>>>jac2(x)array([[3.57224794,0.22646662,-1.80432262],[0.22646662,4.72596213,3.46435663],[-1.80432262,3.46435663,3.70938152]])>>>aarray([[3.57224794,0.22646662,-1.80432262],[0.22646662,4.72596213,3.46435663],[-1.80432262,3.46435663,3.70938152]])Jacobian-Vector Products (Directional Derivatives)In the scalar case,jvpcomputes directional derivatives:>>>v=np.array([0.5,0.6,0.7])# A direction>>>dir_deriv=jvp(f,v)>>>dir_deriv(x)22.725757753354657>>>np.sum(grad(x)*v)22.72575775335481In the multivariate case,jvpgeneralises to Jacobian-vector products:>>>prod=jvp(f2,v)>>>prod(x)array([0.65897811,5.37386023,3.77301973])>>>a@varray([0.65897811,5.37386023,3.77301973])Hessian-Vector Products>>>prod=hvp(f,v)>>>prod(x)array([[0.6589781,5.37386023,3.77301973]])>>>0.5*(a+a.T)@varray([0.65897811,5.37386023,3.77301973])Scalar Derivatives>>>fromfdmimportcentral_fdmLet's try to estimate the first derivative ofnp.sinat1with a
second-order method.>>>central_fdm(order=2,deriv=1)(np.sin,1)-np.cos(1)-1.2914319613699377e-09And let's try to estimate the second derivative ofnp.sinat1with a
third-order method.>>>central_fdm(order=3,deriv=2)(np.sin,1)+np.sin(1)1.6342919018086377e-08Hm.
Let's check the accuracy of this third-order method.
The step size and accuracy of the method are computed upon callingFDM.estimate.>>>central_fdm(order=3,deriv=2).estimate(np.sin,1).acc5.476137293912896e-06We might want a little more accuracy. Let's check the accuracy of a
fifth-order method.>>>central_fdm(order=5,deriv=2).estimate(np.sin,1).acc7.343652562575157e-10And let's estimate the second derivative ofnp.sinat1with a
fifth-order method.>>>central_fdm(order=5,deriv=2)(np.sin,1)+np.sin(1)-1.7121615236703747e-10Hooray!Finally, let us verify that increasing the order generally increases the accuracy.>>>foriinrange(3,10):...print(central_fdm(order=i,deriv=2)(np.sin,1)+np.sin(1))1.6342919018086377e-088.604865264771888e-09-1.7121615236703747e-108.558931341440257e-12-2.147615418834903e-126.80566714095221e-13-1.2434497875801753e-14Testing Sensitivities in a Reverse-Mode Automatic Differentation FrameworkConsider the functiondefmul(a,b):returna*band its sensitivitydefs_mul(s_y,y,a,b):returns_y*b,a*s_yThe sensitivitys_multakes in the sensitivitys_yof the outputy,
the outputy, and  the arguments of the functionmul; and returns a tuple
containing the sensitivities with respect toaandb.
Then functioncheck_sensitivitycan be used to assert that the
implementation ofs_mulis correct:>>>fromfdmimportcheck_sensitivity>>>check_sensitivity(mul,s_mul,(2,3))# Test at arguments `2` and `3`.Suppose that the implementation were wrong, for exampledefs_mul_wrong(s_y,y,a,b):returns_y*b,b*s_y# Used `b` instead of `a` for the second sensitivity!Thencheck_sensitivityshould throw anAssertionError:>>>check_sensitivity(mul,s_mul,(2,3))AssertionError:Sensitivityofargument2offunction"mul"didnotmatchnumericalestimate.