Package: covvvr
CoVVVR: Control Variates & Vegas Variance ReductionThis package is a wrapper over thevegasintegration package. The control variate variance reduction method is applied to the function when integrated along with the techniques applied invegassuch as importance sampling. To understand control variates, lets first look at how Monte carlo works.Monte Carlo BasicsFor a continuous random variable $f(x)$, taking its expectation value where $x$ is drawn from some PDF $p(x)$ is
$$E_p[f]=\int f(x)p(x)\text{d}x$$
where the bounds on the integral are the limits of the PDF. Now, if we draw from the uniform distribution $U[a,b]$, then the PDF is $1/(b-a)$. So
$$E_{U[a,b]}[f]=\frac{1}{b-a}\int_a^bf(x)\text{d}x\qquad\text{or}\qquad(b-a)E[f]=\int_a^bf(x)\text{d}x$$
where the subscript on the expectation value is now assumed. Note that the left-hand side is just a normal integral. Namely, if we are looking to integrate some function $f(x)$ from $b$ to $a$, then that is equivalent to finding $(b-a)E[f]$. For $N$ discrete uniform random variables, the expectation value is just the mean:
$$E[X]=\frac{1}{N}\sum_{i=1}^NX_i$$
and this extends as an approximation for continuous random variables:
$$I=\int_a^bf(x)\text{d}x\approx\frac{b-a}{N}\sum_{i=1}^Nf(x_i)$$
where $x_i\sim U[a,b]$. Alternatively, if we consider the integral as sequentially summing up $N$ rectangles of equal length $(b-a)/N$ from $a$ to $b$, i.e. a Riemann sum, then we see that the $N$ rectangles gives an overall factor of $b-a$. The only difference between the two is one uses random variables and the breaks up the interval evenly and uses each point.Control Variate BasicsEquivalent to $I$ above is integrating $f_c(x)=f(x)+c\left(g(x)-E[g]\right)$ since $\int\left(g(x)-E[g]\right)\text{d}x=0$. So,
$$I=\int_a^bf_c(x)\text{d}x=\int_a^bf(x)-c\Big(g(x)-E[g]\Big)\text{d}x\approx\frac{b-a}{N}\sum_{i=1}^Nf(x_i)-c\Big(g(x_i)-E[g]\Big)$$
where the expectation value of $g(x)$, $E[g]$, is known. Note that $c$ is a free parameter to be chosen. So we choose it to minimize the variance of $f_c(x)$ first by finding $\text{Var}(f_c)$, taking the derivative with respect to $c$ and then setting it equal to zero. We find the optimal value
$$c^*=-\frac{\text{Cov}(f,g)}{\text{Var}(g)}\qquad\text{and}\qquad f_c^*(x)=f(x)+c^*\left(g(x)-E[g]\right)$$
where the $*$ represents the optimal value. Plugging in $c^*$ gives us a final variance of
$$\text{Var}(f_c^*)=\text{Var}(f)-\frac{\text{Cov}^2(f,g)}{\text{Var}(g)}=\left(1-\rho^2(f,g)\right)\text{Var}(f)\qquad\text{where}\qquad\rho(f,g)=\frac{\text{Cov}(f,g)}{\sqrt{\text{Var}(f)\text{Var}(g)}}$$
is the correlation coefficient between $f$ and $g$. Since $|\rho(f,g)|\le1$, then this choice of $c$ will always decrease the variance. The same prescription can be applied for $n$ control variates, rather than just one, which is used in this package.A control variate is applied by usingvegas's importance sampling adaptation. Since $\rho$ is larger when the functions have a linear relationship, we use these previously adapted maps as the control variates. One can specify using the $i$th iteration ofvegasas the control variate when initializing theCVIntegratorclass or specify a list of iterations to use for multiple control variates.InstallationIt can just be installed viapip:python -m pip install control-vegasUsageThe workflow involes creating aFunctionclass and then passing that to theCVIntegrator. TheFunctionclass contains the function to be integrated but also other information such as its name, the true value of the integration (if available) and parameters of the function. TheCVIntegratorclass does the integration and stores the results like mean and variance.Using a Built-In FunctionFor example,fromcovvvrimportCVIntegratorfromcovvvr.functionsimportNGauss# Create 16-dimensional Gaussianng=NGauss(16)# Print out all parameters of the class instanceprint(ng,'\n')# Create integrator class and use the 20th iteration as the control  variatecvi=CVIntegrator(ng,evals=1000,tot_iters=50,cv_iters=20)# Run the integrationcvi.integrate()# Print infocvi.compare(rounding=5)This integrates a 16-dimensional Gaussian with 50 iterations and 5000 evaluations per iteration invegas. And it uses the 20th iteration adapation fromvegasas the control variate. The output isNGauss(dimension=16, name=16D Gaussian, true_value=0.9935086032227194, mu=0.5, sigma=0.2) 

         |   No CVs    |  With CVs   
---------+-------------+-------------
Mean     |     0.99528 |     0.99579
Variance | 4.17808e-06 | 3.54189e-06
St Dev   |     0.00204 |     0.00188
VPR      |             |   15.22696%Adding Your Own FunctionThemake_funcfunction allows you to make your ownFunctionsubclass. As an example, lets say $f(x_1,x_2)=ax_1^2+bx_2^2$ is a 2-dimensional function we want to integrate. Then we can do that byfromcovvvrimportCVIntegrator,make_func# Create function, note that it is vetorized using Numpy slicingdeff(self,x):returnself.a*x[:,0]**2+self.b*x[:,1]# Creating class with name 'WeightedPoly' and assigning values to the parameters in the functionwpoly=make_func('WeightedPoly',dimension=2,function=f,name='Weighted Polynomial',a=0.3,b=0.6)# Print out parameters of class (note `true_value` isn't shown)print(wpoly,'\n')# Create integrator class and use multiple control variatescvi=CVIntegrator(wpoly,evals=1000,tot_iters=50,cv_iters=[10,15,20,25,30,35])# Run the integrationcvi.integrate()# Print infocvi.compare(rounding=5)which outputsWeightedPoly(dimension=2, name=Weighted Polynomial, a=0.3, b=0.6) 

         |   No CVs    |  With CVs   
---------+-------------+-------------
Mean     |     0.39984 |     0.40004
Variance | 5.83781e-08 | 4.90241e-08
St Dev   | 2.41616e-04 | 2.21414e-04
VPR      |             |   16.02309%The reason the function is vectorized is because, on the backend,vegas'sbatchintegrandis used which can greatly speed up the computation.Be lazy!If you're rushed or lazy, you can use theclassic_integratefunction that does the steps above for you and returns theCVIntegratorobject. So to run the previous code block, you would usecvi=classic_integrate(function=f,evals=1000,tot_iters=50,bounds=[(0,1),(0,1)],cv_iters=20,cname="WeightPoly",name="Weighted Polynomial",a=0.3,b=0.6)cvi.compare()which outputs|  No CVs   | With CVs  
---------+-----------+-----------
Mean     |     0.400 |     0.400
Variance | 5.826e-08 | 5.025e-08
St Dev   | 2.414e-04 | 2.242e-04
VPR      |           |   13.746%Note that the arguments arecnameandnameare optional butboundsis not. The dimension of the integral is implied fromboundswhen usingquick_integratewhereas it's taken from theFunctionclass for the previous code blocks.Specifying Control Variate IterationsThere are multiple valid arguments that can be passed tocv_itersfor both theCVIntegratorclass andquick_integratewhich we'll lay out here.For using one control variate, pass a integer representing the iteration to use.For multiple control variates, pass a list of integers representing the iterations to use.The string 'all' will use every iteration.The stringall%nwill use every iteration mod $n$. So if you specifytot_iters=15andcv_iters='all%3', then then the iterations used will be[3, 6, 9, 12]This can be shifted by instead usingall%n+bwhere $b$ is the shift. So fortot_iters=15andcv_iters='all%3+2', you'll get[2, 5, 8, 11, 14].The string 'auto1' will estimate the best single CV to use by sampling each possibility usingauto1_neval1samples specified by the user.Manual Use ofFunctionClassTo access the function call, usefunctionorf. So, using the second example, I can runwpoly.f([1.2,1.5])# returns 1.3319999999999999wpoly.f([0.2,1],[0.8,0.8],[1,2])# returns array([0.612, 0.672, 1.5  ])This wraps around the private, vectorized_function/_fused byvegasso you don't have to worry about the propernumpyarray shapeNotesBy default, functions are integrated from 0 to 1 unlessquick_integrateis used.