Building a Numerical Integration Tool in Python — From Scratch | by Kasper Müller | Towards Data Science

Not so long ago I was doing some research in analytic number theory. It was merely for my own sake at home in the evening. It was just a hobby project to satisfy my curiosity.
At some point, I was left with a nasty double integral and I wanted to check that what I had found was actually correct. To add some context for the interested reader, I was trying to find a formula for the number of twin primes below a given number using the prime counting function and some techniques from complex analysis... Phew!, now that that is out of the way, let’s move on with the story.
So what does one do when faced with such a problem involving a highly discontinuous function? Well, you fetch your laptop, a big cup of coffee and open up a code editor of some sort. In my case, my go-to programming language is Python, so I created an empty python file expecting this to take only 10 to 15 minutes.
I was wrong! The reason? Well, building the custom functions needed such as detecting if a number is a prime, counting them with a loop and topping with some complex discontinuous functions was a breeze. The problem was when I wanted to integrate them.
“Normally, Python’s scientific or data related libraries saves the day, but this time it failed me.”
Now, SciPy has a package (of course) for this sort of thing called quad. And that was why I initially thought that it was going to be easy. Normally, Python’s scientific or data related libraries saves the day and are usually very nice to use, but this time it failed me. The problem was that my integrand (of my double integral) had a pole in the integration interval.
Say you want to calculate the following integral:
I actually tried this example and it didn’t work. The function is perfectly finite in a neighborhood around zero but when I tried calculating it, I kept getting a value of nan. I immediately understood what was going on.
SciPy is using something like Riemann sums or the Trapezoidal rule to calculate the integral and that requires the algorithm to divide the integration interval, into a lot of small intervals. Calculating the areas of the corresponding small rectangles requires it to calculate the function value of all these interval-endpoints — one of them was (very close to) zero!
And of course, trying to divide by zero in Python throws a ZeroDivisonError. As it turns out, according to SciPy’s documentation at least, there should be some ways around it. One can, for example, specify known poles (which of course requires us to know the poles of the function which is not always the case), but again that didn’t work on my double integral (even though it actually does work for the simple example above).
The point of all of this introduction is the following:
A numeric integration tool should only care about convergence — not poles
You shouldn’t have to find the poles yourself or even think of them
You want something hackable that you can modify to fit your needs e.g. modify the error vs runtime parameter (we’ll get into this in a bit)
The errors should be meaningful and understandable
At this point, I was so frustrated, that I decided to build a tool myself.
After realizing that I had to do this, I needed to go fetch some more coffee and think about a design for this tool. You can of course do this in many different ways. I went with the most lightweight approach because I believe that such a tool needs to adapt constantly to fit the user’s needs.
Let’s build it!
I decided to go for an object-oriented design choosing to build a class around the function to be integrated. I did this so that the user would be able to easily add more features to it in the future and to save meta-information about the calculation such as error intervals etc.
Let’s start with the basics. First of all, this program only really depends on Numpy. However, for convenience, we will import quad:
If you don’t have these packages installed, you can install them with
pip install numpy quad
or pip3 depending on your OS.
For the code in this article to work, you need to run it with Python 3.
Let’s create the Integrate class
A few comments are in order here. First, I record the error and you will see why in a moment. Second, we have a sign-attribute defined. This is because when the bounds of the integral swap, then so does the sign of the value and we need to be able to calculate integrals when the lower limit is greater than the upper limit. This is of course just to show you how you could do this.
And now for the main (single) integration method:
I have used the trapezoidal rule here (if you don’t remember, this is basically an average of left and right Riemann sums), but that is no surprise. You might wonder why I have a big try/except clause lying around. That is actually one of the main points. What it does is that if there should be a division by zero hiding in there, then it will be gracefully ignored without input from the user.
We also see that we have stored the error if one wants to calculate by interval arithmetic or just check how accurate the result is.
That’s all well, but how do we use our tool? Say we wanna integrate the function from our example above.
This becomes easy now, despite the “0/0” expression when x = 0.
and now we get an answer of about 1.892166. Pretty good!
But we are not done, remember? We have a double integral to solve, and no working tool for it yet.
So we need to generalize this code to calculate a double integral, but first, let us make sure that we understand the possible errors that we might get. To do that, we need to build a custom error class in Python. This is easy, we just inherit from Exception:
Moreover, since the input intervals should now be a parameter of type list of lists (or something like that), a possible solution is the following:
Note! This is by no means performance code, and if we wanted, we could optimize this with e.g. vectorization in Numpy.
That being said, you can adjust the precision parameter to fit your needs. The higher the value, the more accurate the integral is going to be, but the longer it will take to compute. That’s where the error attribute comes in. You can try as an exercise to optimize this code!
If we wanted, we could of course generalize this to triple integrals, etc. but for the scope of this article, let’s stick to single and double ones.
So let me show you how this works in action.
Say you are working on a project in Python and all of a sudden, you come across the problem of approximating the following integral:
Well, you now have the tool for the job. At the top of your file, import your new integral engine
Further down in the project you can now solve it.
This gives an output of
The result is 3.1415926535897944The accuracy of this result is -1.7319479184152442e-14
This is pretty d... close! To compare against the real deal, the true beginning of pi is 3.141592653589793238... and our error range of course told us exactly that— namely, that we got the first 14 digits right (At least within our limits of integration — we need to be a little careful in our example because technically we are missing the “tails” of the function). Of course, increasing the precision parameter, or the limits of the integral for that matter, would give us even more digits right.
But here’s the important take away:
Even though the double integral above should be calculated over the entire real plane, we can make sure that within our interval we can get arbitrarily close to the true value — and we can check that because of our error attribute. So it is a matter of exchanging accuracy for runtime.
Now I can get back to my work and hopefully, if you encounter some issues in this direction yourself or if you just want to learn more Python, please do not hesitate to use or extend this code.
You can clone the full repository here.
After 7 cups of coffee, I can now go to sleep with ease of mind. I only hope that I am not gonna be chased by mad Riemann sums in my dreams.
Oh.. what about integrating with polar coordinates?...
Maybe next time.