Calculus — The Mathematics of ‘Change’ | by Gaurav Goel | Towards Data Science

Data Science and Machine Learning have fueled up interest in mathematics. A lot of people who are ramping up their skills in ML/AI domain have realized the practical applications of mathematical concepts, for the first time in their lives. During my venture into AI/ML space, I realized how difficult, mathematical ideas (such as calculus and vector algebra) were made in school and college than they really were! I am assuming a lot of people share this feeling. This article is an attempt to explain calculus and its applications, in a fundamental way without using the infamous jargons and big dreaded calculus equations. I anticipate that this reading will unfold the beauty, simplicity, and magic of calculus and mathematics, in general. The idea is to make you will fall in love with it!
Human beings have one great quality — “Curiosity”. Curiosity drives humankind to observe and understand nature. Science can be, broadly called, the study of nature while mathematics is the language to express it.
The understanding or learning is often expressed as a mathematical function that captures the relationship between the entities or dimensions involved. e.g let's say you are driving a car. You are measuring the speed of the car as time is passing by. Some sample measurements can be as follows:
As seen, there is a relationship between speed and time. As time passes by, the speed of the car increases at a constant rate. Every minute, the speed of the car increases by 5 times. This relationship can be expressed as follows:
Speed is a Function of time
Speed = f(time)
S = 5* t
Visually, you can see that there is a linear relationship between speed and time
You can imagine that this relationship is actually representing the equation of a line:
y = mx + b
where m is the slope and b is the intercept of the line.
In our example, the equation becomes s = 5t + 0
Focus on this thing known as “Slope”. What does it represent? We know that as time goes by, the speed of the car is changing. By what factor is this changing? We can find out by checking how much speed has changed for a change in time. e.g between t = 1 and t =2, the speed has changed from 5 to 10. This can be written as:
When things change, a more useful measurement to make is “Rate of Change”. The rate of change gives us an idea of how much one quantity is dependent on the other. If the rate is more, there will be a large correlation between the quantities and vice-versa. (Now think of what you try to do via Machine Learning. Do you see a connection?)
In our case, if we divide the “change in speed” with “change in time”, it gives us the rate of change
This is the rate of change of speed with respect to change in time. For our example, this rate is constant. You can take other time intervals (e.g t=2 and t=5) and measure the change in speed (25–10). The rate is the same — 5
(25-10)/(5–2)
The rate of change is also known as slope or gradient. Notice that greater the rate of change, greater will be the inclination of the line (hence, greater the slope). Compare the slope of the orange and blue lines in the below graph:
What has this to do with calculus? In fact, this is calculus.
Calculus is the branch of mathematics that deals with study of change
Calculus helps in finding out the relationship between two variables (quantities) by measuring how one variable changes when there is a change in another variable and how these changes accumulate over time.
In our example above, the speed of the car is increasing at a constant rate. What will happen if this rate of change is not constant? What if the car is accelerating and the rate of change in speed is different every minute
How to measure the rate of change, in this case, because its different for different time periods? We can not define a single rate of change in speed for this kind of data.
You can notice that the rate of change of speed climbs up as time passes. The mathematical function that captures this relationship is not a straight line but its a “curved line”.
From our earlier example, we know that the slope or gradient of a straight line gives us the rate of change of the Y-axis variable (Speed) with respect to the X-axis variable(Time). The property of “straightness” means that the rate of change was constant.
But...
The property of “curviness” means that rate of change is not constant.
Here comes the beauty of calculus.......
What you can do is to imagine the curve as a collection of lots of “very small straight line segments”
We will now be calculating the slope of a “very small” line segment of the curve. This slope represents a “very small” change in speed with respect to a “very small” change in time.
Let's denote “very small” change in speed as “ds”.
Similarly a “very small” change in time as “dt”
The slope of this “very small” line segment is
It can be calculated as follows:
Note that since “dt” is already very small, so the below will become extremely small and hence can be neglected
Theoretically, we can make ds and dt infinitesimally small, so that this “very small” line segment” becomes a point. The rate of change or slope at this point or instant of time is actually a tangent to the curve.
So, the function and rate of change of this function at a particular instant, in our example, is
In calculus terminology, the process of finding out the rate of change of a variable with respect to another is known as “differentiation”. In other words, if
y = f(x)
then the process of finding out dy/dx is known as “differentiation” or “differentiating”. The ratio dy/dx is called “the differential coefficient of y with respect to x” or “derivative”. Remember, its nothing but the rate of change of y with respect to x.
Keep in mind — “dx” means a very small part of x
What if ‘y’ is a constant. What will be dy/dx in this case? It will be 0 as the value of y is not changing. It's a constant.
The most robust application of Calculus in Machine Learning is the Gradient Descent algorithm in Linear regression (and Neural Networks)
Linear regression involves using data to calculate a line that best fits that data, and then using that line to predict scores on one variable from another. Prediction is simply the process of estimating scores of the outcome (or dependent) variable based on the scores of the predictor (or independent) variable. To generate the regression line, we look for a line of best fit. A line that can explain the relationship between the independent and dependent variable(s), better is said to be the best-fit line. The difference between the observed value and the actual value gives the error. The formula to calculate this error is also called the cost function.
The line of best fit will be expressed mathematically as
Y = m.x + c
where m and c are the slope and intercept of the line respectively. These are the two coefficients that the gradient descent algorithm has to find out.
The error will depend upon the coefficients of the line. If the value of coefficients is not optimal, then the error will be more. The cost function or the amount of error of Linear regression model is dependent upon the value of coefficient chosen.
This is where the calculus is used. We can find the rate of change of error with respect to the different values of coefficients. The value for which the rate of change is minimum( i.e 0, the bottom of the curve) is the optimal value.
So differentiation is used to find the minimum point of the curve. This point gives the optimal value of the coefficient that was being looked for!
There is a second part to Calculus — Integration. It is simply, the reverse of Differentiation.
In differentiation, we break up things into smaller and smaller parts.
In integration, we accumulate or add up, all the smaller parts together. The symbol for integration is:
Whenever you see this symbol, just replace it in your mind by “Add Up all”.
What will be the sum of all small parts of x?
(There is an amazing book “Calculus Made Easy” by Silvanus P. Thompson which explains these concepts beautifully in detail)
Why are we doing it? Why integrate? The simple answer this —
There are many things that can not be understood until and unless you break them up into smaller parts, do some operation for each smaller part and then accumulate or add up the results.
Let us take a rectangle:
We know that area of a rectangle is Length * Width (y *x)
Can you prove it? Think in terms of calculus. Imagine a smaller rectangle by taking a small bit of width(dx).
The whole of rectangle can be thought of sum of all smaller rectangles where width is dx
So, now we have ‘x’ number of small rectangles, each with width dx and length as y.
If you imagine dx to be very very small, the mini rectangle will eventually be reduced to a line (width close to 0 and length as y). The whole rectangle is just a collection of ‘lines’ each of length ‘y’. How many lines? — x.
Total area = Length of Line1 + Length of Line 2 + .......+ Length of Line X
Area = Y * X
Learning (Human or Machine) is all about finding relationships between variables.If the change in the value of a variable is triggered by the change in the value of another variable, its often very useful to find out the rate of change. This rate actually is the measurement of the relationship between the two variables. In mathematical terms, its also known as slope or gradient. Calculus is the branch of mathematics to study “change”Differentiation is the process of finding out the rate of change of one variable with respect to the change in another variable. The ratio dy/dx represents this rate of change. Geometrically, it represents the slope of a tangent to function.In Machine Learning, the regression algorithms use the process of differentiation to measure the rate of change of error in the model with a change in the value of the coefficient chosen. The value of coefficient where this rate of change is 0, gives the optimal value.
Learning (Human or Machine) is all about finding relationships between variables.
If the change in the value of a variable is triggered by the change in the value of another variable, its often very useful to find out the rate of change. This rate actually is the measurement of the relationship between the two variables. In mathematical terms, its also known as slope or gradient. Calculus is the branch of mathematics to study “change”
Differentiation is the process of finding out the rate of change of one variable with respect to the change in another variable. The ratio dy/dx represents this rate of change. Geometrically, it represents the slope of a tangent to function.
In Machine Learning, the regression algorithms use the process of differentiation to measure the rate of change of error in the model with a change in the value of the coefficient chosen. The value of coefficient where this rate of change is 0, gives the optimal value.
Once you start realizing the intuition behind mathematical concepts, the whole way of thinking can change. Differentiation and Integration allow us to view the world as a set of small parts and the sum of these small parts.
You may also like below articles:
Why Data is represented as a Vector in Data Science?Discovering Mathematical MindsetHuman Learning vs Machine Learning
Why Data is represented as a Vector in Data Science?
Discovering Mathematical Mindset
Human Learning vs Machine Learning