Basic Curve Fitting of Scientific Data with Python | by Naveen Venkatesan | Towards Data Science

In addition to plotting data points from our experiments, we must often fit them to a theoretical model to extract important parameters. This short article will serve as a guide on how to fit a set of points to a known model equation, which we will do using the scipy.optimize.curve_fit function. The basics of plotting data in Python for scientific publications can be found in my previous article here. I will go through three types of common non-linear fittings: (1) exponential, (2) power-law, and (3) a Gaussian peak.
To use the curve_fit function we use the following import statement:
# Import curve fitting package from scipyfrom scipy.optimize import curve_fit
In this case, we are only using one specific function from the scipy package, so we can directly import just curve_fit .
Let’s say we have a general exponential function of the following form, and we know this expression fits our data (where a and b are constants we will fit):
First, we must define the exponential function as shown above so curve_fit can use it to do the fitting.
# Function to calculate the exponential with constants a and bdef exponential(x, a, b):    return a*np.exp(b*x)
We will start by generating a “dummy” dataset to fit with this function. To generate a set of points for our x values that are evenly distributed over a specified interval, we can use the np.linspace function.
# Generate dummy datasetx_dummy = np.linspace(start=5, stop=15, num=50)
start — starting value of our sequence
stop — ending value of our sequence (will include this value unless you provide the extra argument endpoint=False )
num — the number of points to split the interval up into (default is 50 )
Note that you do not need to explicitly write out the input names — np.linspace(-5, 5, 100) is equally valid, but for the purposes of this article, it makes things easier to follow.
For our dummy data set, we will set both the values of a and b to 0.5.
# Calculate y-values based on dummy x-valuesy_dummy = exponential(x_dummy, 0.5, 0.5)
To make sure that our dataset is not perfect, we will introduce some noise into our data using np.random.normal , which draws a random number from a normal (Gaussian) distribution. We will then multiply this random value by a scalar factor (in this case 5) to increase the amount of noise:
# Add noise from a Gaussian distributionnoise = 5*np.random.normal(size=y_dummy.size)y_dummy = y_dummy + noise
size — the shape of the output array of random numbers (in this case the same as the size of y_dummy)
Now let’s plot our dummy dataset to inspect what it looks like. Since we have a collection of noisy data points, we will make a scatter plot, which we can easily do using the ax.scatter function. I will skip over a lot of the plot aesthetic modifications, which are discussed in detail in my previous article. To assign the color of the points, I am directly using the hexadecimal code.
# Plot the noisy exponential dataax.scatter(x_dummy, y_dummy, s=20, color='#00b3b3', label='Data')
s — the marker size in units of (points)2, so the marker size is doubled when this value is increased four-fold
An often more-useful method of visualizing exponential data is with a semi-logarithmic plot since it linearizes the data. To set the scale of the y-axis from linear to logarithmic, we add the following line:
# Set the y-axis scaling to logarithmicax.set_yscale('log')
We must also now set the lower y-axis limit to be greater than zero because of the asymptote in the logarithm function. Additionally, for the tick marks, we now will use the LogLocator function:
# Set the y-ticksax.yaxis.set_major_locator(mpl.ticker.LogLocator(base=10.0))# Set the y-axis limitsax.set_ylim(0.1, 1000)
base — the base to use for the major ticks of the logarithmic axis
We can now fit our data to the general exponential function to extract the a and b parameters, and superimpose the fit on the data. Note that although we have presented a semi-log plot above, we have not actually changed the y-data — we have only changed the scale of the y-axis. So, we are still fitting the non-linear data, which is typically better as linearizing the data before fitting can change the residuals and variances of the fit.
# Fit the dummy exponential datapars, cov = curve_fit(f=exponential, xdata=x_dummy, ydata=y_dummy, p0=[0, 0], bounds=(-np.inf, np.inf))
Inputs
f — function used for fitting (in this case exponential)
xdata — array of x-data for fitting
ydata — array of y-data for fitting
p0 — array of initial guesses for the fitting parameters (both a and b as 0)
bounds — bounds for the parameters (-∞ to ∞)
Outputs
pars — array of parameters from fit (in this case [a, b])
cov — the estimated covariance of pars which can be used to determine the standard deviations of the fitting parameters (square roots of the diagonals)
We can extract the parameters and their standard deviations from the curve_fit outputs, and calculate the residuals by subtracting the calculated value (from our fit) from the actual observed values (our dummy data)
# Get the standard deviations of the parameters (square roots of the # diagonal of the covariance)stdevs = np.sqrt(np.diag(cov))# Calculate the residualsres = y_dummy - exponential(x_dummy, *pars)
*pars — allows us to unroll the pars array, i.e. [a, b] gets inputted as a, b
Fit parameters and standard deviations
a = 0.509 ± 0.017
b = 0.499 ± 0.002
We see that both fit parameters are very close to our input values of a = 0.5 and b = 0.5 so the curve_fit function converged to the correct values. Now we can overlay the fit on top of the scatter data, and also plot the residuals, which should be randomly distributed and close to 0, confirming that we have a good fit.
# Plot the fit data as an overlay on the scatter dataax.plot(x_dummy, exponential(x_dummy, *pars), linestyle='--', linewidth=2, color='black')
linestyle — the line style of the plotted line ( -- for a dashed line)
Another commonly-used fitting function is a power law, of which a general formula can be:
Similar to how we did the previous fitting, we first define the function:
# Function to calculate the power-law with constants a and bdef power_law(x, a, b):    return a*np.power(x, b)
We then again can create a dummy dataset, add noise, and plot our power-law function.
# Generate dummy datasetx_dummy = np.linspace(start=1, stop=1000, num=100)y_dummy = power_law(x_dummy, 1, 0.5)# Add noise from a Gaussian distributionnoise = 1.5*np.random.normal(size=y_dummy.size)y_dummy = y_dummy + noise
Similar to the exponential fitting case, data in the form of a power-law function can be linearized by plotting on a logarithmic plot — this time, both the x and y-axes are scaled.
# Set the x and y-axis scaling to logarithmicax.set_xscale('log')ax.set_yscale('log')# Edit the major and minor tick locations of x and y axesax.xaxis.set_major_locator(mpl.ticker.LogLocator(base=10.0))ax.yaxis.set_major_locator(mpl.ticker.LogLocator(base=10.0))# Set the axis limitsax.set_xlim(10, 1000)ax.set_ylim(1, 100)
Now we can follow the same fitting steps as we did for the exponential data:
# Fit the dummy power-law datapars, cov = curve_fit(f=power_law, xdata=x_dummy, ydata=y_dummy, p0=[0, 0], bounds=(-np.inf, np.inf))# Get the standard deviations of the parameters (square roots of the # diagonal of the covariance)stdevs = np.sqrt(np.diag(cov))# Calculate the residualsres = y_dummy - power_law(x_dummy, *pars)
Fit parameters and standard deviations
a = 1.057 ± 0.096
b = 0.492 ± 0.014
Peak fitting with a Gaussian, Lorentzian, or combination of both functions is very commonly used in experiments such as X-ray diffraction and photoluminescence in order to determine line widths and other properties. In this example we will deal with the fitting of a Gaussian peak, with the general formula below:
# Function to calculate the Gaussian with constants a, b, and cdef gaussian(x, a, b, c):    return a*np.exp(-np.power(x - b, 2)/(2*np.power(c, 2)))# Generate dummy datasetx_dummy = np.linspace(start=-10, stop=10, num=100)y_dummy = gaussian(x_dummy, 8, -1, 3)# Add noise from a Gaussian distributionnoise = 0.5*np.random.normal(size=y_dummy.size)y_dummy = y_dummy + noise
Just like in the exponential and power-law fits, we will try to do the Gaussian fit with initial guesses of 0 for each parameter.
# Fit the dummy Gaussian datapars, cov = curve_fit(f=gaussian, xdata=x_dummy, ydata=y_dummy, p0=[0, 0, 0], bounds=(-np.inf, np.inf))# Get the standard deviations of the parameters (square roots of the # diagonal of the covariance)stdevs = np.sqrt(np.diag(cov))# Calculate the residualsres = y_dummy - power_law(x_dummy, *pars)
However, when we do this, we get the following result:
print(pars)>>> [0. 0. 0.]print(stdevs)>>> [inf inf inf]
It appears that our initial guesses did not allow the fit parameters to converge, so we can run the fit again with a more realistic initial guess. You can do this by examining the peak you are trying to fit, and choosing reasonable initial values.
# Fit the dummy Gaussian datapars, cov = curve_fit(f=gaussian, xdata=x_dummy, ydata=y_dummy, p0=[5, -1, 1], bounds=(-np.inf, np.inf))
This time, our fit succeeds, and we are left with the following fit parameters and residuals:
Fit parameters and standard deviations
a = 8.074 ± 0.128
b = –0.948 ± 0.054
c = 2.945 ± 0.054
Hopefully, following the lead of the previous examples, you should now be able to fit your experimental data to any non-linear function! I hope you enjoyed this tutorial and all the examples presented here can be found at this Github repository.