A Classical Least Squares Method for Quantitative Spectral Analysis with Python | by Nicolas Coca, PhD | Towards Data Science

Abstract. In this short tutorial, the Classical Least Squares method to calculate the similarity between spectra will be explained. This method, also known as Beer’s method, can be used to estimate the concentration of different components in an unknown spectrum. First, the physical context and Lambert-Beer’s law will be introduced, followed by the Classical Least Squares theoretical approach and derivation of the Normal equations. To wrap up, a Python implementation will be used to present an application example on how to calculate the concentration of different components on a sample spectrum.
Spectroscopy is the study of emitted, reflected or scattered light by a solid, liquid or gas, as a function of wavelength [1]. Most spectroscopy methods are therefore classified as absorption, emission or scattering spectroscopy. Some examples are Absorption, X-Ray, Infrared and Raman spectroscopy among others.
Sometimes, the substance under spectroscopic study is unknown. If knowing the composition of such substance is required, its measured spectrum (query spectrum) can be compared against a spectral library of known substances (components) in order to be identified. To do so, the unknown spectrum is searched using a search algorithm which computes a score based on a similarity metric, providing a list of nearest matches or concentrations. A similarity metric is a function that quantifies the similarity between two objects. In some sense, such measures are the inverse of distance metrics: they return a large value for similar objects and either zero or negative value for dissimilar objects. When normalized, similarities range between 1 (complete similarity, meaning that the objects are equal) and 0 (no similarity at all). Similarity metrics are also the very basic building block for other activities such as Recommendation systems, clustering, classification or anomaly detection.Regarding spectroscopy, there are several standard spectral search algorithms [2] based on different similarity metrics [3]. In the following, the Classical Least Squares method will be used to identify the spectrum of an unknown substance and to obtain the concentrations of known substances in a mixture. This approach is also known as Direct Least Squares, K-matrix calibration algorithm, or Beer’s method, since it is based on Beer-Lambert’s law.
The Beer-Lambert law relates the attenuation of light to the properties of the material through which the light is traveling. It is commonly applied to physical and chemical analysis measurements and used in understanding light attenuation in physical optics. The Beer-Lambert law says that at any given wavelength i, the light absorbance (A) is proportional to the absorbance coefficient of the pure substance (k) at the chosen wavelength i and the concentration of the pure substance (c):
where the absorbance coefficient (ki = aiL) is the product of the the path length of the light through the material (L) and the molar attenuation coefficient or absorptivity (a) of the pure substance at the chosen wavelength i, which is a molecular property constituting the ‘spectrum’ of such molecule.
The equation Ai = kic applies to a single pure substance or component in a sample. When there are multiple absorbing components, the total absorbance at any wavelength is the sum of the absorbances, at that wavelength, of all the components in the mixture:
If what we have recorded is not a single value A for a given wavelength i, but a whole absorbance spectrum with w points, this is better represented with matrix notation:
or in a more compact manner as
where
so A is our recorded absorbance spectrum (also called query spectrum, or unknown spectrum, since we do not know yet what does it contain), K is the matrix of the pure component spectra (where the spectra values of such components are arranged in columns), and c is a vector with the concentration of the n pure substances.
Therefore, having the values of our measured unknown spectra, Ai, and knowing the ones of the pure components spectra making up the mixture, Kij, we can find the concentrations cj that best determine the mixture absorbance A. However, because the measured spectrum and the pure substances spectra contain noise, there will be, in general, no exact solution for the equation A = Kc. But we can still find c such that the product Kc is as close as possible to A, as measured by the euclidean norm:
In other words, we want to find c such that the sum of the squares of the errors A-Kc is minimized. The errors are defined as the difference between the measured spectra A and the spectra calculated by multiplying K and c. c will be then the best least squares solution for equation A = Kc.
Using the fact that
The Euclidean norm of our error is
In order to find the minimum of such function, we just need to derive it, set the resulting expression equal to zero and solve for c. To do so, we will make use of the property
where x is a vector and M is a symmetric matrix [4].
So,
Setting this expression to 0 we obtain the Normal Equations:
which can be solved to finally obtain the similarity scores between the measured spectrum and the known spectra in the library, or what its the same, the concentrations of pure substances in the unknown mixture:
Note that the second derivative of the squared sum of A-Kc is a positive semidefinite matrix, implying that the solution c is the global minimum (and not a maximum or a saddle point).
More detailed information on this method can be found in the book Chemometric techniques for quantitative analysis [5] or the spectroscopyonline webpage column about CLS [6].
First, the Python packages that will be needed are loaded:
import numpy as npimport matplotlib.pyplot as pltfrom numpy.linalg import inv
The least squares formula to obtain the concentration of pure substances in the unknown mixture is defined as:
def least_sq(sample_spectrum, components):   # sample_spectrum (unknown spectrum): array of w values.   # components (known spectra): array of n (number of components) columns with w values.   # This def returns an array of n values. Each value is the similarity score for the sample_spectrum and a component spectrum.   similarity = np.dot(inv(np.dot(components, components.T)) , np.dot(components, sample_spectrum))    return similarity
Next, some spectra to play with are fabricated. These spectra will be composed of Gaussian peaks. Therefore Gaussian function is defined:
def Gauss(x, mu, sigma, intensity = 1):# x is an array# mu is the expected value# sigma is the square root of the variance# intensity is a multiplication factor# This def returns the Gaussian function of x    return intensity/(sigma * np.sqrt(2*np.pi)) * np.exp(-0.5*((x-mu)/sigma)**2
and use it to create three different components:
# X-axis (Wavelengths)x_range =  np.linspace(100, 200, 1000)# Let's create three different components# Component Amu_a1 = 120sigma_a1 = 2intensity_a1 = 1mu_a2 = 185sigma_a2 = 2intensity_a2 = 0.2gauss_a =  Gauss(x_range, mu_a1, sigma_a1, intensity_a1) + Gauss(x_range, mu_a2, sigma_a2, intensity_a2)# Component Bmu_b = 150sigma_b = 15intensity_b = 1gauss_b = Gauss(x_range, mu_b, sigma_b, intensity_b)# Component Cmu_c1 = 110sigma_c1 = 2intensity_c1 = 0.05mu_c2 = 160sigma_c2 = 10intensity_c2 = 1gauss_c = Gauss(x_range, mu_c1, sigma_c1, intensity_c1) + Gauss(x_range, mu_c2, sigma_c2, intensity_c2)# Spectra normalization:component_a = gauss_a/np.max(gauss_a)component_b = gauss_b/np.max(gauss_b)component_c = gauss_c/np.max(gauss_c)# How do they look?plt.plot(x_range, component_a, label = 'Component 1')plt.plot(x_range, component_b, label = 'Component 2')plt.plot(x_range, component_c, label = 'Component 3')plt.title('Known components in our mixture', fontsize = 15)plt.xlabel('Wavelength', fontsize = 15)plt.ylabel('Normalized intensity', fontsize = 15)plt.legend()plt.show()
This result in the next three different components:
These three spectra will make our component matrix or K-matrix, where the spectra are arranged in rows.
In order to generate a query spectrum, the three components will be multiplied by a concentration and summed up together. To make it a bit more real, some noise will be added.
# What concentrations we want them to have in our mixture:c_a = 0.5c_b = 0.3c_c = 0.2# Let's build the spectrum to be studied: The mixture spectrumquery_spectra = c_a * component_a + c_b * component_b + c_c *component_c# Let's add it some noise for a bit of realism:query_spectra = query_spectra +  np.random.normal(0, 0.02, len(x_range))plt.plot(x_range, query_spectra, color = 'black', label = 'Mixture spectrum with noise')plt.title('Mixture spectrum', fontsize = 15)plt.xlabel('Wavelength', fontsize = 15)plt.ylabel('Intensity',  fontsize = 15)plt.show()
The resulting mixture spectrum looks like:
Our query spectrum is composed of three known components (n=3). Therefore, for this example the system of equations in matrix form looks like:
where w = 1000 for this case (the wavelength axis has 1000 values). So A is a vector composed of 1000 values, c is a vector composed of 3 values (the three component concentrations) and K is a 3x1000 matrix.
Knowing A and K it is possible now to recalculate the 3 components concentration in the ‘unknown’ mixture:
# Generate the components matrix or K matrixcomponents = np.array([component_a, component_b, component_c])# Apply Least squarescs = least_sq(query_spectra, components)# And plot the result:plt.plot(x_range, query_spectra, color = 'black', label = 'Mix spectrum' ) # Plot the unknown spectrumplt.plot(x_range, np.dot(cs,components), color = 'red', linewidth = 2, label = 'Calculation') # Plot the calculated spectrumfor i in np.arange(len(cs)):    plt.plot(x_range, cs[i]*components[i], label = 'c' + str(i)+ ' =               ' + str(np.round(cs[i], 3)))plt.title('Mixture spectrum and calculated components', fontsize = 15)plt.xlabel('Wavelength', fontsize = 15)plt.ylabel('Intensity', fontsize = 15)plt.legend()plt.show()
The calculated concentrations, shown in the legend of the figure, perfectly agree with the set values. Note that the mixture spectra were not normalized. In case that the mixture spectra was normalized, the inverse of such normalization constant should be applied to the obtained concentrations in order to have the right values.
The next figure show three more examples, where the calculated concentration agree with the previously set values (see the figure’s caption).
Least squares with Python libraries: SciPy and Skit-learn
Alternatively, the Least squares problem can also be solved using some Python’s libraries.
As for example with SciPy:
import scipy.optimize as optimizationx0 = np.array([1,1,1])def func(params, xdata, ydata):    return (ydata - np.dot(xdata.T, params))**2optimization.leastsq(func, x0, args=(components, query_spectra))[0]
or with Scikit-learn:
import sklearn.linear_model as linear_modellinear_model.LinearRegression().fit(components.T, query_spectra).coef_
resulting both of them in the concentrations 0.48, 0.30, 0.20 for components A, B and C, respectively.
In this short tutorial, a Classical Least Squares approach was used to calculate the similarity between spectra and show how this is useful to estimate the concentration of different components in a absorbance spectrum (Beer’s method). Three different ways to implement this in Python were presented (based on the Normal equations and using the packages SciPy and Scikit-learn).Importantly, this approach can be applied to not only Absorption spectra, but to any kind of spectral data, as for example Raman spectra, Infrared spectra, etc. However, because the presence of certain artifacts and side effects, several spectral pre-processing steps are required to clean the spectra before applying any similarity metric [2,7]. These steps include cropping and interpolation of the spectra, spikes removal (see my previous post), baseline subtraction, smoothing, and spectra normalization, which are specially important when dealing with weak signal (e.g. Raman spectra). But this will be the topic of a different post.
The original Jupyter Notebook can be found on GitHub.
Acknowledgements. I would like to thank Jorge Luis Hita, Ph.D., Avinash Mishra and Ana Solaguren-Beascoa, PhD for taking the time to proofread this notes.
[1] Spectroscopy. Wikipedia. https://en.wikipedia.org/wiki/Spectroscopy
[2] Khan, S. S., & Madden, M. G. (2012). New similarity metrics for Raman spectroscopy. Chemometrics and Intelligent Laboratory Systems, 114, 99–108.
[3] Implementing the five most popular similarity measures in Python. https://dataaspirant.com/2015/04/11/five-most-popular-similarity-measures-implementation-in-python/
[4] Zico Kolter, Chuong Do (September 30, 2015). Linear Algebra Review and Reference notes. http://cs229.stanford.edu/section/cs229-linalg.pdf
[5] Kramer, R. (1998). Chemometric techniques for quantitative analysis. CRC Press.
[6] Workman, J. and Mark, H. (2010). Chemometrics in Spectroscopy: Classical Least Squares, Part I: Mathematical Theoryhttp://www.spectroscopyonline.com/classical-least-squares-part-i-mathematical-theory?id=&pageID=1&sk=&date=
[7] Bocklitz, T. W., Guo, S., Ryabchykov, O., Vogler, N., & Popp, J. (2016). Raman based molecular imaging and analytics: a magic bullet for biomedical applications!?. Analytical chemistry, 88(1), 133–151.