Package: boxsers
IntroducesBoxSERS, a complete and ready-to-use python library for the application of data augmentation, dimensional reduction, spectral correction, machine learning and other methods specially designed and adapted for vibrational spectra(Raman,FTIR, SERS, etc.).Table of contentsBoxSERS InstallationRequirementsIncluded FeaturesModule misc_toolsModule visual_toolsModule preprocessingModule data augmentationDimensional ReductionUnsupervised Machine LearningSupervised Machine LearningModule validation_metricsLicenseBoxSERS InstallationFrom PypYpipinstallboxsersFrom Githubpipinstallgit+https://github.com/ALebrun-108/BoxSERS.gitRequirementsListed below are the main modules needed to operate the codes:SklearnScipyNumpyPandasMatplotlibTensor flow (GPU or CPU)Labels associated to spectra can be in one of the following three forms:Label TypeExamplesTextCholic, Deoxycholic, Lithocholic, ...Integer0, 3, 1 , ...Binary[1 0 0 0], [0 0 0 1], [0 1 0 0], ...Included FeaturesModuleboxsers.misc_toolsThis module provides functions for a variety of utilities.data_split :Randomly splits an initial set of spectra into two new subsets named in this
function: subset A and subset B.load_rruff :Export a subset of Raman spectra from the RRUFF database in the form of three related lists
containing Raman shifts, intensities and mineral names.Moduleboxsers.visual_toolsThis module provides different tools to visualize vibrational spectra quickly.spectro_plot :Returns a plot with the selected spectrum(s)random_plot :Plot a number of randomly selected spectra from a set of spectra.distribution_plot :Return a bar plot that represents the distributions of spectra for each classes in
a given set of spectra# Code example:fromboxsers.misc_toolsimportdata_splitfromboxsers.visual_toolsimportspectro_plot,random_plot,distribution_plotwn=3spec=5# randomly splits the spectra(spec) and the labels(lab) into test and training subsets.(spec_train,spec_test,lab_train,lab_test)=data_split(wn,spec,b_size=0.4)# resulting train|test set proportions = 0.6|0.4# plots the classes distribution within the training set.distribution_plot(lab_train,title='Train set distribution')# spectra array = spec, raman shift column = wnrandom_plot(wn,spec,random_spectra=4)# plots 4 randomly selected spectraspectro_plot(wn,spec[0],spec[2])# plots first and third spectraModuleboxsers.preprocessingThis module provides multiple functions to preprocess vibrational spectra. These features
improve spectrum quality and can improve performance for machine learning applications.baseline_substraction: Subtracts the baseline signal from the spectrum(s) using
Asymmetric Least Squares estimation.intensity_normalization: Normalizes the spectrum(s) using one of the available norms in this function.savgol_smoothing: Smoothes the spectrum(s) using a Savitzky-Golay polynomial filter.spectral_cut: Subtracts or sets to zero a delimited spectral region of the spectrum(s)spline_interpolation: Performs a one-dimensional interpolation spline on the spectra to reproduce
them with a new x-axis.# Code example:importnumpyasnpfromboxsers.preprocessingimportals_baseline_cor,spectral_cut,spectral_normalization,spline_interpolation# interpolates with splines the spectra and converts them to a new raman shift range(new_wn)new_wn=np.linspace(500,3000,1000)spec_cor=spline_interpolation(spec,wn,new_wn)# removes the baseline signal measured with the als method(spec_cor,baseline)=als_baseline_cor(spec,lam=1e4,p=0.001,niter=10)# normalizes each spectrum individually so that the maximum value equals one and the minimum value zerospec_cor=spectral_normalization(spec)# removes part of the spectra delimited by the Raman shift values wn_start and wn_endspec_cor,wn_cor=spectral_cut(spec,wn,wn_start,wn_end)Moduleboxsers.data_augmentationThis module provides several data augmentation methods that generate new spectra by adding
different variations to existing spectra.aug_mixup: Randomly generates new spectra by mixing together several spectra with a Dirichlet
probability distribution.aug_noise: Randomly generates new spectra with Gaussian noise added.aug_multiplier: Randomly generates new spectra with multiplicative factors applied.aug_ioffset: Randomly generates new spectra shifted in intensity.aug_xshift: Randomly generates new spectra shifted in wavelength.aug_linslope: Randomly generates new spectra with additional linear slopes# Code example:fromboxsers.data_augmentationimportaug_mixup,aug_noisespectra_nse,label_nse=aug_noise(spec,lab,snr=10)spectra_mult,label_mult=aug_multiplier(spectra,labels,0.15,)spectro_plot(wn,spec,spec_nse,spec_mult_sup,spec_mult_inf,legend=legend)spec_nse,lab_nse=aug_noise(spec,lab,param_nse,quantity=2,mode='random')spec_mul,lab_mul=aug_multiplier(spec,lab,mult_lim,quantity=2,mode='random')# stacks all generated spectra and originals in a single arrayspec_aug=np.vstack((x,spec_nse,spec_mul))lab_aug=np.vstack((lab,lab_nse,lab_mul))# spectra and labels are randomly mixedx_aug,y_aug=shuffle(x_aug,y_aug)Moduleboxsers.dimension_reductionThis module provides different techniques to perform dimensionality reduction of
vibrational spectra.SpectroPCA:Returns a plot with the selected spectrum(s)SpectroPCA: Plot a number of randomly selected spectra from a set of spectra.distribution_plot: Return a bar plot that represents the distributions of spectra for each classes in
a given set of spectraDimensional Reduction# Code example:fromboxsers.machine_learning.dimension_reductionimportSpectroPCA,SpectroICApca_model=SpectroPCA(n_comp=50)pca_model.fit_model(spec_train)pca_model.scatter_plot(spec_test,spec_test,targets=classnames,component_x=1,component_y=2)pca_model.component_plot(wn,component=2)spec_pca=pca_model.transform_spectra(spec_test)Unsupervised Machine Learning# Code example:fromboxsers.machine_learningimportSpectroGmixture,SpectroKmeanskmeans_model=SpectroKmeans(n_cluster=5)kmeans_model.fit_model(spec_train)kmeans_model.scatter_plot(spec_test)Supervised Machine LearningConvolutional Neural Networt (3 x Convolutional layer 1D , 2 x Dense layer)fromboxsers.pca_modelimportSpectroPCA,SpectroFA,SpectroICApca_model=SpectroICA(n_comp=50)pca_model.fit_model(x_train)pca_model.scatter_plot(x_test,y_test,targets=classnames,comp_x=1,comp_y=2)pca_model.pca_component(Wn,2)x_pca=pca_model.transform_spectra(x_train)Modulevalidation_metricsThis module provides different tools to evaluate the quality of a modelâ€™s predictions.cf_matrix:  Returns a confusion matrix (built with scikit-learn) generated on a given set of spectra.clf_report: Returns a classification report generated from a given set of spectra