## 2 Materials and Methods
### RIS Image Cube Collection and Pre-Processing
#### 2.2.2 Spectral Training Libraries
The training pigment classes identified in total comprised natural and synthetic, as well as inorganic and organic, pigments (Table 2): azurite (Cu3(CO3)2(OH)2), ultramarine ((NaCa)8 Al6 Si6O24(S,SO)), malachite (Cu2 CO3(OH)2), red lead (Pb2+Pb4+O4), vermiling (HgS), an insect-based red lake, a goethite (FeO(OH))-rich ochre, a hematite (Fe2O3)-rich ochre, lead white (hydrocerusite, Pb3(CO3)2(OH)2), lead-tin yellow, folium (chrozophoridin), and a carbon-based black material. Some class labels are simply the observed color, as we were not able to determine the exact pigment composition: for example, \"Brown\", which has spectral features of an iron earth pigment. For the identification of folium, Surface-Enhanced Raman Spectroscopy (SERS) analysis was carried out on one micro-sample taken from a purple area in the medallion of the illumination _Initial D_ (1964.8.1223) at the Department of Scientific Research of The Metropolitan Museum of Art using a Bruker Santerra Raman spectrometer equipped with a CCD detector and an Olympus 20\(\times\) long working distance microscope objective. Samples were treated with hydrofluoric acid vapor in a polyethylene micro-chamber for 5 min [32] and then covered with a 2 \(\upmu\)L drop of silver nanoparticles, followed by 0.5 \(\upmu\)L of 0.5 M potassium nitrate as aggregating agent. Silver colloids were prepared following a synthesis \begin{table}
\begin{tabular}{l c} \hline \hline
**Illuminations used to train the classification models** & **Accession number** \\ \hline Initial S–[PERSON] [PERSON] as Scribe & 1964.8.1218 \\ Initial T–Monks Singing Before an Altar & 1964.8.1219 \\ Initial D & 1964.8.1221 \\ Initial L & 1964.8.1222 \\ Initial U (?) & 1964.8.1224 \\ Initial N (?)–David in Prayer & 1964.8.1225 \\ Initial I–[PERSON] & 1964.8.1227 \\ Initial C–[PERSON] (King [PERSON]?) & 1964.8.1228 \\ Initial L–Old Testament Prophet & 1964.8.1229 \\ Initial D–[PERSON] & 1964.8.1230 \\ \hline
**Illuminations used to test classification models** & **Accession number** \\ \hline Initial I–[PERSON] & 1964.8.1226 \\ Initial D & 1964.8.1223 \\ Initial A & 1964.8.1220 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Illuminations used in this research. All illuminations are from the Rosenwald Collection, National Gallery of Art, Washington, D.C.  
previously published [33]. Excitation at 488 and 633 nm was provided by a Spectra Physics Cyan solid state laser and a Melles Griot He-Ne laser, respectively. An output laser power of 4 mW was employed for the analysis, with two integrations of 30 s.  
Some training classes ended up being extremely large in number of spectra because they were used extensively throughout all of the illuminations (e.g., \"Red lake\" and \"Malachite\"), while some classes had fewer spectra due to pigments less frequently used, such as \"Red earth\". To limit the impact of imbalanced class sizes on the neural network's training, a random per-class sub-selection was made to reduce the size of the larger classes down to 6000 spectra each while maintaining sufficient spectral variability within the classes (Figure 3c,d). The resultant two training spectral libraries, each containing all twenty-seven classes, were then used to test the different classification methods described in the following section. Figure 3: An example of the regions of interest (ROI) selected to build the Reflectance and Derivative CNN training class spectral libraries. (**a**) ROI selection from RIS image cube of _Initial S_—_King David as Scribe_ (1964.8.1218), with areas of similar spectra grouped together into the same class (denoted in the image by the same color ROI). (**b**) ROI selection from the first-derivative calculated RIS image cube of 1964.8.1218. These areas are the same as those selected in the reflectance cube. (**c**) After extracting spectra across the ten training illuminations, the final classes resulted in thousands of spectra with a small range of spectral variability due to hue/saturation, as shown in these four example classes.