Supervised training sample selection for the estimation of spectral reflectance using a RGB camera

Spectral imaging can provide spectral information f rom which spectral radiance or reflectance can be recove red at each image pixel. Recovery algorithms lead to good spectral and colorimetric performance by directly transformi ng RGB digital counts to spectral reflectances, but his ap proach is sensitive to the size and composition of the traini ng set. What we propose here is a supervised method to select the most appropriate samples from a training database to bul d the transformation matrix relating digital counts to sp ectral reflectances. Thus, this approach is tested with rea l im ges. Introduction Multispectral imaging uses a digital camera coupled or not with colour filters of different spectral bands, ra nging from just one band, as in a monochromatic system, to hundreds of components, as in a ultraspectral system [1]. The m ain advantage of spectral imaging in comparison with co nventional spectroradiometric measurements is that they can pr ovide spectral information from which spectral radiance o r reflectance can be recovered at each image pixel. During the la st years, many different approaches have been proposed for sp ectral reflectance recovery [2]. Among these techniques, r covery algorithms lead to good spectral and colorimetric p erformance by directly transforming RGB digital counts to spectr al reflectances [3]. This approach can be sensitive to the size and composition of the training set of reflectances [4] . Other authors have shown that depending on the spectral a pplic tion (e. g. spectral pigment analysis in art pictures) t he quality of spectral recovery may change dramatically. In this work we propose a supervised training sample selection meth od for spectral reflectance estimation based on a linear p seudo-inverse algorithm. The method uses a set of training sample s selected from a database which are in the neighborhood of th e target sample to optimize the recovery matrix relating dig ital counts to spectral reflectances. What we propose here is to o p imize the building process of the transformation matrix using a learningbased algorithm. Recently different computational results suggest tha t using cut off filters to recover reflectances or illumina nts may or not improve the recovery performance, depending on the presence of noise [5, 7]. As these results are not clear for n isy data, we will also analyze here the effects of adding succes siv cut off filters when natural scenes are captured using a re al RGB camera. Method In the present work, we used real images captured w ith an RGB digital color camera from QImaging (model Retiga 1 300, 12 bits). The images captured were the Gretag Macbe th Color Checker DC and Color Checker rendition chart [8]. In a second step we added cut off filters in front of the camer a lens (GG475 and OG550, from OWIS GMBH) to study possible improvements. Given a set of training spectra S (which can be spectral radiances or reflectances) and the corresponding se t of experimental camera responses ρ, a recovery transformation matrix D is defined by D = Sρ, where ρ is the pseudo-inverse of ρ. If ρ has full rank, then ρ = (ρ ρ) ρ, where ρ is the transpose of ρ. An estimate of Ŝ1 of a set of test spectra S1 may then be obtained from the corresponding set of came r responses ρ1 by applying the transformation D, that is Ŝ1 = Dρ1 [7]. We present here a method to select the most appropr iate samples from a training database to be used as trai ning set to recover reflectances from RGB data of a test image, w ithout any need of knowing any spectral information of the test sample. The first step is to calculate CIELAB coordin ates from both test and training set using real RGB digital cou nts captured with a CCD camera in both cases. Once we hav t is information, CIELAB color difference di is calculated between each test sample and the whole training set. It all ows us to sort the training set for each test sample from minimum to maximum color difference. To choose the training set elements to recover refl ectance for each test sample, we set a sphere in each eleme nt of the test set in the CIELAB space, and we will use the elements of the training set inside this sphere. To calculate the r adius of this sphere, we implement an iterative process as explained in Figure 1. It sets an initial sphere in the test sam ple, looks for the elements of the training set inside this sphere and compute the mean of its distances, giving them a weight tha t depends on the number of iterations and the number of elements inside the sphere. Then, the radius of the sphere is increased nd the process is repeated decreasing the weight. The algorithm description is: Fix a starting radius r0, a constant k, and the number of iterations a. In this work the appropriate values of those constants comes from previous experimental results. Set a first sphere of radius r0 centered on a test sample in the CIELAB space, and look for the elements of the tr aining set inside of the sphere. Those elements will have CIELA B differences di ≤ r0. Now, it is possible to calculate a weighted mean radius T0 as: