# Portable Multispectral System Based on Color Detector for the Analysis of Homogeneous Surfaces

Authors: A. Martínez-Olmos; P. M. Olmos; M. M. Erenas; P. Escobedo
Journal: Journal of Sensors
 (2019)

---

## Abstract

In this work, a compact affordable and portable spectral imaging system is presented. The system is intended to be employed in general applications, such as material classification or determination of the concentration of chemical species together with colorimetric sensors. The imaging device is reduced to a small digital color detector with an active area of3×2mm2. This device provides a quantification of the incident emission in the form of four digital words corresponding to its averaged components blue, green, red, and near infrared. In this way, the size of the image is reduced to one pixel. The wavelength selection is carried out by means of a LED array disposed surrounding the color detector. The LEDs are selected to cover the wavelength range from 360 to 890 nm. A sequential measurement protocol is followed, and the generated data is transmitted to an external portable device via a Bluetooth link where a classification protocol is implemented in a custom-developed Android™ application. The presented system has been applied in three different scenarios involving material classification, meat freshness monitoring, and chemical analysis. The analysis of the data using principal components shows that it is possible to find a set of wavelengths where the classification of the samples is optimal.


---

## Body

## 1. Introduction

Imaging spectrometry was firstly defined by Goetz et al. in 1985 [1] “as a major advance in remote sensing which consists of the acquisition of images in many narrow contiguous spectral bands throughout the visible and solar-reflected infrared spectral bands simultaneously”. With this technique it is possible to acquire a complete reflectance spectrum for each picture element (pixel) in the image that it is considered as a cube of information [2]. The term hyperspectral refers to the multidimensional character of the spectral data set. The technique is known as multispectral imaging when the images are acquired at several discrete wavelengths in the considered range of the spectrum, usually 6 to 10 or even less [3, 4]. Imaging spectrometry is usually applied over a wide range of wavelengths covering from ultraviolet (UV) at 380 nm to near infrared (NIR) up to 1100 nm. It combines two methodologies, spectroscopy and imaging. The imaging part of this technique provides the intensity at every pixel of the image, whereas the spectrometry side generates a single spectrum for the same element [5]. Spectral imaging was introduced as a technique for remote sensing. Since then, this analysis technique has expanded to many other fields, and nowadays it is applied in a diversity of areas such as agriculture, military, environment, geography, medicine, and nutrition, among others. [6–11].The main components that comprise a typical spectral imaging system are lighting system, focusing optics, detection system, and wavelength selector. The design of the illumination system is mostly determined by the method for wavelength selection. First studies on multispectral imaging systems employed a monochrome digital camera to collect the reflected light of the previously filtered illumination [12]. Other systems filter or disperse the reflected light to select the desired wavelength [13, 14]. More recently, wavelength selection is directly achieved within the illumination source by illuminating the sample with quasimonochromatic light. Programmable light sources are commonly used in these kinds of systems [15]. Nowadays, the expansion of very effective light-emitting diodes (LEDs) that are invariant and remarkably stable as compared with white lights has enabled the development of LED-based multispectral imaging systems [16, 17]. Conventional spectral imaging systems include complicated illumination sources, delicate optics, and high-resolution cameras [18–21], so they tend to be expensive and fragile laboratory equipment. Moreover, the system control and the data collection and processing are usually implemented in a personal computer (PC).In a previous work, the authors presented a compact multispectral system based on a Raspberry Pi module [22]. The low cost and compact size of this camera module, facing other in the market, as well as the processing capacity of the Raspberry in a compact and portable instrument, makes this novel system suitable for a variety of applications. Following these design criteria, that is, portability, low cost, and wide range of applications, a new prototype is developed and presented here. The novelty of this instrument lies in the image detector employed and the classification software implemented as an Android™ application. While the previous system, as it happens in most of the described multispectral systems, was based on a camera for the image acquisition, a simple digital color detector is used in this work. This device provides the red, green, blue, and infrared (R, G, B, and IR) components of the incident light on a small active area in a form of 16-bit digital words. This is considered as an image of only one pixel, which can be representative of an extensive and homogeneous area, meaning that the texture and color of the sample are constant on the whole surface. In this way, the complexity of the system and the generated data is reduced, and the color depth is improved in comparison with the images generated with classical CMOS cameras. These one-pixel images are transmitted to a remote portable device such as a smartphone or a tablet where the results of the classification process are presented.The system has been evaluated in three different scenarios. In the first one, a material classification has been carried out to analyze several kinds of white sheets of paper. In the second experience, a package of fresh pork meat has been monitored during eight days. In the third experiment, the instrument has been applied to measure the concentration of potassium in water solutions at pH 9. With a simple dimensionality reduction technique (principal component analysis) combined with a standard low-complexity classification tool (support vector machine), it is possible to determine at which wavelengths the samples can be separated with high precision. The classification algorithm has been implemented as an Android application to be used in a smart device that communicates with the developed instrument via a Bluetooth protocol.


## 2. System Description

As it has been exposed in the previous section, the presented system has been developed as a low cost portable multispectral imaging system where the imaging device is a digital color detector that generates 1-pixel images. These data are transmitted through a Bluetooth protocol to a portable device (smartphone or tablet) or through a USB connection to a computer that completes the system. In both devices, the data are processed and presented to the user. The scheme of the developed system, which is exposed in detail in the following sections, is presented in Figure1.Figure 1
Scheme of the presented system.


### 2.1. Instrument Design

The portable instrument is shown in the picture of Figure2(a). The printed circuit board has dimensions of 8×12cm2, and it is enclosed in a black box of dimensions 9×15×3cm3 to avoid external illumination interference. This box has an aperture facing the sensing area of the instrument composed of the imaging device and the light source. The instrument is placed directly on the surface of the sample to be analyzed; in this way, the distance to the LED array and the imaging device is always the same as depicted in Figure 2(b).Figure 2
Portable instrument (a) and measurement disposition (b).
(a)
(b)The imaging device is the color detector model S11059-02DT (Hamamatsu Photonics K.K., Japan), which is an I2C interface-compatible digital detector sensitive to red (575 to 660 nm,λpeak=615nm), green (455 to 630 nm, λpeak=530nm), blue (400 to 540 nm, λpeak=460nm), and near infrared (700 to 885 nm, λpeak=855nm) radiations. The incident light is directly codified into words of 16 bits of resolution. The sensitivity and integration time can be adjusted so that light measurements can be performed over a wide range. This device presents a photosensitive area of 0.56×1.22mm2.The light source consists of a LED matrix surrounding the color detector. The selected models are the following: VLMU1610-365-135, VLMU3100 (Vishay Intertechnology Inc., USA), LD MSVG-JGLH-46-1 (OSRAM Opto Semiconductors GmbH, Germany), AA3021ZGSK, APTR3216PGW, APT2012NW, APT2012SRCPRV (Kingbright Electronic Co., Ltd., China), SML-LX15HC-RP-TR (Lumex Inc., USA), VSMG2700, and VSMF3710 (Vishay Intertechnology Inc.). The emission peaks of these LEDs are, respectively, 367, 405, 455, 515, 555, 610, 655, 700, 830, and 890 nm as it can be observed in Figure3(a), where the normalized emission spectra of the LEDs are depicted.Figure 3
Normalized emission spectra of the LEDs (a) and scheme of the distribution of the LEDs (b).
(a)
(b)There are two LEDs of each model in the array disposed in symmetrical positions as presented in Figure3(b). In this way, the color detector is always placed in the middle of every two LEDs of the same model. The distribution of the array in couples of LEDs is aimed to generate a uniform irradiance distribution on the sample. It is known that for a two-LED array the irradiance is homogenous at a distance z if the LEDs are separated a maximum distance d given by [23, 24]
(1)d=4m+3⋅z,where m is a number that depends on the relative position of the LED-emitting region from the curvature center of the spherical encapsulant. The value of m is given by the angle θ1/2 defined as the view angle when irradiance is half of the value at 0° (value typically provided by the manufacturer):
(2)m=−ln2lncosθ1/2.In the presented work, the distance to the sample is determined by the height of the box (see Figure2(b)) which is 3 cm. For the selected LED, the angle θ1/2 ranges from 60 to 70°, therefore the value of m in equation (2) varies from 0.65 to 1. This range leads to a maximum distance between LEDs of 3 to 3.14 cm for a distance to the sample z=3cm. In the design of the LED matrix of Figure 3(b), the distance of the LEDs in couple ranges from 2.2 cm in the middle of the square to 3 cm from corner to corner. These values guarantee that the irradiance on the sample is homogeneous. The wavelength selection is carried out activating the LEDs sequentially in a growing wavelength.All the LEDs have been biased at their typical forward voltage by applying a voltage of 5 V to the serial combination of every LED and a resistor of 220Ω. In this situation, the LEDs showed a stable radiance with intensity dispersion below 0.1%. This configuration produces different intensities for different LED models. In order to generate a uniform response of the system, the integration time for each wavelength is defined so that the product Ie×Ti (intensity of emission multiplied by the integration time) is constant. In this way, the integration time associated with a wavelength with a low-intensity emission is higher than the time corresponding to a wavelength generated by a high-intensity LED. These integration times range from 35 to 700 ms.The microcontroller used in this design is the model PIC18F2550 (Microchip Technology Inc., USA). This device integrates a full speed USB 2.0 (12 Mbit/s) interface that has been used for calibration purposes. The microcontroller activates the LEDs of the light source and receives the output of the digital color detector. It transmits these results to an external computer through the USB port or to a remote device via a Bluetooth connection. To implement this wireless communication, a Bluetooth module RN-42 (Microchip Technology Inc.) is included in the design. This module communicates with the microcontroller through a two-wire serial protocol and integrates a small antenna.


### 2.2. Android Application

A user-friendly Android application was developed to use a smartphone as the external reader and processing unit of the multispectral imaging device. The instrument is connected to the smartphone for bidirectional data transmission using the Bluetooth interface. The chosen integrated development environment (IDE) to code the application was Android Studio 3.1.3. The application was designed and tested against API 24 (Android 7.0) using a Samsung smartphone model Galaxy S7, although it supports different Android versions as the lowest API level compatible with the application is API 18 (Android 4.3).The application user interface consists of two screens. The first one, shown in Figure4(a), allows a generic use of the multispectral imaging device. The user can choose between a sequential measurement throughout all the frequencies or the selection of a particular wavelength by means of a slider control. After the measurement is done, the results of the R, G, B, and IR components for each wavelength can be consulted in a plain text report that is saved in the internal memory of the smartphone. Figure 4(b) shows the second user interface of the application, where the user can choose among the three different classification scenarios in which the system has been evaluated. The processing algorithm is changed accordingly to the chosen scenario, and the final results of the measurement are directly displayed on the screen.Figure 4
User interfaces of the developed Android multispectral imaging application for (a) a generic use of the instrument and (b) a specific scenario-based use.
(a)
(b)


## 2.1. Instrument Design

The portable instrument is shown in the picture of Figure2(a). The printed circuit board has dimensions of 8×12cm2, and it is enclosed in a black box of dimensions 9×15×3cm3 to avoid external illumination interference. This box has an aperture facing the sensing area of the instrument composed of the imaging device and the light source. The instrument is placed directly on the surface of the sample to be analyzed; in this way, the distance to the LED array and the imaging device is always the same as depicted in Figure 2(b).Figure 2
Portable instrument (a) and measurement disposition (b).
(a)
(b)The imaging device is the color detector model S11059-02DT (Hamamatsu Photonics K.K., Japan), which is an I2C interface-compatible digital detector sensitive to red (575 to 660 nm,λpeak=615nm), green (455 to 630 nm, λpeak=530nm), blue (400 to 540 nm, λpeak=460nm), and near infrared (700 to 885 nm, λpeak=855nm) radiations. The incident light is directly codified into words of 16 bits of resolution. The sensitivity and integration time can be adjusted so that light measurements can be performed over a wide range. This device presents a photosensitive area of 0.56×1.22mm2.The light source consists of a LED matrix surrounding the color detector. The selected models are the following: VLMU1610-365-135, VLMU3100 (Vishay Intertechnology Inc., USA), LD MSVG-JGLH-46-1 (OSRAM Opto Semiconductors GmbH, Germany), AA3021ZGSK, APTR3216PGW, APT2012NW, APT2012SRCPRV (Kingbright Electronic Co., Ltd., China), SML-LX15HC-RP-TR (Lumex Inc., USA), VSMG2700, and VSMF3710 (Vishay Intertechnology Inc.). The emission peaks of these LEDs are, respectively, 367, 405, 455, 515, 555, 610, 655, 700, 830, and 890 nm as it can be observed in Figure3(a), where the normalized emission spectra of the LEDs are depicted.Figure 3
Normalized emission spectra of the LEDs (a) and scheme of the distribution of the LEDs (b).
(a)
(b)There are two LEDs of each model in the array disposed in symmetrical positions as presented in Figure3(b). In this way, the color detector is always placed in the middle of every two LEDs of the same model. The distribution of the array in couples of LEDs is aimed to generate a uniform irradiance distribution on the sample. It is known that for a two-LED array the irradiance is homogenous at a distance z if the LEDs are separated a maximum distance d given by [23, 24]
(1)d=4m+3⋅z,where m is a number that depends on the relative position of the LED-emitting region from the curvature center of the spherical encapsulant. The value of m is given by the angle θ1/2 defined as the view angle when irradiance is half of the value at 0° (value typically provided by the manufacturer):
(2)m=−ln2lncosθ1/2.In the presented work, the distance to the sample is determined by the height of the box (see Figure2(b)) which is 3 cm. For the selected LED, the angle θ1/2 ranges from 60 to 70°, therefore the value of m in equation (2) varies from 0.65 to 1. This range leads to a maximum distance between LEDs of 3 to 3.14 cm for a distance to the sample z=3cm. In the design of the LED matrix of Figure 3(b), the distance of the LEDs in couple ranges from 2.2 cm in the middle of the square to 3 cm from corner to corner. These values guarantee that the irradiance on the sample is homogeneous. The wavelength selection is carried out activating the LEDs sequentially in a growing wavelength.All the LEDs have been biased at their typical forward voltage by applying a voltage of 5 V to the serial combination of every LED and a resistor of 220Ω. In this situation, the LEDs showed a stable radiance with intensity dispersion below 0.1%. This configuration produces different intensities for different LED models. In order to generate a uniform response of the system, the integration time for each wavelength is defined so that the product Ie×Ti (intensity of emission multiplied by the integration time) is constant. In this way, the integration time associated with a wavelength with a low-intensity emission is higher than the time corresponding to a wavelength generated by a high-intensity LED. These integration times range from 35 to 700 ms.The microcontroller used in this design is the model PIC18F2550 (Microchip Technology Inc., USA). This device integrates a full speed USB 2.0 (12 Mbit/s) interface that has been used for calibration purposes. The microcontroller activates the LEDs of the light source and receives the output of the digital color detector. It transmits these results to an external computer through the USB port or to a remote device via a Bluetooth connection. To implement this wireless communication, a Bluetooth module RN-42 (Microchip Technology Inc.) is included in the design. This module communicates with the microcontroller through a two-wire serial protocol and integrates a small antenna.


## 2.2. Android Application

A user-friendly Android application was developed to use a smartphone as the external reader and processing unit of the multispectral imaging device. The instrument is connected to the smartphone for bidirectional data transmission using the Bluetooth interface. The chosen integrated development environment (IDE) to code the application was Android Studio 3.1.3. The application was designed and tested against API 24 (Android 7.0) using a Samsung smartphone model Galaxy S7, although it supports different Android versions as the lowest API level compatible with the application is API 18 (Android 4.3).The application user interface consists of two screens. The first one, shown in Figure4(a), allows a generic use of the multispectral imaging device. The user can choose between a sequential measurement throughout all the frequencies or the selection of a particular wavelength by means of a slider control. After the measurement is done, the results of the R, G, B, and IR components for each wavelength can be consulted in a plain text report that is saved in the internal memory of the smartphone. Figure 4(b) shows the second user interface of the application, where the user can choose among the three different classification scenarios in which the system has been evaluated. The processing algorithm is changed accordingly to the chosen scenario, and the final results of the measurement are directly displayed on the screen.Figure 4
User interfaces of the developed Android multispectral imaging application for (a) a generic use of the instrument and (b) a specific scenario-based use.
(a)
(b)


## 3. Results and Discussion

The presented system has been applied in three different scenarios with very diverse objectives. The aim is to prove the applicability of this instrument in a wide range of fields.Automatic classification is one of the main applications of spectral imaging [25, 26]. In the first experience, the system has been used to analyze several kinds of white paper with the objective of developing a classification algorithm from the generated data (Scenario I).In the second experience, a package of fresh pork meat has been monitored during 8 days while it was stored at 4°C (Scenario II). It is known that the packaged meat is affected by the microbial activity, which is the main responsible for the food spoilage [27]. Although the external appearance of the meat might not be altered to the naked eye, this bacterial growth, which increases with the storage time, produces quality degradation [28]. The objective of this experience is to analyze the packaged meat daily in order to develop a prediction algorithm that is able to estimate the storage time of packaged meat and, therefore, the status of the content.In the third experiment, the instrument has been applied to measure the concentration of potassium in water solutions at pH 9 (Scenario III). For this purpose, a potassium sensitive membrane has been used. The reagents used to prepare the sensing membrane were 0.8 mg of dibenzo18-crown-6-ether (DB18C6) as ionophore, 1.3 mg of 1,2-benzo-7-(diethyla-mino)-3-(octadecanoylimino) phenoxazine (liphophilized Nile Blue) as lipophilic pH indicator, 63.0 mg of o-nitrophenyloctylether (NPOE) as plasticizer, 1.1 mg of potassium tetrakis (4-chlorophenyl)-borate (TCPB) as lipophilic salt and 26.0 mg of polyvinyl chloride (PVC) as polymer, all dissolved in 1 mL of tetrahydrofurane (THF) [29, 30]. Once dissolved, 60 μL of the sensing cocktail is dropped by spin coating on a Mylar sheet, obtaining in this way a round-shape sensing membrane whose color changes depending on the potassium concentration when it is introduced for 3 minutes in the sample, working in the range from 10-5 to 0.95 M of potassium.In each of the three scenarios, each measurement is a four-dimensional vectorxλ, where λ is the LED-emitting wavelength. We consider ten different values of the emitting wavelength (see Figure 2) and a total of six measurements per wavelength. Figures 5(a)–5(c) shows the resulting data projected in a two-dimensional space using principal component analysis (PCA) for representative wavelengths where the data is visually separated among classes. Data has been standardized to have zero mean. In Figure 5(a), the results for Scenario I when λ=700nm are shown. In Figure 5(b), Scenario II when λ=405nm is depicted. Figure 5(c) shows Scenario III when λ=515nm. In all cases, data points corresponding to the same class (paper, days of storage, or potassio concentration) cluster together, reasonably separated away (compared to the cluster variance) from data corresponding to other classes.Figure 5
Two-dimensional projection with PCA of the dataset in Scenarios I (a), II (b), and III (c) at three wavelengths. Legend in (c) refers to logarithmic pH values.
(a)
(b)
(c)To assess the performance of a classifier that uses the low-dimensional measure embedding described in Figure6, we compare the average classification test error rate for each wavelength for a linear support vector machine (SVM) [31] when the 30% of the data is left out for test. Each experiment is repeated 1000 times for different train/test partitions. The results are shown by means of a box plot, where the dashed central line shows the median of the empirical distribution of the error along the 1000 repetitions of the experiment, the box boundaries represent the first and 3 quartiles of the error, and the lines outside the boxes represent the extreme error values in the sample. It can be observed that the error rate heavily depends on the wavelength, but for every case we can find a wavelength λ for which the error mode is close to zero. Further, only results for Scenarios II and III are presented, as for Scenario I the error rate for all wavelengths is zero (i.e., a linear classifier perfectly separates the available data no matter the choice of the training set). For Scenario II, the average error rate is minimized at λ=515nm (2.25%), and for Scenario III the average error rate is minimized at λ=655nm (0.4%).Figure 6
Classification performance for Scenario II (a) and Scenario III (b) using a linear SVM classifier trained over 70% of the available data.
(a)
(b)Further, the normalized inverse distance to the SVM decision boundary can be used as an approximation to the classifier’s probabilistic confidence [32]. In the experiments summarized in Figure 6, it is numerically observed that the confidence in the right class is typically 10 times larger than for the rest of the classes. For instance, in Scenario II, the average confidence in the right class is 0.567, while for the rest of the classes the average confidence is only 0.042.The results described above demonstrate that the proposed portable multispectral imaging system is able to provide discriminative measurements using only four light bands, and hence it can be used for cheap low-complexity detection device for a wide set of potential industrial applications.The previous experiments have been repeated using the CMOS camera of a smartphone instead of the color detector, maintaining the same geometry presented in Figure2. Since the analyzed samples are homogeneous in texture and color, only 1 pixel of the photography provided by the camera is considered. In this scenario, the difference in the two sets of experiments, the one carried out with the color detector and the other with the CMOS camera, is only the color depth (16 bits for the color detector and 8 bits for the CMOS camera). The prediction errors generated with both systems are presented in Table 1. As it can be observed, the error rates in the prediction where the analysis is carried out with the CMOS camera are much higher than the ones obtained with the original system.Table 1
Comparison of prediction error rate in the 3 different scenarios when the analysis is carried out using the color detector and the CMOS camera.
Lambda (nm)
Error rate Scenario I
Error rate Scenario II
Error rate Scenario III
Color detector
CMOS camera
Color detector
CMOS camera
Color detector
CMOS camera
367
0.000
0.000
0.121
0.882
0.051
0.757
405
0.000
0.000
0.034
0.681
0.027
0.379
455
0.000
0.000
0.042
0.534
0.040
0.439
515
0.000
0.000
0.022
0.441
0.006
0.146
555
0.000
0.162
0.030
0.564
0.011
0.311
610
0.007
0.210
0.530
0.822
0.020
0.226
655
0.000
0.000
0.434
0.544
0.006
0.268
700
0.000
0.153
0.204
0.628
0.023
0.380
830
0.000
0.000
0.252
0.407
0.259
0.723
890
0.000
0.000
0.290
0.337
0.423
0.622


## 4. Conclusions

In this work, we have presented a prototype of a multispectral imaging system for general purposes. It is based on the use of a high-resolution digital color detector as the imaging device and a LED panel for the wavelength selection. The color detector generates one-pixel images in a four-dimension space (R, G, B, and IR) that are considered representative of an extensive homogeneous area. These images are transmitted to a remote smart device through a Bluetooth wireless connection. A custom application for Android operative system has been developed for the acquiring and processing of the images. This scheme implies a simplification of the traditional multispectral imaging system based on a complex camera not only in the hardware requirements but also in terms of signal processing. The result has been a compact and portable field measuring instrument that is easy to use for any nontrained user.The feasibility of the prototype has been proved in three different scenarios. In the first one, the analysis of different white paper samples has been carried out. In the second one, the external appearance of packaged pork meat has been monitored during 8 consecutive days. In the last experience, the system has been applied for the determination of potassium concentration by the analysis of a potassium-sensitive membrane. The generated data in each experiment have been studied by multivariate analysis techniques such as principal component, finding that in every case it is possible to select a wavelength for which the samples can be separated with high precision. Therefore, we have proved that the presented system offers broad application possibilities including automatic samples classification, monitoring of the status of stored food, or colorimetric determination of analytes in solution.



---

*Source: 1519753-2019-01-10.xml*