# Remote Colorimetric and Structural Diagnosis by RGB-ITR Color Laser Scanner Prototype

Authors: Massimiliano Guarneri; Mario Ferri de Collibus; Giorgio Fornetti; Massimo Francucci; Marcello Nuvoli; Roberto Ricci
Journal: Advances in Optical Technologies
 (2012)

---

## Abstract

Since several years ENEA's Artificial Vision laboratory is involved in electrooptics systems development. In the last period the efforts are concentrated on cultural heritage remote diagnosis, trying to develop instruments suitable for multiple purposes concerning restoration, cataloguing, and education. Since last five years a new 3D (three-dimensional) laser scanner prototype (RGB-ITR) based on three amplitude-modulated monochromatic laser sources mixed together by dichroic filters is under development. Five pieces of information per each sampled point (pixel) are collected by three avalanche photodiodes and dedicated electronics: two distances and three target reflectivity signals for each channel, red, green, and blue. The combination of these pieces of information opens new scenarios for remote colorimetry allowing diagnoses without the use of scaffolds. Results concerning the use of RGB-ITR as colorimeter are presented.


---

## Body

## 1. Introduction

One of the biggest problems concerning cultural heritage environment is the study and then the reproduction of pigment color. This difficulty is mainly caused by the tendency to consider the color almost exclusively as subject of visual perception and individual sensibility. It is important to underline that the color is also a physical propriety of the object itself and not only an observer experience that can be measured [1] yet in a nontrivial way.The first work of colorimetry applied to cultural heritage field was in 1953 by Istituto Centrale per il Restauro di Roma for a preliminary study on the restoration of “Maestà” by Duccio di Boninsegna [2]. After that, 40 years elapsed before colorimetry found new applications in this field, mainly in Italy [3]. This is also the consequence of finding in commerce tested and optimized electronic instruments which are mainly developed for industrial inspection. The migration of this type of instruments, suitable for the measure of uniform colored surfaces, in cultural heritage environment is limited by the complexity and continuous variation of polychromatic surfaces. For this reason it is important to have a colorimeter, which is able to measure very small areas better if punctual.Traditional colorimeters [4] work in a very closed range, 0–30 cm, which can be acceptable for canvas or small frescos placed at few meters. For investigating large surfaces in a range of several meters, that is, painted chapels, commercial colorimeters are time-consuming systems and the use of scaffolds is necessary.For all these reasons a remote and punctual system is desirable, but a new class of problems concerning color measurement is introduced: one among all is the independence of color measurement by distance.Since several years a new optoelectronic device is developed in the Research Center ENEA of Frascati (Rome): this instrument can be considered a combination of a 3D scanner and a tristimulus colorimeter. The feature of collecting five pieces of information for each pixel, two distances, and three back-reflected signals from target, introduces the opportunity to carry out remote diagnosis of both the structure and the pigment of investigated surfaces [5].


## 2. Objectives

The main and ambitious aim of methodology and technology described in the paper is to fill the empty let by the use of commercial instruments in cultural heritage environment.The color 3D laser scanner prototype, called RGB-ITR (Red Green Blue Imaging Topological Radar), developed in ENEA laboratory and used in several national and international campaigns, tries to respond at the demand of punctual and remote analysis of polychromatic surfaces. The submillimetric spatial resolution and beam size ensure a continuous cover of the investigating surface.The color calibration methods, presented in the next paragraph, demonstrate the possibility to operate remotely for the detection and prevention of pigment modifications and have to be considered a starting point for the definition of this scanner prototype as colorimeter. In fact the feature of operating remotely in a range of 3–30 meters constrains to put in front of each detector an interferential filter whose transmittance spectra do not mimic the CIE color matching function [1]—so to guarantee the acquisition of measures free by external light sources interference. The calibration methods ensure repeatability of pigment measure during the time, that is, years, and permit to have an objective measure of polychromatic surfaces.


## 3. Methodology

The technique adopted for the simultaneous collection of distance and color information is based on amplitude modulation [6–8] of three monochromatic laser sources (680 nm, 532 nm, and 460 nm), which operate as carrier waves. The distance and color information are extracted from modulating waves by phase-shift and reflectivity responses of the target at the three wavelengths.


### 3.1. Distance Information

The main formula for distance detection is expressed by [6](1)D=vΔφ4πfm,
where fm is the modulation frequency, Δφ the phase difference between the reference modulating wave and the signal back-reflected by the target, and v the light speed in the medium. For laser optical powers such that the shot noise dominates over all other noise sources in the detection process (typically, a few nW at the output of the detection optical fiber), the accuracy σR of distance (i.e., range) measurements can be showed to increase with the modulation frequency fm according to the following formula [6]:(2)σR=v2π2mfmSNRi,
where m is the modulation depth and(3)SNRi=PητhfΓ
the current signal-to-noise ratio, which depends on the laser optical frequency f, the integration time τ, the detector’s quantum efficiency η, the merit factor Γ of the receiver, and the collected optical power P. Here h denotes the Planck constant.


### 3.2. Color Information

The color information is elaborated by the detection of the amplitude of back-reflected modulating signal from the target. The raw data are expressed in Volt and are the result of elaboration by a lock-in amplifier Stanford SR-844, directly connected to an avalanche photodiode by a low-noise amplifier. The ranges for each channel are about 0.1–100 mV, which are digitally converted by three 16-bit ADC. Raw data are normalized in a range of 0-1 by minimum/maximum measurement of the scene, if no calibration is applied.


### 3.3. Color Calibration by Distance Information

The color calibration method as a function of distance is based on the measurement of a white target, Spectralon STR-99-020, placed at different distances and illuminated by the three lasers of RGB-ITR system. Figure1 shows the three curves acquired during a measurement campaign for the 3D digitization of the S. Peter Martyr Oratory, a 15th century chapel located at Rieti (Italy): each point was collected by moving the white target of 50 cm, in a range of 2.5–21 meters. The curves are fitted by Hermite polynomials, which ensure the continuity in the working range.Single channel calibration curves, obtained striking a white target with the three laser beams of RGB-ITR system. Each curve represents the amplitude (V) of the back-reflected signal as a function of the distance, expressed as phase shift (degrees) between reference and back-reflected signals. (a) red, (b) green, and (c) blue curves.
(a)(b)(c)These curves can be considered as a sort of RGB-ITR’s fingerprint: once optical parameters are fixed, like the spot and optical receiving focalization, the color information can be univocally represented. The color correction of data acquired on investigated surface is given by the ratio between the three colors acquired by the instrument and the amplitude value on the three calibration curves selected by the corresponding distance.


### 3.4. Color Calibration by Spectrophotometer

For a standardization of color information detected by RGB-ITR system, respecting CIE standards, a further calibration was introduced.This calibration method was executed in laboratory by using fourteen color Lambertian targets: these targets were acquired by both RGB-ITR system and a MINOLTA spectrophotometer-CM-2600d.The first column on the left of Figure2 shows the RGB-ITR data collected by acquiring the fourteen targets, whereas the last column shows the same targets data collected by spectrophotometer. In the central column, instead, the RGB-ITR data after calibration method obtained by MINOLTA spectrophotometer have been displayed. The differences between two instruments are evident but, while spectrophotometer data are acquired by contact, the RGB-ITR data are collected at a distance of 10 meters.Figure 2
On the left, RGB-ITR raw data; in the center, RGB-ITR data after calibration method obtained by MINOLTA spectrophotometer; on the right, MINOLTA spectrophotometer data.The matching of two color spaces was executed by minimizing the distance between the couple of measures collected by the two instruments. The resulting transformation matrix is obtained solving the equations:(4)[sRMsGMsBMs]=[t11t12t13t14t21t22t23t24t31t32t33t34t41t42t43t44]⋅[RITRGITRBITR1],
where (RM,GM,BM) are the colors triplet in spectrophotometer space, while (RITR,GITR,BITR) are the colors triplet in RGB-ITR space.Arranging the terms and groupings, a homogeneous system of three equations with 16 unknowns fromt11 to t44 is obtained, as shown in (5)t11⋅RITR+t12⋅GITR+t13⋅BITR+t14-t41⋅RITR⋅RM-t42⋅GITR⋅RM-t43⋅BITR⋅RM-t44RM=0t21⋅RITR+t22⋅GITR+t23⋅BITR+t24-t41⋅RITR⋅GM-t42⋅GITR⋅GM-t43⋅BITR⋅GM-t44GM=0t31⋅RITR+t32⋅GITR+t33⋅BITR+t34-t41⋅RITR⋅BM-t42⋅GITR⋅BM-t43⋅BITR⋅BM-t44BM=0.It can be seen that for each point three equations and 16 unknown parameters are interested. Because the measures collected by the two instruments are affected by noise, more than necessary measures for solving the unknowns are considered. The parameter estimation problem is solved by computing the vectorθ=[t11,t12,t13,t14,…,t44]γ that minimizes some cost function of the matrix equation Λ·θ=0.A good estimation using Total Least Squares techniques can be found computing the eigenvector corresponding to the smallest eigenvalue of the matrix Λγ·Λ. The results obtained are shown in the central column of Figure 2.Because of the measure by spectrophotometer can be executed only by contact with the surface, the transformation matrix found by solving (5) has to be applied for each point collected by the method illustrated in Section 3.3. The combination of two calibrations permits to standardize, at each distance, the RGB-ITR data with the MINOLTA spectrophotometer color measures.


## 3.1. Distance Information

The main formula for distance detection is expressed by [6](1)D=vΔφ4πfm,
where fm is the modulation frequency, Δφ the phase difference between the reference modulating wave and the signal back-reflected by the target, and v the light speed in the medium. For laser optical powers such that the shot noise dominates over all other noise sources in the detection process (typically, a few nW at the output of the detection optical fiber), the accuracy σR of distance (i.e., range) measurements can be showed to increase with the modulation frequency fm according to the following formula [6]:(2)σR=v2π2mfmSNRi,
where m is the modulation depth and(3)SNRi=PητhfΓ
the current signal-to-noise ratio, which depends on the laser optical frequency f, the integration time τ, the detector’s quantum efficiency η, the merit factor Γ of the receiver, and the collected optical power P. Here h denotes the Planck constant.


## 3.2. Color Information

The color information is elaborated by the detection of the amplitude of back-reflected modulating signal from the target. The raw data are expressed in Volt and are the result of elaboration by a lock-in amplifier Stanford SR-844, directly connected to an avalanche photodiode by a low-noise amplifier. The ranges for each channel are about 0.1–100 mV, which are digitally converted by three 16-bit ADC. Raw data are normalized in a range of 0-1 by minimum/maximum measurement of the scene, if no calibration is applied.


## 3.3. Color Calibration by Distance Information

The color calibration method as a function of distance is based on the measurement of a white target, Spectralon STR-99-020, placed at different distances and illuminated by the three lasers of RGB-ITR system. Figure1 shows the three curves acquired during a measurement campaign for the 3D digitization of the S. Peter Martyr Oratory, a 15th century chapel located at Rieti (Italy): each point was collected by moving the white target of 50 cm, in a range of 2.5–21 meters. The curves are fitted by Hermite polynomials, which ensure the continuity in the working range.Single channel calibration curves, obtained striking a white target with the three laser beams of RGB-ITR system. Each curve represents the amplitude (V) of the back-reflected signal as a function of the distance, expressed as phase shift (degrees) between reference and back-reflected signals. (a) red, (b) green, and (c) blue curves.
(a)(b)(c)These curves can be considered as a sort of RGB-ITR’s fingerprint: once optical parameters are fixed, like the spot and optical receiving focalization, the color information can be univocally represented. The color correction of data acquired on investigated surface is given by the ratio between the three colors acquired by the instrument and the amplitude value on the three calibration curves selected by the corresponding distance.


## 3.4. Color Calibration by Spectrophotometer

For a standardization of color information detected by RGB-ITR system, respecting CIE standards, a further calibration was introduced.This calibration method was executed in laboratory by using fourteen color Lambertian targets: these targets were acquired by both RGB-ITR system and a MINOLTA spectrophotometer-CM-2600d.The first column on the left of Figure2 shows the RGB-ITR data collected by acquiring the fourteen targets, whereas the last column shows the same targets data collected by spectrophotometer. In the central column, instead, the RGB-ITR data after calibration method obtained by MINOLTA spectrophotometer have been displayed. The differences between two instruments are evident but, while spectrophotometer data are acquired by contact, the RGB-ITR data are collected at a distance of 10 meters.Figure 2
On the left, RGB-ITR raw data; in the center, RGB-ITR data after calibration method obtained by MINOLTA spectrophotometer; on the right, MINOLTA spectrophotometer data.The matching of two color spaces was executed by minimizing the distance between the couple of measures collected by the two instruments. The resulting transformation matrix is obtained solving the equations:(4)[sRMsGMsBMs]=[t11t12t13t14t21t22t23t24t31t32t33t34t41t42t43t44]⋅[RITRGITRBITR1],
where (RM,GM,BM) are the colors triplet in spectrophotometer space, while (RITR,GITR,BITR) are the colors triplet in RGB-ITR space.Arranging the terms and groupings, a homogeneous system of three equations with 16 unknowns fromt11 to t44 is obtained, as shown in (5)t11⋅RITR+t12⋅GITR+t13⋅BITR+t14-t41⋅RITR⋅RM-t42⋅GITR⋅RM-t43⋅BITR⋅RM-t44RM=0t21⋅RITR+t22⋅GITR+t23⋅BITR+t24-t41⋅RITR⋅GM-t42⋅GITR⋅GM-t43⋅BITR⋅GM-t44GM=0t31⋅RITR+t32⋅GITR+t33⋅BITR+t34-t41⋅RITR⋅BM-t42⋅GITR⋅BM-t43⋅BITR⋅BM-t44BM=0.It can be seen that for each point three equations and 16 unknown parameters are interested. Because the measures collected by the two instruments are affected by noise, more than necessary measures for solving the unknowns are considered. The parameter estimation problem is solved by computing the vectorθ=[t11,t12,t13,t14,…,t44]γ that minimizes some cost function of the matrix equation Λ·θ=0.A good estimation using Total Least Squares techniques can be found computing the eigenvector corresponding to the smallest eigenvalue of the matrix Λγ·Λ. The results obtained are shown in the central column of Figure 2.Because of the measure by spectrophotometer can be executed only by contact with the surface, the transformation matrix found by solving (5) has to be applied for each point collected by the method illustrated in Section 3.3. The combination of two calibrations permits to standardize, at each distance, the RGB-ITR data with the MINOLTA spectrophotometer color measures.


## 4. Technology Description

The RGB-ITR is the last offspring of a series of amplitude-modulated (AM), monochromatic 3D laser scanners realized at the ENEA Artificial Vision laboratory (Frascati, Rome), and collectively identified by the acronym ITR (Imaging Topological Radar) [8]. The main new feature improving the last model is the use of three AM laser sources for remote colorimetry.


### 4.1. ITR’s Characteristics and Operating Principle

The RGB-ITR scanner [5, 9] is essentially composed by two main modules: a so-called passive module, which coincides with the system’s optical head, basically including the launching and receiving optics; a so-called active module, which is composed of laser sources, modulators, detectors, and all necessary electronics for collecting and processing data. The two modules are physically separated and optically connected by means of optimized optical fibers. This enables the use of the system in hardly accessible or even hostile environments [10]. Figure 3 shows the entire system assembled as a tower so to guarantee the minimum size and movement facility; the modular configuration permits to adapt the system in the operating environment.Figure 3
RGB-ITR modules assembled all together on a mobile platform. From bottom, the first two boxes are composed by lock-in amplifiers, for laser modulation and signals detection, and a motion controller of the scanning motors; inside the next box, laser sources and detectors; on the top, the optical head with the scanning mirror.One of the most critical parts in this type of instruments is the movement of the scanning system [11], which a lot of times coincides with the movement of the motorized mirror responsible of the laser beam sweep on the target. The way that the motors are assembled on the scanning system defines also the uniformity of the resolution and the possible presence of poles in the resulting images on the entire 3D space.At the moment RGB-ITR scanning system is composed by a mirror mounted on a motor, parallel to the base of the optical head and anchored at a metallic box: the rotational axis is perpendicular at the base of the optical head (horizontal movement). The metallic box can rotate around an axis parallel at the base of the optical head (vertical movement).Due to construction limits, the actual angle of view of RGB-ITR is 80° × 310°, with a point-to-point precision of 0.002°. This resolution permits to obtain a structured and dense point cloud in the working range of the scanner.


### 4.2. ITR Software Modules

All software is assembled and customized in ENEA and it is composed by two main interfaces: the first one, called ScanSystem, controls all the aspects of scanning and data acquisition; the second one, called ITRAnalyzer, is able to build, analyze, and export all ITR data.ScanSystem permits to change a lot of ITR scanner’s parameters, so to guarantee the best solution for the acquisition of the scene under investigation. In Figure4(a) it is possible to observe the dialog interface: the “Motion Controller” section allows to move manually the scanning mirror so to determine the acquisition range, settable in “Scan Parameters” module; the “Acquisition” section is useful for setting the modulation frequency, gain filters, and sampling frequency of the three channels; the acquisition starts pressing the button “Start Scan.” Another interesting feature of ScanSystem resides in SCRIPT programming that permits to divide the acquisition process in several subscans, so achieving a scanning-parameters customization based on scene complexity. In the last period the color calibration section was added at the software.(a) User interface of ScanSystem software module; (b) user interface of ITRAnalyzer software module.
(a)(b)ITRAnalyzer manages all data collected by RGB-ITR system, display color, and structure profiles of the investigated target and permits the registration of meshes obtained by different point-of-view scans. This software can be considered the link between RGB-ITR raw data and commercial software for 3D modeling and photo manipulation. In Figure4(b) it is possible to observe the user interface: the “Building Settings” section allows to set the typology of algorithms to activate before loading the acquired data, like the color calibration versus distance method; the “2D View” panel allows some previews of each channel information and global colored texture; the “3D View” section allows some previews of entire or partial 3D mesh and versors; the last section, “Data Analysis,” is useful for a preliminary on site data analysis of both RGB-ITR data, color, and distance information.


## 4.1. ITR’s Characteristics and Operating Principle

The RGB-ITR scanner [5, 9] is essentially composed by two main modules: a so-called passive module, which coincides with the system’s optical head, basically including the launching and receiving optics; a so-called active module, which is composed of laser sources, modulators, detectors, and all necessary electronics for collecting and processing data. The two modules are physically separated and optically connected by means of optimized optical fibers. This enables the use of the system in hardly accessible or even hostile environments [10]. Figure 3 shows the entire system assembled as a tower so to guarantee the minimum size and movement facility; the modular configuration permits to adapt the system in the operating environment.Figure 3
RGB-ITR modules assembled all together on a mobile platform. From bottom, the first two boxes are composed by lock-in amplifiers, for laser modulation and signals detection, and a motion controller of the scanning motors; inside the next box, laser sources and detectors; on the top, the optical head with the scanning mirror.One of the most critical parts in this type of instruments is the movement of the scanning system [11], which a lot of times coincides with the movement of the motorized mirror responsible of the laser beam sweep on the target. The way that the motors are assembled on the scanning system defines also the uniformity of the resolution and the possible presence of poles in the resulting images on the entire 3D space.At the moment RGB-ITR scanning system is composed by a mirror mounted on a motor, parallel to the base of the optical head and anchored at a metallic box: the rotational axis is perpendicular at the base of the optical head (horizontal movement). The metallic box can rotate around an axis parallel at the base of the optical head (vertical movement).Due to construction limits, the actual angle of view of RGB-ITR is 80° × 310°, with a point-to-point precision of 0.002°. This resolution permits to obtain a structured and dense point cloud in the working range of the scanner.


## 4.2. ITR Software Modules

All software is assembled and customized in ENEA and it is composed by two main interfaces: the first one, called ScanSystem, controls all the aspects of scanning and data acquisition; the second one, called ITRAnalyzer, is able to build, analyze, and export all ITR data.ScanSystem permits to change a lot of ITR scanner’s parameters, so to guarantee the best solution for the acquisition of the scene under investigation. In Figure4(a) it is possible to observe the dialog interface: the “Motion Controller” section allows to move manually the scanning mirror so to determine the acquisition range, settable in “Scan Parameters” module; the “Acquisition” section is useful for setting the modulation frequency, gain filters, and sampling frequency of the three channels; the acquisition starts pressing the button “Start Scan.” Another interesting feature of ScanSystem resides in SCRIPT programming that permits to divide the acquisition process in several subscans, so achieving a scanning-parameters customization based on scene complexity. In the last period the color calibration section was added at the software.(a) User interface of ScanSystem software module; (b) user interface of ITRAnalyzer software module.
(a)(b)ITRAnalyzer manages all data collected by RGB-ITR system, display color, and structure profiles of the investigated target and permits the registration of meshes obtained by different point-of-view scans. This software can be considered the link between RGB-ITR raw data and commercial software for 3D modeling and photo manipulation. In Figure4(b) it is possible to observe the user interface: the “Building Settings” section allows to set the typology of algorithms to activate before loading the acquired data, like the color calibration versus distance method; the “2D View” panel allows some previews of each channel information and global colored texture; the “3D View” section allows some previews of entire or partial 3D mesh and versors; the last section, “Data Analysis,” is useful for a preliminary on site data analysis of both RGB-ITR data, color, and distance information.


## 5. Results

In this section some of the results obtained by RGB-ITR system during a few recent field campaigns are presented. Figure5 shows a scan executed in the church of S. Peter Martyr (Rieti): on the left, raw data obtained directly by the scanner are shown; on the right, color data after the calibration with the methods described above are displayed.(a) RGB-ITR raw data; (b) RGB-ITR data after color calibration.
(a)(b)During this campaign the entire vault was acquired. In Figure6 a screenshot of the entire 3D model and a summary of acquisition main parameters are shown.Figure 6
On the left, the entire vault acquired in the S. Peter Martyr church; on the right, a conclusive table of main parameters of the acquisition.Figure7 shows a detail of a scanning in progress of the ceiling of “Loggia di Amore e Psiche” in Villa Farnesina (Rome). The data shown in this figure are not calibrated by the normalization curves.Figure 7
A detail of a scanning in progress of “Loggia di Amore e Psiche” in Villa Farnesina (Rome).


## 6. Business Benefits

The real economic benefit is the sum of several factors, which have an impact in different sectors of cultural heritage environment: the characteristics such as remote diagnosis of both color and structure information, without the use of scaffolds, noninvasiveness, and versatility offer a competitive and cheap solution for a support to restoration and cataloguing. Moreover, the possibility of collecting simultaneous color and distance information decreases drastically time and costs of elaborated data after production and mesh creation, which can also be used for educational purposes.


## 7. Conclusions

The use of RGB-ITR system for structural and colorimetric remote diagnosis and 3D digitalization was shown. The calibration methods for color acquisition have to be considered a starting point for colorimetric measurements compatible with CIE standards.In summary the use of three AM laser stimuli and narrow interferential filters permits to concurrently acquire five pieces of information (two distances and three-reflectivity signals back-reflected by target, one for each incoming chromatic laser component) per each sampled point by means of three avalanche photodiodes and dedicated electronics and to perform a discrete color analysis that can be considered a three Delta-function analysis.The use of supercontinuum lasers [12] is underinvestigation, so to provide a color analysis in the entire visible spectrum.A new version of ITR system is going to be developed: the main new features are the upgrade of the electronic system, more compact and efficient, and the insertion of new laser sources for infrared and fluorescence analysis.



---

*Source: 512902-2012-04-03.xml*