IOP PUBLISHING

MEASUREMENT SCIENCE AND TECHNOLOGY

Meas. Sci. Technol. 24 (2013) 105401 (7pp)

doi:10.1088/0957-0233/24/10/105401

RGB representation of two-dimensional
multi-spectral acoustic data for object
surface profile imaging
Xinhua Guo, Yuji Wada, Yosuke Mizuno and Kentaro Nakamura
Precision and Intelligence Laboratory, Tokyo Institute of Technology, 4259 Nagatsuta-cho, Midori-ku,
Yokohama 226-8503, Japan
E-mail: guoxinhua@sonic.pi.titech.ac.jp

Received 9 April 2013, in final form 12 July 2013
Published 14 August 2013
Online at stacks.iop.org/MST/24/105401
Abstract
Conventionally, acoustic imaging has been performed using a single frequency or a limited
number of frequencies. However, the rich information on surface profiles, structures hidden
under surfaces and material properties of objects may exhibit frequency dependence. In thisstudy, acoustic imaging on object surface was conducted over a wide frequency range with afine frequency step, and a method for displaying the acquired multi-spectral acoustic data wasproposed. A complicated rigid surface with different profiles was illuminated by sound waves
sweeping over the frequency range from 1 to 20 kHz with a 30 Hz step. The reflected soundwas two-dimensionally recorded using a scanning microphone, and processed using aholographic reconstruction method. The two-dimensional distributions of obtained soundpressure at each frequency were defined as ‘multi-spectral acoustic imaging data’. Next, themulti-spectral acoustic data were transformed into a single RGB-based picture for easy
understanding of the surface characteristics. The acoustic frequencies were allocated to red,green and blue using the RGB filter technique. The depths of the grooves were identified bytheir colours in the RGB image.
Keywords: frequency dependence, multi-spectral acoustic imaging, acoustic holography,

RGB representation(Some figures may appear in colour only in the online journal)

1. Introduction

responses, because frequency-dependent phenomena, such as
resonance, diffraction and attenuation, are essentially related
to the properties of the objects. On the other hand, use of
the optical multi-spectral camera [13, 14], which utilizes
spectral information to obtain the detailed characteristics ofthe objects, has spread recently in research studies on remotesensing and medical tissue diagnostics. As an analogy to multispectral imaging in optics, a few studies on the improvementof the quality of images in acoustics with a limited numberof frequencies have been reported [15, 16]. For example, formedical purposes, interference noise (speckle) is eliminatedby taking average over several images recorded at different
frequencies. However, the frequency coverage and resolution
in the previous studies are definitely insufficient for the
analysis of the resonance characteristics originating from theAs sound waves travel with little attenuation in solids andfluids in which electromagnetic waves do not propagate,
acoustic imaging techniques have been applied to many
fields including underwater sound navigation and ranging(SONAR) [1–5], medical diagnosis [6–8] and nondestructive
evaluation [9–12]. In general, the target object is illuminatedby acoustic waves, and then the reflected or transmitted waves
are recorded. Finally, the object is visualized by appropriate
signal processing.
Acoustic images provide information different from thatof optical images, and are significantly influenced by the elastic
properties, surface structures, shapes and sizes of the targets
to be imaged. These features are reflected in the frequency
0957-0233/13/105401+07$33.00

1© 2013 IOP Publishing Ltd

Printed in the UK & the USA

Meas. Sci. Technol. 24 (2013) 105401

X Guo et al

2.2. Imaging at every frequency
To visualize the characteristics of object surfaces, planar
acoustic holography [17, 18] is applied to the measured
sound pressure data. Let us briefly review the procedure
to clarify the processing method we employ. The pressure
distribution on the surface is derived from the Rayleigh integral
equation [19]
 ∞ ∞pr ( f , xr , yr , zr ) =ph ( f , x, y, zh )−∞−∞×gD ( f , xr − x, yr − y, zr − zh ) dxh dyh ,where pr ( f , xr , yr , zr ) is the multi-spectral complex soundpressure at a point (xr , yr , zr ) on the reconstruction plane,ph ( f , x, y, zh ) is the multi-spectral complex sound pressure ata point (x, y, zh ) on the hologram plane and gD is the Dirichlet
Green’s function that acts as the transfer function of the soundpressure field from one point to another. The expression of
two-dimensional Fourier transforms [19] is employed to treat
the sound field in the k-space: ∞ ∞F̃ (kx , ky ) =f (x, y) e−i(kx x+ky y) dx dy

Figure 1. Reconstruction procedure of acoustic holography.

surface profile, because the response changes drastically withthe frequency and the higher modes sometimes provide further
detailed information.
In this study, to demonstrate the effectiveness ofmulti-spectral acoustic imaging (MSAI) which utilizes a wide
frequency band with a fine frequency step, a complicatedsample surface is illuminated by frequency-swept sound from
1 to 20 kHz with a 30 Hz step. We obtain 635 pieces of
two-dimensional distributed acoustic images, and each image
corresponds to a single frequency. Next, we propose an RGB
representation of the measured results as one of the methods
for presenting the MSAI data. The acoustic frequencies areallocated to three colours, red (R), green (G) and blue (B), and635 MSAI images are fused into one picture.

−∞−∞≡ F [ f (x, y)],(4)where (kx , ky ) refers to the k-space, and (x, y) to the real space.
Since (3) is the expression in a two-dimensional convolution
integral of ph ( f , x, y, zh ) and gD ( f , xr − x, yr − y, zr − zh ),using the convolution theorem [20, 21], (3) yields
pr ( f , xr , yr , zr ) = F − [P̃r (k, kx , ky , zr )]2. Multi-spectral acoustic imaging processing= F − [P̃h (k, kx , ky , zh )G̃D (k, kx , ky , zr − zh )],2.1. Measurement(5)

Multi-spectral acoustic imaging (MSAI) is carried out by
measuring the complex sound pressure (amplitude and phase)at each frequency from 1 to 20 kHz with a 30 Hz step. The
imaging processing technique based on acoustic holography
is used as shown in figure 1. Although impulse response isan alternative method for taking the frequency characteristics,
the frequency-swept method is employed in our experiment toenhance the signal-to-noise ratio.
Assuming that the original amplitude and phase data arerepresented by a( f , x, y, zh ) and θ ( f , x, y, zh ), respectively,
where f is the frequency, x and y are the spatial coordinates, andzh is the constant value that presents the distance between theobject surface and the measurement plane, the multi-spectral
complex sound pressure is expressed as
p0 ( f , x, y, zh ) = a( f , x, y, zh ) ejθ ( f ,x,y,zh ) .

where F − indicates the inverse Fourier transform. This gives
the holographic reconstruction of the three-dimensional spacepr ( f , xr , yr , zr ) in terms of the Fourier transformed formP̃h (k, kx , ky , zh ) derived from the hologram data ph ( f , x, y, zh ).
The operator G̃D (k, kx , ky , zr − zh ) is the multi-spectral andtwo-dimensional spatial Fourier transform of the Dirichlet
Green’s function.
2.3. RGB representation of the MSAI data
RGB-based colour image, where different colours represent
different acoustic frequencies, is a good representation toindicate frequency characteristics at a glance. Three sets ofsignal data are prepared by using the RGB windows as shownin figure 2. The complex sound pressure ph is processed
through the RGB windows as(1)

To reduce the influence of frequency characteristics ofillumination, reflected signals from the surface are calibrated
by the reference signal pc ( f ). The reference signal pc ( f )is generated as the average of the several selected datap0 ( f , xi , y j , zh ), where i and j indicate the data positions.
The original data contain the phase circulation due to theillumination chirp signal. Before proceeding to the next
operation, the phase is compensated using the reference databy the following equation:ph ( f , x, y, zh ) = p0 ( f , x, y, zh ) e−j arg[pc ( f )] .

(3)

p1 (x, y, zh , i) =nw ( fm , i) ph ( fm , x, y, zh ),m=1

i = [Red, Green, Blue] ,(6)where p1 (x, y, zh , i) is a vector that represents the RGB
components of one pixel at (x, y, zh ), and w( fm , i) represents
the RGB windows, f 1 is the lowest frequency, and fn is thehighest frequency for the measurement. In our experiment,
f 1 = 1 kHz, fn = 20 kHz, (fn − f 1)/n = 30 Hz, and n = 635.

(2)
2

Meas. Sci. Technol. 24 (2013) 105401

X Guo et al

To explore high contrast images, the operation of rotating
the phase by parameter ϕ is applied according to the following
procedure:p4 (x, y, zh , i) = p2 (x, y, zh , i) − 1,(10)

p4 (x, y, zh , i)p5 (x, y, zh , i) =.
(11)
3 |σ (p4 , i)|
The positive real value of the signal p5 (x, y, zh , i) with phaseshift of ϕ is transformed to the unsigned 8-bit integer for
display by the following equations:p6 (x, y, zh , i, ϕ) = uint8{255Re[p5 (x, y, zh , i) ejϕ ]},⎧⎨255
uint8 (x) = 0⎩
round (x)Figure 2. RGB windows.

The trapezoidal windows are given in hue-saturation-value(HSV) [22] colour expression as⎧f − fl (i)⎪⎪fc (i) − fl (i)  f < fc (i) − fl (i)/2⎪⎪
fl (i)/2⎪⎪⎪⎨
1
fc (i) − fl (i)/2  f < fc (i) + fr (i)/2,
w( f , i) =fr (i) − f⎪⎪⎪fc (i) + fr (i)/2  f < fc (i) + fr (i)⎪⎪f r(i)/2⎪⎪⎩0otherwise.
(7)

p1 (x, y, zh , i).
pc (i)

3.1. Sample
A sample for demonstration was a rigid block having a planesurface of 100 × 100 mm2 made of polyvinylchloride (PVC).
The surface was engraved with five different letters ‘T’, ‘I’,
‘T’, ‘U’ and ‘O’ as shown in figure 3(a). Here, the upper
left ‘T’ was defined as ‘T1’, and the upper right ‘T’ as ‘T2’.
The five letters formed cavities that were characterized bydifferent geometrical features: all of them had the same widthof 6 mm, and differed in depth. The letters ‘T1’, ‘I’, ‘T2’, ‘U’and ‘O’ had depths of 6, 9, 13, 18 and 22 mm, respectively.
The other dimensions are indicated in figure 3(a). When thecavities were excited by incident waves, every letter exhibited
its own resonance frequencies since they had different depths
and shapes.

(8)
3.2. Experimental set-up and data acquisition

Then, the central value is adjusted as the pixel data,
p3 (x, y, zh , i) =128|p2 (x, y, zh , i) | − 128+ 128,
3 |σ (p2 , i)|(13)

3. Experiments

where fc (i), fl (i) and fr (i) are the central, the left and theright corner frequencies of the band, respectively. In thisstudy, the trapezoidal windows with the central frequenciesof (R, G, B) = (2, 5, 8) kHz are used.
First, the data p1 (x, y, zh , i) are normalized by thereference value pc (i) as
p2 (x, y, zh , i) =(x > 255)(x < 0)otherwise.

(12)

The experimental set-up is shown in figure 3(b). A loudspeaker(Fostex, T90A) and a needle microphone (B&K, 4182) were
mounted on the X–Y motorized stage, and moved along thehologram plane parallel to the sample surface. The distancefrom the microphone to the sample surface was 2 mm, and(9)where σ is the standard deviation. P3 (x, y, zh , i) is themagnitude of the RGB representation.

(a)

(b)

Figure 3. (a) Sample with the detailed dimensions (optical image), and (b) experimental set-up.
3

Meas. Sci. Technol. 24 (2013) 105401(a)

X Guo et al

from the loudspeaker to the sample surface was 150 mm.
The incident angle was 15 degrees. The continuous soundwave (linear chirp signal) was radiated onto the object surface.
The reflected sound was detected using the needle microphonethat had smooth frequency response characteristic between
1 Hz and 20 kHz. At every microphone position (x, y), theloudspeaker transmitted a frequency-swept sound from 1 to
20 kHz with a step of 30 Hz. Using a frequency response
analyzer (FRA, NF5097), the received data of the amplitude
and phase at each frequency was transferred to a personal
computer (PC) for further processing. The signal source
driving the loudspeaker was generated by the same FRA,
and was used as the reference signal for determining thephase of the sound pressure. The X–Y scanning points were
40 × 40, and the grid spacing was 2.5 mm. The time requiredfor data acquisition was 12 h for all the measurement andpreprocessing.
After the measurement was completed, acquired data werearranged into two-dimensional distributions of the amplitude
and phase at each frequency as shown in figure 4. We had
acoustic holograms at every 30 Hz from 1 to 20 kHz, thus
the total number of obtained holograms was 635. These wererecognized as the MSAI data. In this example, six points inthe data (x, y) = (15, 10), (25, 10), (10, 20), (20, 20), (30, 20),
(20, 30) were selected for the calibration stated in section 2.1.

(b)

Figure 4. (a) Amplitude and (b) phase distributions (MSAI data).

Figure 5. Frequency responses for five letters.

(a)

(b)

(c)

(d)

(e)

(f)

(g)

(h)

Figure 6. (a)–(h) Reconstructed images at different frequencies of the object surface.
4Meas. Sci. Technol. 24 (2013) 105401

X Guo et al

(a)

(b)

(c)

(d)

(e)

(f)

(g)

(h)

(i)

(j)

Figure 7. (a) Amplitude, (b)–(i) RGB representations with rotating phases ϕ = 0, π /4, π /2, 3π /4, π , 5π /4, 3π /2 and 7π /4, respectively,and ( j) colour bar.

5

Meas. Sci. Technol. 24 (2013) 105401

X Guo et al

4. Results

Since the frequency responses varied by the position along
one letter, mean frequency responses of each letter were used
for simplicity. The mean values were calculated from themeasured points located along the letters. For focusing on
the resonance of depth mode, the characteristics of the meanfrequency responses for each letter are shown in figure 5. The
fundamental, second and third harmonics of the letter ‘O’ wereobserved, while the fundamental and second harmonics of the‘U’ and ‘T2’, and the fundamental harmonic of the letters ‘I’and ‘T1’ were found.
According to the theory for a one-dimensional closed end
tube, resonating at the odd harmonics, i.e. λ/4, 3 λ/4, 5 λ/4,
7 λ/4, . . . , and n · λ/4 (λ represents the sound wavelength,
and n denotes the odd integers), the resonant frequencies forthe different depths of the letters (TITUO) were calculated to
be 14.2, 9.4, 6.5, 4.7 and 3.9 kHz, corresponding to the depths
of 6, 9, 13, 18 and 22 mm, respectively. The measured resonant
frequencies were 15.1, 9.1, 6.3, 4.5 and 3.5 kHz as shown infigure 5. Although there were errors in the results, the depthsof the letters were roughly identified by the mean frequency tosome extent. For more precise estimation, three-dimensional
discussion about the acoustic resonance of cavity is required.

were fused into one RGB image. The amplitude result isshown in figure 7(a). The five letters were clearly observed
and had different frequency responses displayed by different
colours. To explore higher contrast of the images, RGB images
with various phases, ϕ = 0, π /4, π /2, 3π /4, π , 5π /4,
3π /2 and 7π /4, were created as shown in figures 7(b)–(i).
The case for ϕ = π /2 showed the high contrast image asshown in figure 7(d). The frequency characteristics of eachletter were successfully observed in one picture using the
RGB representation. Figure 7( j) shows the colour bar andthe corresponding acoustic frequency. The wavelengths for
1/4- and 3/4-wavelength resonances of a one-dimensional
single-end tube are also displayed in the figure. Based on
the colour bar corresponding to the frequency band, weroughly estimated which grooves were the deepest and theshallowest. The grooves of the letters ‘T1’, ‘I’, ‘T2’, ‘U’ and ‘O’corresponded to blue, deep sky-blue, yellow, coral and fuchsia,
respectively, which meant that the groove of the letter ‘T1’ was
the shallowest one with the highest resonant frequency, andthe groove of the letter ‘O’ was the deepest. In addition, it wasobserved that the distribution of the colour in each letter was
not even. This was probably affected by the horizontal acoustic
mode in the grooves. The letter ‘O’ had the second harmonic
at 8–10 kHz, thus the letter ‘O’ was displayed by two mixed
colours.

4.2. Imaging at each frequency

5. Conclusions

All 635 holograms from 1 to 20 kHz with a 30 Hz step werereconstructed by the form of the Rayleigh integral expressedin (5). Eight reconstructed images of the magnitude of soundpressure were picked up from the 635 images and are shownin figures 6(a)–(h). With increasing frequency, the five letters‘TITUO’ engraved in the surface showed different responses.
The deepest letter ‘O’ responded at the lowest frequency as
shown in figure 6(a), and the second and the higher harmonics
were observed in figures 6(e), (g) and (h), respectively. The
letter ‘U’ was observed at the fundamental resonant frequency
in figure 6(b) and the second harmonic in figures 6(e) and( f ). ‘T2’ was seen at the fundamental resonant frequency infigure 6(c) and the second harmonic in figures 6(g) and (h).
The letters ‘I’ and ‘T1’ were observed in figures 6(d) and( f ), respectively. Although it was difficult to discriminate‘T1’ from ‘T2’ using the optical images, they were easily
recognized by the MSAI technique. The letters ‘T1’ and ‘T2’
were not observed at the same frequencies of 12 940, 13 780
and 19 510 Hz in figures 6(e), ( f ) and (h). However, both ‘T1’and ‘T2’ were observed at the same frequency of 18 970 Hz
as shown in figure 6(g). In addition, the responses of the four
letters were observed at the frequency of 12 940 Hz except for
the letter ‘T2’ as shown in figure 6(e). These results indicatedthat the surface profiles of the object were seen by their own
acoustic spectral characteristics.

The effectiveness of the MSAI was demonstrated using a wide
frequency band from 1–20 kHz with a fine frequency step of
30 Hz. To understand the surface profile using one picture,
the RGB representation was proposed. The images of letters‘TITUO’ engraved on a rigid surface were observed at eachfrequency by acoustic holography. The different structures ofthe surface were distinguished by the characteristics of theiracoustic frequency responses. It was verified that the multispectral imaging was also a powerful tool in acoustics, andgave us different information from optical imaging.
The typical trapezoidal RGB windows were employedin this study for the RGB representation. Different designsof the RGB windows may result in different images. The
central frequencies of the RGB windows should be selectedaccording to the applications. We should note that, in the RGB
representation, only a part of the rich information involved in
the MSAI data is displayed, and the rest of the information is
lost.

4.1. Basic frequency responses

References[1] Wallace R H, Hillery H V, Barnard G R, Marks B M
and McKinney C M 1975 Experimental investigation ofseveral passive sonar targets J. Acoust. Soc. Am. 57 862–9[2] Capéran T, Hayward G and Chapman R 1999 A 3D simulator
for the design and evaluation of sonar system
instrumentation Meas. Sci. Technol. 10 1116–26[3] Iwata T, Goto Y and Susaki H 2001 Application of themultiple signal classification (MUSIC) method for
one-pulse burst-echo Doppler sonar data Meas. Sci.
Technol. 12 2178–84

4.3. RGB representation
Based on the transformation using the RGB filters, the MSAI
data, all the frequency characteristics of the surface profiles,
6

Meas. Sci. Technol. 24 (2013) 105401

X Guo et al[4] Hickman G and Krolik J L 2004 Matched-field depth
estimation for active sonar J. Acoust. Soc. Am. 115 620–9[5] Nakahira K, Kodama T, Furuhashi T and Okuma S 2004 A
self-adapting sonar ranging system based on digital polarity
correlators Meas. Sci. Technol. 15 347–52[6] Chivers R C and Parry R J 1978 Ultrasonic velocity andattenuation in mammalian tissues J. Acoust. Soc. Am.
63 940–53[7] Nock L, Trahey G E and Smith S W 1989 Phase aberration
correction in medical ultrasound using speckle brightness asa quality factor J. Acoust. Soc. Am. 85 1819–33[8] Hinkelman L M, Liu D L, Waag R C, Zhu Q
and Steinberg B D 1995 Measurement and correction ofultrasonic pulse distortion produced by the human breast
J. Acoust. Soc. Am. 97 1958–69[9] Chen E P 1977 Impact response of a layered composite
containing a crack J. Acoust. Soc. Am. 61 727–30[10] Han X, Favro L D and Thomas R L 2011 Sonic IR imaging ofdelaminations and disbonds in composites J. Phys. D: Appl.
Phys. 44 034013[11] Bowler N 2011 Four-point potential drop measurements
for materials characterization Meas. Sci. Technol.
22 012001[12] Mitri F G, Kinnick R R, Greenleaf J F and Fatemi M 2009
Continuous-wave ultrasound reflectometry for surface
roughness imaging applications Ultrasonics 49 10–4
[13] Sakanoi T, Okano S, Obuchi Y, Kobayashi T, Ejiri M,
Asamura K and Hirahara M 2003 Development of themulti-spectral auroral camera onboard the index satellite
Adv. Space Res. 32 379–84[14] Basiri A, Nabili M, Mathews S, Libin A, Groah S,
Noordmans H J and Ramella-Roman J C 2010 Use of amulti-spectral camera in the characterization of skinwounds Opt. Express 18 3244–57[15] Yoshizumi N, Saito S, Koyama D, Nakamura K, Ohya A
and Akiyama I 2009 Multiple-frequency ultrasonic imaging
by transmitting pulsed waves of two frequencies J. Med.
Ultrason. 36 53–60[16] Akiyama I, Yoshizumi N, Saito S, Wada Y, Koyama D
and Nakamura K 2012 Development of multiple-frequency
ultrasonic imaging system using multiple resonancepiezoelectric transducer Japan. J. Appl. Phys. 51 07GF02[17] Maynard J D, Williams E G and Lee Y 1985 Nearfield
acoustical holography: I. Theory of generalized holography
and the development of NAH J. Acoust. Soc. Am.
78 1395–413[18] Veronesi W A and Maynard J D 1987 Nearfield acoustic
holography (NAH) II. Holographic reconstruction
algorithms and computer implementation J. Acoust. Soc.
Am. 81 1307–22[19] Williams E G 1999 Fourier Acoustics—Sound Radiation and
Nearfield Acoustical Holography (San Diego, CA:
Academic) pp 31–5
[20] Goodman J W 1968 Introduction to Fourier Optics (NewYork: McGraw-Hill) p 10
[21] Williams E G 1999 Fourier Acoustics—Sound Radiation and
Nearfield Acoustical Holography (San Diego, CA:
Academic) p 3[22] Smith A R 1978 Color gamut transform pairs Comput. Graph.
12 12–97