# Space-frequency analysis of pseudo-periodic patterns for subpixel position control

P. Sandoz, J. Galeano Zea

► To cite this version:

P. Sandoz, J. Galeano Zea. Space-frequency analysis of pseudo-periodic patterns for subpixel position control. International Symposium on Optomechatronic Technologies, Sep 2009, Istanbul, Turkey. pp.16, 10.1109/ISOT.2009.5326092. [hal-00472307](https://hal.science/hal-00472307v1)

HAL Id: hal-00472307

https://hal.science/hal-00472307v1

Submitted on 26 Apr 2021

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# Space-frequency analysis of pseudo-periodic patterns for subpixel position control

Patrick Sandoz and July A. Galeano Zea

Institut FEMTO-ST, Département d'Optique PM Duffieux,

UMR CNRS Université de Franche-Comté 6174 F-25030 Besançon cedex, France

Email: patrick.sandoz@univ-fcomte.fr and july.galeano_zea@univ-fcomte.fr

Telephone: +33 (0) 381 666 420 - Fax: +33 (0) 381 666 423

*Abstract*—This paper reports on visual position measurement based on space-frequency analysis of a pseudo-periodic pattern. This approach can be seen as a two-dimensional extension of the Vernier principle as used in the Vernier caliper. The pseudo-periodic pattern fixed on the target is seen as a secondary scale that is compared to the reference scale formed by the image pixel frame. By performing space-frequency analysis and phase computations, the center of the pseudo-periodic pattern is retrieved with a subpixel accuracy and the in-plane orientation is determined as well. Several configurations allow different kinds of measurement. A single camera vision system leads to in-plane pose estimation. Stroboscopic illumination can be used to quantify vibration amplitudes down to the nanometer range. A two camera setup can be used for displacement measurements along the three spatial directions while the choice of an interference objective, sensitive to out-of-plane direction, can complement the measurement to address the six degrees of freedom. An extended pseudo-periodic pattern was also designed to locate any field of observation within a wide dimension surface.

## I. INTRODUCTION AND PRINCIPLE

Vision is a convenient way to measure position or displacement with a high resolution without contact. However image sequences generate a huge amount of data that induce significant processing times. If the position data has to be used in a feedback loop, the shorter the processing time the wider the control bandwidth. The response time is thus an important criterion to consider in determining the right way to measure position. While video rate can be achieved in some configurations, a short response time is not set as a priority in our approach. We investigated how a well-chosen feature fixed on the target can be used for achieving high resolution with an usual vision system. In the case of pseudo-periodic patterns, the spatial phase was fund to be very powerful for performing pixel interpolation[1], [2]. The working principle can be compared with the Vernier scale widely known through the Vernier caliper as represented in Fig.1.a. The Vernier caliper achieves an interpolation between units of the main scale from the position where the two different scales match. In this case, the period of the secondary scale with respect to that of the main one's determines the degree of interpolation that is constant. Let us consider Fig.1.b. The set of regularly distributed points can be seen as a 2D secondary scale with respect to the main one formed by the 2D image pixel frame. By performing digital processing of the image, the pattern periodicity can be used for determining very accurately the centre position

<figure><img src="image_2.png" /><figcaption>Fig. 1. Similarly to the Vernier caliper (a), the position of the regular distribution of dots (b) with respect to the 2D pixel frame can be used for subpixel interpolation in position measurement.</figcaption></figure>

of the pattern. Phase measurements are involved in those computations as described in section II and the degree of pixel interpolation depends mainly on pattern design and signal to noise ratio. The paper is aimed to present the basic principle of our approach as well as different configurations dedicated to the measurement of 2D and 3D position, displacement or vibration amplitudes. We present also an interferometric configuration addressing the six degrees of freedom and a pattern design for in-plane position encryption of an extended 2D surface.

## II. IMAGE PROCESSING AND POSE RETRIEVAL

The image processing is aimed to determine with the best accuracy the position of the pattern represented in Fig.1.b with respect to the image pixel frame. This in-plane position is given by the center coordinates and the pattern orientation. The different steps of position retrieval are illustrated in Fig.2. In this case, the pseudo periodic pattern has an hexagonal shape with a truncated summit (cf Fig.2.a) in order to break the $\pi/6$ symmetry and thus avoid orientation ambiguity. (Square and hexagonal patterns are compared below). At the final step, the pose is given by the best intersection of three phase planes that are representative of the point position along the main direction of the pattern. Those phase planes are reconstructed independently from each other through the following space-frequency processing. Fig.2.b represents the Fourier spectrum of the hexagonal pattern. This spectrum is
<figure><img src="image_3.png" /><figcaption>Fig. 2. a) Hexagonal pattern with a truncated summit for unambiguous angle determination; b) Fourier spectrum of the pseudo-periodic pattern; c) Single Fourier lobe obtained by bandpass filtering; d) One-directional fringe pattern given by the real part of the inverse Fourier transform of c); e) Wrapped phase obtained by inverse Fourier transform; f) Unwrapped phase plane.</figcaption></figure>

made of well separated spatial frequencies that correspond to the different modulation directions of the pattern. In Fig.2.c, a single lobe has been filtered out from the spectrum that corresponds to a single direction of modulation. This lobe results from the application of a gaussian filter to the spectrum; the width of filter determines the space-frequency trade-off of the process. After inverse Fourier transform of the filtered spectrum, a fringe-like modulation is given by the real part (cf Fig.2.d) while the argument of this inverse transform leads to a wrapped phase map shown in Fig.2.e. This kind of phase extraction from Fourier processing was proposed by Takeda in the field of shape measurement by fringe projection[3]. The phase is representative of the pattern position with a high accuracy. For instance, the fringe pattern of Fig.2.c can be expressed as :

$$I(x, y) = I_0 \cdot \cos(\Phi(x, y)) \quad \text{with :} \quad (1)$$

$$\Phi(x, y) = 2\pi \cdot (\nu_x \cdot (x - x_0) + \nu_y \cdot (y - y_0)) \quad (2)$$

where $(\nu_x, \nu_y)$ are the coordinates of the frequency lobe of Fig.2.c and $(x_0, y_0)$ are the coordinates of the pattern center. The phase map is thus known to be a plane with the property that each intensity maximum corresponds to a phase equal to

<figure><img src="image_4.png" /><figcaption>Fig. 3. a) Pattern frontiers given by the modulus of the inverse Fourier transform; b) Result of cross-correlation between the initial image and a digitally generated one.</figcaption></figure>

0 modulo $2\pi$. To come back to the phase plane, the phase returned by the argument of the inverse Fourier transform (cf Fig.2.e) has to be unwrapped as represented in Fig.2.f. In this phase map, we choose to keep only the central region for avoiding phase distortions due to side effects. This phase plane is representative of the dot pattern position along the direction perpendicular to the fringes of Fig.2.d (modulo $2\pi$; i.e. one period) but not along the fringe direction. Complementary information is obtained by applying the same processing to two other lobes of the Fourier spectrum that correspond to directions rotated by 60° and 120° respectively. We thus have to combine these three phase maps to determine the pattern position. For that purpose, it is necessary to determine a coarse pattern position in order to remove the $2\pi$ phase ambiguities. This can be done following two different approaches that are represented in Fig.3. Fig.3.a corresponds to the modulus of the inverse Fourier transform of the filtered spectrum of Fig.2.c. The pattern contours are well defined and can be used for that coarse center determination if the signal to noise ratio is large enough. In other cases, we perform a cross-correlation between the initial image and a computer generated one. The latter is a digital copy of the pattern with period and orientation derived from the unwrapped phase map (cf Fig.2.f). The result of this cross-correlation is represented in Fig.3.b in which the position of the intensity maximum gives the pattern center.

Once the coarse pattern center position is known, the unwrapped phase maps are adjusted with a multiple of $2\pi$ to match with equation 2. Then the phase maps are approximated by least square fitting and the pattern center position is determined with a subpixel accuracy. The pattern orientation is given by the phase map slope and the $\pi/6$ ambiguity is removed from the altered summit position. The latter is determined either from the modulus (Fig.3.a) or by performing six cross-correlations; each of them with a different position of the truncated summit.

With respect to an hexagonal shape, the square design of Fig.1.b has the disadvantage to have only two main directions at 90° from each other. The performances were fund to be dependent on the relative angle between the pattern directions and the image pixel frame while it is not the case for the hexagonal pattern since it present three directions at 60° from
<figure><img src="image_5.png" /><figcaption>Fig. 4. a) Distribution of 100 measurements without displacement. Full scale is only 3 · 10⁻² pixel; b) Detection of a target motion: full scale is 0.1 × 0.2 pixel; i.e. one 50th of a pixel.</figcaption></figure>

each other.

### III. CONFIGURATIONS AND RESULTS

#### A. In-plane pose estimation

In-plane pose estimation can be performed by any vision system. The actual size of the pattern has simply to be adapted to the optical magnification in order to match with the system capabilities. Fig.4 presents results obtained under an optical microscope with a 10× objective observing a square pattern with a period of 10µm. The measurement repeatability was tested by performing 100 measurements without target displacement (except mechanical drift) and results are presented in Fig.4.a. The peak-valley deviation is smaller than 10⁻² pixel in the Y direction. It is slightly larger in the X direction because of the contribution of the line start trigger of the video signal. The actual size on the target is about 1µm per pixel leading to a resolution of about 4nm to be compared with the pattern period of 10µm observed with a 10× objective. Fig.4.b presents the position reconstructed with the same setup while the target described the word 'NANO' by means of a piezoelectric transducer. With that resolution, it would be possible to observe 50 times the word 'NANO' within a single image pixel.

#### B. Vibration characterization

The technique was applied to the monitoring of the vibration amplitude of a shear-force microscope probe[4], [5]. The latter is made of a quartz tuning fork loaded with a sharp tungsten tip. A focused ion beam was used to etch a dot pattern on the end faces of the two prongs with a period of 3µm. The shear-force microscope was operated on a transparent glass sample in order to allow the observation of the dot pattern by means of a 20× microscope objective. A stroboscopic illumination was chosen to shift the resonance frequency of the tuning fork (~33kHz at first mode) within the 25 fps bandwidth of a standard CCD camera. A 2Hz shift was kept between tuning fork dithering frequency and strobe frequency in order to observe the prong motion at an apparent frequency of 2Hz, compatible with the image rate. This setup allowed the determination of the tuning fork resonances as well as the quantification of the prong motion. Fig.5.a presents the dot pattern orientation as reconstructed while the tuning fork was dithered at 181.552kHz. This high order vibration mode induces a torsion of the prongs that is well resolved by our method. The torsion amplitude is about 10⁻² deg. in this case and the rotation resolution was evaluated to be 10⁻³ deg. with a good signal to noise ratio (256 gray levels).

In fact, the resolution in pattern orientation was fund to be much more dependent on the pattern period than the center coordinates. The shorter the period, the better the orientation resolution while the actual pattern dimensions are smaller for a shorter period. This can be justified by a better definition of the spatial frequency that is increased when the period is reduced.

Fig.5.b presents the vibration amplitude of the prong as recorded during three shear-force interaction cycles during which the tungsten tip is approached until contact with the surface and then retracted. The vibration damping due to tip-surface contact can be easily measured as well as a drift during the first cycle.

#### C. Addressing 3D displacements

Fig.6.a presents a two camera setup for sensing along the three spatial directions. Two pseudo-periodic patterns are used with a relative orientation of 90 deg. to be perpendicular to the optical axis of the two vision systems. If telecentric objectives are used the measurements along the three directions remain mostly independent from each other. Otherwise the geometrical behavior of the lenses has to be taken into account and a calibration is necessary to convert pixel information into actual space coordinates. Fig.6.b presents a 3D displacement produced by a piezoelectric transducer and describing our department acronym. In this configuration, the magnification is of 32.555µm per pixel and the pose resolution was evaluated to 29nm to be compared to a working distance of 20cm and a measurement range around 3cm³.

#### D. 6 D.O.F. sensing by using an interference objective

In order to widen the method capabilities we investigated the possibility to work with an interference objective that is
<figure><img src="image_6.png" /><figcaption>Fig. 5. a) Prong torsion angle of a tuning fork vibrating at 181.552kHz as measured by stroboscopy; b) Measured vibration amplitude damping produced by successive tip-surface contacts.</figcaption></figure>

sensitive to out of plane features. Such devices are widely known in surface profilometry[6], [7] and we combined both approaches to be sensitive to the six degrees of freedom[8]. Fig.7.a presents the principle of a Mirau interference objective. A beam splitter is placed in the focusing part of the beam and a fraction of the light is focused on a reference mirror fixed inside the lens. The other part of the beam is focused on the surface as usually. After reflection, the beam splitter combines the light reflected by the mirror and sample surfaces and interferograms are recorded in this way. The latter encode the surface height within the interference intensity following a modulated cosine function. To remove ambiguities, phase-shifted interferograms are recorded by introducing a slight variation of the optical path difference with a calibrated PZT. Then algorithms are available for reconstructing the surface profile from a set of phase-shifted interferograms. In order to be sensitive to position instead of surface profile, we used a specific sample whose profile is represented in Fig.7.b. A plate silicon sample was micro-machined to present a pseudo periodic pattern in relief. The peak height was chosen to be 50nm that is convenient for measurement by phase shifting interferometry.

<figure><img src="image_7.png" /><figcaption>Fig. 6. a) Two camera configuration of the vision system; Reconstructed three dimensional displacements.</figcaption></figure>

<table><caption>Table 1: Statistical results obtained from 100 measurements without target displacement.</caption><thead><tr><th>Repeatability</th><th>Mean</th><th>Peak-Valley</th><th>rms.</th></tr></thead><tbody><tr><td>X (pix)</td><td>277.012</td><td>15.10<sup>-3</sup></td><td><strong>3.2.10<sup>-3</sup></strong></td></tr><tr><td>X (nm)</td><td>NS</td><td>12.4</td><td><strong>2.66</strong></td></tr><tr><td>Y (pix)</td><td>329.086</td><td>59.10<sup>-3</sup></td><td><strong>6.5.10<sup>-3</sup></strong></td></tr><tr><td>Y (nm)</td><td>NS</td><td>49.6</td><td><strong>5.42</strong></td></tr><tr><td>θ<sub>Z</sub> (rad.)</td><td>0.046</td><td>5.04.10<sup>-4</sup></td><td><strong>1.05.10<sup>-4</sup></strong></td></tr><tr><td>Z (nm)</td><td>NS</td><td>16.9</td><td><strong>2.93</strong></td></tr><tr><td>θ<sub>X</sub> (μrad.)</td><td>-35935</td><td>8.0</td><td><strong>1.4</strong></td></tr><tr><td>θ<sub>Y</sub> (μrad.)</td><td>28775</td><td>4.0</td><td><strong>0.8</strong></td></tr><tr><td>Pix size (nm/pix)</td><td>830.65</td><td>0.64</td><td><strong>0.15</strong></td></tr></tbody></table>

The pose estimation requires a profile measurement from a set of four phase-shifted interferograms. Once the profile is reconstructed (cf Fig.7.b), the in-plane position is derived from the peak locations (after mean plane removal for an uniform peak visibility) by applying the image processing described earlier. Then, the mean plane of the surface profile is computed by considering only the outside of the pattern to avoid peak distortions. An equation is derived that is suitable for determining the tilt angles of the surface as well as the height of the pattern center. These results complete the in-plane ones to provide the pose along the six degrees of freedom. Table 1 summarizes the repeatability of the method as measured experimentally with a 10× Mirau objective. The
<figure><img src="image_8.png" /><figcaption>Fig. 7. a) Schematic description of a Mirau interference objective; b) 3D profile of a patterned specimen as reconstructed with nanometer accuracy.</figcaption></figure>

standard deviation is a few nm for position and about $10^{-6}$ and $10^{-4} \text{rad}$ for $\Theta_x, \Theta_y$ and $\Theta_z$ respectively. This configuration requires the sample to be static during measurement and the allowed out of plane slopes are restricted to ensure a correct interference fringe sampling, depending on the lens magnification [8]. Despite these constraints, the system could be used for the calibration of actuators with nanometer resolutions and less restrictive interference configurations could be proposed to be more time efficient.

### E. Pseudo-periodic pattern for extended surface encryption

The configurations and patterns presented until now require the pattern to remain visible in the field of view during the whole process. This condition can be restrictive and impose a trade-off between resolution (linked to high magnification) and displacement range (linked to low magnification). To extend the method capabilities, we designed a continuous pseudo-periodic pattern that allows the identification of the position of any field of view with respect to the whole pattern. In this way, magnification and size of encoded field can be chosen independently. The actual application developed concerns biological inspection with optical microscopy. The problem is to come back easily on a particular site of interest of a cell culture while the sample has been removed from the microscope to return in a culture chamber for a few hours [9]. The pseudo-periodic pattern proposed differs from the previous ones only in the identification of the line and column numbers. The subpixel resolution achieved remains based on phase computations applied to the spatial frequencies associated with the periodic frame of the pattern. Fig.8 presents the kind of alteration made to the periodic frame in order to encrypt the in-plane position. Some dots are missing and their distribution follows a codification of the X and Y orders of the dots. This codification is based on linear shift register sequences as described elsewhere [10]. Pose retrieval involves complementary image processing to identify the location of the missing points and thus to return the line and column orders necessary to complete the fine position provided by phase computations.

<figure><img src="image_9.png" /><figcaption>Fig. 8. Extended pseudo-periodic pattern. Line and column numbers are encrypted in the periodic frame through the location of the missing points.</figcaption></figure>

## IV. OUT OF FOCUS CAPABILITIES

A particularly interesting property of the method concerns the extra working depth allowed by the space-frequency analysis. Since the position retrieval is based on phase computations associated to a single spatial frequency, it is not necessary to keep a sharp image on the camera. Image blur due to defocus can be supported as long as the spatial frequency of interest is seen with a sufficient contrast. Fig.9.a presents the contrast variation while the pseudo-periodic pattern is scanned through focus. The highest values are obtained while the sample is exactly in focus and lower values are observed elsewhere. The two square marks delimit the interval between which the image processing succeeds in determining the pose properly. We can see that the working depth is about ten times larger than the depth of focus of the lens used. Fig.9.b shows the pseudo-periodic images recorded at extreme positions that are compared to the in-focus image. The loss in position accuracy due to defocus has still to be characterized. However this capability to work on an axial range larger than the focus depth of the lens is of great interest for actual control applications.

## V. CONCLUSION

The paper presents a subpixel pose estimation method that is based on space-frequency analysis of a pseudo-periodic pattern fixed onto the target. The position is obtained by combining fine and coarse measurements that are complementary as in the case of the Vernier caliper. Fine measurement, providing subpixel accuracy, is based on phase computations performed after a Fourier filtering aimed to separate the different directions of modulation of the pattern. The coarse measurement can be done in different ways, either by analyzing the pattern contours, by image cross-correlation or by decrypting the distribution of points missing in the periodic frame. A high resolution is achieved in pose retrieval and range and resolution can be made independent by using an extended pseudo-periodic pattern. Different configurations and applications are reviewed that demonstrate the main capabilities of that kind
<figure><img src="image_10.png" /><figcaption>Fig. 9. Top: Contrast variation while pseudo periodic pattern is scanned through focus; Bottom: In focus image (middle) compred to extreme images (sides) allowing correct position retrieval.</figcaption></figure>

of approach. The method combines subpixel accuracy, an extended in-plane range and a working depth much larger than the lens depth of focus. These properties seem to be of interest for applications in pose control. Processing times are of the order of seconds by using Matlab but could be drastically reduced by software optimization or hardware implementation.

## ACKNOWLEDGMENT

The authors would like to thank Région de Franche-Comté and Institut National du Cancer (INCa) for project funding. We also acknowledge colleagues who contributed to some configurations presented in this work.

## REFERENCES

[1] P. Sandoz, S. Dembelé, J.C. Ravassard, A. Janex, *Phase-sensitive vision method for high accuracy position measurement of moving targets*, IEEE Trans. on Instrum. and Meas., **49**, pp. 867–872, 2000.

[2] P. Sandoz, V. Bonnans, T. Gharbi, *High-accuracy position and orientation measurement of extended 2D surfaces by a phase-sensitive vision method*, Appl. Opt., **41**, pp. 5503-5511, 2002.

[3] Takeda, M. and Mutoh, K., *Fourier transform profilometry for the automatic measurement of 3-D object shapes*, Appl. opt., **22**(24), 3977-3982, 1983.

[4] P. Sandoz, J.M. Friedt, E. Carry, *In-plane rigid-body vibration mode characterization with nanometer resolution by stroboscopic imaging of a microstructured pattern*, Rev. of Scient. Instrum., **78**, 023706, 2007.

[5] P. Sandoz, J.M. Friedt, E. Carry, *Vibration amplitude of a tip-loaded quartz tuning fork during shear force microscopy scanning*, Rev. of Scient. Instrum., **79**, 086102, 2008.

[6] K. Creath: *Phase-Measurement Interferometry Techniques*, in Progress in optics, **26**, E. Wolf ed., Elsevier, Amsterdam, 349393, 1988.

[7] P. Sandoz, *Wavelet transform as a processing tool in white-light interferometry*, Optics letters, **22**(14), 1065-1067, 1997.

[8] P. Sandoz, *Nanometric Position and Displacement Measurement versus the Six Degrees of Freedom by Means of a patterned Surface Element*, Appl. Opt., **44**, pp. 1449–1453, 2005.

[9] P. Sandoz, R. Zeggari, L. Froehly, J.L. Prétet, C. Mougin, *Position referencing in optical microscopy thanks to sample holders with out-of-focus encoded patterns*, J. of Microsc., **255**, pp. 293-303, 2007.

[10] J. Galeano Zea, P. Sandoz, *Position encryption of extended surfaces for subpixel localization of small-sized fields of observation*, submitted to International Symposium on Optomechatronic Technology, Istanbul, Turkey, Sept. 21-23, 2009.