# Pseudo-periodic patterns for subpixel accuracy visual control: principle, pattern designs, and performances

J. Galeano Zea, P. Sandoz

► To cite this version:

J. Galeano Zea, P. Sandoz. Pseudo-periodic patterns for subpixel accuracy visual control: principle, pattern designs, and performances. SPIE Europe Optical Metrology, Jun 2009, Munich, Germany. 10.1117/12.827485. [https://doi.org/10.1117/12.827485](https://doi.org/10.1117/12.827485). [arXiv:0906.5110v1](https://arxiv.org/abs/0906.5110)

HAL Id: hal-00472309

https://hal.science/hal-00472309v1

Submitted on 25 May 2021

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
Pseudo-periodic patterns for subpixel accuracy visual control:
principle, pattern designs and performances

July A. Galeano Zea, Patrick Sandoz

Département d'Optique P.M. Duffieux, Institut FEMTO-ST,
U.M.R C.N.R.S. / Université de Franche-Comté,
F 25030 Besançon cedex, France;

ABSTRACT

A vision system is used for measuring in-plane target displacement, position and orientation. Pseudo-periodic patterns fixed on the target forms a phase reference. Absolute position is determined with subpixel accuracy by phase computations. Various position encoding designs are proposed for different displacement ranges and resolution. Performances obtained are compared and discussed for both displacement and orientation measurements. The capability to resolve position on depth ranges larger than the lens depth of focus is demonstrated.

**Keywords:** Visual control, in-plane position and displacement measurement, subpixel resolution, angle measurement, phase computation, space-frequency analysis

1. INTRODUCTION AND PRINCIPLE

Visual control is widely used in robotics and automation as a position sensor in servo-control loops of actuators and robots.¹ However, visual control is also used in other fields, for instance for precise alignment, for easy localization of regions of interest in micro- and nanotechnology or more generally in many applications using microscopy. Several approaches were reported, based on processing of objects features or specific marks, and using dedicated image filtering. A trade-off has usually to be found between position accuracy, measurement rate and system complexity and cost.

Our approach achieves in-plane position and orientation measurements with a high level of pixel interpolation.²,³ It is based on the processing of a pseudo-periodic pattern fixed on the target. The principle can be seen as a two-dimensional extension of the Vernier principle as depicted in Fig.1. The Vernier principle is based on two complementary scales with slightly different periods. The position of coincidence between the two scales gives the relative shift between the scales and provides a sub-unit measurement that refines the coarse measurement given by the main scale. Our method also uses two scales. One is given by a two-dimensional set of dots distributed along a periodical grid and with a finite spatial extension. The second scale is simply given by the image pixel frame as schematized by the dotted lines in the figure. As described in Sect.2, phase measurements allow a fine comparison of the two scales and the high accurate identification of the orientation and centre position of the dot pattern with respect to the pixel frame. Achieved resolution depends mainly on the signal to noise ratio of recorded images; it is in the range of 10⁻³ pixel for centre position identification with an image dynamics of 8 bits, while the angle resolution is a few 10⁻³ degree.

2. PATTERN DESIGNS AND POSITION RECONSTRUCTION

The pattern design used depends on the actual application and we distinguish mainly two cases. In the first one, the allowed displacement range is much larger than the vision system field of observation. The pattern design has thus to cover the whole measurement range and to allow position retrieval from any local image given by the pattern area located under the field of observation at the measurement time instant. We reported an original solution⁴ formed by a periodic grid modified by a two-dimensional position encryption based on linear feedback shift register sequences.

<footnote>Further author information:</footnote>

<footnote>E-mail: july.galeano_zea@univ-fcomte.fr</footnote>

<footnote>patrick.sandoz@univ-fcomte.fr</footnote>
<figure><img src="image_2.png" /><figcaption>**Figure 1.** a) Interpolation principle of the Vernier scale; b) The regular distribution of dots with respect to the 2D pixel frame can be seen as a generalization of the Vernier scale for subpixel interpolation in position measurement.</figcaption></figure>

<figure><img src="image_3.png" /><figcaption>**Figure 2.** a) Regular distribution of dots following a square design. b) Corresponding Fourier spectrum; c) Regular distribution of dots following a hexagonal design. The alteration introduced breaks symmetry for avoiding orientation ambiguities. d) Corresponding Fourier spectrum.</figcaption></figure>
<figure><img src="image_4.png" /><figcaption>**Figure 3.** a1-3) Spatial frequency band extracted from Fourier spectrum for reconstruction of one-directional stripes. b1-3) One-directional stripes obtained by inverse Fourier transform of filtered spectrum a1-3). c1-3) Wrapped phase corresponding to stripes b1-3). d1-3) Unwrapped phase over a central region of the pattern.</figcaption></figure>
<figure><img src="image_5.png" /><figcaption>**Figure 4.** a) Test pattern reconstructed with period and orientation retrieved from previous steps; b) Result of correlation between initial image and test pattern of a).</figcaption></figure>

In the second case, concerned displacements remain small compared to the vision system field of observation. Then the pattern design is simply a finite set of periodical points. This set is aimed to be remain visible within the recorded image during the whole measurement. This solution is well suited for the measurement of small displacements, of vibrations⁵,⁶ or for interferometric measurement along the six degrees of freedom.⁷ The basic principle of position reconstruction is similar in the two cases and benefit from Fourier filtering and phase computations. Figures 2.a and 2.c present square and hexagonal designs explored for small displacement measurement. Let us first describe the position reconstruction technique before to compare between the two designs. Figures 2.b and 2.d present the Fourier spectra obtained with the square and hexagonal designs respectively. Different lobes are associated to the different directions of point distribution and they can be filtered out independently from each other. This is illustrated in Figure 3 in the case of the hexagonal shape. The same procedure is applied to three Fourier lobes corresponding to the main direction of point distribution. Figures 3.a1 through 3.a3 show the Fourier lobe extracted from the Fourier spectrum as represented in Fig.2.d. After inverse Fourier transform, phase and fringe images are reconstructed for each lobe as represented in Fig.3.b1 through Fig.3.c3. To avoid side effects, the unwrapped phase is considered only in the central region and the expected phase plane corresponding to a linear distribution of lines is retrieved by least square fitting. Three phase equations are thus obtained that give different information. The pattern period can be derived from these phase equations as well as the pattern orientation (modulo $\pi/3$). These three equations provide also a fine measure (modulo $2\pi$) of the pattern centre position that only requires the removal of $2\pi$ phase ambiguities in the unwrapped phase. This step is performed by image correlation as illustrated in Fig.4. From the knowledge of the pattern period and orientation, a set of six digital patterns is generated numerically and correlated with the recorded image. These six patterns differs in the summit that is reduced by four points. This alteration is aimed to break the pattern symmetry and allows unambiguous pattern orientation determination. Among the results of image correlation with the six patterns, the highest correlation value is obtained for the right orientation of the pattern orientation and the central point is identified by the highest intensity observed in Fig.4.b. This coarse position determination allows phase adjustment of the unwrapped phase equations and their intersection determine the final pattern centre position unambiguously and with a high resolution.

In the case of a square pattern as shown in Fig.2.a, the position is retrieved in the same way from only two phase maps.³ The $\pi/2$ pattern symmetry could also be broken by some alteration to allow unambiguous orientation determination. The main difference between square and hexagonal designs is related to their similarity with the image pixel frame. Image pixels are distributed along a 2D grid and significant performance differences
<figure><img src="image_6.png" /><figcaption>**Figure 5.** Comparison of angle retrieval performances of square and hexagonal designs with almost the same number of dots. Results obtained by simulation by introducing random noise with a rms. value of one gray level.</figcaption></figure>

were observed³ relatively to orientation of the square pattern with respect to the image line and raw directions. This problem is no more critical in the case of the hexagonal design since the pattern main orientations differ from a square grid.

## Square and Hexagonal Performances

Square and hexagonal designs were firstly compared through simulated images in order to avoid experimental bias. Fig.5 presents the error observed in the reconstruction of the pattern orientation for the two designs and for and orientation varying between 0 and 45 degrees (all orientations are thus covered due to pattern symmetries). This result was obtained by introducing random noise in the pattern images with a standard deviation equal to one gray level (among 256). Peaks and valleys are randomly distributed in the angle measurement error because of noise but on the average, the hexagonal pattern was found to provide better results than the square one as in the case of the figure. However, the different parameters involved in the Fourier filtering, for instance band-pass filter width and size of the unwrapped area, were found to be as important as the pattern design. Once the latter are optimized, the error in the angular measurement is at least 10% lower for the hexagonal design. (The introduction of zero-padding before Fourier transform is also important for excellent position and orientation retrieval, especially for small images of 128 × 128 pixels).

This advantage of the hexagonal design for orientation determination seems to be experimentally counter-balanced by lower performances in pattern centre position measurement. This point will be discussed in more details in the conference presentation.

## 3. EXPERIMENTAL PERFORMANCES

Experiment reported in this section were obtained by using the extended pseudo-periodic pattern that allows position measurement over a range much larger than the vision system field of observation.⁴ Performances were evaluated by fixing the pseudo-periodic pattern on a 3D piezo-electric translator (PI) in order to apply known displacements. However, because of load and servo-control limitations of this equipment, performances in position and orientation measurements were evaluated through complementary data analysis.

Fig.6 presents the pattern position reconstructed by the proposed method during a linear displacement of 5µm by steps of 0.01µm. Deviations from a straight line are mainly due to the translator uncertainty as confirmed by comparison with a reference method. The latter consists in image correlation of each recorded image with the
<figure><img src="image_7.png" /><figcaption>**Figure 6.** Comparison of positions reconstructed from pseudo-periodic pattern processing and from sub-pixel image correlation.</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>**Figure 7.** Reconstructed angle value during progressive object rotation.</figcaption></figure>

first one. Since the pattern is pseudo-periodic, the correlation is also periodic and thus allows pixel interpolation by means of spectral phase measurement. The excellent agreement between the results of these independent methods demonstrate the translator responsibility and provides an alternative for performance evaluation. The standard deviation of the difference between these independent measurements was found to be 25nm to be compared with a pattern period of 4µm observed by means of a 20× microscope objective.

Fig.7 presents the result of angle measurement while the same piezo-translator was used for introducing a rotation of the experimental setup. In this case, image correlation can not access to the actual angle and we
<figure><img src="image_9.png" /><figcaption>**Figure 8.** Left: Evolution of a contrast parameter while the pseudo-periodic pattern is scanned through focus. Black squares indicate the interval in which position retrieval works. Right: Extreme images leading to correct position identification compared to the best focus image.</figcaption></figure>

are unable to distinguish between translator uncertainty and error in angle measurement. However, the peak to
peak deviation from a straight line is about $10^{-3}$ degree that forms an upper evaluation of the reconstructed
angle error.

One particularly interesting property of the method concerns its capability to perform pattern position identification from out of focus images as illustrated in Fig.8. A 20× microscope objective with 0.5 numerical aperture is used for observing the pseudo-periodic pattern. The resulting depth of focus is about 2.9µm as shown by the vertical lines in the figure. The solid line describes the evolution of a contrast parameter⁸ while the pattern is scanned through focus in front of the lens. The focus range allowing correct position retrieval is delimited by the two black squares and correspond to almost ten times the depth of focus of the lens used. This capability results from the filtering of sharp spatial frequencies in the Fourier spectrum and in robust software implementation. The extreme images allowing position retrieval are shown on the right side of the figure and compared to the in-focus image. We can see that the pattern period is relatively far from the Niquist limit of two pixels per period and this is a first reason of the low method sensitivity to defocusing. (The side lobe observed in the contrast parameter is due to some imaging effect due to the pattern periodicity). The loss in position measurement resolution due to defocusing has still to be investigated experimentally but the possibility to perform correct measurement on such an extended focus range is of great interest, for instance for application in actuator localization in micro-robotics.

# 4. CONCLUSION

This paper proposes a synthetic view of in-plane position and orientation measurement capabilities offered by processing in the Fourier domain images of pseudo-periodic patterns. Different pattern designs are explored. Square and hexagonal designs are compact and simple and well suited for small displacement ranges. Extended displacement ranges require an extended pseudo-periodic pattern as realized by the encryption of the absolute position within a periodic distribution of points. Performances are about $10^{-3}$ pixel for centre position identification and $10^{-3}$ degree for angle determination. Finally, we demonstrate that this method is able to provide a correct position measure from out of focus images with the measurement range as large as then times the objective depth of focus.
REFERENCES

1. Wilson, W., Hulls, C., and Bell, G., "Relative end-effector control using Cartesian position based visual servoing: Special section on vision-based control of robot manipulators," *IEEE Transactions on Robotics and Automation* **12**(5), 684–696 (1996).

2. Sandoz, P., Ravassard, J., Dembelé, S., and Janex, A., "Phase-sensitive vision technique for high accuracy positionmeasurement of moving targets," *IEEE Transactions on Instrumentation and Measurement* **49**(4), 867–872 (2000).

3. Sandoz, P., Bonnans, V., and Gharbi, T., "High-accuracy position and orientation measurement of extended two-dimensional surfaces by a phase-sensitive vision method," *Applied optics* **41**(26), 5503–5511 (2002).

4. Sandoz, P., Zeggari, R., Froehly, L., Pretet, J., and Mougin, C., "Position referencing in optical microscopy thanks to sample holders with out-of-focus encoded patterns," *Journal of Microscopy* **225**(3), 292–302 (2007).

5. Sandoz, P., Friedt, J., and Carry, E., "In-plane rigid-body vibration mode characterization with a nanometer resolution by stroboscopic imaging of a microstructured pattern," *Review of Scientific Instruments* **78**, 023706 (2007).

6. Sandoz, P., Friedt, J., and Carry, É., "Vibration amplitude of a tip-loaded quartz tuning fork during shear force microscopy scanning," *Review of Scientific Instruments* **79**, 086102 (2008).

7. Sandoz, P., "Nanometric position and displacement measurement of the six degrees of freedom by means of a patterned surface element," *Applied Optics* **44**(8), 1449–1453 (2005).

8. Cho, H., [*Optomechatronics: Fusion of optical and mechatronic engineering*], CRC Press (2006).