# Position encryption of extended surfaces for subpixel localization of small-sized fields of observation

J. Galeano, Laurent Robert, P. Sandoz

► To cite this version:

J. Galeano, Laurent Robert, P. Sandoz. Position encryption of extended surfaces for subpixel localization of small-sized fields of observation. ISOT 2009. International Symposium on Optomechatronic Technologies, Sep 2009, Istambul, Turkey. pp.22-27, 10.1109/ISOT.2009.5326093. [hal-00472305](https://hal.science/hal-00472305v1)

HAL Id: hal-00472305

https://hal.science/hal-00472305v1

Submitted on 15 Apr 2021

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# Position encryption of extended surfaces for subpixel localization of small-sized fields of observation

July A. Galeano Zea and Patrick Sandoz

Institut FEMTO-ST, Département d'Optique PM Duffieux
UMR CNRS Université de Franche-Comté 6174
F-25030 Besançon cedex, France

Email: july.galeano_zea@univ-fcomte.fr and patrick.sandoz@univ-fcomte.fr
Telephone: +33 (0) 381 666 420 - Fax: +33 (0) 381 666 423

Laurent Robert

Institut FEMTO-ST, MIMENTO
UMR CNRS Université de Franche-Comté 6174
F-25044 Besançon cedex, France

Email: laurent.robert@femto-st.fr
Telephone: +33 (0) 381 853 999

***Abstract***—A vision system is used for measuring in-plane target position, displacement and orientation. An encrypted pseudo-periodic pattern fixed on the target forms a phase reference. Absolute position is determined with subpixel accuracy by phase computations from any local view of a small zone of the pattern. Method principle is presented and performances are characterized. The capability to resolve position on depth ranges larger than the lens depth of focus is demonstrated. The method is applied to position referencing of live cell culture boxes and could be used as visual control sensor in other applications.

## I. INTRODUCTION

Visual control is widely used in robotics and automatics as a position sensor in order to determine the actual position of a body and according to this, to execute a control. As examples of the use of visual systems are tracking methods, visual-servoing, video-microscopy or more generally precise alignment in micro- and nanotechnology. Several approaches were reported based on phase correlation[1], stereo-vision and processing a specific mark or feature in the target by using dedicated image filtering. A trade-off has usually to be found between position accuracy, measurement rate and system complexity and cost.

In our approach a pseudo-periodic pattern is fixed on the target and observed by a vision system. Then in-plane position and orientation of the target are reconstructed with a high resolution. [2], [3], [4]. This pattern is based on a periodic distribution of dots that is distorted in such a way to encrypt the position information. As described in section III absolute position is retrieved by combining fine and coarse data computed independently. Firstly, phase computation processing are used to determine with a high accuracy the relative position of the periodic grid of the pattern with respect to the image pixel frame. At this stage, ambiguities of a whole number of periods are present. Those ambiguities are removed in a second step by analyzing the periodic grid distortions. This stage is aimed to decrypt the absolute position information inserted in the grid by means of missing points. Finally the target position is retrieved in an absolute way with a subpixel accuracy.

Performances are evaluated by using a servo-controlled piezoelectric transducer and by comparing results with reference data. Resolution better than $10^{-2}$ pixel and $10^{-3}$ degree are demonstrated in position and angle reconstruction respectively. Finally, the method is applied to the referencing of cell culture boxes in order to observe the same region of interest of a specimen while the latter is transferred from microscope stage to culture chamber after every observation. The pseudo-periodic effectiveness is demonstrated by obtaining excellent image superimposition.

## II. OBJECTIVE AND PATTERN DESIGN

### A. Objective

The objective of the pattern is to obtain 2D absolute position and orientation of any surface zone observed with respect to the whole encoded surface as depicted in figure 1. A vision system observes an area ($w \times h$) among the whole encoded area ($W \times H$). The zone $v$ observed by the vision system has an orientation ($\theta$) with respect to angular reference provided by the pattern on the target. The aim of the pseudo periodic pattern and of the image processing presented in the following sections is to allow the retrieval of the position ($x, y, \theta$) with high-precision from any local image. Furthermore angular and lateral image adjustments are expected in order to superimpose images in a common position reference system and to allow site by site comparison to render of the dynamics of a specific target.

### B. Pseudo-periodic code for position encryption

In order to obtain high-precision measurement a design permitting phase calculation has been chosen. The latter is made of a pseudo-periodic dot distribution as depicted in figure 2. If the pattern was perfectly periodic, it would not be possible to distinguish between consecutive lines and columns of dots. Therefore, the aim of the missing points is to distort the regular grid in an unique way for the identification of the actual order of each line and each column of dots (from the upper left corner of the encoded area). The encoding technique is based
<figure><img src="image_2.png" /><figcaption>Fig. 1. Definition of the position (x, y, θ) of a zone (v) of size (w × h) observed under microscope with respect to the encoded surface (Σ) of size (W × H).</figcaption></figure>

on so-called *pseudo-random sequences* of bits that are useful for test procedures in the field of digital signal processing and telecommunications because of their statistical properties[5]. The basic principle consists in encoding a sequence of (2^N) positions by using a shift register of length N. With appropriate linear feedback of the shift register with XOR gates, the words produced successively by the N register values describe all possibilities, i.e. 2^N - 1 different words (only the zero value is missing and can be introduced in case of computer generated sequences). The effect of this encryption method is depicted in Figure 3 for the case of words of three bits. Because of the shift register principle, successive words share necessarily N-1 bits with immediate neighbors. It is thus possible to retrieve the word value by combining N consecutive bits centered on the current position. By using this technique, the encoding of a sequence of 2^N words requires only 2^N + N - 1 bits. The only requirement is the knowledge of a look-up table for the conversion of the words read into their actual position along the sequence. The extension of this principle to two dimensions is obtained by reproducing identical sequences among which some are shifted with respect to the other ones as can be seen in the decoded words of Fig.4. In the 2D code, the reference lines give the horizontal position. It means that in a zone under test, reference lines match to each other and give the same binary value while the shifted lines correspond to the value of the reference ones plus an offset which is proportional to the position along the vertical direction. The vertical position is thus provided by the difference between the reference value and the shifted one[4].

By choice the binary code is inserted in the periodic frame with a low density of only one bit per 10 dots (5lines × 2 columns). The choice keeps complete lines of dots that are used to separate lines from each other. The code encryption is shown in Fig.5 on three lines with an alternative localization of the missing dot encoding the bit 0.

<figure><img src="image_3.png" /><figcaption>Fig. 2. The pseudo-periodic pattern is made of a distribution of dots among which some of them have been removed for absolute position encoding and retrieval</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>Fig. 3. Example of an imbricated binary sequence with words of N=3 bits: a) Usual sequence, each word requires three bits to be coded; b) Example of a LFSR sequence, each word shares N-1 bits with its neighbors and thus the bit number necessary for the whole sequence tends to be equal to the number of words.</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>Fig. 4. Binary encryption for 2-D position determination: from a binary code area the corresponding decimal value is taken in order to find in a LUT the exact bidimensional encoding position.</figcaption></figure>
<figure><img src="image_6.png" /><figcaption>Fig. 5. Dots' distribution of a position encoding along lines with a pseudo-random sequence of binary code.</figcaption></figure>

### III. IMAGE PROCESSING FOR POSITION RETRIEVAL

The position retrieval is performed in two steps as described below.

#### A. Phase computation for relative position measurement

Firstly a fine measurement is obtained from phase computations. This fine measurement corresponds to the position of the periodic frame with respect to the image pixel frame but without the identification of the line and column orders. This steps involves Fourier processing of the pseudo-periodic pattern. Fig.6.a presents the power spectrum of the pattern. It is made of a series of lobes, each of them corresponding to a specific fringe direction. Single direction fringe patterns can be obtained by inverse Fourier transform applied to a filtered spectrum keeping a single lobe. Fig.6.b presents the fringe pattern resulting from the filtering of the lobe indicated by the arrow V in Fig.6.a while Fig.6.c results from the lobe marked by the arrow H. These fringe patterns can be seen as elementary constituents of the periodic pattern that are multiplied to each other to form the periodic frame. Their cosine distribution of intensity is representative of the fringe position with respect to the image pixel frame. This information is accurately encoded in the phase of the inverse Fourier transform[6] as depicted in Fig.6.d and 6.e. Because of the periodicity of the inverse tangent function involved in this calculation the phase has to be compensated with $2\pi$ constants. This operation is known as phase unwrapping. Finally a phase plane is obtained as shown in Fig.6.f for the vertical direction. This plane can be fitted by a first degree equation due to the linear distribution of the phase with respect to the pixel image. This equation is obtained by means of least square fitting as follow for the two directions:

$$ \Phi_V(i, j) = A_V \cdot i + B_V \cdot j + C_V \quad (1) $$

$$ \Phi_H(i, j) = A_H \cdot i + B_H \cdot j + C_H \quad (2) $$

In these equations the constant parameter $C_V$ and $C_H$ are obtained with an ambiguity of a multiple of $2\pi$ since the phase unwrapping process depends on the wrapped phase at the starting point, necessarily in the interval $(-\pi, \pi]$. On the contrary the parameters A and B are fully representative of the phase slope versus the pixel frame. The orientation of the

dot pattern with respect to the image pixel frame is thus given by:

$$ \alpha = \tan^{-1}(B_V/A_V) \quad \text{or} \quad \tan^{-1}(B_H/A_H). \quad (3) $$

Once this procedure is done, the initial image is rotated by an angle $-\alpha$ in order to align the dot distribution with the image pixel frame. The two unwrapped phase planes are rotated as well by an angle $-\alpha$ and they are then represented by simplified equations:

$$ \Phi_L(i, j) = A_L \cdot j + \varepsilon_L \cdot i + C_L \quad (4) $$

$$ \Phi_C(i, j) = B_C \cdot i + \varepsilon_C \cdot j + C_C \quad (5) $$

where subscripts L and C indicates lines and columns respectively. Eq.4 and 5 are relative to the observed area and are aligned with the encoded area $\Sigma$ but present $2k\pi$ ambiguities. Therefore the corresponding phase planes obtained are not suitable for an absolute position determination. The decoding of the missing point distribution is still necessary to calculate the correct phase offsets $2k_L\pi$ and $2k_C\pi$ suitable to remove the ambiguities on the constant parameters $C_L$ and $C_C$.

<figure><img src="image_7.png" /><figcaption>Fig. 6. Analysis in the Frequency Domaine after Fourier Transform: (a)Power spectrum of the pseudo-periodic pattern. (b) and (c) Fringes in the vertical and horizontal direction after filtering of the lobes indicated by arrows V and H in Fig.6(a). (d) and (e) wrapped phase related with the vertical and horizontal fringes respectively. (f) Unwrapped phase plane.</figcaption></figure>
## B. Pseudo-periodic code decoding for absolute position

The identification of the absent dots in the pseudo-periodic pattern is an essential step for the decoding of the actual position. The decrypted line and column orders are necessary to adjust the phase constants implied in the phase plane equations and obtain the exact position. This treatment involves mainly binary image processing for the identification of the lines and columns used in the codification and the derivation of the reference and shifted sequence values. However, for images that present shadows or non uniform illumination as in Fig.7.a, this identification is problematic without locally adapted thresholds as depicted in Fig.7.b where some missing dots are not identified. To solve this problem, we compute a local contrast value that makes use of the phase data. We know that the dot centers correspond to a phase equal to 0 modulo $2\pi$ and that points in the middle between dots have a phase equal to $\pi$ modulo $2\pi$. From this knowledge we locate the dot centers as well as intermediate points as depicted in Fig.7.c. Thus a local value of the dot visibility is derived from the neighborhood that is used as threshold value to determine if the dot is present or not. This adaptive process was fund to be very robust. We can observe in Fig.7.d that all the missing points are identified properly, even in case of shadow or low illumination.

<figure><img src="image_8.png" /><figcaption>Fig. 7. Absent Dots detection: (a)Pseudo-periodic pattern with shadows and non-uniform illumination due to observation under microscope for biological use. (b) In previous work there was no possible to identify the absent dots placed in the disturbed regions of the pattern. (c) Color representation of the intensity value of a specific zone in Fig.7(a) A local contrast is computed from colored sites. The green circles corresponds to the background and the red one corresponds to the absen/present dots. (d) Absent dots identification by means of the closer visibility criteria.</figcaption></figure>

# IV. REALIZATION, PERFORMANCES AND APPLICATION

## A. Technological realization

Photolithography processes were developed to imprint the dot pattern on both glass and plastic substrates. Fig.8 show the pseudo-periodic pattern as reproduced on the photo-resist layer

deposited on a glass substrate as well as a Pétri dish in which a set of four pseudo-periodic pattern has been printed. Dots are made of aluminium over a transparent background, their size is $2\mu m$ for a period of $4\mu m$ and represent about 20% of the surface. Such Pétri dishes are used in the biological application describer below.

<figure><img src="image_9.png" /><figcaption>Fig. 8. Left: View of the dot pattern as reproduced on the photo-resist layer on a glass plate. Right: Pétri dish with the pseudo periodic pattern reproduced on bottom side for further microscopy observation.</figcaption></figure>

## B. Characterization of method capabilities

The pattern mask is attached to a servo-controlled piezoelectric transducer (PZT) in order to evaluate the method capabilities in measuring sub-micrometer displacements with different kinds of tests. For this purpose the mask is illuminated with a red light emitting diode an imaged by a CCD camera through a $10 \times$ magnification microscope objective. Once the mechanical stability of the structure is obtained, the motor is activated by steps of $0.01\mu m$ and an image is recorded for each position. Then the image set is processed by the principles explained previously. A second method based on image correlation is also implemented and used as a reference to compare the results obtained by our algorithm. Fig.9 presents the reconstructed positions. Displacement non-linearity is due to PZT behavior. We can see that the two curves are very close from each other; the standard deviation of the position difference is equal to $2.5nm$. This value gives a good idea of the method resolution. It can be compared to the $4\mu m$ pattern period and to an area of $0.4\mu m \times 0.4\mu m$ equivalent to a single image pixel on the target. The resolution level is thus better than $10^{-2}$ image pixel. Using the same mechanical system we compared the position measurements obtained from the pattern images with the position given by the capacitive sensor of the PZT. For this experiment voltage steps of 0.1V were applied to the PZT and 20 images were recorded at each step. Results are depicted in Fig.10. The behavior is correct even if the capacitive sensor has a response time much shorter than the integration time of the camera and thus presents sharper variations. The system reproducibility was also measured in both X and Y directions by maintaining the PZT static. Retrieved position values are plotted in Fig.11. The peak-valley deviations are $21.4nm$ and $14.5nm$ in X and Y directions respectively while the standard deviations are $8.3nm$ and $5.1nm$. For the pattern orientation, the angle deviations obtained are respectively $1.9 \times 10^{-3}deg$.
<figure><img src="image_10.png" /><figcaption>Fig. 9. Comparison of positions reconstructed from pseudo-periodic pattern processing and from sub-pixel image correlation.</figcaption></figure>

<figure><img src="image_11.png" /><figcaption>Fig. 10. Comparison of displacement measurements by means of two methods: correlation and pseudo-pattern image processing.</figcaption></figure>

and $4.9 \times 10^{-4} \text{deg.}$ for the peak-valley and standard deviations.

<figure><img src="image_12.png" /><figcaption>Fig. 11. Evolution of the retrieved position values while the PZT is static.</figcaption></figure>

One particularly interesting property of the method concerns its capability to perform pattern position identification from out of focus images as illustrated in Fig.12. A 20x microscope objective with 0.5 numerical aperture is used for observing the pseudo-periodic pattern. The resulting depth of focus is

about $2.9\mu m$ as shown by the vertical lines in the figure. The solid line describes the evolution of a contrast parameter while the pattern is scanned through focus in front of the lens. The focus range allowing correct position retrieval is delimited by the two black squares and correspond to almost ten times the depth of focus of the lens used. This capability results from the filtering of sharp spatial frequencies in the Fourier spectrum and in robust software implementation. The extreme images allowing position retrieval are shown on the bottom of the figure and compared to the in-focus image. We can see that the pattern period is relatively far from the Niquist limit of two pixels per period and this is a first reason of the low method sensitivity to defocusing. (The side lobe observed in the contrast parameter is due to some imaging effect due to the pattern periodicity). The loss in position measurement resolution due to defocusing has still to be investigated experimentally but the possibility to perform correct measurement on such an extended focus range is of great interest, for instance for application in actuator localization in micro-robotics.

<figure><img src="image_13.png" /><figcaption>Fig. 12. Top: Evolution of a contrast parameter while the pseudo-periodic pattern is scanned through focus. Black squares indicate the interval in which position retrieval works. Bottom: Extreme images leading to correct position identification compared to the best focus image.</figcaption></figure>

### C. Application to live cell culture boxes

The method was used for a biology application. The problem is to find several times the same location of a specimen under an optical microscope while the specimen is transferred from the microscope stage to the culture chamber after each observation. Accurate image superimposition are expected in order to evaluate the modifications occurring in the biological medium versus time. For this purpose, the pseudo periodic pattern printed in the Pétri dish is used. The microscope is firstly focused on the pseudo-periodic pattern and its image is used to compute the position of the current view. Then the distance from the zone of interest is derived and the specimen is shifted in order to retrieve the right location. Following this procedure, we are able to focus rapidly the microscope on the expected zone of the biological specimen and to record
an updated image of the medium under test. This process is illustrated in Fig.13. For convenience of observation, the black and white images are put in the green and red channels of a RGB image. These images are then displayed in a common reference system thanks to the absolute position given by the pseudo-periodic pattern. Fig.13.a presents the positions at the beginning of the procedure. The green and red images are far from each other and the specimen has to be shifted towards the zone of interest. Fig.13.b presents the position after a first manual shift of the Pétri dish with the microscope stage. We have a partial overlap but the sample has still to be shifted. After a second adjustment, an excellent zone overlapping is obtained as can be seen in the pattern images of Fig.13.c. By applying numerically the same coordinate adjustment to the biological specimen images, a perfect superimposition is obtained as depicted in Fig.13.d. Once a sequence of images of the same zone has been recorded during the necessary lapse of time, the biological interpretation can be done from excellent data.

<figure><img src="image_14.png" /><figcaption>Fig. 13. Progressive localization of a region of interest. a) At the beginning, the current view is far from the expected one; b) Slight view overlap after a first sample shift; c) Excellent view overlap after second position adjustment; resulting biological image superimposition. (B&W images are put in the red and green channels of a RGB image for visual convenience).</figcaption></figure>

## V. CONCLUSION

We present a pseudo-periodic pattern suitable for position encryption of an extended 2D surface as well as subpixel resolution in position reconstruction. The position is retrieved from any view of a local zone of small dimensions with respect to encoded one. The processing of the local fraction of the pseudo-periodic pattern allows the accurate localization of the current view within the whole encrypted surface in both position and orientation. We firstly describe the method of surface encoding by a distorted periodic grid of points as well as the use of linear feedback shift register sequences for efficient encryption of position. Then we present the

image processing developed for position reconstruction. It is based on two consecutive steps providing with coarse and fine measurements that are combined with each other. The performing of phase computations relative to the periodic frame of points is responsible for the high resolution obtained. System performances are evaluated by inducing nanometer displacements with a piezoelectric transducer and by comparing results obtained to reference data. Finally, we present how the technique can be applied to the position referencing of cell culture boxes in order to come back to chosen regions of interest on the microscope stage while the biological specimen make round trip travels from microscope stage to culture chamber. Excellent biological image superimposition are obtained for further interpretation of biological process from excellent specimen documentation.

The technique could also be used for micro-mechatronics actuator localization over extended work planes and benefit from subpixel resolution as well as out-of-focus detection capabilities.

## ACKNOWLEDGMENT

We would like to acknowledge Région de Franche-Comté for project funding (UFC 39SB) and Institut National du Cancer (INCa) for supporting PhD work of July Andrea Galeano Zea.

## REFERENCES

[1] M. Balci and H. Foroosh, *Subpixel Registration Directly from the Phase Difference*. EURASIP Journal on Applied Signal Processing, Volume 2006, Article ID 60796, Pages 1-11.

[2] P. Sandoz, J. Ravassard, J. Dembele and A. Janex, *Phase-sensitive vision technique for high accuracy position measurement of moving targets*. IEEE Transactions on Instrumentation and Measurement 49(4),867872 (2000).

[3] P. Sandoz, V. Bonnans and T. Gharbi, *High-accuracy position and orientation measurement of extended two-dimensional surfaces by a phase-sensitive vision method*. Applied optics 41(26), 55035511 (2002).

[4] P. Sandoz, R. Zeggari, L. Froehly, J. Pretet and C. Mougin, *osition referencing in optical microscopy thanks to sample holders with out-of-focus encoded patterns*. Journal of Microscopy 225(3), 292302 (2007).

[5] S. Golomb, *Shift register sequences*. Aegean Park Press Laguna Hills, CA, USA (1981).

[6] Takeda, M. and Mutoh, K., *Fourier transform profilometry for the automatic measurement of 3-D object shapes*, Appl. opt., **22**(24), 3977-3982, 1983.