# Efficient Separation Between Projected Patterns for Multiple Projector 3D People Scanning

Tomislav Petković, Tomislav Pribanić, Matea Donlić, Peter Sturm

► To cite this version:

Tomislav Petković, Tomislav Pribanić, Matea Donlić, Peter Sturm. Efficient Separation Between Projected Patterns for Multiple Projector 3D People Scanning. PeopleCap ICCV-Workshop: Capturing and modelling human bodies, faces and hands, Oct 2017, Venice, Italy. pp.815-823, 10.1109/ICCVW.2017.101. hal-04722811

HAL Id: hal-04722811

https://inria.hal.science/hal-04722811v1

Submitted on 6 Oct 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# Efficient Separation between Projected Patterns for Multiple Projector 3D People Scanning

Tomislav Petković, Tomislav Pribanić and Matea Đonlić

University of Zagreb Faculty of Electrical Engineering and Computing
Unska 3, HR-10000 Zagreb, Croatia

{tomislav.petkovic.jr, tomislav.pribanic, matea.donlic}@fer.hr

Peter Sturm

INRIA Grenoble Rhône-Alpes

Inovallée, 655 Avenue de l'Europe, Montbonnot, 38334 Saint Ismier Cedex, France

peter.sturm@inria.fr

## Abstract

Structured light 3D surface scanners are usually comprised of one projector and of one camera which provide a limited view of the object's surface. Multiple projectors and cameras must be used to reconstruct the whole surface profile. Using multiple projectors in structured light profilometry is a challenging problem due to inter-projector interferences which make pattern separation difficult. We propose the use of sinusoidal fringe patterns where each projector has its own specifically chosen set of temporal phase shifts which together comprise a DFT<sub>2P+1</sub> basis, where P is the number of projectors. Such a choice enables simple and efficient separation between projected patterns. The proposed method does not impose a limit on the number of projectors used and does not impose a limit on the projector placement. We demonstrate the applicability of the proposed method on three projectors and six cameras structured light system for human body scanning.

## 1. Introduction

Structured light (SL) 3D surface scanning using sinusoidal patterns, the fringe projection profilometry (FPP), is one of the most robust and well accepted techniques in practical applications including (but not limited to) human body scanning, digital model generation, reverse engineering, rapid prototyping, and quality control [7]. In FPP a projector projects a sinusoidal fringe pattern onto an object which is then recorded by a camera. The phase of the deformed fringe pattern observed by the camera encodes all information required to reconstruct the surface profile.

In many applications of FPP a complete object's surface

profile is required. Obtaining a complete surface profile is impossible using only one projector and only one camera due to occlusions and limited fields of view. While adding a camera to a FPP system is simple, adding a projector to increase the illuminated area is a challenging problem due to interference between projected patterns. Indeed, almost all of SL patterns described in the literature [21, 22, 1] are designed for a single projector. Considering that the only option for using the existing SL patterns in a multi-projector SL system is to use projectors in turns which eliminates any interference [5].

There are currently only several multi-projector FPP systems described in the literature which employ simultaneous pattern projection. Servin et al. [24, 25] describe a multi-projector single-camera system which relies on a very precise spatial placement of projectors (so called co-phased profilometry), and therefore has limited general applicability. Woolford and Burnett [29] describe a more general two-projector multi-camera system which uses spatial fringe multiplexing where spatial orientations of fringe patterns vary between projectors: one projector uses vertical and another horizontal fringes. Such spatial fringe multiplexing is commonly used in single-projector FPP systems [31, 27], however, in multi-projector FPP systems it places undesirable constraints on spatial projector placement and is not easily extendable to an arbitrary number of projectors. Interestingly, multi-projector FPP systems which use an acquisition strategy where projectors are employed sequentially are also rarely described [26, 5].

Non FPP-based multi-projector multi-camera systems are also uncommon in the literature. Sagawa et al. [20, 19] describe a multi-projector system which uses line patterns with different spatial line orientations. Similar to spatial
multiplexing in FPP systems [29, 31] using different spa-
tial orientations places undesirable constraints on spatial
projector placement. Yan et al. [30] describe a theoretical
multi-projector system which combines De Bruijn color se-
quences and random dot patterns; unfortunately, their work
only provides simulated results making practical applicabil-
ity questionable.

We propose a novel multi-projector multi-camera struc-
tured light surface scanning system which uses gray-level
sinusoidal fringe patterns where each projector has its own
carefully chosen set of temporal phase shifts. The idea
of separating mixed sinusoids from a sufficient number
of observations was previously applied to single projector
FPP systems [16, 9, 8] only. In our approach for a scan-
ner using *P* projectors we select temporal phase shifts to
comprise a discrete Fourier transform in 2*P* + 1 points
(DFT<sub>2*P*+1</sub>) which enables simple and efficient decoupling
of projected fringes. Compared to existing multi-projector
multi-camera FPP systems the proposed method does not
impose a limit on the number of projectors used and does
not impose constraints on the projector placement. To the
best of our knowledge there is currently no multi-projector
multi-camera FPP system described in the literature which
does not impose constraints on projector placement.

This paper is organized as follows: In Section 2 we
present the proposed method. In Section 3 we compare the
proposed method to classical sequential employment. Re-
sults and discussion are presented in Section 3. We con-
clude in Section 5.

## 2. The Proposed Method

Before presenting the proposed method we give a brief
review of a classical one-projector temporal phase shift-
ing strategy [2, 11]. We then extend this first to the two-
projectors case in Section 2.2 and then to the arbitrary num-
ber of projectors in Section 2.3.

2.1. Classical Temporal Phase Shifting

Classical approaches to temporal phase shifting use one
projector only. The projector projects a set of *N* phase-
shifted gray-level sinusoidal fringes which are observed by
one or more cameras sequentially in time. Each of *N* pro-
jected fringes has a different temporal phase shift $\varphi[n]$ given
by

$$
\varphi[n] = 2\pi n / N, \quad n = 0, \dots, N - 1, \qquad (1)
$$

where *n* is a frame number (time-step). The intensity of the
*n*th fringe in projector’s coordinate system ($x_{PRJ}$, $y_{PRJ}$) is:

$$
I_{PRJ}(x_{PRJ}, y_{PRJ}) = I_0 \left( 1 + \cos(\omega x_{PRJ} + \varphi[n]) \right) / 2, \quad (2)
$$

where $I_0$ is projector's intensity and where $\omega$ is spatial
frequency. The intensity of the $n$th fringe observed by

the camera and expressed in camera's coordinate system
($x_{CAM}, y_{CAM}$) is:

$$
I_{\text{CAM}}(x_{\text{CAM}}, y_{\text{CAM}}) = I_{\text{AMB}} + hI_{\text{PRJ}}, \quad (3)
$$

where $I_{\text{AMB}}$ is a constant ambient illumination and where $h$,
$0 \le h < 1$, models pixel-dependent channel loss. Combin-
ing Eqs. (2) and (3) yields:

$$
I_{\text{CAM}}(x_{\text{CAM}}, y_{\text{CAM}}) = a + b \cos(\omega x_{\text{PRJ}} + \varphi[n]), \quad (4)
$$

where $a = I_{\text{AMB}} + \frac{1}{2}hI_0$ is a time-invariant component
comprised of both ambient and projector illumination and
where $b = \frac{1}{2}hI_0$ is the amplitude of the observed fringe
or contrast. Eq. (4) defines a spatio-temporal signal where
$\varphi[n]$ are temporal phase shifts and where $\omega x_{\text{PRJ}} = \Phi$ is
a spatial phase. The spatial phase $\Phi$ cannot be recovered
directly from the $N$ acquired frames. Instead, a wrapped
phase $\phi \equiv \Phi (\text{mod } 2\pi)$ is recovered from Eq. (4) using

$$
\phi = \operatorname{atan2}(Y, X) \tag{5}
$$

where

$$
\begin{align*}
Y &= - \sum_{n=0}^{N-1} I_{\text{CAM}}(x_{\text{CAM}}, y_{\text{CAM}}) \sin(\varphi[n]) \\
X &= \sum_{n=0}^{N-1} I_{\text{CAM}}(x_{\text{CAM}}, y_{\text{CAM}}) \cos(\varphi[n])
\end{align*}
$$

The wrapped phase $\phi$ is then unwrapped to obtain $\Phi$ using
one of the well known phase unwrapping algorithms [32, 6].
Then for each camera pixel ($x_{\text{CAM}}, y_{\text{CAM}}$) the spatial pro-
jector coordinate $x_{\text{PRJ}}$ is recovered from the spatial phase $\Phi$
using

$$
x_{\text{PRJ}} = \frac{\Phi}{\omega}. \qquad (6)
$$

Finally, the surface profile is reconstructed via triangulation
[10].

2.2. Two Projectors

Let us now consider the case of two projectors where
both projectors simultaneously project sinusoidal fringes
patterns so camera observes an additive combination of two
sinusoidal fringes for each of *N* recorded frames.

Adding the contribution of the second projector to
Eq. (4) yields

$$
I_{\text{CAM}} = a + b_1 \cos(\omega_1 x_1 + \varphi_1) + b_2 \cos(\omega_2 x_2 + \varphi_2), \quad (7)
$$

where $b_k$, $\omega_k$ and $\varphi_k$ are contrast, spatial frequency, and
temporal phase shift of $k$th projector (spatial coordinates are
omitted for clarity). In Eq. (7) the term $a$ contains the over-
all contribution of ambient illumination and of both projec-
tors,

$$
a = I_{\text{AMB}} + \frac{1}{2}hI_1 + \frac{1}{2}hI_2, \qquad (8)
$$
where $I_1$ and $I_2$ replace $I_0$ of Eq. (2). Let $\phi_1 = \omega_1 x_1$ and $\phi_2 = \omega_2 x_2$ be spatial wrapped phases of the first and the second projector respectively. Then

$$I_{\text{CAM}}[n] = a+b_1 \cos(\phi_1+\varphi_1[n])+b_2 \cos(\phi_2+\varphi_2[n]), \quad (9)$$

which may be decomposed into a sum of five complex exponentials

$$I_{\text{CAM}}[n] = ae^{j0} + \frac{1}{2}b_1 e^{j\phi_1} e^{j\varphi_1[n]} + \frac{1}{2}b_1 e^{-j\phi_1} e^{-j\varphi_1[n]} \\ + \frac{1}{2}b_2 e^{j\phi_2} e^{j\varphi_2[n]} + \frac{1}{2}b_2 e^{-j\phi_2} e^{-j\varphi_2[n]}. \quad (10)$$

Recall a standard notation from digital signal processing [15] where a time-discrete complex exponential with the period $N$ is denoted by

$$W_N^n = \exp(-2\pi jn/N). \quad (11)$$

Specifically, $W_N$ is the $N$th root of unity and

$$\begin{aligned} \langle W_N^{nk_1}, W_N^{nk_2} \rangle &= \sum_{n=0}^{N-1} W_N^{-nk_1} W_N^{nk_2} \\ &= \begin{cases} 0, & k_1 \not\equiv k_2 \pmod{N} \\ N, & k_1 \equiv k_2 \pmod{N} \end{cases} \end{aligned} \quad (12)$$

holds. We say that time-discrete complex exponentials of period $N$ form an orthogonal basis with $N$ basis vectors $v_k[n] = W_N^{nk}$, where $n = 0, \dots, N-1$ is time-step and where $k = 0, \dots, N-1$ is basis vector index.

We propose that each projector uses a different set of temporal phase shifts so

$$\varphi_1[n] = 2\pi n/N \text{ and } \varphi_2[n] = 4\pi n/N = 2\varphi_1[n]. \quad (13)$$

Eq. (10) may be rewritten using Eqs. (11) and (13) to obtain

$$I_{\text{CAM}}[n] = aW_N^{0-n} + \frac{1}{2}b_1 e^{j\phi_1} W_N^{-n} + \frac{1}{2}b_1 e^{-j\phi_1} W_N^{n} \\ + \frac{1}{2}b_2 e^{j\phi_2} W_N^{-2n} + \frac{1}{2}b_2 e^{-j\phi_2} W_N^{2n}, \quad (14)$$

which is a decomposition of the time-discrete signal $I_{\text{CAM}}[n]$ into a sum of complex exponentials $W_N^{nk}$. The decomposition of Eq. (14) is recognizable as the discrete Fourier transform in $N$ points of $I_{\text{CAM}}[n]$ [15]. More specifically

$$\begin{aligned} \hat{I}_{\text{CAM}}[k] = \text{DFT}_N[I_{\text{CAM}}[n]] = & \\ = & \left\{ \frac{N}{2}b_2 e^{j\phi_2}, \frac{N}{2}b_1 e^{j\phi_1}, \underline{Na}, \frac{N}{2}b_1 e^{-j\phi_1}, \frac{N}{2}b_2 e^{-j\phi_2} \right\}. \end{aligned} \quad (15)$$

We say $\hat{I}_{\text{CAM}}[k]$ is a five component discrete spectrum of $I_{\text{CAM}}[n]$. In Eq. (15) the DC spectral component models the ambient illumination and mean projector intensities. Two additional spectral components are required to model each

projector, $k = \pm 1$ for the first projector and $k = \pm 2$ for the second. If the number of temporal phase shifts $N$ satisfies $N \ge 2 \cdot 2 + 1 = 5$ then all complex exponentials in Eq. (14) are orthogonal and the decomposition of $N$ observed frames into contributions of each projector is possible.

To recover the $k$th wrapped phase $\phi_k$ the fast Fourier transform [4] may be used to pixel-wise decompose $N$ acquired frames and compute the spectrum (15). The wrapped phase $\phi_k$ is the argument of the spectrum at the index $k$:

$$\phi_k = -\operatorname{Arg}(\hat{I}_{\text{CAM}}[k]) = -\operatorname{Arg}\left(\frac{N}{2}b_k e^{-j\phi_k}\right). \quad (16)$$

Similarly, the contrast $b_k$ may be computed from the absolute value of the spectrum at the index $k$:

$$b_k = \frac{2}{N} |\hat{I}_{\text{CAM}}[k]|. \quad (17)$$

Although the contrast $b_k$ is not required for phase recovery it is used to determine if a particular camera pixel is illuminated by the $k$th projector. The value of contrast is compared to some threshold $T$ and pixels for which

$$b_k > T \quad (18)$$

holds are considered illuminated by the $k$-th projector. The threshold $T$ is best set as a percentage of $\frac{1}{2}I_k$, e.g. setting $T$ to 10% of $\frac{1}{2}I_k$ requires that channel loss $h$ is no lower than 0.1. Once wrapped phases $\phi_1$ and $\phi_2$ are computed the surface profile may be reconstructed for each projector-camera pair as was described in Section 2.1.

## 2.3. Arbitrary Number of Projectors

Let us now consider the arbitrary number of projectors where all projectors project their sinusoidal fringes simultaneously. The camera now observes an additive combination of all fringes in each of $N$ acquired frames.

Let $P \ge 1$ be the number of projectors. Then Eq. (9) generalizes to

$$I_{\text{CAM}}[n] = a + \sum_{k=1}^{P} b_k \cos(\phi_k + \varphi_k[n]). \quad (19)$$

Instead of a single sinusoidal fringe we are observing a sum of $P$ fringes. We propose to select the $P$ temporal phase shifts $\varphi_k[n]$ so

$$\varphi_k[n] = 2\pi kn/N, \quad k = 1, \dots, P. \quad (20)$$

Using the selected temporal phase shifts Eq. (19) transforms to

$$I_{\text{CAM}}[n] = a + \sum_{k=1}^{P} \frac{1}{2} b_k (e^{j\phi_k} W_N^{-kn} + e^{-j\phi_k} W_N^{kn}), \quad (21)$$

which is a discrete Fourier transform decomposition of the signal $I_{\text{CAM}}[n]$ in $2P + 1$ points and is a generalization of
Eq. (14). In the proposed decomposition the ambient illumination and the combined mean projector intensity of all P projectors are modeled by the term a which has the basis vector $W_N^{0:n}$. The sinusoidal fringe of the kth projector is modeled by a single summand of the sum in Eq. (21) which uses two basis vectors $W_N^{-k:n}$ and $W_N^{k:n}$. As each projector uses a different phase-shift step k the decomposition of Eq. (21) is unambiguous if

$$N \geq 2P + 1. \qquad (22)$$

Wrapped phases and contrast may then be recovered using Eqs. (16) and (17). Applying a phase unwrapping algorithm [32, 6] recovers coordinates $x_{PRJ}$ which together with known $x_{CAM}$ and $y_{CAM}$ enable surface reconstruction via triangulation [10].

To summarize, the proposed method for multi-projector multi-camera FPP scanning is comprised of the following steps:

1. For each of P projectors generate a set of $N \geq 2P + 1$ fringe patterns using Eq. (2) where the temporal phase shift of kth projector is given by Eq. (20).

2. Set P projectors to simultaneously project fringes so each camera observes N frames containing an additive combination of projected fringes.

3. For each camera decode the set of N observed frames as follows:

(a) Decompose in time the set of N frames using the fast Fourier transform in N points.

(b) Determine the area illuminated by the kth projector using Eq. (18).

(c) Recover the wrapped phase $\phi_k$ for the area illuminated by the kth projector using Eq. (16).

(d) Unwrap the wrapped phase $\phi_k$ using any of the algorithms from [32, 6] to obtain $\Phi_k$.

(e) Recover the projector coordinate $x_{PRJ,k}$ of kth projector from $\Phi_k$ using Eq. (6).

4. Triangulate object's surface using all collected camera and projector coordinate pairs.

### 3. Comparison to Sequential Employment

The simplest data acquisition strategy for multi-projector multi-camera SL system which avoids inter-projector interference is to use each projector in turn [26, 5], a so-called sequential projector employment. The proposed method uses projectors simultaneously which allows the reduction in the total number of acquisitions, where an acquisition means a simultaneous frame capture on all cameras.

<figure><img src="image_2.png" /><figcaption>Figure 1. Multi-projector multi-camera structured light 3D scanner comprised of three projectors and six cameras mounted on three vertical poles.</figcaption></figure>

<figure><img src="image_3.png" /><figcaption>Figure 2. Spatial arrangement for 3D scanner. One projector and one camera-pair are mounted on the same vertical pole as follows: C1-P1-C2 on the first pole, C3-P2-C4 on the second, and C5-P3-C6 on the third. Points for one position of a planar calibration board are shown as black crosses.</figcaption></figure>

For a single projector the minimum number of sinusoidal fringes to project is 3 [2, 3], so for P projectors employed sequentially the minimum number of acquisitions is $3P$. The proposed method reduces $3P$ to $2P + 1$, which is a clear advantage. Furthermore, in practice the number of phase shifts is never the minimum 3 due to noise sensitivity; common choices are 7 or more shifts [3]. For the proposed method all acquired frames contain data of more (all) projectors, therefore for three or more projectors we achieve 7 or more shifts for each projector as $2P + 1 > 7$.

A disadvantage of the proposed method compared to sequential employment is that simultaneous employment makes camera's dynamic range shared between projectors. In an ideal one-projector setup a whole camera's dynamic
<figure><img src="image_4.png" /><figcaption>Figure 4. A planar calibration board used for quantitative analysis as seen by C4: (a) board under white illumination; (b) board under illumination by P3 using $\omega_2$; and (c) board under illumination by all three projectors simultaneously. Note the difference between the pure fringe in (b) and the multi-projector interference pattern in (c).</figcaption></figure>

range *D* is allocated to that projector. However, in multi-projector setup each camera observes the sum which reduces the maximal dynamic range allocated to one projector to *D*/*P*. This reduction increases measurement noise and may affect wrapped phase estimation. Fortunately, the loss of resolution in amplitude due to sharing of the dynamic range may be compensated by using a better camera or by increasing the temporal resolution by using more phase shifts.

In practice the loss of resolution in amplitude has a firm limit which is determined by the number of projectors with overlapping FOVs. That limit is normally less than the total number of projectors used, e.g. to scan a complete human body *P* projectors are placed to surround the human which in practice limits the overlap to about 2 or 3 projectors instead of *P*.

In the context of multiplexed-illumination as proposed by Schechner et al. [23] a multi-projector SL scanner operates in saturation-limited range: camera exposure must be fixed to match the common refresh rate of all projectors, therefore only lens iris is used to avoid oversaturation. Then, comparing sequential acquisition using *M* shifts for each of *P* spatial frequencies with the proposed simultaneous acquisition using *N* = 2*P* + 1 shifts yields an SNR gain of *N*/*M* in the signal amplitude (contrast) *b* and of (*N* · *B*<sub>1</sub><sup>2</sup>)/(*M* · *B*<sub>2</sub><sup>2</sup>) in the signal phase $\varphi$ [14, 12], where *B*<sub>1</sub> = *D*/*P* and *B*<sub>2</sub> = *D* are fringe modulations. In practice using an 8-bit camera is sufficient to separate patterns of at least three overlapping projectors.

## 4. Results and Discussion

To demonstrate the applicability of the proposed method we have built a SL scanner comprised of three projectors and six cameras shown in Fig. 1. Of three projectors one is Canon LV-WX310ST (P1) and two are Acer S1383WHne (P2,P3). All projectors are operated at their

<table><caption>Table 2. Absolute mean error of recovered projector coordinate $x_{PRJ}$ expressed in projector pixels and its standard deviation for a planar calibration board shown in Fig. 4. Note the average error is in subpixel range at about 0.14 px.</caption><thead><tr><th></th><th>P1</th><th>P2</th><th>P3</th></tr></thead><tbody><tr><td>C1</td><td>0.15 ± 3.26</td><td>0.10 ± 0.08</td><td>0.12 ± 0.12</td></tr><tr><td>C2</td><td>0.15 ± 2.73</td><td>0.09 ± 0.08</td><td>0.11 ± 0.10</td></tr><tr><td>C3</td><td>0.16 ± 3.72</td><td>0.13 ± 0.12</td><td>0.17 ± 4.23</td></tr><tr><td>C4</td><td>0.21 ± 5.52</td><td>0.21 ± 5.31</td><td>0.15 ± 4.20</td></tr><tr><td>C5</td><td>0.12 ± 0.11</td><td>0.11 ± 0.10</td><td>0.15 ± 3.68</td></tr><tr><td>C6</td><td>0.18 ± 4.08</td><td>0.13 ± 0.12</td><td>0.15 ± 3.05</td></tr></tbody></table>

native resolution of 1280 × 800. All six cameras are Point-Grey's Grasshopper3 GS3-U3-23S6C-C; four cameras are fitted with Fujinon HF12.5SA-1 lenses (C1,C2,C3,C4) and two with Kowa LM8JCM lenses (C5,C6). The scanner is synchronized in software and acquires images at 20 FPS. The system was calibrated using a planar calibration board (Fig. 4) and the method of [17, 10]. The spatial arrangement of projectors and cameras is shown in Fig. 2; note that Fig. 1 is a view toward the scanner and Fig. 2 is a view toward the object. Such an arrangement was deliberately chosen to maximize projector overlap and to better demonstrate the proposed method; there is no camera-projector pair for which the common field-of-view is empty. In real-world applications and especially in human body scanning a setup where projectors and cameras surround the object would normally be used.

We have selected a multiple-phase shifting (MPS) unwrapping strategy [18] with three spatial frequencies $\omega_0 = 2\pi \frac{13}{W}$, $\omega_1 = 2\pi \frac{17}{W}$, and $\omega_2 = 2\pi \frac{21}{W}$, where *W* = 1280 px is projector width. Fringes are projected in a round-robin fashion: first P1 projects $\omega_0$, P2 $\omega_1$, and P3 $\omega_2$ using *N* phase-shifts each; then *p*th projector switches its spatial frequency from $\omega_m$ to $\omega_{m+1}$ (mod 3) etc. The number of phase shifts was set to *N* = 12. The phase steps were chosen to be 1, 3, and 5 so P1 used $\varphi_1[n] = \frac{1}{6}\pi n$, P2 $\varphi_2[n] = \frac{1}{2}\pi$, and P3 $\varphi_3[n] = \frac{5}{6}\pi n$. We have performed several experiments using these parameters: (i) a quantitative comparison of wrapped phases; (ii) a quantitative comparison of recovered projector coordinate; and (iii) a 3D reconstruction of a person. Concerning the operating scanner frame rate of 20 FPS, the number of 12 phase shifts, the number of 3 frequencies, and the number of 3 projectors involved, the acquisition times for sequential and simultaneous scanning are approximately 5.5 and 1.8 seconds respectively.

We quantitatively compare wrapped phases and recovered projector coordinate to the classical MPS strategy which is taken as the ground truth. Recorded object is a
<figure><img src="image_5.png" /><figcaption>Figure 3. Spectral decomposition for [C5]. Twelve input frames are decomposed using the proposed method to obtain contributions of each individual projector. Note that $\phi_1$ has spatial frequency $\omega_0 = 2\pi \frac{13}{W}$, $\phi_2$ has $\omega_1 = 2\pi \frac{17}{W}$, and $\phi_3$ has $\omega_2 = 2\pi \frac{21}{W}$.</figcaption></figure>

stationary calibration board (Fig. 4) which was first imaged using sequential employment (classical MPS) and then using the proposed method (simultaneous employment). Wrapped phases are compared using the absolute circular distance $d_{\text{circular}}(\cdot, \cdot)$ as the error metric:

$$d_{\text{circular}}(\phi, \phi_{\text{MPS}}) = |\text{Arg}(e^{i\phi}/e^{i\phi_{\text{MPS}}})|, \quad (23)$$

where $\phi$ is the wrapped phase obtained using Eq. (16) and $\phi_{\text{MPS}}$ is the ground truth. The error metric was computed for illuminated pixels only using the threshold $T$ of 5%. Table 1 lists errors for all $3 \cdot 3 \cdot 6 = 54$ (number of spatial frequencies used times number of projectors times number of cameras) wrapped phases. On average the wrapped phase error is $0.0168$ rad = $0.96^\circ$. The absolute error in the recovered projector coordinate, $|x_{\text{PRJ}} - x_{\text{PRJ,MPS}}|$, is listed in Table 2 for all $3 \cdot 6 = 18$ camera-projector combinations. On average the projector coordinate error is $0.14$ px. Note that the distribution of errors is not Gaussian and that quantitative results are presented using non-filtered data. Overall, almost all (> 95%) points have error below the twice the mean and only a small number of outliers affects the deviation for some views. These quantitative results clearly indicate the applicability of the proposed method.

For a qualitative evaluation of the proposed scanner we

have recorded a female model in an office during the day under normal ambient illumination. An example of spectral decomposition of the first 12 input frames for [C5] is shown in Fig. 3. Note the following: (1) the DC component $a$ contains both ambient and average illumination of all projectors; (2) the contrast $b_k$ indicates which areas are well illuminated by $k$th projector; and (3) the wrapped phase $\phi_k$ is recovered even for poorly illuminated areas as $\phi_k$ is mostly independent of $b_k$. The spectral decomposition is then applied to all 36 input frames to obtain three wrapped phases for each of six cameras. Wrapped phases may then be unwrapped using the method of [18]. An example of nine wrapped phases and of recovered projector coordinates is shown in Fig. 5. In Figs. 3 and 5 processing results are shown for all pixels, i.e. even pixels which are not illuminated according to Eq. (18) were processed.

Once projector coordinates are known for each camera a 3D point cloud is obtained via triangulation [10]. Recovered 3D surface points may be textured using the value of $a$ as the gray-level intensity. Furthermore, if cameras are recording in color then the DC spectral coefficient $a$ may be computed for each color channel separately. This yields a full-color texture with the sinusoidal fringe removed: there is no need to acquire an additional frame with all projec-
36 input frames in three groups of 12 frames are first decomposed into nine wrapped phases from which $x_{PRJ}$ is computed.

<figure><img src="image_6.png" /><figcaption>Figure 5. Recovered wrapped phases and projector coordinates for [C2]. Note different shadows in the projector coordinate $x_{PRJ}$ for three projectors.</figcaption></figure>

<table><caption>Table 1. Absolute wrapped phase error of recovered wrapped phases $\phi$ in degrees and its standard deviation for a planar calibration board shown in Fig. 4.</caption><thead><tr><th></th><th colspan="3">P1</th><th colspan="3">P2</th><th colspan="3">P3</th></tr><tr><th></th><th>ω<sub>0</sub></th><th>ω<sub>1</sub></th><th>ω<sub>2</sub></th><th>ω<sub>0</sub></th><th>ω<sub>1</sub></th><th>ω<sub>2</sub></th><th>ω<sub>0</sub></th><th>ω<sub>1</sub></th><th>ω<sub>2</sub></th></tr></thead><tbody><tr><th>C1</th><td>0.91 ± 0.81</td><td>1.16 ± 1.75</td><td>0.91 ± 0.75</td><td>0.95 ± 0.74</td><td>0.75 ± 0.58</td><td>0.75 ± 0.60</td><td>0.76 ± 0.62</td><td>0.76 ± 0.62</td><td>1.07 ± 1.14</td></tr><tr><th>C2</th><td>0.95 ± 0.89</td><td>1.20 ± 1.85</td><td>0.94 ± 0.77</td><td>0.92 ± 0.73</td><td>0.58 ± 0.43</td><td>0.77 ± 0.56</td><td>0.77 ± 0.63</td><td>0.77 ± 0.62</td><td>0.99 ± 0.94</td></tr><tr><th>C3</th><td>1.12 ± 4.51</td><td>0.86 ± 0.70</td><td>0.89 ± 1.71</td><td>0.96 ± 0.93</td><td>0.96 ± 0.88</td><td>1.25 ± 1.23</td><td>0.86 ± 0.75</td><td>1.01 ± 1.82</td><td>0.88 ± 0.78</td></tr><tr><th>C4</th><td>1.41 ± 9.32</td><td>0.83 ± 0.93</td><td>0.86 ± 3.53</td><td>1.26 ± 4.91</td><td>1.06 ± 1.00</td><td>2.02 ± 11.24</td><td>0.84 ± 0.71</td><td>0.96 ± 0.86</td><td>0.86 ± 0.74</td></tr><tr><th>C5</th><td>0.79 ± 0.66</td><td>0.78 ± 0.65</td><td>1.15 ± 1.35</td><td>0.86 ± 0.80</td><td>1.02 ± 1.01</td><td>0.85 ± 0.80</td><td>1.04 ± 3.15</td><td>0.89 ± 0.73</td><td>0.89 ± 0.77</td></tr><tr><th>C6</th><td>0.82 ± 0.97</td><td>0.79 ± 0.72</td><td>1.28 ± 3.01</td><td>1.00 ± 0.89</td><td>1.32 ± 1.19</td><td>1.00 ± 0.87</td><td>1.01 ± 1.30</td><td>0.87 ± 0.73</td><td>0.87 ± 0.75</td></tr></tbody></table>
<figure><img src="image_7.png" /><figcaption>Figure 6. Textured point clouds for a recorded female model. Holes in the reconstructed point clouds originated from the limited camera dynamic range, required to capture both very dark and very bright surface parts.</figcaption></figure>

tors turned off (or projecting all-white pattern). A textured
point clouds for a female model are shown in Fig. 6. For
this reconstruction we have used the threshold *T* of 10%
which effectively removes dark areas such as model's hair
and boots together with parts of a hardwood floor. Note
that except for selection of illuminated points no additional
filtering or smoothing was performed on the data shown in
Fig. 6. We also point out that fitting a plane to the 3D re-
constructed planar object have shown essentially the same
result both for sequential and simultaneous approach (not
shown due to the space limitation).

The proposed method uses sinusoidal fringes which are
robust to projector and camera blur: the phase is unaffected
by blurring due to spatial symmetry of the blurring OTF
[13]. This is a highly desirable property and is a clear ad-
vantage of the proposed multi-projector FPP approach over
non-FPP multi-projector approaches such as [20, 19, 30].

The proposed method opens an exciting possibility of
combining FPP and photometric stereo [28] when many
projectors are used. Consider three images showing the re-
covered contrast coefficients $b_k$ which are shown in Fig. 3:
the viewing direction is constant while the direction of in-
cident illumination depends on the projector. If sufficient
number of viewing directions is known surface orientation
may be recovered from the contrast coefficients $b_k$.

5. Conclusion

We have proposed a method for efficient separation be-
tween projected sinusoidal fringe patterns in fringe projec-
tion profilometry which is based on a specific choice of tem-
poral phase shifts.

We have shown that for a system of *P* projectors a selec-
tion of temporal phase shifts exists which enables decompo-
sition of the projected fringes in terms of the basis vectors
of the discrete Fourier transform in 2*P* + 1 or more points.
Such decomposition in turn enables efficient separation be-
tween projected sinusoidal fringe patterns.

The proposed method does not impose a limit on the
number of projectors nor on their placement. This in turn
enables construction of large FPP systems comprised of
many projectors and cameras which may be particularly
useful in human body scanning.

Acknowledgment

This work has been supported by the Croatian Science
Foundation's funding of the project IP-11-2013-3717.

References

[1] J. Batlle, E. Mouaddib, and J. Salvi. Recent progress in coded structured light as a technique to solve the correspondence problem: a survey. *Pattern Recognition*, 31(7):963-982, 1998. 1

[2] K. Creath. V phase-measurement interferometry techniques. volume 26 of *Progress in Optics*, pages 349–393. Elsevier, 1988. 2, 4

[3] P. de Groot. Derivation of algorithms for phase-shifting interferometry using the concept of a data-sampling window. *Appl. Opt.*, 34(22):4723–4730, Aug 1995. 4

[4] M. Frigo and S. G. Johnson. FFTW: an adaptive software architecture for the FFT. In *Acoustics, Speech and Signal Processing, 1998. Proceedings of the 1998 International Conference on*, volume 3, pages 1381–1384 vol.3, May 1998. 3

[5] R. R. Garcia and A. Zakhor. Markerless motion capture with multi-view structured light. *Electronic Imaging*, 2016(21):3DIPM-050.1-3DIPM-050.7, 2016. 1, 4

[6] D. C. Ghiglia and M. D. Pritt. *Two-Dimensional Phase Unwrapping: Theory, Algorithms, and Software*. Wiley, May 1998. 2, 4

[7] S. S. Gorthi and P. Rastogi. Fringe projection techniques: Whither we are? *Optics and Lasers in Engineering*, 48(2):133–140, 2010. 1

[8] J. Gu, T. Kobayashi, M. Gupta, and S. K. Nayar. Multiplexed illumination for scene recovery in the presence of global illumination. In *2011 International Conference on Computer Vision*, pages 691–698, Nov 2011. 2

[9] M. Gupta and S. K. Nayar. Micro phase shifting. In *2012 IEEE Conference on Computer Vision and Pattern Recognition*, pages 813–820, June 2012. 2
[10] R. Hartley and A. Zisserman. *Multiple view geometry in computer vision*. Cambridge university press, 2003. 2, 4, 5, 6

[11] J. M. Huntley and H. O. Saldner. Shape measurement by temporal phase unwrapping: comparison of unwrapping algorithms. *Measurement Science and Technology*, 8(9):986, 1997. 2

[12] B. Kamgar-Parsi and B. Kamgar-Parsi. Evaluation of quantization error in computer vision. *IEEE Transactions on Pattern Analysis and Machine Intelligence*, 11(9):929-940, Sep 1989. 5

[13] B. Li, N. Karpinsky, and S. Zhang. Novel calibration method for structured-light system with an out-of-focus projector. *Appl. Opt.*, 53(16):3415-3426, Jun 2014. 8

[14] J. Li, L. G. Hassebrook, and C. Guan. Optimized two-frequency phase-measuring-profilometry light-sensor temporal-noise sensitivity. *J. Opt. Soc. Am. A*, 20(1):106-115, Jan 2003. 5

[15] S. K. Mitra. *Digital Signal Processing: A Computer-Based Approach*. McGraw-Hill Companies, 3 edition, Jan. 2005. 3

[16] D. Moreno, K. Son, and G. Taubin. Embedded phase shifting: Robust phase shifting with embedded signals. In *2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)*, pages 2301-2309, June 2015. 2

[17] D. Moreno and G. Taubin. Simple, accurate, and robust projector-camera calibration. In *2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization Transmission*, pages 464-471, Oct 2012. 5

[18] T. Petković, T. Pribanić, and M. Đonlić. Temporal phase unwrapping using orthographic projection. *Optics and Lasers in Engineering*, 90:34-47, 2017. 5, 6

[19] R. Sagawa, R. Furukawa, and H. Kawasaki. Dense 3d reconstruction from high frame-rate video using a static grid pattern. *IEEE Transactions on Pattern Analysis and Machine Intelligence*, 36(9):1733-1747, Sept 2014. 1, 8

[20] R. Sagawa, Y. Ota, Y. Yagi, R. Furukawa, N. Asada, and H. Kawasaki. Dense 3d reconstruction method using a single pattern for fast moving object. In *2009 IEEE 12th International Conference on Computer Vision*, pages 1779-1786, Sept 2009. 1, 8

[21] J. Salvi, S. Fernandez, T. Pribanić, and X. Lladó. A state of the art in structured light patterns for surface profilometry. *Pattern Recognition*, 43(8):2666-2680, 2010. 1

[22] J. Salvi, J. Pagès, and J. Balle. Pattern codification strategies in structured light systems. *Pattern Recognition*, 37(4):827-849, 2004. 1

[23] Y. Y. Schechner, S. K. Nayar, and P. N. Belhumeur. Multiplexing for optimal lighting. *IEEE Transactions on Pattern Analysis and Machine Intelligence*, 29(8):1339-1354, Aug 2007. 5

[24] M. Servin, G. Garnica, J. C. Estrada, and A. Quiroga. Coherent digital demodulation of single-camera n-projections for 3d-object shape measurement: Co-phased profilometry. *Opt. Express*, 21(21):24873-24878, Oct 2013. 1

[25] M. Servin, M. Padilla, G. Garnica, and A. Gonzalez. Profilometry of three-dimensional discontinuous solids by combining two-steps temporal phase unwrapping, co-phased profilometry and phase-shifting interferometry. *Optics and Lasers in Engineering*, 87:75-82, 2016. 1

[26] W.-H. Su, C.-Y. Kuo, C.-C. Wang, and C.-F. Tu. Projected fringe profilometry with multiple measurements to form an entire shape. *Opt. Express*, 16(6):4069-4077, Mar 2008. 1, 4

[27] L. X. Tang, J. Chen, and H. L. Chen. A novel structured light pattern for one-shot shape acquisition. In *Frontiers of Manufacturing Science and Measuring Technology III*, volume 401 of *Applied Mechanics and Materials*, pages 1191-1194. Trans Tech Publications, 12 2013. 1

[28] R. J. Woodham. Photometric method for determining surface orientation from multiple images. *Optical Engineering*, 19(1):191139-191139-, 1980. 8

[29] S. Woolford and I. S. Burnett. Toward a one shot multi-projector profilometry system for full field of view object measurement. In *2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)*, pages 569-573, May 2014. 1, 2

[30] Z. Yan, L. Yu, Y. Yang, and Q. Liu. Beyond the interference problem: hierarchical patterns for multiple-projector structured light system. *Appl. Opt.*, 53(17):3621-3632, Jun 2014. 2, 8

[31] Z. Zhang. Review of single-shot 3d shape measurement by phase calculation-based fringe projection techniques. *Optics and Lasers in Engineering*, 50(8):1097-1106, 2012. 1, 2

[32] C. Zuo, L. Huang, M. Zhang, Q. Chen, and A. Asundi. Temporal phase unwrapping algorithms for fringe projection profilometry: A comparative review. *Optics and Lasers in Engineering*, 85:84-103, 2016. 2, 4