# Improved Crosstalk Reduction on Multiview 3D Display by Using BILS Algorithm

Authors: Xiaoyan Wang; Chunping Hou
Journal: Journal of Applied Mathematics
 (2014)

---

## Abstract

In multiview three-dimensional (3D) displays, crosstalk is one of the most annoying artefacts degrading the quality of the 3D image. In this paper, we present a system-introduced crosstalk measurement method and derive an improved crosstalk reduction method. The proposed measurement method is applied to measure the exact crosstalk
among subpixels corresponding to different view images and the obtained results are very effective for crosstalk reduction method. Furthermore, an improved crosstalk reduction method is proposed to alleviate crosstalk by searching for the optimal integral intensity values of subpixels on the synthetic image. The derived algorithm based on modified Schnorr-Euchner strategy is implemented to seek the optimal solution to this box-constrained integer least squares (BILS) problem, such that the Euclidean distance between solution and its target decreases substantially. The method
we develop is applicable to both multiview 3D parallax barrier displays and multiview 3D lenticular displays. Both simulation and experimental results indicate that the derived method is capable of improving 3D image quality more effectively than the existing method on multiview 3D displays.


---

## Body

## 1. Introduction

Crosstalk is a critical factor affecting the image quality in multiview three-dimensional (3D) displays [1], which is caused by the incomplete isolation of different image channels. In order to mitigate the Moiré fringe [2] and balance the horizontal versus vertical resolution [3] of 3D displays, the slanted parallax barrier or lenticular lens array is used [4]. However, the shape of subpixels on the display screen is rectangular or triangle and so on, and the shape of visible image observed through a slanted slit cannot exactly coincide with boundary of subpixels. This causes the viewers to observe not only the intended view image but also the other unintended view images and leads to the crosstalk observed at the optimal viewing positions on multiview 3D displays.As introduced in [5], the available mathematical definitions of crosstalk are diverse and sometimes contradictory. In 2009, Huang et al. defined two terms [6]: system crosstalk that is independent of the content (determined only by the display) and viewer crosstalk that varies depending on the content. Most of literatures concerning the methodology [7, 8] for measuring system-introduced crosstalk on multiview 3D displays provided the methods based on the illumination measurement. However, it is often difficult to measure the amount of crosstalk exactly, due to the complexity of the system, lack of measurements, reluctance of manufacturers to release data, and difficulty of making the measurement [5].A number of methods [9, 10] for crosstalk reduction on multiview 3D displays have been proposed in the literatures to date. For instance, lowing aperture ratio and fixing a special 3D barriers are both effective for crosstalk cancellation. But these methods are at the cost of the intensity or extra devices. Recently, Li et al. [11] have suggested a method to eliminate the crosstalk by correcting the subpixels values on synthetic image. Despite the fact that this method to some extent reduced the crosstalk, the crosstalk coefficients matrix was obtained by the illumination measurement. The experiment error of crosstalk measurement influenced the results of crosstalk reduction. Moreover, how to obtain the solutions to the equations of crosstalk elimination was not mentioned.Few research efforts have been devoted to reducing crosstalk effectively on multiview 3D displays by taking it as a box-constrained integer least squares (BILS) problem. Lattice reduction [12] is a powerful mathematical tool for solving diverse problems involving point lattices. In [13], the enumeration strategies by Schnorr-Euchner, Pohst, and Kannan were introduced. It was found that the Schnorr-Euchner strategy is substantially faster than the other two [14]. An efficient search method based on the Schnorr-Euchner enumeration strategy (see [15]) for solving integer least squares (ILS) problem was implemented in [16]. The method was then modified in [17] to solve the BILS problem by taking the box constraint into account.In this paper, we extend the definition of system-introduced crosstalk to the multiview 3D displays and present a mathematical method for exactly measuring the system-introduced crosstalk observed at the optimal viewing position. Crosstalk between the neighboring view images is not always equal to the exact crosstalk between subpixels corresponding to different view images. Our method is applied to obtain the crosstalk coefficient matrixA, in which precise crosstalk between the subpixels is presented. The obtained results are more effective for crosstalk reduction method than for the one via the typical experiment. Moreover, we adopt the variable a to simulate the slight movement of viewable image, while the viewer is moving parallel to the display screen in the viewing zone such that the more exact crosstalk at the optimal viewing position is obtained. One of major contributions in this paper is the proposal of reasonable measurement method of system-introduced crosstalk between the neighbouring subpixels above and below corresponding to the different view images.In actual display devices, the subpixel values are constrained to the integer setZ. Without regard to it, the crosstalk elimination result still remains the biggist accumulated quantization error, which has the negative impact on image quality. That means the crosstalk elimination [11] on multiview 3D displays is not as complete as expected. Hence, it is effective by taking reduction process as a BILS problem. In this paper, we concentrate our efforts on developing a crosstalk reduction method, which gives a few modifications of Schnorr-Euchner strategy based algorithms given in [17], to correct for the synthetic image with the minimum quantization error. The experiments demonstrate that the proposed method reduces crosstalk effectively and greatly improves the 3D image quality. Moreover, based on the fact that the proposed method is based on the geometrical relationship of parallax barrier and subpixels, it is applied to eliminate crosstalk between the neighbouring subpixels above and below, rather than the neighbouring view images. If the distribution mode of view images changes on the synthetic images [18], that is, the neighbouring subpixels above and below do not correspond to the neighbouring view images, the method [11] becomes invalid. In contrast, the improved method still remains effective.The following notation is used in the paper.Rm×n and Rn denote the set of all real m×n matrices and the set of all real n-vectors, respectively. Zm×n and Zn denote the set of all integer m×n matrices and the set of all integern-vectors, respectively.Im×n∈Zm×n denotes m×n matrix with all entries equal to 1. For a real scalar z, ⌊z⌉ stands for its nearest integer, ⌊z⌋ stands for the largest integer less than or equal to z, and ⌈z⌉ stands for the smallest integer greater than or equal to z. The operation sign⁡(z) returns -1 if z≤0 and 1 if z>0. For a matrix A,aj denotes the jth column of A,akjdenotes thekth entry of vector aj, and AT denotes the transposed matrix of A. Bold upper case letters and bold lower case letters denote matrices and vectors, respectively.


## 2.  Crosstalk Measurement and Crosstalk Reduction Method

### 2.1. Crosstalk Measurement Method

Multiview 3D displays provide 3D perception without requiring any form of special glasses or other user-mounted devices. Among them, parallax barrier displays and lenticular displays are prevalent in actual devices, as indicated in Figure1. The synthetic image prepared for the 8-view 3D display is synthesized by 8 different view images, as can be seen in Figure 2. The rectangles represent the subpixels, and the numbers marked in them indicate view image numbers. The subpixels illuminated with color in Figure 2(a) are prepared for the observer at the first optimal viewing position. At the same time, the visible image observed through the slits is illustrated in Figure 2(b). They do not overlap perfectly; that is the main cause of crosstalk observed at optimal viewing position.Structures of two kinds of multiview 3D displays. (a) A parallax barrier display. (b) A lenticular display.
(a)
(b)Distribution mode of 8 different view images’ subpixels on the synthetic image. (a) View image prepared for the first viewing position. (b) Visible image observed through the slits at the first viewing position.
(a)
(b)We extend the definition of system-introduced crosstalk [6] from stereoscopic display to the multiview 3D display and precisely define the system-introduced crosstalk observed at thenth optimal viewing position on the N-view 3D display as follows:
(1)χn=∑l=1,l≠nN‍Yl-YBYn-YB,
where Yl(l=1⋯N) denotes the luminance measured at the nth optimal viewing position with white in the lth view image and black in all of other view images on the synthetic image and YB denotes the environmental light intensity. This definition is black-white crosstalk, since it uses full-black and full-white images in the testing scheme. Full-white and full-black are used because maximum leakage usually occurs when the pixels in the desired view image are full-black and the pixels in the other view images are full-white. The crosstalk between the neighboring view images on N-view 3D display was considered in formula (1), rather than the crosstalk between the left eye image and the right eye image given in [6].For theN-view 3D display, ∑l=1,l≠nN‍Yl-YB in formula (1) is replaced with s1, which is the viewable area of unintended view images observed at the nth optimum viewing position. Yn-YB is replaced withs2, which is the viewable area of intended view image observed at the nth optimum viewing position. Thus, we define the system-introduced crosstalk as follows:
(2)χn=s1s2=∑m=1v‍∑i=2h‍s4(i-1,m)+∑m=1v‍∑i=1h-1‍s5(i+1,m)∑m=1v‍∑i=1h‍s3(i,m),
where (i,m) denotes the subpixel that is in the ith row on synthetic image observed through the mth slit at the nth optimal viewing position, v denotes the number of slits, and h denotes the vertical resolution of the synthetic image. s3 denotes the viewable area of subpixel belonging to the intended view image and s4 and s5 denote the viewable areas of neighboring unintended subpixel above and below respectively. The distributions of s3, s4, ands5are illustrated in Figure 3. The dotted lines present the edges of slits. The rectangles present the subpixels on the display screen.Figure 3
Illustration of system-introduced crosstalk measurement method.Based on the image composition theory, the subpixels showing the intended view image are determined in all subpixels observed at thenth optimal viewing position. Calculate s3, s4, and s5 and substitute the results into expression (2) to obtain the system-introduced crosstalk.Let the visible area’s width observed through a slit on the synthetic image bet (this equals the width of one subpixel). As a result, s3, s4, and s5 remain constant when m varies over the range of [1,v]. When we substitute s3(i)=s3(i,1), s4(i-1)=s4(i-1,1), s5(i+1)=s5(i+1,1) for s3(i,m),s4(i-1,m),s5(i+1,m) respectively, formula (2) is still valid. m=1 means that subpixels are observed through the first integral slit. Therefore, the system-introduced crosstalk on the multiview 3D display at the nth viewing position is derived as
(3)χn=s1s2=∑i=2h‍s4(i-1)+∑i=1h-1‍s5(i+1)∑i=1h‍s3(i).Depending on the slant angle of parallax barrierθ, the system-introduced crosstalk measurement is proposed as follows.(1) Crosstalk measurement whenθ≤arctan(1/3): when the subpixel is divided equally by the left side of the slit, the intercept length on the bottom edge of the subpixel ist-Δ0=t-|(3t·tanθ-t)/2|=(t+3ttanθ)/2. Let the intercept of subpixel on the lower limb by the left edge of the first integral slit be Δ1(i)=3it·tanθ-⌊3it·tanθ⌋.WhenΔ1(i)≥t-Δ0, the subpixel in the ith row ⌊3i·tanθ⌋+2th column on the synthetic image shows the color belonging to intended view image. Consider
(4)s3(i)=32t2+|Δ1(i)+Δ0-t|×3t,s4(i-1)=(2Δ0+Δ1(i)-t)22tanθ,s5(i+1)=0.WhenΔ1(i)<t-Δ0, the subpixel in the ith row ⌊3i·tanθ⌋+1th column on the synthetic image shows the color belonging to intended view image.IfΔ0≥t-Δ0-Δ1(i),
(5)s3(i)=32t2+|Δ1(i)+Δ0-t|×3t,s4(i-1)=0,s5(i+1)=(t-Δ1(i))22tanθ.IfΔ0<t-Δ0-Δ1(i),
(6)s3(i)=3t2-Δ12(i)2tanθ-(t-2Δ0-Δ1(i))22tanθs4(i-1)=0,s5(i+1)=(t-Δ1(i))22tanθ.(2) Crosstalk measurement whenθ>arctan(1/3): when the subpixel is evenly divided by the left side of the slit, the intercept length on the bottom edge of this subpixel is Δ0=|(3t·tanθ-t)/2|. When the subpixel in theith row is intercepted by the left edge of the first integral slit, Δ1(i)=3it·tanθ-⌊3it·tanθ⌋.WhenΔ1(i)>Δ0, the subpixel in the ith row ⌊3i·tanθ⌋+1th column on the synthetic image shows the color belonging to intended view image.IfΔ1(i)-Δ0≤Δ0,
(7)s3(i)=(2Δ1(i)+t)t2tanθ-Δ12(i)2tanθ,s4(i-1)=0,s5(i+1)=(t-Δ1(i))22tanθ.IfΔ1(i)-Δ0>Δ0,
(8)s3(i)=3t2-Δ12(i)2tanθ-(t+2Δ0-Δ1(i))22tanθ,s4(i-1)=(Δ1(i)-2Δ0)22tanθ,s5(i+1)=(t-Δ1(i))22tanθ.WhenΔ1(i)≤Δ0, the subpixel in the ith row ⌊3i·tanθ⌋th column on the synthetic image shows color belonging to intended view image.IfΔ0-Δ1(i)≤t-Δ0,
(9)s3(i)=3t2-(2Δ0-Δ1(i))22tanθ-(2Δ1(i)+t)t2tanθ,s4(i-1)=(t+Δ1(i)-2Δ0)22tanθ,s5(i+1)=0.IfΔ0-Δ1(i)>t-Δ0,
(10)s3(i)=3t2-[2(2Δ0-Δ1(i)-t)+t]t2tanθ-(2Δ1(i)+t)t2tanθ,s4(i-1)=0,s5(i+1)=0.Substituting these results into expression (3) obtains the system-introduced crosstalk for arbitrary multiview 3D display.In the method given above, the top left corner of the visible image observed through the first integral slit and the top left corner of synthetic image coincide. Variablea is used to denote the distance from the top left corner of the visible image observed through the first integral slit to the top left corner of synthetic image. Let a be within the range of [0,t]. Then, Δ1(i)=3it·tanθ+a-⌊3it·tanθ+a⌋. The following calculation procedure is the same as that given above.Varyinga from 0 to t simulates the slight movement of visible image on the synthetic image, while the viewing position is moving parallel to the display screen in the viewing zone. The crosstalk varies with the variable a. Thus, we define the system-introduced crosstalk as χn=min⁡a∈[0,t]χn(a) that is the system-introduced crosstalk observed at the optimal viewing point.


### 2.2. Improved Crosstalk Reduction Method

For crosstalk reduction, the neighboring subpixels in the same column should be considered because the subpixels in same column on synthetic image show the same color, which is the main leakage among different view images. What we want to do is to correct the subpixel value on the synthetic image fromb(i,j) to x(i,j) such that the leakage from the neighboring subpixels in same column will become useful for the image quality of multiview 3D displays; that is,
(11)b(i,j)={s3(i)x(i,j)+s5(i+1)x(i+1,j)s3(i)+s5(i+1)vvvvvvvvvvvvvvvvvvvvvvvvvi=1s4(i-1)x(i-1,j)+s3(i)x(i,j)+s5(i+1)x(i+1,j)s4(i-1)+s3(i)+s5(i+1)vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv2≤i≤h-1s4(i-1)x(i-1,j)+s3(i)x(i,j)s4(i-1)+s3(i)vvvvvvvvvvvvvvvvvvvvvvvvi=h,
where b(i,j) and x(i,j) denote the subpixel values in the ith row j column on the synthetic image before and after correction, respectively. That means that the intensity obtained at the nth optimal viewing position is equal to the intensity of the nth intended view image, after correction. To solve this problem, it can be written as follows:
(12)AX=B,
where B∈Zw×h and X∈Zw×h are the synthetic images before and after correction, respectively. A∈Rh×h is the crosstalk coefficient matrix. The entry of A, a(p,q) denotes the leakage degree from the subpixel in the qth row jth column to the one in the pth row jth column on the synthetic image. p and q are both the numbers from 1 to h. We obtain a(p,q) by the proposed method, which exactly reflects the crosstalk among the subpixels corresponding to different view images on the synthetic image, rather than among the whole neighboring view images.In the actual display devices,x(i,j) is set to be 0, if x(i,j)<0; x(i,j) is set to be 255, if x(i,j)>255; and x(i,j) is set to be ⌊x(i,j)⌉, if x(i,j)∉Z. This operation causes the fact that the unique solution to problem (12), X=A-1B, may not be the optimal solution, which means crosstalk elimination does not perform completely.If the rank ofA is h, which means A is full rank, the equation set has the unique solution X=A-1B. However, B is constrained to the box BB={B∈Zw×h:L≤B≤U}, where L=0×Iw×h and U=255×Iw×h. Hence X may be in following two boxes:(1)
B1={X∈Zw×h:X<LorX>U};(2)
B2={X∉Zw×h:L≤X≤U}.We propose an alternative method to obtain the optimum solution to equations set (12). For the given B, X is the solution to a BILS problem:
(13)min⁡X∈BX∥B-AX∥22BX={X∈Zw×h:L≤X≤U},
where ∥·∥2 denotes the Euclidean norm.Given a realh-vector bj and a real h×h matrix A with full column rank, lattice reduction is applied to solve the problem:
(14)min⁡xj∈Bx∥bj-Axj∥22,Bx={xj∈Zh:lj≤xj≤uj}
for j=1,2,…,w. The optimal solution to problem (14) is also the solution to (13). In lattice theory, A is called the generator matrix of the lattice L(A)={Axj:xj∈Zh}, bj is called the input vector, and (14) is referred to as a closest-point problem, since it is the problem to find a point in the lattice which is closest to the given input point bj.At first, we transform the matrixA to an upper triangular matrix, which has good properties to make the search algorithm more efficient. Here, this can be performed by the QR decomposition of A:
(15)AP=QR,
where P∈Zh×h is a permutation matrix, Q∈Rh×h is orthogonal, and R∈Rh×h is nonsingular upper triangular. With the QR decomposition, we have
(16)∥bj-Axj∥22=∥QTbj-RPTxj∥22.
Define
(17)b¯j=QTbj,zj=PTxj,l¯j=PTlj,u¯j=PTuj,
where lj is the lower bound of xj and uj is the upper bound of xj.l¯j is the lower bound of zj; u¯j is the upper bound of zj. Here, l1j=l2j=⋯=lhj=0; u1j=u2j=⋯=uhj=255.WhenP is an identity matrix, BILS problem (14) is equivalent to the following one:
(18)min⁡zj∈Bz∥b¯j-Rzj∥22Bz={zj∈Zh:l¯j≤zj≤u¯j},
forj=1,2,…,w.Then the proposed algorithm tries to solve (18). Note that, if z^jis the solution to (18), then x^j=Pz^j=z^j is the solution to (14) and X^ is the solution to (13).We provide a search algorithm which avoids some drawbacks of algorithm BGBF (Algorithm1) [17]. Our algorithm is applied to obtain the correction result of synthetic image one column by one column. When we input b=b-j (j is the number from 1 to w), the correction result of this column z^=z^j is obtained. After w times operations, synthetic image is processed to the correction result Z^. Thus, we obtain the final correction result X^=Z^.Algorithm 1:Crosstalk reduction algorithm.
Input:  The upper triangular matrix R∈Rh×h with positive diagonal entries, the vector
b=b-j∈Zh, the lower bound vector l=l-j∈Zh, the upper bound vector
u =u-j∈Zh, the initial hyper-ellipsoid bound β.
Output: The solution z^∈Zh to the BILS Problem (18).
Step  1. (Initialization) Set k:=h, Tk:=0,
Compute sk:=bk/rkk, zk:=⌊sk⌉,
zk:=max⁡(zk,lk), zk:=min⁡(zk,uk),
ρk:=(sk-zk)rkk, Δk:=-sign⁡(ρk).
Step  2.  Compute T:=Tk+ρk2,
if  T<β and k≠1, then
Go to Step  3
else
Go to Step  4
end if
Step  3. Set Tk-1:=T, k:=k-1
Compute sk:=(bk-∑m=k+1h‍rkmzm)/rkk,
zk:=⌊sk⌉, zk:=max⁡(zk,lk), zk:=min⁡(zk,uk),
ρk:=(sk-zk)rkk, Δk:=-sign⁡(ρk).
Go to Step  2
Step  4. If  T<β, then
Set β:=T, z^:=z, k:=k+1,
else if  k=h, then
Terminate
else
k:=k+1
if  ρk=0, then
K:=k
repeat
if  k=h, then
Terminate
end if
k:=k+1
until  ρk≠0
if  k≠h, then
k:=K
end if
end if
end if
Step  5. If  zk+Δk>uk, then
Set Δk:=-1
Compute zk:=zk+Δk
else if  zk+Δk<lk, then
Set Δk:=1
Compute zk:=zk+Δk
else
Compute zk:=zk+Δk, Δk:=-Δk-sign⁡(Δk)
end if
Compute ρk:=(sk-zk)rkk
Go to Step  2.In Step 1 or Step 3, the last value ofzk may be the lower bound lk or the upper bound uk. To avoid enumerating some integers outside the interval [lk,uk] in Step 5, we set Δk=-sign⁡(ρk). For instance, when ⌊sk⌉>uk in Step 1 or Step 3, zk=⌊sk⌉>uk, zk=min⁡(zk,uk)=uk, and the offset variable Δk=-1 (since ρk=(sk-zk)rkk=(sk-uk)rkk>0), which means the next value is zk:=zk-1=uk-1. This process mostly avoids enumerating integer beyond the box constraint. However, when uk-0.5≤sk<uk, ⌊sk⌉=uk, ρk=(sk-zk)rkk=(sk-uk)rkk<0, and Δk=-sign⁡(ρk)=1, zk=zk+Δk will be outside the interval [lk,uk]. To avoid this happening, we set Δk:=-1 and compute zk:=zk+Δk if zk+Δk>uk in Step 5; we set Δk:=1 and compute zk:=zk+Δk if zk+Δk<lk; otherwise, we compute zk:=zk+Δk and Δk:=-Δk-sign⁡(Δk). Therefore, our algorithm always enumerates the integers within the box constraint at each level.In algorithm BGBF, ifT>β and k≠h in Step 4, that means no any other integer will satisfy inequality ∑l=kh‍rll2(sl-zl)2<β in level k. If ρl=0(l=k+1,k+2⋯h) at that time, there is still not any other integer satisfying the inequality and we should exit loop directly and return the optimal solution z^. Therefore, if T>β, k<h, and ρl=0(l=k+1,k+2⋯h), our algorithm exits the loop and returns the optimal solution z^=z^j; otherwise go back to level k+1.


## 2.1. Crosstalk Measurement Method

Multiview 3D displays provide 3D perception without requiring any form of special glasses or other user-mounted devices. Among them, parallax barrier displays and lenticular displays are prevalent in actual devices, as indicated in Figure1. The synthetic image prepared for the 8-view 3D display is synthesized by 8 different view images, as can be seen in Figure 2. The rectangles represent the subpixels, and the numbers marked in them indicate view image numbers. The subpixels illuminated with color in Figure 2(a) are prepared for the observer at the first optimal viewing position. At the same time, the visible image observed through the slits is illustrated in Figure 2(b). They do not overlap perfectly; that is the main cause of crosstalk observed at optimal viewing position.Structures of two kinds of multiview 3D displays. (a) A parallax barrier display. (b) A lenticular display.
(a)
(b)Distribution mode of 8 different view images’ subpixels on the synthetic image. (a) View image prepared for the first viewing position. (b) Visible image observed through the slits at the first viewing position.
(a)
(b)We extend the definition of system-introduced crosstalk [6] from stereoscopic display to the multiview 3D display and precisely define the system-introduced crosstalk observed at thenth optimal viewing position on the N-view 3D display as follows:
(1)χn=∑l=1,l≠nN‍Yl-YBYn-YB,
where Yl(l=1⋯N) denotes the luminance measured at the nth optimal viewing position with white in the lth view image and black in all of other view images on the synthetic image and YB denotes the environmental light intensity. This definition is black-white crosstalk, since it uses full-black and full-white images in the testing scheme. Full-white and full-black are used because maximum leakage usually occurs when the pixels in the desired view image are full-black and the pixels in the other view images are full-white. The crosstalk between the neighboring view images on N-view 3D display was considered in formula (1), rather than the crosstalk between the left eye image and the right eye image given in [6].For theN-view 3D display, ∑l=1,l≠nN‍Yl-YB in formula (1) is replaced with s1, which is the viewable area of unintended view images observed at the nth optimum viewing position. Yn-YB is replaced withs2, which is the viewable area of intended view image observed at the nth optimum viewing position. Thus, we define the system-introduced crosstalk as follows:
(2)χn=s1s2=∑m=1v‍∑i=2h‍s4(i-1,m)+∑m=1v‍∑i=1h-1‍s5(i+1,m)∑m=1v‍∑i=1h‍s3(i,m),
where (i,m) denotes the subpixel that is in the ith row on synthetic image observed through the mth slit at the nth optimal viewing position, v denotes the number of slits, and h denotes the vertical resolution of the synthetic image. s3 denotes the viewable area of subpixel belonging to the intended view image and s4 and s5 denote the viewable areas of neighboring unintended subpixel above and below respectively. The distributions of s3, s4, ands5are illustrated in Figure 3. The dotted lines present the edges of slits. The rectangles present the subpixels on the display screen.Figure 3
Illustration of system-introduced crosstalk measurement method.Based on the image composition theory, the subpixels showing the intended view image are determined in all subpixels observed at thenth optimal viewing position. Calculate s3, s4, and s5 and substitute the results into expression (2) to obtain the system-introduced crosstalk.Let the visible area’s width observed through a slit on the synthetic image bet (this equals the width of one subpixel). As a result, s3, s4, and s5 remain constant when m varies over the range of [1,v]. When we substitute s3(i)=s3(i,1), s4(i-1)=s4(i-1,1), s5(i+1)=s5(i+1,1) for s3(i,m),s4(i-1,m),s5(i+1,m) respectively, formula (2) is still valid. m=1 means that subpixels are observed through the first integral slit. Therefore, the system-introduced crosstalk on the multiview 3D display at the nth viewing position is derived as
(3)χn=s1s2=∑i=2h‍s4(i-1)+∑i=1h-1‍s5(i+1)∑i=1h‍s3(i).Depending on the slant angle of parallax barrierθ, the system-introduced crosstalk measurement is proposed as follows.(1) Crosstalk measurement whenθ≤arctan(1/3): when the subpixel is divided equally by the left side of the slit, the intercept length on the bottom edge of the subpixel ist-Δ0=t-|(3t·tanθ-t)/2|=(t+3ttanθ)/2. Let the intercept of subpixel on the lower limb by the left edge of the first integral slit be Δ1(i)=3it·tanθ-⌊3it·tanθ⌋.WhenΔ1(i)≥t-Δ0, the subpixel in the ith row ⌊3i·tanθ⌋+2th column on the synthetic image shows the color belonging to intended view image. Consider
(4)s3(i)=32t2+|Δ1(i)+Δ0-t|×3t,s4(i-1)=(2Δ0+Δ1(i)-t)22tanθ,s5(i+1)=0.WhenΔ1(i)<t-Δ0, the subpixel in the ith row ⌊3i·tanθ⌋+1th column on the synthetic image shows the color belonging to intended view image.IfΔ0≥t-Δ0-Δ1(i),
(5)s3(i)=32t2+|Δ1(i)+Δ0-t|×3t,s4(i-1)=0,s5(i+1)=(t-Δ1(i))22tanθ.IfΔ0<t-Δ0-Δ1(i),
(6)s3(i)=3t2-Δ12(i)2tanθ-(t-2Δ0-Δ1(i))22tanθs4(i-1)=0,s5(i+1)=(t-Δ1(i))22tanθ.(2) Crosstalk measurement whenθ>arctan(1/3): when the subpixel is evenly divided by the left side of the slit, the intercept length on the bottom edge of this subpixel is Δ0=|(3t·tanθ-t)/2|. When the subpixel in theith row is intercepted by the left edge of the first integral slit, Δ1(i)=3it·tanθ-⌊3it·tanθ⌋.WhenΔ1(i)>Δ0, the subpixel in the ith row ⌊3i·tanθ⌋+1th column on the synthetic image shows the color belonging to intended view image.IfΔ1(i)-Δ0≤Δ0,
(7)s3(i)=(2Δ1(i)+t)t2tanθ-Δ12(i)2tanθ,s4(i-1)=0,s5(i+1)=(t-Δ1(i))22tanθ.IfΔ1(i)-Δ0>Δ0,
(8)s3(i)=3t2-Δ12(i)2tanθ-(t+2Δ0-Δ1(i))22tanθ,s4(i-1)=(Δ1(i)-2Δ0)22tanθ,s5(i+1)=(t-Δ1(i))22tanθ.WhenΔ1(i)≤Δ0, the subpixel in the ith row ⌊3i·tanθ⌋th column on the synthetic image shows color belonging to intended view image.IfΔ0-Δ1(i)≤t-Δ0,
(9)s3(i)=3t2-(2Δ0-Δ1(i))22tanθ-(2Δ1(i)+t)t2tanθ,s4(i-1)=(t+Δ1(i)-2Δ0)22tanθ,s5(i+1)=0.IfΔ0-Δ1(i)>t-Δ0,
(10)s3(i)=3t2-[2(2Δ0-Δ1(i)-t)+t]t2tanθ-(2Δ1(i)+t)t2tanθ,s4(i-1)=0,s5(i+1)=0.Substituting these results into expression (3) obtains the system-introduced crosstalk for arbitrary multiview 3D display.In the method given above, the top left corner of the visible image observed through the first integral slit and the top left corner of synthetic image coincide. Variablea is used to denote the distance from the top left corner of the visible image observed through the first integral slit to the top left corner of synthetic image. Let a be within the range of [0,t]. Then, Δ1(i)=3it·tanθ+a-⌊3it·tanθ+a⌋. The following calculation procedure is the same as that given above.Varyinga from 0 to t simulates the slight movement of visible image on the synthetic image, while the viewing position is moving parallel to the display screen in the viewing zone. The crosstalk varies with the variable a. Thus, we define the system-introduced crosstalk as χn=min⁡a∈[0,t]χn(a) that is the system-introduced crosstalk observed at the optimal viewing point.


## 2.2. Improved Crosstalk Reduction Method

For crosstalk reduction, the neighboring subpixels in the same column should be considered because the subpixels in same column on synthetic image show the same color, which is the main leakage among different view images. What we want to do is to correct the subpixel value on the synthetic image fromb(i,j) to x(i,j) such that the leakage from the neighboring subpixels in same column will become useful for the image quality of multiview 3D displays; that is,
(11)b(i,j)={s3(i)x(i,j)+s5(i+1)x(i+1,j)s3(i)+s5(i+1)vvvvvvvvvvvvvvvvvvvvvvvvvi=1s4(i-1)x(i-1,j)+s3(i)x(i,j)+s5(i+1)x(i+1,j)s4(i-1)+s3(i)+s5(i+1)vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv2≤i≤h-1s4(i-1)x(i-1,j)+s3(i)x(i,j)s4(i-1)+s3(i)vvvvvvvvvvvvvvvvvvvvvvvvi=h,
where b(i,j) and x(i,j) denote the subpixel values in the ith row j column on the synthetic image before and after correction, respectively. That means that the intensity obtained at the nth optimal viewing position is equal to the intensity of the nth intended view image, after correction. To solve this problem, it can be written as follows:
(12)AX=B,
where B∈Zw×h and X∈Zw×h are the synthetic images before and after correction, respectively. A∈Rh×h is the crosstalk coefficient matrix. The entry of A, a(p,q) denotes the leakage degree from the subpixel in the qth row jth column to the one in the pth row jth column on the synthetic image. p and q are both the numbers from 1 to h. We obtain a(p,q) by the proposed method, which exactly reflects the crosstalk among the subpixels corresponding to different view images on the synthetic image, rather than among the whole neighboring view images.In the actual display devices,x(i,j) is set to be 0, if x(i,j)<0; x(i,j) is set to be 255, if x(i,j)>255; and x(i,j) is set to be ⌊x(i,j)⌉, if x(i,j)∉Z. This operation causes the fact that the unique solution to problem (12), X=A-1B, may not be the optimal solution, which means crosstalk elimination does not perform completely.If the rank ofA is h, which means A is full rank, the equation set has the unique solution X=A-1B. However, B is constrained to the box BB={B∈Zw×h:L≤B≤U}, where L=0×Iw×h and U=255×Iw×h. Hence X may be in following two boxes:(1)
B1={X∈Zw×h:X<LorX>U};(2)
B2={X∉Zw×h:L≤X≤U}.We propose an alternative method to obtain the optimum solution to equations set (12). For the given B, X is the solution to a BILS problem:
(13)min⁡X∈BX∥B-AX∥22BX={X∈Zw×h:L≤X≤U},
where ∥·∥2 denotes the Euclidean norm.Given a realh-vector bj and a real h×h matrix A with full column rank, lattice reduction is applied to solve the problem:
(14)min⁡xj∈Bx∥bj-Axj∥22,Bx={xj∈Zh:lj≤xj≤uj}
for j=1,2,…,w. The optimal solution to problem (14) is also the solution to (13). In lattice theory, A is called the generator matrix of the lattice L(A)={Axj:xj∈Zh}, bj is called the input vector, and (14) is referred to as a closest-point problem, since it is the problem to find a point in the lattice which is closest to the given input point bj.At first, we transform the matrixA to an upper triangular matrix, which has good properties to make the search algorithm more efficient. Here, this can be performed by the QR decomposition of A:
(15)AP=QR,
where P∈Zh×h is a permutation matrix, Q∈Rh×h is orthogonal, and R∈Rh×h is nonsingular upper triangular. With the QR decomposition, we have
(16)∥bj-Axj∥22=∥QTbj-RPTxj∥22.
Define
(17)b¯j=QTbj,zj=PTxj,l¯j=PTlj,u¯j=PTuj,
where lj is the lower bound of xj and uj is the upper bound of xj.l¯j is the lower bound of zj; u¯j is the upper bound of zj. Here, l1j=l2j=⋯=lhj=0; u1j=u2j=⋯=uhj=255.WhenP is an identity matrix, BILS problem (14) is equivalent to the following one:
(18)min⁡zj∈Bz∥b¯j-Rzj∥22Bz={zj∈Zh:l¯j≤zj≤u¯j},
forj=1,2,…,w.Then the proposed algorithm tries to solve (18). Note that, if z^jis the solution to (18), then x^j=Pz^j=z^j is the solution to (14) and X^ is the solution to (13).We provide a search algorithm which avoids some drawbacks of algorithm BGBF (Algorithm1) [17]. Our algorithm is applied to obtain the correction result of synthetic image one column by one column. When we input b=b-j (j is the number from 1 to w), the correction result of this column z^=z^j is obtained. After w times operations, synthetic image is processed to the correction result Z^. Thus, we obtain the final correction result X^=Z^.Algorithm 1:Crosstalk reduction algorithm.
Input:  The upper triangular matrix R∈Rh×h with positive diagonal entries, the vector
b=b-j∈Zh, the lower bound vector l=l-j∈Zh, the upper bound vector
u =u-j∈Zh, the initial hyper-ellipsoid bound β.
Output: The solution z^∈Zh to the BILS Problem (18).
Step  1. (Initialization) Set k:=h, Tk:=0,
Compute sk:=bk/rkk, zk:=⌊sk⌉,
zk:=max⁡(zk,lk), zk:=min⁡(zk,uk),
ρk:=(sk-zk)rkk, Δk:=-sign⁡(ρk).
Step  2.  Compute T:=Tk+ρk2,
if  T<β and k≠1, then
Go to Step  3
else
Go to Step  4
end if
Step  3. Set Tk-1:=T, k:=k-1
Compute sk:=(bk-∑m=k+1h‍rkmzm)/rkk,
zk:=⌊sk⌉, zk:=max⁡(zk,lk), zk:=min⁡(zk,uk),
ρk:=(sk-zk)rkk, Δk:=-sign⁡(ρk).
Go to Step  2
Step  4. If  T<β, then
Set β:=T, z^:=z, k:=k+1,
else if  k=h, then
Terminate
else
k:=k+1
if  ρk=0, then
K:=k
repeat
if  k=h, then
Terminate
end if
k:=k+1
until  ρk≠0
if  k≠h, then
k:=K
end if
end if
end if
Step  5. If  zk+Δk>uk, then
Set Δk:=-1
Compute zk:=zk+Δk
else if  zk+Δk<lk, then
Set Δk:=1
Compute zk:=zk+Δk
else
Compute zk:=zk+Δk, Δk:=-Δk-sign⁡(Δk)
end if
Compute ρk:=(sk-zk)rkk
Go to Step  2.In Step 1 or Step 3, the last value ofzk may be the lower bound lk or the upper bound uk. To avoid enumerating some integers outside the interval [lk,uk] in Step 5, we set Δk=-sign⁡(ρk). For instance, when ⌊sk⌉>uk in Step 1 or Step 3, zk=⌊sk⌉>uk, zk=min⁡(zk,uk)=uk, and the offset variable Δk=-1 (since ρk=(sk-zk)rkk=(sk-uk)rkk>0), which means the next value is zk:=zk-1=uk-1. This process mostly avoids enumerating integer beyond the box constraint. However, when uk-0.5≤sk<uk, ⌊sk⌉=uk, ρk=(sk-zk)rkk=(sk-uk)rkk<0, and Δk=-sign⁡(ρk)=1, zk=zk+Δk will be outside the interval [lk,uk]. To avoid this happening, we set Δk:=-1 and compute zk:=zk+Δk if zk+Δk>uk in Step 5; we set Δk:=1 and compute zk:=zk+Δk if zk+Δk<lk; otherwise, we compute zk:=zk+Δk and Δk:=-Δk-sign⁡(Δk). Therefore, our algorithm always enumerates the integers within the box constraint at each level.In algorithm BGBF, ifT>β and k≠h in Step 4, that means no any other integer will satisfy inequality ∑l=kh‍rll2(sl-zl)2<β in level k. If ρl=0(l=k+1,k+2⋯h) at that time, there is still not any other integer satisfying the inequality and we should exit loop directly and return the optimal solution z^. Therefore, if T>β, k<h, and ρl=0(l=k+1,k+2⋯h), our algorithm exits the loop and returns the optimal solution z^=z^j; otherwise go back to level k+1.


## 3. Results and Discussion

In this section, we aim to validate the improved crosstalk reduction method on multiview 3D displays, both qualitatively and quantitatively. We first give a specific example to demonstrate that the improved method alleviates the crosstalk on multiview 3D lenticular display effectively. Then computer simulation is carried out to compare the crosstalk correction results with typical method on two test sequences. Numerical results are presented to validate the performance of the improved method in terms of the mean square error (MSE) and peak signal-to-noise ratio (PSNR). Finally, a photometer is used for measurement to compare the crosstalk reduction results by the improved method and typical method. The experiment results indicate that the improved method eliminates the crosstalk more effectively and improves the quality of the 3D images further.At first, we compare the crosstalk values obtained via the proposed method and measured through the experimental method. To verify the effectiveness of the proposed crosstalk measurement method, a photometer (FS-5500TLS) is used to measure the crosstalk at the optimal viewing position. At first, 8 black-and-white test images are obtained in which each view image is white (that means that the subpixel values corresponding to the certain view image are equal to 255) and, in turn, presented on the display screen. In the black-and-white test image, only subpixel values corresponding to the certain view image are equal to 255, and all of the other subpixel values are equal to 0. We perform the experiment, using a multiview 3D parallax barrier display screen (42HD, provided by Tianjin 3-D Imaging Technique Co. Ltd.) is used to display synthetic images. Then, measure environmental light intensity and each test images intensity, respectively, along the horizontal direction at the optimal viewing distance. The photometer is used for luminance measurements. The luminancesYl and YB are obtained; then crosstalk χn=0.38 is got through formula (1). Through the proposed crosstalk measurement method, we get the crosstalk χn=0.38 when α=arctan(2/9) on the multiview 3D display. The result well matches the actual measured crosstalk value. The close agreement between the proposed method and measured crosstalk results demonstrates that the proposed crosstalk measurement method can be used to measure the crosstalk on the multiview 3D display.In our implementation,a^=argmin⁡aχn(a) and the crosstalk coefficient matrices A∈Rh×h are obtained by the proposed system-introduced crosstalk measurement method. The improved crosstalk reduction method is applied to calculate the correction values of subpixels one column by one column. However, when dimension h is large, the search time becomes significant. So A is rewritten as
(19)A=(A¯10⋯00A¯2⋯0⋮⋮⋱⋮00⋯A¯⌈h/r⌉),
where A¯l∈Rr×r. To determine r, we analyze the relationship of neighboring subpixels in the same column. For example, when parallax barrier is slanted at the angle of arctan(2/9), we get the distribution mode of different view images’ subpixels on the synthetic image, illustrated in Figure 2. Taking the first column on the left as an example, the third and fourth subpixels belong to the seventh view image and are observed at the seventh optimal viewing position. That means the third and fourth subpixels do not interact and are not related to each other. Consequently, we consider that every 3 subpixel values are correlative in one column; that is, r=3 when θ=arctan(2/9). Obviously, when θ<arctan(1/3), r can be determined easily in the same way. When θ≥arctan(1/3), we can determine r, according to the need.We input the vectorb=QlTbjl and upper triangle matrix Rl∈Rr×r, where Rl and Ql∈Rr×r are the results ofQR composition of A-l, bjl=(bl,j,bl+1,j,…,bl+r-1,j)T (l is the number from 1 to ⌈h/r⌉;jis the number from 1 to w). After one operation, output z^=(z^l,j,z^l+1,j,…,z^l+r-1,j)T. After ⌈h/r⌉×wtimes operations, the correction result X^ is obtained.First, we give a specific example to demonstrate the effectiveness of the proposed method for a group of images of a scene “Euro.” A multiview 3D lenticular display screen (46HD, provided by SuperD Co. Ltd.) is used to display synthetic images. The result before correction and the result by the method in [11] and our method are presented on the display screen, respectively. Figure 4 shows the photos taken at the fourth optimal viewing position. As can be seen the edges of “Euro” are blurring in the result before correction and it is still not good enough in the result by the method in [11]. Compared with Figures 4(a) and 4(b), the photo in Figure 4(c) is the clearest, especially at the boundaries of “golden bricks.” Furthermore, it can be observed from Figure 4 that objects’ jagged edges (such as Euro’ edges and the golden stick’ edges) are still obvious in the result by the method in [11]. In contrast, our method well solves this problem; the phenomenon of contour jaggies largely disappears in the result by our method. It can be discovered from the experiment results that the proposed method reduces the crosstalk more effectively on the multiview 3D lenticular display.Photos of a scene “Euro” taken at the fourth optimal viewing position. (a) Result before correction. (b) Result by method given in [11]. (c) Result by our method.
(a)
(b)
(c)Then, we compare the correction results obtained by the proposed method and the method in [11] for two test sequences in terms of MSE and PSNR. Two test sequences are used in our implementation, namely, “Fireman” and “Lotus.” They are obtained from Tianjin 3D Imaging Technique Co. Ltd., Tianjin, China, which are 3D test sequences with proper parallax between adjacent views. The resolution of “Fireman” is 1024 × 600, while the resolution of “Lotus” is 1920 × 1024. To be fair, the commonly used metric MSE is used to evaluate the residual crosstalk, which is defined as MSE=(1/(h×w))∥AX-B∥22. Note that the MSE is converted to the PSNR based upon the relationship PSNR=10log⁡10 (2552/MSE). Figure 5 plots the PSNR values versus both the frame and the view numbers in two sequences. For a close-up look, as an example, Figure 6 compares our correction results with the results by the method in [11]. The PSNR values versus the frame number for each view of “Fireman” are shown in the figure. Furthermore, the MSE curves for each view of “Lotus” are plotted in Figure 7, showing the frame-level residual crosstalk (MSE) of each view versus the frame number. It can be concluded that the improved method performs better than the typical method for both sequences in terms of MSE and PSNR.PSNR versus both the frame and the view numbers in corrected sequences. (a) PSNR of corrected “Lotus” by our method. (b) PSNR of corrected “Lotus” by the method in [11]. (c) PSNR of corrected “Fireman” by our method. (d) PSNR of corrected “Fireman” by the method in [11].
(a)
(b)
(c)
(d)PSNR versus frames for each view in “Lotus.” (a) PSNR of view 1 in “Lotus.” (b) PSNR of view 2 in “Lotus.” (c) PSNR of view 3 in “Lotus.” (d) PSNR of view 4 in “Lotus.” (e) PSNR of view 5 in “Lotus.” (f) PSNR of view 6 in “Lotus.” (g) PSNR of view 7 in “Lotus.” (h) PSNR of view 8 in “Lotus.”
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)MSE versus frames for each view in “Fireman.” (a) MSE of view 1 in “Fireman.” (b) MSE of view 2 in “Fireman.” (c) MSE of view 3 in “Fireman.” (d) MSE of view 4 in “Fireman.” (e) MSE of view 5 in “Fireman.” (f) MSE of view 6 in “Fireman.” (g) MSE of view 7 in “Fireman.” (h) MSE of view 8 in “Fireman.”
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)The MSE per frame in results by the method in [11] and our method for two test sequences is compared. Figure 8 plots the MSE versus the frames for the sequences “Fireman” and “Lotus.” As can be seen, the intensity observed at the optimal viewing position in the correction result by our method is very near to the intensity of intended view image; that is, the residual crosstalk reduces greatly in the results by our method.MSE versus frames in “Fireman” and “Lotus.”
(a)
(b)We then estimate the average PSNR per view for the two test sequences. Table1 lists the average residual crosstalk (PSNR) of each view of the two test sequences in results by the method in [11] and our method. These comparative results clearly demonstrate that the crosstalk reduction is better performed in our results.Table 1
Crosstalk reduction results (PSNR) comparison in two sequences.
Sequences
PSNR (dB)
Before correction
Method in [11]
Our method
“Fireman”
View 1
26.0711
29.0970
32.5948
View 2
31.3607
34.7981
40.8481
View 3
31.5242
34.9284
41.7979
View 4
32.1086
35.2510
41.5759
View 5
32.0063
35.5148
43.1056
View 6
31.3716
34.7109
41.0957
View 7
31.3093
34.5597
40.3676
View 8
26.1231
29.2921
33.0626
“Lotus”
View 1
30.1898
34.1157
39.7694
View 2
36.0433
39.9944
46.7904
View 3
36.0373
40.0222
47.4171
View 4
36.0175
39.9877
47.2790
View 5
36.0111
39.9765
47.2212
View 6
35.9936
39.9391
47.0899
View 7
35.9739
39.8958
46.5392
View 8
30.1766
34.6226
42.0747To further verify the effectiveness of the proposed method, a photometer is used to measure the luminance at the optimal viewing position in our experiment. The screen we use for the measurements is the same 8-view 3D lenticular display as mentioned above. The screen is placed in a dark room and test images (before and after correction) and various reference images are displayed on it alternatively. The typical experiment measures crosstalk based on maximum pixel value. However, the pixel value is a random variable from 0 to 255 in the practical application, which is rarely 0 or 255. Furthermore, while the intended view image is observed, the other view images are all observed at the same time, rather than being observed alternatively. In typical experiment, when the luminance measured at thenth optimal viewing position is near to the intended luminance in the nth test image, the influence on the other viewing position is not evaluated. To be fair, we decide to use only one test image and 8 reference images for comparison. The test image is composed of 8 view images, V1=V2=20, V3=V4=220, V5=V6=40, and V7=V8=200, where Vn denotes pixel value of the nth view image. The luminance In is measured at thenth optimal viewing position by the photometer when the test image is presented on the screen. The intended view image’s luminance Irefn for the nth viewing position is obtained at the nth viewing position when the nth reference image is presented on the screen. V1=V2=⋯=V8=Vn in the nth reference image; that is, there is no crosstalk for nth view image. Absolute deviation is used to evaluate the performance, which is defined as |In-Irefn|. The corrected test images by the method in [11] and our method are presented on the display screen, respectively. Table 2 lists the deviation values of the correction results for each view by our method and the method in [11]. These comparative results clearly demonstrate that luminance measured in our result is very close to the intended view image’s luminance, which means the viewer crosstalk is eliminated largely by the proposed method.Table 2
Crosstalk reduction results (absolute deviation) comparison.
View number
Absolute deviation (Cd/m2)
Method in [11]
Our method
1
6.609
5.968
2
8.532
7.919
3
18.503
2.000
4
16.945
0.855
5
3.040
2.584
6
11.063
10.624
7
21.049
2.042
8
21.173
1.741


## 4. Conclusions

Despite the fact that many researchers have been devoted to evaluating and eliminating the crosstalk in the stereoscopic display, there is still a lack of work on crosstalk reduction on multiview 3D displays. Typical methods for crosstalk measurement are based on the experiments, which cannot measure the precise crosstalk. In this paper, a system-introduced crosstalk measurement method is proposed to obtain crosstalk when slant angle of parallax barrier is an independent variable. Through this method, the crosstalk coefficient matrixA is obtained. Moreover, we improve the lattice search algorithm to reduce the crosstalk on multiview 3D displays, which overcomes some shortcomings of algorithm BGBF. Both simulations and experimental results are presented to demonstrate that proposed method is capable of effectively reducing crosstalk on multiview 3D displays.In practical applications, our system-introduced crosstalk measurement method can be employed to study the performance of multiview 3D techniques and our crosstalk reduction technology can improve the multiview 3D viewing quality. We hope the reader can implement our algorithms without difficulty.



---

*Source: 428602-2014-07-24.xml*