# Generation of Stereo Images Based on a View Synthesis Network
## 3 The Proposed Approach
### View Synthesis of Objects  
We only retained the disocclusion-aware AFN (DOAFN); we discarded the visibility map after training, which makes the network synonymous with the AFN, and modified some parts to fit our model. In addition to the visibility map, the differences introduced in using the DOAFN were that we changed the size of the rotation input from \(17\times 1\times 1\) to \(3\times 1\times 1\) and used bicubic sampling on the appearance flow. One of the limitations of previous view synthesis networks [17,18,20] is that the trained model of the network is only for one object category. For example, if we train the network using the training data of cars, we cannot apply the trained model to the testing data of chairs. This is due to the complexity of the task of view synthesis, which usually requires \(360^{\circ}\) views to be generated using the input view. However, in our model, we rotate objects from the left-eye view to the right-eye view, which means that the object views change by only a few degrees and always rotate clockwise. In the DOAFN, the size of the rotation input is \(17\times 1\times 1\), which is a one-hot array representing angles from \(20^{\circ}\) to \(340^{\circ}\) with an interval of \(20^{\circ}\). For example, \([0,1,0,0,\dots\ 0]\) represents a \(40^{\circ}\) rotation. However, in our model, we use a \(3\times 1\times 1\) array, representing angles from \(5^{\circ}\) to \(15^{\circ}\). Because of the relatively small difference between the left-eye view and the right-eye view, not only did we reduce the range of rotation, but we also lessened the interval from \(20^{\circ}\) to \(5^{\circ}\), which produces subtler changes. In addition, we were not required to consider negative degrees, such as \(-15^{\circ}\) or \(345^{\circ}\), which is a benefit of the property of clockwise rotation. These changes reduce the complexity of view synthesis and allow the training of various categories of objects at the same time. Our model is trained with 38 categories of indoor objects from the ShapeNet [29], including beds, benches, bookshelves, chairs, sofas and tables. The total number of objects in the training data set is 8233. When we applied the trained model with our modified AFN to the objects segmented from the input image, the rotation index \(r_{i}\) was sent into the one-hot array using linear interpolation, as follows:  
\[x_{[r_{i}]}=1-(r_{i}-[r_{i}]),\ x_{[r_{i}]+1}=1-x_{[r_{i}]},\]where \(x_{[r_{i}]}\) and \(x_{[r_{i}]+1}\) are elements in the one-hot array. For example, if \(r_{i}=1.8\), the one-hot array will be [0.2, 0.8, 0], which represents \(9^{\circ}\) ((\(5\times 0.2\)) + (\(10\times 0.8\)) = 9). Thus, the degrees of rotation are consecutive. In the DOAFN, a differentiable sampling layer was observed following the appearance flow output. These layers are the kernels of the network. The appearance flow is a two-channel map that represents the correlation between pixels in the target view and in the input view, as presented in Figure 4. Figure 4a shows the input view of the objects, Figure 4b shows the RGB (red-green-blue) image of the appearance flow for visualization, and Figure 4c shows the target view of the objects generated using the input view and appearance flow. Following the appearance flow, the network can use bilinear sampling to sample pixels from the input view and then generate a new view. The bilinear sampling [17; 20] equation is as follows:  
\[I_{t}^{i,j}=\sum_{(h,w)\in N}I_{s}^{h,w}\text{max}(0,\ 1-\left|F_{y}^{i,j}-h \right|)\text{max}(0,\ 1-\left|F_{x}^{i,j}-w\right|)\]  
where \(I_{t}^{i,j}\) is the target view, \(I_{s}^{i,j}\) is the input view, \((i,j)\) are the coordinates of the pixel in the target view, and \((h,w)\in N\) represents its four neighbors. \(F_{x}^{i,j}\) and \(F_{y}^{i,j}\) indicate the \(x\) and \(y\) coordinates of a target location \((i,j)\) in the appearance flow \(F\). When the DOAFN is trained, this operation is converted to the differentiable sampling layer that efficiently computes time.