\begin{figure}
\centering
\begin{subfigure}{0.45\linewidth}
\includegraphics[width=\linewidth]{figure_18_1.pdf}
\end{subfigure}
\begin{subfigure}{0.45\linewidth}
\includegraphics[width=\linewidth]{figure_18_2.pdf}
\end{subfigure}
\begin{subfigure}{0.45\linewidth}
\includegraphics[width=\linewidth]{figure_18_3.pdf}
\end{subfigure}
\begin{subfigure}{0.45\linewidth}
\includegraphics[width=\linewidth]{figure_18_4.pdf}
\end{subfigure}
\caption{Failure case illustration. First Row: we test our method in wide baseline cases of light field, segmentation tends to inconsistent due to large view point changes. Second Row: The visual results of the non-Lambertian cases. Our method discovers correct object boundaries, but mistakenly separates objects due to incorrect depth matches.}
\label{figure_18}
\end{figure}

\begin{figure}
\centering
\begin{subfigure}{0.45\linewidth}
\includegraphics[width=\linewidth]{figure_19_1.pdf}
\end{subfigure}
\begin{subfigure}{0.45\linewidth}
\includegraphics[width=\linewidth]{figure_19_2.pdf}
\end{subfigure}
\begin{subfigure}{0.45\linewidth}
\includegraphics[width=\linewidth]{figure_19_3.pdf}
\end{subfigure}
\begin{subfigure}{0.45\linewidth}
\includegraphics[width=\linewidth]{figure_19_4.pdf}
\end{subfigure}
\caption{User guided object segmentation. Regions comprising a single object are selected by a user. The regions themselves are not manually altered.}
\label{figure_19}
\end{figure}

\section{APPLICATIONS}

Segmentation is a starting point for many processes in image manipulation and computer vision. In the following we highlight several applications of our light field segmentation method.

\subsection{User-guided Object Segmentation}

Like most 2D and video segmentation methods, our method segments the light field into regions of consistent appearance, but not into semantic objects. However with a simple user interface, we can manually select multiple regions that comprise a single object. Examples of this user-guided object segmentation are shown in Fig. \ref{fig:figure_19}.

\subsection{Light Field Flattening}

Image flattening refers to the suppression of texture detail while preserving strong scene edges and overall image structure. Here, we extend an existing 2D method \cite{Li_2015} to 4D. Specifically, we take into consideration the $L_1$ sparsity in spatial slices, angular patches as well as the 4D light field segmentation, and jointly minimize the pixel variation and approximation error as detailed in the following.

\textit{Spatial Term.} $f_i$ is the $Lab$ feature vector of pixel $p_i$.

\begin{equation}
E_l = \sum_i \sum_{p_j \in N_h(p_i)} w_{ij}\|L(i) - L(j)\|_1,
\end{equation}

where $N_h(p_i)$ is a spatial local $h \times h$ patch. $w_{ij}$ is the affinity between pixel $p_i$ and $p_j$. Here, we simply use Euclidean distance with a normalization function.

\textit{Angular Term.} We prefer a uniform intensity values over simple angular patches of the light field, and smooth exposure variation in different spatial slices. Similar to Eq. 24, we formulate our angular flattening term as

\begin{equation}
E_a = \sum_i \sum_{p_j \in N_a(p_i)} w_{ij}\|L(i) - L(j)\|_1,
\end{equation}

where $N_a(p_i)$ is the angular patch that $p_i$ lies in.

\textit{Segmentation Term.} The segmentation provides extra cues to include more pixels for avoiding the influence of shading, reflectance or noise.

\begin{equation}
E_s = \sum_{p_i \in S_k} \sum_{p_j \in S_k} w_{ij}\|L(i) - L(j)\|_1,
\end{equation}

\textit{Data Fidelity Term.} To avoid trivial solution, smoothed light field should be similar to original light field, which is formulated as,

\begin{equation}
E_d = \sum_i \|L(i) - L^{ini}(i)\|_2,
\end{equation}

where $L^{ini}$ is original light field data. The overall objective function is the sum of those terms,

\begin{equation}
E = E_d + \alpha E_l + \beta E_a + \gamma E_s,
\end{equation}

where $\alpha$, $\beta$, $\gamma$ are weight parameters.

Fig. 20 shows the results of the light field segmentation, where we then utilize the segmentation cue to remove fine details and preserve the main edges of the light field.

In the example of Fig. 21, we visualize our light field segmentation, edge detection results and pencil sketching. We first utilize our light field segmentation for removing fine details of light field. Then, we apply conventional edge detection method \cite{Dollar_Zitnick_2015}.