# Optical Flow Estimation with Occlusion Detection
## 2 Related Work  
Most optical flow approaches are based on a variational formulation since it can obtain sub-pixel level results naturally. Occlusion phenomena are ubiquitous in optical flow estimation, and there is abundant literature on this topic. In this section, we briefly review the variational methods and the occlusion-aware optical flow estimation.  
Following the variational model of [PERSON] and [PERSON] [23], modern optical flow estimation is usually posed as an energy minimization problem, which includes an energy function containing a brightness constancy assumption and a smoothness assumption. Hierarchical (pyramid-based) algorithms have a long history [24], and it is widely used in optical flow estimation presently, such as S2F [25] and MirrowFlow [21]. Hierarchical algorithms first estimate the optical flow on smaller, lower-resolution images and then use them to initialize higher-resolution estimates. The advantages of which include increased computation efficiency and with the ability to find better solutions (escape from local minima). [PERSON] et al. [26] systematically summarized the previous work on the data term and the smoothing term. [PERSON] et al. [27] did further work on the selection of the smoothing terms, and they found that the automatic setup of decreasing scalar functions(DF-Auto) for regularization can achieve a good balance between robustness and accuracy. LDOF [7] creatively integrated the histograms of oriented gradients (HOG) descriptors into the coarse-to-fine warping methods, and achieved the remarkable results. A quantitative analysis of optical flow estimation was proposed by [PERSON] et al. [28], and they verified that the importance of the median filtering.  
Occluded pixels have undefined flow, so that good estimation of flow depends on occlusion and vice versa. Based on the methodology, we divide them into three major categories. The first category treats occlusion as outliers and suppresses the influence of occlusion by complex penalty terms, the L1-norm potentials were introduced by [PERSON] [29] and have been widely used by many other researchers due to its robustness. Motivated by the statistics of optical flow, the Lorentzian potentials and the Charbonnier potentials are proposed successively. The second category deals with occlusion by exploiting the symmetric property of optical flow and analyzing the bidirectional optical flow. [PERSON] et al. [30] proposed one such solution, computing symmetrically dense optical flow which is consistent from frame 1 to frame 2 and frame 2 to frame 1. This is achieved by minimizing an energy function which respects occlusion. The last category builds more sophisticated frameworks to reasoning occlusion. [PERSON] et al. [31] proposed a triangulation-based framework using a geometric model that can directly account for occlusion effects. FullFlow [32] optimized a classical optical flow objective over the full space of mappings between discrete grids, which can deal with occlusion simultaneously. MR-Flow [33] proposed a deep learning framework for rigid region reasoning, which makes use of its integrity to circumvent the occlusion detection process.