# Lester: Rotoscope Animation through Video Object Segmentation and Tracking
## 3 Methodology
### Contours Simplification  
One of the desired choices for the target visual style is the possibility to display very simple geometries, i.e., shape contours with short polygonal chains, in order to mimic the visual aspect of, e.g., \"Another World\". To achieve it, a series of steps are carried out. Figure 3: Workflow of the segmentation and tracking stage for the first two frames (inputs on the left, output on the right). For each frame, the result of the segmentation and tracking step is a mask with each pixel assigned an RGB color representing an object label. The mask is first divided into submasks, one for each color/label. Each submask is divided, in turn, into contours with the algorithm from [30]. Each individual contour is represented with a vector of the vertex coordinates of the boundary line segments. The number of line segments of each contour is reduced with the Douglas-Peucker algorithm [3]. The algorithm starts by joining the endpoints of the contour with a straight line. Vertices closer to the line than a given tolerance value \(t\) are removed. Next, the line is split at the point farthest from the reference line, creating two new lines. The algorithm works recursively until all vertices within the tolerance are removed. Tolerance \(t\) is one of the optional parameters of the proposed method and let the user control the level of simplification. As each contour is simplified independently, undesirable spaces can appear between the geometries of bordering contours that previously were perfectly aligned. To avoid this, and before the simplification step, all contours are dilated convolving the submask with a 4 \(\times\) 4 kernel. It is worth mentioning that the simplification algorithm may fail for certain contour geometries and values of \(t\). In that situations (usually very small contours) the original contour is kept. See Algorithm 1 and Figure 4.  
```
0:\(M\) a list containing submasks, \(\alpha\) a minimum contour area, \(t\) a tolerance value for simplification
0:\(R\) a list of contours where \(R_{i,j}\) is the simplified contour \(j\) of submask \(i\)
1:functionSimplify(\(M,\alpha,t\))
2:for\(i\gets 0\) to \(length(M)-1\)do
3:\(C_{i}\gets getContours(M_{i})\)
4:for\(j\gets 0\) to \(length(C_{i})-1\)do
5:if\(areaOf(C_{i,j})>\alpha\)then
6:\(D_{i,j}\gets dilateContour(C_{i,j})\)
7:if\(contourCanBeSimplified(D_{i,j},t)\)then
8:\(R_{i,j}\gets simplifyContour(D_{i,j},t)\)
9:else
10:\(R_{i,j}\gets D_{i,j}\)
11:endif
12:endif
13:endfor
14:endfor
15:return\(R\)
16:endfunction
17:
18:\(contourCanBeSimplified\) returns false if the Douglas-Peucker algorithm fails, as may happen for certain contour geometries and values of \(t\). Figure 4: Graphical overview of the contour simplification stage of one frame: (**a**) result of the segmentation and tracking step; (**b**) contours detection; (**c**) contours dilation; (**d**) contours simplification; (**e**) resulting colored frame. ### Finishing Details

The third and final step of the method entails applying customizable finishing details, granting users flexibility in shaping the visual style of the outcome. The most important aspect is the color. The method generates a visual guide with unique identifiers for each color segment, as shown in Figure 5. One of the inputs to the method is a color palette, which indicates the RGB values that will be applied to each segment identifier. Another finishing detail, which is optional, is the possibility of adding some schematic-looking facial features. The features are generated from the 68 facial landmarks that can be obtained using the frontal face detector described in [31], which employs a Histogram of Oriented Gradients (HOG) [32] combined with a linear classifier, an image pyramid and a sliding window detection scheme. Another option is the possibility of adding a very simple rim light effect by superimposing the same image with different brightness and displaced horizontally.