# Color Harmonization, Deharmonization and Balancing in Augmented Reality
## 2 Related Work
### Recoloring Techniques for Images and Videos
[48] enhanced the harmony of the colors contained in images or photographs, where the recolored image remains as much faithful as possible to the original one. The solution is based on the harmonic schemes proposed by [49; 50] that define relationships among harmonic colors on the hue wheel in HSV color space. In this way, the best set of colors is calculated by solving an optimization problem through a cut-graph optimization while ensuring the spatial coherence of colors. The main limitation of this approach is that it cannot recognize automatically disconnected areas that belong to the same object, so that these could be shifted to a different sector of the template. [PERSON] et al. [51] presented an approach to harmonize both images and videos taking inspiration from [PERSON]'s work [48]. In particular, in the harmonization of videos, they observed a flickering effect in the post-processed video due to the lack of continuity in changing colors between adjacent frames. To solve this issue, authors introduced two different improvements. Firstly, they concentrated on processing of the frames in groups rather than individually. Secondly, an overlapping of frames between two adjacent groups was utilized to avoid inter-group flickering effect. Based on [48], [PERSON] et al. [52] presented a technique to find the best sets of colors to harmonize images based on the use of the predominant hue in the image. In this way, the best angle and the best template to minimize the harmonization function are found by a continuous use of a graph-cut optimization to guarantee the color coherence. [PERSON] et al. [53] proposed an improvement in colors harmonization process to preserve the coherence of pixels. A two-level graph cut algorithm was utilized which consists of two main parts: a region-level graph and a pixel-level graph. Firstly, the region-level graph allows for shifting regions with similar hue values to the same sector of the selected template, even if they are not close to each other. Subsequently, it generates seeds as pixels that can be interpreted as constraints under which the pixel-level graph harmonizes the colors of the image. The same approach was extended to videos in which foreground and background are treated separately by also employing optical flow estimation, region growing, and foreground extraction algorithm. [PERSON] et al. [54] developed an algorithm to harmonize composite images. Three different cases of harmonization were proposed: (1) harmonization of both foreground and background; (2) harmonization of the background without adjusting the foreground; (3) harmonization of the foreground maintaining the background unaltered. An assigning process was followed to change the hue value of pixels by employing a novel cost function for optimization. This was formulated by considering that pixels very close to each other, which have similar anharmonic hues, need to be harmonized in the same way. [PERSON] et al. [55] focused their efforts on improving works proposed by [PERSON] et al. [48] and [PERSON] [49]. The authors have introduced a saliency map to accurately predict which areas of images are more visually attractive and fit for driving the estimation of the harmonic template. Additionally, the use of a new color mapping function and a dedicated color segmentation was adopted to obtain consistent results after the harmonization process.