# Color Harmonization, Deharmonization and Balancing in Augmented Reality
## 1 Introduction  
Modern digital technologies such as Augmented Reality (AR) allow us to enjoy new experiences in exciting ways. AR superimposes digital information (3D models, text, images, videos) directly into the real environment by providing additional information to the user [1, 2, 3, 4]. In most cases, AR content is generated by 3D artists and application developers who define its visual properties without any prior knowledge about the real-world environment in which virtual objects will be integrated. This makes it difficult to predict how digital content will appear and if it will fit into its real scenario, with adverse consequences on the AR visualization.  
Perceptual issues in AR have been previously documented [5, 6]. The color scheme used, as well as the variety of an environment, can hinder a correct perception. This could lead to depth problems [7]. Specific AR colors can hinder the outcomes of augmentation based on the similarity with the chosen color scheme. A typical example occurs with labels where text legibility is problematic due to uncontrollable environmental conditions such as colors and textures of the background on which the augmenting information is overlaid [8, 9, 10]. This can lead to perceptual issues for users who may not be able to appropriately distinguish virtual text from real objects.  
Another common problem occurs where the color of digital content may lead to a poor perception due to a conflict with the colors of the real background for interior design [11, 12], where virtual furniture is displayed in an AR environment so that virtualinformation and real-world scenes can co-exist in harmony. In this way, users can pick the perfect furniture to fit their homes or offices, taking into account their own creativity and imagination. Therefore, virtual objects blended with real ones contribute to an aesthetically pleasing environment with marginal mental efforts by users in choosing colors to be applied to virtual content. Another example of the pertinence of this problem is related to AR applications for gaming and entertainment [13; 14], where colors associated with 3D models of characters are usually predefined by developers. In these circumstances, it is very difficult to recognize if colors are fit to ensure a good visibility of virtual information.  
To address these issues, this paper proposes a set of recoloring methods to recolor AR content that is integrated into the real world. Given an arbitrary real background, the colors associated with virtual objects are automatically modified by taking into account real colors. In particular, the recoloring process can follow three different approaches, i.e., harmonic, disharmonic, and balance.  
The first approach aims to harmonize the colors of virtual objects to make them consistent with the colors of the real background and reach a more pleasing effect to a human eye. The second approach is based on the generation of disharmonious colors with respect to real ones to be associated with virtual objects. The aim is to improve the visibility of the virtual content by obtaining a high contrast between real and virtual objects. The third approach combines both the aforementioned ones and aims to generate harmonious colors while ensuring a good contrast between virtual and real objects. To evaluate the proposed recoloring methods, a preliminary experimentation has been carried out with three different case studies in an indoor controlled environment.  
The rest of the paper is structured as follows: Section 2 is a review of the related works, while Section 3 describes the proposed AR recoloring methodology in detail. Experiments and discussions are detailed in Section 4, followed by conclusions.