# Exploring the role of generative AI in environment design for animated films: Challenges in capturing atmospheres

Khaoula Raboudi

► To cite this version:

Khaoula Raboudi. Exploring the role of generative AI in environment design for animated films: Challenges in capturing atmospheres. Conférence Internationale Multidisciplinaire sur l'Intelligence Artificielle et le Multimédia, Institut Supérieur des Arts Multimédia de la Manouba, Nov 2024,HAM-MAMET, Tunisia. hal-04868102v2

HAL Id: hal-04868102

https://hal.science/hal-04868102v2

Submitted on 10 Jan 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Exploring the role of generative AI in environment
design for animated films: Challenges in capturing
ambiances

Khaoula Raboudi

Univ. Manouba, ISAMM,
2086 Manouba, Tunisia

Univ. Grenoble Alpes, CNRS, ENSAG,
AAU-CRESSON,
38000 Grenoble, France

https://orcid.org/0009-0004-1494-953X

***Abstract***— This paper examines the role of generative AI in environment design for animated films, focusing on the challenge of creating immersive ambiances. We emphasize the importance of ambiance in enhancing narrative depth and viewer engagement. While technologies like Stable Diffusion and GANs can generate visually appealing images, they often fail to capture the complex sensory dimensions of ambiance, including both material and immaterial aspects. We argue that for AI-generated designs to be effective, a deeper understanding of human perception is essential.

***Keywords***— *Environment design, Animated film, Ambiances, Generative AI.*

I. INTRODUCTION

The animation industry currently encompasses cartoons and animated films, supported by art, science, and technology [1]. It has consistently embraced technological advancements and rapid digital transformation. Today marks a distinctive era, characterized by remarkable advancements in artificial intelligence (AI). AI, along with machine learning (ML) and deep learning (DL), is driving a new wave of innovation that enhances computational design capabilities. This technology has significantly impacted numerous fields, leading to a radical paradigm shift, with generative AI capturing global attention like never before. Generative AI refers to systems capable of creating new content, including novel concepts, characters, and environment designs. In this paper, we focus specifically on environment design in animated films and explore how generative AI can be leveraged to create content that enhances the overall cinematic experience.

II. ENVIRONMENT DESIGN AND AMBIANCE

Environmental design is more than just a backdrop; it plays a crucial role in the narrative development of the story and in evoking character emotions, much like in video games. It involves crafting visual spaces in a film that not only support the story but also create a spatial narrative. This process involves organizing space, shaping characters' postures, and their interactions with the environment. All of which guide the audience to deepen their narrative experience. Thus, environment design goes beyond just lights, color tones, textures, or shot composition; it fundamentally shapes the scene's ambiance. We adopt here the definition of ambiance used by Jean Paul Thibaud [2], which describes it as the manifestations of phenomena and their perception by the user. Whether consciously or unconsciously, ambiance engages in complex mental representation processes, combining psychosensory perceptions with cognitive constructs.

Huang [3] argues that ambiance is a crucial element in animation, playing a decisive role in the final effects of the film design. Although the term "ambiance" is not always explicitly addressed in animated films, its significance is evident in various cinematic works, where the term "mood" often emphasizes the emotional responses elicited from viewers. This concern is integral to the narratives of Hayao Miyazaki's films, where landscapes are crafted to convey deep symbolic meanings, fostering a symbiotic connection between the viewer and the depicted world [4].

Additionally, animated films exemplify the impact of environment design on emotional engagement. For instance, the vibrant and diverse color palette in Zootopia's various districts—such as the lush greens of the Rainforest District and the icy blues of Tundratown—conveys different moods, immersing viewers in the world of Zootopia. In The Incredibles, the sleek, modern design of Syndrome's Nomanisan Island contrasts with the more traditional design of the Parr family's home, highlighting the dichotomy between the ordinary and extraordinary in the film's narrative.

These elements of environment design are essential for crafting immersive ambiances that enhance storytelling. While they are not the only instances, they effectively illustrate how environment design amplifies both the narrative and the audience's connection to the characters and the world they inhabit.

III. GENERATIVE AI FOR ENVIRONMENT DESIGN

Generative AI has significantly transformed the whole process involved in producing an artwork. Particularly, AI technology is reshaping the animation industry's value chain, encompassing everything from conceptualization and pre-production to film marketing. Safavi Jahromi and Ghazinoory [5] argue that AI technologies provide animation studios with opportunities to boost creativity, optimize production workflows, and deliver high-quality content to their audiences. Specifically, in the pre-production phase, considered as the most labor-intensive part of a project, Song & Yip [6] emphasize that AI-based tools enhance creativity in ideation, enabling filmmakers to develop richer content, craft compelling characters, and design innovative scenes.

AI algorithms can generate background elements and even entire scenes based on textual, commonly known as the user prompts. Tools such as Midjourney [7], Adobe Firefly [8], Stable Diffusion XL [9] and DALL-E [10] mark the dawn of a new era of creative exploration. As of today, these design
generators are still in their early stages and may seem
rudimentary, particularly at the time of this writing, especially
when compared to what is expected to become widespread in
the near future—possibly within the next few months or
weeks.

Furthermore, film environment design is enhanced by AI-
powered rendering engines. AI-based upscaling and image
enhancement techniques improve visual quality by increasing
resolution and refining details, ultimately enriching the
viewing experience and enabling companies to deliver high-
quality content. Popular tools for film background design now
feature AI plugins. For instance, Blender, a widely used tool
for visual effects (VFX) and 3D animation, incorporates AI
plugins for automated texture generation, real-time scene
optimization, and advanced lighting effects. Similarly,
Houdini integrates AI-driven features such as procedural
modeling and simulation, enabling artists to design complex
environments with greater efficiency

In this context, several authors [11] [12] [13] have
explored the integration of AI into the design process through
image generators. The findings suggest that AI offers a
distinct perspective, opening new avenues for enhancing the
design process. Chiou and al. [12] argues that AI integration
contributes to a fresh form of self-expression and
communication, providing new opportunities for artistic
expression.

AI is also enhancing scene design and content creation in
Virtual Reality (VR) and Augmented Reality (AR)
environments. Recent studies demonstrate that VR technology
significantly improves the quality of film and television
animation through advanced light control and image
alignment techniques. AI-generated content is being
integrated into AR displays [14], offering innovative user
experiences, while systems like GetWild [15] facilitate
efficient 3D modeling and editing in immersive virtual spaces.

IV. CHALLENGES AND CONCLUSION

AI-based generation tools have profoundly impacted the
work of many artists, expanding the possibilities for artistic
expression. However, in this context raises an important
question: in an era of increasingly complex relationships
between creativity and technology, to what extent can
generative AI truly understand and effectively generate
environmental ambiances?

While AI can mimic certain visual and sensory cues,
creating immersive environments requires a deeper
understanding of how humans perceive and interact with
space. The challenge lies in teaching AI not only to replicate
design elements but also to capture the subtle nuances of
atmospheric experiences. As Jean Paul Thibault [2] argues,
ambiance exists within a material framework but it goes
beyond being just a physical environment. It comprises both
physical elements, such as light, textures, and colors, and
intangible qualities like emotions, perceptions, and social
interactions. Ambiance is inherently multisensory, engaging
all modes of perception—vision, hearing, olfaction, touch,
taste, and movement.

Ultimately, designing an environment involves crafting
the scene's ambiance by focusing on lighting, color tones, shot
composition, sound, and the interactions and postures of
characters to create a convincing and memorable experience
for viewers. However, the techniques currently employed in

AI-based environment design, such as Stable Diffusion and
GANs, present both potential and limitations. GANs consist
of two neural networks —the generator and the
discriminator—which work against each other to create
images. Whereas stable diffusion is built on a diffusion
process, which gradually transforms random noise into images
through a series of denoising steps. While both techniques
promise exciting possibilities for environment design, they do
not come without limitations. These models require vast
amounts of data and they are not capable of capturing
ambiances nuances, primarily, because they are trained only
on images and text descriptions. While images can convey
atmospheric experiences, they represent only a thin slice of
these experiences, leaving many sensory and atmospheric
dimensions unaddressed. They simultaneously impoverish the
experience by failing to capture key aspects such as auditory,
and kinesthetic sensations. Moreover, the used images for
training are often static, unable to portray movement and
human interactions—crucial elements that contribute to a full
sense of immersion in a place. Thus, while these techniques
effectively convey the material qualities of environments, they
fall short in representing the immaterial qualities that are
equally significant.

In summary, we argue that the future of AI-generated
designs depends not only on technological advances but also
on how we integrate human sensitivity. For AI to become a
truly effective tool in ambiance creation, it must possess a
deep understanding of human perception, taking into account
the often-elusive dimensions of the environment. Therefore, a
thoughtful examination is needed to identify the data types and
formats that should be integrated into AI systems to enhance
their ability to understand and create richer designs
Additionally, the concept of ambiance should be approached
holistically; while it can be decomposed, it must be
understood as a cohesive whole within a systemic framework.
To our knowledge, this critical reflection has yet to be
thoroughly explored by researchers.

REFERENCES

[1] H. Yoon, « Globalization of the animation industry: multi-scalar linkages of six animation production centers », Int. J. Cult. Policy, vol. 23, no 5, p. 634-651, sept. 2017, doi: 10.1080/10286632.2015.1084298.

[2] J.-P. Thibaud, « Ambiance », in *Psychologie environnementale* : 100 notions clés, Dunod, in *Univers Psy.*, Paris, 2022.

[3] X.-Y. Huang, « Explore scene atmosphere in animation creation », in 2016 International Conference on Applied System Innovation (ICASI), Okinawa, Japan: IEEE, mai 2016, p. 1-4. doi: 10.1109/ICASI.2016.7539924.

[4] [C. Bartolomei, A. Ippolito, et D. Mezzino, « Representation of Landscape and Ecological Vision in Miyazaki's Filmography », Sustainability, 2023, [En ligne]. Disponible sur: https://api.semanticscholar.org/CorpusID:264454164

[5] G. Safavi Jahromi et S. Ghazinoory, « From Superman to Garfield: Four scenarios regarding the effects of industry 4.0 on animation companies », Technol. Soc., vol. 77, p. 102511, juin 2024, doi: 10.1016/j.techsoc.2024.102511.

[6] J. Song et D. Yip, « Exploring the Intersection of AI Art and Film: A Case Study of Giant », in 2023 IEEE International Conference on Multimedia and Expo Workshops (ICMEW), Brisbane, Australia: IEEE, juill. 2023, p. 347-352. doi: 10.1109/ICMEW59549.2023.00066.

[7] D. Holz, « Midjourney ». [En ligne]. Disponible sur: https://www.midjourney.com

[8] Adobe, « Adobe Firefly ». [En ligne]. Disponible sur: https://firefly.adobe.com/
[9] E. Mostaque, « Stable Diffusion XL ». [En ligne]. Disponible sur :
https://stability.ai/

[10] OpenAI, « DALL·E 3 ». [En ligne]. Disponible sur :
https://openai.com/dall-e-3

[11] Y. Duan et J. Zhang, « A Novel AI-Based Visual Stimuli Generation
Approach for Environment Concept Design », Comput. Intell.
Neurosci., vol. 2022, p. 1-12, juin 2022, doi: 10.1155/2022/8015492.

[12] L.-Y. Chiou, P.-K. Hung, R.-H. Liang, et C.-T. Wang, « Designing with AI: An Exploration of Co-Ideation with Image Generators », in Proceedings of the 2023 ACM Designing Interactive Systems Conference, Pittsburgh PA USA: ACM, juill. 2023, p. 1941-1954. doi: 10.1145/3563657.3596001.

[13] H. Tong, A. Türel, H. Şenkal, S. F. Yagci Ergun, O. Z. Güzelci, et S. Alaçam, « Can AI Function As A New Mode of Sketching: A Teaching Experiment with Freshman », Int. J. Emerg. Technol. Learn. IJET, vol. 18, no 18, p. 234-248, sept. 2023, doi: 10.3991/ijet.v18i18.42603.

[14] Y. Hu et al., « Investigating the Design Considerations for Integrating
Text-to-Image Generative AI within Augmented Reality
Environments », 2023, arXiv. doi: 10.48550/ARXIV.2303.16593.

[15] S. M. Wong, C.-W. Chen, T.-Y. Pan, H.-K. Chu, et M.-C. Hu, « GetWild: A VR Editing System with AI-Generated 3D Object and Terrain », in Proceedings of the 30th ACM International Conference on Multimedia, Lisboa Portugal: ACM, oct. 2022, p. 6988-6990. doi: 10.1145/3503161.3547733.