XLerated AI — Part V — Hollywood
How AI is going to impact our lives
Category: Virtual Reality
Author: Ram Nathaniel

XLerated AI — Part V — Hollywood
How AI is going to impact our lives
This is part 5 of a series of posts about how AI is going to impact our lives. In previous posts, we talked about types of AI, and how AI is going to revolutionize the field of cleaning, sales and the internet of things. In this post I will talk about how AI is going to impact the movie industry.
What does AI have to do with making movies?
Hollywood, at its core, is all about story telling. Acting, props, soundtracks, and animations are all trying to make the story more believable, aimed at getting us to emotionally respond.
AI can play many important roles in this industry, from helping make special effects seem more real and impressive, to enabling new technologies for a more immersive experience, to interactively generating content.
New technologies are coming
Augmented reality. Source: Microsoft hololens
There are two disruptive technologies knocking on our doors, but haven’t been adopted by the mass market just yet:
Virtual reality (VR) is a technology that creates the experience as if you are in a virtual or imaginary environment. The image you see depends on the orientation you look at as if you would have seen it if you were there.
Augmented reality (AR) is a technology that lets you experience the real world, but added with virtual elements that appear to be consistent with the real world.
Augmented Reality. Source: Magic Leap
While some implementation of these technologies can be created without AI, in most of today’s implementations AI plays an important role in lowering costs. The bigger contribution that AI can do, however, is by making these worlds interactive.
Holoportation: how AR will influence our communication
Generating content
One thing that deep learning seems to be very good at is copying styles. The technique is called generative adversarial networks - or GAN. The idea is that if we have content, e.g., a story, image or a video, where we train one deep learning network generate a fake one, and another to tell apart the fake from the real. The two networks are trained together, and so the first network gets better and better at creating content which is similar to the original one.
Once trained, this network can be used to create more content with this style.
Original image, and generated image (of my wife and me :-)
This technology can mean a major breakthrough and the dream of every Hollywood producer — to be able to create content that the human eye can not distinguish from the real thing.
Imagine a world where Hollywood can take one actor’s face and place it over another actor’s face perfectly, with all facial expressions in tact. This can mean that the production can add scenes to a movie without the need of the original cast on the set. This technology is already available, and was used in order to create fake porn movies by celebrities like the one made with wonder woman Gal Gadot.
Edit: The idea of replacing faces is booming now, known as Deep Fake.
A more elaborated explanation can be found here.
3D face reconstruction. Source: Matan Sela, Technion, Israel
Next steps
In the very near future, all of these technologies will dramatically influence the industry. The roadmap forward is pretty clear:
Step 1 — moving to a personal experience
The movies industry started in cinemas where hundreds of viewers experienced it together. Since then, it made a switch to the smaller screen in our living rooms, where several people experience it together. The next step would be adapting to a the single viewer. This opens up the possibility to adapt the experience to each viewer personally.
This has already started to take effect, for example in YouTube, where using AI the system automatically plays the next video, based on its understanding and profiling of the user.
How fast will the general public adopt AR/VR? This is still unclear, as it requires new hardware, something that the industry has always had a hard time selling.
The movie Avatar — computer generated effects
Step 2 — AI generated appearance
In some aspects this is already happening in many of the special effects we see on the screen. What we don’t realise is that even today many of the characters we see on the screen in animated cartoons were created by some form of AI. These are usually the extras or the background characters — like a flock of birds. So far it is usually used to save work for the animators.
Soon, however, the role of AI will be much bigger — the ability to put any face and body onto any actor has so far been used mainly for fictional characters. This used to require a lot of work and at great cost. However, soon we will see how this can be achieved by anyone in real time using smartphones with a level not detectable by the human eye. We are going to reach an age where the fact that you saw something on video doesn’t mean it happened.
Step 3 — Interactive scripts
Computer games are today a huge production, basically forming a story line that is dependent on the player’s interaction. While the level of involvement of a player in a computer game is very different then the passive viewer who watches TV, AI will soon be able to sense the viewer’s reaction to the content, which will allow a more interactive behavior even if the viewer does not actively participate.
Imagine watching a movie with an emotional scene. One viewer is deeply touched, and the scene that took 2:30 minutes was just right for him. Another viewer is loosing patience and just wants to “cut to the chase”. If the director has prepared more than one version of that scene, the AI can decide in real time, according to the viewer’s reaction and preferences, which version to show.
Step 4 — Interactive content
While the previous step had a movie with more than one plot line, and the TV decided which plot line to show based on the sensing of the viewer, in this stage we will start seeing computer generated plot lines that are created in real time based on how the TV senses the the viewer’s reaction.
The creator will provide basic guidelines, and basic style, and perhaps some basic scenes, and the AI will decide on-the-fly how to best create the viewing experience based on our emotional response and our personal preferences.
Although it will be technologically possible one day I believe that most, if not all of the content, will be guided by a human to some narrative and general plot line.
I must emphasize that even though people have different flavours and different emotional triggers, we want to experience entertainment together, talk about it together, and feel part of a group.
The human touch
Between a more immersive experience through AR and VR, and computer generated content that is responsive and interactive, the human element will rapidly change in the industry.
Today, it requires 400 people to produce a movie, all working together and in sync. I predict that that is going to dramatically change:
Actors will create a hard drive with video clips of that character, each one separately, not knowing in which scenes it will eventually be embedded.
The locations and scenery will be filmed separately by the production.
The writer will enter a general plot and general messaging, either in the form of a book, or something not as finished.
Photographers will create a sampling of the photography style, or choose one that was already created in the past.
The sound crew will create a collection of sounds to use, not knowing how or when they will be used.
The AI will put it all together, and compose it into an interactive experience. The director will point the path, and the AI will discover it.
Low cost productions will use libraries of scenery and actors off-the-shelf to turn a book into a movie.
Movies are an art form, and as such will probably never die. Not every movie will be interactive, and not every movie will be composed by AI, but the AI generated interactive movies will claim their share in the near future.