Avo, Bandersnatch, and the New Grammar of Television (and Games)— Part 3: Interacting with Attention-Seeking Cameras
By: Dan Hill
Tags: Netflix, Augmented Reality, Film, Television, Videogames

Playdeo’s balance of code, film and interaction has created a generative space which has to walk through these possibilities, automatically (or algorithmically), and in real-time. The attention-seeking cameras have to automatically do what Hitchcock was doing — not in a Hitchcock style, although it might be interesting to provide training data for the cameras based on different directors! Activate Godard-mode! Activate Bruckheimer-mode!— but in terms of “eliminating anything that interferes.”

Hitchcock said he reduced everything to this purpose:

“I’m not concerned with plausibility; that’s the easiest part of it, so why bother? … I am against virtuosity for its own sake. Technique should enrich the action. One doesn’t set the camera at a certain angle just because the cameraman happens to be enthusiastic about that spot. The only thing that matters is whether the installation of the camera at a given angle is going to give the scene its maximum impact. The beauty of image and movement, the rhythm and the effects — everything must be subordinated to the purpose.”

Making a hybrid of game and TV programme is extraordinarily complex, yet that complexity has been neatly “subordinated to the purpose” by Playdeo. Its image and movement, rhythm and effects are being calibrated by code in real-time, subordinate to this balance of player’s agency and plot development.

Truffaut later points out to Hitchcock that “in your technique everything is subordinated to the dramatic impact; the camera, in fact, accompanies the characters like an escort.” This is entirely how the virtual cameras hover around Avo, and your interactions with the character, just as Hitchcock would never show a train speeding through the countryside from a distance (“giving us the viewpoint of a cow”, he snorted) but his cameras would stay on the train, with the characters. Hitchcock had his rules, even about how, as Truffaut put it, the cameras should not “precede the action instead of accompanying it … the camera should never anticipate what’s about to follow.” Hitchcock added, “If a character moves around and you want to retain the emotion on his face, the only way to do that is to travel the close up.”

In Avo, the cameras swing back for aesthetic effect only, when Avo has to travel a distance, or there’s a chance of a wide. When conveying emotion, to Hitchcock’s point, the camera is tight on Avo. One can see how rules could be developed, to enable this array of virtual cameras for CGI and attention-seeking cameras for predetermined elements. (Playdeo’s founders almost exemplify these various inputs themselves: Timo Arnall’s ability to fluidly synthesise high-craft film with design; Nick Ludlam’s code creating a vocabulary for this new dialogue, for the off-screen chatter between those cameras and the CGI; Jack Schulze’s capability with interactive product design, as well as character and narrative.)

Playdeo team

New Grammar of the Film Language

So does this suggest a new rule-set, a new grammar?

Daniel Arijon’s 1976 book, Grammar of the Film Language, became the de facto guide to filmmaking, in terms of framing, editing and shooting for films and TV. Its focus is on the visual grammar itself: how to shoot a narrative sequence, essentially. It provides a vocabulary, and visual language, for making film and TV. Across 600 incredibly detailed and illustrated pages, Arijon describes the way cameras ought to be positioned or move, the way that cuts should happen, the way scenes should be framed, the entire grammar of filmmaking, shot by shot, scene by scene, story by story.

Yet in those 600 pages, there is, of course, little or no mention of audience or viewers, or their context. The grammar is constrained to the frame of the film, and the depth within that frame. The output device is assumed to be a cinema, perhaps a TV in a living room, and so the interaction is necessarily lean-back, and the spatial environment perhaps assumed to be an armchair, or row of armchairs.