Projet ANR-12-JS02-0008 – CineCitta : Cinématography Virtuelle Interactive
Programme : JCJC - SIMI 2 - Science informatique et applications (JCJC SIMI 2)
Année de financement : 2012
Coordinateur du projet : Monsieur Marc Christie (Université de Rennes 1 / Institut de Recherche en Informatique et Systèmes Aléatoires)
Partenaires : Institut de Recherche en Informatique et Systèmes Aléatoires
Montant : 208166.0 euros
Date de début : 2012
Durée : 36 mois
Résumé : Cinematographie Virtuelle InteractiveThe main objective of this proposal is to propose and evaluate a new workflow which mixes user interaction using motion-tracked cameras and automated computation aspects for interactive virtual cinematography that will better support user creativity. We propose a novel cinematographic workflow that features a dynamic collaboration of a creative human filmmaker with an automated virtual camera planner.
A novel workflow for intelligent virtual cinematographyThe underlying scientific and technical challenges are :(i) the ability to generate relevant viewpoint suggestions following classical cinematic conventions. (ii)  the ability to formalize and represent a number of characteristic elements of cinematographic style.(iii)  the integration of motion-tracked cameras in the workflow.
ApproachApproach:(i)   provide a computationally efficient and expressive model to formalize a wide range of characteristic composition elements;(ii)   propose a methodology to extract characteristic elements of cinematographic style and genre using reinforcement learning;(iii)   propose novel interactions metaphors for handling complex scenes with motion-tracked cameras;(iv)   gather feedback from professionals and evaluate our approaches through collaborations with film schools (ESRA, LISAA,...).
Résumé de la soumission : The main objective of this proposal is to propose and evaluate a new workflow which mixes user interaction using motion-tracked cameras and automated computation aspects for interactive virtual cinematography that will better support user creativity. We propose a novel cinematographic workflow that features a dynamic collaboration of a creative human filmmaker with an automated virtual camera planner. This process aims at enhancing the filmmaker’s creative potential by enabling very rapid exploration of a wide range of viewpoint suggestions of a 3D environment. It looks at enhancing the quality and utility of the automated planner’s suggestions by adapting and reacting to the creative choices made by the filmmaker. This requires three advances in the field. First, the ability to generate relevant viewpoint suggestions following classical cinematic conventions. The  formalization of these conventions in a computationally efficient and expressive model is a challenging task in order to select and propose the user with a relevant subset of viewpoints among millions of possibilities. Second, the ability to analyze data from real movies in order to formalize some elements of cinematographic style and genre. The issue here is to encode these characteristic elements of style and genre and let the users select which genre they prefer on a given scene. In this project we propose to characterize elements of style and genre using reinforcement learning techniques from hand-annotated real movies. Third, the integration of motion-tracked cameras in the workflow. Motion-tracked cameras represent a great potential for cinematographic content creation. However given that tracking spaces are of limited size, there is a need to provide novel interaction metaphors to ease the process of content creation with tracked cameras. Finally we propose an evaluation of our tool involving both professionals and film schools.Proposing tools to interactively assist users in the design of shots, edits and camera paths is a novel and ambitious research path which strongly contrasts with main research in the field, more involved in the design of declarative viewpoint computation, optimization and planning techniques. Furthermore this project displays a strong potential for industrial transfer.
Résultats : RAS
Perspectives : RAS
Publications et brevets : Soumissions en cours (SCA, Pacific Graphics)