Statistical signal processing techniques for visual post-production

Digital media post-production is an industry standard step in media creation. Now that issues of speed and physical storage have largely been rendered less problematic, the emphasis has shifted toward increasing levels of automation. This thesis makes several contributions in the domain of visual post-production in an attempt to bring advanced statistical digital signal processing techniques to bear on several key problem. The problem of transfer of statistics is rst addressed. It transpires that this fundamental process underlies several key problems in lm grading. This can be understood by considering an image as of features samples. Then texture or colour aspects can be described by the statistics of texture or colour samples. The rst and major problem is that of colour grading. In that process the colour of each lm frame has to be adjusted in order to match lighting and atmosphere throughout the shot. Consider a scene shot at midday and the same scene shot later in the afternoon, a typical procedure in movie production. The lm shot during the midday shoot will have a totally di erent look and feel than the afternoon shoot. In addition, brightness can change from frame to frame due to lm degradation, editing of di erent physical lm material and even uctuating lighting (e.g. uorescent). In all the cases the solution is to design an automated process that alters the distribution of colour and brightness of each frame in such a way that it balances across all shots. The thesis proposes a new mechanism for doing this based on the iterative matching of projections of the colour distribution on di erent set of colour axes. In addition, algorithms for robustness to grain artefacts artefacts caused by stretched histogram matching, and occlusion problems are also proposed. The second part of the thesis deals with tracking of objects and contours. Both methods proposed are founded on Bayesian grounds but adopt di erent optimisation strategies. Contour tracking is achieved by Bayesian Filtering exploiting a new prior that extracts local direction information from the image. Object tracking is achieved by a Viterbi algorithm that incorporates candidates for object position available in each frame. Both these tracking tools have implications for post-production. Contour tracking for rotoscoping is well established, but up till now incorporating edge information has been di cult. MCMC Bayesian lters have had great success for object tracking in recent times, but this thesis shows that given low level content knowledge it is possible to exploit deterministic strategies instead of MCMC and yield the same or better performance. The thesis ends with work that proposes a new method for simplifying images implicitly. The idea is to create representations that are transmitted or compressed easily without exploiting heavy content information. A bit like in cartoonisation, images are re-expressed in a way which is not realistic but which conveys the same content while throwing away confusing details. Thus a textured object could be replaced by a simpli ed object in which the texture is removed for instance. The tools discussed previously could all be used in tandem to create such an e ect. This section of the work considers instead a purely implicit way to nd similar patches in images and replace them by a unique patch representation. The work draws from the success of recent texture synthesis results that exploit nonparametric image models. The result is an image which has a better viewability and can be better compressed.