Approximating methods for intractable probabilistic models: Applications in neuroscience

This thesis investigates various methods for carrying out approximate inference in intractable probabilistic models. By capturing the relationships between random variables, the framework of graphical models hints at which sets of random variables pose a problem to the inferential step. The approximating techniques used in this thesis originate from the eld of statistical physics which for decades has been facing the same type of intractable computations when analyzing large systems of interacting variables e.g. magnetic spin systems. In general, these approximating techniques are known as mean eld methods. The thesis provides a brief introduction to the basic methodology of learning and inference in graphical models as well as a short review of the various types of mean eld approximations which recently have been shown to be e cient for carrying out approximate inference in intractable probabilistic models. Starting from the naive mean eld approximation we derive for the independent component analysis (ICA) model with instantaneous mixing general expressions for the posterior quantities needed to perform learning by Expectation-Maximization (EM). Furthermore, we explore the feasibility of going beyond the naive mean eld approximation for this model. In fact, it turns out that the overcomplete ICA problem can be solved using a simple linear response correction to the mean su cient statistics obtained by naive mean eld approximation. In addition, we apply to the ICA problem an adaptive version of the [Thouless, Anderson and Palmer 1977] (TAP) mean eld approach which is due to [Opper and Winther 2000c]. To illustrate the methodology on a real world problem, an explorative analysis of a functional magnetic resonance imaging (fMRI) dataset from a visual activation study is carried out using ICA with binary sources. It is shown this approach, which is computationally e cient, infers reasonable brain activation functions. Finally, we outline various ways of carrying out approximate message passing in probabilistic models for which marginalization over some of the clique variables is intractable. Resumé ( I nærværende afhandling undersøgers forskellige teknikker til at udføre approximativ inferens i beregningsmæssigt tunge probabilistiske modeller. Med udgangspunkt i teorien om gra ske modeller er det muligt direkte at få et indblik i, hvornår approximerende metoder er påkrævede. De teknikker, som er benyttet i denne afhandling, har alle deres oprindelse i statistisk fysik. I løbet af de sidste årtier har folk indenfor dette område udviklet approximerende metoder for at kunne analysere systemer med mange vekselvirkende enheder som f.eks. magnetiske spin systemer. Disse metoder går samlet under betegnelsen middelfeltsmetoder. Denne afhandling giver foruden en kort introduktion til inferens og parameterestimation i gra ske modeller også en oversigt over de forskellige metoder, som i tidens løb har vist sig egnede for approximativ inferens i probabilistiske modeller. Med udgangspunkt i den naive middelfeltsapproximation udleder vi generelle udtryk for kilde-posteriorsandsynligheden i en model for independent component analysis (ICA) hvor kilderne blandes instantant. For denne model undersøger vi fordelene ved at benytte mere advancerede approximationer. Det viser sig, at det underbestemte tilfælde, hvor der er ere kilder end mikrofoner, kan løses ved en lineær responskorrektion af de su ciente statistikker, som fås fra den naive middelfeltsapproximation. Endeligt anvender vi til dette ICA problem en adaptiv version af [Thouless, Anderson and Palmer 1977] (TAP) middelfeltsmetoden, som blev foreslået af [Opper and Winther 2000c]. Vi illustrerer metoden i en explorativ analyse af en sekvens af dynamiske hjerneskanbilleder optaget under et visuelt aktiveringsstudie. Det er vist, at denne metode, som er beregningsmæssig e ektiv, rent faktisk er i stand til at nde plausible hjerneaktiveringsmønstre. Endelig gives der forslag til, hvordan det er muligt at lave approximativ sekventiel inferens i probabilistiske modeller, hvor eksakt marginalisering over enkelte klikkepotentialer er umuligt.