hi this is Jeff Dean welcome to applications of deep neural networks of Washington University in this video we're going to look at how we can use ganz to generate additional training data for the latest on my a I course and projects click subscribe in the bell next to it to be notified of every new video Dan's have a wide array of uses beyond just the face generation that you

GANS for Semi-Supervised Learning in Keras (7.4)


often see them use for they can definitely generate other types of images but they can also work on tabular data and really any sort of data where you are attempting to have a neural network that is generating data that should be real or should or could be classified as fake the key element to having something as again is having that discriminator that tells the difference


in the generator that actually generates the data another area that we are seeing ganz use for a great deal is in the area of semi supervised training so let's first talk about what semi-supervised training actually is and see how again can be used to implement this first let's talk about supervised training and unsupervised training which you've probably seen in previous machine


learning literature but just in case you haven't supervised training is what we've been doing up to this point I would say probably the vast majority of this class is in the area of supervised learning this is where you have multiple axes in the case of tabular data or grids and other things in the case of image data but you have some sort of input coming in which is the X and you


know what the correct Y's are you are going to train the model to produce these Y's when you have these X's because later on you're going to have X's coming in where you don't know what the Y is and that's where you want the neural network or other model to be able to give you some estimate as far as what the Y value is going to actually be unsupervised training is where we have


the X's it could look just like this it would work with image data tabular or really just about anything but there is no y we're letting the neural network or whatever model it is and you know typically by the way use neural networks for unsupervised training this is usually the area of things like k-means clustering and other things your classic unsupervised training is


just going to take the inputs and cluster them in such a way so that similar ones are together these could be similar images these could be similar inputs in tabular data a variety of things semi supervised training it's actually much closer to supervised training I would say than unsupervised and this is where gams really shine and semi-supervised training you have X's


just like you have in these others but you don't have a label or a Y for every single one of them you might have a small number of them by oh no means have the complete data set label traditionally what would be done is these values that were not labeled would be left out because they there was no way to feed them into traditional supervised learning or you would train


it on the ones that you did have Y's for with classic back propagation or however you were training that particular model then you would create predictions Y predictions for all the missing values and then retrain the whole thing on the predictive values with the others in practice I never had a great deal of success with that technique but there is some theoretical basis for it with semi


supervised training and Ganz will see that there's a way that we are able to actually make use of these now semi-supervised training this does make sense from a biological standpoint if you think about a child who is seeing all sorts of vehicles as they go about their daily lives with their parents or whoever they're with and they're seeing all these vehicles as they pass on the


street and they're not labeled nobody is telling them even though that's a vehicle seeing just a barrage of images as they as they grow up they learn edges they learn other sorts of things they learn how to classify if something is on top of something else just by observing there's no particular labels then eventually somebody says hey that's a that's a bus that's that's a train


that's a bicycle using that that small handful of labels that they're given when somebody actually tells them what they're looking at or they verify it independently that is semi-supervised training because it is building on those years and years of having unlabeled data that they they didn't know what they were looking at but they knew they were looking at something and it just it gives them


additional reference that's exactly the same thing with supervised training these values even though we don't have wise they're still valuable for the neural network to be learning structure in this data as it is learning to predict the ones that we do actually in fact have the wise heart so let's look at the structure for this this is the structure of a normal image generating gang


baseline so to speak where they research started we saw this before but just to quickly review we have actual images they go into a discriminator and we have the generated images that the generator so the cyan pieces those are the two neural networks random seed values are causing that generator to generate images the discriminator is learning to better and better discriminate between


actual and generated the generator is learning to create better and better images that fool the discriminate now once this is all done you keep the generator because it generates images for you and you likely throw away the discriminator it was just there for the generator to practice against will see that this flips for semi-supervised learning and semi-supervised learning we


care about the discriminator and not so much the generator we typically throw the generator away this is how you would train a semi supervised classification neural network it's very very similar to the diagram that we just looked at in this case we're looking at how we would train it on tabular data it's a medical record the discriminator would learn to tell the difference between a fake


medical record or whatever the generator is generating this parts all the same as the previous one as is as is this part the difference is we're training it now to tell not just the difference between fake and real these are the real and this is this is fake we're teaching it to learn classes so there's four different classes of say medical record that we're looking at maybe four


different health levels we're teaching it as a classification neural network to classify between five things the four classes that were actually interested in and is it a fake once we're done training the whole thing we now have this discriminator that can tell the difference between fake and what the what the classes are we also have the generator that is able to generate these


fake medical records but we can then throw away the generator and we'll use the discriminator really truly as our actual neural network now for the medical records where we don't have the Y so we're missing this we still feed those in it's just now we're evaluating it not based on if it classified it correctly but just if it knew the difference between fake and real the


street house view data set is a image data set that is often used to demonstrate semi-supervised game learning and I have a link to a curious example external to this class that demonstrates this if you're interested in this sort of technology but what this does is you have data on these addresses from images that were taken on the sides of buildings and not all of those are


labeled or you simulate them not all being labeled and you see that the Gann is capable of learning to classify these 10-digit types even though it doesn't have labels on each of those now if you want to do the same thing for regression it becomes very similar you have two outputs so you have a multi output neural network one is the actual regression value that you're trying to


train it on and the other is the probability that it's a fake record being generated now these I'm doing tabular as just the example again these could be medical records and perhaps the regression output would be a health level or maybe a guess at how old the patient is or some other value perhaps if they have a current disease or not a prediction so it's it's doing the same


two things when you feed an medical records where we don't know the Y output then we want to see that this regression on the fake record when we're feeding in values where we have the medical record where you don't have the Y we just want to make sure that the probability that the fake record is fairly high and that's built into the training we don't so much care about what its regressing


on or what the Russian output is for ones where we do have it we're penalizing it based on how close or how far away it was from the expected Y from this and just like the classification one when we're all done with this we throw away the generator in the discriminator becomes the semi-supervised neural network that was trained on this now if you want to go further with this semi-supervised


learning technique i've given you a couple of lengths of articles that i found useful for this there is a link to the actual house data set that's a pretty interesting data set to look at it has all those house numbers above you can deal with in several ways you can deal with classifying the individual digits they give you the bounding rectangles around the digits they also


give you just the bounding rectangle of the entire set of digits if you want to so you can be classifying digits or you can be classifying the entire address it just depends on how you want to set up the problem the examples that I give you here we're using individual digits this is the original paper that first started looking at this unsupervised representation learning with deep


convolutional generative Gass general generative adversarial Network I have a link to this paper in the module thank you for watching this video in the next video we're going to take a look at some of the most cutting-edge and current research into Ganz it's a very active area of research this content change is often so subscribe to the channel to stay akha date on this course and other


you're one of the one of the only people who dared boldly to try to formalize our the idea of artificial general intelligence to have a mathematical framework for intelligence just like as we mentioned termed IHC AI X I so let me ask the basic question what is IX e ok so let me first say what it stands for because letter stands for actually that's probably the

Marcus Hutter: What is AIXI? | AI Podcast Clips


more basic question but it the first question is usually how how it's pronounced but finally I put it on the website how it's pronounced and you figured it out yeah the name comes from AI artificial intelligence and the X I is the Greek letter X I which are used for solo manav's distribution for quite stupid reasons which I'm not willing to repeat here in front of camera so it


just happened to be more less arbitrary I chose this I but it also has nice other interpretations so their actions and perceptions in this model around an agent has actions and perceptions and overtime so this is a Index IX index I so this action at time I and then followed by reception at time I will go with that I lit out the first part yes I'm just kidding I have some more


interpretations so at some point maybe five years ago or ten years ago I discovered in in Barcelona it wasn't a big church there was in you know it's stone engraved some text and the word I see appeared there very surprised and and and and happy about it and I looked it up so it is Catalan language and it means with some interpretation of dead say that's the right thing to do yeah


Eureka oh so it's almost like destined somehow came came to you in a dream so Osceola there's a Chinese word I she also written our galaxy if you transcribe that opinion then the final one is there is AI crossed with interaction be her status and it's going more to the content now so good old-fashioned AI is more about you know planning known the domestic world and induction is more


about often you know iid data and inferring models and essentially what this is a model does is combining these two and I actually also recently I think heard that in Japanese AI means love so so if you can combine excise somehow with that I think we can there might be some interesting ideas there so I let's then take the next step can you maybe talk at the big level of what is this


mathematical framework yeah so it consists essentially of two parts one is the learning and induction and prediction part and the other one is the planning part so let's come first to the learning induction prediction part which essentially explained already before so what we need for any agent to act well is that it can somehow predict what happens I mean if you have no idea what


your actions do how can you decide which acts not good or not so you need to have some model of what your actions affect so what you do is you have some experience you build models like scientists you know of your experience then you hope these models are roughly correct and then you use these models for prediction and the model is sorry to interrupt the model is based on your perception of


the world how your actions will affect our world that's not so what is the important part but it is technically important but at this stage we can just think about predicting say stock market data weather data or IQ sequences one two three four five what comes next yeah so of course our actions affect what we're doing what I come back to that in a second so and I'll keep interrupting


so just to draw a line between prediction and planning or what do you mean by prediction in this in this way it's trying to predict the the environment without your long-term action in the environment what is prediction okay if you want to put the actions in now okay then let's put in a now yes so another question ok so the simplest form of prediction is that you just have data


which you passively observe and you want to predict what happens without you know interfering as I said weather forecasting stock market IQ sequences or just anything okay and Salama of theory of induction based on compression so you look for the shortest program which describes your data sequence and then you take this program run it it reproduces your data


sequence by definition and then you let it continue running and then it will produce some predictions and you can rigorously prove that for any prediction task this is essentially the best possible predictor of course if there's a prediction task our task which is unpredictable like you know your fair coin flips yeah I cannot predict the next reckon but Solomon of Tarsus says


ok next head is probably 50% it's the best you can do so if something is unpredictable so Lamar will also not magically predict it but if there is some pattern and predictability then Solomonov induction we'll figure that out eventually and not just eventually but rather quickly and you can have proof convergence rates whatever your data is so there's pure magic in a sense


what's the catch well the catch is that is not computable and we come back to that later you cannot just implement it in even this Google resources here and run it and you know predict the stock market and become rich I mean if ray solomonoff already you know tried it at the time but the basic task is you know you're in the environment and you interact with environment to try to


learn a model the environment and the model is in the space as these all these programs and your goal is to get a bunch of programs that are simple and so let's let's go to the actions now but actually good that you asked usually I skip this part also that is also a minor contribution which I did so the action part but they're usually sort of just jump to the decision path so let me


explain to the action partner thanks for asking so you have to modify it a little bit by now not just predicting a sequence which just comes to you but you have an observation then you act somehow and then you want to predict the next observation based on the past observation and your action then you take the next action you don't care about predicting it because


you're doing it and then you get the next observation and you want more before you get it you want to predict it again based on your past excellent observation sequence you just condition extra on your actions there's an interesting alternative that you also try to predict your own actions if you want oh in the past or the future what are your future actions


wait let me wrap I think my brain is broke we should maybe discuss that later Biff after I've explained the aixi model that's an interesting variation but there's a really interesting variation and a quick comment I don't know if you want to insert that in here but you're looking at in terms of observations you're looking at the entire the big history a long history of the


observations exact that's very important the whole history from birth sort of of the agent and we can come back to that I'm also why this is important here often you know in RL you have MVPs Markov decision processes which are much more limiting okay so now we can predict conditioned on actions so even if the influence environment but prediction is not all we want to do right we also want


to act really in the world and the question is how to choose the actions and we don't want to greedily choose the actions you know just you know what is best in in the next time step and we first I should say you know what is you know how to be measure performance so we measure performance by giving the agent reward that's the so called reinforcement learning framework so


every time step you can give it a positive reward or negative reward or baby no reward it could be a very scarce right like if you play chess just at the end of the game you give +1 for winning or -1 for losing so in the aixi framework that's completely sufficient so occasionally you give a reward signal and you ask the agent to maximize river but not greedily sort of you know the


next one next one because that's very bad in the long run if you're greedy so but over the lifetime of the agent so let's assume the agent lives for M times that was there it dies in sort of hundred years sharp that's just you know the simples model to explain so it looks at the future reward some and ask what is my action sequence or actually more precisely my policy which


leads in expectation because of the know the world to the maximum reward some let me give you an analogy in chess for instance we know how to play optimally in theory it's just a minimax strategy I play the move which seems best to me under the assumption that the opponent plays the move which is best for him so best so worst for me and the assumption that he I play again the best move and


then you have this expecting max three to the end of the game and then you back propagate and then you get the best possible move so that is the optimal strategy which for norman already figured out a long time ago for playing adversarial games luckily or maybe unluckily for the theory it becomes harder the world is not always adversarial so it can be if the other


humans in cooperative fear or nature is usually I mean the dead nature is stochastic you know you know things just happen randomly or don't care about you so what you have to take into account is in noise yeah and not necessarily doesn't really so you'll replace the minimum on the opponent's side by an expectation which is general enough to include also adversarial cases so now


instead of a minimax trials you have an expected max strategy so far so good so that is well known it's called sequential decision theory but the question is on which probability distribution do you base that if I have the true probability distribution like say I play backgammon right there's dice and there's certain randomness involved you know I can calculate probabilities


and feed it in the expecting max or the signatures eg come up with the optimal decision if I have enough compute but in the before the real world we don't know that you know what is the probability you drive in front of me brakes and I don't know you know so depends on all kinds of things and especially new situations I don't know so is it this unknown thing about prediction and


there's where solomonov comes in so what you do is in sequential decision jury it just replace the true distribution which we don't know by this Universal distribution I didn't expect a talk about it but this is used for universal prediction and plug it into the sequential decision mechanism and then you get the best of both worlds you have a long-term planning agent but it


doesn't need to know anything about the world because the Solomon reduction part learns can you explicitly try to describe the universal distribution and how some of induction plays a role here yeah I'm trying to understand so what it does it so in the simplest case I said take the shortest program describing your data run it have a prediction which would be deterministic yes okay but you


should not just take a shortest program but also consider the longer ones but keep it lower a priori probability so in the Bayesian framework you say our priori any distribution which is a model or stochastic program has a certain a pre or probability which is two - two - and why - ders - length you know I could explain length of this program so longer programs are punished yeah a priori and


then you multiplied with the so called likelihood function yeah which is as the name suggests is how likely is this model given the data at hand so if you have a very wrong model it's very unlikely that this model is true so it this very small number so even if the model is simple it gets penalized by that and what you do is then you take just the sum word this is the average


over it and this gives you a probability distribution so it was universal distribution of Solomon of distribution so it's weighed by the simplicity of the program and likelihood yes it's kind of a nice idea yeah so okay and then you said there's you're playing N or M or forgot the letter steps into the future so how difficult is that problem what's involved there okay so here's a cop to


mutation problem what do we do yeah so you have a planning problem up to the horizon M and that's exponential time in in the horizon M which is I mean it's computable but in fact intractable I mean even for chess it's already intractable to do that exactly and you know it could be also discounted kind of framework or so so having a hard horizon you know at number of years it's just for simplicity


of discussing the model and also sometimes the mass is simple but there are lots of variations actually quite interesting parameter it's it's there's nothing really problematic about it but it's very interesting so for instance you think no let's let's pin let's let the parameter M tend to infinity right you want an agent which lives forever all right if you do it novel you have


two problems first the mathematics breaks down because you have an infinite revert sum which may give infinity and getting river 0.1 in the time step is infinity and giving you got one every time service affinity so equally good not really what we want another problem is that if you have an infinite life you can be lazy for as long as you want for ten years yeah and then catch up with


the same expected reward and you know think about yourself or you know or maybe you know some friends or so if they knew they lived forever you know why work hard now or you know just enjoy your life you know and then catch up later so that's another problem with infinite horizon and you mentioned yes we can go to discounting but then the standard discounting is so called


geometric discounting so $1 today is about worth as much as you know one dollar and five cents tomorrow so if you do this local geometric discounting you have introduced an effective horizon so the agent is now motivated to look ahead a certain amount of time effectively it's likely moving horizon and for any fixed effective horizon there is a problem to solve which requires larger


horizon so if I look ahead you know five time steps I'm a terrible chess player right and I need to look ahead longer if I play go I probably have to look ahead even longer so for every problem no forever horizon there is a problem which this horizon cannot solve yes but I introduced the so-called near harmonic horizon which goes down with one over T rather than


exponential in T which produces an agent which effectively looks into the future proportional to its age so if it's five years old it plans for five years if it's hundred years older than plans for hundred years interesting and a little bit similar to humans - right and my children don't I had very long within we get the doll to a play I had more longer maybe when


we get all very old I mean we know that we don't live forever and maybe then how horizon shrinks again so just adjusting the horizon what is there some mathematical benefit of that of or is it just a nice I mean intuitively empirically we'll probably a good idea to sort of push a horizon back to uh extend the horizon as you experience more of the world but is


there some mathematical conclusions here that are beneficial mr. lamagno who talks to the prediction party of extremely strong finite time but no finite data result so you have sown so much data then you lose sown so much so the dt r is really great with the aixi model with the planning part many results are only asymptotic which well this is what is asymptotic means you can


prove for instance that in the long run if the agent you know x long enough then you know it performs optimal or some nice things happens so but you don't know how fast it converges yeah so it may converge fast but we're just not able to prove it because the typical problem or maybe there's a bug in the in the in the model so that is really dead slow yeah so so that is what asymptotic


means sort of eventually but we don't know how fast and if I give the agent a fixed horizon M yeah then I cannot prove our synthetic results right so I mean sort of pivot dies in hundred years then it number uses over cannot say eventually so this is the advantage of the discounting that I can prove our synthetic results so just to clarify so so I okay I made I've built up a model


well now in a moment I've have this way of looking several steps ahead how do I pick what action I will take it's like with the playing chess right you do this minimax in this case here do expect the max based on the solo mode of distribution you propagate back and then while inaction falls out the action which maximizes the future expected reward on the Solano's distribution and


then you take the section and then repeat that you get a new observation and you feed it in this extant observation then you repeat and the reward so on yeah so you rewrote - yeah and then maybe you can even predict your own action I love the idea but ok this big framework what is it this is I mean it's kind of a beautiful mathematical framework to think about


artificial general intelligence what can you what does it help you into it about how to build such systems or maybe from another perspective what does it help us to in understanding AGI so when I started in the field I was always interested two things one was you know AGI i'm the name didn't exist 10 2014 area wa strong AI and physics c or everything so i switched back and forth


between computer science and physics quite often you said the theory of everything the theory of everything just alike it was a basic that regulators problems before all all of humanity yeah I can explain if you wanted some later time you know why I'm interested in these two questions Nestle and a small tangent if if if one to be it was one to be solved which one would you if one if you were


if an apple fell in your head and there was a brilliant insight and you could arrive at the solution to one would it be a GI or the theory of everything definitely a GI because once the a GI problem solve they can ask the a GI to solve the other problem for me yeah brilliantly okay so so as you were saying about it okay so and the reason why I didn't settle I mean this thought


about you know once you have solved a GI it solves all kinds of other not just as your every problem but all kinds of use more useful problems to humanity it's very appealing to many people and you know and it is thought also but I was quite disappointed with the state of the art of the field of AI there was some theory you know about logical reasoning but I was never convinced that this will


fly and then there was this for more holistic approaches with neural networks and I didn't like these heuristics so and also I didn't have any good idea myself so that's the reason why I toggle back and forth quite some violent even work so four and a half years and a company developing software something completely unrelated but then I had this idea about the ITC model and so what it gives you


it gives you a gold standard so I have proven that this is the most intelligent agents which anybody could build built in quotation mark right because it's just mathematical and you need infinite compute yeah but this is the limit and this is completely specified it's not just a framework and it you know every year tens of frameworks are developed with just have skeletons and then pieces


are missing and usually this missing pieces you know turn out to be really really difficult and so this is completely and uniquely defined and we can analyze that mathematically and we've also developed some approximations I can talk about it a little bit later that would dissolve the top-down approach like say for Normans minimax theory that's the theoretical optimal


play of games and now we need to approximate it put heuristics in prune the tree blah blah blah and so on so we can do that also with a hike symbol but for generally I it can also inspire those and most of most researchers go bottom-up right they have the systems that try to make it more general more intelligent it can inspire in which direction to go what do you mean by that


so if you have some choice to make right so how should they evaluate my system if I can't do cross validation how should I do my learning if my standard regularization doesn't work well yeah so the answers always this we have a system which does everything that's actually it's just you know completing the ivory tower completely useless from a practical point of view but you can look


at it and see oh yeah maybe you know I can take some aspects and you know instead of Kolmogorov complexity there just take some compressors which has been developed so far and for the planning well we have used it here which is also you know being used in go and it I at least it's inspired me a lot to have this formal definition and if you look at other fields you know like I


always come back to physics because I'm a physics background think about the phenomena of energy that was a long time a mysterious concept and at some point it was completely formalized and that really helped a lot and you can point out a lot of these things which were first mysterious and wake and then they have been rigorously formalized speed and acceleration has been confused tried


until it was formally defined here there was a time like this and and people you know often you know I don't have any background you know still confuse it so and this is a model or the the intelligence definitions which is sort of the dual to it we come back to that later formalizes the notion of intelligence uniquely and rigorously so in in a sense it serves as kind of the


light at the end of the tunnel so yeah so I mean there's a million questions I could ask her so maybe the kind of okay let's feel around in the dark a little bit so there's been here a deep mind but in general been a lot of breakthrough ideas just like we've been saying around reinforcement learning so how do you see the progress in reinforcement learning is different like which subset of IHC


does it occupy the current like you said the maybe the Markov assumption is made quite often in reinforce for learning the there's this other assumptions made in order to make the system work what do you see is the difference connection between reinforcement learning and I AXI and so the major difference is that essentially all other approaches they make stronger assumptions so in


reinforcement learning the Markov assumption is that the the next state or next observation only depends on the on the previous observation and not the whole history which makes of course the mathematics much easier rather than dealing with histories of course their profit from it also because then you have algorithms which run on current computers and do something practically


useful for generally re all the assumptions which are made by other approaches we know already now they are limiting so for instance usually you need a go digital assumption in the MTP framework in order to learn it goes this T essentially means that you can recover from your mistakes and that they are not traps in the environment and if you make this assumption then essentially you can


you know go back to a previous state go there a couple of times and then learn what what statistics and what the state is like and then in the long run perform well in this state yeah but there are no fundamental problems but in real life we know you know there can be one single action you know one second of being inattentive while driving a car fast you know you can ruin the rest of my life I


can become quadriplegic or whatever so and there's no recovery anymore so the real world is not err gorica i always say you know there are traps and there are situations we are not recover from and very little theory has been developed for this case what about what do you see in there in the context of I see as the role of exploration sort of you mentioned you know in the in the


real world and get into trouble and we make the wrong decisions and really pay for it but exploration is seems to be fundamentally important for learning about this world for gaining new knowledge so is it his exploration baked in another way to ask it what are the parameters of this of IHC it can be controlled yeah I say the good thing is that there are no parameters to control


and some other people drag knobs to control and you can do that I mean you can modify axes so that you have some knobs to play with if you want to but the exploration is directly baked in and that comes from the Bayesian learning and the long-term planning so these together already imply exploration you can nicely and explicitly prove that for a simple problem


like so-called bandit problems where you say to give a real good example say you have two medical treatments a and B you don't know the effectiveness you try a a little bit be a little bit but you don't want to harm too many patients so you have to sort of trade-off exploring yeah and at some point you want to explore and you can do the mathematics and figure out the optimal strategy it took


a Bayesian agents they're also non-bayesian agents but it shows that this patient framework by taking a prior over possible worlds doing the Bayesian mixture than the Bayes optimal decision with long term planning that is important automatically implies exploration also to the proper extent not too much exploration and not too little in this very simple settings in


the aixi model and was also able to prove that it is a self optimizing theorem or asymptotic optimality theorems or later only asymptotic not finite time bounds it seems like the long term planning is a really important but the long term part of the planet is really important yes and also I mean maybe a quick tangent how important do you think is removing the Markov


assumption and looking at the full history sort of intuitively of course it's important but is it like fundamentally transformative to the entirety of the problem what's your sense of it like because we all we make that assumption quite often it's just throwing away the past now I think it's absolutely crucial the question is whether there's a way to deal with it in


a more holistic and still sufficiently well way so I have to come everything up and fly but you know you have say some you know key event in your life you know a long time ago you know in some city or something you realized you know that's a really dangerous street or whatever right here and you want to remember that forever right in case you come back they're kind of a selective kind of


memory so you remember that all the important events in the past but somehow selecting the importance is see that's very hard yeah and I'm not concerned about you know just storing the whole history just you can calculate you know human life says 30 or 100 years doesn't matter right how much data comes in through the vision system and the auditory system you compress it a little


bit in this case La Salette and store it we are soon in the means of just storing it yeah but you still need to the selection for the planning part and the compression for the understanding part the raw storage I'm really not concerned about and I think we should just store if you develop an agent preferably just restore all the interaction history and then you build


[Music] so Wolfgang helpfully laid out the dichotomy between industry people and academics and experimentalists and computational people and if you're wondering which one I am the answer is yes so this is I'm going to mostly describe work that happened in my lab at Harvard and work by Bill Lauder who is actually in industry at us doing a start-up because that's what everyone

Predictive Coding Models of Perception


does these days but I recently also took a gig as the director of a new MIT IBM collaboration a quarter-million-dollar AI Institute so if you if you're interested in that come and find me and then I'm going to tell you about some deep learning work we've done but then I'm also going to tell you at the end if I have time about some experimental work that it's inspired in

Predictive Coding Models of Perception


my lab just to kind of reinforce this idea that there's sort of a loop that we can be driving between models and experiments so we all know that deep learning has kind of a maybe kind of connection to biology so we have units and they have synapses and connections between them okay that's that's part of it that's for their artificial neural network part comes from and then they're

Predictive Coding Models of Perception


deep and you know we know that perceptual hierarchies for instance in the and the primate are are sort of deep hierarchical systems and deep CN n sort of capture that but really when you get right down to it you know that's been most of the interplay between these two and I think a lot of us here are sort of trying to think well how can we go back to the brain and get more inspiration

Predictive Coding Models of Perception


but then also how can we use the in you know the deep learning to actually help us understand the brain a little bit better so again this virtuous loop you're gonna hear this I think again and again and again and that's that's wherever we're coming from so it turns out the deep learning so I'm interested in perception so a different part of the stack then the the previous two talks

Predictive Coding Models of Perception


imaged in how we perceive objects and make sense of this sort of terrible complicated flow of information coming in through our senses and it turns out that by accident deep learning systems convolution neural networks turned out to be the best model for doing this so people have built all kinds of computational models of how visual processing worked in the ventral visual

Predictive Coding Models of Perception


pathway and primates but it turns out that when you just started training deep Nets you just took the internal representations compare them to actual neuronal population responses and that's the best model so so here this is a paper from my my former PhD advisor Jim - Carlos lab and basically you see those are sort of model fit quality for different kinds of models that came

Predictive Coding Models of Perception


before including some of my own and then basically once we have deep Nets those explained the data fit the data way better than anything else and you know this has been going on and basically the bottom line is the better the models get on image net the better the fit seems to be between what's going what the representational space looks like in the in the deep net and what the

Predictive Coding Models of Perception


representational space seems to look like in the population so this led to this idea that well maybe maybe visions just tapped out and this is a solved problem and then maybe we don't need to worry about this anymore the answer is deep Nets there's a problem because all the work we've done so far is all looking at static representations and really our visual systems are built for

Predictive Coding Models of Perception


dynamic situations and it's even worse than that because even if you show a static stimulus neuronal populations will not produce static outputs and this is one of the first things you notice when you're a graduate students sticking electrodes into the brain of the monkey which I did for five years if you show a static picture of a monkey face you get these sort of transient

Predictive Coding Models of Perception


responses and in fact the only way to drive a sustained response in an IT neuron at the end of the ventral visual pathway is to show a dynamic stimulus so this is very much at odds with how cnn's work because they have no intrinsic notion of time you put in an input you get an output it's a static thing so something is clearly going on here and there's all kinds of interesting rich

Predictive Coding Models of Perception


dynamics here where sometimes it's sustained sometimes it's not sometimes it looks like it might be oscillating so we don't really have a good picture yet of what that's all about and to that and you know that part which seems really salient isn't actually captured by simple cnn's and it gets even weirder than that too because there's this great experiment by Karl Olsen's lab where he

Predictive Coding Models of Perception


showed successive presentations of image image image image and in some cases the the second image the B image was predicted by the first image so b4 always happened after a4 and you saw this a couple hundred times and b5 was always seen after a5 showed a couple hundred times and then basically you see is you don't get a response to beef-up before if it's preceded by a for does

Predictive Coding Models of Perception


that mean that the cell just doesn't like b4 well no actually it will respond to b4 but only when it's preceded by a different image so there's some kind of higher-order temporal track that's going on but again CNN's just don't capture because they don't have any intrinsic notion of time the other thing too which i think is actually a salient problem for deep learning is

Predictive Coding Models of Perception


that most of the datasets we have to train these things you need tons and tons and tons of data you know if you want to train you know a dog detector you need you know thousands and thousands and thousands of dogs and thousands and thousands of thousands of things that aren't dogs and that's just not the way we learned I don't sit down for my daughter and shower dog dog dog

Predictive Coding Models of Perception


dog dog Cat Cat Cat Cat that's just not the way it works in fact yeah we can just do that experiment right now does anyone know what this is raise your hands if you do okay we have a few electrical engineers in the audience okay even though you've even though you've only seen this for the first time is it present in this image yes how many in this image - how about that one yeah

Predictive Coding Models of Perception


but it's a little weird right so even though you only saw one example you were immediately able to determine what was there you immediately an expert on this kind of object that's called one-shot learning and you know just if you need more evidence that deep nets don't quite work yet is anyone can anyone say what this is it's actually Meret Oppenheim it's called luncheon in fur but everyone

Predictive Coding Models of Perception


agrees it's a cup a saucer in the spoon state of the art deep net says it's a teddy bear and then it gets worse than that even even when things are very clear to the right object so this is a state of the art our CNN detection correctly detects this as a bird but you just put a few objects in the don't belong and it starts you know calls this a cat says that's it you know it's this

Predictive Coding Models of Perception


is a television but so it's calling that a cat so there's something about there's something brittle and adversarial examples we could go on and on and on about ways in which deep nets really aren't quite solving the problem yet and you know the I I and other people think that unsupervised learning is an important piece of this so how can we without labels gather up a lot of

Predictive Coding Models of Perception


structure about the world and build representations that are really good and then that can feed into things like reinforcement learning and all that kind of stuff so this is just a different part of the stack I think we all agree on there's multiple kinds of learning happening here but the kind that I'm really interested in and have been for a long time is this idea of temporal

Predictive Coding Models of Perception


learning if you just look at the environment and let it play out the environment is almost always showing you its structure so you look at a person doing a tennis serve you can see how they're articulated you can see they're put together you can see how the shadows move around on an object that's just all played out in time you don't need supervision you can just

Predictive Coding Models of Perception


observe and learn a lot about the world so that's what we're interested doing it and you know brains from the from the neuroscience literature seemed to be exquisitely tuned for these kinds of temporal statistics so this is one of my favorite studies it's like a physics where basically they took faces and then they rotated them and then had subjects to passively watch these but as a little

Predictive Coding Models of Perception


trick sometimes they would morph the face as it moved and people generally didn't notice this but if you ask the same different later ones that had morphed and they had seen them morph a few times they would incorrectly sort of associate them as being the same same object and then you can even go further this is let me I didn't over my PhD you can even have presented a peripheral

Predictive Coding Models of Perception


object and then have while the subject is a Codding you flip it out for a different object before their eyes land their eyes land on a different object and then you ask them to do same different on peripheral versus fovea and they'll make incorrect associations so it seems like the brain is constantly collecting up these sort of temporal associations to make sense of the world

Predictive Coding Models of Perception


so we have we have experimental of us this might be happening but we don't have really models for it now there have been prior attempts so slow Fourier analysis is a is an attempt to sort of extract those signals that are moving slowly and this has been influential but you know it hasn't sort of unleashed you know a revolution in how we do things but it's an interesting and important

Predictive Coding Models of Perception


set of ideas but what I want to do is look specifically at prediction as an unsupervised learning rule so basically the idea that can we just use the idea of predicting what a future frame is going to look like to help us build better representations and I'm gonna make the argument that brains are particularly adept at prediction I'm gonna use this person who's this person

Predictive Coding Models of Perception


Sarina yes not Venus Sarina you guys are our experts so anyway so she can she can do tennis serve at 207 km/h that's 57 meters per second that means the ball traverses the court in 400 milliseconds about now the latency from the retina to the primary visual cortex is about 60 milliseconds so that means that v1 is operating 3 meters in the past and if we go all the way through the ventral

Predictive Coding Models of Perception


visual hierarchy then we're talking about about a hundred and seventy millisecond latency in a human it's about nine meters so there's a very deep sense in which if you think you saw Serena Williams's tennis serve you couldn't have because your brain you know this year line latency of your brain was way in the past so one suggestion that that's been made is if you're returning that

Predictive Coding Models of Perception


serve a big part of what you're doing is you're looking at the lined up and you're predicting where the ball is going to be and that's where you're putting your racket so you know and this this idea has been sort of you know it can be shown there's a little bit of nuance here but if this is called the flash lag you lose inch so if you look at this dot right here and then watch

Predictive Coding Models of Perception


this sort of clock hand going around you see how there's like a little flashing straight line now is that line behind in front of or lined up with the clock hand behind of course and the rule with all optical illusions is if it looks like it's not lined up it is lined up and if it looks like it is lined up it's not lined up so so this is perfectly collinear and the sort of classical interpretation

Predictive Coding Models of Perception


and there's some nuance here is that basically this line has to go through the full pipeline latency of the visual system but this tracking line is predictable so your visual your perception just sort of puts it where it really is in real time so so this is and then there's another weird things everyone's kind of see that it looks like it's tilted as well I can just hold

Predictive Coding Models of Perception


that thought all right so so so so we wanted to get into this so we were we were you know working in sort of machine learning computer vision and right around the time we were working on this we got interesting this idea of future framed video prediction so basically the idea is that if we see a sequence of images transforming the goal is to basically predict what the next image

Predictive Coding Models of Perception


looks like and this is now that there were there was a little bit of work around the time we were doing this and that's it's a much bigger field now just to acknowledge that there are other people working in this area and we did some very simple things that you would do in in deep learning and again this is sort of fits in the same spirit of what Matt was talking about where so you just

Predictive Coding Models of Perception


want the simplest possible models to start with and just see how much your problem statement sort of you know sort of it makes the problem happen so basically we did is we took an autoencoder type architecture and we just wedged a recurrent neural network in this case an LS TM in the middle and then asked and then we also had a fancier version with gans back then

Predictive Coding Models of Perception


there were only four Gann papers so it seemed exotic and how there's like 4,000 Gann papers but anyway so basically what we discovered is we can build these sort of generative networks that could actually basically sort of renderer faces and this doesn't seem so surprising anymore but at the time I was shocked how well this worked and there's some details about whether you

Predictive Coding Models of Perception


use the Gann or not and whether the ear appears or not but the basic bottom line is this is there a future frame video prediction problem where we see a sequence of frames and then we predict what the next frame might look like it is actually a tractable thing we can do with neural networks now and then the interesting and of course it works on all kinds of different faces including

Predictive Coding Models of Perception


faces you haven't seen before but the interesting thing here was we were really trying to look at what kind of representations do we implicitly induce when we train a network to do this kind of future frame prediction so all we're training we're training the network with back prop but the loss is coming entirely from how well does it reconstruct a future frame that it

Predictive Coding Models of Perception


hasn't seen before and the interesting interesting thing that we find is if we kind of just take that internal representation and just sort of pipe it off and just ask how well can we decode things about other things like the identity of the face what we find is if we do a simple you know 50 Way face recognition task where we need to tell which of 50 people it is and we only get

Predictive Coding Models of Perception


to see 1 2 3 4 so on views of the person what we see is if we start using these predictive networks they're able to do a little bit better with a little bit less data and this is what we're trying to get towards can we get representations where we can do more with less training data and because we're sort of extracting some sort of sort of deep part of you know some sort of deep

Predictive Coding Models of Perception


structure of the image now this was back in in 2015 and then you know we also discovered that you know for instance you could internal within to these the representations that we learned you could also find directions that you can move around in that would do things like make the face more male or more female these are less surprising now that ganzar around and we can do all kinds of

Predictive Coding Models of Perception


fancy things with them but what we really we're driving towards just to get more to the punchline was not this sort of very simple auto encoder with an RNN in the middle but really getting towards this idea of predictive coding which came from neuroscience so the basic idea here this is a paper from 1999 that popularized predictive coding but the idea actually goes back about a decade

Predictive Coding Models of Perception


earlier and the basic idea is this we start with an input we have a feed-forward signal we try and predict away the input and we subtract that off and we only send forward the differences now the original idea here was an efficient coding idea let's try and reduce the number of spikes we have to send because if we subtract away the things we already know then all we need to send forward is the

Predictive Coding Models of Perception


is the the difference from that so what we did was basically to take yeah on what we think the signal is if I'm looking at the outside and I'm trying to submit position of a ball and predicting where the ball is going to be is a different stories and if I'm trying to estimate suppose it's octave all right and then do in how many examples do we actually know what it is that we're

Predictive Coding Models of Perception


trying to predict I mean really for sure right this is only doing that thing right yeah that's that's a great matter question and and I think we can discuss that like what are we estimating in this case for the purposes of this we just took the simplest path I mean again this is sort of the modus operandi is take the simplest possible thing and then see where you get with it so here we're

Predictive Coding Models of Perception


actually gonna generate whole frames so we actually want to confabulate and sort of imagine what the future frames gonna look like so you could imagine in that flash lag illusion example you're percept in that case is actually an imagined thing it's like it you're actually what you're perceiving at a given moment is is a fabricated future State so so we're gonna take it at the

Predictive Coding Models of Perception


pixel level but I agree because if you were interests in different things you might you might use different sort of targets for your prediction but what we did basically was to take the sort of classic idea of predictive coding and a state instantiated in the simplest possible way we could with with deep networks so basically what we do whose inputs come in here we have a running

Predictive Coding Models of Perception


prediction of what the input should look like and then we subtract them off to get an error map and then we send that forward to the next layer and then we have a recurrent layer at each at each stage of the hierarchy that's trying to build up this prediction and it can get feedback just the same as in the brain and it can also get local recurrence and then you basically just subtract off and

Predictive Coding Models of Perception


there's different ways you can do the subtraction doesn't much matter and then you get these errors you only send the errors forward through the network so one nice thing about this is at time one this is a CNN as nothing's come through so the extent that CNN's are a good model for the ventral visual pathway this is the CNN on the first time step and then the other thing that's nice

Predictive Coding Models of Perception


about it is it's also a classic generative model so if we put something in on the top it'll render down into an actual predicted image here so we can see what the network is is seeing or perceiving and I'm given moment and we called we started off by calling these particular works and then we don't maybe deep prediction that's not good name pretty deep no bill net no

Predictive Coding Models of Perception


can't believe it's not Alex net we ended up with pregnant because their rules their rules to how you do this and you have to you know you have to get them right so we're calling these pred nets for better or for worse and what we found is that these networks that had the recurrence at every layer we're able to you know our previous ones were only able to get sort of one degree

Predictive Coding Models of Perception


of freedom and a rotating object now we could actually get as many degrees of freedom basically as we wanted in these synthetic objects so here are examples and basically that just for all these I'm gonna show is this is the actual sequence of incoming images and then what's next but below it is the time-shifted prediction so you can compare the prediction to what actually

Predictive Coding Models of Perception


came and you can see on the first frame you get these weird potato thinks as it doesn't know which way it's going to go but then once it knows which way it's gonna go it locks on and then you start getting pretty good predictions and it can do this with faces it hasn't seen before and all that kind of stuff but again the the reason we're doing this is not for future frame prediction per se

Predictive Coding Models of Perception


but to see if we can actually learn good representations because the idea is in order to predict what's coming next you have to implicitly know lots of things about the structure of the object how light works how shadows work all that kind of stuff and what we find is basically if we try and build decoders again in the same sort of spirit as that face recognition

Predictive Coding Models of Perception


task where we just sort of peel off the representation and just do linear decoding we discover that we can decode lots of different parameters of the image like how fast it's moving what the starting angle is we can also look at things about principal components of the identity so you it sort of fits with this idea again that by virtue of learning good predictions you learn good

Predictive Coding Models of Perception


representations that are useful for lots of things which is kind of what you need to have in an unsupervised or a semi-supervised learning sort of setting we also did a face-recognition version of this again and what we found is that these presents at least at the time were performing on par better than the best semi-supervised learning algorithms that were available called latter networks so

Predictive Coding Models of Perception


you might wonder what happens if we put in things that aren't faces what does it do it actually does something pretty reasonable so this is a network that was trained on faces and we put in this image of a top and you know something came out pretty okay it doesn't always turn out okay here's a here's like a little toy car and you can see it's desperately trying to turn it

Predictive Coding Models of Perception


into some kind of face it's like this like when scream kind trying to happen there but you know and but the bottom line is if you train it on complicated things and enough variety you can get networks that can do pretty good predictions on just about anything you want to do and of course just about anything you want to do is cars right autonomous cars like at least in this

Predictive Coding Models of Perception


year we did this work that's the thing you want to be doing prediction on because that's where all the money is so we took my student bill without any prompting took some some car mounted camera datasets so this we train it on the kiddy data set which is in Germany and then what I'm showing you here is the test set which is the Caltech pedestrian data set and then this is

Predictive Coding Models of Perception


what the predictions look like so they're not perfect they're a little bit blurry if we put a Gann on it we can make it less blurry sure but we're really just focusing on sort of learning what kind of representations we can pull out of this and a couple interesting things here one is it seems to implicitly know a lot about perspective and flow it knows that things need to expand

Predictive Coding Models of Perception


outwards and needs you knows the things that are far away need to you know change less it also knows to infill road here if we look at other examples as well you can see things like you know it knows a little bit about occlusion so this car is going to include this golf cart it knows that that one should go in front it knows you know it knows all kinds of things about you know

Predictive Coding Models of Perception


implicitly about the flow of content in the image and again what we're trying to do here it's interesting to do it to look at feature frame prediction and we can also do farther out future frame predictions as well if we can go out as much as five frames it gets blurrier but it's it's still okay but again the goal here is to say well by virtue of learning how to predict as sort of a

Predictive Coding Models of Perception


surrogate loss can we learn how to decode other things that might be useful and in a car one of the things you might want to learn about is what's the steering angle of the car and what we find again in the same way we can take the representation peel that off put it into a linear decoder and we can discover that without any prior training on these steering angles this can

Predictive Coding Models of Perception


actually outperform a system that was purpose-built to decode steering angles so this is from comma this is a startup that's doing autonomous cars they had a reference CNN on their data set and then basically the president just by learning how to predict the future also implicitly learned what the steering angle of the car was that was kind of what you need to do to

Predictive Coding Models of Perception


do the task yeah not one not just the recurrency but also the fact that you're training it on continuously sample images representation is due to the prediction as opposed to merely the sequence of training images all right so so we have so we've done predictive versions and we've also done sort of a just a autoencoder style and the prediction outperforms the details

Predictive Coding Models of Perception


about are all in the paper how much additional how much of a better representation of you gain by adding the predictive component as opposed to just having from whatever image your sequence of images you have you mean specifically for this the steering angle version so the way the the comma one works is it it takes in frames and it puts them to a CNN and it so it has the temporal

Predictive Coding Models of Perception


component but it doesn't have any notion of prediction it wasn't it wasn't sort of trained to do prediction and what we see here is these were the comma reference CNN's so how well they performed and then this is with different numbers of input frames to the decoder and you can see it can can do quite well with with relatively little data so it seems to be a useful thing

Predictive Coding Models of Perception


but again what we're trying to do is trying to drive towards how might this be a principle that the brain could use to organize itself yeah when we're doing rollouts what do you mean oh into the future so their unique way school what we do is we re inject recurrent recursively the the predictions and then we fine-tune that so so basically if you take the prediction put it in again get the next

Predictive Coding Models of Perception


prediction put it in again put the next prediction and you can get its it starts to get blurrier and blurrier over time but that's basically what we're doing very briefly showed we can do five frames ahead the predictions just get murkier it makes a little bit of a difference it gets murkier because you don't know which way the car is going to go I mean realistic what we should be doing

Predictive Coding Models of Perception


what we are doing now is probabilistically we should actually be putting out a probability distribution of all the possible outcomes and that's that's hard but that's what we're doing and you know there's a lot of interesting sort of wrinkles but even just with this but the straight up let's just does this come up with a mean outcome because there's a problem that

Predictive Coding Models of Perception


if you don't know it's gonna go left or right then you you don't want to split the difference and get blurry what you'd really like to do is but have a system that produces samples that are sort of the correct distribution of actual outputs you could have and again that's something something we're working on something brain publi does - of course then we want to actually since this is a

Predictive Coding Models of Perception


neuroscience talk we want to go back and look at like well does this do something that explains something in neuroscience that we didn't understand before well here's something that everything does so it turns out you get good bores everything you know every neuron that work you train on anything no matter what you do it's like a rule that you'll get Gabor is out and and orientation

Predictive Coding Models of Perception


tuning but more interesting than that is this notion that I sort of flagged earlier which is if you put in a static input the brain doesn't give you a static output it gives you a dynamic output so there's usually a delay there's a burst of activity and then you have this sort of activity that falls off and sometimes you also get off responses so when the stimulus goes away

Predictive Coding Models of Perception


you get a fresh response and it's probably not a great surprise but pride nets do this as well and it's not hard to see why so this is the average of units in the error representation of the pregnant hard to figure out why it's doing this basically what happens is on that first hit it can't predict anything when the first image first flash is on so you get a big bunch of activity but then as it

Predictive Coding Models of Perception


locks on and learns to sort of explain away the data then it goes down and then when you get the image off that's another sort of surprise and then you get another burst of activity yeah if you sprinkled the pixels randomly like you know Toby Delbert's event cameras or something true continue so so it's true we're doing this in discrete time because we're talking about worker L

Predictive Coding Models of Perception


STM's and things like that convalesced hams so we can't accommodate continuous time but I don't think that's I don't think that's a huge huge difference well I mean these aren't like different frames it's like continuous frames so it's a relatively smooth progression through the space of images and when you have nothing and then you put on something that is the kind of

Predictive Coding Models of Perception


discontinuity we're talking about it is in discrete time yeah so what we did is we literally put the image up you know blank screen put the image up and then take the image off and so that's like the standard sort of primate experiment version of this and basically the present that does exactly the same thing has exactly the same dynamics that you would find in in primary visual cortex

Predictive Coding Models of Perception


so this is this is a bit of circumstantial evidence at least that part sort of matches up with expectations originally actually particular coding was it was designed to explain a phenomenon called N stopping which is if you have a bar that's oriented the way a v1 cell likes and then you make the bar longer and longer and longer and longer the response will go up to some point and then once you

Predictive Coding Models of Perception


make it longer actually the response gets suppressed so it's like too much of a good thing you know the longer the bar is that that's in the orientation that likes it actually reduces the response and you know true to form the the present version of predictive coding also has this sort of n stopping response which which is a nice nice thing to have but it goes further than

Predictive Coding Models of Perception


this there's also this notion of surround depression so if you have a stimulus which is a dot and you larger and larger and larger and larger the e1 cells will like it better and better and better and respond more and more and more up to a point but then beyond that they'll start being suppressed as if there's a suppressive surround around the response and what we

Predictive Coding Models of Perception


find is that the the pet net also has this quality so if you look in again the bottom layer the response goes up to a certain size of a dot stimulus and then it's suppressed but interestingly Rick borns lab at Harvard back in 2013 found that if you cool downstream visual areas so if you cool v2 and then record in v1 so in activate feedback connections you actually would find that these surround

Predictive Coding Models of Perception


suppression effects were themselves suppressed they go away if you take away the top-down feedback and this is a way easier experiment to do in in a network because you can just turn off the feedback connections and see what happens and lo and behold you get almost exactly the same pattern of feedback so in the pred net sit extent there surround suppression it's happening as a

Predictive Coding Models of Perception


top-down feedback phenomenon just the same way it seems to be happening in the primate and then you know as we sort of march through these things we also have this interesting sort of sequence learning effects that you see in the end of the ventral visual pathway and lo and behold if you show it pregnant these things so the pred Nets just trained on natural images like our videos and then

Predictive Coding Models of Perception


we're gonna show it these stimuli that are basically the same as what you would find in these experiments in an untrained pred net you get these sort of funny effects where it's you know you flash up an image and it takes a little while to sort of predict it away but then as you train these these sequences so that are expected you get sharper and sharper transitions from between these

Predictive Coding Models of Perception


images and then what happens is basically the same exact result so if B is predictable from a in serial presentations again this is trained on cars and then we just subsequently show it a few of the you know a comparable number hundreds of examples then B will be suppressed if a explains away B but if we show that very same B and you know with something else that's not a that

Predictive Coding Models of Perception


doesn't predict it in front of it we get the exact same result that the Carlos and company got so it's able to track with almost exactly the same number of trials some of these interesting sequence learning facts with no no extra machinery needed to be added and then you might also ask okay that that flash lag illusion so you might remember you know this this clocks

Predictive Coding Models of Perception


going around this bar is flashing it's actually lined up but it looks like it's lagging behind so you might ask oh is that this is the same thing happen with with the pred net and the cool thing about generative neural networks is you can see what they're experiencing because we can just visualize the error layer and so this is what it's seeing now there's a caveat here does anyone

Predictive Coding Models of Perception


know the caveat is yeah yeah so you're getting a double dose so so the network is seeing it and then you're seeing it on top of what the network is seen so if we really want to do this we need to we need to look at freeze frames and we can see exactly what what the network's seeing and there it is there's that weird lagging and there's the tilt too right that was weird

Predictive Coding Models of Perception


there wasn't a good there wasn't a satisfy satisfy explanation for that tilt before and it just sort of falls out of the pregnant yeah just predictable on a longer time scale yeah so it's the same thing with with with the human version of it right like it's predictable but there seems to be some window beyond which it's not it's effectively not predictable that's right

Predictive Coding Models of Perception


there's also weird things like you see these like weird ripoli things happening and I don't know what those are but they're really cool and I like looking at them and we'll figure out what they're for eventually and then I love the scientific community and I love archive so we posted this and a group in Japan picked it up and started using it so they basically wanted to see these

Predictive Coding Models of Perception


illusory motion stimuli they wanted to see what would happen if they showed them to pred nuts so so the way these work hopefully you're experiencing the illusion right now this is a static image but it looks like these are are sort of these ones are sort of are sort of rotating and these ones are not so this is a classic you know sort of illusory motion sort of stimulus and it

Predictive Coding Models of Perception


turns out if you if you train a pret net on these sort of rotating so it's seen real objects rotating in the world or synthetic images of like propellers and things moving in the world and then you compute the flow vectors on the actual predictions if you just let a pregnant look at these static images the pregnant actually produces flow vectors that are consistent with the illusion so when

Predictive Coding Models of Perception


humans see the illusion the pregnant has optic flow that looks like what we see and when humans don't see the illusion there's no optic flow so so this is this is interesting especially because the usual explanation for these things has to do with sort of epiphenomena about the relative Layton sees visual responses and now that might still be true but this at least gives you another

Predictive Coding Models of Perception


possible explanation which is that you have a system that's trained with the predictive loss it's gonna naturally have some of these biases that come from the sort of statistics of the world so with all things neural network you know like we just hide all the like the parameter search that went into it there is indeed so the failure mode in general when it can't when it doesn't learn so

Predictive Coding Models of Perception


if we have like the hyper parameters wrong or whatever is that it predicts the last frame that's usually our benchmark against which to see if it's working so we take the reconstruction lost if you just said it's the same as the last frame and then compare how much we improve the error and then you're right it if it's too herky-jerky then it has trouble learning so so I don't have

Predictive Coding Models of Perception


a good quantitative answer for you but there there is indeed a sweet spot and yeah I mean it depends on how fast you run the framerate we were running these at 10 frame frame rates so that's kind of yeah about about 100 milliseconds it depends on how fast things are moving as well so anyway so I promised that we would get back to a neuroscience experiment so I'm

Predictive Coding Models of Perception


going to tell you about a story about an actual neuroscience experiment that the neuroscience said the wet lab part of my lab did in response to the computational work that was going on in my lab so you might remember that we could read out the steering angle from a self-driving car using the internal representations in the pregnant we also had the idea well you know wouldn't it make sense to

Predictive Coding Models of Perception


actually take the efference copy of the steering wheel all the odometry of the car we could just feed that into the representation surely the network would do a better job of predicting so if I'm trying to predict how the world is going to change if I know the wheels turn this way I can make a better prediction about how the world's going to change I might also be able to do cool things like if I

Predictive Coding Models of Perception


could sort of fictive Li imagine like conditionally generate if I turn the wheel like this what would the world look like so this is something we started doing just because it made sense from a machine learning standpoint and no big surprise when you don't have a friend's copy which is basically the sort of vanilla pret net you get certain convergence over time you know so you

Predictive Coding Models of Perception


get get down to some mean error and with with training epoch but if you include this extra information unsurprisingly the network converges faster and comes up with a better result so that's that's all fine and good but if this were true and this is what was happening in the brain that would imply that in visual cortex we should have signals from motor cortex right they should be there and if

Predictive Coding Models of Perception


they're there we should be able to decode them even in the dark potentially so so my student Greg thought this was a great idea he was doing 24/7 recordings in visual cortex in a rat at the time so he's had boatloads of boatloads of data so we just had that lab meeting well hey well I just put them in a dark box like so there's no light completely light-tight and see if you can actually

Predictive Coding Models of Perception


and you have an accelerometer on the animal's head anyway because you know the minute you're putting electrodes and you might as well put an accelerometer in there too and then so we can record from from many tête roads so we have you know 60 16 tetrode 64 electrodes and we can record local field potentials and then we also have these accelerometer signals so there's a

Predictive Coding Models of Perception


there's a hypothesis that comes from the computational work that says well we should be able to decode you see friends copy signals in you know potentially even in v1 and it turns out that that is exactly what we can do so so these blue bars are in complete darkness and visual cortex so there's no visual stimulus they're not getting any visual optic flow but we can decode a little bit of

Predictive Coding Models of Perception


you know sort of six degree of freedom information about which way the animals nose is pointing and then interestingly if you inject mute samal into area M two which is the putative place where these motor signals you friends copies would be coming from you know this is all still new so you know don't get futzed if this ends up not holding up because we still only get more animals at least

Predictive Coding Models of Perception


in the first animal we did we can actually abolish that signal so if we take away motor cortex signals we actually can't any longer decode the position of the animal's head in 3-space so this is a case where we actually built them up we built a model that was useful in its own right it helped explain some things about neuroscience more or less along the way without

Predictive Coding Models of Perception


having to fit anything and then by looking at it we can make predictions which then led to experiments that we could do to maybe learn something new about how the brain is organized corollary discharged we can't tell and in an electrophysiology we just have some Tet Rhodes recording from some places and we're not exactly sure where we can't reconstruct the image to fully

Predictive Coding Models of Perception


disambiguate that we also have two photon imaging going on in my lab so there is an idea that we could go and be getting you know hundreds of cells that are thousands of cells at a time you know and have a prayer of actually doing those those decoding experiments and that's work that we have that's the song going but but you know right now we're just in the stage where we said can we

Predictive Coding Models of Perception


decode any information and the answer seems to be yes I agree there's a lot of questions that come downstream of that like well you know predictive coding implies certain kinds of correlational structures that we can start to look at we also have the ability to image synapses in the two-photon image of the synapses differently separately from the cell body so we can actually look at

Predictive Coding Models of Perception


feedback synapses specifically and ask how the information that they contained is different than the the cell bodies in layer 2 3 so these are all experiments that are ongoing but I don't have a good answer for you but that's the right kind of question and that's the right kind of question we can ask you know when we have these kinds of models they can guide our experiments ok so this is my

Predictive Coding Models of Perception


pitch so apparently a deep mind it's a it's a circle for me it's just a it's just a ping-pong I don't know I like the circle better maybe I'll change it top away oh yeah that's right so people are picking this up Gooding neuroscientists and we love that if you want to pick it up there it is the codes all free I'm told it's not hard for other people to get working so that's

Predictive Coding Models of Perception


great that's a testament to Bill's hard work to share things so I just want to try and get us back on schedule so this is my lab just went to acknowledge everyone particularly Bill water and then all the people who funded this stuff and then just one shameless plug you know so I've just started as the director of this institute if you're interested in sort of AI and

Predictive Coding Models of Perception


neuroscience Nexus the kinds of stuff that I'm showing you here and you reached in doing that in industry you know find me during one of the breaks I'd be happy to talk right thanks [Applause] yes oh the most of the mud so the motor modulation it's there's a whole cottage industry of like my son trackballs when they run versus when they don't run and and basically there's more information

Predictive Coding Models of Perception


about whether the running or not then there seems to be about the visual system that's sort of like a first approximation but but a lot of the there's been a sort of a dichotomy of people who are either either think you know maybe this is a predictive process and kind of thing versus people who think it's more of like an arousal sort of like when the animals running they're

Predictive Coding Models of Perception


on and when they're not running they're off or something and you can probably imagine which one I think is the more likely scenario and so so so the idea that we can actually get information about the actual direction the nose is pointing I think is interesting and a little bit surprising and and feels like at odds with the idea that it's just sort of an on versus an off kind of

Predictive Coding Models of Perception


transformation now in terms of vestibular signals yeah I mean we're we've talked about you know the you know the experiments aren't pretty but you can imagine what they're like you know it's like I'm just moving I'm moving the rat in the dark Here I am with my rat moving in the dark so you know like sometimes this thought you have to do the science and and it's not it's not

Predictive Coding Models of Perception


glamorous so we say so so we aren't in those experiments I mean it doesn't sort of go against the general theme here like wherever you get the signals from you'd be crazy not to use them I think the fact that we see some reduction in them when we knock out em to suggest a little bit that maybe it's more of a motor you friends copy but it's entirely possible that this tabular system the

Predictive Coding Models of Perception


vestibular inputs are getting through to v1 through so m2 is I mean it's like it's sort of like frontal orienting fields in a monkey it's like an orienting area yeah so as near as we can tell it doesn't seem to so we've done let's go to all the marginal comparisons so you'd expect it to have some effect it turns out you can actually just scoop out all of them fun

Predictive Coding Models of Perception


all the motor cortex by my neighbor at Harvard Ben selves Eskie did this experiment are you basically just carpet bomb all of motor cortex and the animals can still do all kinds of complicated motor tasks so that that's alarming if you study motor cortex but but from our perspective it's it's actually a good things I mean so it's near as we can tell we're not changing with the

Predictive Coding Models of Perception


statistics of the movement one of things were moving towards mister is trying to get more quantitative predictions about what would the actual predictive coatings that are subtractive having a model at least gives you a prayer of being able to sort of ask those kinds of questions I agree it's complicated it's not at the end not the end-all be-all and and also in monkey and primate IT

Predictive Coding Models of Perception


there was there some beautiful studies that were they were kind of lesser-known where they showed that in complete darkness they were psychotic eye movement signals presence he could decode when cicadas happened so so these ideas aren't aren't new it's just for their sort of driving us in new directions [Music] but when you're comparing that to the neural state law of attraction the first

Predictive Coding Models of Perception


example if you're showing if you're comparing that to the N stopping signals or the transient dynamics yeah it turns out if you look at them they almost all look the same which which is disheartening for people who want to disambiguate different populations I mean at least when you marginalize them in this way the the e neurons look like pop and then off response but so did the

Predictive Coding Models of Perception


activation neurons and so there are recurrent neurons that are sort of sitting there and like like in Matt's talk like they're ones that are doing the things you know they have to be doing but there are ones in the in the recurrent layer that also have the sort of transient dynamics as well so so the long story is like if you were sticking electrodes into a monkey brain like I

Predictive Coding Models of Perception


did for five years you know you wouldn't know which ones of these you were getting and they all look surprisingly similar I mean so in the in the cases where we're doing the cutting we're actually decoding from the ours so but I mean you you can decode lots of things from lots of places the arse we thought the arse made a lot of sense because they're the ones who should be

Predictive Coding Models of Perception


holding information about context and representation but I mean there's obviously a lot going on yeah as you get further along though this is a CNN in the static case on the first time step because you basically there's nothing to cancel out this is all zeroes so you just go up through like a CNN so you would expect there to be sort of different levels of representation of

Predictive Coding Models of Perception


higher level and lower level features so it's only long story short it's complicated the the simple fact of training it in this predictive mode is gonna affect all of these weights so it's going to induce representations even in these A's so the bottom line is like you can pretty much decode from anywhere this is bad news for Neuroscience right you can decode from

Predictive Coding Models of Perception


anywhere and all of them qualitatively look very similar in their dynamics so I think about the relationship between what you're doing here and the stuff that I've been working on but honestly really the ideas that were put forth by hoc right originally about supervise and that work would suggest that this system should be able to do something even richer which is to do

Predictive Coding Models of Perception


something like visual system identification so in the phenomena that you've you've talked about you look at the first frame and you can make predictions about what's going to happen because you live in this world but there are situations where I can't predict this observing the dynamics helps me predict the dynamic so if I like let me say it's seeing or someone with a fuse

Predictive Coding Models of Perception


joint or something should be able to do that kind of system identification on the I don't know but I agree and we're I think we're all kind of using the same market fuel right it's like Ellis TMS can do a lot and these are sort of two separate instantiations of like recurrent Nets can can actually do do a lot yeah exactly exactly exactly and and there is a spot that you know they even

Predictive Coding Models of Perception


after you fix things and it's running you know it has state that's that it's accumulating so it can you can do I think system identification so I mean this isn't this isn't fit exactly into your what you're talking about but we also had it doing things like predicting balls bouncing around and things like that and you know and it works you know like the balls bounce off the walls yeah

Predictive Coding Models of Perception


and if we have these like have like electrostatic repulsion or something you could imagine them learning dynamically like well you know this is this is the new rules so yeah and this is the kind of setting I would probably prefer to do it in rather than the other one cuz you can't control the natural world as easily but but I think it's really interesting I mean that there are

Predictive Coding Models of Perception


limits of the capacity of this thing to learn so it you can't you know if you have very complicated movies and things it doesn't do a great job of predicting the next frame so we've far from crack the nut but it just feels like we're kind of moving maybe a little bit in right direction so so much like the project I spent 20 years of my life working I've had so

Predictive Coding Models of Perception


many people tell me that it ultimately got published at nips as unsupervised pixel prediction and it was just trying to predict a moving dot no recognition of anything it was only doing the motion different continuity in time I wrestled with it it was awful it worked kind of okay yeah yeah while I was working as a coder and an algorithm guy and stuff in Silicon Valley what some C++ and a job I

Predictive Coding Models of Perception


worked at Redwood Neuroscience but ultimately what happened was I rely I alternate Lee gave a presentation here that it worked with just using matrices and just the algorithms forget the neurons it was that hard then I throw away even the matrices and finally got a zero parameter model where I just assumed volumetric 3d computational medium intracellular space so if you

Predictive Coding Models of Perception


assume brains just do 3d computations I so I understand I think I think we understand continuity in time and it's just what he said yeah but trying to build it on top of deep learning which was built around essentially n dimensional hyperspace all we have to do with 3d continuous it's not easier oh this is a soft key in fits paper is that the one I think I think I

Predictive Coding Models of Perception


think we I mean I think we I think we saw I think we cited you but the I think I think I think the you know the tools I mean the thing that's nice about deep learning and maybe others have this sort of intuition about it it sort of nice to be able to say here's our like I'm saying like presto magico deep-learning what I'm saying like I think we've gotten to the stage where but that's all

Predictive Coding Models of Perception


abstraction now what we can say is if we just optimize future frame prediction what else obtains and I think that's the right framing for using deep learning to sort of it's not that we're like doing neurons or that they are neurons like this has back prop like clearly you know like there's gonna be a couple suggestions on how we can do it better it's more like if you just optimize this

Predictive Coding Models of Perception


one thing and deep learning lets you very effectively optimize that one thing this is what comes out of it right so that that's kind of the mode we're thinking about it and you know could it be reduced to some other simpler thing sure but we just wanted to have the tools that take us to there to that to that point and I think we're I think we're out of time unfortunate this or

Predictive Coding Models of Perception


that feature of object this is the kind of work you did in your thesis and your adviser certainly you know should not fire at all if it's correctly representing this thing and so I'm trying to picture how the same room can be both representing something and they're not representing something and and the question is maybe the answer is in that your last picture when you said

Predictive Coding Models of Perception


so on the first pass to reader model basically the representational kind of thing is getting passed forward through these error units which are the only things passing anything for which is a pretty strong constraint in their model and so but then later those same units must be passing forward something it means something different which is you know the the non predictive part only

Predictive Coding Models of Perception


those are two very different things at very different at different times and so are you is that the correct correct interpretation which is the sort of there's a time multiplexing where maybe a first pass through the ventral Strait is the representational pass and could lead to successful recognition of an object and then later that thing is somehow shut off because everything is

Predictive Coding Models of Perception


predictable or what have suggested time multiplexing in the responses before so there's a paper from from yesterday's group I could be wrong about that where they're basically claiming that the initial pop of activity had different information than the latter part I would say when when when Jim and I have said in the past that this neuron represents that thing we're taking time like cat

Predictive Coding Models of Perception


spike counts within 100 millisecond window after you know 100 milliseconds after onset of stimulus that that same neuron is categorically not doing that 100 milliseconds after that right like you know there's this weird phenomena that a lot of the neurons are shut down so I think this idea that these neurons the firing of these neurons signal that you know without any further additional

Predictive Coding Models of Perception


context I think that that kind of has to be wrong that's on the level I mean the fact that these things are so dynamic I think the end of the day all the system cares about is that makes the right call and initiates the right behavior I don't think is any requirement that there be labeled line neurons that signal features but so this is an example where you have neurons that will look that way

Predictive Coding Models of Perception


I kept trying to get this secret to work for me for such a long time and it just wouldn't work until I changed one thing and I just want to show you what that one thing is are you ready let's go when it did work what happened is I manifested this no I'm only joking watch this because actually what I manifest was much more than just a five thousand dollar massage chair

Why 'The Secret' Wonâ€™t Work For You Until You Do This.. [Law of Attraction]


aside from a $20,000 watch I don't know if you can see out here and then this all became my lifestyle alongside that beautiful little boat there now your question is why am i showing all of this to you and what a show-off who is this guy anyway I just want to break it all down for you because it's not about me and I showed you all of this stuff it's because I want you to know that it is


possible one of the biggest things is that we need to believe before we can manifest and that's the biggest problem that a lot of people don't know how to get so I want to break it down for you into three simple steps in this video how to actually manifest and attract what you like what you desire in life using the law of attraction or the principles from the secret but before we


get into it so real hit it Bob today isn't that they're struggling it's not they can't get out of this it's because the that they're in isn't big enough and why I say that is because if that was big enough right now if the people had a girl acquaintance or that head they would actually get that ass moving always understand that emotions is what these emotion is energy in motion so if your