conclusions from this but you know humans draw on an animal's drawn evolution and innate behavior but I think it's just more efficient learning and another kind of reaction to this is we draw our background knowledge about about the world and that's true the big question I'm asking here is what does that come from how do we acquire all the background knowledge we have about the world that allows us to learn a new task very quickly so so all the success that we you see in practical machine learning today almost all of it


is due to supervised running and we all know what what that means right to give let's say you want to do image recognition you give an image to the Machine and if the machine doesn't give you the right answer you tell you what a right a right answer is and you adjust its internal parameters using stochastic gradient descent or something like that a gradient based method to get the output closer to the one you want the amount of information you give to the machine at every trial is relatively small even in the case of something like

Yann LeCun: "Energy-Based Self-Supervised Learning"


image net which has 1,000 categories you tell it with the correct categories and that's less than 10 bits of information so you're asking efficient to predict a very small amount of information every time as a result you need a lot of samples to try need to do anything reinforcement learning is even worse so reinforcement learning a situation where you don't tell the Machine the correct answer you only tell it whether the answer it produced was good or bad ok now there is like a harder form of reinforcement learning where what the


Machine sees next types type depends on the answer we got and then there is a problem of exploration exploitation etc but even without talking about this if you look at how long it takes even to learn to for a machine to learn to place the condition that it just runs by reinforcements come into play an Atari game very simple Atari game from the 1980s it takes the equivalent on average of 80 hours of training to reach the performance that any human can reach in about 15 minutes those machines actually get to superhuman performance but it takes them


a long time the the go system that was produced by deep mine and the one that was produced by Facebook a little bit later I can I know the numbers for Facebook because they published and also they're my friends this takes about 20 million of self play games to to reach super human performance running on 2003 P news for two weeks this is a lot of games mortal games that any humans can play go yes go is complicated and Starcraft so this is a recent the paper actually just appeared last week but the results have


been known for a while from deepmind the Alpha star system takes about 200 years of equivalent real time to learn to play on a single map for every single player a single type of player if you want and that's an enormous amount of computation there's rumors that just to train this for a week or two that team took more computational resources at Google that auto research okay and you know similar for there is a you know recent demo by open AI and of course when he paper are using the first month on internal manipulation from simulation and then


you can sort of transfer this to a real real robot and it takes the equivalent of ten thousand years of real time in simulation so you can run simulation fast or you can run it in parallel it just costs money or power or co2 emissions but but it doesn't work in the real world so if you want to train a car to drive itself and you don't have accurately no simulation to turn on this in simulation it's not gonna work you will have the car you'll need a car to drive itself or you know millions of hours caused thousands


of accidents destroyed South multiple times it will have to run out forklift multiple times before it realizes it's a bad idea to another cliff when it starts it doesn't know anything about gravity or anything like that and so it's not practical for the real world although it may be practical in simulation if you can do an accurate enough simulation but it's gonna cost you a lot in terms of computation so how is it that humans can learn to drive a car in about 20 hours of training for most of us without causing any accidents also for most of


us right that's a big question how do humans and animals run so quickly and what happens is there is that it's not supervised running it's not refunds my drowning it's something else and so when you look at babies you talk to a cognitive scientist development to a cognitive scientist and and you ask them you know when do baby learn basic things like gravity you know when do they learn that objects are supposed to fall I'll tell you around nine months so before nine month old you should be the scenario here but there's a little car


on the platform you push it off and the car appears to float in the air it's a trick of course babies barely pay attention that's another thing they see how is your Kendall stuff doing there that day they learn from every single one of them but that doesn't surprise them after nine months old they've run it by gravity and they look at this like a little girl here they're really really surprised because in the meantime they've learned that objects are not supposed to you know kind of float in the air they're supposed to Foley's are


not supported and so that's actually a trick that a method we just I would say that psychosis is used to identify when babies learn new concepts so you know babies run face-tracking very quickly and you know this is could be you know there are computational models that learn kind of face detection based on emotion you know self supervise being on motion that learning in minutes so like that could be learn but really quickly the notion of object permanence the fact that when an object is hidden behind another one


is still there we don't seem to be born with this but we learn this quite quickly as well so many moles are born with this the distinction between animate and inanimate objects that's run around three months they're objects whose trajectory are completely predictable and others that are not animate objects and you go gravity inertia a conservation of momentum basically what we call intuitive physics that that comes much later around nine months and it looks as if or maybe that's our hypothesis but you know babies kind of


learn sort of basic concepts kind of a you know step survive stage pieces sort of building more abstract concept on top of simpler ones so for example you know are we born with the concept that the world is three-dimensional or do we learn this I think it's a good hypothesis to think that we learned this a lot of psychologists will tell you we're born with it but I don't see like how the cortex could be wired to sort of you know you know tell us how to compute depth right although there is certainly some bias in the in the wiring that


makes this favorable in the sense that you know connections from the left eye and the right eye actually go to the same place in the cortex so if the if the cortex wants to compute disparity it's easy for it the work the wires are there okay but the function not not really and so here is how you could compute how you could learn that the world is two-dimensional if you train your visual system to predict what the world is going to look like when you move your head the best explanation for how the world changes is the fact that


every pixel every location in the world has a depth right because then you get parallax motion so implicitly if you want to predict what the wall is going to look like when you move your head you're gonna have to learn that implicitly even if you have no idea that the world is two-dimensional that's the best way to explain how the world changes okay so that's not that's an idea that suggests how we can learn very simple concepts just by learning to predict essentially and that's going to be the general theme of this talk which


is learning to predict prediction is the essence of intelligence in my opinion and so we build models of the world that allows us to learn to drive into in 20 hours to know kinds of stuff but animals do that too so I really love this video of this little baby or wrong song here is being shown a magic trick where you put a cherry in a cup and and then the cherries removed but he doesn't see that and then the cup is empty and he was on the floor laughing okay so he's his model of the world is obviously being violated and he finds that funny Hey I


mean there's the these two things that happen when your model of the world is violated either you find that funny or you find out or you find it scary because here is something you didn't predict it could kill you in both cases you pay attention okay so that brings us to this this audio cell supervised running this idea of running by prediction okay so not learning a task not running to classify objects in categories that you know come to you from a deus ex machina but learning the structure of the world by just observing


the world essentially so the basic hypothesis of this is or the principle that you can base this on is predict everything from everything else what do I mean by this so let's say you have a piece of data for the sake of concreteness here let's let's think about a video clip for example there's going to be a piece of that data that you're gonna tell the Machine you can look at it and there's another piece that the Machine pretends it doesn't know it doesn't see here it's the the future frames of the video okay so it


looks at the video up to a point and then it tries to predict the rest of the video but it pretends it doesn't know it yet and then it trends itself to predict it of course it can just wait and observe what's going to happen in the world and it trains and predicts it by just observing what happened another form of this is this it's called a mast cell supervised running you give a piece of data it's very popular in the context of natural language processing these days take a window of text a bunch of words you remove some of the words and you ask


the machine to predict the words that are missing in the process of doing so the machine has to basically develop a representation of language that this allows it to make those predictions and basically in the process of doing this it kind of understands language not completely not deeply but but still but really more generally is the idea of taking a piece of data and asking the machine to predict a piece of it from the piece that it sees so as I just mentioned this type of learning is extremely has become in the last year


has become extremely popular in natural my which processing is actually part about a huge improvement in performance of all natural language processing systems including translation search to Google there's been a series of ideas you know going back to the 90s on this but really sort of a paper that that convinced everyone that's you know decide to be the thing came up on archive in October last year from from Google from Google Google a I was on Google brain actually and they use a particular type of neural net a gigantic


one called a transformer architecture so transformer architecture is kind of a funny kind of neural net where groups of neurons basically implement some sort of memory module differentiable memory module so they know they don't just compute weight it's on the complete weighted sums but then they compare those weighted sum with with vectors that called keys and then that gives them scores that you normalized to one and then you can compute a linear combination of other vectors and sort of complicated but it's kind of an


associative memory every module in there is an associative memory and you put 40 layers of those with hundreds of millions of parameters you train this on billions of words of text and you train it in the following way you take a window of a few hundred words you take out 15% of the words and you train the machine to just predict the missing words now the machine cannot do a perfect job at this so what it outputs is for each word there are missing it that puts a probability vector whose size is the size of the of the


dictionary and it gives you a probability for every every word right so that's the way it handles uncertainty in the prediction it produces a large probability vector this has completely revolutionized NLP everybody does this now it works so well that Google deployed this within like in in the last few weeks they basically deployed this as a way of for example if you ask a question to Google it will produce an answer the answer is computed by anything like this Facebook has deployed as a develop things like this for translation and content


filtering hate speech detection all kinds of stuff yes right well yeah here you know what those words are right it's just you you it's not right it's not it's supervised running with two differences what is you don't have a extraneous piece of data us submission to predict so basically you're not asking you to perform any task although that understanding the input data the internal structure of input data second thing is that prediction cannot be known exactly because you know you can't predict exactly which word is going to


go here and so you need to deal with uncertainty and those are crucial key points that so distinguish this from sort of regular supervisor on human okay it doesn't work so well for yes yes it produces a probability vector over all words yeah it's a separate one for every word by the way so yeah so there's no consistency between if you pick one where you can put another word independently format from that distribution vector yeah so of course people try to do this for images so the equivalent of this would be take an


image you know blank out some of the areas of this image and then train a neural net the coefficient net or something to predict the missing parts the problem with this is that now the distribution of outputs is all is over a high dimensional continuous space and we don't know how to parameterize good distributions over those so those so far have not been very successful not to the extent that those have been successful so the way you use those things is you train this network and then you take the internal representation of language that


those things learn and use it as input to a supervised task his fish detection answering questions you know whatever this group to students at Facebook in Paris who have used this for training your translator so you give a sentence a sentence in English and a sentence in French you remove different words randomly from the two sentences and then you ask the system to translate and the magic thing is that because some of the words that are removed from the French version are present in English version it learns to produce a


representation that is independent of language so what you get in the end is a meaning representation that works for English in French and you have two encoders went for engagement from French Google has a version of this has you know a hundred languages Facebook now has a version of this that has compiled languages those are massive networks the latest biggest ones are tens of billions of parameters it's just ridiculously large yeah it's inventing on steroids exactly yeah yeah so because you can't you can't pick it you know you can't


pick pixels independent of each other so this is a trick that deepmind has proposed if a former postdoc of my daddy find Carl Gregor which is you make you make the prediction of pixels sequential and you turn it into a classification problem for the great scale where it's you know one among 256 its it just strikes me as wrong you know it kind of work surprisingly well but you know it can be the ultimate answer no I think we'll find something better so there is actually yeah there is there is a second studies about this actually people who


have tried to study what the representation inside so Chris Manning at Stanford has done some some work on this and his various groups it seems that those things actually represent meeting to some extent right it's not a deep understanding of text you know but its shadow because those are words that are not connected with the real world right i mean the thing only sees text this big question so it has the linguistics community up in arms because it basically you know breaks the entire universe okay of like you know what


about grammar what about you know semantics what about all those things is all statistics you know what about symbol manipulation right those things basically just represent everything by vectors they embed everything in vector spaces and so the the chomskyan linguistics say oh my god they write books against this okay so supervised running you you you train a system with sort of a pretext task which is not really a task it just reconstruction or prediction and as I said as we said I said it works with you well for texts


and symbols and people use this now for like you know DNA sequences all kinds of stuff it's very new images yeah not so much video not so much either signal audio not so much either there is some results in other words so improves the steadily out a little bit but they're not as successful as in NLP NLP they're incredibly successful okay there's another reason why we might want to use our supervised learning which is and it goes back to this idea of training a car to drive itself the reason why we can learn to drive a car in 20 hours with


that crashing most of the time is that we have this model of the world that allows us to predict the consequences the consequences of our actions so we know that if we drive next to a cliff and we turn the wheel to the right you know the car with we're off to the right it will run off the cliff and crashing on the bottom because we know about gravity and nothing good is going to come out of it so we don't even try right because we had this predictive model we can predict the consequences of some of our actions at least so the way


this works is actually a very standard thing for optimal control theory which is if you have a predictive forward model of the world they gives you the state of the world at time T plus 1 as a function of state the world time T your action and perhaps some latent variable that represents all those stuff you don't know about the world that you can you can sort of run this in your head you can you know run your world model in your head with a proposed sequence of action and see what the result will be and you can measure the cost of it you


know you can have an eternal cost for a good things are you know and I don't want to cry right and so you can you can sort of run this model forward and perhaps infer a action sequence that will minimize your costs right that model will have to be learned with self supervised running basically here is a state of the world let me take an action and see what the result is or not taking action just because the world is being the world and that's the same problem we need to solve here the self supervised learning


problem and the main part the main issue is that the world is not deterministic okay so that it be to this picture of the three paradigms of running if you want reinforcement learning supervised running and soft supervising the difference is how much feedback information you give to the system at every trial or every sample here is just one scalar here it's just a few bits for example and here it's basically a whole video right it's a huge amount of information you give to the machine so the hope is that you can try and


gigantic networks without them being you know ridiculously overpriced and it will learn a lot of the structure about the world just by observation without actually taking any risk and without you spending money correcting labels that's probably how humans and animals learn so much that might be how common sense emerges right the accumulation of all the background knowledge we have about the world by that we accumulate by observation that's how that's the basis of common sense essentially so we need to get machines to do this and for I


mean I've been sort of advocating for this for a while and you know what Middle's obnoxious you know slides without reinforcement learning being a cherry on the cake of machine learning and supervised reading being the dark matter of AI we don't know what it is there's actually the dark energy you've more you know it's most of the energy okay so the next revolution is I will not be supervised you will not be reinforced either I saw this from Alyosha of course my colleagues Eterna bamileke who's from Berkeley also in Facebook


here says labels are the opium of the machine learning researcher so you know it's all like revolutionary statements and some dude actually produces a t-shirt that you can buy okay so that gives me two inertia base models which you really really is a proposal for how we approach this problem so the main problem is how do we predict with uncertainty so if I if I do an experiment I'll come back to this if I do an experiment which is I put a pen on the table and a lady go and in a minute and if I I repeat the experiment


all table times and it's not the same video clip every time the pen will fall in the different direction and if I ask you to predict what is the state what is going to be the set of the world in two seconds you can tell that the pen is going to fall but you can't tell really in which direction right most of the time so if you train a deterministic function to make one prediction the best thing you can do is predict the average of all the possible futures which would be a transparent pen in all possible configurations and if you actually do


this you train a system to predict video to pointing video frames you're the fourth frame or the the first four frames of the videos are observed the last two are predicted you get blurry predictions they're basically the average of all the stuff that could happen and then machines can't decide which one it has to make one prediction yes yes No okay so address all that works ten try to solve the same problem in a different way from the one I'm going to explain but I'll come back to that analogy later okay so the point is if you have an


input a deterministic function that produces one output and some cost function here that measures the discrepancy if this cost function is only zero when when the prediction and the observation are the same then this guy can only predict the average now to me to come back to your point if you make this cost function complicated in such a way that it doesn't compare points but it compares then you know distribution so example then yes but then that becomes complicated that's what adversarial training is about you


have to train this thing to do the to copy all distributions which is hard okay so we're not going to use a deterministic function so here is the the crux of energy based models and it's very connected to things like factor graphs that the people were talking about in the context of graphical models and Bayesian networks and stuff like that so basically you have an input and observation you have an observed or hypothesis for prediction and you have an energy function here that measures the compatibility between the two if the


two are compatible if Y is a good prediction for X then the energy produced by this is Rho if they are not compatible it's high okay and the way you do inference is that I give you an X you find a why that might be multiple that produces a low energy okay so for example here this is x and y if I give you this X there are two possible answers for why I have low energy right this is the manifold of data that's why they does that our sample form and you know you'll you'll have to answer so I'm not telling you how to do this


minimization and how to produce multiple answers but that's the inference mechanism so the energy here is not used for training is used for inference it's very different so you could say well alright you know that's an energy function but you know you can take the exponential and normalize with a gives distribution and it gives you a probability yes except that I don't actually want the log of a probability I don't want the energy to be the log of a probability here the probability is a set of measure zero its peak like if I


know anything about math right you know it's a thin plate and so the the distribution here would be you know you know infinity on that point on that on that manifold and 0 just epsilon that side of it which means this energy function or ever you parameterize it will have to have infinite parameters infinite weights something you can set your own net and it's not very useful because you can't do inference with this it becomes a golf course you can do inference what you want is a function that is smooth so that at any point here


the gradient of that function might tell you where to go to get a point from the manifold so this is very emphatic you do not want to learn distributions they're bad for you right maximum rank liquid sucks it just doesn't do the right thing this is big mistake that actually gets to the original formulation insists that you know this should be you know one here zero outside and that's just it's just a bad idea so that's an example where you know applying probability theory blindly actually is bad for you and I've been


trying to snap people out of it for twenty years without success so far okay so this is the what I just showed you is the conditional version and there's an unconditional version where you don't actually have an observation the only thing you want to know is like model the internal consistency of why and really those two problems are not that different from each other in the first case you know a priori which set of variables are observable in the second case you don't know which part of why it's going to be observed and so here


the what the model gives you is kind of a dependency you know a function that gives you the dependency between y1 and y2 in this case so things like auto encoders or this type yes okay it's akin to negate you look like a hood but you don't want to train you to with maximum maximum okay good or not at least without heavily regularizing it and you don't need the normalization because it's not like you're gonna sample from it anyway so in the end it's just an energy that's the more elementary concept that you can derive from and you


know physicists in the audience know that energy is more fundamental than probabilities probability is going to derive from from it or Hamiltonian if you're a quantum physicist but it's not probability this amplitude well whatever okay so how do we trade on energy base model so of course we're gonna primate Francis in some way right it's going to be some sort of neural net with a particular architecture and it's gonna have parameters in it and we need to train it in such a way that it takes to shape so that the data the training data


we observe take low energy and everything else has higher energy and not specifying that it should be the you know that the difference of energy should be akin to wish you know difference of love of our abilities insist on that and there are two classes of methods for doing this without contrastive methods and architectural methods so quadratic methods basically consists in pushing down on the energy of points of data points right so give a give a pair XY to the model and twist the parameters so that the energy coming


out of it goes down easy and then the contrastive term which prevents this thing from just collapse into zero everywhere picks points intelligently outside and pushed their energy up okay and the problem is how intelligently intelligent you have to be - can I pick those points and by the way guns are an example of this so again for example where the discriminator is this energy function and the generator is the smart system that picks out the points whose energy going to push up that's called energy B's gets any paper on this


a few years ago and then there is architectural methods and those architectural methods consist in building the energy function in such a way that the volume of stuff they can take low energy is is limited or minimized okay so either about construction or through some regularization term and I'll come to how you do this in a minute but let's start with okay so there are all kinds of you know traditional and supervised learning methods that you can caste that in that language and I said you know basically the construct the basic idea contrastive


methods is push down the energy of data point push up everywhere else or which what is maximum likelihood does if they have if you have tractable partition function approximation put down the energy of data points and push up on chosen location and maximum likelihood with Monte Carlo Markov chain Monte Carlo an alternate Monte Carlo contrasted divergence metric you learn English for matching all the stuff basically are different versions of this including gas and the third one is trying to function at most points of the


data manifold two points on the data manifold that's called denoising auto-encoder and that's why those NLP it was largely a pea model i was telling you about do that's called a master to encode particular case of denoising auto-encoder I'm gonna mention a little bit metric learning because it's one of the few cases where it works and it's in fact the only case we know it works in the context of images so it's kind of important and it's those results are reason like last week and then there's architectural methods some of them some


of you I'm sure know so I think like PCA so PCA you make sure the whole space is not reconstructed because the representation is constrained to be a low dimensional k-means just make sure model square is here etcetera these are the ones I'm going to talk about because that's where my money is right now so I think like sparse coding sparse or two encoders which some of you of course I've heard of and then the other ones I'm not gonna mention okay so how does it work in the context of PCA k-means so in PCA the the the region of the space


that is perfectly reconstructed is the principal subspace right in this case here if the data points are sampled from this from the sparrow the principal subspace here dimension 1 is this so this has become our reconstruction error 0 and everything else grows could racket quadratically right because you take a point you project it on this and so if it's already there reconstruction here at zero if it's here reconstruction error scores that the square of the Euclidean distance not a good model of a spiral as you can tell k-means says so


k-means interesting because it has a latent variable in it so the energy function is not directly a function is the minimum of some other more elementary energy function okay so it's the the min over a vector Z of the squared distance between the data point and the this Z vector multiplied by a matrix whose columns are the prototypes of the the k-means model and you constrain this vector to be a one heart vector okay so only one one only one component can be one the other ones have to be 0 and so you have to do this search exhaustively which is


akin to a nearest neighbor and what you see here those of you black areas are the minima centered on the prototypes ok so KB's just put you know prototypes more or less equally distant over the many fold it looks great in two dimensions and how dimension k-means really doesn't work that well but what's interesting about those both of those cases is that they work because the capacity the volume of the white space that can take low energy is limited okay that's a key concept or you talked about the maximum likelihood so I'm going to


keep that okay so that leads us to this idea of latent variable models right so so energy models f of x and y that are actually defined by minimizing a more elementary energy function e of X Y Z with respect to Z or by marginalizing over Z which is equivalent to defining f of X Y as minus 1 over beta which you can think of as a new verse temperature log integral over Z exponential minus beta the energy ok this is a log partition function for those of you know and this is a free energy which is like Oh F for physicists in the room so


there's the conditional version and the unconditional version which only consists in taking X out ok so that's that's what the model looks like you have observe variable variable you need to predict and some latent variable you have to minimize over now why is that interesting latent variables are interesting because they are an essential tool for making a system be able to predict multiple outputs instead of just one so if I build a system out of deterministic functions here I have you know a neural net with a few layers


it produces some representation of the observed the observed variables and then I feed this to another neuron that I call the decoder together with a little variable by varying this latent variable over a set I can make the output very over set may be something complicated manifold of this if this network is complicated and basically that allows me to solve this problem of not really take the average right I can just you know predict the actual thing I'm observing by finding the latent variable that will make my model pretty the best thing so


here is how you train an image based model there you show it an x and y you find the Z that minimizes the the reconstruction error okay and if that's not perfect and we're one step O's to caste gradient descent you update the parameters of whatever functions you're using to make this small okay this works great except there's a there's a slight problem with this which is that imagine that Z has the same dimension as Y okay so in C are the same dimension as Y and the decoder is not you know the generate function then it's kind of a powerful


parameterize function then for any Y you show the Machine there's always going to be a Z that's going to reconstruct it perfectly which means your energy surface can be completely flat it's not a good model dependency of Y on X because your energy function doesn't tell you you know which Y is good so again what we're going to have to do is limit the information capacity of Z like we did with camis basically we're going to have to limit the volume of white space they can take low energy and that volume will have to be commensurate with


the you know the volume of our or data manifold essentially okay but let's start with contrastive embedding so church embedding is following idea to handle the fact that multiple Y's are compatible with X you feed both x and y to neural nets and those neural nets will have invariances and so you're going to be able to modify Y for a given X without changing the output okay because of the embarrasses built into the system and that's a way of handling the fact that there are multiple ways are compatible with the next but now the


way you need to train this is that you need to tell it ok here are two images they are actually the same you know conceptually so whatever representation you extract from this image should be similar to the representation you extract from this image so basically I want HNH fine here to be as close to each other as possible because really they represent the same thing but if you if you only do this you see you collapse you see basically those Network completely ignoring the inputs and producing constant vectors right so you


need a contrastive term the contractive term which is kind of a subclass so this side you are pushing up the energy of so if you don't want is you show pairs of examples that are dissimilar and then you train those networks to produce outputs that are different from each other and these various loss functions to do this so this is in the business it's called Siamese neural net and it's another idea but it's been kind of revived more recently it's been successful for training of face recognition system or there's a the


paper that just came out that actually beats to actually use some supervised learning in vision to improve performance of a super purely supervised learning is this one moko and it's using a trick to kind of slow down one of those the main issue here is the difficulty of finding hard- so here you have to mine your entire data set for images that the system thinks are similar to this one but really aren't and that's really where where things become complicated but that idea this paper I just mentioned actually improves


the performance of image recognition systems over purely supervised yes/no nights you know whatever your favorite primary classes but in these cases large compositional nets in fact the name of the architecture is right here this means resonate 50 with four times the size of the future maps resonate 50 is going to a standard architecture for image recognition so the the bird system that's used for an LP those those mashed auto-encoder or denoising auto-encoder the the diagram looks like this you start with a piece


of data you corrupt it which means you remove some pieces you run it through a few layers of neuron that there's a latent variable which is implicit in those models which is like which of the outputs is picked as a function of the of the probability distribution on the output and then you compare this with the the actual data that you observed and you train the entire system to minimize the reconstruction error and in continued space the conceptually what that does is that if you imagine that your data manifold is this okay those


points you take a point you corrupt it so you you add noise to it for example in this case and then you train the your parameterize neural net to map this input to the output okay you feed this as input and you tell it you should map it here once the system is trained you can actually plot the vector field of you know those are little vectors that point in the direction where the neural net if you feed with this input would would take you I mean you have to lengthen the the thing but they almost all take you to the to the manifold here


and the color here indicates the energy so the energy is low on the manifold which is what you want it's high outside except there's a problem right here there's a ridge here and it's a it's a kind of a flat Ridge which is not good so here the reconstruction error is actually zero because the system when this train doesn't can't decide whether to go this way or that way so there's a flaw with this with this thing there ways to fix it but no clear the main issue with this is that it doesn't scale with well in high dimension because in


high dimension there are many many many ways to be different from a sample and you're never going to explore the entire dimension of the all the space it's right it could be either but the part I'm abusing the reconstruction error is that here the reconstruction error is zero so which means the energy is zero spent so it's a phantom Low Energy I think there are ways to fix it there are no cheap you sure I'm not gonna go too okay so prediction with latent variables as I told you before I give you an X I'll give you Y you find the Z that minimizes


the reconstruction error and unfortunately because if Z is high-capacity this is gonna give you a flat energy surface so the solution to this is your regular Y Z you basically add a term in the energy along that x RZ where our Z measures you know basically tells you if you are at a particular region of space that you're happy with and so basically you pay a price for making Z go outside of that region a good example of this which is familiar to many of you is our Z could be the l1 norm of Z so if you put the other one


normal Z the sum of the absolute values of the components of Z to make this small you have to basically make many of the components of Z zero as many as possible and so you end up with sparse sparse representation and that actually limits the the volume of space that is the has low energy essentially this is this is what what you get so this is sort of the unconditional version of it where there is no X is you just modeling Y and here I give you a Y Z the regular as before Z is the other one norm Z is multiplied by a matrix color item matrix


he won't decoding decoding matrix it produces a reconstruction you measure the square Euclidean distance between the two and that's your energy function so a classical in the applied math community at least you could generalize this and so what you get is when you turn this on this little spiral here yeah you get that the low-energy regions are basically piecewise linear approximations of with sort of low dimensionality in your subspaces of the entire system that works really well in high dimension that's the cool thing and


it's been studied you know a lot one thing that is not yet studied is what if you make the decoder nonlinear let's say instead of having a matrix here that you multiply Z by you have an entire neural net what happens I'll tell you about this little bit later now here is the problem though finding you know finding I'm sorry it's not what I wanted to show finding Z for a given X a pair X Y finding the Z that minimizes the sum of those two terms can be expensive you have to back popping equation today is to go in and decide this could be a non


smooth functions you have to do those l1 l2 optimization you know Easter or whatever it can be expensive so one idea is you actually train and you're on that to predict the optimal solution to that optimization problem okay so ignore the the great part for now I'll give you an X on you I find the Z that minimizes the sum of this and that and then use this as a target to train a neural net which from x and y is going to predict this guy and then if this guy is well trained that I don't need to run the optimization algorithm for inference


anymore I just need to run through the encoder so that becomes very clear that is very important to limit the information content of Z because the system can cheat here it actually has access to the answer and you can just you know copy the answer on the output and so unless you have a way of restricting information content of this asset ml children completely ignore X okay so if you have the unconditional version of this we don't have this part this is called a regularized auto encoder or sparse auto encoder this


works really well in the sense that if you train as possible to encoder like this weather color is linear and the encoder is a few layers of a neural net on the at least the columns of the decoding matrix end up being little parts of characters which means you can reconstruct any character with a linear combination of a small number of those things people call this call this these things atoms if you try our natural image patches this is a running algorithm running you end up with oriented edge detectors which is


great you have to do a little bit of whitening for the images if you do this in a compositional mode where the the decoder is actually conditional so Z is not a vector it's a bunch of feature maps and then you run them through collisions and you compute the sum and that's how you decode you get beautiful so these are the the business functions in the decoder the kernels that I used to reconstruct the the outputs and these are weights in the first layer of the uncoated encoder on only has two layers in this case and they're basically


mirror images of of the decoder and this is four one two four eight 16 32 64 filters you get a very high diversity of filter centers around gratings oriented edges at various frequencies it's really nice this is these are ten year old results more recently we can revive this technology because it's very interesting so this is again filters that are learned on natural image patches from the c4 dataset 9x9 kernels and those are corresponding feature maps that are extremely sparse that can reconstruct basically any image on so far with real


chip very good accuracy some other work we're doing along those lines is having multi-layer decoders so basically here is an image and then take a bunch of feature mats here run them through completions and value and commissions on value when we construct and then you can sort of stack multiple layers of those on train this carefully if you know how to do it she's not good not easy but it kind of works these are reconstructions this is the original and these are kind of reconstructions you obtained with sparse


representation so if you only reconstruct from here you ignore the rest you get you get sort of high frequency information if you only reconstruct from here ignore this you just run through this network you can reconstruct a low-resolution version so this is you can think of this as like nonlinear wavelets if you want write the system so naturally learns to represent this let me skip this ok talk about this really quickly so something that's become very popular in the in the business is something called version of tiny coders


and variation over 20 coders are basically auto-encoder models they could be made conditional if you want but I cannot grade this out and they are an example of a model where you also limit the capacity of the of the the representation here in the middle and the way you limit the information capacity of this vector is that you add noise okay so basically here's a why you run to an encoder you produce a prediction for what the code should be and then you add additive Gaussian noise to it and you run to the decoder and


there's a constraint here that it's a penalty really used during learning that the the norm of the outputs of the encoder need needs to be as small as possible okay so it's l2 regularization if you want during learning now what how does that give me the information content of the code well let's say that you train without noise right so you train your two encoder is going to assign a code this is in code space it's going to assign a code vector to every training sample these are all of the training samples now I'd know as to


those guys you turn them into fuzzy balls okay and those fuzzy balls might overlap so for example this this sample and that sample mind up my end up being confused with each other because when you add noise you can turn one into the other and so the reconstruction error will probably increase so what is the system going to do very easy it's gonna make those fuzzy ball fly away from each other right so that they don't overlap and that really is not that interesting you know it just makes the norm of the output of the encoder just larger but it


you know it doesn't do anything for you so what you do is you play a trick you attach each of those little fuzzy balls to the origin with a spring okay you tell them okay you know you can fly away but not too far so you cannot have to overlap with each and construct some sort of data manifold if you want and two bubbles will overlap to the extent that the reconstruction error is not dramatic on the output okay so there's a trade-off between the strength of that spring the size of those bubbles which in the case of


rational coders autoencoders are maximized actually and and and things like that and if you if you read all the papers and rational tone colors it's never formulated like this it's formulated as you know some version will lower bound or some probability distribution but it's a mechanical analogy I mean it makes it completely clear this is just a way of reducing the information content to keep the information capacity of the code okay I'm gonna end with an application of of all this which is the problem of predicting what the world around you is


going to do for things like avoiding to bump into other cars for example right so I already talked about this idea that if you have a forward model of the world that gives you the state of the world at time T plus one of the state the function of the state at time T and the action you're gonna take you can sort of roll out a an action in your head with using this model and there has planned a sequence of action that will minimize your cost here the cost being I want to stay in my lane I don't want to bump into other car I don't want to get too


close to any other cars okay and that's a different triple cost so I'm not talking about reinforcement running everything is differentiable everything is computable I don't need to try anything I mean I don't need to like estimate gradients of stuff by by trial and error everything is differentiable so the part name of course is that this this model of what cars around you are going to do is not deterministic right there's a lot of things that cars around you are going to do that you know you may not predict


and so there is a latent variable in the model that you're going to need to sample which is which is going to parameterize the set of all stupid things that cars are on you can do and and non stupid things as well okay so you start from a state which you observe this is your current state this is what you wear the cars around you are and you sample the certain variable you take an action of your action and then the system gives you a prediction for where the car and you are gonna be proud chiptune you okay if you decide to turn the wheel the


world around you is gonna reel is gonna rotate okay so this is predicting what the world around you is going to look like and then what you could do is you can back propagate gradient from the cost to a network here that is supposed to predict the correct action from the state so should I turn the wheel should I break sure that accelerate and by sampling multiple samples and running this on different initial conditions you you might have a car that trains itself to drive without actually driving just by just thinking about it having trained


is forward modelled by observing all the cars driving so the way we do this is that there is a camera that looks at a highway from the top and then you track every car and you extract a little rectangle around every car centered on every car and it turns with a car and so that's the world around every car and then you you can record sequences of those little things by tracking every car and that constitutes a training set the set of videos centered on every car and so you give a few frames of this thing observe frames and you train a


system that has written variables and all that stuff to predict the next frame so Z Z we represent all the stuff you can predict that the other cars are gonna do essentially right oh I see it's that's a good question I think it's I think is 256 dimension vector of 206 dimensions so this is so for inference you need to kind of sample z4 for training Z is given to you by by an encoder basically right you train but you need one of those information capacity reduction here which in our case is done by a combination of adding noise and what we


call dropout but it's basically set Z to zero it forces it to be zero half the time and so it tells the system like you know even if you don't have a latent variable do a good job at predicting whatever you can and then half the time it lets the system use Z and the the latent variable is combined additively with the representation extracted from the predictor so that zero has kind of a special meaning if you want so this is what it produces so this is recording of the real world this is a prediction when you set Z to zero all the time and so


you get blurry predictions and what you see here are I'm gonna restart if it wants to restart so what you see here are four different predictions you know run kind of recursively for different samplings of the Z variables and you see they predict different features and it's indicated by the square on the circle here the indicate cars that do different things for the different simply the cost function for training this thing is very simple it's you know whether the car is in lane or whether it's and how far it


is from its neighbors and so you can train this policy by just by popping in gradient of the cost through the entire system all the way down to the policy Network if you do this it doesn't work because what happens is the system gets into regions of the space where the foreign model does a really bad job at predicting what happens to have low cost okay so the car can it goes off the road or something like that and this can be due also to flaws in the in a cost function but but basically it doesn't do what you want so what you have to do is


regularize it by forcing the system to stay within regions of the forward model that where the foreign model is pretty sure of its predictions so that the system doesn't try to drive crazy stuff in crazy ways that are not present in the training set and and where its forward model can't really predict what's going to happen accurately and you do this by estimating the uncertainty in the prediction to the forward model by sampling the output of the forward model with these random variables you can sample like the the


drop out of in the network computing the variance of it and then using this as a term in the cost function so it forces the system to stay within a region of space where predictions are fairly reliable with low variance and this is where the system does so so this is the car being driven the green cars are recorded videos and the white dot indicates whether the car wants to turn accelerate brake etc and they it's perhaps more visible in this example so the yellow car is the car that is in the recorded video the blue car is the one


that we are driving and it didn't change lane the problem is that the blue car invisible to the other ones and so it get squeezed and it has to escape because the other cars will just record it right so they don't they don't see the blue car is another example they're you know they are there is gonna less issues it's trying to stay sort of halfway between the cars in front in the back okay so that's slide I think this whole idea of supervised running is associated machine learning this don't necessarily believe me but that's where


everybody is I think we can learn complex hierarchical feature for low resource tasks which is which is becoming really important using supervised running actually in natural language it works is very important for natural language for example it's important for Facebook to be able to translate Burmese into English or to more precisely to actually train a classification system that detects his speech in Bernie's because there is a sneek conflicts in in in Myanmar and so you want to be able to detect a speech


to prevent bad things from happening but how much data how much training data we have in Burmese so one way to do this is to kind of turn text into a language independent representation and then train a speech detector independently of language it's very important for low resource languages like Burmese or whatever I mean there is 2,000 language is something that people use on Facebook the advantage of that is that we can train massive networks it can accumulate a lot of background knowledge about the world in an on task dependent way and


then we can use several techniques that handle uncertainty by to learn forward models for model-based control and reinforcement learning model base reinforcement random so my money currently is on energy based approaches latent variable models so that it can handle multi modality regularize rates and viable models to prevent this collapse problem in particular sparse latent variable models although the precise way how to make that sparse is now clear and then latent variable prediction through a trainable encoder that's what I'm


hi this is Jeff Dean welcome to applications of deep neural networks of Washington University in this video we're going to look at how we can use ganz to generate additional training data for the latest on my a I course and projects click subscribe in the bell next to it to be notified of every new video Dan's have a wide array of uses beyond just the face generation that you often see them use for they can definitely generate other types of images but they can also work on tabular data and really any sort of data where

GANS for Semi-Supervised Learning in Keras (7.4)


you are attempting to have a neural network that is generating data that should be real or should or could be classified as fake the key element to having something as again is having that discriminator that tells the difference in the generator that actually generates the data another area that we are seeing ganz use for a great deal is in the area of semi supervised training so let's first talk about what semi-supervised training actually is and see how again can be used to implement this first let's talk about supervised training and


unsupervised training which you've probably seen in previous machine learning literature but just in case you haven't supervised training is what we've been doing up to this point I would say probably the vast majority of this class is in the area of supervised learning this is where you have multiple axes in the case of tabular data or grids and other things in the case of image data but you have some sort of input coming in which is the X and you know what the correct Y's are you are going to train the model to produce


these Y's when you have these X's because later on you're going to have X's coming in where you don't know what the Y is and that's where you want the neural network or other model to be able to give you some estimate as far as what the Y value is going to actually be unsupervised training is where we have the X's it could look just like this it would work with image data tabular or really just about anything but there is no y we're letting the neural network or whatever model it is and you know typically by


the way use neural networks for unsupervised training this is usually the area of things like k-means clustering and other things your classic unsupervised training is just going to take the inputs and cluster them in such a way so that similar ones are together these could be similar images these could be similar inputs in tabular data a variety of things semi supervised training it's actually much closer to supervised training I would say than unsupervised and this is where gams really shine and semi-supervised training you have X's


just like you have in these others but you don't have a label or a Y for every single one of them you might have a small number of them by oh no means have the complete data set label traditionally what would be done is these values that were not labeled would be left out because they there was no way to feed them into traditional supervised learning or you would train it on the ones that you did have Y's for with classic back propagation or however you were training that particular model then you would create predictions Y


predictions for all the missing values and then retrain the whole thing on the predictive values with the others in practice I never had a great deal of success with that technique but there is some theoretical basis for it with semi supervised training and Ganz will see that there's a way that we are able to actually make use of these now semi-supervised training this does make sense from a biological standpoint if you think about a child who is seeing all sorts of vehicles as they go about their daily lives with their parents or


whoever they're with and they're seeing all these vehicles as they pass on the street and they're not labeled nobody is telling them even though that's a vehicle seeing just a barrage of images as they as they grow up they learn edges they learn other sorts of things they learn how to classify if something is on top of something else just by observing there's no particular labels then eventually somebody says hey that's a that's a bus that's that's a train that's a bicycle using that that small handful of labels that they're given


when somebody actually tells them what they're looking at or they verify it independently that is semi-supervised training because it is building on those years and years of having unlabeled data that they they didn't know what they were looking at but they knew they were looking at something and it just it gives them additional reference that's exactly the same thing with supervised training these values even though we don't have wise they're still valuable for the neural network to be learning structure in this


data as it is learning to predict the ones that we do actually in fact have the wise heart so let's look at the structure for this this is the structure of a normal image generating gang baseline so to speak where they research started we saw this before but just to quickly review we have actual images they go into a discriminator and we have the generated images that the generator so the cyan pieces those are the two neural networks random seed values are causing that generator to generate images the discriminator is learning to


better and better discriminate between actual and generated the generator is learning to create better and better images that fool the discriminate now once this is all done you keep the generator because it generates images for you and you likely throw away the discriminator it was just there for the generator to practice against will see that this flips for semi-supervised learning and semi-supervised learning we care about the discriminator and not so much the generator we typically throw the generator away this is how you would


train a semi supervised classification neural network it's very very similar to the diagram that we just looked at in this case we're looking at how we would train it on tabular data it's a medical record the discriminator would learn to tell the difference between a fake medical record or whatever the generator is generating this parts all the same as the previous one as is as is this part the difference is we're training it now to tell not just the difference between fake and real these are the real and this is this is fake we're teaching


it to learn classes so there's four different classes of say medical record that we're looking at maybe four different health levels we're teaching it as a classification neural network to classify between five things the four classes that were actually interested in and is it a fake once we're done training the whole thing we now have this discriminator that can tell the difference between fake and what the what the classes are we also have the generator that is able to generate these fake medical records but we can then


throw away the generator and we'll use the discriminator really truly as our actual neural network now for the medical records where we don't have the Y so we're missing this we still feed those in it's just now we're evaluating it not based on if it classified it correctly but just if it knew the difference between fake and real the street house view data set is a image data set that is often used to demonstrate semi-supervised game learning and I have a link to a curious example external to this class that


demonstrates this if you're interested in this sort of technology but what this does is you have data on these addresses from images that were taken on the sides of buildings and not all of those are labeled or you simulate them not all being labeled and you see that the Gann is capable of learning to classify these 10-digit types even though it doesn't have labels on each of those now if you want to do the same thing for regression it becomes very similar you have two outputs so you have a multi output neural network one is the actual


regression value that you're trying to train it on and the other is the probability that it's a fake record being generated now these I'm doing tabular as just the example again these could be medical records and perhaps the regression output would be a health level or maybe a guess at how old the patient is or some other value perhaps if they have a current disease or not a prediction so it's it's doing the same two things when you feed an medical records where we don't know the Y output then we want to see that this regression


on the fake record when we're feeding in values where we have the medical record where you don't have the Y we just want to make sure that the probability that the fake record is fairly high and that's built into the training we don't so much care about what its regressing on or what the Russian output is for ones where we do have it we're penalizing it based on how close or how far away it was from the expected Y from this and just like the classification one when we're all done with this we throw away the generator in the


discriminator becomes the semi-supervised neural network that was trained on this now if you want to go further with this semi-supervised learning technique i've given you a couple of lengths of articles that i found useful for this there is a link to the actual house data set that's a pretty interesting data set to look at it has all those house numbers above you can deal with in several ways you can deal with classifying the individual digits they give you the bounding rectangles around the digits they also


give you just the bounding rectangle of the entire set of digits if you want to so you can be classifying digits or you can be classifying the entire address it just depends on how you want to set up the problem the examples that I give you here we're using individual digits this is the original paper that first started looking at this unsupervised representation learning with deep convolutional generative Gass general generative adversarial Network I have a link to this paper in the module thank you for watching this video in the next


once it is set maybe high level vision show some of the ideas I think are the big ideas in future learning I think the agenda of deep learning as the idea of using brain simulations to make learning algorithms might better than easy to use and also make revolution advances in machine learning and AI so come back to this later but you know once upon a time I guess when I was in high school I think I joined the field of machine learning because I want to work on AI but somehow that got lost and instead of actually doing AI we

Andrew Ng: "Deep Learning, Self-Taught Learning and Unsupervised Feature Learning"


wound up spending our lives doing curve fitting which is not what I signed up to do uh and and deep learning was for the first time in many years made me think about the bigger dreams again I should come back and say a bit more about that and and again I would say you know the sort of vision and ideas on a share is really not mine but as I think shared by large community including you know young Jeff Fenton yoshua bengio and many others that you hear from in the next couple weeks what about computers do with our data


right we want to talk with images and label them lock of audio listen to audio and do speech recognition have text and do stuff with text and it turns out that machine learning is our best shot and most of these applications today but it is very difficult to get these applications work right so while back i ôll some of my students at stanford to use like a state-of-the-art computer vision algorithm to to write the motorcycle detector and this was the resulting god and this is typical in computer vision right so um well even


though learning algorithms works each of those lines is like a you know six months to two years of work for a team of engineers and and we like these algorithms to be less work to build and also maybe perform better so let me start to explain some of these ideas using computer vision but and then I will talk about about audio and apply these albums other modalities as well so why is this problem hard right so obviously a motorcycle how on earth could a computer fail to recognize what this is zooming into small part of the


image zooming into whether little red square is where you and I see a motorcycle the computer sees this so the computer vision problem is to look at all those pixel intensity values and tell you that all those numbers represent the exhaust pipe of a motorcycle seems like you need a very complicated function to do that and how do we do this so machine learning you know machine learning guys like me say oh just feed the data to the learning algorithm and let the learning algorithm do its job right when I teach my machine


learning cause I draw pictures like this and this is just not how it works so let's pick a couple pixels and let's plot some examples right so take that image there and because pixel one is relatively dark and pixel two is relatively bright that image you know it has has that position in this figure now let's take a different example a different motorcycle image has a this has a bright two pixel one in the darker pixel too so that second image gets passed a different location and then let's do this for a few negative


examples as well no motorcycles and what you find is that if you plot a set of positive and negative motorcycle and non motorcycle images that your positive negative examples are extremely jumbled together and so if you feed this data to you know certainly a linear classifier it doesn't work so what is done is that the machine learning is that we want to be nice if you could come up with what's called a feature representation if you could write a piece of code that tells you does this image have handlebars in


it does this image have tires or wheels in it and if you could do that then your data looks more like this on the lower right and it then becomes easy much easier for say a linear classifier like a swivel vector machine the logistic regression to distinguish the motorcycles from the banal motor signals right but the story goes on so will this in this illustrative example or saying well whether we could write a piece of code to tell us that their handlebars and wheels but we don't actually know how to do that and so in computer vision


what is done is actually the following this is how people actually come up with features in computer vision this kind of notional illustrative example but this is what I wanna do when I take my motorcycle I make I'm going to detect edges at four different orientations so look for vertical edges horizontal edges 45-degree ages 135 degree edges and then what this number or point seven and the upper right means right that number is saying that the density of vertical edges in the upper right hand quadrant of my image is 0.7 and


what and what this number down here says is that the density of horizontal edges you know in the lower right hand quadrant of my image is 0.5 and in case you're getting the sense that sort of like oh my god what on earth is going on there this seems hardly complicated how interrupts it will come up with this piece of code you know that's that's that's the point and sadly this is the way that a lot of computer vision is done today so we'll probably this notion of a feature representation is pervasive throughout machine learning in fact I


guess I live in I live in Silicon Valley and if you walk around Silicon Valley and look at what where people are spending all the engineering time is often in coming up with these feature representations so let's let's look what's delve deeper right so where do these features come from um since they're the primary lens through which our algorithms see the world this gives them a certain importance right so how about computer vision oh and then facts are interesting and in fact this notion of feature representations is pervasive


you know for vision audio text even other applications so what do you get the features from in computer vision the state of the art answer for where the features come from is that teams of tens hundreds or teams of some of tens hundreds or maybe thousands of computer vision researchers have spent decades of their lives hand engineering features for computer vision the figure on the upper left is a figure that I took from the stiff paper the stiff paper is the single most highly cited paper in computer vision like


fifteen years and I read the paper maybe about five times now and I still have no idea what it's doing right this is something so complex piece of code that David loathes good friend David will tell you this himself it took David literally ten years I'm knocking he's he'll say 10 years himself of you know filling with pieces of the code in order to come up with the SIP feature which works pretty well but you know you have to ask is there a better way to design features than this that's vision how about audio same thing


right teams of tens hundreds of thousands of audio researchers working on features for audio ever CC is shown on the upper right that's actually a pretty clever surprisingly hard to beat but again you know honestly to this day I have a hard time understanding what some bits of the M FCC album are doing and natural language in fact I think most of natural language processing text processing today is unapologetically about finding better features so think about pauses right there's a lot of NLP work on parsers um this piece of


software that tells you where the noun phrases are in your sentence I mean why on earth do I care where the noun phrases are in my sentences I really don't need software to tell me that the only reason we spend so much time working on pauses is because we hope that this will give us useful features to then feed to some later downstream application like anti-spam web search machine translation that we actually care about so common features is difficult time consuming requires expert knowledge and we're working with them


applications machine learning you know we spent a long time there's a way if you look at apply machine learning where they will walk anew oh the company local people doing apply machine learning this is really what they spend the vast majority of the time on it's coming up with features so can we do better so the next piece in a lot of deep learning there's a like many people in very like many of you I tend to treat biological inspiration with a great deal of caution and even a healthy dose of skepticism but for me a


lot of the some of my thinking about deep learning has been taken inspiration from biology so I only share of you but you know some some cool cool ideas from really by washing inspiration so turns out there's a fascinating hypothesis that much of human intelligence can be explained by a single learning algorithm since cause the one learning our hypothesis let me share with you some evidence for this right so this one's experiment first done on ferrets in MIT on that very piece of brain tissue shown on the slide that's your auditory cortex


the way that your understanding my words now is that your ears is routing the sound signal to that piece of great brain tissue and this processing the sound and then that's how you you eventually get to understand what I'm saying so neuroscientists did the following experiments which is in cut the wire between the ears and the auditory cortex and do what's called a neural rewiring experiment so that eventually the signal from the eyes gets routed to the auditory cortex it turns out if you do this that great piece of


brain tissue learns to see and this is the word see this has been replicated in multiple labs on four species of animals and these animals can quote see in every single sense of the word that I know how to use the word see these animals they can do visual discrimination talks we can look at things and make correct decisions based on you know an image in front of them using that rare piece of brain tissue another example this rare piece of brain tissue is your somatosensory cortex is responsible for your sense of touch do a similar neural


rewiring experiment and your somatosensory cortex Lucy um so more generally this is idea that if the same physical piece of brain tissue right the same physical bit of your brain can process sight or sound or touch or maybe even other things then maybe there's a single learning algorithm they can process sight or sound or touch or maybe other things and that it can you know discover some approximation to that learning algorithm when we discover a totally different algorithm but it accomplishes the same thing then that might be a better way to


us making progress in AI then hand engineering separate pieces of code in each of these individual application silos which which we have been doing for decades now just a few more fun examples it turns out you can plug in other sensors to the brain and the brain kind of figures out how to do of it shown on the upper left is a scene with your tongue right so this is actually undergoing FDA trials of now to help blind people see a system called brain port so the way it works is you strap a camera as your forehead takes a


low-resolution grayscale image of what's in front of you run a wire to a rectangular array of electrodes that you place on top of the tongue so the each pixel maps to a point on your tongue and maybe a high voltage is a bright pixel and a low voltage is a dark pixel and even as adults you and I today would be able to learn to see you about tongues in like tens they only ten-thirty 10 20 minutes human echolocation well you need a you know snap your fingers right I'll click your tongue and there are there actually schools today training blind


children to learn to interpret the pattern of sounds bouncing off the environments as human sonar a haptic belt is you know bring a buzzes around your waist program the one facing off the bus and then you get the directions and you just magically know where North is or similar to how birds sense direction you can plug a third eye into frog and you know the farm learns how to how to deal with it it doesn't work in every single instance there are cases where this doesn't work but I think to a surprisingly large extent it's almost as


if you can plug in you know not quite any sensor but almost the large range of sensors onto almost any part of the brain kind of going to do of it so want to be cool if you get a learning algorithm to do the same so let's see oh let's take a break could you guys on I think you now know enough to work look at the questions one through three in the handout do you guys want to take a few minutes so just write down do write down what you think is the right answer and when you've done so you know discuss what you wrote down with


your neighbors and and see if you agree or disagree for question one I had d4 question two I had auditory cortex learns to see and the question three I don't know different people have different ideas I guess I tend to use the wording that much of human intelligence can be explained by a single learning algorithm but there are lots of other Worthing's that lots of other ways of distracting it alright so given this you know what are the implications for machine learning right so here if we think that without visual


system computes an incredibly complicated function of the input right it looks all those numbers and those pixel values and tells you that that's the motorcycle exhaust pipe and so two approaches that we could try to build such a system as you could try to directly implement this complicated function which is what I think of as a hand engineering approach or maybe you can try to learn this function instead right and in kind of a side comment maybe only for the aficionados the machine learning is that if you look at


a train learning algorithm you know a learning algorithm after has trained with all the parameter values there's a very complex thing but the learning algorithm itself is relatively simple most learning algorithms can be described in like half a page of pseudocode so the complexity of the things we're training usually comes from the complexity of the data rather than the complexity of the algorithm and then that's a good thing because we know how to get complex data you just year which is an images or around us but coming up with complex


algorithms is hot right so here's a here's a problem that I guess I post a few years ago which is you know can we learn a better feature representation for vision or audio or what have you so concretely can you come up with an algorithm they just examine examine it's a bunch of images like these and automatically comes up with a better way to represent images than the raw pixels and if you can do that maybe you can apply the same algorithm to audio and have the same algorithm trained along a bunch of audio clips and have it find a


better way to represent audio than the raw data okay so let's let's write down the mathematical formalism of this problem right which is given a 14 by 14 image X image patch X one way to represent the image patch is with a list of 196 row numbers corresponding to the pixel intensity values the probably one opposes can we come up with a better feature vector to represent those pixels okay and if you can do so then this is what you can do here's a problem called a self-taught learning I guess which is well so in in traditional machine


learning right if you want to learn to distinguish your motorcycles from non motorcycles you have a training set with some and this is a pain because there's a lot of work to come up with a lot of pictures and motorcycles it was like tens of thousands of them so in the unsupervised feature learning on the self or learning problem what you do is instead we're going to give you a large source of unlabeled images then give you an infinite source of unlabeled images because of the web where we all have an effectively infinite source of images


and the task is can all those random images up there somehow can pictures of trees and sunsets and horses and so on can that help you to do a better job figuring out that this picture down here is a mobile cycle okay and so one way to do that is that we have an algorithm that can look at these on label images and learn a much better representation of images than just the raw pixels and if that superior representation allows us to then look at a small label training set and this my superior representation allows us to use


the small label training set to do a much better job figuring out what this tested images okay so I guess in machine learning there are sort of three standard three and a few common formalisms right there's the supervised learning setting which is the olders will standard one that most you best know so let's see it goes to record distinguish between cars and motorcycles right so the standard Nero school like 50 year old supervised learning setting 30 year old supervisor is I think you need to come a large training set up a


bottle cause a lot of motorcycles okay Oh about 10 15 years ago deciding Andrew McCollum Tom Mitchell or maybe even others before them start to talk about semi-supervised learning the idea of using unlabeled data and that was exciting but in semi-supervised learning as is simply conceived um you know the ability to use unlabeled data is great but in semi supervised learning typically the unlabeled data is all images or causing over cycles and it turns out that this is a semi-supervised learning this sort of model is not


widely used because it turns out that um you know rarely do you have a data set of where all the images are either caused or motorcycles and nothing else but the only thing we're doing is just missing the label so this is kind of useful but isn't why they use where as in um what I call self-taught learning the goal is to take you know totally random images that may be caused may be old cycles may be totally other random things and to can you can use somehow use this to learn to distinguish the miles and many cycles and one weight is


what would I like to think about it is that you know the first time that a child sees a new object where someone invents a new vehicle right the first time that you and I saw a Segway we learn to recognize the Segway very quickly just from seeing it once and I think the reason that we learn to recognize a Segway very quickly is because you're in my visual system prior to that had had several decades of experience looking at random natural images just seeing the world and was by looking at these random unlabeled images


that allow us that allowed you in my visual system to learn enough about the structure of the world to come up with better features if you will so that the first time you saw a Segway you very quickly learn to recognize what a Segway is right so just to make sure you've got this concept could you please a look at question four and just do that map this to a new example and so someone called the answer what first first part a PCOD second part and third part all right also all right that was easy cool so how do you actually do this in


order to come up with an algorithm to learn features let's turn one last time to biological motivation turns out that when your brain gets an image the first thing it does is look for edges in the image right so first stage of visual processing on the brain is called visual cortical area view what I think y'all might have mentioned is yesterday and the first thing it does is look for edges or lines I'm going to use the term lines and edges interchangeably so in your brain right now there's probably a neuron that is looking for a 45 degree


line 45-degree edge like this shown on the left with the dark region next to a bright region and there's probably a different neuron in your brain right now there's looking for a vertical line like this one right here okay so um how can we get our software to maybe mimic the brain and also find edges like this what we don't want to do is code this up by hand because you know what I don't want to do is tell us the neuroscientist and then work really hard to right hand engineer software the replica I think what's most much more


interesting is we can have an algorithm learn these things by itself and that there is such an algorithm very old ones like a what 16 year old result now do 200 thousand a few called sparse coding you talked about this a bit yesterday did you write cool so now go through this very quickly even a small coding was obviously conceived as like a theoretical neuroscience model so you know Brunello thousand will tell you right he never envisioned that this would be used as a machine learning algorithm this is like a theoretical


neuroscience result used to try to explain you know computations in the brain or something like that right and this is how the atom works is an unsupervised learning algorithm so the way where else is you feed it a set of m images X 1 X 2 up to X M so each input example is and let's say an N by n matrix yo quicker like a 14 by 14 image patch what's fast coding does is it learns the dictionary of basis functions Phi 1 Phi 2 up to 5 K such that each of your training images X subject to the constraint that the a J's are mostly 0


sparse and then the way this is implemented is on with a l1 constraint where both work you know minimize the sum of absolute value terms on the on their coefficients AJ okay the sparsity penalty term and so if you do this then on by the way I think this is the only equation I have for this first hour so I hope you enjoyed it so same thing in pictures if you train sparse coding on unnatural images every single time you're on it they'll learn the set of basis functions that look a lot like the edge detectors that you know we believe visual protocol


error view one is looking for and then given a test example give it a test image X right what what what they will do is it will select out let's say three out of my 64 basis functions here and it will take that test example and explain it or decompose it into a linear combination of in this case just three out of 64 of my basis functions okay so speaking loosely this algorithm has to quote invented edge detection right the algorithm is free to choose absolutely any basis functions at once but it shows but you know if you run it


every time it chooses to learn basis functions that look like these edges and what this decomposition says is that this image X is 0.8 times H number 36 plus 0.3 times H number 42 plus 0.5 times H number 63 okay so if you will this is saying this is now decompose the image in terms of what edges appear in this image and this gives a high-level more succinct more compact representation of the image and also probably a more useful one right because it's more useful to know where the edges are in the image that you know where the


pixels are moreover this gives us a alternative way to represent the image instead of representing the image patch using a list of 196 pixel values we can instead using this vector of numbers a 1 through a 64 these are the coefficients multiplying into the basis functions just a few more examples so the method in density attention those represent an image in terms a just appear in it and it turns out that a funeral scientists have done that had to invent a chef have done quantitative comparisons between sparse coding and oh and visual cortical


area v1 and found that you know it's not as by no means a perfect explanation of visual to everyone but but it matches of unknown so surprisingly well on not all but on many dimensions so that's vision power other input modalities so this is a slide I got from Evan Smith from his PhD thesis work with Michael Ricky so what Evan did was he appoints false coding to audio data and what I've shown here is 20 basis functions learned by sparse coding when trained on natural sounds ok so this is a grid of 5 by 4 you know what 5 by 4 the lower audio


clips I guess audio basis functions so this 20 basis functions learn by sparse coding um what he did was he then went to the cat or the tree system since the biologists in Boston had been you know using electro recordings to figure out what early auditory processing in a cat does and for each of these 20 things learned by his algorithm he found the closest match in the biological data and the closest matches are shown over the in red ok so the same algorithm that only one hand gives a you know he's an explanation for early


visual processing and on the other hand use may be a you know by no means perfect but but the reason why X which is some explanation for review of it auditory processing as well and it turns out you can do a similar study on them early somatosensory processing as well this is work done by Andrew Sacco Stanford where he collected touch data how do you call it touch data right so the way that done Andrew Andrew Sachs did it was um you know so we hold things of our hands all the time when I'm holding this thing but


how do you how do you actually collect data for how I'm holding it so the way Andrew Sachs did it was um he took a cloth and he took an object and he sprayed talcum powder all over the object and then when you take a glove and you hope this object and then you let go the pattern of talcum powder you know on your glove tells you where you came into contact with the object and moreover the density of talcum powder actually corresponds a little bit to the to the pressure and and he not sure why he did this but um he didn't actually


found so so what type of objects do people hold or I won't we don't know so we're collecting data you want to be representative of what animals do so fortunately it turns out that there were two biologists that has spent about a year of their lives sitting on some Island watching monkeys and carefully documenting every single way that monkeys pick up different things so thank God I'm up computer science is right and so Andrew Sachs you know took that distribution of data and he wearing his glove picked up objects using the


same distribution of drawers as was documented in these monkeys on an island oh and that was his data um I think that story was pretty fun but totally unnecessary oh but but Carlyle sorry showing a zero so training training on data like this that turns out that you learn basic functions using sparse coding there are I should say by no means a perfect match to one to what is known to what is believed to happen in somatosensory cortex but this may be a surprisingly good match dimensions right so that's fast coding


um and let me could you take could you do a five and six on the handle so it was the answer four five and six wait what's wait once what is must actually where was it four six again okay come on all right be right all right cool let success on the same for every image but is the coefficients they want to vacate the look features for the specific image okay um all right great so that's sparse coding and it turns out that on the different ways to implement you know sparse coding and what what I just talked about was maybe the the


original way biocells and view in 1986 they're much route there there are different ways now I think young mother talked about encoded decode architectures I'll talk a bit more about that later today but I think this intuition of learning sparse features has been kind of key for is one of the ideas I guess that allows us to learn very useful features even from unlabeled data come back to this later as well there are there other ways to do it if any of you are familiar with ICA actually how many of you have heard of


the ICA independent components out oh cool all of you awesome so it turns out that there's a you know the deep mathematical relationship between ICA and sparse coding with the turns out the two algorithms are doing something very similar for me personally these days I tend to use the ICA version so sparse coding rather than the version I just talked about but later today also talked about about sparse altering colors different ways of learning sparse features we'll get to that later but so what do you do instead of your we just


described is you know one layer of one of these spas feature learning algorithms maybe sparse coding may be as possible to encoder may be a sparse VPN or sparse RBM and turns out what you can do really building on a jet engines work what you can do is recursively apply this procedure where instead of just going from pixels to images excuse me pixels to edges you can recursively apply this procedure and you know just as you can group together pixels to form agency can group together edges to form combinations of edges and group together


combinations of edges to form higher level features so let me show an example that this is an example run by Hall actually it's not a michigan professor but what huh laughs did was she trained one air of virtuous paws dbn and first layer you know Avram mercy group together pixels for mages another level up learn significant edges to form models of object parts right so this I should say this was an example of sparse coding train just on pictures of faces so the entire dataset was pictures of faces right and then I'm requesting you apply


this the next level up and doing some more complete models of faces so let me make sure that this visualization makes sense right um when I have this little square here shown here what this little red tag what does little square means is that I have learned a neuron in the first level that is looking for a vertical edge like that one okay going one level up and and and I've shown all these rectangles the same size but higher up features are actually looking at bigger regions of the image okay it's just a resize all of the same but one


level up this is actually looking at the bigger vision of the image but one level up you know this rectangle here means that at that next level one of the neurons has learned to detect eyes that look like that great and then the highest level you know if you look at the upper leftmost square say with that visualization is showing that there's a neuron that has learned to detect faces that you look local bit like that person okay if you train the same algorithm on different object classes you end up with different decompositions of different


option classes into different object parts then more complete models of objects if you train the algorithm on a mix of four different classes of objects so there's an algorithm trained on a data set that includes cars faces bikes and airplanes then you know you end up with at the mid level you get features that are shared among the different object classes where I don't know maybe what I guess are new cars and motorbikes both have real tire like shapes or your features that kind of shared between multiple object parts and then the


highest level you get object specific features okay yeah is there any sort of variance in see yes there is there's a point so because of the nature of the visualization I showed them as though images but yeah it it there's some amounts of Indians that's hard to visualize it oh I remember I have a better example later today of a where where we more carefully document the invariant zones have a better example later okay so was this good for when you when when you hear your research isn't deep learning like me talk you you see


people like yawn in me and Jeff intern is hello stories IQs but you know so you can learn features so what is it good for well it turns out the Hollywood to benchmark could stand the benchmark in computer vision where the task is to watch a short video clip and decide whether you know any of a small number of activities took place in this video you know whether two people kids to do hugs almost driving so as eating's or was running the visor activities like the theater computer vision has tried out many different combinations that


features last year probably soon as standard found that by learning rather than hand engineering the features he was able to significantly outperform the previous AVR all right how about audio it turns out you can apply similar ideas to audio so this is a spectrogram which is a different representation for audio you can take slices of spectrograms and apply sparse coding to that it turns out if you do this then on this is a dictionary of basis functions learning for speech I guess I'm not an excellent speech but in impervia probably a


slightly optimistic as a reading of these you know the basis functions learn by sparse coding correspond roughly to phonemes there's a slightly optimistic interpretation I should say and so but if under this slightly optimistic interpretation when say informally that sparse coding has learned to decompose speech data you know very loosely into it because the phonemes to the appearance speech and moreover you can take this recursively apply this idea just as we saw earlier to build higher and higher level features and I guess a few years ago