special is that brilliant insight yeah that can create something totally new yes so now let's look at the extreme case let's look at the set of all possible problems that you can formally describe which is infinite which should be the next problem that a scientist or power-play is going to solve well it should be the easiest problem that goes beyond what you already know


so it should be the simplest problem that the current problems all of that you have which can already sold 100 problems that he cannot solve yet by just generalizing so it has to be new so it has to require a modification of the problem solver such that the new problem solver can solve this new thing but the old problem solver cannot do it and in addition to that we have to make

Juergen Schmidhuber: Godel Machines, Meta-Learning, and LSTMs | Lex Fridman Podcast #11


sure that the problem solver doesn't forget any of the previous solutions right and so by definition power play is now trying always to search and this pair of in in the set of pairs of problems and problems over modifications for a combination that minimize the time to achieve these criteria so as always trying to find the problem which is easiest to add to the repertoire so just


like grad students and academics and researchers can spend the whole career in a local minima hmm stuck trying to come up with interesting questions but ultimately doing very little do you think it's easy well in this approach of looking for the simplest unsolvable problem to get stuck in a local minima is not never really discovering new you know really jumping


outside of the hundred problems the very solved in a genuine creative way no because that's the nature of power play that it's always trying to break its current generalization abilities by coming up with a new problem which is beyond the current horizon just shifting the horizon of knowledge a little bit out there breaking the existing rules search says the new thing becomes


solvable but wasn't solvable by the old thing so like adding a new axiom like what Google did when he came up with these new sentences new theorems that didn't have a proof in the phone system which means you can add them to the repertoire hoping that that they are not going to damage the consistency of the whole thing so in the paper with the amazing title formal theory of creativity fun in


intrinsic motivation you talk about discovery as intrinsic reward so if you view humans as intelligent agents what do you think is the purpose and meaning of life far as humans is you've talked about this discovery do you see humans as an instance of power play agents yeah so humans are curious and that means they behave like scientists not only the official scientists but


even the babies behave like scientists and they play around with toys to figure out how the world works and how it is responding to their actions and that's how they learn about gravity and everything and yeah in 1990 we had the first systems like the hand would just try to to play around with the environment and come up with situations that go beyond what they knew at that


time and then get a reward for creating these situations and then becoming more general problem solvers and being able to understand more of the world so yeah I think in principle that that that curiosity strategy or sophisticated versions of whether chess is quiet they are what we have built-in as well because evolution discovered that's a good way of exploring the unknown world


and a guy who explores the unknown world has a higher chance of solving problems that he needs to survive in this world on the other hand those guys who were too curious they were weeded out as well so you have to find this trade-off evolution found a certain trade-off apparently in our society there are as a certain percentage of extremely exploitive guy and it doesn't matter if they die


because many of the others are more conservative and and and so yeah it would be surprising to me if if that principle of artificial curiosity wouldn't be present and almost exactly the same form here in our brains so you're a bit of a musician and an artist so continuing on this topic of creativity what do you think is the role of creativity and intelligence so you've


kind of implied that it's essential for intelligence if you think of intelligence as a problem-solving system as ability to solve problems but do you think it's essential this idea of creativity we never have a program a sub program that is called creativity or something it's just a side effect of when our problem solvers do they are searching a space of problems or a space


of candidates of solution candidates until they hopefully find a solution to have given from them but then there are these two types of creativity and both of them are now present in our machines the first one has been around for a long time which is human gives problem to machine machine tries to find a solution to that and this has been happening for many decades and for many decades


machines have found creative solutions to interesting problems where humans were not aware of these particularly in creative solutions but then appreciated that the machine found that the second is the pure creativity that I would call what I just mentioned I would call the applied creativity like applied art where somebody tells you now make a nice picture off of this Pope and you will


get money for that okay so here is the artist and he makes a convincing picture of the Pope and the Pope likes it and gives him the money and then there is the pure creative creativity which is more like the power play and the artificial curiosity thing where you have the freedom to select your own problem like a scientist who defines his own question to study and so


that is the pure creativity of UL and opposed to the applied creativity which serves another and in that distinction there's almost echoes of narrow AI versus general AI so this kind of constrained painting of a pope seems like the the approaches of what people are calling narrow AI and pure creativity seems to be maybe I'm just biased as a human but it seems to be an


essential element of human level intelligence is that what you're implying to a degree if you zoom back a little bit and you just look at a general problem-solving machine which is trying to solve arbitrary problems then this machine will figure out in the course of solving problems that it's good to be curious so all of what I said just now about this prewired curiosity and this


will to invent new problems that the system doesn't know how to solve yet should be just a byproduct of the general search however apparently evolution has built it into us because it turned out to be so successful a pre-wiring a buyer's a very successful exploratory buyers that that we are born with and you've also said that consciousness in the same kind of way


may be a byproduct of problem-solving you know do you think do you find it's an interesting by-product you think it's a useful by-product what are your thoughts on consciousness in general or is it simply a byproduct of greater and greater capabilities of problem-solving that's that's similar to creativity in that sense yeah we never have a procedure called consciousness in our


machines however we get as side effects of what these machines are doing things that seem to be closely related to what people call consciousness so for example in 1990 we had simple systems which were basically recurrent networks and therefore universal computers trying to map incoming data into actions that lead to success maximizing reward in a given environment


always finding the charging station in time whenever the battery's low and negative signals are coming from the battery always finds the charging station in time without bumping against painful obstacles on the way so complicated things but very easily motivated and then we give these little a separate we can all network which is just predicting what's happening if I do


that in that what will happen as a consequence of these actions that I'm executing and it's just trained on the long and long history of interactions with the world so it becomes a predictive model loss of art basically and therefore also a compressor our theme observations after what because whatever you can predict you don't have to store extras or compression is a side


effect of prediction and how does this record Network impress well it's inventing little sub programs little sub Network networks that stand for everything that frequently appears in the environment like bottles and microphones and faces maybe lots of faces in my environment so I'm learning to create something like a prototype face and a new face comes along and all


I have to encode are the deviations from the prototype so it's compressing all the time the stuff that frequently appears there's one thing that appears all the time that is present all the time when the agent is interacting with its environment which is the agent itself so just for data compression reasons it is extremely natural for this we can network to come up with little sub


networks that stand for the properties of the agents the hand you know the the other actuators and all the stuff that you need to better encode the data which is influenced by the actions of the agent so they're just as a side effect of data compression during problem-solving you have inter myself models now you can use this model of the world to plan your future and that's what yours have done


since 1990 so the recurrent Network which is the controller which is trying to maximize reward can use this model as a network of the what is this model network as a wild this predictive model of the world to plan ahead and say let's not do this action sequence let's do this action sequence instead because it leads to more predictor to rewards and whenever it's waking up these layers of


networks let's stand for itself and it's thinking about itself and it's thinking about itself and it's exploring mentally the consequences of its own actions and and now you tell me what is still missing missing the next the gap to consciousness yeah hi there there isn't that's a really beautiful idea that you know if life is a collection of data and in life is a process of compressing that


data to act efficiently you in that data you yourself appear very often so it's useful to form compressions of yourself and it's a really beautiful formulation of what consciousness is a necessary side-effect it's actually quite compelling to me you've described our nen's developed LST aims long short-term memory networks the there type of recurrent neural networks they have


gotten a lot of success recently so these are networks that model the temporal aspects in the data temporal patterns in the data and you've called them the deepest of the Newell networks right so what do you think is the value of depth in the models that we use to learn since you mentioned the long short-term memory and the lsdm I have to mention the names of the brilliant


students of course that's worse first of all and my first student ever set for writer who had fundamental insights already in this diploma thesis then Felix Kias had additional important contributions Alex gray is a guy from Scotland who is mostly responsible for this CTC algorithm which is now often used to to train the Alice TM to do the speech recognition on all the Google Android


phones and whatever and Siri and so on so these guys without these guys I would be nothing it's a lot of incredible work what is now the depth what is the importance of depth well most problems in the real world are deep in the sense that the current input doesn't tell you all you need to know about the environment mm-hmm so instead you have to have a memory of what


happened in the past and often important parts of that memory are dated they are pretty old and so when you're doing speech recognition for example and somebody says eleven then that's about half a second or something like that which means it's already fifty-eight time steps and another guy or the same guy says seven so the ending is the same Evan but now the system has to see the


distinction between seven and eleven and the only way I can see the differences it has to store that fifty steps ago there wasn't or a nerve eleven or seven so there you have already a problem of depth fifty because for each time step you have something like a virtual a layer and the expanded unrolled version of this Riccar network which is doing the speech recognition so these long


time lags they translate into problem depth and most problems and this world Asajj that you really have to look far back in time to understand what is the problem and to solvent but just like with our CMS you don't necessarily need to when you look back in time remember every aspect you just need to remember the important aspects that's right the network has to


learn to put the important stuff in into memory and to ignore the unimportant noise so but in that sense deeper and deeper is better or is there a limitation is is there I mean LCM is one of the great examples of architectures that do something beyond just deeper and deeper networks there's clever mechanisms for filtering data for remembering and forgetting so


do you think that that kind of thinking is necessary if you think about LCM is a leap a big leap forward over traditional vanilla are nuns what do you think is the next leap hmm it within this context so LCM is a very clever improvement but LCM still don't have the same kind of ability to see far back in the future in the in the past as us humans do the credit assignment problem across way


back not just 50 times steps or a hundred or a thousand but millions and billions it's not clear what are the practical limits of the lsdm when it comes to looking back already in 2006 I think we had examples where it not only looked back tens of thousands of steps but really millions of steps and who won Paris artists in my lab I think was the first author of a paper where we really


was a 2006 or something had examples word learn to look back for more than 10 million steps so for most problems of speech recognition it's not necessary to look that far back but there are examples where it does now so looking back thing [Music] that's rather easy because there is only one past but there are many possible futures and so a reinforcement learning


system which is trying to maximize its future expected rewards and doesn't know yet which of these many possible future should I select given this one single past it's facing problems that the LCN by itself cannot solve so the other sim is good for coming up with a compact representation of the history so far of the history and observations in action so far but now how do you plan in an


efficient and good way among all these how do you select one of these many possible action sequences that a reinforcement learning system has to consider to maximize reward in this unknown future so again it behaves this basic setup where you have one week on network which gets in the video and the speech and whatever and it's executing actions and is trying to maximize reward


so there is no teacher who tells it what to do at which point in time and then there's the other network which is just predicting what's going to happen if I do that then and that could be an LCM Network and it allows to look back all the way to make better predictions of the next time step so essentially although it's men predicting only the next time step it is motivated to learn


to put into memory something that happened maybe a million steps ago because it's important to memorize that if you want to predict that at the next time step the next event you know how can a model of the world like that a predictive model of the world be used by the first guy let's call it the controller and the model the controller and the model how can the model be used


by the controller to efficiently select among these many possible futures so naive way we had about 30 years ago was let's just use the model of the world as a stand-in as a simulation of the wall and millisecond by millisecond we planned the future and that means we have to roll it out really in detail and it will work only as the model is really good and it will still be inefficient


because we have to look at all these possible futures and and there are so many of them so instead what we do now since 2015 and our cm systems controller model systems we give the controller the opportunity to learn by itself how to use the potentially relevant parts of the M of the model network to solve new problems more quickly and if it wants to it can learn to ignore the M and


sometimes it's a good idea to ignore the the M because it's really bad it's a bad predictor in this particular situation of life where the control is currently trying to maximize r1 however it can also allow and to address and exploit some of the sub programs that came about in the model network through compressing the data by predicting it so it now has an opportunity to reuse that code the


ethnic information in the modern are trying to reduce its own search space such that it can solve a new problem more quickly than without the model compression so you're ultimately optimistic and excited about the power of ära of reinforcement learning in the context of real systems absolutely yeah so you see RL as a potential having a huge impact beyond just sort of the M


part is often develop on supervised learning methods you see RL as a four problems of cell traffic cars or any kind of applied cyber BOTS X that's the correct interesting direction for research in your view I do think so we have a company called Mason's Mason's which has applied to enforcement learning to little Howdy's there are DS which learn to park without


a teacher the same principles were used of course so these little Audi's they are small maybe like that so I'm much smaller than the real Howdy's but they have all the sensors that you find the real howdy is you find the cameras that lead on sensors they go up to 120 20 kilometres an hour if you if they want to and and they are from pain sensors basically and they don't want to bump


against obstacles and other Howdy's and so they must learn like little babies to a park take the wrong vision input and translate that into actions that lead to successful packing behavior which is a rewarding thing and yes they learn that they are salt we have examples like that and it's only in the beginning this is just the tip of the iceberg and I believe the next wave of a line is going


to be all about that so at the moment the current wave of AI is about passive pattern observation and prediction and and that's what you have on your smartphone and what the major companies on the Pacific of em are using to sell you ads to do marketing that's the current sort of profit in AI and that's only one or two percent of the world economy which is big enough to make


these company is pretty much the most valuable companies in the world but there's a much much bigger fraction of the economy going to be affected by the next wave which is really about machines that shape the data through our own actions and you think simulation is ultimately the biggest way that that though those methods will be successful in the next 10 20 years we're not


talking about a hundred years from now we're talking about sort of the near-term impact of RL do you think really good simulation is required or is there other techniques like imitation learning you know observing other humans yeah operating in the real world where do you think this success will come from so at the moment we have a tendency of using physics simulations to learn


behavior for machines that learn to solve problems that humans also do not know how to solve however this is not the future because the future is and what little babies do they don't use a physics engine to simulate the world no they learn a predictive model of the world which maybe sometimes is wrong in many ways but captures all kinds of important abstract high-level


predictions which are really important to be successful and and that's what is what was the future thirty years ago when you started that type of research but it's still the future and now we are know much better how to go there to to move there to move forward and to really make working systems based on that where you have a learning model of the world a model of the world that learns to


predict what's going to happen if I do that and that and then the controller uses that model to more quickly learn successful action sequences and then of course always this crazy thing in the beginning the model is stupid so the controller should be motivated to come up with experiments with action sequences that lead to data that improve the model do you think


improving the model constructing an understanding of the world in this connection is the in now the popular approaches have been successful you know grounded in ideas of neural networks but in the 80s with expert systems there's symbolic AI approaches which to us humans are more intuitive in a sense that it makes sense that you build up knowledge in this knowledge


representation what kind of lessons can we draw in our current approaches mmm for from expert systems from symbolic yeah so I became aware of all of that in the 80s and back then a logic program logic programming was a huge thing was inspiring to yourself did you find it compelling because most a lot of your work was not so much in that realm mary is more in learning systems yes or no


but we did all of that so we my first publication ever actually was 1987 was a the implementation of genetic algorithm of a genetic programming system in prologue prologue that's what you learn back then which is a logic programming language and the Japanese the anthers huge fifth-generation AI project which was mostly about logic programming back then although a neural networks existed


and were well known back then and deep learning has existed since 1965 since this guy and the UK and even anko started it but the Japanese and many other people they focus really on this logic programming and I was influenced to the extent that I said okay let's take these biologically inspired rules like evolution programs and and and implement that in the language which I know which


was Prolog for example back then and then in in many ways as came back later because the Garuda machine for example has approved search on board and without that it would not be optimal well Marcus what does universal algorithm for solving all well-defined problems as approved search on board so that's very much logic programming without that it would not be a Centanni optimum but then


on the other hand because we have a very pragmatic is also we focused on we cannula networks and and and some optimal stuff such as gradient based search and program space rather than provably optimal things the logic programming does it certainly has a usefulness in when you're trying to construct something provably optimal or probably good or something like that but


is it useful for for practical problems it's really useful at volunteer improving the best theorem provers today are not neural networks right no say our logic programming systems and they are much better theorem provers than most math students and the first or second semester on but for reasoning to for playing games of go or chess or for robots autonomous vehicles that operate


in the real world or object manipulation you know you think learning yeah as long as the problems have little to do with with C or improving themselves then as long as that is not the case you you just want to have better pattern recognition so to build a self-driving car you want to have better pattern recognition and and pedestrian recognition and all these


things and you want to your minimum you want to minimize the number of false positives which is currently is slowing down self-driving cars in many ways and and all that has very little to do with logic programming yeah what are you most excited about in terms of directions of artificial intelligence at this moment in the next few years in your own research and in the broader community so


I think in the not so distant future we will have for the first time little robots that learn like kids and I will be able to say to the robot um look here robot we are going to assemble a smartphone it's takes a slab of plastic and the school driver and let's screw in the screw like that no no not like that like so hmm not like that like that and I don't have a data glove or something


he will see me and he will hear me and he will try to do something with his own actuators which will be really different from mine but he will understand the difference and will learn to imitate me but not in the supervised way where a teacher is giving target signals for all his muscles all the time no by doing this high level imitation where he first has to learn to imitate


me and then to interpret these additional noises coming from my mouth as helping helpful signals to to do that Hannah and then it will by itself come up with faster ways and more efficient ways of doing the same thing and finally I stopped his learning algorithm and make a million copies and sell it and so at the moment this is not possible but we already see how we are going to get


there and you can imagine to the extent that this works economically and cheaply it's going to change everything almost all our production is going to be affected by that and a much bigger wave much bigger ai wave is coming than the one that we are currently witnessing which is mostly about passive pattern recognition on your smartphone this is about active machines that shapes data


Susy actions they are executing and they learn to do that in a good way so many of the traditional industries are going to be affected by that all the companies that are building machines well equip these machines with cameras and other sensors and they are going to learn to solve all kinds of problems through interaction with humans but also a lot on their own to improve what they


already can do and lots of old economy is going to be affected by that and in recent years I have seen that all the economy is actually waking up and realizing that those vacations and are you optimistic about the future are you concerned there's a lot of people concerned in the near term about the transformation of the nature of work the kind of ideas that you just suggested


would have a significant impact of what kind of things could be automated are you optimistic about that future are you nervous about that future and looking a little bit farther into the future there's people like you la musk - a rustle concerned about the existential threats of that future so in the near term job loss in the long term existential threat are these concerns to


you or yalta mele optimistic so let's first address the near future we have had predictions of job losses for many decades for example when industrial robots came along many people many people predicted and lots of jobs are going to get lost and in a sense say were right because back then there were car factories and hundreds of people and these factories assembled cars and today


the same car factories have hundreds of robots and maybe three guys watching the robots on the other hand those countries that have lots of robots per capita Japan Korea and Germany Switzerland a couple of other countries they have really low unemployment rates somehow all kinds of new jobs were created back then nobody anticipated those jobs and decades ago I already


said it's really easy to say which jobs are going to get lost but it's really hard to predict the new ones 30 years ago who would have predicted all these people making money as YouTube bloggers 200 years ago 60% of all people used to work in agriculture today maybe 1% but still only I don't know 5% unemployment lots of new jobs were created and Homo Luden's the the playing man is inventing


new jobs all the time most of these jobs are not existentially necessary for the survival of our species there are only very few existentially necessary jobs such as farming and building houses and and warming up the houses but less than 10% of the population is doing that and most of these newly invented jobs are about interacting with other people in new ways through new media and so on


getting new high types of kudos and forms of likes and whatever and even making money through that so homo Luden's the playing man doesn't want to be unemployed and that's why he is inventing new jobs all the time and he keeps considering these jobs as really important and is investing a lot of energy and hours of work into into those and new jobs it's quite beautifully put were really


nervous about the future because we can't predict what kind of new jobs would be created but your ultimate ly optimistic that we humans are so Restless that we create and give meaning to newer in your jobs telling you likes on faith things that get likes on Facebook or whatever the social platform is so what about long-term existential threat of AI where our whole civilization may be swallowed


up by this ultra super intelligent systems maybe it's not going to be smaller DUP but I'd be surprised if B were B humans were the last step and the evolution of the universe you you've actually at this beautiful comment somewhere that I've seen saying that artificial quite insightful artificial general intelligence systems just like us humans will likely not want to


interact with humans they'll just interact amongst themselves just like ants interact amongst themselves and only tangentially interact with humans hmm and it's quite an interesting idea that once we create a GI that will lose interest in humans and and have compete for their own Facebook Likes on their own social platforms so within that quite elegant idea how do we know in a hypothetical


sense that there's not already intelligent systems out there how do you think broadly of general intelligence greater than us how do we know it's out there mmm how would we know it's around us and could it already be I'd be surprised even with within the next few decades or something like that we we won't have a eyes that truly smarts in every single way and better problem


solvers and almost every single important way and I'd be surprised as they wouldn't realize what we have realized a long time ago which is that almost all physical resources are not here and this biosphere but for thou the rest of the solar system gets 2 billion times more solar energy than our little planet there's lots of material out there that you can use to build


robots and self-replicating robot factories and all this stuff and they are going to do that and there will be scientists and curious and they will explore what they can do and in the beginning they will be fascinated by life and by their own origins and our civilization they will want to understand that completely just like people today would like to understand


how life works and um and also the history of our own existence and civilization and also on the physical laws that created all of that so they in the beginning they will be fascinated my life once they understand that I was interest like anybody who loses interest and things he understands and then as you said the most interesting sources information for them will be others of


their own kind so at least in the long run there seems to be some sort of protection through lack of interest on the other side and now it seems also clear as far as we understand physics you need matter and energy to compute and to build more robots and infrastructure and more AI civilization and III ecology is consisting of trillions of different types of AIS and and so it seems


inconceivable to me that this thing is not going to expand some AI ecology not controlled by one AI but one by trillions of different types of AI is competing and all kinds of quickly evolving and disappearing ecological niches in ways that we cannot fathom at the moment but it's going to expand limited by Lightspeed and physics it's going to expand and and now we realize


that the universe is still young it's only 13.8 billion years old and it's going to be a thousand times older than that so there's plenty of time to conquer the entire universe and to fill it with intelligence and senders and receivers such that AI scan trouble the way they are traveling in our labs today which is by radio from sender to receiver and let's call the current age


of the universe one Eon one Eon now it will take just a few eons from now and the entire visible universe is going to be full of that stuff and let's look ahead to a time when the universe is going to be one thousand times older than it is now they will look back and they will say look almost immediately after the Big Bang only a few eons later the entire universe started to become intelligent


now to your question how do we see whether anything like that has already happened or is already in a more advanced stage in some other part of the universe of the visible universe we are trying to look out there and nothing like that has happened so far or is that her do you think we'll recognize it or how do we know it's not among us how do we know planets aren't in themselves


intelligent beings how do we know ants seen as a collective are not much greater intelligence in our own these kinds of ideas no but it was a boy I was thinking about these things and I thought hmm maybe it has already happened because back then I know I knew I learned from popular physics books that the structure the large-scale structure of the universe is not


homogeneous and you have these clusters of galaxies and then in between there are these huge empty spaces and I thought hmm maybe they aren't really empty it's just that in the middle of that some AI civilization already has expanded and then has covered a bottle of a billion light-years diameter and is using all the energy of all the stars within that bubble for its own


unfathomable purposes and so it always happened and we just failed to interpret the signs but then alarmed effect gravity by itself explains the large-scale structure of the universe and that this is not a convincing explanation and then I thought maybe maybe it's the dark matter because as far as we know today 80% of the measurable matter is invisible and we


know that because otherwise our galaxy or other galaxies would fall apart they would they are rotating too quickly and then the idea was maybe all us he is AI civilizations and hourly out there they they just invisible because they are really efficient in using the energies at their own local systems and that's why they appear dark to us but this is awesome at a convincing explanation


because then the question becomes why is there are there still any visible stars left in our own galaxy which also must have a lot of dark matter so that is also not a convincing thing and today I like to think it's quite plausible that maybe are the first at least in our local light cone within a few hundreds of millions of light years that we can reliably observe is there exciting to


hi there what you're seeing here is an energy based model that learns the concept of a shape from a demonstration on the left so on the left you can see a demonstration of data point sampled from a shape in these cases circles or squares and then the corresponding energy function that the model in first from that and then it can replicate that shape on the right using that energy

Concept Learning with Energy-Based Models (Paper Explained)


function so the paper we're going to analyze today is called concept learning with energy based models by yegor more Dutch of open AI and this is a very cool paper or at least I think it's a very cool paper but it is also a very hard paper so therefore first I want to kind of make a bit of an introduction into the concepts that we are facing in this paper so the first thing you need to


know are energy functions or energy based models what is an energy function an energy function sometimes called e is simply a function with one or multiple inputs let's call them X and you can make the if the energy function is happy with X it will be the value 0 and if the energy function is not happy with X it will be a high value like larger than zero so this is happy this is not happy


so let's give some examples of this we can formulate almost any machine learning problem in terms of an energy function let's say we have a classifier the classifier is takes as an input image here may be of a cat and the label so if the label is cat then the energy will be zero if the energy function is of course working correctly and if but if we give the energy function the same


image but we give it a wrong label dog then it is very high in the case of the classifier of course we can to simply take the loss function as the energy function and we are altom automatically get an energy based model so the loss function here would be something like the negative log probability of the of the sorry of the correct class but in any case it is just


going to be a high number let's call it 10 to the 9 so the energy function says ha this is very this is very bad this thing here is very bad the entire thing you input it won't tell you yet what's bad about it so that also means you can change any of the two things to make the classifier happy now usually we're concerned with changing the label it's like tell me which other label do I need


to input to make you happy and if we make the labels differentiable of course we never input the true label we actually input like a distribution softmax distribution over labels and that's a differentiable we can use gradient descent to update the dog label we can use gradient descent to find the label that would make the energy function more happy so we could use


gradient descent to get the cat level if we had a good classifier but we can also we can also optimize the image to make it compatible with the dog label right that's things that if you ever saw deep dream or something like this those models do exactly that they optimize the input image for a particular label and there you can view the entire neural network including the loss function as


the energy function so what's another example another example is let's say you have a k-means model and the energy function is simply input a data point and for the data point what you're going to do is you're going to find the min cluster index the min Kay over you know you have your multiple clusters here and your data point might be here so you're going to fight the cluster that's


closest and then the distance here this since Dee will be the energy of that so the model is very happy when your data point it comes from one of the clusters but your model is not happy when the data point is far away and that would be the cost function of the k-means function so that's an energy based model too now currently energy based models have come into fashion through things


like Gans or any conservative noise contrastive estimation so in a jet in a gam what you have is you have a discriminator and the discriminator will basically learn a function to differentiate data from non data so that by itself is an energy function so the discriminator will learn a function and that function will be low wherever the discriminator thinks there is a data


right so it will usually do this around the data point so the data points form the valleys right here and then the generator will basically take that discriminator function and will try to infer points that are also in these valleys to produce points that are also in the valleys and then you basically have an energy learning competition the discriminator now tries to push down on


the energy where the true data is and push up on the energy where the generated data is and that will give you basically a steeper energy based function in the future I hope so in this case the discriminator neural network is the energy function and the D generator just tries to produce data that is compatible with that energy function so I hope that concept of what an energy


function is is a bit clearer and in any again any machine learning problem can be formulated in terms of an energy function now what is not done so far is what we alluded to a little bit before in the classifier example and also here so right now when we want to Train again we simply take the generator to whose data now what's the generator skull the generators goal is to hit


those valleys in the energy function and we produce a generator into in one shot produce this data but what we could also do is of course we could just start somewhere let's say here we pick a random data point and then we use gradient descent because the energy function in this case is smooth we use gradient descent to just drop down this valley and then find ourselves in this


valley so without ever training a generator we can use this methods to produce points that are in the valley of the energy function right and this I don't know if people I can guess people have trained gams like this the reason why it doesn't work let's say in the real world is because that procedure will just produce adversarial examples for the discriminator and those usually look


like nothing like data because if you keep the discriminator just stable and gradient descent against it what you'll get isn't really qualitatively good but in principle if the discriminator was a good energy function for the date to describe the data we could use gradient descent the same up here in order to find a good label for an image given that we have a good energy function


right so this is that we could simply gradient descent on the label in order to find a better in order to find a better label so in this paper we're going to have a situation where we say we're given an energy function and we're given a bunch of inputs they are then called X a and W and if I have my energy function already if I have given my energy function and I have given two of


those three things any two right I can infer the last thing simply by gradient descent on my energy function because I know the energy function is zero when all of these when the energy function is happy with the input so when all of these things agree basically the energy function is happy it will output zero otherwise it will output a high value therefore if I given any of those


two any two of those three things I can find a compatible third thing by descending and then of course over here in these machine learning problems the task was always actually to learn an energy function right so usually in the training dates that we are given images and labels and we want to learn this energy function which would be parameterized so we want to learn the


parameters and the same here in our general case if we are now given three things but we are not given the parameters of the energy function we don't know what those are as long as we're given all of the inputs and our training data set and our training data set guarantees these are actually you know these are inputs that are compatible with each other the energy function should below we can


simply gradient descent on the parameters of the energy function so in a sense there are four things right there are these three inputs and then there are the parameters of the energy function if we're given any three of those four we can gradient descent on the rest and that's going to be the basis so the X here is going to be the so-called state and the state in this


paper is going to be images of entities the entities sorry is not going to be images but the entities are these little circles that you're going to see and each of those entities can have an exposition a Y position and I believe a color so our G and B so each of those can have that and then the concatenation of all of those attributes is a one big vector and that is your X that's your


state so state is number of entities and their attributes a is going to be an attention mask over the state so a is going to be here you have four entities so a will have four entries telling you which of these entities you should pay attention to right now and W is going to be a concept vector so called so W is going to be the embedding of a concept now what a concept is in this case is very


general I can give you an example one a concept is do any of do the entities that the a pays attention to are they close to each other so in this case you see we have two entities that a has a high value one and this is this all up here and this ball down here now if the concept vector is the embedding for the concept of being close to each other then the energy function would be very


happy if those two things are close to each other and it would be very unhappy if those two things aren't close to each other but in the very same situations of the same X the same attention mask but a different concept so a different W vector right here then the the energy function would be maybe very happy if the two things are far apart and maybe unhappy if the two things are close so


the question is always how are the three things that you put into the energy function compatible with each other and given all but one of these things you can infer the other so let's say you have a perfect energy function for this this all of these for this situation you're just given the energy function you can trust it and you are given let's make an example you are given the X so


you're given the state I'm gonna draw the state down here right okay this is the state and you were given the W and the W is the embedding it's a vector but the embedding space but the embedding is for a line right so the geometric the geometric unit of a line now your task is to find a the attention mask that will make the energy function happy and as you can see right here


while you would do is you would put a lot of weight on this this this and this ball and no weight on that ball because those make a line and since everything here is differentiable so the state is differentiable the attentions differentiable and the concepts or vectors that are differentiable you can use gradient descent to find that another example if you're given again


the same W so line and you were given this following thing and you were given now you're given the attention on these three and you say please find the X please find the X the states that makes this energy function happy now this here you would call the starting state the x0 your ear task is going to be find the x1 find the state how do you have to change this state such that the energy function


is happy and of course the answer is going to be is to push this ball here inward until it is in the middle of the two others so the three form a line right these three form a line you don't you don't have to do anything to this ball up here because there is no attention on it and the attention it's only is the concept for the things that you put attention on yeah and the state


are those three in agreement and the energy function is happy okay we have covered the basics now let's dive into the paper I think this is the longest introduction ever but I think it will pay off once see so they they specifically or this this author I think it's a single author identifies two different things that you can do with an energy function here of


course you can do more as we saw but they identify two so here is where you have given the initial state and an attention mask and you want to find the x1 the state that satisfies the concept and the tension the most this the author calls generation as you can see here these four things that you have the attention on are pushed around until they make a square because the concept


right now is square and in the other case where you are given this x0 and x1 just call this X right here just call this thing X if you're given those two and you are given the concept Square and you're tasked with finding a the attention mask of course you're going to put the attention on these right here and that is going to happen through gradient descent again we're not


learning a model to give you that attention like in again we're learning a generator to just one shot give it to you right now what we're going to do is we're going to gradient descent optimize on our smooth energy function to give us that perfect attention mask that satisfies the energy function all right so this is the difference right here gradient descent is part of the output


procedure of the model usually we just use it to learn and we learn a one-shot model but here gradient descent is part of the model so they introduce energy functions here and they say okay we can have a policy on X so if we're given a concept W and if we're given an A we can have a policy over X which basically means we can find X's that are compatible with that by running gradient


descent here you see there is an XK minus 1 and we are running gradient descent on the energy function with respect to X to find a better X that satisfies the energy function given those inputs and the same if we want to find an attention mask we are running gradient descent on the attention mask again in order to satisfy the same energy function so you see the inputs


are both times the same the concept here we can input square here we can input square but the difference is what we're running gradient descent on and what we keep constant and I would get I would add a third line here actually because we can also if we're given an X and an A we can also infer a W and that's going to be an integral part so if I have this right here and this situation and I have


say I have a tension on these for now I can ask the model so I'm given X and I'm given a I can ask the model to infer W and the model should ideally output ha this is square now the model isn't going to output square the model is going to output a vector representation of square right so the model is going to output square but as a vector of numbers because that's how we've trained it W


isn't it is the embedding but what we can then do later is we can say okay I'm not going to tell you it's a square you just come up with a vector W it describes this situation and now I'm going to take that vector W that you came up with miss mister or missus model and I'm going to take tell you a new situation this situation right here and I'm going to now give you X and I'm


going to give you the W that you yourself have output and now please tell me what's the a and then the model is of course supposed to tell you oh these four here or the a so without without ever telling that it should be a square what you can do is you can let the model infer a W from one example situation and then transfer that W to a new situation so it can identify you can just say


whatever concept I have up here please apply that same concept which is the W down here and this is the entire paper now this is the concept learning through energy based models okay so that is kind of a third line I would add down here you can infer a concept vector if you're given the X and the a so in order to do all this their energy function is going to be a so called relational neural


network so what you'll have is you'll have a simple neural network a multi-layer perceptron that always connects two entities to each other with the concept vector and then this is a belief a sigmoid that connects the attention masks of the two and then you simply sum over all pairs of two entries in your model and then you send that through an MLP sorry through an MLP


again this I believe is not so important it's just important that they can feed this entire situation the X the a and the W they can basically feed into a neural network in the neural network comes up with a number of how well those three things fit together and then you can transfer these concepts that's pretty cool now the only question is of course and we've always said we're given


an energy function or just we just have it but of course this is a neural network and the neural network has parameters and the parameters we don't know what good parameters are at the beginning so we need to train this thing and again the reason why these are toy problems right here is I mean we'll get to why it's computational but this is kind of a new field


I believe in machine learning at least I come from classical machine learning and we only ever have used like SGD to train and we only have her have produced models that one shot produce something and here we this is a I believe there's a new concept where you use gradient descent as part of the output and that makes a lot of trouble and so that's why we work


in toy problems so what this this here is the situation I described you have a demo event where you're given the X and the a and you're supposed to infer the W so the question here is what's the W and the model will come up with a W and you're not gonna do anything you're not right now you're simply gonna take that W and tell it oh well here is a so called test event so please apply the W you


came up with in this test event and please find me the the a in this case that satisfies the W and the X I give you here and of course the a right here is as you can see even you don't know that it's a square and the actual concept here is move the grey ball to the middle of the square right that that is it here but no one has told me this I just looked at the picture so the the


correct answer here would be to place attention on those four things and then to take this thing and move it to the middle right here in in the in this over here so that would be the correct answer now the question is how do you train something like this and they they show that they so this is the loss function right here the loss function is they give you a concept and an initial


situation and you're supposed to infer the x1 and the a and the loss function is simply the negative log likelihood of that but what does that mean so will will make it easier if if you have this this procedure right here where you have demo event this up here this is demo and this is a test event how are you going this entire procedure how are you going to learn the energy function well in


this case this entire procedure this entire thing is one training sample sample but usually we have input and label and now here it's much more complicated because so we have input okay that's this X and this a cool but then we have SGD as integral part of the procedure to determine the W and now what we could do is just apply a loss to the W but we don't because we don't know


what the embedding space for the concepts is we could maybe train a classifier but in this case we want to train the ability to transfer these concepts so our training sample needs to be one time transferring a concept so SGD for one is part of our process here and not only that but then this this X here of course is also part of our training sample write this up here as X


0 and this here is X 1 and now we need to find this a this attention mask and that is an SGD again remember inferring anything through the energy function is a gradient descent process so ultimately our one training example consists of X 0 a at the beginning so let's call that a zero it consists of the SGD procedure to find W it consists of X 1 and they consist of the SGD procedure to find a


the a 1 the output a and then that will give us the output a the a 1 so this here is our input in the classical machine and this would be our X and this here would be our label Y and that's what we trained on we trained so it such that the output right here the this is of course sorry this is of course the Y hat this is what we predict and in the training sample we just write


a little generator that will you know make this situation that knows what the concept is right it will say okay I'm gonna make an example for a square then it make this will make the attention mask for a square and then it will make the new situation again with a square but not tell us the attention mask there and it will make the attention mask into the true Y so at the end we can compare


what our model output the attention mask we output here without ever knowing that it should be a square and we have the true label which comes out of the generator that at the beginning decided that it should be a square and then the loss in the distance between those two that's our loss this is an in this is an enormous procedure to get a loss and most crucially you have to back


propagate through optimization procedures and this is something that we just can't do yet in our models if you take an image a resonate 50 right right now we do one forward propagation to get a label in this procedure if you had two back propagate through the optimization procedure for each sample you would need to basically back propagate through 50 forward passes of the resonate if you if


your optimization procedure is 50 steps long and that is just not feasible right now so that's why we don't do it but I believe maybe once we find a smart way of back propping through optimization procedures a whole lot of these things will become the new and new wave and machine learning I really I'm excited - I'm pretty sure it doesn't work yet and this is very figley fiddly work but I'm


excited by the prospect that we can do this so this is the training procedure right you've given X 0 x1 and a and you Optima is in order to infer the concept behind it right the generator that your level generator of your training data it knows the concept it has a concept in mind when it generated this but you're not telling your model what the concept is it needs


to infer that and then using the model thing that the model inferred you can either give it x0 and x1 and infer a or you can give it the X and the a and in forex you can do either of those right these are called identification or generation respectively and then you compare the output here to what the generator at the beginning thought again it's not telling you it's that's because


that's the label and you compare this to that and that will be your loss to train your energy function parameters so your training samples if you think of this entire thing as one forward pass of the model then it's just classic machine learning right you have a training sample which is one forward pass and you have a corresponding label that you infirm so let's jump to the experiments


right here experiments are actually pretty cool so what they've done is for example have taken the concept of being far apart from something now being far apart so that the little X needs to be as far away as possible from the ball that has the attention on it so if you do generation and you start the little X right here and you ask the model where please infer the next state of the world


it will push that little X away right here and in color you can see the energy function valleys of the position of the X so it pushes it away from this thing but if you take the same concept embedding the concept embedding of being far away but you don't do generation you do identification which means you infer the a then it will simply tell you that this ball right here is the furthest


away from the X right so you can do all sorts of things like this and transferring concepts I find this here pretty interesting so they had to have two different concepts one concept is read as an identification you need to identify the red ball but the other concept is you need to turn something red right you need to take a ball that is maybe now blue and of course the


color you can gradient descent on the colors you'd need to make it red and since the energy function it just takes three input X a and W it doesn't you you you're not going to tell it right now in which situation you are it has to create create this W embedding space through learning and if you do it with those two concepts then it will put the make something red concept and the


is something red concepts in the same places so this is a PCA and then blue I think these blue is the attention codes for identify the red things and in red or the generation code for make something red and they will be put in the same place which is pretty cool it means that the energy function really learns the feature of something being red I find this pretty pretty neat and then


here they they have some experiments where they basically show we need that gradient descent optimization procedure because only after many steps will will the energy function basically be aligned with the concept that you want so if you have a zero shot model like just one forward pass as we do here you'll see that the energy function that is supposed to make a circle from samples


right this is the example concept right here it if you just have a one shot model it will it cannot but in this case at least it doesn't learn to one shot produce only if you opt in for a few steps will it get this so you optimize at inference time and that seems to be very important you can see again here demonstrations of this so the example is this and then the model as


you can see after 20 steps learn optimizes the points to go to these locations whereas after only one step it didn't do that yet so there are complex things at work here and this column here is where you don't have a relational or neural network so you can't basically capture dependencies between things so you you have no chance of making a square because you don't know where the


things are in relation to each other but that's more of an engineering question their point is basically that if you have models that do an optimization at inference time they are much more powerful than models that just do a one-shot forward pass it's sort of like an auto regressive model in NLP versus a non auto regressive model that produces all words at once if you produce all


words of a sentence at once no word can depend on any other word and you can just come loose independent or you can just produce independent things which will make the sentence often not make any sense they also have this KL objective which is a regularizer which I believe that's just a trial and error they built it in because but it is a regularizer I don't want to really go into that and


then they they do demonstration in and they re-enacted on a robot the demonstration here is that there is a situation where two things have a tension on and you're supposed to move something into the middle of the two things so that's the content you don't tell the robot the concept it needs to learn that from data and then infer that this is the concept that you want and


then transfer that to the other environment now you know this you look you know there's this robot environment but ultimately they still encode the positions of these things and the position of that and really all you have to do different here is that instead of moving this actuator directly you need to like calculate what you need to do to the individual joints in the robot so I


think this is maybe because it's open AI and it needs to you know look robot II and stuff but the problem here is not really different it's it's not even it's not real-world transfer or anything so yeah let's let go through some of the things they can learn with this so you can see here they can learn these regional geometric shapes and so on the left is the example


event that the model needs to take the concept from now this is this is I believe very much identification so what they did is they trained with a data set where all of these appear right so this there are squares there are lines there are circles so this is maybe my criticism here that it is not so much to generally infer a concept it is more like identify the concept so the model


basically just needs to decide is this line is the circle or is this square because that was those things were in the training data set it would be nice to see how this generalizes to general concepts or if we can even make that if we can have a zero shot concept inference and then transfer those concepts to other things maybe that's already happening I don't I don't know so here the spatial


arrangement is to either be close to something or to be between two things so if the attention is on two things you want in between so you see the top ones are the demonstrations it needs to recognize the concept and it needs to basically optimize to fulfill that concept shapes so to make shapes is mmm oh yeah there's a triangle right again this this this just very much I believe


relies on recognition and not actual understanding of what a triangle here you have proximity being closer being far apart what else is cool oh yeah if the recognition for the same task right you need to identify the ball that is closer for and here you really also see the optimization procedure in action where for example at the beginning of each flicker you kind of


see the attention being everywhere and then stabilizing to one or two points so if two points are equally close or far apart you'll see the attention being on multiple points which is pretty cool right so that means the model really learns this this is concept here's the count quantity so you can either have one two or larger than three or something yeah that seems like they


tried three and four and didn't work so they just said I will just do larger than three and here is this robot thing where it also always needs to move in between now this this is the part that I'm not really impressed with but you know whatever whatever you want okay I hope this was a good introduction to energy functions what you can do with them what I think of them end of this


The "Dirty Jobs" crew and I were called to a little town in Colorado, called Craig. It's only a couple dozen square miles. It's in the Rockies. And the job in question was sheep rancher. My role on the show, for those of you who haven't seen it -- it's pretty simple. I'm an apprentice, and I work with the people who do the jobs in question. And my responsibilities are to simply try and keep up,

Learning from dirty jobs | Mike Rowe


and give an honest account of what it's like to be these people for one day in their life. The job in question: herding sheep. Great. We go to Craig and we check into a hotel, and I realize the next day that castration is going to be an absolute part of this work. Normally, I never do any research at all. But this is a touchy subject, and I work for the Discovery Channel,

Learning from dirty jobs | Mike Rowe


and we want to portray accurately whatever it is we do. And we certainly want to do it with a lot of respect for the animals. So I call the Humane Society and I say, "Look, I'm going to be castrating some lambs. Can you tell me the deal?" And they're like, "Yeah, it's pretty straightforward." They use a band, basically, a rubber band, like this, only a little smaller.

Learning from dirty jobs | Mike Rowe


This one was actually around the playing cards I got yesterday -- (Laughter) But it had a certain familiarity to it. And I said, "Well, what exactly is the process?" And they said, "The band is applied to the tail, tightly. And then another band is applied to the scrotum, tightly. Blood flow is slowly retarded; a week later the parts in question fall off. "Great -- got it."

Learning from dirty jobs | Mike Rowe


OK, I call the SPCA to confirm this. They confirm it. I also call PETA just for fun, and they don't like it, but they confirm it. OK, that's basically how you do it. So the next day I go out. And I'm given a horse and we go get the lambs and we take them to a pen that we built, and we go about the business of animal husbandry. Melanie is the wife of Albert. Albert is the shepherd in question.

Learning from dirty jobs | Mike Rowe


Melanie picks up the lamb, one hand on both legs on the right, likewise on the left. Lamb goes on the post, she opens it up. Alright. Great. Albert goes in, I follow Albert, the crew is around. I always watch the process done the first time before I try it. Being an apprentice, you know, you do that. Albert reaches in his pocket to pull out, you know, this black rubber band,

Learning from dirty jobs | Mike Rowe


but what comes out instead is a knife. And I'm like, "Hmm, that's not rubber at all," you know? (Laughter) And he kind of flicked it open in a way that caught the sun that was just coming over the Rockies, it was very -- (Laughter) It was ... it was impressive. In the space of about two seconds, Albert had the knife between the cartilage of the tail, right next to the butt of the lamb,

Learning from dirty jobs | Mike Rowe


and very quickly, the tail was gone and in the bucket that I was holding. A second later, with a big thumb and a well-calloused forefinger, he had the scrotum firmly in his grasp. And he pulled it toward him, like so, and he took the knife and he put it on the tip. "Now, you think you know what's coming, Michael, You don't, OK?" (Laughter) He snips it, throws the tip over his shoulder,

Learning from dirty jobs | Mike Rowe


and then grabs the scrotum and pushes it upward, and then his head dips down, obscuring my view. But what I hear is a slurping sound, and a noise that sounds like Velcro being yanked off a sticky wall, and I am not even kidding. Can we roll the video? No, I'm kidding, we don't -- (Laughter) I thought it best to talk in pictures. I do something now I've never, ever done on a "Dirty Jobs" shoot, ever.

Learning from dirty jobs | Mike Rowe


I say, "Time out. Stop." You guys know the show, we use take one; we don't do take two. There's no writing, there's no scripting, there's no nonsense. We don't fool around, we don't rehearse -- we shoot what we get! I said, "Stop. This is nuts." I mean -- (Laughter) "This is crazy. We can't do this." And Albert's like, "What?" And I'm like, "I don't know what just happened,

Learning from dirty jobs | Mike Rowe


but there are testicles in this bucket, and that's not how we do it." He said "Well, that's how we do it." I said, "Why would you do it this way?" And before I even let him explain, I said, "I want to do it the right way, with the rubber bands." And he says, "Like the Humane Society?" I said, "Yes, like the Humane Society. Let's do something that doesn't make the lamb squeal and bleed.

Learning from dirty jobs | Mike Rowe


We're on in five continents, dude! We're on twice a day on the Discovery -- we can't do this." He says, "OK." He goes to his box and pulls out a bag of these little rubber bands. Melanie picks up another lamb, puts it on the post, band goes on the tail, band goes on the scrotum. Lamb goes on the ground, lamb takes two steps, falls down, gets up, shakes a little,

Learning from dirty jobs | Mike Rowe


takes another couple steps, falls down. I'm like, this is not a good sign for this lamb, at all. Gets up, walks to the corner. It's quivering, and it lies down and it's in obvious distress. And I'm looking at the lamb and I say, "Albert, how long? When does he get up?" He's like, "A day?" I said, "A day! How long does it take them to fall off?" "A week." Meanwhile, the lamb that he had just done his little procedure on

Learning from dirty jobs | Mike Rowe


is, you know, he's just prancing around, bleeding stopped. He's, you know, nibbling on some grass, frolicking. And I was just so blown away at how completely wrong I was, in that second. And I was reminded how utterly wrong I am, so much of the time. (Laughter) And I was especially reminded of what a ridiculously short straw I had that day, because now I had to do what Albert had just done,

Learning from dirty jobs | Mike Rowe


and there are like 100 of these lambs in the pen. And suddenly, this whole thing's starting to feel like a German porno, and I'm like -- (Laughter) Melanie picks up the lamb, puts it on the post, opens it up. Albert hands me the knife. I go in, tail comes off. I go in, I grab the scrotum, tip comes off. Albert instructs, "Push it way up there." I do. "Push it further."

Learning from dirty jobs | Mike Rowe


I do. The testicles emerge. They look like thumbs, coming right at you. And he says, "Bite 'em. Just bite 'em off." (Laughter) And I heard him, I heard all the words -- (Laughter) Like, how did I get here? How did -- I mean -- how did I get here? It's just -- it's one of those moments where the brain goes off on its own, and suddenly, I'm standing there in the Rockies,

Learning from dirty jobs | Mike Rowe


and all I can think of is the Aristotelian definition of a tragedy. You know, Aristotle says a tragedy is that moment when the hero comes face to face with his true identity. (Laughter) And I'm like, "What is this jacked-up metaphor? I don't like what I'm thinking right now." And I can't get this thought out of my head, and I can't get that vision out of my sight,

Learning from dirty jobs | Mike Rowe


so I did what I had to do. I went in and I took them. I took them like this, and I yanked my head back. And I'm standing there with two testicles on my chin. (Laughter) And now I can't get -- I can't shake the metaphor. I'm still in "Poetics," in Aristotle, and I'm thinking -- out of nowhere, two terms come crashing into my head, that I hadn't heard since my classics professor in college drilled them there.

Learning from dirty jobs | Mike Rowe


And they are "anagnorisis" and "peripeteia." Anagnorisis and peripeteia. Anagnorisis is the Greek word for discovery. Literally, the transition from ignorance to knowledge is anagnorisis. It's what our network does; it's what "Dirty Jobs" is. And I'm up to my neck in anagnorises every single day. Great. The other word, peripeteia, that's the moment in the great tragedies --

Learning from dirty jobs | Mike Rowe


Euripides and Sophocles. That's the moment where Oedipus has his moment, where he suddenly realizes that hot chick he's been sleeping with and having babies with is his mother. That's peripety, or peripeteia. And this metaphor in my head -- I've got anagnorisis and peripeteia on my chin -- (Laughter) I've got to tell you, it's such a great device, though. When you start to look for peripeteia,

Learning from dirty jobs | Mike Rowe


you find it everywhere. I mean, Bruce Willis in "The Sixth Sense," right? Spends the whole movie trying to help the little kid who sees dead people, and then -- boom! -- "Oh, I'm dead." Peripeteia. You know? It's crushing when the audience sees it the right way. Neo in "The Matrix," you know? "Oh, I'm living in a computer program. That's weird." These discoveries that lead to sudden realizations.

Learning from dirty jobs | Mike Rowe


And I've been having them, over 200 dirty jobs, I have them all the time, but that one -- that one drilled something home in a way that I just wasn't prepared for. And, as I stood there, looking at the happy lamb that I had just defiled -- but it looked OK; looking at that poor other little thing that I'd done it the right way on, and I just was struck by --

Learning from dirty jobs | Mike Rowe


if I'm wrong about that, and if I'm wrong so often, in a literal way, what other peripatetic misconceptions might I be able to comment upon? Because, look -- I'm not a social anthropologist, but I have a friend who is. And I talk to him. (Laughter) And he says, "Hey Mike, look. I don't know if your brain is interested in this sort of thing or not, but do you realize you've shot in every state?

Learning from dirty jobs | Mike Rowe


You've worked in mining, you've worked in fishing, you've worked in steel, you've worked in every major industry. You've had your back shoulder to shoulder with these guys that our politicians are desperate to relate to every four years, right?" I can still see Hillary doing the shots of rye, dribbling down her chin, with the steel workers. I mean, these are the people that I work with every single day.

Learning from dirty jobs | Mike Rowe


"And if you have something to say about their thoughts, collectively, it might be time to think about it. Because, dude, you know, four years." So, that's in my head, testicles are on my chin, thoughts are bouncing around. And, after that shoot, "Dirty Jobs" really didn't change, in terms of what the show is, but it changed for me, personally. And now, when I talk about the show,

Learning from dirty jobs | Mike Rowe


I no longer just tell the story you heard and 190 like it. I do, but I also start to talk about some of the other things I got wrong; some of the other notions of work that I've just been assuming are sacrosanct, and they're not. People with dirty jobs are happier than you think. As a group, they're the happiest people I know. And I don't want to start whistling "Look for the Union Label,"

Learning from dirty jobs | Mike Rowe


and all that happy-worker crap. I'm just telling you that these are balanced people who do unthinkable work. Roadkill picker-uppers whistle while they work, I swear to God -- I did it with them. They've got this amazing sort of symmetry to their life. And I see it over and over and over again. So I started to wonder what would happen if we challenged some of these sacred cows?

Learning from dirty jobs | Mike Rowe


Follow your passion -- we've been talking about it here for the last 36 hours. Follow your passion -- what could possibly be wrong with that? It's probably the worst advice I ever got. (Laughter) Follow your dreams and go broke, right? I mean, that's all I heard growing up. I didn't know what to do with my life, but I was told if you follow your passion, it's going to work out.

Learning from dirty jobs | Mike Rowe


I can give you 30 examples right now. Bob Combs, the pig farmer in Las Vegas who collects the uneaten scraps of food from the casinos and feeds them to his swine. Why? Because there's so much protein in the stuff we don't eat, his pigs grow at twice the normal speed, and he's one rich pig farmer. He's good for the environment, he spends his days doing this incredible service,

Learning from dirty jobs | Mike Rowe


and he smells like hell, but God bless him. He's making a great living. You ask him, "Did you follow your passion here?" and he'd laugh at you. The guy's worth -- he just got offered like 60 million dollars for his farm and turned it down, outside of Vegas. He didn't follow his passion. He stepped back and he watched where everybody was going, and he went the other way.

Learning from dirty jobs | Mike Rowe


And I hear that story over and over. Matt Freund, a dairy farmer in New Canaan, Connecticut, who woke up one day and realized the crap from his cows was worth more than their milk, if he could use it to make these biodegradable flowerpots. Now he's selling them to Walmart, right? Follow his passion? The guy's -- come on. So I started to look at passion, I started to look at efficiency vs. effectiveness.

Learning from dirty jobs | Mike Rowe


As Tim talked about earlier, that's a huge distinction. I started to look at teamwork and determination. And basically, all those platitudes they call "successories" that hang with that schmaltzy art in boardrooms around the world right now, that stuff -- it's suddenly all been turned on its head. Safety. Safety first is ... Going back to OSHA and PETA and the Humane Society:

Learning from dirty jobs | Mike Rowe


What if OSHA got it wrong? I mean -- this is heresy, what I'm about to say -- but what if it's really safety third? Right? (Laughter) No, I mean, really. What I mean to say is: I value my safety on these crazy jobs as much as the people that I'm working with, but the ones who really get it done -- they're not out there talking about safety first. They know that other things come first --

Learning from dirty jobs | Mike Rowe


the business of doing the work comes first, the business of getting it done. And I'll never forget, up in the Bering Sea, I was on a crab boat with the "Deadliest Catch" guys -- which I also work on in the first season. We were about 100 miles off the coast of Russia: 50-foot seas, big waves, green water coming over the wheelhouse, right? Most hazardous environment I'd ever seen,

Learning from dirty jobs | Mike Rowe


and I was back with a guy, lashing the pots down. So I'm 40 feet off the deck, which is like looking down at the top of your shoe, you know, and it's doing this in the ocean. Unspeakably dangerous. I scamper down, I go into the wheelhouse and I say, with some level of incredulity, "Captain -- OSHA?" And he says, "OSHA? Ocean." And he points out there. (Laughter)

Learning from dirty jobs | Mike Rowe


But in that moment, what he said next can't be repeated in the Lower 48. It can't be repeated on any factory floor or any construction site. But he looked at me and said, "Son," -- he's my age, by the way, he calls me "son," I love that -- he says, "Son, I'm the captain of a crab boat. My responsibility is not to get you home alive. My responsibility is to get you home rich."

Learning from dirty jobs | Mike Rowe


(Laughter) You want to get home alive, that's on you." And for the rest of that day -- safety first. I mean, I was like -- So, the idea that we create this sense of complacency when all we do is talk about somebody else's responsibility as though it's our own, and vice versa. Anyhow, a whole lot of things. I could talk at length about the many little distinctions we made

Learning from dirty jobs | Mike Rowe


and the endless list of ways that I got it wrong. But what it all comes down to is this: I've formed a theory, and I'm going to share it now in my remaining 2 minutes and 30 seconds. It goes like this: we've declared war on work, as a society -- all of us. It's a civil war. It's a cold war, really. We didn't set out to do it and we didn't twist our mustache in some Machiavellian way,

Learning from dirty jobs | Mike Rowe


but we've done it. And we've waged this war on at least four fronts, certainly in Hollywood. The way we portray working people on TV -- it's laughable. If there's a plumber, he's 300 pounds and he's got a giant butt crack, admit it. You see him all the time. That's what plumbers look like, right? We turn them into heroes, or we turn them into punch lines. That's what TV does.

Learning from dirty jobs | Mike Rowe


We try hard on "Dirty Jobs" not to do that, which is why I do the work and I don't cheat. But, we've waged this war on Madison Avenue. So many of the commercials that come out there in the way of a message -- what's really being said? "Your life would be better if you could work a little less, didn't have to work so hard, got home a little earlier, could retire a little faster, punch out a little sooner."

Learning from dirty jobs | Mike Rowe


It's all in there, over and over, again and again. Washington? I can't even begin to talk about the deals and policies in place that affect the bottom-line reality of the available jobs, because I don't really know; I just know that that's a front in this war. And right here, guys -- Silicon Valley. I mean -- how many people have an iPhone on them right now?

Learning from dirty jobs | Mike Rowe


How many people have their BlackBerry? We're plugged in; we're connected. I would never suggest for a second that something bad has come out of the tech revolution. Good grief, not to this crowd. (Laughter) But I would suggest that innovation without imitation is a complete waste of time. And nobody celebrates imitation the way "Dirty Jobs" guys know it has to be done.

Learning from dirty jobs | Mike Rowe


Your iPhone without those people making the same interface, the same circuitry, the same board, over and over -- all of that -- that's what makes it equally as possible as the genius that goes inside of it. So, we've got this new toolbox. You know? Our tools today don't look like shovels and picks. They look like the stuff we walk around with. And so the collective effect of all of that

Learning from dirty jobs | Mike Rowe


has been this marginalization of lots and lots of jobs. And I realized, probably too late in this game -- I hope not, because I don't know if I can do 200 more of these things -- but we're going to do as many as we can. And to me, the most important thing to know and to really come face to face with, is that fact that I got it wrong about a lot of things, not just the testicles on my chin.

Learning from dirty jobs | Mike Rowe


I got a lot wrong. So, we're thinking -- by "we," I mean me -- (Laughter) that the thing to do is to talk about a PR campaign for work -- manual labor, skilled labor. Somebody needs to be out there, talking about the forgotten benefits. I'm talking about grandfather stuff, the stuff a lot us probably grew up with but we've kind of -- you know, kind of lost a little.

Learning from dirty jobs | Mike Rowe


Barack wants to create two and a half million jobs. The infrastructure is a huge deal. This war on work that I suppose exists, has casualties like any other war. The infrastructure is the first one, declining trade school enrollments are the second one. Every single year, fewer electricians, fewer carpenters, fewer plumbers, fewer welders, fewer pipe fitters, fewer steam fitters.

Learning from dirty jobs | Mike Rowe


The infrastructure jobs that everybody is talking about creating are those guys -- the ones that have been in decline, over and over. Meanwhile, we've got two trillion dollars, at a minimum, according to the American Society of Civil Engineers, that we need to expend to even make a dent in the infrastructure, which is currently rated at a D minus. So, if I were running for anything -- and I'm not --

Learning from dirty jobs | Mike Rowe


I would simply say that the jobs we hope to make and the jobs we hope to create aren't going to stick unless they're jobs that people want. And I know the point of this conference is to celebrate things that are near and dear to us, but I also know that clean and dirty aren't opposites. They're two sides of the same coin, just like innovation and imitation, like risk and responsibility, like peripeteia and anagnorisis,

Learning from dirty jobs | Mike Rowe


hi there check out these clusters of images right here and just have a look at how all of them are pretty much showing the same object so here's balloons here's birds here's sharks or other fish these are from images from the image net data set and you can see that these clusters are pretty much the object classes themselves so there's all the frogs right here here all the all

Learning To Classify Images Without Labels (Paper Explained)


the people that have caught fish so this the astonishing thing about this is that these clusters have been obtained without any labels of the image net dataset of course the data set has labels but this method doesn't use the labels it learns to classify images without labels so today we're looking at this paper learning to classify images without labels by water from Guns Becca


Simon Van Daan hender stung stamatis Georg Ulis mark pro Simmons and Luke fungal and on a high level overview they have a three-step procedure basically first they they use self supervised learning in order to get good representations second they do a clustering so they do a sort of k nearest neighbor clustering but they do clustering on top of those things but


they're doing in a kind of special way and then third they do a refinement through self labeling so if you know what all of these are you basically understand the paper already but there's a bit of tricky steps in there and it's pretty cool that at the end it works out like you just saw so before we dive in as always if you're here and not subscribed then please do and if you


liked the video share it out and leave a comment if you feel like commenting cool so as we already stated the problem they ask is it possible to automatically classify images without the use of ground truth annotations or even when the classes themselves are not known a priori now you might seem like you might think that this is outrageous how can you class high when you don't even know


what the classes are and so on so the way you have to imagine it going forward and they're sort of they don't explicitly explain it but it's it's sort of assumed that if you have a data set dataset ba-da-da-da-da and you learn to classify it what basically that means is you cluster it right you put some of the data points in in the same clusters okay and then of course the data set I'm


gonna draw the same data set right here the same data set would have an actual classification thing so this would be class zero this here maybe class one and this year might be class 2 now you can't possibly know how the classes are like called or something which one is the first which one is a second so at test time basically if you have a method like this that doesn't use labels what you're


going to do is you're basically going to find you're going to be as generous as possible and in the assignment of these and say I'll look if I assign this here to cluster zero and this here to cluster 2 and this year to cluster 1 and I just you know carry over the labels what would my accuracy be under that labeling so you've asked generous as possible with the assignments of the labels so


that's how it's going to work right that's a way you have to keep in mind we're basically developing an algorithm that gives us this kind of clustering of the data and then if that clustering partitions the data in the same way as the actual labeling would the actual labeling with the test labels then we think it's a good algorithm okay so they claim they have a


okay in this paper we deviate from recent works and advocate a two-step approach and it's actually a three step approach but we're feature learning and clustering are decoupled okay why why is that so they argue what you could do what people have done is and I'm going to well this is just a wall of text so what you could do is you could just basically cluster the data like who says


you can't use clustering algorithms but then the question is what what do you cluster them by like you need a distance so if I have points in 2d it sort of makes sense to use the Euclidean distance here but if I have images of cats and dogs and whatnot then the Euclidean distance between the pixels is really not a good a good thing but also so you might think we could actually we


could you use a deep neural network and then basically send the image that's the image right here send the image through the deep neural network and then either take this last state right here so it goes through and through and through and we could get take either of the hidden states or we could just take you know the last state that is the sort of hidden representation right here and do


the clustering with that but then of course the question is what do you what which neural network do you take how do you train that neural network and there have been a few approaches such as a deep cluster which try to formulate basically an objective for that neural network where you first you send all the images through right you send a bunch of images through to get you in embedding


space you get you points and then in embedding space you think well the features that are in the embedding space they are somehow latent and they you know if basically the entire thing is if this neural network was used to classify images you would have a classification head on top and a classification head this is like a five class classification that is nothing else than a linear


classifier boundary that you put on top of this hidden representation so if you were to use this neural network for classification it must be possible to draw a linear boundary between the classes and therefore the either things like the inner product distance or the Euclidean distance must make sense in that space if they don't make sense in the picture space but they must make


sense in the hidden representation space because what you're going to do with them is exactly linear classification the last classification head of a neural network is just a linear classifier so the assumption is that and the conclusion is well in this space you should be able to cluster by Euclidian distance so what deep cluster does alternate like is first get the


representations you start off with a random neural network then cluster these representations then basically label self label the images in a way now Oh way oversimplifying that technique right here but you have these alternative steps of clustering and then kind of finding better representation and then clustering these representations and what it basically says is that the CNN


itself is such a is like a prior because it's the translation invariant works very good for very well for natural images the CNN itself will lead to good representations if we do it in this way and there's some good results there but this paper argues that if you do that then the the algorithm tends to focus a lot on very low-level features so if the pixel on the bottom right here is blue


right then you can and the neural network by chance puts two of those images where the blue pixel on the bottom right it puts them close together then in the next step it will because they're close together will cluster them together and then it will basically feed back the new representation should put the two in the same class right it will feed back that it should focus even more


on that blue pixel so it's very very dependent on initializations and it can jump super easily onto these low-level of features that have nothing to do with what the high level task you're ultimately trying to solve which is to classify these images later so what this paper does is it says we can eliminate this we can eliminate this the fact that these methods will predict


will produce neural networks that focus on low-level features and how do we do that we do that by representation learning so representation you're learning you might know this as self supervised learning and this is the task they solve in the first step of their objective so let's go through this this right here is an image now the T is a transformation of that image and in self supervised


learning there are several methods that you can transform an image so for example you can random crop an image you can just cut out like a piece right here and scale that up to be as large as the original image or you can use for example data augmentation which means you take the image and you basically so if there is I don't know the cat right here you kind of convolve it with some


things it's there's like a very squiggly cat okay I'm terrible is you can you can rotate it for example so it's like this okay so these are all these are all sets including the crop sets of this transformation T so you transform it in some way and you want after you've transformed it you send your original image that it should be red you send your original image and the transformed


image through a neural network each one by themselves okay and then after then this you say the hidden representation here should be close to each other Oh this is this is basically the self supervised training task it's it's been shown to work very very well as a pre-training method for classification neural networks you you have an image and it's augmented version and you


minimize the inner product or the Euclidean distance between the two versions in the hidden space and the rationale is exactly the same the rationale is that this hidden space of course should be linearly classifiable and so the distance between those should be close and the rationale between having these tasks is that well if I flip the image right if I flip the image


to the right it cannot focus on the pixel on the bottom right anymore because that's not going to be the pixel on the bottom right here and I'm not always going to flip it into the same direction and sometimes I'm gonna crop it so it also can't focus on the pics on the bottom right because in the crop that pixel is like out here it's not even in the crop so basically what


you're looking to do with these self supervised methods is you are looking to destroy this low level of information that's that's all you're looking to build a pipeline of a neural network here that destroys deliberately low level information and you do that by coming up with tasks like this self supervision tasks that this that deliberately exclude this information


from being used I think that's what's going on generally in the self supervised learning thing okay so this here as you can see is the neural network that you train you send both images the original and the Augmented version through the same neural network and then you minimize some distance which is usually like the inner product or the Euclidean distance in this


embedding space okay and what you train you can see right here you train the parameters of this neural network so the transformations are fixed or sampled and the distance is fixed you train the neural networks such that your embeddings minimize this task now this is nothing new this has been this has been used for a couple of years now to get better representation self


supervised learning the thing but they basically say we can use this as an initialization step for this clustering procedure because if we don't do that we focus on these low-level features okay and notice you don't need any labels for this procedure that's why it's called self supervise okay so the second second part is the clustering now they cluster but they


don't just cluster these representations that would be that doesn't perform very well in their in their experiments what they instead do is they minimize this entire objective right here and we'll go through it step by step so they train a new neural network okay this thing right here this is a new neural network so first you have you already have the neural network which was called what was


it even called the one that gives you the embedding with the theta okay it's called five theta it's the same architecture and I think they initialize one with the other so in step 1 you get Phi Theta 5 theta goes if from from X gives you a representation of X ok let's call it hidden X so that's via self supervised learning but in step 2 you train an entirely new new neural


network this phi ada here and you initialize it with this one but now you train it to do the following again you want to minimize sorry you want to maximize the inner product right here see that's the inner product you want to maximize the inner product between two things now that's the same thing as before we want to minimize the distance between two things and the dot product