side it's become possible to make sounds that sound like normal speech but are actually interpreted as a different sentence than the human here's the level of perceptibility of the adversarial perturbation is still kind of high the when you listen to the recording it sounds like there's some noise in the background just like rustling sounds but those rustling sounds are actually the adversarial perturbation that makes the phone hear a completely different sentence yeah that's so fascinating Peter Norvig mention that you're writing


the deep learning chapter for the fourth edition of the artificial intelligence the modern approach book so how do you even begin summarizing the field of deep learning in a chapter well in my case I waited like a year before I actually read anything is it even having written a full length textbook before it's still pretty intimidating to try to start writing just one chapter that covers everything one thing that helped me make that plan was actually the experience of having ridden the full book before and then

Ian Goodfellow: Generative Adversarial Networks (GANs) | Lex Fridman Podcast #19


watching how the field changed after the book came out I realized there's a lot of topics that were maybe extraneous in the first book and just seeing what stood the test of a few years of being published and what seems a little bit less important to have included now helped me pare down the topics I wanted to cover for the book it's also really nice now that the field is kind of stabilized to the point where some core ideas from the 1980s are still used today when I first started studying machine learning almost everything from


the 1980s had been rejected and now some of it has come back so that stuff that's really stood the test of time is what I focused on putting into the book there's also I guess two different philosophies about how you might write a book one philosophy is you try to write a reference that covers everything and the other philosophy is you try to provide a high level summary that gives people the language to understand a field and tells them what the most important concepts are the first deep learning book that I


wrote with Yahshua and Aaron was somewhere between the the two philosophies that it's trying to be both a reference and an introductory guide writing this chapter for Russell and Norvig book I was able to focus more on just a concise introduction of the key concepts and the language you need to read about them more and a lot of cases actually just wrote paragraphs that said here's a rapidly evolving area that you should pay attention to it's it's pointless to try to tell you what the latest and best version of a you know


learn to learn model is right you know I can I can point you to a paper that's recent right now but there isn't a whole lot of a reason to delve into exactly what's going on with the latest learning to learn approach or the latest module produced by learning to learn algorithm you should know that learning to learn is a thing and that it may very well be the source of the latest and greatest convolutional net or recurrent net module that you would want to use in your latest project but there isn't a lot of point in trying to summarize


exactly which architecture in which learning approach got to which level of performance so you maybe focus more on the basics of the methodology so from back propagation to feed-forward to recur in your networks convolutional that kind of thing yeah yeah so if I were to ask you I remember I took algorithms and data structures algorithm there of course remember the professor asked what is an algorithm and yelled at everybody in a good way that nobody was answering it correctly everybody knew what the alkyl it was graduate course everybody knew


what an algorithm was but they weren't able to answer it well let me ask you in that same spirit what is deep learning I would say deep learning is any kind of machine learning that involves learning parameters of more than one consecutive step so that I mean shallow learning is things where you learn a lot of operations that happen in parallel you might have a system that makes multiple steps like you might have had designed feature extractors but really only one step is learned deep learning is anything where you have multiple


operations in sequence and that includes the things that are really popular today like convolutional networks and recurrent networks but it also includes some of the things that have died out like Bolton machines where we weren't using back propagation today I hear a lot of people define deep learning as gradient descent applied to these differentiable functions and I think that's a legitimate usage of the term it's just different from the way that I use the term myself so what's an example of deep learning that is not gradient


descent on differentiable functions in your I mean not specifically perhaps but more even looking into the future what's your thought about that space of approaches yeah so I tend to think of machine learning algorithms as decomposed into really three different pieces there's the model which can be something like a neural nut or a Bolton machine or a recurrent model and I basically just described how do you take data and how do you take parameters and you know what function do you use to make a prediction given the


data and the parameters another piece of the learning algorithm is the optimization algorithm or not every algorithm can be really described in terms of optimization but what's the algorithm for updating the parameters or updating whatever the state of the network is and then the the last part is the the data set like how do you actually represent the world as it comes into your machine learning system so I think of deep learning as telling us something about what does the model look like and basically to qualify as deep I


say that it just has to have multiple layers that can be multiple steps in a feed-forward differentiable computation that can be multiple layers in a graphical model there's a lot of ways that you could satisfy me that something has multiple steps that are each parameterised separately I think of gradient descent as being all about that other piece the how do you actually update the parameters piece so you can imagine having a deep model like a convolutional net and training it with something like evolution or a genetic


algorithm and I would say that still qualifies as deep learning and then in terms of models that aren't necessarily differentiable I guess Boltzmann machines are probably the main example of something where you can't really take a derivative and use that for the learning process but you you can still argue that the model has many steps of processing that it applies when you run inference in the model so that's the steps of processing that's key so geoff hinton suggests that we need to throw away back prop back


propagation and start all over what do you think about that what could an alternative direction of training nil networks look like I don't know that back propagation is going to go away entirely most of this time when we decide that a machine learning algorithm isn't on the critical path to research for improving AI the algorithm doesn't die it just becomes used for some specialized set of things a lot of algorithms like logistic regression don't seem that exciting to AI researchers who are working on things


like speech recognition or autonomous cars today but there's still a lot of use for logistic regression and things like analyzing really noisy data and medicine and finance or making really rapid predictions in really time-limited contexts so I think I think back propagation and gradient descent are around to stay but they may not end up being everything that we need to get to real human level or superhuman AI are you optimistic about us discovering you know back propagation has been around for a few decades


so I optimistic bus about us as a community being able to discover something better yeah I am I think I think we likely will find something that works better you could imagine things like having stacks of models where some of the lower level models predict parameters of the higher level models and so at the top level you're not learning in terms of literally calculating gradients but just predicting how different values will perform you can kind of see that already in some areas like Bayesian optimization where you have a Gaussian process that


predicts how well different parameter values will perform we already used those kinds of algorithms for things like hyper parameter optimization and in general we know a lot of things other than back prep that work really well for specific problems the main thing we haven't found is a way of taking one of these other non back based algorithms and having it really advanced the state-of-the-art on an AI level problem right but I wouldn't be surprised if eventually we find that some of these algorithms that even the ones that already exists not


even necessarily a new one we might find some way of customizing one of these algorithms to do something really interesting at the level of cognition or or the the level of I think one system that we really don't have working quite right yet is like short-term memory we have things like LST M's they're called long short-term memory they still don't do quite what a human does with short-term memory like gradient descent to learn a specific fact has to do multiple steps on that fact like if I I tell you the meeting today is at 3 p.m. I don't need


to say over and over again it's at 3 p.m. it's not 3 p.m. it's at 3 p.m. it's a 3 p.m. right for you to do a gradient step on each one you just hear it once and you remember it there's been some work on things like self attention and attention like mechanisms like the neural Turing machine that can write to memory cells and update themselves with facts like that right away but I don't think we've really nailed it yet and that's one area where I'd imagine that new optimization algorithms are different ways of applying existing


optimization algorithms could give us a way of just lightning-fast updating the state of a machine learning system to contain a specific fact like that without needing to have it presented over and over and over again so some of the success of symbolic systems in the 80s is they were able to assemble these kinds of facts better but dude there's a lot of expert input required and it's very limited in that sense do you ever look back to that as something that will have to return to eventually sort of dust off the book from the shelf and


think about how we build knowledge representation knowledge place well we have to use graph searches searches right and like first-order logic and entailment and things like that a thing yeah exactly in my particular line of work which has mostly been machine learning security and and also generative modeling I haven't usually found myself moving in that direction for generative models I could see a little bit of it could be useful if you had something like a differentiable knowledge base or some other kind of knowledge base where it's


possible for some of our fuzzier machine learning algorithms to interact with the knowledge base immanuel Network is kind of like that it's a differentiable knowledge base of sorts yeah but if if we had a really easy way of giving feedback to machine learning models that would clearly helped a lot with with generative models and so you could imagine one way of getting there would be get a lot better at natural language processing but another way of getting there would be take some kind of knowledge base and figure out a way for


it to actually interact with a neural network being able to have a chat within y'all network yes so like one thing in generative models we see a lot today is you'll get things like faces that are not symmetrical like like people that have two eyes that are different colors and I mean there are people with eyes that are different colors in real life but not nearly as many of them as you tend to see in the machine learning generated data so if if you had either a knowledge base that could contain the fact people's faces are generally


approximately symmetric and eye color is especially likely to be the same on both sides being able to just inject that hint into the machine learning model without it having to discover that itself after studying a lot of data it would be a really useful feature I could see a lot of ways of getting there without bringing back some of the 1980s technology but I also see some ways that you could imagine extending the 1980s technology to play nice with neural nets and have it help get there awesome so you talked about the story of


you coming up with idea of Gans at a bar with some friends you were arguing that this you know Gans would work Jenner of adversarial networks and the others didn't think so then he went home at midnight coated up and it worked so if I was a friend of yours at the bar I would also have doubts it's a really nice idea but I'm very skeptical that it would work what was the basis of their skepticism what was the basis of your intuition why he should work I don't want to be someone who goes around promoting alcohol for the science in


this case I do actually think that drinking helped a little bit mm-hmm when your inhibitions are lowered you're more willing to try out things that you wouldn't try out otherwise so I I have noticed it in general that I'm less prone to shooting down some of my own ideas when I'm when I have had a little bit to drink I think if I had had that idea at lunch time yeah I probably would have thought it it's hard enough I mean one neural net you can't train a second neuron that in the inner loop of the outer neural net that was basically my


friends action was that trying to train two neural nets at the same time would be too hard so it was more about the training process unless so my skepticism would be you know I'm sure you could train it but the thing would converge to would not be able to generate anything reasonable and any kind of reasonable realism yeah so so part of what all of us were thinking about when we had this conversation was deep Bolton machines which a lot of us in the lab including me were a big fan of deep bolts and machines at the time they involved two


separate processes running at the same time one of them is called the positive phase where you load data into the model and tell the model to make the data more likely the owners called the negative phase where you draw samples from the model and tell the model to make those samples less likely in a deep Bolton machine it's not trivial to generate a sample you have to actually run an iterative process that gets better and better samples coming closer and closer to the distribution the model represents so during the training process you're


always running these two systems at the same time one that's updating the parameters of the model and another one that's trying to generate samples from the model and they worked really well on things like Amnesty a lot of us in the lab including me had tried to get the Boltzmann machines to scale past em inist to things like generating color photos and we just couldn't get the two processes to stay synchronized so when I had the idea for Gans a lot of people thought that the discriminator would have more or less the same problem as


the negative phase in the Boltzmann machine that trying to train the discriminator in the inner loop you just couldn't get it to keep up with the generator and the outer loop and that would prevent it from converging to anything useful yeah I share that intuition yeah what turns out to not be the case a lot of the time with machine learning algorithms it's really hard to predict ahead of time how well they'll actually perform you have to just run the experiment and see what happens and I would say I still today don't have


like one factor I can put my finger on it say this is why ganz worked for photo generation and deep Boltzmann machines don't there are a lot of theory papers showing that under some theoretical settings the the gun algorithm does actually converge but those settings are restricted enough that they don't necessarily explain the whole picture in terms of all the results that we see in practice so taking a step back can you in the same way as we talked about deep learning can you tell me what generative adversarial


networks are yeah so generative adversarial networks are a particular kind of generative model a generative model is a machine learning model that can train on some set of data like so you have a collection of photos of cats and you want to generate more photos of cats or you want to estimate a probability distribution over cats so you can ask how likely it is that some new image is a photo of a cat ganzar one way of doing this some generative models are good at creating new data other generative models are good at estimating that


density function and telling you how likely particular pieces of data are to come from the same distribution as a training data gans are more focused on generating samples rather than estimating the density function there are some kinds of games like flow gun that can do both but mostly guns are about generating samples of generating new photos of cats that look realistic and they do that completely from scratch it's analogous to human imagination when again creates a new image of a cat it's using a neural network to produce a cat


that has not existed before it isn't doing something like compositing photos together you're not you're not literally taking the eye off of one cat on the ear off of another cat it's it's more of this digestive process where the the neural net trains on a lot of data and comes up with some representation of the probability distribution and generates entirely new cats there are a lot of different ways of building a generative model what's specific against is that we have a two-player game in the game theoretic sense and as the players in


this game compete one of them becomes able to generate realistic data the first player is called the generator it produces output data such as just images for example and at the start of the learning process it'll just produce completely random images the other player is called the discriminator the discriminator takes images as input and guesses whether they're real or fake you train it both on real data so photos that come from your training set actual photos of cats and you try to say that those are real you also train it on images that come


from the generator network and you train it to say that those are fake as the two players compete in this game the discriminator tries to become better at recognizing where their images are real or fake and the generator becomes better at fooling the discriminator into thinking that its outputs are are real and you can analyze this through the language of game theory and find that there's a Nash equilibrium where the generator has captured the correct probability distribution so in the cat example it makes perfectly realistic cat


photos and the discriminator is unable to do better than random guessing because all the all the samples coming from both the data and the generator look equally likely to have come from either source so do you ever do sit back and does it just blow your mind that this thing works so from very so it's able to estimate that density function enough to generate generate realistic images I mean does it yeah do you ever sit back yeah how does this even why this is quite incredible especially where Gant's have gone in terms of


realism yeah and and not just to flatter my own work but generative models all of them have this property that if they really did what we asked them to do they would do nothing but memorize the training data right some models that are based on maximizing the likelihood the way that you obtain the maximum likelihood for a specific training set is you assign all of your probability mass to the training examples and nowhere else forgets the game is played using a training set so the way that you become unbeatable in the game is you literally


memorize training examples one of my former interns wrote a paper his name is a Vaishnav nagarajan and he showed that it's actually hard for the generator to memorize the training data hard in a statistical learning theory sense that you can actually create reasons for why it would require quite a lot of learning steps and and a lot of observations of of different latent variables before you could memorize the training data that still doesn't really explain why when you produce samples that are new why do you get compelling


images rather than you know just garbage that's different from the training set and I don't think we really have a good answer for that especially if you think about how many possible images are out there and how few images the generative model sees during training it seems just unreasonable that generative models create new images as well as they do especially considering that we're basically training them to memorize rather than generalize I think part of the answer is there's a paper called deep image prior where they show that


you can take a convolutional net and you don't even need to learn the parameters of it at all you just use the model architecture and it's already useful for things like in painting images I think that shows us that the convolutional network architecture captures something really important about the structure of images and we don't need to actually use learning to capture all the information coming out of the convolutional net that would that would imply that it would be much harder to make generative models in


other domains so far we're able to make reasonable speech models and things like that but to be honest we haven't actually explored a whole lot of different data sets all that much we don't for example see a lot of deep learning models of like biology datasets where you have lots of microarrays measuring the amount of different enzymes and things like that so we may find that some of the progress that we've seen for images and speech turns out to really rely heavily on the model architecture and we were able to do what


we did for vision by trying to reverse-engineer the human visual system and maybe it'll turn out that we can't just use that same trick for arbitrary kinds of data all right so there's aspects of the human vision system the hardware of it that makes it without learning without cognition just makes it really effective at detecting the patterns we've seen the visual world yeah that's yeah that's really interesting what in a big quick overview in your view in your view what types of Gans are there and what other generative models besides


games are there yeah so it's maybe a little bit easier to start with what kinds of generative models are there other than Gans so most generative models are likelihood based where to train them you have a model that tells you how how much probability it assigns to a particular example and you just maximize the probability assigned to all the training examples it turns out that it's hard to design a model that can create really complicated images or really complicated audio waveforms and still have it be possible to estimate the the likelihood


function from a computational point of view most interesting models that you would just write down intuitively it turns out that it's almost impossible to calculate the amount of probability they assign to a particular point so there's a few different schools of generative models in the likelyhood family one approach is to very carefully design the model so that it is computationally tractable to measure the density it assigns to a particular point so there are things like auto regressive models like pixel CN n those basically break


down the probability distribution into a product over every single feature so for an image you estimate the probability of each pixel given all of the pixels that came before it hmm there's tricks where if you want to measure the density function you can actually calculate the density for all these pixels more or less in parallel generating the image still tends to require you to go one pixel at a time and that can be very slow but there again tricks for doing this in a hierarchical pattern where you can keep the runtime under control or the


quality of the images it generates putting runtime aside pretty good they're reasonable yeah the I would say a lot of the best results are from Gans these days but it can be hard to tell how much of that is based on who's studying which type of algorithm if that makes sense the amount of effort invest in it but yeah or like the kind of expertise so a lot of people who've traditionally been excited about graphics or art and things like that have gotten interested in Gans and to some extent it's hard to tell our Gans


doing better because they have a lot of graphics and art experts behind them or our Gans doing better because they're more computationally efficient or our Gans doing better because they prioritize the realism of samples over the accuracy of the density function I think I think all of those are potentially valid explanations and it's it's hard to tell so can you give a brief history of Gans from 2014 we paid for 13 yeah so a few highlights in the first paper we just showed that Gans basically work if you look back at the


samples we had now they looked terrible on the CFR 10 dataset you can't even recognize objects in them your papers I will use CFR 10 we use em NIST which is little handwritten digits we used the Toronto face database which is small grayscale photos of faces we did have recognizable faces my colleague Bing Xu put together the first again face model for that paper we also had the CFR 10 dataset which is things like very small 32 by 32 pixels of cars and cats and dogs for that we didn't get recognizable objects but all the deep


learning people back then we're really used to looking at these failed samples and kind of reading them like tea leaves right and people who are used to reading the tea leaves recognize that our tea leaves at least look different right maybe not necessarily better but there was something unusual about them and that got a lot of us excited one of the next really big steps was lap gown by Emily Denton and seemeth chintala at Facebook AI research where they actually got really good high-resolution photos working with gans for the first time


they had a complicated system where they generated the image starting at low res and then scaling up to high res but they were able to get it to work and then in 2015 I believe later that same year palek Radford and sumh intelli and Luke Metz published the DC gain paper which it stands for deep convolutional again it's kind of a non unique name because these days basically all gans and even some before that were deep in convolutional but they just kind of picked a name for a really great recipe where they were able to actually using


only one model instead of a multi-step process actually generate realistic images of faces and things like that that was sort of like the beginning of the Cambrian explosion of gans like you know once once you got animals that had a backbone you suddenly got lots of different versions of you know like fish and right they have four-legged animals and things like that so so DC Gann became kind of the backbone for many different models that came out used as a baseline even still yeah yeah and so from there I would say some interesting


things we've seen are there's a lot you can say about how just the quality of standard image generation ganz has increased but what's also maybe more interesting on an intellectual level is how the things you can use guns for has also changed one thing is that you can use them to learn classifiers without having to have class labels for every example in your your training set so that's called semi-supervised learning my colleague at open AI Tim Solomon's who's at at brain now wrote a paper called improved techniques for training


guns I'm a co-author on this paper but I can't claim any credit for this particular part one thing he showed in the paper is that you can take the gun discriminator and use it as a classifier that actually tells you you know this image is a cat this image is a dog this image is a car this image is a truck and so and not just to say whether the image is real or fake but if it is real to say specifically what kind of object it is and he found that you can train these classifiers with far fewer labeled examples learn traditional classifiers


so a few supervised based on also not just your discrimination ability but your ability to classify you're going to do much you're going to convert much faster to being effective at being a discriminator yeah so for example for the emne status set you want to look at an image of a handwritten digit and say whether it's a 0 a 1 or 2 and so on to get down to less than 1% accuracy required around 60,000 examples until maybe about 2014 or so in 2016 with this semi-supervised degan project tim was able to get below 1% error using only a


hundred labeled examples so that was about a 600 X decrease in the amount of labels that he needed he's still using more images in that but he doesn't need to have each of them labeled as you know this one's a 1 this one's a 2 this one's a 0 and so on then to be able to for Ganz to be able to generate recognizable objects so object for a particular class you still need labelled data because you need to know what it means to be a particular class cat dog how do you think we can move away from that yeah some researchers at brain Zurich


actually just released a really great paper on semi-supervised de Gans whether their goal isn't to classify its to make recognizable objects despite not having a lot of label data they were working off of deep minds big gun project and they showed that they can match the performance of began using only 10% I believe of the of the labels big gun was trained on the image net dataset which is about 1.2 million images and had all of them labelled this latest project from brain Zurich shows that they're able to get away with only having about


10% of the of the images labeled and they do that essentially using a clustering algorithm where the discriminator learns to assign the objects to groups and then this understanding that objects can be grouped into you know similar types helps it to form more realistic ideas of what should be appearing in the image because it knows that every image it creates has to come from one of these archetypal groups rather than just being some arbitrary image if you train again with no class labels you tend to get things that look sort of like grass or


water or brick or dirt but but without necessarily a lot going on in them and I think that's partly because if you look at a large image net image the object doesn't necessarily occupy the whole image and so you learn to create realistic sets of pixels but you don't necessarily learn that the object is the star of the show and you want it to be in every image you make yeah you've heard you talk about the the horse the zebra cycle Gann mapping and how it turns out again thought provoking that horses are usually on grass and zebras


are usually on drier terrain so when you're doing that kind of generation you're going to end up generating greener horses or whatever so those are connected together it's not just yeah yeah be able to you're not able to segment yeah it's generating the segments away so there are other types of games you come across in your mind that neural networks can play with each other to to to be able to solve problems yeah the the one that I spend most of my time on is insecurity you can model most interactions as a game where there's


attackers trying to break your system and you order the defender trying to build a resilient system there's also domain adversarial learning which is an approach to domain adaptation that looks really a lot like Ganz the the author's had the idea before the game paper came out their paper came out a little bit later and you know they they're very nice and sighted again paper but I know that they actually had the idea before I came out domain adaptation is when you want to train a machine learning model in 1:1 setting called a


domain and then deploy it in another domain later and he would like it to perform well in the new domain even though the new domain is different from how it was trained so for example you might want to train on a really clean image data set like image net but then deploy on users phones where the user is taking you know pictures in the dark or pictures while moving quickly and just pictures that aren't really centered or composed all that well when you take a normal machine learning model it often degrades really badly


when you move to the new domain because it looks so different from what the model was trained on domain adaptation algorithms try to smooth out that gap and the domain adverse oral approach is based on training a feature extractor where the features have the same statistics regardless of which domain you extracted them on so in the domain adversarial game you have one player that's a feature extractor and another player that's a domain recognizer the domain recognizer wants to look at the output of the feature extractor and


guess which of the two domains oh the features came from so it's a lot like the real versus fake discriminator and ends and then the feature extractor you can think of as loosely analogous to the generator in games except what's trying to do here is both fool the domain recognizer and two not knowing which domain the data came from and also extract features that are good for classification so at the end of the day you can in in the cases where it works out you can actually get features that work about the same in both domains


sometimes this has a drawback where in order to make things work the same in both domains it just gets worse at the first one but there are a lot of cases where it actually works out well on both do you think gas being useful in the context of data augmentation yeah one thing you could hope for with Kenz is you could imagine I've got a limited training set and I'd like to make more training data to train something else like a classifier you could train Magan on the training set and then create more data and then maybe the classifier would


perform better on the test set after training on those big ERG and generated data set so that's the simplest version of of something you might hope would work I've never heard of that particular approach working but I think there's some there's some closely related things that that I think could work in the future and some that actually already have worked so if you think a little bit about what we'd be hoping for if we use the gun to make more training data we're hoping that again we'll generalize to new examples better than the classifier


would have generalized if it was trained on the same buddy at us and I don't know of any reason to believe that the Gann would generalize better than the classifier would but what we might hope for is that the Gann could generalize differently from a specific classifier so one thing I think is worth trying that I haven't personally tried but someone could try is what have you trained a whole lot of different generative models on the same training set create samples from all of them and then train a classifier on that


because each of the generative models might generalize in a slightly different way they might capture many different axes of variation that one individual model wouldn't and then the classifier can capture all of those ideas by training in all of their data so we'd be a little bit like making an ensemble of classifiers and I say oh of gans yeah in a way I think that could generalize better the other thing that gans are really good for is not necessarily generating new data that's exactly like what you already have but


by generating new data that has different properties from the data you already had one thing that you can do is you can create differentially private data so suppose that you have something like medical records and you don't want to train a classifier on the medical records and then publish the classifier because someone might be able to reverse-engineer some of the medical records you trained on there's a paper from Casey greens lab that shows how you can train again using differential privacy and then the samples one again


still have the same differential privacy guarantees as the parameters that again so you can make fake patient data for other researchers to use and they can do almost anything they want with that data because it doesn't come from real people and the differential privacy mechanism gives you clear guarantees on how much the original people's data has been protected that's really interesting actually I haven't heard you talk about that before in terms of fairness I've seen from triple AI your talk how can an adversarial machine learning


help models be more fair with respect to sensitive variables yeah there was a paper from Amos Torquay's lab about how to learn machine learning models that are incapable of using specific variables so to say for example you wanted to make predictions that are not affected by gender it isn't enough to just leave gender out of the input to the model you can often infer gender from a lot of other characteristics like say that you have the person's name but you're not told their gender well right if if their name


is Ian they're kind of obviously a man so what you'd like to do is make a machine learning model that can still take in a lot of different attributes and make a really accurate informed prediction but be confident that it isn't reverse engineering gender or another sensitive variable internally you can do that using something very similar to the domain adversarial approach where you have one player that's a feature extractor and another player that's a feature analyzer and you want to make sure that the feature


analyzer is not able to guess the value of the sensitive variable that you're trying to keep private right that's yeah I love this approach so we'll yeah with the with the feature you're not able to infer right this sensitive variables yeah brilliant it's quite quite brilliant and simple actually another way I think that Ganz in particular could be used for fairness would be to make something like a cycle again where you can take data from one domain and convert it into another we've seen cycle again turning horses into zebras we've


seen other unsupervised gains made by Ming Yue Lu doing things like turning day photos into night photos I think for fairness you could imagine taking records for people in one group and transforming them into analogous people in another group and testing to see if they're they're treated equitably across those two groups there's a lot of things that be hard to get right to make sure that the conversion process itself is fair and I don't think it's anywhere near something that we could actually use yet but if you could design that


conversion process very carefully it might give you a way of doing audits where you say what if we took people from this group converted them into equivalent people in another group does the system actually treat them how it ought to that's also really interesting you know in a popular in popular press and in general in our imagination you think well gangs are able to generate data and use to think about deep fakes or being able to sort of maliciously generate data that fakes the identity of other people is this something of a concern to you is


this something if you look 10 20 years into the future is that something that pops up in your work in the work of the community that's working on generating models I'm a lot less concerned about 20 years from now than the next few years I think there will be a kind of bumpy cultural transition as people encounter this idea that there can be very realistic videos and audio that aren't real I think 20 years from now people will mostly understand that you shouldn't believe something is real just because you saw a video of it people


will expect to see that it's been cryptographically signed or or have some other mechanism to make them believe the the content is real there's already people working on this like there's a startup called true pic that provides a lot of mechanisms for authenticating that an image is real there they're maybe not quite up to having a state actor try to to evade their their verification techniques but it's something people are already working on and I think we'll get right eventually so you think authentication will will


eventually went out so being able to authenticate that this is real and this is not yeah as opposed to gas just getting better and better or generative models being able to get better and better to where the nature of what is real I don't think we'll ever be able to look at the pixels of a photo and tell you for sure that it's real or not real and I think it would actually be somewhat dangerous to rely on that approach too much if you make a really good fake detector and then someone's able to fool your fake detector and your


fake detector says this image is not fake then it's even more credible than if you've never made a fake detector in the first place what I do think we'll get to is systems that we can kind of use behind the scenes for to make estimates of what's going on and maybe not like use them in court for a definitive analysis I also think we will likely get better authentication systems where you know if a match every phone cryptographically signs everything that comes out of it you wouldn't go to conclusively tell that an


image was real but you would be able to tell somebody who knew the appropriate private key for this phone was actually able to sign this image and upload it to this server at this timestamp so you could imagine maybe you make phones that have the private keys Hardware embedded in them if like a State Security Agency really wants to infiltrate the company they could probably you know plant a private key of their choice or break open the chip and learn the private key or something like that but it would make it a lot harder for an adversary with


fewer resources to fake things most of us yeah okay okay so you mentioned the beer and the bar and the new ideas you were able to implement this or come up with this new idea pretty quickly and implement it pretty quickly do you think there are still many such groundbreaking ideas and deep learning that could be developed so quickly yeah I do think that there are a lot of ideas that can be developed really quickly guns were probably a little bit of an outlier on the whole like one-hour timescale right but just in terms of a like low resource


ideas where you do something really different on the algorithm scale and get a big payback I think it's not as likely that you'll see that in terms of things like core machine learning technologies like a better classifier or a better reinforcement learning algorithm or a better generative model if I had the gun idea today it would be a lot harder to prove that it was useful than it was back in 2014 because I would need to get it running on something like image net or celibate high resolution you know those take a while to train you couldn't


you couldn't train it in an hour and know that it was something really new and exciting back in 2014 shredding an amnesty was enough but there are other areas of machine learning where I think a new idea could actually be developed really quickly with low resources what's your intuition about what areas of machine learning are ripe for this yeah so I think fairness and interpretability our areas where we just really don't have any idea how anything should be done yet like for interpretability I don't think we even have the right definitions and


even just defining a really useful concept you don't even need to run any experiments could have a huge impact on the field we've seen that for example in differential privacy that uh Cynthia Dworkin her collaborators made this technical definition of privacy where before a lot of things are really mushy and then with that definition you could actually design randomized algorithms for accessing databases and guarantee that they preserved individual people's privacy in a in like a mathematical quantitative sense right now we all talk


a lot about how interpretable different machine learning algorithms are but it's really just people's opinion and everybody probably has a different idea of what interpretability means in their head if we could define some concept related to interpretability that's actually measurable that would be a huge leap forward even without a new algorithm that increases that quantity and also once once we had the definition of differential privacy it was fast to get the algorithms that guaranteed it so you could imagine once we have


definitions of good concepts and interpretability we might be able to provide the algorithms that have the interpretability guarantees quickly to what do you think it takes to build a system with human level intelligence as we quickly venture into the philosophical so artificial general intelligence what do you think I I think that it definitely takes better environments than we currently have for training agents that we want them to have a really wide diversity of experiences I also think it's going to take really a lot of computation it's


hard to imagine exactly how much so you're optimistic about simulation simulating a variety of environments is the path forward I think it's a necessary ingredient yeah I don't think that we're going to get to artificial general intelligence by training on fixed datasets or by thinking really hard about the problem I think that the the agent really needs to interact and have a variety of experiences within the same lifespan and today we have many different models that can each do one thing and we tend to train them on one data set or one RL


environment sometimes they're actually papers about getting one set of parameters to perform well in many different RL environments but we don't really have anything like an agent that goes seamlessly from one type of experience to another and and really integrates all the different things that it does over the course of its life when we do see multi agent environments they tend to be there are so many multi environment agents they tend to be similar environments like all of them are playing like an action based video


game we don't really have an agent that goes from you know playing a video game to like reading The Wall Street Journal to predicting how effective a molecule will be as a drug or something like that what do you think is a good test for intelligence in you view it's been a lot of benchmarks started with the with Alan Turing a natural conversation being good being a good benchmark for intelligence what what are what would you and good fellows sit back and be really damn impressed if a system was able to accomplish something that doesn't take a


lot of glue from human engineers so imagine that instead of having to go to the CFR website and download CFR 10 and then write a Python script to parse it and all that you could just point an agent at the CFR 10 problem and it downloads and extracts the data and trains a model and starts giving you predictions I feel like something that doesn't need to have every step of the pipeline assembled for it it definitely understands what it's doing is Auto ml moving into that direction are you thinking wave and bigger autosomal has


mostly been moving toward once we've built all the glue can the machine learning system to design the architecture really well so I'm we're saying like if something knows how to pre-process the data so that it successfully accomplishes the task then it would be very hard to argue that it doesn't truly understand the task in some fundamental sense and I don't necessarily know that that's like the philosophical definition of intelligence but that's something that would be really cool to build that would be really useful and would impress me


and would convince me that we've made a step forward in real AI so you give it like the URL for Wikipedia and then next day expected to be able to solve CFR 10 or like you type in a paragraph explaining what you want it to do and it figures out what web searches it should run and downloads all the whole unnecessary ingredients so you have a very clear calm way of speaking no arms easy to edit I've seen comments for both you and I have been identified as both potentially being robots if you have to prove to the world that you are indeed


human how would you do it but I can understand thinking that I'm a robot it's the flipside yeah touring test I think yeah yeah the proof prove your human test I mean I lecture so you have to is there something that's truly unique in your mind I suppose it doesn't go back to just natural language again just being able to so proving proving that I'm not a robot with today's technology yeah that's pretty straightforward too like my conversation today hasn't veered off into you know talking about the stock market or something because in my


training data but I think it's more generally trying to prove that something is real from the content alone it was incredibly hard that's one of the main things I've gotten out of my can research that you can simulate almost anything and so you have to really step back to a separate channel to prove that slang is real so like I guess I should have had myself stamped on a blockchain when I was born or something but I didn't do that so according to my own research methodology there's just no way to know at this point so what


last question problem stands all for you that you're really excited about challenging in the near future so I think resistance to adversarial examples figuring out how to make machine learning secure against an adversary who wants to interfere it in control with it is one of the most important things researchers today could solve in all domains in image language driving in I guess I'm most concerned about domains we haven't really encountered yet like like imagine twenty years from now when we're using advanced day eyes to do


things we haven't even thought of yet like if you ask people what are the important problems in security of phones in in like 2002 I don't think we would have anticipated that we're using them for you know nearly as many things as we're using them for today I think it's going to be like that with AI that you can kind of try to speculate about where it's going but really the business opportunities that end up taking off would be hard to predict ahead of time well you can predict ahead of time is that almost anything you can do with


machine learning you would like to make sure that people can't get it to do what they want rather than what you want just by showing it a funny QR code or a funny input pattern and you think that the set of methodology to do that can be bigger than you want domain and that's I think so yeah yeah like one methodology that I think is not not a specific methodology but like a category of solutions that I'm excited about today is making dynamic models that change every time they make a prediction so right now we tend to train models and


then after they're trained we freeze them and we just use the same rule to classify everything that comes in from then on that's really a sitting duck from a security point of view if you always output the same answer for the same input then people can just run inputs through until they find a mistake that benefits them and then they use the same mistake over and over and over again I think having a model that updates its predictions so that it's harder to predict what you're going to get will make it harder for the for an


hi there so you may have seen this already there's a cvpr paper called pulse and what it does is it's a method to up sample a pixelated image in a way that makes it look realistic but also that the again down sampled variant matches the original down sampled image so it's kind of a cycle consistency loss together with a again and all in all it's a method to demonstrate how you could do this now this has been trained on this face dataset among others there it was a user Bomb Z that made this into a collapse of

[Drama] Yann LeCun against Twitter on Dataset Bias


people could try it out and tweet it this out and as you can see it works pretty nicely it gives pretty nice results on this particular data set but of course people started playing around with it and gave fairly funny results like this or that that gets more into the horrible category these so you can see these ones I particularly liked from being made into the little child so you can see as soon as you get away from the original kind of dataset modality you are going to get these results that are off and people started to notice that so


here you input Barack Obama and what comes out is a fairly standard Caucasian person someone tweeted out saying this image speaks volumes about the dangers of bias in AI I guess here is where the entire story starts so young Luca weighs in and says ml systems are biased when data is biased this face up sampling system makes everyone look white because the network was pre trained on flick face HQ which mainly contains white people picks train the exact same system on a date set from Senegal and everyone will look African so this is pointing


out why this happens namely because the data set is mainly Caucasian people so the results of up sampling are going to be mainly Caucasian people I mean this is like a straightforward explanation of why we're seeing what we're seeing but of course this was not okay and here is where the piling starts as an interjection we have to talk about bias in machine learning technically there is a statistical notion of bias which has a very rigorous definition and there is the societal definition of bias and these two things even though they're the


same word they're totally different a machine learning system mainly consists of four different parts there is a data set the model the loss function and the optimization procedure statistical bias means whenever the model the loss or the optimization procedure leads to a situation where the outcome doesn't reflect the distribution of the data that you input this for example is achieved when you regularize your model which means that you put some prior knowledge onto the model you introduce bias and therefore you choose to not


accurately represent your data distribution regularize it to a more bias distribution that in turn has lower variance we know this as the bias-variance tradeoff it's actually very simple right you you have the Ferraris and the Lamborghinis and you want make a model that predicts the accident probability now it just so happens that the Ferrari drivers are a bit more reckless and they do slightly higher accidents and now I train my logistic regression and it tells me okay 6040 cool but now I train my logistic regression with an l1 penalty and I say


I want my model to be you know explainable so I wanted to be sparse I want the least amount of variables to be contributing to it what's the model gonna say the models gonna say Ferrari drivers add Lamborghini drivers good societal bias and machine learning is way different an example for this is when face detection systems work well on Caucasian people but don't work so well faced with people from other Heritage's and these societal biases are in the dataset as young account points out here if you change the dataset


you'll change these biases notably the societal biases can only be in the data set otherwise you'd have to argue something like logistic regression itself has a preference for white people or something like this now there is a considerable interaction effect between the two but as Jung Lacan points out the actual societal bias of the final system is a direct result of the bias in the dataset and he is very correct if you train that system on a different data set it will exhibit different biases societal bias cannot be


in the other parts of the machine learning pipeline they can serve to exaggerate or mitigate that bias in the data set but they themselves can only be statistically biased and not societally biased but ya'know can make the terrible mistake of pinpointing the exact root cause of this problem and not addressing the I guess wider ranging problems in the field as some people perceive it and he shouldn't have to write he pretty clearly says this is why it happens we can solve it by swapping the dataset he doesn't say anything about anything else


namely he doesn't say that general bias in the field is not a problem he doesn't say that this doesn't harm anyone none of that he simply suggests a solution Jonathan Peck says well yes that's the point ml researchers need to be more careful selecting their data so that they don't encode biases like this and Lacan responds with not so much ml researchers but ml engineers the consequences of bias are considerably more dire in a deployed product than in an academic paper which is also correct this paper was about the method showing


that this method works on this dataset now assume if here makes a interesting point which I agree with saying that today ml researchers are inadvertently powering product of a lot of non-ai companies who ignorant lis start with a pre trained birth or ResNet or Yola from the internet probably ignoring the license readme and so on which is a valid point right there are going to be people that take this and think oh this is a face up sampler cool I can use that without noting that this is simply an example and implementation


on an example data set so you can argue that there might be some responsibilities of the researchers right here that doesn't make yung Lacan not correct but I still consider this to be like a fruitful discussion between individuals right here but now we go on this person saying train it on the whole American population with an l2 Lawson almost everyone will look white or train it on the whole American population with an l1 loss and more people might look black stop pretending that bias does not also come from algorithmic choices young


the car never says it doesn't right the car responds now saying the most efficient way to do it though is to equalize the frequencies of the categories of samples during training this forces the network to pay attention to all the relevant features for all the sample categories and training with an l1 instead of an l2 will not even begin to solve the problem I would pretty much argue training with an l1 loss here would exacerbate the problem because the l2 loss is much more sensitive to outliers drawl Sutton says serious


question why do you feel that it's important to make this point are you worried that people are going to start suing cycle gang and Lacan says because people should be aware of this problem and know its cause so they can fix it how terrible yawn how terrible you dare pinpoint the exact cause of the problem so that people can fix it the correct thing to do is to point out that everything is problematic so Tim the giver says Jung I suggest you watch me and Emily's tutorial or a number of scholars who are expert in


this area you can't just reduce harms to dataset bias for once listen to us people from marginalized communities and what we tell you if not now during worldwide protests not sure when so again I feel the argument here is that you can't simply point out that it's the data set bias you must point out the bigger problems which the on account does not ever deny he simply says this particular problem can be solved by switching the data set Nikola LaRue says Jung was in my PhD jury I am indebted for him for everything he taught me but this


constant dismissal of the harms caused directly or indirectly by the m/l community is highly problematic where or when have I dismissed the harm caused by the m/l community I'm pointing out the cause of the harm so it can be fixed you can't fix the harm unless you know what causes it know the roux says causes of the biases are numerous only pointing out data set bias deflects the attention away from the other more pervasive ones that make the whole field of bias in ml many people try to your attention about these issues but


you kept focus on the data set because the dataset is the problem right here he doesn't dismiss any of the other things he simply says here the data set is the problem if your problem is that it doesn't work as well for non-caucasian people which was never the intent of this the intent of this was to showcase the method I mean imagenet is like 60% dog species and still people trained on it to showcase their image recognition techniques no one training on image net makes a claim that they have solved computer vision for all the


classes in the world in a fair manner Tim McGee Bru goes on saying I'm sick of this framing tired of it many people have tried to explain many scholars listen to us you can't just reduce the harms caused by ML to dataset bias doesn't do that doesn't do it so someone asks her is he engaging in any ways with you it's appalling to see that he answers to everybody but you yet maybe there is a conversation going on in private and I don't want to jeopardize it note that young Lacoste tweet has 500 retweets 1.9 K likes and comments as far


as you can scroll to what she responds to with yep but I'm used to white men refusing to engage with black and brown women even on issues of bias that mostly affect us I mean he literally has ignored a whole body of work by people from that demographic hence the statement so not surprised I mean in absence of the fact that an argument should be independent of the person making the argument that is a low blow heart Meru says I respectfully disagree with Yun here as long as progress is benchmarked unbiased data such biases


will also be reflected in the inductive biases of ML systems advancing ml with biased benchmarks and asking engineers to simply retrain models with unbiased data is not helpful I don't disagree with you here I don't think my tweet contradicts your statement which it doesn't people are reading into this because he doesn't conform to the orthodoxy of pointing out that everything and everything is problematic and the pinpoints a particular problem he must be thinking all the wrong things Jeff Dean says this is a clear example


here is an illustration that seemingly minor choices in learning algorithms or loss can have significant effects so bias in ML systems is about much more than just avoid data bias ml researchers and practitioners must pay attention to these issues and I think they are and Lacan doesn't say anything against that he says as I point out in my comment to this tweet is much more efficient to correct this kind of bias note that Yann Lacan actually differentiates between the different kinds of biases by equalizing the


frequencies of categories of samples during training than be hacking the loss function correct because if you hack the loss function you're trying to counter one kind of bias by another kind of bias Meredith Whittaker says this is very racist and even if it recognized non-white people it would be very racist this is Coptic it's designed to allow those with power to surveil and control those with less power diverse training sets aren't going to fix it advocating that we should never build these systems and that's a discussion to be had but


let me break this to you this isn't going to help the cops this isn't actually giving you the face of the person that was down pixeled this is simply going to give you the most likely face associated with that down pixel picture given the dataset the algorithm was trained on I don't see this whenever any machine learning algorithm does anything with faces at all people jumping up going like this is cop technology well in line with all the broader impact statement advice can't it also be used to find lost children from


very very bad security camera footage and if I already mentioned that this doesn't actually give you back the person on the down stamp old image it will give you back the most likely person given the data set so with that I want to conclude this section please stop the witch hunting young account made a completely fine tweet here and there's no reason why people should pile on him this hard he doesn't dismiss any of the other problems just because he doesn't mention them and while we all enjoy a good discussion


where people disagree genuinely is not helpful to accuse him of things he never said or meant I mean where does this all lead the result of this is going to be that small labs that don't have the resources to collect their own data sets or check for all the possible biases in their models that are reliant on the data sets that we do have even if they are biased and flawed we'll just be disincentivized from publishing their code or actually doing research at all so this as every other additional constraint on research is going to help


the large corporations with lots of money and maybe that's just my opinion but we should be able to just talk about a problem and the solution to it without always having to make sure that we rabble down all the different things that are and might be wrong according to the Canada and big props to young lecounte here for holding his own 90% of people by now would probably be like oh yes I'm so sorry I did a not thoughtful comment blah blah blah props tyrion keep going and with that I conclude this section let me know what


It's a scientific fact that the hormones of stress downregulate genes and create disease. Long-term effects. Human beings because of the size of the neocortex, we can turn on the stress response just by thought alone as I think about our problems and turn on those chemicals That means then our thoughts Could make us sick So if it's possible, that our thoughts could make us sick then it is possible then our thoughts could make us well, the answer is absolutely yes Everybody welcome to Impact Theory our goal with this show and company is to introduce you to the people and ideas that will help you

How To BRAINWASH Yourself For Success & Destroy NEGATIVE THOUGHTS! | Dr. Joe Dispenza


Actually execute on your dreams Alright today's guest is a New York Times bestselling author and one of the most sought-after speakers in the world He's lectured and given advanced workshops in more than 30 countries Across five continents all with the aim of helping people better understand and unlock the power of their mind His expertise is the intersection of the fields of neuroscience Epigenetics and quantum physics and he's partnered with other scientists across multiple disciplines to perform extensive research on the effects of meditation


Using advanced technologies such as epigenetic testing brain mapping with EEG s and gas-discharge visualization technology. Through his work He is endeavouring to help advance both the scientific community and the public at large as understanding of mind derived health optimization, a topic he covered extensively in his groundbreaking book, You are the placebo. His teaching has had such a profound impact on the way that people perceive a wide range of brain related topics around Mindfulness and well-being that he's a faculty member at the quantum University in Hawaii the Omega Institute for holistic studies in New York


And the Kerr Paulo Centre for yoga and health in Stockbridge, Massachusetts He's also an invited chair of the research committee at life University in Atlanta As well as a corporate consultant where he delivers his lectures and workshops for businesses So, please help me in welcoming the man who has appeared in such films as Heal, People versus the state of illusion and Unleashing creativity The author of the recent book Becoming supernatural. Dr. Joe Dispenza Thanks for being here So, diving into your world and how you perceive the sense of self and


the way that you marry science to - the way that we form memories the way that we live in a perpetual state of Reliving our past and things like that It's really, really incredible and I want to dive into the whole notion of you sort of being a habitual Construct like what? What is that? What is the habit of you? Well a habit is a redundant set of Automatic unconscious thoughts, behaviors and emotions that's acquired through repetition The habit is when you've done done something so many times that your body now knows how to do it better than your mind


So if you think about it people wake up in the morning they Begin to think about their problems Those problems are circuits, memories in the brain, each One of those memories are connected to people and things at certain times and places and if the brain is a record of the past The moment they start their day, they're already thinking in the past. Each one of those memories has an emotion Emotions are the end product of past experiences So the moment they recall those memories of their problems, they all of a sudden feel unhappy, they feel sad, they feel pain


Now how you think and how you feel creates your state of being. So the person's entire State of being when they start their day is in the past. So what does that mean? The familiar past will sooner or later be predictable future so if you believe that your thoughts have something to do with your destiny and You can't think greater than how you feel Or feelings have become the means of thinking by very definition of emotions you're thinking in the past And for the most part you're going to keep creating the same life, so then people grab their cell phone


They check their WhatsApp. They check their texts. They check their emails. They check Facebook They take a picture of their feet. They post it on Facebook. They tweet something, they do Instagram they check the news and now they feel really connected to everything that's known in their life And then they go through a series of routine behaviors They get out of bed on the same side. They go to the toilet. They get a cup of coffee They take a shower, they get dressed, they drive to work the same way. They do the same things


They see the same people that pushed the same emotional buttons and that becomes the routine and it becomes like a program So now they've lost their free will To a program and there's no unseen hand doing it to them. So when it comes time to change the Redundancy of that cycle becomes a subconscious program. So now 95% of who we are by the time we're 35 years old is a Memorized set of behaviors, emotional reactions, unconscious habits, hardwired attitudes, beliefs and perceptions that function like a computer program


So then person can say with their five percent of their conscious mind. I want to be healthy I want to be happy. I want to be free but the body's on a whole different program So then how do you begin to make those changes? Well? you have to get beyond the analytical mind because what separates the conscious mind from the Subconscious mind is the analytical mind and that's where meditation comes in because you can teach people through practice how to change their brainwaves, slow them down and when they do that


Properly they do enter the operating system where they can begin to make some really important changes. So Most people then wait for crisis or trauma or disease or diagnosis, you know, they wait for loss some tragedy to make up their mind to change and my message is why wait and and You can learn and change in a state of pain and suffering or you can learn and change in a state of joy and inspiration I think right now the cool thing is that people are waking up that's really interesting and where I found the


the deepest hooks into how powerful this can be for somebody is when you talk about trauma and you've talked about how People experience a traumatic event, but they then basically rehearse it and how that then has this knock-on effect. So, what is that? Why do people find it so hard to get past trauma? Well? the the stronger the emotional reaction You have to some experience in your life the higher the emotional quotient The more you pay attention to the cause and the moment the brain puts all of its attention on the cause


It takes a snapshot and that's called a memory. So long-term memories are created from very highly Emotional experiences. So what happens then is that people think neurologically within the circuitry of that experience and they feel chemically within the boundaries of those emotions and So when you have an emotional reaction to someone or something most people think that they can't control their emotional reaction Well, it turns out if you allow that emotional reaction, it's called a refractory period to last for hours or days


That's called the mood. I say to someone. Hey, what's up think so I'm gonna move well, why are you in a mood? well I had this thing happen to me five days ago and I'm having one long emotional reaction if you keep that same emotional reaction going on for weeks or months That's called temperament. Why is he so bitter? I don't know. Let's ask him. Why is he so bitter? Why are you bitter? Well, I had this thing happened to me nine months ago And if you keep that same emotional reaction going on for years on end that's called a personality trait


And so learning how to shorten your refractory period of emotional reactions is really where that work starts So then people when they have an event what they do is they keep recalling the event because the Emotions of stress hormones the survival emotions are saying pay attention to what happened Because you want to be prepared if it happens again Turns out most people spend 70% of their life living in survival and living in stress. So they're they're always Anticipating the worst-case scenario based on a past experience and they're literally out of the infinite


potentials in the quantum field they're selecting the worst possible outcome and they're beginning to emotionally embrace it with fear and their Conditioning their body into a state of fear do that enough times Body has a panic attack without you you you can't even predict it because it's programmed subconsciously So then you say to the person why are you this way? And they'll say I am this way because of this event that happened to me 15 or 20 years ago and what that means from biological standpoint is that they haven't been able to change since that event


So then the emotions from the experience tend to give the body and the brain a rush of energy So people become addicted To the rush of those emotions and they use the problems and conditions in their life to reaffirm their limitation So at least they can feel something. So now when it comes time to change you say the person why are you this way? Well, every time they recall the event they're producing the same chemistry in their brain and body as if the event is occurring firing and wiring the same circuits and


Settings the same emotional signature to the body. Well, what's the revelant behind that? Well your body is the unconscious mind It doesn't know the difference between the experience that's creating the emotion and the emotion that you're creating by thought alone So the body's believing it's living in the same past experience 24 hours a day seven days a week 365 days a year and so then when those emotions influence certain thoughts and they do and Then those thoughts create the same emotions and those same emotions influence the same thoughts


Now the entire person's state of being is in the past. So then the hardest part about change is not making the same choice as you did the day before a period and The moment you decide to make a different choice get ready because it's going to feel uncomfortable It's going to feel unfamiliar. It's there's gonna be something so why does it feel so uncomfortable? Is it because of the the neurons that fire together wire together so I've there's like an Easiness to that loop just because literally and you've talked very eloquently


about this the way that the neurons connect in the brain how rapidly I've seen you show footage of how Rapidly those connections happen, which is pretty incredible Is is that what makes it so? discomforting for people I think that I think that the bigger thing is that we we keep Firing and wiring those circuits they become more hardwired. So there you have a thought and then the program runs but it's the emotion that follows the thought if you have a if you have a Fearful thought you're gonna feel anxiety the moment you feel anxiety your brains checking in with your body and saying yeah, you're pretty anxious


so then you start thinking more corresponding thoughts equaled how you While the redundancy of that cycle conditions the body to become the minds. So now when it comes time to change Person's steps into that river of change and they make a different choice in all of a sudden They don't they don't feel the same way So the body says well you've been doing this for 35 years Well, you're gonna just stop feel suffering and stop feeling guilty and stop feeling shameful and you're not gonna complain or blame or make excuses


Or feel sorry for yourself long The body's in the unknown so the body says I want to return back to familiar territory so the body starts influencing the mind then it says Start tomorrow, you're too much like your mother. You'll never change. This isn't gonna work for you. This doesn't feel right And so if you respond to that thought as if it's true that same thought will lead to the same choice Which will lead to the same behavior, which will create the same experience which produce the same emotion I want to talk about that notion of


Give me a little more detail. We mean by the body becomes the mind or the unconscious mind. What do you mean by that exactly? Well, those are two different things your body is your unconscious mind in a sense if you're sitting down and you start thinking about Some future worst-case scenario that you're conjuring up in your mind and you begin to feel the emotion of that event your body doesn't know the difference between The event that's taking place in your world outer world and what you're creating by emotion or thought alone. So most people then


They're they're constantly reaffirming their emotional states So when it comes time to give up that emotion they can say I really want to do it but really the body is stronger than the mind because it's been conditioned that way so The servant now has become the master and the person all of a sudden once they step into that unknown They'd rather feel guilt and suffering because at least they can predict it being in the unknown Is a scary place for most people because the unknown is uncertain people say to me. Well, I can't predict my future


I'm in the unknown and I always say the best way to predict your futures have created Not from the known but from the unknown what thoughts? Do you want to fire and wire in your brain? what behaviors do you want to demonstrate in one day the act of rehearsing the mentally closing your eyes and rehearsing the action the rehearsing the reaction of what you want or the action of what you want by closing your eyes and mentally rehearsing some action if You're truly present. The brain does not know the difference between what you're imaging and what you're experiencing in 3d world


so then you begin to install the Neurological Hardware in your brain to look like the event has already occurred Now your brain is no longer a record of the past now It's a map to the future and if you keep doing it priming it that way the hardware becomes a software program and who knows you just may start acting like a happy person and then I think the hardest part is To teach our body emotionally what the future will feel like ahead of the actual experience. So, what does that mean? You can't wait for your success to feel empowered. You can't wait for your wealth to feel abundant


you can't wait for your your new relationship to feel love or Your healing to feel whole I mean that's the old model of reality of cause and effect, you know Waiting for something outside of us to change how we feel inside of us and when we feel better inside of us We pay attention to ever or whatever caused it But what that means then is that from the Newtonian world that most people spend their whole life living in lack Will hitting through something to change out their what do you mean the Newtonian world?


Newtonian world is all about the predictable It's all about predicting the future But the quantum model of reality isn't is about causing an effect the moment you start feeling Abundant and worthy you are generating wealth the moment you're empowered and feel it You're beginning to step towards your success the moment. You start feeling whole Your healing begins and when you love yourself and you love all of life You'll create an equal and now you're causing an effect and I think that's that the difference between living as a victim in


Your world saying I am this way because of this person or that thing or this experience They made me think and feel this way when you switch that around you become a creator of your world and you start My thinking and my feeling is changing an outcome in my life And now that's a whole different game and we start believing more that were creators of reality. So, how do we go from? Okay, I have this negative emotion. It's controlling my life. It's got me in this cycle of I think about this emotion which triggers a chemical reaction which trains my body to feel that way which makes it easier more likely I will do it again and


so now I'm in this vicious line unconscious and it's unconscious right and you You said does your thinking create your environment orders your environment create your thinking which I thought was really really interesting. So how do we then go from that like mechanistically To begin this visualization process of something that's empowering its me in a different state. It's my future self Is it meditation is it what does that look like if you're not being defined by a vision of the future? Then you're left with the old memories of the past and you will be predictable in your life


and If you wake up in the morning and you're not being defined by a vision in the future as you see the same people and you go to the same places and You do the exact same thing at the exact same time It's no longer that your personality is creating your personal reality Now your personal reality is affecting or creating your personality Your environment is really controlling how you think and feel? unconsciously because every person every thing every place every experience has a neurological network in your brain every


Experience that you have with every person produces an emotion. So some people will use their boss to reaffirm their addiction to judgment They'll use their enemy to reaffirm their addiction to hatred to use their friends that we affirm their addiction to suffering So now they need the outer world to feel something. So To change them is to be greater than your environment to be greater than the conditions in your world and the environment Is that seductive so then why is meditation the tool well? Let's sit down. Let's close our eyes. Let's disconnect


from your outer environment So if you're seeing less things is less stimulation going to your brain if you're playing soft music or you have earplugs in Less sensory information coming to your brain. So you're disconnecting from environment if you can sit your body down and Tell it to stay like an animal stay right here. I'm gonna feed you when we're done You can get up and check your emails You can do all your texts, but right now you're gonna sit there and obey me So then when you do that properly and the you're not eating anything or smelling anything or tasting anything?


You're not up experiencing and feeling anything. You would have to agree with me that you're being defined by a thought, right? So when the body wants to go back to its emotional past And you become aware that your attention is on that emotion And where you place your attention is where you place your energy? you're siphoning your energy out of the present moment into the past and you become aware of that and You settle your body back down in the present moment because it's saying well, it's eight o'clock You normally get upset because you're in traffic around this time and here you are sitting and we're used to feeling anger and you're off


Schedule. Oh, it's 11 o'clock and usually check your emails and judge everybody. Well, the body is looking for that that predictable chemical state every time you become aware that you're doing that and your body is craving those emotions and You settle it back down into the present moment. You're telling the body it's no longer the mind that you're the mind and now your will is Getting greater than the program and if you keep doing this over and over again over and over again over and over again Just like training a stallion or a dog. It's just gonna say


I'm gonna sit and the moment that happens when the body's no longer the mind when it finally surrenders There's a liberation of energy We go from particle to wave from matter to energy and we free ourselves from the chains Of those emotions that keep us in the in the familiar past and we've seen this Thousands of times. In fact, we can actually predict it now on a brain scan. That's so interesting Let's go a little bit harder on Metacognition the notion that you don't have to believe everything you think I love the way that you talk about that. Hmm


Yeah, and we have a huge frontal lobe. It's 40% of our entire brain and most people when they have a thought they just think that that's the truth and I think one of my greatest Realizations in my own journey was just because you have a thought it doesn't necessarily mean it's true so if you think 60 to 70 thousand thoughts in one day and we do and 90% of those thoughts are the same thoughts as The day before and you believe that your thoughts have something to do with your destiny Your life's not gonna change very much


Because the same thought leads to the same choice the same choice leads to the same behavior the same behavior creates the same experience and the same experience produces the same motion and so then the act of becoming conscious of this process to to begin to become more aware of How you think how you act in how you feel? It's called metacognition and so then why is that important because the more conscious you become of those unconscious states of mind and body the Less likely you're gonna go unconscious during the day and that thought is not gonna slip by your awareness unchecked


because your It means to know thyself and the word meditation means to become familiar with So as you become familiar with the thoughts the behaviors and the emotions of the old self You're retiring that old self as you fire and wire new thoughts and condition the body into a new emotional state if you do that Enough times it'll begin to become familiar to you. So it's so important Just like a garden if you're planting a garden, you've got to get rid of the weeds You got to take the plants from the past year and you got to pull them out the rocks that sift to the top that


Are like our emotional blocks they have to be removed that soil has to be tenderized and broken down We have to we have to make room to plant the new garden So primarily we learn the most about ourselves and others when we're uncomfortable because the moment you move into that uncomfortable state normally a program jumps in When that program jumps ins because the person doesn't want to be in the present moment and engage it consciously So when you teach people how to do that with a meditative process Turns out that when they're in their life


They're less likely to emotionally react they're less likely to be so rigid and believe the thoughts they were thinking they're more aware of when they go unconscious back into a habit and that is what starts the process of change and So we have to unlearn Before we relearn we have to break the habit of the old self before we reinvent a new self We have to pre synaptic connections and sprout new connections. We have to unfired unwire and refire and rewire. We have to unmemorable Body to a new mind into a new emotion like the program and reprogram that's the act and it's a two-step process


Yeah, I like the way that you call that out as an action There was another thing that you said that I thought was really powerful about how insights themselves are essentially inert. They don't do anything What what then do we do with an insight? How do we take a breakthrough moment and make sure that it's not just a breakthrough moment Like I guarantee people watching right now are having like a hundred aha moments for sure That was definitely the case for me as I was researching you and when you said that I was like and that's the danger that


You have the AHA and then nothing. Yeah Yeah and that's it's a it is a danger because then people will will shrink back into mediocracy and they'll use the insight to Excuse them from taking a leap. They'll say yeah, you know, I have a chemical imbalance in my brain. Yeah, my father was Really overbearing he was a perfectionist. That's why I am the way I am you know people they come up with stuff to to excuse themselves. The insight is Actually giving them permission to stay limited and it's an amazing idea because they'll say to you


And they really want to get over their anxiety. But let's ok. Let's take your ex-husband. Let's put him in a straitjacket Let's duct tape them and shoot them to the moon know what I mean. What are you gonna do now? You still have to make those changes. And so then the person's enemy dies or they're something shifts in their life And that person's gone. They'll find another person to hate. This is just how we function as human beings. We just slide another Reason to feel those emotions. So I think I think when people start to understand this, you know,


I think knowledge is power but knowledge about yourself is self empowerment. So how much of this is really learning to? just bifurcate the world into there's negative emotions that have negative neuro chemistry associated with and you said that in those states if you're living in a perpetual state of stress hormones and things like that illness is like a step away and Then just the other side of that is understanding but there's this whole other side of positive energy which happiness joy Empowerment whatever that you know neurochemical cocktail is but that when you're on that side


Your immune system is more likely to function. Well, like is that Just sort of bringing it down to like a really basal. Yeah, that's sort of one of the biggies Well, let's talk about it in terms of survival or creation As I said 70% of the time people live in stress and living in stress is living in survival now All organisms in nature can tolerate short-term stress, you know a deer gets chased by a pack of coyotes when it out runs the Coyotes it goes back to grazing and the event is over and The definition of stress is when your brain and body are knocked out of balance out of homeostasis


The stress response is what the body and Nate Lee does to return itself back to order. So you're driving down the road Someone cuts you off you jam on the brakes You may give them the finger and then you settle back down and the event is over and boom now. Everything's back back to normal But what if it's not a predator that's waiting for you outside the cave, but what if it's your coworker? Sitting right next to you and all day long you're turning on those chemicals because they're pushing all your emotional buttons


When you turn on the stress response, and you can't turn it off Now you're headed for a disease because no organism in nature can live an emergency mode for that extended period of time It's a scientific fact that the hormones of stress down regulate genes and create disease long term affects Human beings because of the size of the neocortex we can turn on the stress response just by thought alone Which I think about our problems and turn on those chemicals That means then our thoughts Could make us sick So if it's possible that our thoughts could make us sick. Is it possible that our thoughts could make us? Well, the answer is absolutely


Yes So then what are the emotions that are connected to survival? Let's name them anger aggression hostility hatred competition fear Anxiety worry pain suffering guilt shame unworthiness the envy jealousy. Those are all Created by the hormones of stressin and psychology calls them normal human states of consciousness I call those altered states of consciousness So then we tend to remember those traumatic events more because in survival, you better be ready if it happens again that's an and in one's survival gene is switched on you could have ten really great things that happen to you in your day and


you just have one bad thing that happens and you cannot take your attention off that bad that that unhappy thing because The survival gene is switched on it's really interesting How does epigenetics come into play and all this like what's actually happening? You've talked pretty profoundly about? Proteins and like really at a deep level how we're signalling to our genetics to create these kinds of changes What does that actually look like? Well epigenetics epi means above the gene and Many years ago after the DNA helix was discovered by Watson and Crick


They said the blueprints of life, you know, all diseases are created from genes it turns out less than 5% more like 1% of people on the planet are born with a genetic condition like type 1 diabetes or Tay-sachs disease or sickle cell anemia the other 95 to 99 percent Are created by lifestyle and by choices you can take to identical twins Exact same genome one dies at 51. The other one dies at 85 same gene different environment, so All of a sudden they said we lied That was wrong. It's not genes that create disease. It's the environment that signals the gene that creates disease. Well, ok, but


That's not the whole truth too because you could have two people working side by side in the same factory One gets cancer after being exposed to a carcinogenic for 25 years both working for 25 years The other one has no cancer at all. So there must be some internal order That would cause one person to not get it while another one does So is it possible then if? The environment signals the gene and it does and the end product of an experience in the environment is called an emotion Can you signal the gene ahead of the environment by embracing an elevated emotion?


We've done the research on this where we measured 7,500 different gene expressions in a group of people it came to an advanced event for four days and we Had them doing a seated meditation a walking meditation a laying down meditation a standing meditation and at the end of four days Just four days The common eight genes that were upregulated two genes to suppress cancer cells and tumor growth Two genes for neurogenesis the growth of new neurons in response to novel experiences and learning the gene that signals stem cells


To go to damaged areas and repair them the gene for oxidative stress was upregulated We started seeing all these genes that are very very healthy to cause the body to flourish Imagine if people were doing that for three months. We also measured telomeres the little Shoestrings on the end of DNA that tell us our biological age. We asked people to Do the work meditation five out of seven days for 60 days Measure their telomeres that determine their biological age sixty days later seventy four percent of the people lengthen their telomeres 40 percent


significant change twenty percent a very remarkable change That means that they got a little bit of their life back if it lengthened by ten percent They got 10% of their life back. That's incredible Before I ask my last question tell these guys where they can find you online Sure. My website is just dr. Joe Dispenza dot-com. You can follow us on Facebook Twitter Instagram We're all over and then my final question. What's the impact that you want to have on the world? I? think that the end game for me is to empower people to


Such a degree that they realize that they need less things outside of them to make them happy less things outside of them to regulate their moods and their behaviors and that they begin to use the kind of the power that we All have access to and into really and to change the world to make a difference so that there's more peace There's more homeless. There's more connection that we support and love each other and we serve better and and I think that we have to start for the most part if everybody's working on themselves and


and Trying doing their best to present the greatest ideal of themselves to the world. I think the world would be a better place. And so That's my passion and I'm witnessing it happening now The more than I ever thought I would was incredible Joe. Thank you so much for being here and amazing having you Guys Go watch this man's videos They are some of the best explanations of what's going on inside the mind that I've ever come across There were of several that I literally have people in my life that I'm going to force to sit down and watch these things


It's just incredible explanations of how you create yourself out of the things you do Habitually the way that you think creates a feeling the way that you feel creates thinking that matches that and then you get in this cycle and that coming down to that personality ultimately being a finite set of patterns in your brain I think is really really illuminating in terms of how we actually experience the world and I think when people understand that that it's within your control that you don't have to believe every thought that you think that you can