after a long day, turns off the key in his car and says, "Aah, I'm home." Because we all know where home is. So this is a piece which goes from away to home. I'm going to play it all the way through and you're going to follow. B, C, B, C, B, C, B -- down to A, down to G, down to F. Almost goes to E, but otherwise the play would be over. He goes back up to B, he gets very excited. Goes to F-sharp. Goes to E. It's the wrong chord. It's the wrong chord. And finally goes to E, and it's home. And what you're going to see is one-buttock playing.


(Laughter) Because for me, to join the B to the E, I have to stop thinking about every single note along the way, and start thinking about the long, long line from B to E. You know, we were just in South Africa, and you can't go to South Africa without thinking of Mandela in jail for 27 years. What was he thinking about? Lunch? No, he was thinking about the vision for South Africa and for human beings. This is about vision. This is about the long line. Like the bird who flies over the field and doesn't care about the fences underneath, all right?

The transformative power of classical music | Benjamin Zander


So now, you're going to follow the line all the way from B to E. And I've one last request before I play this piece all the way through. Would you think of somebody who you adore, who's no longer there? A beloved grandmother, a lover -- somebody in your life who you love with all your heart, but that person is no longer with you. Bring that person into your mind, and at the same time, follow the line all the way from B to E, and you'll hear everything that Chopin had to say. (Music) (Music ends) (Applause) Now, you may be wondering --


(Applause) (Applause ends) You may be wondering why I'm clapping. Well, I did this at a school in Boston with about 70 seventh graders, 12-year-olds. I did exactly what I did with you, and I explained the whole thing. At the end, they went crazy, clapping. I was clapping. They were clapping. Finally, I said, "Why am I clapping?" And one of them said, "Because we were listening." (Laughter) Think of it. 1,600 people, busy people, involved in all sorts of different things, listening, understanding and being moved


by a piece by Chopin. Now, that is something. Am I sure that every single person followed that, understood it, was moved by it? Of course, I can't be sure. But I'll tell you what happened to me in Ireland during the Troubles, 10 years ago, and I was working with some Catholic and Protestant kids on conflict resolution. And I did this with them -- a risky thing to do, because they were street kids. And one of them came to me the next morning and he said, "You know, I've never listened to classical music in my life,


but when you played that shopping piece ..." (Laughter) He said, "My brother was shot last year and I didn't cry for him. But last night, when you played that piece, he was the one I was thinking about. And I felt the tears streaming down my face. And it felt really good to cry for my brother." So I made up my mind at that moment that classical music is for everybody. Everybody. Now, how would you walk -- my profession, the music profession doesn't see it that way. They say three percent of the population likes classical music.


If only we could move it to four percent, our problems would be over. (Laughter) How would you walk? How would you talk? How would you be? If you thought, "Three percent of the population likes classical music, if only we could move it to four percent." How would you walk or talk? How would you be? If you thought, "Everybody loves classical music -- they just haven't found out about it yet." See, these are totally different worlds. Now, I had an amazing experience. I was 45 years old, I'd been conducting for 20 years,


and I suddenly had a realization. The conductor of an orchestra doesn't make a sound. My picture appears on the front of the CD -- (Laughter) But the conductor doesn't make a sound. He depends, for his power, on his ability to make other people powerful. And that changed everything for me. It was totally life-changing. People in my orchestra said, "Ben, what happened?" That's what happened. I realized my job was to awaken possibility in other people. And of course, I wanted to know whether I was doing that. How do you find out?


You look at their eyes. If their eyes are shining, you know you're doing it. You could light up a village with this guy's eyes. (Laughter) Right. So if the eyes are shining, you know you're doing it. If the eyes are not shining, you get to ask a question. And this is the question: who am I being that my players' eyes are not shining? We can do that with our children, too. Who am I being, that my children's eyes are not shining? That's a totally different world. Now, we're all about to end this magical, on-the-mountain week,


we're going back into the world. And I say, it's appropriate for us to ask the question, who are we being as we go back out into the world? And you know, I have a definition of success. For me, it's very simple. It's not about wealth and fame and power. It's about how many shining eyes I have around me. So now, I have one last thought, which is that it really makes a difference what we say -- the words that come out of our mouth. I learned this from a woman who survived Auschwitz, one of the rare survivors. She went to Auschwitz when she was 15 years old.


And ... And her brother was eight, and the parents were lost. And she told me this, she said, "We were in the train going to Auschwitz, and I looked down and saw my brother's shoes were missing. I said, 'Why are you so stupid, can't you keep your things together for goodness' sake?'" The way an elder sister might speak to a younger brother. Unfortunately, it was the last thing she ever said to him, because she never saw him again. He did not survive. And so when she came out of Auschwitz, she made a vow. She told me this.


you hi I'm professor Pete Carr I'm been at the University of Minnesota for about thirty-eight years I've worked with many many graduate students in class and in my research lab and I find it useful to work with students um to teach them how to read a paper on this first slide that I want to show you is an outline of of the way a typical scientific paper is organized and I think most beginning students instinctively start reading a paper in in order as the paper is printed for instance they read the title then they go on to the abstract then

How to Read a Paper Efficiently (By Prof. Pete Carr)


they read the introduction and so on and so forth working their way from the beginning to the end of the article don't do this this is not a good use of your time there's a better way to do things which is what I'm going to tell you about today let me jump in here at this point and tell you about a fairly simple algorithm if you will about how to get the most out of a paper with with the least effort and I think to do this you have to think of reading a paper as a two-phase process in the first phase you're surveying the paper the article


to see if it's really worth investing a lot of time in and this is the way you you keep up with what's going on in in the literature you'll probably have some sort of service that provides you with with papers based upon keywords which are by and large taken from the abstract of of the paper off so the first step to keep more first thing to keep in mind is that you're allowed to stop this process at any point when you become disinterested in going further next you will look undoubtedly at the key words and and the title of the paper if these


don't interest you at all you stop next thing to look at really is the abstract it's the most important part of the paper for getting acquainted with the paper but next I think you want to jump to the conclusions you don't read the intermediate steps you don't look at the experimental and the introduction and the results and discussion look at the conclusions because if the conclusions are not relevant to you probably you don't want to go any further so that this basically at this point you've surveyed the paper and you


know whether or not it's really worth your while to invest any time on it the next thing that I think it's best to do again because it's fairly fast is to take a good look at the tables and the figures and the captions because you can do this quickly and it will tell you the main things that went on when the when the scientists did their work and again it will help you decide do I really want to dig into this paper or not if that's the case and you want to dig in then the place to start is the introduction and


this unit have to start reading seriously and the introduction will provide you with essential background information that's one of the roles of the introduction another role of the introduction is to let you know why the authors of the paper did did the particular study and I think these are important things for you to know before digging in the real heart of a paper is the results and discussion section of the paper here's where you're going to spend most of your time and going through the paper finally at this point you may decide to stop


you've had enough but if the paper is really extremely relevant to what you are currently working on then it's time to dig very deeply and get into the details of the experimental section of the paper and this is where you really learn what the authors actually did but more importantly it's how you it's where you learn how they did things and you may need that level of detail in your own work um once once you've done reading the paper uh you can you can stop however I think it's a really good idea to develop some kind of system


where you take some notes on the paper these notes aren't going to serve you any good next week or maybe even next month but down the road maybe when you start to write your own first paper having some notes on these papers that you've read will be very beneficial and will really save you a lot of time it'll tell you which papers you should reread before you start writing which papers you don't need to include in as references in your own manuscript because they're not relevant so taking some notes when you can as you finish


reading a paper is a really good idea and and these notes should be in a notebook or in some system and not simply written on the PDF of the paper because that you can't collect those those notes very easily you want these notes readily accessible for instance on an index card or a bunch of index cards so that you can flip through them quickly and take a look at all the all the relevant papers there's an old saying I think it's Chinese in origin that the faintest writing is better than the best memory and in the course of my time


you've used a pre-trained model to make predictions that gives you great results if you want to classify images into the categories used by the original models but what if you have a new use case and you don't categorize images in exactly the same way as the categories for the pre trained model for example I might want a model that can tell if a photo was taken in an urban area or a rural area my pre trained model doesn't classify images into those two specific categories we can build a new model from scratch for this specific purpose but to

Transfer Learning | Kaggle


get good results we need thousands of photos with labels for which our urban and which are rural something called transfer learning will give us good results with far less data transfer learning takes what a model learns while solving one problem and applies it to a new application remember that early layers of a deep learning model identify simple shapes later layers identify more complex visual patterns and the very last layer makes predictions so most layers from a pre trained model are useful in new applications because most

Transfer Learning | Kaggle


computer vision problems involve similar low-level visual patterns so we'll reuse most of the pre trained ResNet model and just replace that final layer that was used to make predictions some layers before that in the pre trained model may identify features like roads buildings windows open fields etc will drop in a replacement for the last layer of the rezident model this new last layer will predict whether an image is rural or urban based on the results of that previous layer let's look at this a little closer here we see that the

Transfer Learning | Kaggle


resident model has many layers we cut off the last layer the last layer of what's left has information about our photo content stored as a series of numbers in a tensor it should be a one-dimensional tensor which is also called a vector the vector can be shown as a series of dots each dot is called a node the first node represents the first number in the vector the second node represents the second number and so on practical models have far more nodes than we've on here we want to classify the image into two categories urban and rural so

Transfer Learning | Kaggle


after the last layer we keep the pre-trained model we add a new layer with two nodes one node to capture how urban the photo is and another to capture how rural it is in theory any node in the last layer before prediction might inform how urban it is so the urban measure can depend on all the nodes in this layer we draw connections to show that possible relationship for the same reason the information at each node might affect our measure of how rural the photo is so our structure looks like this we have a lot of

Transfer Learning | Kaggle


connections here and we'll use training data to determine which nodes suggest an images urban which suggests it is rural and which don't matter that is we'll use data to train the last layer of the model in practice that training data will be photos that are labeled as either being urban or rural we'll cover more mathematical detail on this training step in a later video notice that we allow all features from one layer to influence or be connected with a prediction layer when this happens we describe the last layer as

Transfer Learning | Kaggle


being a dense layer one other note when classifying something into only two categories we could get by with only one node at the output in this case a prediction of how urban a photo is would also be a measure of how rural it is if a photo is 80 percent likely to be urban its twenty percent likely to be rural but we've kept two separate nodes at the output layer using a separate node for each possible category in the output layer will help us transition into cases when we want to predict with more than two categories in both the current case

Transfer Learning | Kaggle


and the case with more categories we'll get a score for each category and then apply a function called softmax the softmax function will transform the scores to probabilities so they'll all be positive and I'll sum to one we could then work with those probabilities however we want let's see it in code we'll introduce two new classes from Charis first is sequential this is just saying we're going to have a model that's a sequence of layers one after the other there are some exotic models that don't fit into this structure and

Transfer Learning | Kaggle


we'll get to other types of models later for now all models you would want to build are sequential we'll also want to add a dense layer so we import that in this application we classify photos into two categories or classes urban and rural we'll save that as num classes now we build the model we set up a sequential model that we can add layers to first we add all of a pre-trained ResNet 50 model we've written include top equals false this is how we specify that we want to exclude the layer that makes predictions

Transfer Learning | Kaggle


into the thousands of categories used in the imagenet competition we'll also use a file that doesn't include the weights for that last layer we hand this argument pooling equals average that says that if we had extra channels in our tensor at the end of this step we want to collapse them to a 1d tensor by taking an average across channels we'll come back to intricacies of pooling in a later lesson but now we have a pre-trained model that creates the layer you saw in the graphic will add a dense layers to make predictions

Transfer Learning | Kaggle


we specify the number of nodes in this layer which in this case is the number of classes like we talked about earlier then we say we want to apply the softmax function to turn it into probabilities well tell tens flow not to Train the first layer which is the ResNet 50 model because that's the model that was already pre trained with the imagenet data now we'll get to a more complex line of code in a compile command I'll describe the broad concept here and we'll give a more complete explanation of the underlying theory in a couple

Transfer Learning | Kaggle


videos the compiled command tells tensorflow how to update the relationships in the dense connections when we're doing the training with our data we have a measure of loss or inaccuracy we want to minimize we specify as categorical cross entropy in case you are familiar with the law gloss this is another term for the same thing we use an algorithm called stochastic gradient descent to minimize the categorical cross entropy loss function again we'll cover this in our Theory video we asked it to report the accuracy metric that is what fraction of

Transfer Learning | Kaggle


predictions were correct this is easier to interpret than categorical cross entropy scores so it's nice to print it out and see how the model is doing our raw data is broken into a directory of training data and a directory of validation data within each of those we have one subdirectory for the urban pictures and another for the rural pictures caris provides a great tool for working with images grouped into directories by their label this is the image data generator there's two steps to using image data generator first

Transfer Learning | Kaggle


we'll create any generator object in the abstract I'll tell it that we want to apply the ResNet pre-processing function every time it reads an image you use this function before to be consistent with how the rezident model is created then we use the flow from directory command we tell it what directory that data is in what size image we want how many images to read in at a time and we tell it we're classifying data into different categories well add batch size to our list of information covered in the upcoming theory video for now assume

Transfer Learning | Kaggle


you want categorical class mode almost every time we do the same thing to setup a way to read the validation data that creates a validation generator the image data generator is especially valuable when working with large data sets because we don't need to hold the whole data set in memory at once but it's nice here even with a small data set now we fit the model we tell it the training data comes from trained generator we said to read 12 images at a time and we have 72 images so we'll go through 6 steps of 12 images then we say that

Transfer Learning | Kaggle


validation data comes from validation generator validation generator reads 20 images at a time and we have 20 images of validation data so we can use just one step as the model training is running well see progress updates showing with our loss function and the accuracy it updates the connections in the dense layer that is the models impression of what makes an urban photo and what makes a rural photo and it makes those updates in six steps when it's done it got 79 percent of the training data right then it examines the validation data it gets

Transfer Learning | Kaggle


95 percent those right 19 out of 20 we trained on 72 photos you could easily take that many photos on your phone upload them to kaggle and build a very accurate model to distinguish almost anything you care about I think that's incredibly cool this may feel like a lot of new ideas for you to take in here's our plan we have one exercise for you to build a model yourself using chance for learning after you've done transfer learning hands-on I'll show you a simple but powerful trick called data augmentation data augmentation really

Transfer Learning | Kaggle


[Music] Tocantins the way human whip [Music] [Music] with the powers code we are able to create technology that solves some of the most fundamental issues with humans in the first place we can cure diseases we can solve world problems we can do so much that we never put down without the power of technology feels like home in terms of conferencing I like the format a lot it's very the audience is more technical than usual so I definitely appreciate this conference as a sort of contrast to what usually takes place in many venues

WeAreDevelopers Congress Vienna 2019 Aftermovie


[Music] I'm a programmer myself you know so basically talking to other developers is the next experience just to see you know how other people are dealing with the same maybe constraints that you are dealing to [Music] but I was very amazing I really like the location so it's really really cool to be at Hope work [Music] [Applause] both changes the way people think and people find in Kord impacts the way that different social challenges can be solved [Music] [Applause] and the cause changes the way we consume


supervised learning updates the parameters of a neural network to match predicted class labels with the ground truth labels the construction of these ground truth class vectors is typically done with one hot encoding but other techniques such as label smoothing and knowledge distillation have been developed to overcome the limitations of one hot encoded ground truth class labels Mehta pseudo labels that uses the meta learning framework to dynamically adapt the target distribution or ground truth class labels throughout training

Meta Pseudo Labels


of a student classification network to maximize its resulting validation set accuracy this is done by training the classification model or student network on pseudo labels labeled by a teacher network the teacher network is then updated to maximize the classification models accuracy on the validation set after it trains and updates itself through back propagation supervised learning on the pseudo labels from the teacher network this involves an interesting gradient through a gradient operation to train the teacher network

Meta Pseudo Labels


Mehta pseudo labels achieves 80 6.9% top 1 imagenet accuracy through semi-supervised learning with additional data and also impressive performances in the limited data settings the authors also introduced a reduced MPL framework to avoid the memory bottleneck of having two high capacity models in memory for the meta learning framework this video will explain meta pseudo labels from researchers at Google ai this video will explain meta pseudo labels from researchers at Google ai meta sudo labels is a new way to use meta learning

Meta Pseudo Labels


to adapt the ground truth class labels during training by using a teacher network to label data and then a student network that learns from those labels a quick overview of the meta pseudo labels algorithm is that a teacher model is trained along with a student model to set the students target distributions and adapt to the students learning state so typically these target distributions are these one hot encoded vectors where you might have like zero cat one dog and then zero for all of the other classes in the case of say C fart n so the idea

Meta Pseudo Labels


here is to have the teacher network produce the way of labeling the data points to say zero point zero three cat zero point seven dog zero point zero four horse these kind of distributions are going to be assigned by the teacher network rather than heuristic lis encoded with something like one hot encoding label smoothing or even the knowledge distillation pipeline with to temperature tuning so then the idea is to adapt these target distributions to the students learning state so the way this pipeline works is that the

Meta Pseudo Labels


teacher network parameterised by phi is again taking the same training data set and then produce a pseudo label distribution and then the student network is going to try to fit this label distribution that was produced by the teacher network so it's going to do back propagation using the cross entropy loss function between the predictions from the student Network y-prime and then these pseudo labels Q Phi of X so it's going to back prop this and then update the parameters to theta T plus 1 so now these new parameters that have

Meta Pseudo Labels


been updated by training on the pseudo labels from the teacher network are then going to be evaluated to provide a reward signal for the teacher network by taking those parameters and then having them classify a held-out validation set so the performance on the validation set is the reward signal that goes back through the teacher by taking a gradient through a gradient or do something we'll get into more in the later on in the video the idea is that the teacher model is going to be training changing this distribution of class labels to maximize

Meta Pseudo Labels


the performance of the student Network and the held-out validation set these are some examples of the most common target distributions our ground truth class label vectors Y that are used in machine learning the most common of which is one hot encoded vectors this is how datasets like C fart n are labeled if this is the case of a dog image in the class label corresponding to that image you'd have a one in the position of the dog index and then zero everywhere else for the other class labels so say one dog zero cat zero

Meta Pseudo Labels


truck ship this is how the sea far 10 data set it's labeled so one problem with labeling data sets one hot and code vectors is that the model is going to have these overconfident or over fitted predictions to this kind of a class label distribution you can say it applies any probability density to another class it's gonna have a penalty from the cross entropy loss for doing so so if it's a C's this dog image and tries to do label it as 0.75 dog and 0.2 cat he's unsure whether it's a cat or a dog it's gonna be penalized for that as

Meta Pseudo Labels


if the cat is just as different from a dog as a truck or a ship or a frog or these other C Bartek classes so one solution to the overconfident predictions are overfitting on the one hand code of vectors is label smoothing so label smoothing is where you apply this uniform weight to all the other class labels in the class suitable vector and then another solution to assigning these target distributions knowledge distillation knowledge distillation in the form of self-training with noisy student currently has the state VR for imagenet

Meta Pseudo Labels


classification and it's also a really powerful technique for model compression such as say distilled Bert or you have these high capacity models then you use the high capacity model to produce a new class label distribution that is better than the one hunter for training the student network and then the student network learned that a combination of this distillation class distribution as well as the ground truth what hot encoded vectors so these are some examples of different target distributions that have been explored in

Meta Pseudo Labels


machine learning and are commonly used to prevent overfitting and then to you know train these models with supervised learning so code from the paper is that from the success of these heuristic tricks it is clear that how to construct the target distribution plays an important role in the algorithm design and a proper method could lead to a sizeable game motivating this exploration for meta learning the target distributions during training so again we have this problem of what should be this target distribution should we have

Meta Pseudo Labels


one hot encode to class level vectors should we smoothing out the labels by putting uniform weight on the other class labels or should we use this teacher-to-student pipeline and knowledge installation but the solution explored in this paper is to metal learn the pseudo label distribution or the targets that the student networks and we trying to fit during training there are two phases of learning in the meta pseudo labels framework in phase one the student learns from the teacher the parameters theta of the student

Meta Pseudo Labels


classification network are updated by taking the cross entropy loss between the predictions piece of theta of X and then the pseudo label distribution that is produced when you pass these X data points through the teacher Network parameterised by Phi denoted Q sub 5 X to denote this new pseudo label distribution that comes out of the teacher phase 2 is the teacher learns from the students validation loss this is a much more complex way of structuring this loss this gradient through a gradient meta learning idea of

Meta Pseudo Labels


training the teacher network so the teacher network is evaluated on the validation set performance of the student network after it updated as parameters to theta t plus 1 so the way that this reward is propagated back into the teachers parameters Phi because you have to take the derivative of how much each of these five parameters and their labels and data points impacts the gradient of the student network to change it in the direction of this validation loss it's difficult to completely derive this idea of gradient

Meta Pseudo Labels


through a gradient and in if you're using frameworks like tensorflow PI torch you can utilize automatic differentiation to automatically implement this for you and you don't have to exactly know the math of how say this parameter from x1 input to the hidden state a in the teacher Network gets this lost signal from the student network that is then updated with the gradient of this new label data set and then moved in the direction of the validation loss but the idea the high-level idea and I think maybe visualizing these two networks even

Meta Pseudo Labels


though in practice the teacher network is gonna be a multi-layer perceptron like this but the student network is one of these high capacity classification models like ResNet wide ResNet or efficient that so the idea is you want to say update this weight from the input to a hidden state in the teacher network and you're gonna try to take the partial derivative with respect to this way in the teacher network with respect to the validation loss on the the student network after at theta t plus 1 on that validation set so this is trained with

Meta Pseudo Labels


policy gradients on this reward signal because this isn't like Y prime minus y there's no ground truth with respect to the validation loss of the student achieves so you're just taking that validation loss reward and treating that as like a reward and like say pac-man or Atari and using policy gradients to update the parameters but basically say like if this parameter contributed a lot to the output and then you get a high reward do more of this like increase the weight from this connection to do more of it to get more of this reward that's

Meta Pseudo Labels


kind of a high-level idea of policy gradients but the idea is that in order to get this derivative to find out like how much of this weight contributed to the validation loss you have to take a gradient through a gradient which is a pretty complex idea that's maybe better explained in the next equation from the paper hopefully this equation from the paper were further explain the idea of taking a gradient through a gradient to update the parameters v from the teacher network with respect to the parameters theta T plus 1 that are evaluated on the

Meta Pseudo Labels


validation set stadia was we're taking a gradient through a gradient so the parameters theta T plus 1 that are responsible for this validation loss reward signal we're trying to update the teacher network with our updated by taking the parameters theta T and then updating them with a gradient so we want to know how much each parameter infi is responsible for the gradient that updates this so you're taking the derivative with respect to the-- of this validation loss well fee is you know contributes to this validation

Meta Pseudo Labels


loss through the gradient so you have to find out you know how much each of these parameters and the fee network you know as in something like this how much does this parameter the fee network or this parameter contribute to the gradient and then the gradient is what updates the parameters and gives you this new validation loss so you're taking the partial derivative of feet with respect to this gradient through a gradient idea which is a little complicated but you know really an interesting idea with the meta learning and this medicine labels

Meta Pseudo Labels


algorithm the next idea introduced in the meta pseudo labels is to avoid this memory requirement of having two high capacity classification models in memory because say you have an efficient net as the teacher network as well as the student network now you have to keep both these models in memory especially when you're doing the gradient update for the teacher Network si idea to avoid that is do first train a large teacher network on the label dataset and then use it to produce this new distribution on the unlabeled data and then you use a

Meta Pseudo Labels


smaller teacher network so say you originally trained the teacher network with like an efficient net and then you move it to a multi-layer perceptron because all it's doing now is adjusting the original distribution that was produced by this high capacity model so this high capacity model is already producing a pretty useful target distribution as in knowledge distillation and then the smaller teacher has enough capacity to be adapting it during training as in the meta learning framework the authors are going to test the performance of meta

Meta Pseudo Labels


pseudo labels in the limited data setting as well as the semi supervised learning setting semi-supervised learning is responsible for most of the image net state of the arts like self training with noisy student where they have the labeled image net and they also leverage this unlabeled jft 300 million data set to get more performance out of the model also the billion scale weekly semi-supervised learning framework from facebook uses the labeled image net data set and the unlabeled Instagram images that are weakly labeled with their

Meta Pseudo Labels


hashtags to take advantage of the semi-supervised learning framework which is probably going to be the paradigm that leads forward since it's so easy to get this unlabeled data compared to label data so they're experimenting with meta pseudo labels on the efficient net architecture for the student network on the full-si 410 image net street view house numbers data set plus extra unlabeled data so in the case of C 410 this is tiny images in image net it's why FCC 100 million in street view house numbers there's an additional like

Meta Pseudo Labels


500,000 data points that come with the data set to use optionally if you want to test out these kind of algorithms so they achieve 98.6% see of our 10 accuracy and then 6.9% topple and imagenet then here are some other papers to check out if you're interested in semi-supervised learning that have also come out recently and are really successful in this kind of space these are the results of meta pseudo labels in the semi-supervised learning framework compared with supervised learning and then the self training with

Meta Pseudo Labels


noisy student pipeline you see gains in the sea far ten dataset small gains in the street view house members and then big gains on the image net dataset one reason that the authors point towards these small gains for the street view house numbers is that the extra unlabeled data in street view house numbers is in it's in the distribution so there is this distinction between out of distribution data and in distribution data so the case of image net where you're trying to take in this new data from this Y of cc100 million data you

Meta Pseudo Labels


would call that out of distribution data because it's not like the image net data whereas the street view house numbers that extra data is like really the same exact data that the training set has in terms of this like kind of underlying distribution idea so the idea here is that the meta Studio labels this adaptive adjustment of changing the labels during training is more crucial when the extra unlabeled data is more out of distribution so if you're dealing with a computer vision problem and you're curious that this algorithm is

Meta Pseudo Labels


gonna work for your problem it's interesting to say you know it's the unlabeled a do you have how out of distribution is it how noisy is it and this MPL meta student-level framework is likely to have a bigger gain if this is noisier data compared to in distribution data the authors also test the meta pseudo labels algorithm in the limited data setting where you have say only 4,000 labeled images in C far 10 1000 in Street View house numbers or 10% of the labeled data in image net in this case you see the performance of meta pseudo

Meta Pseudo Labels


labels compared to supervised learning with all the labels Simms CL are fixed match or unsupervised data augmentation which are all other algorithms that are successful at doing this kind of learning with limited data this plot shows the performance of these models with respect to the different limited data settings changes as you increase the percentage of labelled data points the interesting part about this plot is this top left area where you have the smaller percentage of labeled data and you see a huge gain of the unsupervised

Meta Pseudo Labels


data augmentation plus the menace to toe labels algorithm compared to supervised learning or the Rand augment data augmentation algorithm this table shows the gains of meta pseudo labels with respect to the sea far 10 limited data setting and in the street view house number limited data setting see the performance of different algorithms like just training with supervised learning on Limited data using that label smoothing way of putting uniform weights on the other class labels then using supervised learning plus meta pseudo labels and

Meta Pseudo Labels


then stacking medicine labels with the ran augment and unsupervised data augmentation algorithms these are some of the algorithms that are explored to stack on top of meta pseudo labels unsupervised data augmentation enforces predictions Y given X to be consistent with the same X data point after it's gone through a data augmentation so in some cases data augmentation might mean like rotating an image translating it or horizontally flipping it in the case of natural language processing it might mean translating the sentence to German

Meta Pseudo Labels


and then translating it back to English which is known as a back translation these are the different ways of augmenting data points and then forcing this cycle consistency to have similar predictions on the data point before and after it's been augmented another these algorithm says stacked on top of meta Steudle labels is Rand augment Rand augment is this automated data augmentation algorithm similar to like auto augment or population-based augmentation but the idea here is to have this simpler parameterization of

Meta Pseudo Labels


the space that actually shows to work better with respect to constructing these automated data augmentation pipelines and the next sort of algorithm to be looking at as well and comparing this with is self-rated with noisy student which is this really popular way of doing knowledge isolation which is where you take the pseudo labels you apply a lot of noise with respect to training the student model on that teacher target distribution which is the whole idea of meta studio labels is looking at the ways to structure this

Meta Pseudo Labels


target distribution to train these neural networks on the authors explore the behavior of the teacher network in the meta studio label framework so they've learned the Medus to delay bol teacher fits the validation gradient it's not just label Corrections and is not only a regularization or preventing overfitting strategy the authors explore this idea that the teacher encourages the students training gradient to be similar to the students validation gradient on this two moons data set because it's really difficult

Meta Pseudo Labels


for them to do this kind of cosine similarity between validation and training data gradients with these larger data sets like CFR 10 or imagenet so they show the cosine similarity between the training and validation data as gradient with respect to the training progress and showing that the medicine learning framework the teacher is trying to steer the gradient in the direction of this validation data sets gradient as well the next idea is to explore whether the teacher network is so performing label correction or trying to

Meta Pseudo Labels


mimic the behavior of supervised learning with perfect labels so this plot is showing that if it was doing this then these accuracies of the student network should be high as well and have a similar kind of curve as supervised learning so this is showing that the teacher network isn't just trying to fit the training data it's trying to help with this regularization and preventing overfitting this visualization shows the teacher network isn't just doing preventing of overfitting with respect to how its labeling this data can you see

Meta Pseudo Labels


interesting behaviors with respect to the developments of how it's labeling each of these data points and see if our 10 data set throughout the training you see that it the label with the highest confidence doesn't change it doesn't get steeper between 50 and 75 percent it doesn't do things like flipping labels or dampening distributions in obvious ways that are typically heuristically explored with respect to you know doing regularization in the class label space another interesting algorithm in the space of meta learning these different

Meta Pseudo Labels


components that make up the supervised learning problem is generative teaching networks generative teaching networks have a generator that uses this gradient through a gradient in order to generate this data set that is used to train the student network so it could be interesting to see if you could stack this meta suit of labeling or having this adaptive labeling with the generated data set as well also definitely a confusing gradient or you could maybe stack the generator and the label are sort of similar to how like

Meta Pseudo Labels


alphago zeroes combines the policy and value network into one architecture but it's definitely an interesting kind of space of emerging algorithms these meta learning algorithms that are generating data generating adaptive labels during the training a lot of different areas where meta learning is being developed and producing these interesting algorithms thanks for watching this explanation of meta pseudo labels a really interesting meta learning algorithm that adapts the target distribution for the student network as

Meta Pseudo Labels


it's learning throughout training to maximize the accuracy on a held out validation set this is a really interesting use of meta learning and this gradient through gradient training to have the teacher-student paradigm where the teacher is taking apart this different component of the supervised learning framework particularly in this case the target distribution and then the student is learning with the teacher network in this simultaneous I like dual optimization or coevolution framework of training these two models in the meta

Meta Pseudo Labels


hi-yah thanks a lot for joining us today thank you for inviting me actually I was glad to be here yeah one of the world's most visible deep learning researchers life asked me to share a bit about your personal story so how do you end up doing this work that you now do yeah that sounds great I guess I first became interested in machine learning right before I met you actually I've been working on neuroscience and my undergraduate adviser Jerry Kane a Stanford encouraged me to take your internet AI class I didn't know that

Heroes of Deep Learning: Andrew Ng interviews Ian Goodfellow


okay so I had always thought that AI was a good idea but that in practice the main thing I knew that was happening was like game AI where people have a lot of hard-coded rules for non player characters and games to say different scripted lines at different points in time and then when I took you were injured AI class and you covered topics like linear regression and the bias and various variants decomposition of the error linear regression I started to realize that this is a real science and I could actually have a scientific


career in AI rather than nursing great and then what happe well I came back an IT a two-year course I'm late I see great education so the really big turning point for me was while I was teeing that course one of the students my friend Ethan Dreyfus got interested in geoff hinton deep belief that paper and the two of us ended up building one of the first GPU CUDA based machines at Stanford in order to run Boulton machines in our spare time over winter break and at that point I started to have a very strong intuition that deep


learning was the way to go in the future that a lot of the other algorithms I was working with like support vector machines didn't seem to have the right asymptotics that you add more training data and it gets lower or for the same amount of training data it's hard to make them perform a lot better by changing other settings and at that point I I started to focus on deep learning as much as possible indeed and remember Magette rain has very old GPU paper race acknowledges you for having done a lot already work yeah yeah


as I was written using some of the missions that we built the first machine I built was just something that Ethan and I built that Ethan's mom's house that I would wait with our own money and then later we have money to build the first to agree for the Stanford lab wow that's great I never knew that story of Jason oh and then today one of the you know things that's really taken the deepening world by storm is you invention of Gans so how did you come up with that I just studying generative models for a long time sort of a Gans


our way of doing generative modeling where you have a lot of training data and you'd like to learn to produce more examples that resemble the training data but but they're imaginary they've never been seen exactly in that form before there were several other ways of doing generative models that had been popular for several years before I had the idea for again and after I've been working on all those other methods throughout most of my PhD I knew a lot about the advantages and disadvantages of all the other frameworks like Bolton machines


and sparse coding and all the other approaches that had been really popular for years I was looking for something how to avoid all of those disadvantages at the same time and then finally when I was arguing about 200 models with my friends in a bar something clicked into place and I started telling them you need to do this this and this and I swear it'll work and my friends didn't believe me that it would work I was supposed to be writing the deep learning text book at the time but I believed strongly enough that it would work that


I went home and coded it up the same night that it worked so thank you one evening to implement the first version as I if I implemented it around midnight after going home from the bar where my friend's house is going-away party and the first version of it worked which is very very fortunate I didn't have to search for hyper parameters or anything it was just for me I read it somewhere where you had a deaf experience and that reaffirm your commitment to AI tell me that yeah I was I wasn't actually near deaf but I


briefly thought that I was I had a very bad headache and some of the doctors thought I might have a brain hemorrhage and during the time that I was waiting for my MRI results to find out whether I had a brain hemorrhage or not I realized that most of the fact I was having worry about making sure that other people would eventually try out the research ideas that I had at the time in retrospect they're all pretty silly research ideas but at that point I realized that this was actually one of my highest priorities in life was


carrying out my machine learning research work yeah that's great that when you thought you might be dying soon you're just thinking how we get the research done yeah yeah that that that that's commitment yes yeah so today you're still at the center of allow the activities with scans of generative atmosphere networks so tell me how you see the future of Gans right now Jen's are used for a lot of different things like so my supervised learning generating training data for other models and even simulating scientific


experiments in principle all of these things could be done but other kinds of generative models so I think that games are at an important crossroads right now right now they work well some of the time but it can be more of an art than a science to really bring that performance out of them it was more or less how people felt about deep learning in general 10 years ago and back then we were using deep belief networks with bolts and machines as a building blocks they were very very finicky over time we switched to think like rectified linear


units and Bachelor realization and deep learning became a lot more reliable if we can make games become as reliable as deep learning has become but I think we'll keep seeing games used in all the places they're used with much greater success if we aren't able to figure out how to stabilize games but I think their main contribution to the history of deep learning is that they will have shown people how to do all these tasks that involve generative modeling and eventually we will replace them with other forms of generative models so I


spend maybe about 40% of my time right now working on stabilizing games he's cool oh and so just as a lot of people they join deep learning about ten years ago such as itself ended up being pioneers maybe the people they joined Gans today if it works out could end up the early pioneers yeah a lot of people already are early pioneers of games and I think if you wanted to give any kind of history of again so far you'd really need to mention other groups like indigo and Facebook and Berkeley for all the different things that they've done so in


addition to all your research you also co-authored a book on these learnings oh that guy that's right with Joshua Benji oh and Aaron kohrville who were my PhD coat visors we wrote the first textbook on the modern version of deep learning and that has been very popular both in the English edition and the Chinese Edition we sold about I think around 70,000 copies total between those two languages and I've had a lot of feedback from students who said that they've learned a lot from it one thing that we did a little bit differently than some


other books is we start with a very focused introduction to the kind of math that you need to do deep learning I think one thing that I got from your courses at Stanford is the linear algebra and probability are very important that people get excited about the machine learning algorithms but if you want to be a really excellent practitioner you've got to master the basic math that underlies the whole approach in the first place so we make sure to give a very focused presentation of the basics that's a start book that way you


don't need to go ahead and learn all of linear algebra but you can get a very quick crash course in the piece of the linear algebra that are the most useful for deep learning so even someone whose math you know is real shaky you've ever seen in math for a few years we're going to start from the beginning of your book and get that background and get into deep learning all of the facts that you would need to know are there it would definitely take some focused efforts and practice that making use of them great


if someone's really afraid of method it might be a bit of a painful experience but but if you're ready for the learning experience and you believe you can master it I think all the all the tools that you need are there as someone does work in designing for a long time I'd be curious if you look back over the years tell me about how about how you're thinking of AI and a deep learning has evolved over the years ten years ago I felt like as a community the biggest challenge in machine learning was just how to get it working for AI related


tasks at all we have really good tools that we can use for simpler tasks where we wanted to recognize patterns in hand extracted features where a human designer could do a lot of the work by creating those features and then hand it off to the computer and that was really good for different things like predicting which adds the user would click on or different kinds of basic scientific analysis but we really struggled to do anything involving millions of pixels in an image or a raw audio waveform where the system hasn't


built all of its understanding from scratch we finally got over the hurdle really thoroughly maybe five years ago and now we're at a point where there are so many different paths opens that someone who wants to get involved in AI may be the hardest problem they face is choosing which path they want to go down do you want to make reinforcement learning work as well as supervised learning works do you want to make unsupervised learning work as well as supervised works do you want to make sure that machine learning algorithms are fair and


reflect biases that we prefer to avoid do you want to make sure that the societal issues surrounding AI work out well that we are able to make sure that a AI benefits everyone rather than causing social of people and trouble with lots of jobs I think right now this is really an amazing amount of different things it can be done both to prevent downsides from AI but also to make sure that we leverage all of the upsides that it offers us and so today there are a lot of people wanting to get into AI so what advice would you have for someone


like that I think a lot of people that want to get into a I start thinking that they absolutely need to get a PhD or some other kind of credential like that I don't think that's actually a requirement anymore one way that you could get a lot of attention is to write good code and put it on github if you have an interesting project that solves the problem that someone working at the top lab one is itself once they find your github repositories they'll come find you and ask you to come work there a lot of the people that I've hired or


recruited at opening I last year or at Google this year I first became interested in working with them because it's something that I saw that they released in open-source form on the internet writing papers and putting them in archives can also be good a lot of the time it's harder to reach the point where you have something polished enough to really be a new academic contribution to the scientific literature but you can often get to the point of having a useful software product much earlier so sort of you know Nietzsche book


practices and materials and post like github and maybe on archive I think if you if you learn by reading the book it's really important to also work on a project at the same time to either choose some way of applying machine learning to an area that you're already interested in like if you're a field biologist and you want to deep-learning maybe you could use it to identify birds or if you don't have an idea for how you'd like to use machine learning in your own life you could pick something like making a tree view house


numbers classifier where all the data sets are set up to make it very straightforward for you and that way you get to exercise all of the basic skills while you read the book or while you watch Coursera videos that explains the concepts to you so over the last couple years have also seen you do one will work on adversarial examples and tell us a bit about that yeah I think every searle examples are the beginning of new fields that I called machine learning security in the past we've seen computer security issues where attackers could


fool a computer into running the run code that's called application level security and there's been attacks where people can fool a computer into believing that messages on a network come from somebody that is not actually who they says they say they are and that's called Network level security now we're starting to see that you can also fool machine learning algorithms into doing things they shouldn't even if the program running the machine learning algorithm is running the correct code even if the program running the machine


learning algorithm knows who all the messages on the network really came from and I think it's important to build security into a new technology near the start of its development we found that it's very hard to build a working system first and then add security later so I am really excited about the idea that if we dive in and start anticipating security problems with machine learning now we can make sure that these algorithms are secure from the start instead of trying to patch it in richer actively years later thank you fellas


and when you're living by the hormones or stress not a time to create no out of time to open your heart not a time to learn sometimes we do a meditation we start opening our heart and start elevating the body's energy and then those emotions can drive certain thoughts of your future you have to understand that if 95% of who you are is a set of unconscious programs then the first step is lighting a match in a dark place this will be one of the most powerful videos you watch today right now you're about to discover the three

DO THIS Everyday To Unlock The FULL POWER Of Your Mind! | Joe Dispenza


secrets to unlock the power of your mind with dr. Joe Dispenza I hope you enjoy [Music] how do we change our energy and how do we sustain it for extended period of time how long is that time need to be until we really started to see that sometimes immediate okay so so our research and we've done in the last six years because we were seeing so many incredible incredible things going on in our workshops I mean people stepping out of wheelchairs and all kinds of crazy things my church look kind of like a mega church but hopefully not that based


in science yeah but but but isn't it amazing that some of these churches when people get to believe whether they have science backing it or not they just it's the belief when they step out into the unknown step out of your body and who your size right instantaneously instantaneously and we do see that a lot and some people like yeah that's yeah can that be possible how can it be possible we we've done research now I just assembled a team of scientists we've done 8,500 brain scans I can tell you I can tell you when a person's about


ready to change I can tell you why really don't change I can tell you what it takes to change so what's it take to change well do you change most people keep their attention always their awareness on their body it keeps their attention on everything in their environment with people and things there the brain is always scanning everything around us to determine what's known as safe and unsafe right and you know we do that all the time so our research shows that the moment you take your attention off your body and you go from somebody


to nobody you take your attention off the people in your life and go from who you identify with from someone to no one and so many people spend their whole life building an identity of being someone take your attention off your cell phone your computer your car and go from something to nothing take your attention off where you're sitting where you need to be someplace you have to go go from somewhere to nowhere and take your attention off time linear thinking about the predictable future they're familiar past and fall into the generous present


moment and go from some time to no time then all you're left with is consciousness and that's the moment are no longer playing by the same rules matter to matter and there's a very elegant moment that takes place in the brain in fact I was just showing my research to a group of researchers and Santa Cruz this past week and they were blown away and I said now watch this person this person is going to have a transformational moment they said how do you know I buy I've seen enough of these and the next moment the whole brain just


lights up that person is switched on they'll never be the same person again they're having a transcendental moment and we could actually predict it and teach it now it's a formula hmm just like you doing sports if it just becomes a formula and then you change the formula and you add to it right so when you no longer are you're identifying with your body your environment and time that's the moment your pure consciousness now you're just an idea you're an awareness awareness awareness that has nothing to do with local space and time


and now if you're no longer yond you're anything you can go beyond and that's when the brain because the brain doesn't change the brain it takes a long time it takes a long time for the personality to change the personality for the ego to change the ego the programs that change the programs takes forever matter takes a long time to change matter but when you're in this moment you're no longer playing by those rules consciousness is the phenomenon above matter in fact consciousness is beginning to activate or manipulate circuits in the


brain people just think the brain is creating conscious no consciousness is executing the brain right so then if the brain can change then the mind doesn't change the brain mine is the brain in action is consciousness that changes it so when people begin to disengage and get beyond themselves you are at your absolute best when you get beyond yourself and getting the person to that point how does someone get to that point yeah so we teach them that formula we teach them to that point where all of a sudden they reach that generous present


moment where they just feel connected and when they're in that place all the things they thought they wanted they actually no longer want because they feel like they already have them so then imagine living your life from that place you would be less judgmental you would be less frustrated us impatient reactive and so so the formula then is that it requires a clear intention which is a coherent brain and when you're living stressed out and something goes wrong and you're threatened or you can't predict an outcome or you have the perception that


something's getting worse or you can't control it you switch on that fight-or-flight nervous system they've talked about now here's what happens when that occurs you start shifting your attention from one person to one problem the one thing to another person to another place because your brain is trying to predict the next moment well every one of those people and things in places has a neurological network in your brain so as you shift your attention from one to the next it's like a lightning storm in the clouds


your brain starts firing very incoherent lis when your brain is incoherent you're incoherent and when you're living by the hormones of stress not a time to create no not a time to open your heart not a time to learn not a time to trust and it's a time to run fight or hide so people spend 70% of their time of their life living in the state Wow so think about it so miserable yes so then when you're under stress if there's if there's a cougar around the corner you're not going to sit down and meditate sit still right but but so I'm


a tree you got the survival genes switched on and nobody is gonna believe in possibility when you're living in survival right yeah so then when you're living in stress what happens is you narrow your focus on the cause you now your focus on matter the object the thing and so people get switched on and all of their attention is on their outer world when the hormones of stress kick on the body gets an arousal now your attention is on the body and of course when you're under stress you're trying to predict the future based on the past


and now you're literally enslaved into three-dimensional reality so then how do you get what you want you gotta try harder you force it more you got a war cardio to fight for it it's matter trying to change matters Austin people just burnout right so then we now know that when you go from a narrow focus on something and you begin to open your focus you create sense and awareness that the act of opening your focus causes you to stop thinking and if you stop thing can you no longer activate those circuits and you start to slow your


brainwaves down hmm as you slow your brainwaves down you start connecting to that autonomic nervous system the thing that's giving you life and all of a sudden when you get beyond yourself it says he's gone let's step in and just clean up this mess before he gets back really and its job is to create order and balance your body will start to do that for you the innate intelligence will step right in once you've connect you got to connect so you got to know how to change your brainwaves you can't change your brainwaves you stay in an


active state you're basically moving furniture around you're analyzing your life within some disturbing emotion and I can tell you after looking at all those brain stands if you're analyzing your life within some disturbing emotion you're going to make your brain worse in fact you are thinking in the past right so you teach people the formula how to open their focus change their brainwaves connect to that invisible field and all of a sudden different compartments of the brain start synchronizing the front of the brain starts talking to the back


of the brain the right side starts talking to the left side and all of a sudden what sinks in the brain links in the brain all of a sudden you see this person starting to feel more like themselves and when you see those two hemispheres the brains start lighting up watch out because that person's gonna feel really hope they're gonna start loving life they're gonna feel like they're gonna be in love with life because the union of polarity and duality is wholeness at the exact same time coherent brain when you're


resentful when you're judgmental when you're impatient your heart beats out of rhythm why you're stepping on the gas and you're stepping on the brake at the same time your body and its intelligence living in survival is saying t-rex is back there but you're not running because you're sitting across the table looking at somebody smiling and your body's revved up right so the heart is beating and rhythmically and when that happens you're you're squandering or you're using all the body's life force and turning it into chemistry right


using all that energy to survive as opposed to think beyond right right so you're drawing from your vital life force that invisible field around your body and you're turning into the chemistry you actually are going to shrink your own feel the hormones of stress caused us to be materialists right we when one of the stress were we're using our senses to determine reality so now you feel more like matter unless like energy more separate from possibility so then to teach a person then how to regulate that heart center


and we do this we've done 6000 heart scans why because if I can teach you how to get in that heart state and I can teach you how to activate that Center and I can teach you how to regulate an elevated emotion the heart starts to create a very coherent signature and when the heart starts beating like a drum like dropping a pebble in water it begins to produce a measurable magnetic field up to 3 meters wide now you're more energy than matter more wave than particle Wow now that field that's being created is measurable and that's an


energy and energy is frequency and all frequency carries information so what is the information when it makes it here that you're sharing in the world it could carry the thought of your healing why because it's consistent with the energy guilt isn't gonna carry the thought of your healing it's a different frequency and all of a sudden now the person is elevating their emotional state and they're allowing their thought to be carried on their frequency they're broadcasting a whole new energetic signature but thoughts are the language


of the brain and feelings of the language of the body and how you think and how you feel creates your state of being so then the question is if you keep practicing creating that state of being it should become familiar to you yes or no the word meditation literally means to become familiar with so then if you're practicing moving into these elevated states and your heart is coherent and we're measuring and I can say Louis you got it now do it for 30 minutes now do it for 60 minutes and you practice creating that coherence you'll


know when you're there and when you're not yes or no sure and then you would be able to say like a skill like anything else give me a minute I'm gonna step out and you're gonna go back in the heart coherence and bring up that state now we get there in the heart then we practice a formula again rest your attention stop calling up elevated emotions and when you start seeing that that starts happening then you sustain it then you keep practicing and all of a sudden it gets longer and longer and longer now what's the relevance behind that well


we've measured neurotransmitters so when a person actually activates their heart the heart releases a chemical called oxytocin oxytocin is actually love chemical not oxytocin signals nitric oxide nitric oxide signals another chemical called endothelial derived relaxing factor what does that do causes the vessels in your heart to swell you will literally have energy in your heart you will literally feel like your heart is full now once you have that feeling you're not gonna want to trade that feeling for anyone or


anything you're gonna say well why would I judge that person if I judge that person will lose this feeling now all of a sudden you're self-regulating now once the heart is activated I just was at the research lab this week once the heart is activated it acts as an amplifier and it amplifies energy in the brain so once you start opening that heart and it begins to signal the brain you're going to suppress the survival centers in fact the research shows it will reset your baseline in other words if you're anxious and vigilant and you learn how


to self-regulate you'll actually reset the baseline and you'll say well the trauma was 15 years ago I saw my somebody get murdered or whatever and they will say yeah yeah yeah but the moment the heart not the brain it's the heart that actually resets the amygdala and all of a sudden the person all of a sudden switches down and all something like that I just don't have anxiety we have thousands of brain scans with anxiety and depression from people from all walks of life they've reset and all of a sudden they


don't have that excited they don't have to take medications or do anything is they know how to self-regulate where does all anxiety stem from mmm and anxiety is doing this living in the survival when you're living in survival I'll tell you this when the survival gene is activated out of the infinite potentials in the quantum field you'll always choose the worst-case scenario why because if you're in survival and you're preparing for the worst there's always better chances of surviving if anything less happens so people are


always selecting the worst thing in their mind and they begin to emotionally embrace that future before it happens thought and emotion you start conditioning so you're conditioning the body to become the mind of fear you keep doing that enough times once the body becomes them it's a subconscious program person as a panic attack try as you made a controller with your conscious mind you can't you programmed it subconsciously now you worry about the next panic attack and as you start worrying about the next panic attack


that's the vigilance that creates the next one well now here's what's happening in our work people who are self-regulating and creating these elevated states we have we have heart scans of them sustaining heart coherence for a whole hour during a meditation then at the end of the day they're still wearing the the monitor it's eight o'clock at night they're not even in a meditation and for a whole entire hour they're in heart coherence we say to the woman what's going on here she said I have no idea I was just getting ready


for bed and all of a sudden my heart just swelled up it was so intense I had to lay on my back and surrender to love instead of surrendering to fear she had a spontaneous love attack instead of a spontaneous panic attack now I would call that the natural state of being so then if you're living by those elevated states and you know how to feel that emotion of your future before it happens you're less likely to wait for it to happen because you'll feel like it already happened you'll less likely try to control it you'll


know that the moment you lose the feeling you just disconnected and you're gonna make your way back and when you get good at it no person no thing no experience can take it away from you well now you're empowered and if you understand the laws of how creation happens then you're less likely to compete and rush to get what you want you're gonna know that it's gonna come to you and now that's the new model of how do we create knowing is gonna come to us at the right time what if we want it faster no you just do it again


hmmm but remember if you're trying to make it happen faster you're back to the old self right then yourself would never do that the new self would constantly stay there and so then how does it appear it appears in a way that you can't expect because if you can predict it it's the known it's gonna come in a way that you haven't thought of an unknown and it's and it's got a rock your world it's got to catch you off guard it's got to leave you no doubt that what you've been doing inside of you that produces some effect outside of


you and when you correlate what you've been doing inside of you with the effect that you produced outside of you pay attention to what you did and do it again and the energy uh the joy that you feel when it happens you're gonna use that energy to create again now people say to me well I'm this way because of that person in that thing I would say to them so you mean them that person or that experience out there is controlling your thoughts and feelings that means you're a victim to your environment but when you start changing your thoughts


and feelings and it starts to produce an effect in your environment you're gonna change the belief that you're a victim consciously or subconsciously of your life to becoming more of a creator of your life and now all of a sudden you become more a creator of your life you can't blame anybody you can't say well that person and I think you'd have to say I got to be greater than that environmental condition who in history can I study that had the same challenges now what was what was what did they do let me just work that into my rehearsal


so that I can improve right just like you've done with sports is the same process yeah [Music] was more powerful than our thoughts or our emotions and do our emotions change our thoughts or do our thoughts change our emotions yeah the answer is yes the answer is both I mean thoughts to me produce an electrical charge in the quantum field and feelings produce a magnetic charge in the quantum field thoughts wait thoughts produce a what an electrical charge okay and feelings produce a magnetic charge and how you


think and how you feel broadcasts an electromagnetic signature that influences every single atom in your life the thought sends the signal out I'll think about this and the feeling draws the event back so you could have the intent that you want wealth you want health you want success that's your intent that's your thought but if you're waiting for the experience to happen to feel it then you're not drawing the experience to you because you're not feeling the emotion right so then teaching people once again how to


balance their thoughts and feelings because you can you can enter that cycle either place sometimes we do a meditation we start opening our heart we start elevating the body's energy and then those emotions can drive certain thoughts of your future other times you open your awareness you create brain coherence you have the vision of your future you begin to emotionally experience it however you want to jump on that cycle and then sustain it because the longer you're conscious of that energy the more you're drawing your


future to you so then most people spend their lives right they we live in this realm called space-time three-dimensional reality and you move your body through space and three-dimensional reality it takes time yeah so everything all your goals all your dreams all your visions you're gonna have to get your body up and drag it through space every day to pay off that you know that home that's in your future right right when you create from the field instead of from matter when you're the vibrational match between


your energy and some potential and your thoughts and feelings are coherent now you are going to begin to collapse time and space or the experience is going to be drawn to now now your the vortex to your destiny and now you don't have to go anywhere to get it because you're not playing by the rules of three-dimensional reality you're playing by the rules of energy in the quantum so teaching people how to do this in getting better at it then all of a sudden they're not forcing and controlling outcomes in fact they're


trusting and surrendering to outcomes because they don't want to get in the way because the moment you start trying to predict when it's going to happen or how it's gonna happen you're overlaying a known over a place where there should be an unknown right so teaching people how to do that means we have to lay down the very thing we used our whole life to get what we want for something greater to occur right and so that transcendental moment is something that we're working on the mystifying and and you could be gluten


free person you could be a gluten full person you could drink wine not drink wine you could be rich you could be poor you could be any color any shape any size in fact you can't tell me you're too old to do this work you can't tell me that we got elders in this work that we show you other brain scans and you'd be blown away but they they know how to do it you can't tell me you're too sick to do this work we got people that have reversed stage four cancer in numerous times and yeah it took a Herculean effort to do it but they love themselves


for you can't tell me that you're too out of shape or too overweight or too under weight you can't I've seen it in all shapes and sizes you can't even tell me that you had a brutal past and people that have had very very dismal pasts that are free they're happy people you can't even tell me you're you never meditated before in fact our research shows that many people have never meditated before have the most profound experiences because they're not trying to make anything happen they're just following the instructions right and and


they don't have a habit of doing it so so we don't want to exclude anybody in the process we want to include everybody so it turns out that our events tend to draw a good portion of men because of the science we have a lot of children now that are you know teenagers that are coming and people in their 20s we have great community of elders we have you know in our events sometimes 63 different cultures well coming to countries coming to our events between 50 and you know 65 so so we want it we want to make it so inclusive that


community becomes the side effects because because with a community of like-minded like like antler similar energy of people everybody understands they get one another you know you you communities tends to be the thing now that in terms of our social medium and the feedback we're getting everybody wants more community because you get a you get a thousand people in the audience and their energy synchronized now you're talking us something so much bigger we're just gonna measure this I just talked to a researcher yesterday


we're going to measure a thousand people when they reach that synchronized moment when they're we can we know that the entire social coherence in the room is orderly then if you're producing a ambience coherent magnetic field in your heart and you're tuning into a thought or an intent and you got a thousand people doing that and your energy is gonna start interfering and commingling with the person next to you when that Energy starts to synchronize it's gonna produce a bigger wave the higher the amplitude the higher the wave the more


energy there is so now you have one mind in one heart and now when it comes to healing others and we've done the research on this now and we're collecting the data that we're teaching people how to administer a change in energy and the person that's laying there because it's not matter that emits a field that's the wrong way to think about it it's the field that creates matter you change the field you change matter you know it's not your job to change the tumor the tumor is the illusion it's the pattern in the field


that's that that has to be changed so once people start reversing this then you start seeing the tumors disappearing you start seeing blind people seeing deaf people hearing you start seeing people with Parkinson's disease switch on I mean you start seeing stage four cancer is reversing because now they're you're you're you're swimming upstream you're going to the headwater in making that change so pushing the envelope and them seeing that in a community when a community synchronized towards the second half of a week-long


event I mean as I said before we started the show I I'm more surprised than anybody was it's crazy what is the we talked about I heard you say consciousness a couple of times what's the difference between mindset and consciousness to me consciousness is awareness awareness is paying attention and noticing and so 95% of who we are by the time were 35 years old is a set of unconscious automatic programs that we've just practiced so many times that we're not consciously thinking about those so in order for you to change to


answer the initial question that you asked the first step is you got to become conscious of your unconscious thoughts and you got to you got to start looking at those hardwired thoughts that that you think every day that it's just circuits that have been fired and wired together how do we do that should we take write a list at the end of the day or one of the most common thoughts we had that day like how does someone become aware you don't have to do that you just have to sit down close your eyes and not move and then you'll get


you'll you'll start seeing what am I thinking about right yeah and all you want to do is observe the thought because when you begin to observe that thought you're no longer the program now you're the consciousness observing the program and you're starting to pull out of the program thinking about the thinking yeah who's doing the thinking of the thinking about the thinking that's who you are when you're not the program that's awareness right you got to become aware of how you speak how you act become so conscious so aware of it


that you won't go unconscious and let that thought or that behavior run you you got to say oh my god this feeling that I've been living by for the last 20 years is actually guilt I didn't know it was guilt because it just feels like me and all of a sudden as you start becoming conscious of it you're beginning to objectify your subjective self here you're pulling you out of those programs and nobody likes to do that because it's uncomfortable they'd rather turn on their cell phones start texting get on the internet


you know watch TV to distract them from that moment and that is what they have to move through in order to get to their to their own personal freedom so the first step is becoming conscious and meditation means to become familiar with to become conscious of to to become so conscious of your unconscious self that you won't go unconscious to any thought any behavior or emotion and get ready because it takes a tremendous amount of energy to do that and awareness that conscious to stay conscious and so we fall from grace yeah fine


you got you got your wake you got another day let's go again how often do you fall oh my gosh I mean how many times have I done it thousands but I'm not gonna give up because the moments in which I do connect or the moments that I do have that transcendental experience what matters the most after it when I have that transcendental moment I look back at all of those difficult meditations those difficult days and those are the ones you remember you don't remember the good meditations you remember the ones where you came up


against yourself yeah and you want a little further and you say I'm gonna go we'll further and go further or you had a rough day and you just went in and you just you at the end of the day you surrender and you have the classic oh my god moment there's no linear correlation it's just whether you're willing to live in creation instead of living in survival and so you get better at it you know we just get better at it and for me staying consciousness thing where in staying present is an art because you you know when someone's present with you


in your life because they're paying attention to you you know when they're not present with you because they're not paying attention to you so imagine this field of informations is this intelligence that lives within you and I that's governing everything material in this world it's a self-organizing intelligence you have access to it so you better get present with it as well as you can get present with anything else and just because you can't see it doesn't mean it doesn't exist it that that that realm you can't


experience with your senses you can only experience with your awareness so then people have to take their attend off their bodies and go from uh somebody to a nobody hmm take their attention off the people in their life and go from that they identify with and go from uh someone to a no one take their attention off the things in their life their cell phone their computer the car and go from something to nothing take their attention off where they sleep where they work where they're sitting and go from somewhere to nowhere take their


attention off the predictable future and the familiar past and time and go from some time to no time and now if you're taking all of your attention off of everything material in this three-dimensional reality now there's only one other thing that's left that means you're an awareness your consciousness and now that is the bridge that is the door to the quantum field and you can't enter the quantum field as a somebody so if someone has spent their whole life working on having the perfect body or so much so they have so much


attention on their pain where you place your attention is where you place your energy it's going to take some work for them to take all of their attention off their body right because they'll go they'll do it and then look go back let's see if the pain so that all the pain still there so it's a little bit of a waltz in the beginning but as people start applying this you start getting better at it as an example we had Bond University University and Australia on the Gold Coast senior researcher took a large majority of my brain scans and


they hadn't she had them analyzed by her graduate students and they statistically looked at everything one of the most startling things for the research team was our community's ability to go to to get to that point where there nobody no one no thing nowhere no time I'm talking for seconds I'm talking five seconds I'm talking nine seconds just like just give me a second I know how to do this [Music] I feel like we've brainwashed ourselves over the years to believe a story of the past is who we are and who we will


always be in the future so how do we brainwash ourselves in times of stress and anxiety in order to become more peaceful loving and successful in the future how do we brainwash ourselves in a different way sure well that's what I and and Wow I mean the first and most important thing is that you have to understand that if 95% of who you are is a set of unconscious programs then the first step is lighting a match in a dark place if you want to become someone else you got to become aware of who you are yes that means you


got to start thinking about what you've been thinking about you got to start paying attention to how you speak that be conscious being conscious to every thought action most word emotion feeling expression body language everything you need to be aware of and conscious of Selzer is there a way that you help people to track this besides of just like okay I'm aware of this in the moment do they journal the thoughts throughout the day or when a negative thought comes up today remember the way their body languages throughout the day


closed off and guarded or open do they how do they self reflect where they can track it better mmm well that's a big question but I will tell you this that you demystify the word meditation and the word meditation literally means become familiar with yeah when you become so familiar with your thoughts so aware of your emotions so conscious of your habits that you wouldn't go unconscious to them again now you're no longer the program right so so getting people disentangled from that program we found out as a formula and when we teach


people how to do certain things with their focus and opening their awareness when we teach them how to create a very disorderly brain that has been driven by the hormones of stress into a more orderly coherent plane and teach them how to open their focus and practice that they'll come up against those thoughts and they'll become so familiar with them listen to they won't believe him anymore when they come up any longer Wow and so when they hear them in their day they'll be like that's not gonna stop me from my future


so if they're sitting there and they want to quit just because they're sitting still and they're not quitting then they're developing a will that's greater than those programs and you're breaking out of the shell and you keep doing that you're gonna get up and do the work every day because you did it yesterday and you're gonna want to do more of that because you're getting out of your past and it feels better and if you keep doing that and you keep feeling better every day the question is why wouldn't you do it every day because you


would ultimately just feel better and then the more hole you feel means the more you are connected you should imagine if you felt every single day and this is what I do work on if I could stay connected to the emotions of my future all day long there's no way I would be looking for when it would be happening how could I look for when it would be happening if I feel like it's happening I wouldn't look anymore which means I wouldn't be separate from it and that's when you start creating the magic right that's when you're in that zone


and that's when that reaffirms that personality that you're becoming and now you don't you know wake up in the morning go I got to create my future jump out of bed excited you're not gonna want that magic to end that's the right so so you teach people how to do this and they start seeing the events in their life they're not gonna want to miss a day in in really just getting beyond their personality and listen it's so cool because it's amazing you see all these people come in for a success and new careers and new relationships and


healing from a disease and mystical experience that come in for all these different kind of reasons healing from a childhood trauma but really they just want wholeness right and so as I start becoming more whole and they start feeling more whole is not coming from anywhere out there it's not coming from out there nothing out there is making them feel whole when the novelty of the thing wears off you feel empty again they're feeling whole from within oh this is different this is a different game so why wouldn't


you want to keep feeling more whole that you no longer want anything now now you're not here now you're not living in separation anymore if someone is so disconnected to their future their greater future self if they're so negative thoughts suicidal thoughts often hurting themselves potentially often just don't have many close friends don't feel like they identify with themselves in the world don't like anyone understands them no one accepts them no one gets them like this all sounds great in theory but when


you're in a place of survival mode and your thoughts constantly how can someone like that without having to go through the workshop that doesn't have the opportunity to go right now what can they start to do to just give some a little bit of relief and peace in their heart yeah it's simple knowledge experience wisdom philosophy initiate that philosophy master it yep mind body soul thinking doing being learning with your head applying with your hands knowing it by heart and this is the journey of knowledge because when


you learn that information and you really study it you are going to begin to see the world differently because your brain is changing then when you start saying how can I use this how can I apply it how can I personalize that how could I do something initiates this information what am I going to do how do I get my behaviors to match my intentions now this is the act of trial and error it's so important you don't make it the first time you don't give up you get up and you try to walk again and you start learning how to do this and so


as you begin to do it over and over again you start having new experiences well new experiences enrich the circuits in your brain philosophically you know the brain makes a chemical and now you're feeling more unlimited you're feeling more whole you're teaching your body chemically to understand what your mind intellectually understood and now you are literally literally starting to embody that knowledge yes coming signaling new G's it's new information but but you can't do it one day and expect your wealth to come you gotta do


it over and over again yeah so the repetition of practicing over and over again neuro chemically conditions the mind and body begin to work as one you've done it so many times the he now knows how to do it subconsciously because just like it just like it knew how to subconsciously lean into trauma on victim mode right that kids now the body's getting new information it's going to adapt and and now you're going to literally become that knowledge you're gonna be coming that's what you're gonna become and so now that's


when you no longer have to try it's who you are it's yes yeah you've memorized an internal order that's greater than anything in your outer world that's gonna tell you something else that's key right there and you become immunity to negative thoughts or negative viruses where if something tries to someone tries to say something to your body just rejects it automatically just as if your physical body would reject some virus coming in is that correct that's absolutely what I said dr. Joe you can't do it you're


stupid you're ugly you're not smart enough you're not this you won't even reverse they won't even just bounce right off you you wouldn't even receive it because your field is so powerful it's pushing all that away no because it's not the truth there you go it's just laughs it'd just be like okay that's just not real but if I kept telling you you know you need this product you need this drug to feel better to look better to be better let's appeal to your lack and if you buy this it'll make that feeling go away and you


try for a while then the feeling doesn't go away anymore you got to try something else and now watch the news and listen to all that information that's telling you you're limited you're you know you're limited it's something out there it's going to get you there's nothing wrong with that this but if you're constantly saying it's traffic it's the news it's politics it's my ex that's making me feel and think this way then you're subconsciously not consciously affected by your environment and you'll be more affected by your environments not a kind


of process but the more you say I'm going to in spite of the fear that we talked about or the anxiety or the frustration or the aggression or the hostility or whatever it is I'm paying the suffering instead of saying that I can't change that you know what I'm gonna think I'm gonna see if I can where can I find that information where and all of a sudden you find people that are doing it and it makes sense to you and you're like well okay I'm feeling really anxious instead of taking something that's going to chemically change me let's see if I


could chemically change me without something out there let's see if I can make my own pharmacy of any depressants I don't know let's see if I can make my own pain relievers let's see if I can make my own chemicals that cause my immune system to get stronger I'm just curious let's see now now the person is there out of the bleachers and they're on the field so they'll start believing that they can do it even if they change it a little bit and if they don't get it the first three days but they've seen a testimony of someone who has and you see