that start inside the sphere converge and none of the spheres intersect so the sphere of point i doesn't intersect with the sphere of point j so that's where we say all these spheres are non-intersecting we say x i is retrieved if the iteration equation three converged to the single fixed point in that sphere the retrieval error is the distance so you'll notice you have two things you have x i this is the actual pattern and you have x i star this is the retrieved pattern so these hopf they don't always have to


give you the same thing that you stored that's part of the the nature of continuous neural networks whatnot so for every sphere we say there is a pattern there is a sphere now we as pattern is stored if every i can start wherever i want in this sphere okay wherever i want it will always converge to a point that's inside the sphere okay and maybe that point isn't the pattern that i stored but actually this point right here but wherever i start i will always converge to that particular point if that's the case then i have stored

Hopfield Networks is All You Need (Paper Explained)


this particular pattern now the fact is i don't retrieve this particular pattern i retrieve the blue thing but i can then define the error of retrieval the error of retrieval is simply the distance between the two things ideally this distance is very small right but you know we can't can't guarantee it now there are going to be theorems that deal exactly with this retrieval error but first you can see that here if if these spheres become larger you you can't accurately store a pattern anymore so this is the kind of ideal situation


but there are also situations where these spheres you know if i have these patterns right here these spheres are so large kind of the the attractions of the patterns are so large that if i start let's say here then i don't converge to either of these two patterns i converge to like something in the middle i converge to maybe this point right here and that's going to be one of these meta stable states okay we're going to encounter situations like this but we're also going to encounter situations like this and the bottom thing isn't necessarily


bad and that's what you have to keep in mind um and yeah as i said we will get to it but just keep this kind of sphere image in mind okay so first we'll just deal with the you know the up the top situation where we store patterns and then retrieve patterns so we'll we'll assume a failure probability which is p and p is going to be you know pretty pretty low for their example so they have p equals 0.001 you know like a 0.1 percent error probability of retrieving your pattern things like this and randomly chosen patterns on the


sphere with radius m we define some constants then with probability 1 minus p the number of random patterns that can be stored and stored in the sense of having these spheres around them so that you can retrieve them accurately or at least you can retrieve something that's close to them is is bounded lower bounded by this quantity right here so there's the square root of p there is this constant c but then you see that d is in the exponent right here so that means it's exponential in the number of dimensions


so that's that's pretty cool so if you add a dimension you exponentially increase the number of the number of patterns you can store and you know that's that is a kind of i mean it's it's been known for modern hopfield networks with binary strings so it's not uber surprising but if you have you know it's not what you would imagine like that okay so they may give a few examples of these co you have to set these constants you know in a particular fashion such that this is given and so on um but they say you know examples here


are where c is something like three and d is 20. um so if you were to add a 21st dimension then your i guess storage capacity would increase by a factor of three which pretty cool all right so this is how many that we can store infinitely not sorry exponentially many patterns um in these networks now they deal they say the next theorem states that the update rule typically converges after one update if the patterns are well separated so if we're in a situation where these patterns are well separated which is


kind of like this but you can also imagine this in terms of dot products because we operate in the space of dot products so if the patterns are well separated that sort of means that they all kind of sort of point away from each other and this notion of separation is going to be captured by this quantity right here this is the separation of example of pattern i which is just the inner product with itself uh minus the maximum inner product with any other uh pattern and this quantity is going to be large when no


other pattern is close to it so when the separation is large then the update rule the retrieval rule of um calculating you know i have a query calculate the inner product with all of those then i re-weigh all of the patterns by that inner product by the softmax then i used that new thing as a query again and so on as we discussed it will converge to the closest pattern but this theorem says it actually converges pretty fast and here i have my problems with saying that it converges after one step um typically converges after one update


because that you know genuinely depends on a lot of constants as we'll see but it does converge exponentially fast in this separation constant as the theorem 4 says with query psi after one update the distance of the new point to the fixed point is exponentially small in the separation delta i the precise bound using the jacobian and its value in the mean value theorem are the following so here you can see this is the distance between the updated psi after one step and the um and the fixed point right here this is what it converges to


is going to be the distance as it was before times this thing right here so you can see since this is a this is a multiplicative update um and in this jacobian so this is expanded down here this is this you can see here you have the you have this sorry yeah this is this so this is bounded by that you have the exponent the exponential function negative this separation right here so the higher the separation the faster this algorithm converges okay to say that it converges after one step is you know it might be a bit of of


bragging i don't know if this is a common thing if you have like an exponential convergence uh that you are allowed to say it's after one step i'm not sure especially what i'm not sure about is that you have n here as linear constants in that factor okay so if you if you and that's what they do in their code so if you look at their code and the code's available which is pretty cool it's implemented in pi torch as a general module that can you can just drop in so this is not only for transformers this is for you can replace like lstms you can


replace pooling mechanisms um you can you know do a whole bunch of stuff in their paper in the accompanying paper they do this multi-instance learning with giant uh sets um on using these hopfield layers so pretty pretty cool this code is definitely worth kind of checking out and maybe you want to replace some stuff with you but the question is how many of these update steps should you do right because we looked at the diagram at least in the attention mechanism it seems like you have attention layers right you have a transformer


and the transformer consists of you know you have this input right here and you go through layer layer layer layer layer and in each layer there's contained in it and one of these attention mechanism right this entire thing is in this layer okay and now if you interpret this as a hopfield network and you want to do multiple steps that means you go this branch right here so in each layer potentially you do multiple steps of these things so for whatever computational constraints um transformers had already this will


certainly make it worse but also you need to decide how many steps you want to do now you can hard code that of course but they say you should do these steps until this norm here until the norm between the old and the new is small enough so where is that so you can't measure how close you are to the convergence points right because you don't know in practice but you can measure how far you're away you can measure where did we have it you can measure this quantity right here that's something you can measure how far


twit rates are apart so what you'll simply do is you'll measure that and if that is small enough then you'll you'll stop but that i guess is very related to this so how if you we've already proven it converges to this x star so i guess we can approximate this quantity right here with the quantity above and that tells you how many updates you need to do and that quantity is linear not only linear but actually here quadratic in n i don't care you know yes it's exponential uh amd uh separation but it's quadratic in n and if i've


learned anything from kind of my fast code courses is that constants actually matter when you're not dealing with infinity with an infinite number of steps so the number of the number of steps you need to do i guess will depend on the sequence length in a quadratic fashion so i'm not sure you can always claim this is converges in one step now i might be super mistaken here and none of this uh will can none of this actually makes a difference in the in the light of the exponential decay here but i would just i'm just a bit worried


saying this usually converges in one step it's clear i guess why they do it right because the attention mechanism in transformers is a one-step application of this rule and this here is kind of a theoretical justification for interpreting this precisely as a hopfield network because you'd say well in a hopfield network you would do multiple steps but wait wait we can actually prove that even if you interpret it as a hub field network you it can it usually converges after one step so what you're actually doing in a


transformer is applying a hopfield network update rule uh to convergence so yeah i'm not yeah i might be bickering on a high level here luxury problems theorem five then says so theorem four is how fast does this converge um theorem five the last theorem right here uh says that the retrieval error of a pattern and so this is the this is what you converge to and this is what you've stored um is bounded by again something that's exponential in the separation right here as you can see okay so that was the theorem so if we go


quickly through them again theorems one and two deal with the convergence of this algorithm and the fact that it actually minimizes the proposed energy then theorem three says you can store exponentially many patterns in terms of the dimension of your space and theorems four and five say that this update rule will converge exponentially fast after after one step if you believe that and the retrieval error will also go down exponentially fast with the number of update steps that you do okay that sounds pretty pretty pretty


good but we've heard it it's very dependent on how well separated these patterns are and it turns out that it's you know at least in transformers they aren't always well separated and that might be on purpose remember the these states here the the patterns aren't pre-stored like in a classic hopfield network but the patterns if you interpret an attention mechanism as this are also generated by the network itself so the pattern matrix that you retrieve from and the query are generated by the attention mechanism in in this case as i said this


is applicable to many many more uh domains than just this but um yeah so there's another slight modification that you have to do to make this actually equivalent to an attention mechanism and that is you'll have to recast the value because usually what you'll do is you have some sort of input and then you make queries keys and values from that using different heads the only thing to make it formally equivalent is you have to make the values generated from the keys so the keys give rise to the values as you can see


right here that you first multiply with the key matrix and then with the value matrix i think that's you know that i don't i doubt that this will will change anything um if you if you the only way that could really change anything is if this matrix here would be super low rank like collapse the space of um into like very few dimensions which the value matrix wouldn't do so you know but just letting you know that the technical equality requires this slight modification okay now we said that um it might not you know be that this is


always super well separate and you retrieve a single pattern and that's what they research here in a pre-trained bert model so they take a pre-trained model from i guess from hogging face and they run they just run a data set through it and what they do is so for each for each query and sorry for each attention head because you have multiple ones of these attention heads right um in each layer so in each layer you have multiple ones of these heads for each head they look at over the course of the whole data set


how do these softmax distributions look like so when you believe that this is a hopfield network and you believe that this converges in one step then if the patterns are well separated what we would expect is a distribution as we said like this okay there would be one dominant pattern that you retrieve you know that's what you want to retrieve that's what comes out but a bang um you retrieve that accurate pattern anything else would mean that the hopfield network sort of failed right it wouldn't give you back


one particular pattern so they have i think that's a pretty it's a pretty smart experiment they look how many bars do we need to add how many of these bars in the soft max distribution do we need to add to reach 90 percent right so it depends a bit on the temperature of the softmax which is hardcoded and attention mechanism pdb is one this squared over d um so they say how many do we need to add to get to 0.9 to 90 percent of the mass of this distribution and if this is the hopfield network where you retrieve one pattern


then one will be enough right one of these bars will probably be i don't know like 99 okay but there are other cases imagine the case where the patterns and the query you retrieve the spheres that it gives rise to are all like overlapping okay so what that will do is it won't converge to any particular pattern but the attractor space in this kind so you can imagine if you have two spheres that are apart from each other the update rule converges either so if it's closer to here it converge here if it's closer to here it'll converge


here but if they are overlapping like this the energy landscape will actually make it such that it will neither if it starts somewhere it will neither converge to here nor to here it will actually converge to somewhere in the middle okay into the mean of the stored patterns and if we take that to the extreme what could be is it could be that the softmax distribution looks completely uniform okay which would basically mean that you know i don't care where my information comes from just average and this has its applications so if you


for example want to make a sentiment classifier a very cheap way to do that is to simply take pre-trained word embeddings like glove or word to back you know assign each word word embedding and then just average the word embeddings okay and you count on the fact if there are a lot of kind of negative words in there like bad sad angry the word embedding kind of will you know reflect that and the average word embedding will point more into the bad direction and if there's a lot of happy words the average will point into the happy


direction okay so there are applications of averaging information not caring particularly where it comes from and um in that case what we'd expect is that this number and we'll call that so we'll call that the number k in this case it equals one but in this case k equals i guess n the number of inputs okay because we need well not maybe n but you know approximately we need almost all of them to uh to reach the 90 percent okay and there there is an in between and these are called these meta stable states where


and the in between is something like you'd have a couple of patterns here a couple here and a couple maybe here it's almost like a clustering like and these overlap and these overlap and these overlap but they don't overlap with each other which means that if you start somewhere here you would converge to the mean but not to the mean of all the patterns but just to the mean of these patterns and here here and here here so this this is like a clustering in latent space right so you can interpret these hopfield


update rules as somehow you know getting not going to a particular pattern but going to sort of a cluster and this is if you ask something like hey is there any adjective around right and all of these patterns they kind of overlap in that space in that query space of adjective they overlap and therefore the update rule would converge to sort of the mean which would basically say yes there is an adjective here right and the information would not be routed so that the distribution if we start here right and we converge


to this the distribution would look something like small small small and then you'd have a couple of large ones right you'd have like maybe two or three or four of large ones and these would exactly correspond to the patterns here so the information will be routed from all of those in that cluster to this particular node that asks the query okay these are these are what's called these meta stable states and what they do is they calculate over the entire data set this number k and here they show you the distribution so in these plots


what you'll see is over the entire data set um k goes into that direction so i guess let's go to tis here this this seems pretty easy so k uh is in this direction and this is simply the amount of like how so in each you you let a data point run through it you measure k for that particular layer one you see this is layer one head four okay this is one layer one attention head and then you can see that the number k is distributed like this okay so contrast this to this head right here where it's a lot of weight on the number


one or like very few numbers okay so these blue ones would be these are your typical like when you retrieve one particular pattern so this attention head we can sort of conclude in this particular tension head this is very specific it looks at its input it looks at its token and it decides what information do i want and it retrieves one particular thing from the other nodes okay whereas here it's more like kind of an an averaging it's more like i want this kind of information and on average i don't even know what


the sequence length is here i guess it's maybe 512. uh so of the 512 the median this number is always the median in median it collects information from 231 of them okay so you can see that this corresponds this green and orange ones correspond to these meta-stable states where uh there's kind of an implicit clustering done in the in this space of attention whereas the blue ones they correspond to attention heads that ask for particular information retrieve one particular maybe a few patterns and um happy with that and the red ones


here you can see that they often just average they just you know because k is so high means that i need all of the i need all of these bars to get to the 90 or i need almost all of them which basically means it's a uniform distribution right so it's like i don't care where information comes from just average whatever average i just want the average you know some particular uh space and as we said that also has its uses interesting how this translate through so this here is as we go down the vert model on the bottom you have layer one


you see there are a lot of these averaging operations going on so a lot of the heads are simply doing averaging and as you go up the layers the heads get more and more specific in the types of information they seek but then again in the last layers interestingly you get into a lot of these meta stable states again which i guess again interpret this as you as you want i'm going to leave this up to you but it sort of says like here you want kind of general patterns at the bottom and then the middle layers are kind of


the logical workhorses so you look for very specific things in the input this is i guess this is where i guess this is where the thinking happens um so this is sort of pre-processing i'm just making stuff up here by the way this is this must be in no way true this is maybe thinking and this this here this might already be output again because you know after that you have language modeling or classification so this might already be like aggregating uh types of information this is how i sort of interpret it okay uh yeah so so this these these


experiments are pretty pretty pretty interesting and here they have they do these are the last experiments for this paper um they do an interesting experiment where they actually replace the attention heads by simply an average mechanism and later they actually replace them by gaussians but in this case they simply average and they show that look if i replace layer one with just averaging the perplexity doesn't rise that much right so it's pretty good um even if i replace an entire layer here with averaging uh it it perplexity goes more up


and you can see the corresponds if you remember the previous plot the correspondence is pretty one to one with how much blue and green uh heads there are as in contrast to how much red uh and orange ones there are so here you have lots of blue ones and you can see that the error kind of goes up and interestingly here you have more meta-stable states at the end but still the perplexity goes up uh more so i guess you can only really replace the red ones with the averaging so this is always averaging in one particular layer


and they go into more detail here where they say look this is this is layer six and this is layer 12. so this is one particular tension head from layer 6 and layer 12 and the updates don't be confused it goes in this direction okay i was confused at first and you can see right here this number k at first you know it's kind of spread out but then it pretty quickly converges to a very small number and there is this kind of point right here i don't know if the learning rate's decreased i don't think so i think that's just kind of a


a phase transition right here this is the blue line by the way the blue training line a face transition where all of a sudden these just these attention heads they somehow decide okay this is the thing i want to specialize in this is the type of task i want like a sub task of linguistic subtask i want to specialize in and then they concentrate on one particular pattern per input so they are really specializing whereas in the last layer you see here that even during training they are sort of continuously learning


so first they also do this averaging then they go into this meta-stable region right this is this meta-stable region k isn't one but also k isn't a very high number um so they continuously learn and it's even indicative of this training might not be done here first of all and second of all it would be really interesting to see how this works out with you know sizes of transformers and like especially these these huge transformers just the fact that they can keep learning the more we train them might be you know


be interpreted in the light of what kind of states they converge to and the fact that there are tension heads i don't know how does this go on do they stay in the meta stable states because it makes sense to have metastable states as i said it makes sense to kind of cluster things or are they simply is this simply an intermediate step and if you go really far down they would actually also converge to the k equals one where they really specialize or maybe do we need more attention heads for this i don't know it it's just i think this is just the


the beginning of kind of research in this direction i think just this kind of number k um how it's how it's made it's pretty simple and apparently it's pretty pretty revealing so you know that's pretty cool so that was the paper uh and its experiments it's it's a pretty sizable paper as i said even the paper itself is uh 10 pages and then there is this immune repertoire classification which uh i will like spend one minute looking at it so you have you have these set classifications so for each human you obtain a set of immune receptors


and you simply obtain one label whether that human is immune to a particular disease or not and your task is kind and then a different human has a different set you have no idea which one of these things is responsible for it being for the human being um for the human being immune or not in fact there is a it you can't even decide based on these you can only decide based on like sub sequences uh of these and they might be in combination with each other so there might not be a single one responsible but like a combination but you don't


have labels for the individual ones and you have different ones per human and they are different lengths all of this is just a giant um giant task and you have many of them you have tens of thousands per human right so they build a system here where first they do these 1d convolutions to process the inside sequences and then they do this hopfield um attention mechanism all with with learned queries uh over these things and then they train on the output label and surprisingly uh that actually works even with tens of thousands of inside


sequences and only one label for all of them and so they they achieve i guess uh favorable results compared to other baselines on this task using these hopfield network which is pretty interesting but i'll let you look at that paper yourself so i hope this somehow uh made it a bit clear what happens here and it would actually be pretty interesting um if we you know to see what happens if we just do maybe two rounds of these updates is this even desirable right uh is it desirable to run this to convergence is there something good


in 1948 Claude Bristol wrote a best-selling book the magic of believing this very popular book continues to sell in paperback form today Claude Bristol died in 1951 however we feel that in order to maintain the compelling nature of the book we would like to give you this in a form as close to the author's original style as possible just as you hear the voice of a writer as you read a book with the help of an actor William Caine we hope to bring you the voice of Claude Bristol and just as you can stop reading a book

"The Magic of Believing" By Claude Bristol


most conveniently at the end of a chapter we'll create natural places for you to stop time to think about what you've heard time to take it in listen to the magic of believing by Claude Bristol it could change the course of your life chapter 1 how to tap the power of belief is there a force a factor a power a science call it what you will are something which a few people understand and use to overcome their difficulties and achieve outstanding success I firmly believe that there is it is my purpose here to attempt to explain it so that


you may use it if you desire I realize I have run across something that is workable but I don't consider it as anything mystical except in the sense that it is unknown to the majority of people and is little understood by the average person I'm aware that there are forces powerful forces at work in this country that would dominate us substituting a kind of regimentation for the competitive system which has made America great among nations I believe that we must continue to retain the wealth of spirit of our forefathers if


we don't we shall find ourselves dominated and everything we do by a mighty few will become serfs in fact if not in name I hope this work will help develop individual thinking and doing some may call Mia a crackpot or a screwball I'm well aware of that let me say that I am past the half-century mark and have had many years of hard practical business experience as well as a goodly number of years as a newspaperman I started as a police reporter police reporters are trained to get the facts and take nothing for


granted apparently I was born with a huge bump of curiosity I've always had an insatiable yearning to seek explanations and answers this yearning has taken me to many strange places brought to light many peculiar cases and has caused me to read every book I could get my hand on dealing with religions cults and both physical and mental sciences I have read literally thousands of books on modern psychology metaphysics ancient magic voodoo ISM yoga 'sm theosophy Christian Science unity truth new thought and many


other dealings it's what I call mind stuff many of these books were nonsensical others strange and many very profound gradually I discovered that there is a golden thread that runs through all the teachings and makes them work for those who sincerely accept and apply them that thread can be named in a single word belief it is the same element or factor belief which causes people to be cured through mental healing enables others to climb the ladder of success and gets phenomenal results for all who accept it why belief


is a miracle worker is something that cannot be satisfactorily explained but have no doubt about it there's genuine magic in believing the magic of believing became a phrase around which my thoughts steadily revolved I've tried to put down these thoughts as simply and as clearly as I could so that everyone can understand my hope is that anyone who listens will be helped in reaching their goal in life I would like to start by relating a few experiences of my own life with the hope that by hearing them you will gain a better understanding of


the entire science early in 1918 I landed in France as a casual soldier unattached to a regular company as a result it was several weeks before my service record necessary for my pay caught up with me during that time I was without money to buy gum candy cigarettes and the like every time I saw a man light a cigarette or chew stick a gum the thought came to me that I was without money to spend on myself certainly I was eating in the army clothed me and provided me with a place on the ground to sleep but I grew bitter because I had no


spending money and no way of getting any one night on route to the forward area on a crowded troop train sleep was out of the question I made up my mind then that when I returned a civilian life I would have a lot of money the whole pattern of my life was altered at that moment I didn't realize it then that at that moment I was laying the groundwork for a new direction in my life groundwork that would unleash forces that would bring accomplishment as a matter of fact the idea that I could with my thinking and believing develop a


fortune never entered my mind money is not the only desire you may have it doesn't matter to what end the science is used it will be effective in achieving the object of your desires and in this connection let me tell another experience some years ago I decided on a trip to the Orient and sailed on a ship called the Empress of Japan something was working for me on that trip I had no claim to anything but ordinary service however I sat at the executive officers table and was frequently his personal guest in his quarters as well as on


inspection trips through the ship well naturally the treatment I received made a great impression on me and in Honolulu I often had the thought it would be nice to receive comparable treatment on my journey home on another ship one afternoon I got the sudden impulse to leave for the mainland it was about closing time when I arrived at the ticket agency I was told that a ship was leaving the next day at noon and I could get the only remaining cabin ticket I bought it and the next day just a few minutes before noon I started up the


gangplank in an offhand manner I said to myself they treated you as a king on the Empress of Japan the least you can do here is sit at the captain's table sure you'll sit at the captain's table the ship got underway and as we steamed out of the harbor word was received from the dining room steward for passengers to appear in the dining room for assignments to tables about half the assignments had been made when I came before him he asked me for my ticket which I placed on the table he glanced at it and then to me saying oh yes table


a seat number five it was the captain's table and I was seated directly across from him many things happened aboard that ship which pertained to the subject the most prominent being a party supposed to be an honor of my birthday just an idea of the captain's because my birthday was months off in laying before you this very workable science I am aware that the subject has been handled before from many angles I am also cognizant that many people shy away from anything that smacks of religion the occult or the metaphysical


accordingly I am presenting it in the language of a businessman who believes that sincere thinking and plain speaking will get any message across to the people in using this science which is given to you with the confident knowledge that no matter how you use it it will get results I wish to warn you never use it for harmful or evil purposes since the beginning of man there have been two great subtle forces in the world good and evil both are terrifically powerful in their respective scopes and cycles the basic


principle operating both is mind power masked mind power therefore take great care that you do not misuse the science of mind stuff I cannot emphasize this too strongly for if you employ it for him harmful or evil purposes it will boomerang and destroy you just as it has others down through the centuries these are not idle words but solemn words of warning chapter 2 the power of thought glance around you if you are in a furnished room your eyes tell you that you are looking at a number of inanimate objects that's true


so far as visual perception is concerned but in reality you are actually looking at thoughts or ideas which have come into materialization through the creative work of some human being it was a thought first that created the furniture fashion the window glass and gave form to the draperies and coverings the automobile the skyscraper the Great Plains that sweep the stratosphere the sewing machine the tiny pin a thousand and one things yes millions of objects where did they come from originally only one source from that strange force thought


as we look further we realize that these achievements and in fact all our possessions came as a result of creative thinking thought is the original source of all wealth all success all material gain all great discoveries inventions and of all achievements with that in mind it becomes easy to understand that a man's thoughts make or break him and Shakespeare's words become more intelligible there is nothing either good or bad but thinking makes it so many people feel that success comes with hard work however I would like to point


out that hard work alone will not bring success the world is filled with people who have worked hard but have little to show for it something more than hard work is necessary it is creative thinking and firm belief in your ability to execute your ideas the successful people in history have succeeded through their thinking their hands were merely helpers to the brains another important point is that one essential to success is that your desire be an all obsessing one your thoughts and aims be coordinated and your energy be concentrated and applied


without let-up it may be riches or fame or position or knowledge that you want for each person has his own idea of what success means to him but whatever you consider it to be you can have it provided you are willing to make the objective the burning desire of your life a big order you say no not at all by using the dynamic force of believing you can set all your inner forces in motion and they in turn will help you reach your goal now that you have a clearer idea of the part that thought and desire play in your daily


lives the first thing to determine is precisely what you want starting in with the general idea that you merely want to be a success as most people do is to indefinite you must have a mental pattern clearly drawn in your mind ask yourself where am I headed what is my goal have I visualized just what I really want if success is to be measured in terms of wealth can you fix the amount in figures if in terms of achievement can you specify it definitely I ask these questions for in their answers are the factors which will


determine your whole life from now on strange as it may appear not one out of a hundred people can answer these questions most people have a general idea that they would like to be a success but beyond that everything is vague they go along from day to day figuring that if they have a job today they will have it tomorrow that somehow they will be looked after in their old age they are like a cork on the water floating aimlessly drawn this way and that by various currents being washed up on the shore or becoming waterlogged and


eventually sinking therefore it is vital that you know what you want of life you must know where you were headed and you must keep a fixed goal in your view only then will you get what you're after so you begin with desire if you ever hope to achieve anything or gain more than you have now however as we shall see there is more to it than mere desire it has been said that thought attracts that upon which it is directed thought attracts that upon which it is directed it was Jobe who said for the thing which I greatly


feared has come upon me our fearful thoughts are just as creative or just as magnetic and attracting troubles to us as are the constructive and positive ones and attracting positive results so no matter what the character of the thought it does create after its kind when this sinks into a man's consciousness he gets some inkling of the awe-inspiring power which is his to use I cling to the theory that while thoughts do create an exercise control far beyond any limits yet known to man they create only according to their


pitch intensity emotional quality depth of feeling or vibratory plane in other words comparable to the wavelength and wattage of a radio station thoughts have a creative or controlling force in the exact ratio of their constancy intensity and power let me try to clarify that while many explanations have been offered no one knows where the thought is a form of electrical energy or something else yet to be defined but I have been an experimenter in that branch of electricity known as high frequency pioneered by the great genius Nikola


Tesla and whenever I think of thought and its radiations and vibrations I instinctively link them up with electricity and it's phenomena in this manner they become more understandable to me all persons living in high altitudes have felt and sometimes observed the electric spark resulting from walking across the room then touching some metallic substance that of course is a form of static electricity generated by friction it gives you an idea of how one kind of electricity can be developed through the body the Sigmund Freud the famous Austrian


psychoanalyst brought the world's attention to the hypothesis that there was a powerful force within us an unenumerated part of the mind separate from the conscious mind constantly at work molding our thoughts feelings and actions others have called this division of our mental existence the soul some call it the super-ego the inner power the super consciousness the unconscious the subconscious and various other names it isn't an organ or so-called physical matter such as we know the brain to be nevertheless it is there and from the


beginning of recorded time man has known that it exists the ancients often referred to it as the spirit Paracelsus called it the will others have called it the mind an adjunct to the brain some have referred to it as conscience the creator of the still small voice within still others called it intelligence and have asserted that it is a part of the supreme intelligence to which we are all linked no matter what we call it I prefer the word subconscious it is recognized as the essence of life and the limits of


its powers are unknown it never sleeps it comes to our support in times of great trouble it warns us of impending danger often it aids us in what seems impossible it guides us in many ways and when properly employed perform so-called miracles perhaps the most effective method of bringing the subconscious into practical action is through the process of making mental pictures using the imagination perfecting an image of the thing or situation as you would have it exist in physical form this is usually referred to as visualization


however before this visualization can work you must really believe I refer now to deep-seated belief a firm and positive conviction that goes through every fiber of your being when you believe at heart and soul as the saying goes I'll call it a phase of emotion a spiritual force a type of electrical vibration anything you please but that's the force that brings outstanding results it sets the law of attraction into operation and enables sustained thought to correlate with its object this belief changes the tempo of the mind or thought


frequency and like a huge magnet draws the subconscious forces into play changing your whole aura and affecting everything about you and often people and objects at great distances it brings into your individual sphere of life results that are sometimes startling often results you never dreamed possible Chapter three what the subconscious is a good staff kelly the distinguished french psychologist and author of from the unconscious to the conscious once wrote there is no artist man of science or writer of any distinction however


little disposed to self-analysis who is not aware by personal experience of the unequaled importance of the subconscious he also said that the best results in life were obtained by a close harmony and cooperation between the conscious and subconscious mind as a subconscious plays a very important part in the magic of believing it will bring you to a quicker understanding of this science if you have a clear and detailed picture of what the subconscious mind is where it is located and how it functions now it is the


conscious mind that is the source of thought also it's the mind that gives us the sense of awareness in our normal waking life the knowledge that we are ourselves here and now the recognition and understanding of our environments the power to rule over our mental faculties to recall the events of our past life and to understand our emotions and their significance more concretely it enables us to have a rational understanding of the objects and persons about us of our successes or shortcomings of the validity of an


argument or the beauty of a work of art many times the solution of our problems result from the use of the conscious mind but now and then when the solution is not forthcoming we become exhausted with continued trying we begin to lose confidence in ourselves and we often resign ourselves to the idea that we have failed that nothing can be done about it here is where the subconscious mind comes in it helps us to renew our belief in ourselves it assists us to overcome our difficulty and to put us on the road to achievement and success


just as the conscious mind is a source of thought so the subconscious is the source of power also it is one of the greatest realities in human life it is rooted in instinct and is aware of the most elemental desires of the individual yet it is always pressing upward into conscious existence the powers of the subconscious are many the chief of which our intuition emotion certitude inspiration suggestion deduction imagination organization and of course memory and dynamic energy it is a distinct entity it possesses powers and


functions with unique mental organization all its own now the subconscious mind has three primary functions first with its intuitive understanding of the bodily needs it maintains and preserves the well-being and indeed the very life of the body unaided by the conscious mind second in times of great emergency it springs into immediate action again independent of the conscious mind it takes supreme command acting with incredible certitude rapidity accuracy and understanding in the saving of the life of the individual


third it is operative in the psychic world in which the psychic powers of the subconscious are manifested in such phenomena as telepathy clairvoyance and psychokinesis but also it can be summoned to help the conscious mind in times of great personal necessity when the conscious calls upon the subconscious to use its powers and resources to solve a vital problem or bring to pass that which is sought or desired by the individual it is the third function that we are most concerned with here to draw upon the resources and powers of the subconscious


and awaken it into action you must first be sure that you are asking for something that is rightfully yours to have and is within your ability to handle the subconscious manifests itself only according to the capabilities of the person then you must have patience and absolute faith Theodore Simon Geoffroy the French philosopher said the subconscious mind will not take the trouble to work for those who do not believe in it next in conveying your need to the subconscious it must be in the spirit that the work has already


been done so while it is necessary for you to feel and think yourself successful it is important for you to go one step further and actually see yourself as already successful either in the performance of some selected task or as actually occupying the position to which you are aspiring for the next and final step you must wait patiently while the subconscious is assimilating the elements of your problem and then goes about its own way to work it out for you in due course with the flowing of ideas and plans of the subconscious into your


waiting conscious mind the solution of your problem will be revealed to you the correct course of action will be indicated you must follow those indications immediately and unquestioningly there must be no hesitation on your part no mental reservation no deliberation you must receive the message from the subconscious freely and after understanding it you must act on it at once only by doing that will you make your subconscious serve you and continue to respond whenever you call upon it however your problem may be one that


cannot be solved in such a manner instead of receiving the solution in the form of a blueprint as it were you may instead feel some mysterious force urging you at intervals to do certain things that seem to have no special significance or logical connection nevertheless you must continue to believe in the power and the wisdom of the subconscious and obediently perform the seemingly irrelevant things one day you will find yourself in the position you sought through the aid of the subconscious and doing the work you


envision for yourself then when you look back you will see how the things you were called upon to do all formed a logical line of events the last one of which was your final arriving the reward of your sincerest hopes and desires your own triumphant personal success chapter four suggestion is power after studying the various mystical religions and different teachings and systems of mind stuff one is impressed with the fact that they all have the same basic modus operandi and that is through repetition the repeating of certain


mantras words formulas or just plain mumbo-jumbo is common with witch doctors voodoo high priests hexes and many other followers of strange cults they use them to evoke the spirits or work black magic one finds the same principle at work in chants incantations litanies daily lessons also the frequent praying of the Buddhists and Muslims alike the affirmation of the Theosophists and the followers of unity the absolute true New Thought divine science in fact it is basic to all religions although here it is white magic instead of black magic


this brings us to the law of suggestion through which all forces operating within its limits are capable of producing phenomenal results that is it is the power of suggestion and Auto suggestion your own to yourself or hetero suggestion coming to you from outside sources that starts the machinery into operation or causes the subconscious mind to begin its creative work and right here is where the affirmations and repetitions play their part it's the repetition of the same chant the same incantation the same affirmations that lead to belief and


once that belief becomes a deep conviction things begin to happen now this is the same identical force and the same mechanics that Hitler used in building up the German people to attack the world a reading of mind camp will verify that dr. Rene faux vel a famous French psychologist explained it by saying that Hitler had a remarkable understanding of the law of suggestion and it's different forms of application it was with uncanny skill and masterly showmanship that he mobilized every instrument of propaganda in his mighty


campaign of suggestion Hitler openly stated that the psychology of suggestion was a terrible weapon in the hands of anyone who knew how to use it let's see how he worked it to make the Germans believe what he wanted him to and once that belief took hold how they started their campaign of terror slogans huge signs posters masked flags appeared throughout Germany Hitler's picture was everywhere one Reich one folk one leader became the chant it was heard everywhere today we own Germany tomorrow the entire world the marching


song of the German news came from thousands of throats daily such slogans as Germany has waited long enough stand up you are the aristocrats of the Third Reich Germany is behind Hitler to a man and hundreds others bombarded the people 24 hours a day from billboards sides of buildings the radio and the press every time they move turned around or spoke to one another they got the idea that they were a superior race and under the hypnotic influence of this belief strengthened by repeated suggestion they started out to prove it


unfortunately for them there were other nations who also had strong national beliefs that eventually became the means of bringing defeat to the Germans let's go into the field of sports for everyone who has ever witnessed a football or baseball game has actually seen this power of suggestion at work the late Knute Rockne famous coach at Notre Dame knew the value of suggestion and used it repeatedly but he always suited his method of applying it to the temperament of the individual team a story is told that on one Sunday afternoon Notre Dame


was playing a particularly grueling game and at the end of the first half was trailing badly the players were in their dressing room nervously awaiting the arrival of Rockne finally the Douro and Rocky's head came in slowly his eyes swept inquiringly over the squad oh excuse me I made a mistake I thought these were the quarters of the Notre Dame team the door closed and Rockne was gone puzzled and then stung with fury the team went out for the second half and won the game in the Depression years and there may be years like them in the


future we saw this same suggestive force working overtime day after day we heard the expression times a hard businesses pour the banks of failing prosperity hasn't a chance and while stories about business failures on every hand until they became the national chant millions believe that prosperous days would never return hundred just thousands of strong-willed men went down under the constant hammering the continuous tap tapping of the same fear vibratory thoughts money always sensitive runs to cover when fear


suggestions begin to circulate and business failures and unemployment follow quickly we heard thousands of stories of bank failures huge concerns going to the wall and people believe them readily and acted accordingly there will never be another business depression if people generally realize that it is with their own fear thoughts that they literally create hard times they think hard times and hard times follow doctor walk till Scott eminent psychologist and longtime president of Northwestern University told the whole


story when he said success or failure in businesses caused more by mental attitude rather than by mental capacities let's consider charms talismans amulets good-luck pieces four-leaf clovers old horseshoes a rabbit's foot and countless other trinkets which thousands of people believe in by themselves they are inanimate harmless objects without power but people breathe life into them by thinking they do have power or even though the power isn't in them per se the power comes only with the believing which alone makes them


effective - outstanding illustrations of this are found in the stories of Alexander the Great and Napoleon in Alexander's day an Oracle proclaimed that whoever unloosened the Gordian knot would become ruler of all Asia Alexander you may remember with one stroke of his sword cut the knot and rose to tremendous Heights and power napoleon was given a star sapphire when he was a child with the prophecy that it would bring him luck and someday make him emperor of france could it have been anything but the supreme belief in the


prophecy that carried these two great men to a place in the Hall of Fame they became great men because of their super normal beliefs here's a simple experiment that will demonstrate to you the strange power of attraction through visualization making the mental picture actually work find a few small stones or pebbles which you can easily throw locate a tree or a post of six to ten inches in diameter stand 25 to 30 feet away from it start throwing pebbles at the tree trying to hit it if you're an average person most of the stones will


go wide of their mark now stop and tell yourself that you can hit the objective get a mental picture of the tree figuratively stepping forward to meet the stone imagine the rock actually colliding with the tree in the spot where you want it to strike you'll soon find yourself making a perfect score don't say it's impossible try it and you'll prove that it can be done if you will only believe it chapter 5 the art of mental pictures to become the person that you would like to be you create a mental picture of your


newly conceived self and if you continue to hold it the day will come when you are in reality that person Shakespeare said assume the virtue if you have it not now let's take this great truth and follow some of its implications in assuming the virtue you are assuming via your imagination but here we must make a distinction between daydreaming and a true mental picture or proper use of the imagination perhaps there is some genie who will drop $100,000 into your lap or overnight provide you with a mansion luxuriously furnished I have never had


the pleasure of meeting one but daydreaming or mere undirected wishful thinking doesn't have the power to release the latent forces within you that will bring you the $100,000 over the mansion when you employ your imagination properly you see yourself doing a thing and you go ahead and do it it's the doing the thing you have pictured to yourself that brings it into actual existence in this connection think about the use of the magnifying glass when properly focused it will gather the light from the Sun and concentrate it so that the heat will


burn a hole in the object on which the Rays are focused it must be held steady before the heat power is developed and so it is with the holding of the image or the mental picture however it is very difficult for the average person to concentrate for any length of time to say nothing of holding on to a mental picture for any great period you are constantly being swayed by what you read and hear and as a result the coordinating part of this creative force turns to gathering together all these scattered elements in a focused mass


instead of devoting itself to making a clear and dynamic picture of your desire often I have thought of this matter of desire and suggestion in connection with the planting of veg or flower seeds once the soil is prepared and the tiny seeds are placed in it it only takes a short time until they begin to root and sprouts begin to appear the moment they start upward through the soil in search of light sunshine and moisture obstacles mean nothing to them they will push aside small stones or bits of wood and if they can't do that


their extend themselves and grow around them so it can be with you when the suggestions you give to your subconscious mind the results will be pure or complex depending upon the original seed in the attention which you give it in other words plant the right kind of seed and habitually feed it with strong affirmative thought always directed toward the same end it will grow into a mighty force finding ways and means of overcoming all obstacles I have been in the private offices of a great many industrial leaders


businessmen great bankers and others long before this magic of belief was understood by me I was impressed with the pictures photographs slogans bits of statuary and so forth which were to be found in the inner sanctums of great firms undoubtedly many of you have seen or heard of such displays but has it ever occurred to you what their purpose was there can only be one answer and that is they serve as a constant reminder getting the picture over to the occupant of the room that he too can succeed as those did before him in


common with other great men Thomas a Edison obviously knew the value of the repeated suggestion and made use of it among the articles found in his desk was a piece of paper that said when down in the mouth remembered Jonah he came out all right Edison must have thought well of that expression and perhaps reflected much upon it so let's get down to the mechanics find yourself three or four cards ordinary business cards will do in your office your home your room or any other place where you can have privacy sit down and ask yourself what you


desire above everything else when the answer comes and you certain that it is your uppermost desire then at the top of one card write a word picture of it one or two words may be sufficient a job a better job or money a home of your own then on each card duplicate the word picture from the original carry one in your billfold or handbag place another alongside your bed or fasten it to your bedstead place another on your saving mirror or dressing table and still another on your desk the whole idea as you may have


guessed is to enable you to see mentally the picture at all hours of the day just before going to sleep at night and upon waking in the morning are two very important moments of the 24 hours in which to concentrate upon your thoughts with added force but don't stop just with those two periods the more often you can visualize the desire by this method or one of your own devising for that matter the speedier the materialization at the start you may have no idea of how the results are to come don't worry just leave it to the


subconscious mind which has its own ways of making contacts and of opening doors and avenues that you may never even have thought of you will receive assistance from the most unexpected sources you may be suddenly struck with the idea of seeing a person that you have not heard from in a long time or calling upon a man you've never seen before you may get the idea of writing a letter or making a telephone call whatever the idea is follow it it cannot be too strongly emphasized that you should tell no one just what the words on the cards mean


don't give anyone an inkling of what you desire the truth is that when you talk about what you're going to do you scatter your forces you lose the close connection you have with the subconscious and you frequently find that unless you do is direct it you will have to start all over again in your program of achievement go and tell no man still holds true suppose you want a better job or promotion not only use the cards but keep telling yourself constantly continuously that you are going to get that job you have already visualized it


if you have accepted this science but the repetition will be the means of driving the suggestion deeply and firmly into the subconscious mind this may be compared to driving a nail into a board the first tap puts the nail in place but it is only by a number of heavy strokes that the nail is driven home it has been my observation that those who consciously use this science as well as those who may be using it unconsciously are people of tremendous energies virtually human dynamos they are people who not only use their imagination and


hold strong beliefs and convictions but they are great doers in action and that brings me to this most important statement faith without action is dead chapter 6 the mirror technique there is another device which I call the mirror technique before explaining it I want to tell you how I happen to discover what a truly wonderful thing it is and how it can be used to bring quicker and more effective results many years ago I was the dinner guest of a very wealthy man who owned many patents covering logging and sawmill machinery he had invited a


number of newspaper publishers bankers and industrial leaders to his suite in a prominent hotel in order to explain a new method he had devised for mill operations dinner was late in being served and as there had been plenty of liquor offered the host found himself in an embarrassing state of intoxication just before dinner was served I noticed him staggering into his bedroom and pulling himself up abruptly before his dresser thinking I might help him I followed him to the door of his room as I stood there I saw him grab the edge of


the dresser top with both hands and stare into the mirror all the time mumbling as a drunken man sometimes does then his words began to make sense and I moved back a little to watch the performance I heard him say John you own they tried to get you drunk but you're going to fool them you're sober cold sober this is your party and you've got to be sober as he kept repeating these words while continuing to stare at the reflection of his eyes in the mirror I noticed that a Transfiguration was taking place his body was becoming more erect the muscles


of his face were tightening and his drunken look was disappearing the whole performance was over in about five minutes but in all my experience as a newspaperman and more especially as a police reporter I had never seen such a rapid change not wanting him to know that I'd been watching him I made for the bathroom when I got back to the dining room I found the host at the head of the table and file his face was still a little flushed to all appearances he was sober at the end of the dinner he presented a very dramatic and convincing


picture of his new plans it wasn't until long afterwards when I got a better understanding of the power of the subconscious mind that I understood the science involved in transforming the obviously drunken man into a cold sober host many great orators preachers actors and statesmen have used this mirror technique for years Winston Churchill according to Drew Pearson never made a speech of importance unless he made it before a mirror first Woodrow Wilson also employed the same technique it's what I call a super charging method of stepping


up the subconscious forces this mirror technique gives a clue to the power and magnetism of Billy Sunday the great evangelist according to Eric severide in his book not so wild a dream Billy Sunday would bound about his hotel room now peering intently out the window with one foot on the sill now grasping the dressing table firmly in both hands while lecturing his reflection in the mirror now to outline the technique stand in front of a mirror it need not be a full-length mirror but it should be big enough so that you may at least see


your body from the waist up those of you who have been in the army know what it means to come to attention stand fully erect bring your heels together pull in your stomach keep your chest out your head up now breathe three or four times until you feel a sense of power strength and determination next look into the very depths of your eyes tell yourself that you're going to get what you want name it out loud so that you can see your lips move and you can hear the words make a ritual of it practice doing it at least twice a day mornings and


evenings and you'll be surprised at the results within a few days you have developed a sense of confidence that you never realize that you could build within yourself this power will give you that penetrating gaze that causes others to think you are looking into their very souls sooner or later there will come an intensity that will reveal the intensity of your thought emerson wrote that every man carries in his eyes the exact indication of his rank remember that your own gradation or position in life is marked by what you carry in your eyes


so develop eyes that say confidence the mirror will help you a word of warning here I take it for granted that none of you assume that the techniques I'm showing you here is an open sesame' to riches and fame overnight certainly it wouldn't be wise to rush into undertakings far beyond your capabilities or your development but by using this science you could learn the various steps which will take you to the top but you must have a plan of action before any program is undertaken you've got to know what you want and be


specific about it as long as you hold on to the mental picture of your idea and begin to develop it with action nothing can stop you from succeeding for the subconscious mind never fails to obey any order given to it clearly and emphatically Chapter seven how to project your thoughts in this section I want to talk about several points that I think pertain to mind stuff call it a pot pourri we seldom realize how much our emotional vibrations affect others and how much we're affected by there's an extremely nervous person in a position


of authority can put nearly every person associated with them into a nervous state it's always important to remember that a negative person can raise havoc in an organization or a home the same amount of damage can be done by a strong negative personality as good can be done by a positive one when the two are pitted together against one another the negative frequently becomes the more powerful to get a better understanding of the effect of these suggestive vibrations you need only to read your varying feelings when entering different


offices or homes the atmosphere which is the creation of the people living there can be instantly detected as being upsetting disturbing trend core harmonious the vibrations set up by others affect us much more than we realize we take on the characteristics of those with whom we are more or less constantly associated if you want to remain a positive type avoid associating too much with anyone who has a negative or pessimistic personality this brings me to another point a person who desires riches must go where the riches are


alone on a desert I have on a man would probably have a tough time eking out a living to say nothing of trying to amass a fortune so it is in everyday pursuits therefore if you want money you have to associate yourself with people who have it or who know how to make it this may sound rather gross but the truth is that if it's money you're after you must go where it is and where it is being spent also you must become personally acquainted with those who have the authority to spend it if you're a Salesman selling advertising and you


know the head of the firm is the man with the final say it's a waste of time trying to convince minor clerks and junior executives the same holds true if you're trying to sell other commodities or what is more important trying to sell yourself and finally the right mental attitude being properly attired keeping your eyes straight ahead and fixed on your goal throwing around you the proper aura which is done by an act of your imagination or an extension of your personal magnetism will work wonders Theo's Bernard in his penthouse of the


gods learned this when he was cornered and stoned by a crowd of natives in Tibet in his book he says his first reaction was to fight but the thought was immediately dismissed when he recalled that he had been taught to assume and maintain his aura thus he straightened his shoulders lifted high his head directed his eyes straight ahead and moved forward with a firm and rapid stride not only did the crowd give way but others came forward and made a path for him when man fully comprehends the great power of his mind


and honestly puts it to work he will have dominion over this earth and everything on it you yourself have this inner spark but it must be fanned until the fire is of white-hot intensity and it must be constantly stoked which you do by adding fuel ideas ideas more ideas and action chapter 8 belief makes things happen I have tried to make plain how this power through belief can be developed and to take you up the ladder as far as you wish to go it is necessary though to point out that it is easy to lose ones belief or faith thousands have risen to


great heights of success only to stumble row or fall to undreamed-of depths others seeking health have appeared to be more or less miraculously cured only to find that in later years or even months there is a recurrence of their ailments there are many weakening factors and influences all suggestive in nature which we in unguarded moments allowed to slip into our subconscious minds once these influences begin the destructive work they can undo all the good accomplished by our constructive forces so step out in front head toward the Sun


keep facing it and the dark shadows will not cross your path I know that it is difficult for the average person who knows nothing of the subject to accept the idea that all is within but surely the most materialistic person must realize that as far as he himself is concerned nothing exists on the outside plane unless he has knowledge of it or unless it becomes fixed in his consciousness it is the image created in his mind that gives reality to the world outside of him happiness sought by many and found by few therefore is a matter


entirely within ourselves our environment and the everyday happenings of life have absolutely no effect on our happiness except as we permit mental images of the outside to enter our consciousness happiness is wholly independent of position wealth or material possessions it is a state of mind which we ourselves have the power to control and that control lies with our thinking Emerson said what is the hardest task in the world to think obviously this is so when one considers that most of us are victims of mass thinking and feed upon suggestions


from others we all know that the law of cause and effect is inviolable yet how many of us ever pause to consider its workings the entire course of a man's life has many times been changed by a single thought which coming to him in a flash became a mighty power that altered the whole current of human events history is replete with the stories of strong-minded resolutely willed individuals who steadfastly holding to their inner convictions have been able to inspire their fellow man and in the face of tremendous and determined


opposition have literally created out of nothing great businesses huge empires and new worlds they had no monopoly of thought power you and every man and woman have it all you have to do is use it you will then become the person you envisage in your imagination know yourself know your power faithfully use the cards and the mirror techniques and you will get results far beyond your fondest expectations just believe that there is a genuine creative magic in believing and magic there will be for belief will supply the power which will


Transcriber: Leonardo Silva Reviewer: Denise RQ So, we all have some behavior that we would like to change about ourselves. And we certainly all want to help someone else change their behavior in a positive way. So, maybe it's your kid, your spouse, your colleague. So I want to share some new research with you that I think reveals something really important about what gets people to change their behavior. But before I do that, let's zoom in on one strategy that I think you probably use a lot. So, let's say you're trying to stop yourself from snacking.

How to motivate yourself to change your behavior | Tali Sharot | TEDxCambridge


What do you tell yourself? Well, most people, in a monologue, will say, "Beware. You'll be fat." And if this was your kid, you would probably tell him that smoking kills and, by the way, he's in big, big trouble. (Laughter) So, what we're trying to do here is we're trying to scare ourselves and others into changing their behavior. And it's not just us. Warnings and threats are really common in health campaigns, in policy. It's because we all share this deep-rooted belief that if you threaten people, if fear is induced,


it will get them to act. And it seems like a really reasonable assumption, except for the fact that the science shows that warnings have very limited impact on behavior. So, graphic images on cigarette packets, for example, do not deter smokers from smoking, and one study found that, after looking at those images, quitting actually became a lower priority for smokers. So, I'm not saying that warnings and threats never work, but what I'm saying is, on average, they seem to have a very limited impact. And so, the question is: why?


Why are we resistant to warnings? Well, if you think about animals, when you induce fear in an animal, the most common response you will see is freezing or fleeing; fighting, not as much. And so, humans are the same. So if something scares us, we tend to shut down and we try to eliminate the negative feelings. So, we might use rationalizations. For example, you might tell yourself: "My grandpa smoked. He lived to be 90. So, I have really good genes and absolutely nothing to worry about." And this process can actually make you feel more resilient


than you did before, which is why warnings sometimes have this boomerang effect. In other times, we simply put our head in the ground. (Laughter) Take the stock market for example. Do you know when people pull their head out of the ground to look at their accounts -- not to make a transaction, just to log in to check their account? So, what you're seeing here, in black, is the S&P 500 over two years, and in gray, is the number of times that people logged in to their account just to check. And this is data from Karlsson, Loewenstein & Seppi,


it's control [data] for all the obvious confounds. So, what do we see? When the market is high, people log in all the time, because positive information makes you feel good, so you seek it out. And when the market is low, people avoid logging in, because negative information makes us feel bad, so we try to avoid it altogether. And all this is true as long as bad information can reasonably be avoided. So, what you don't see here is what happened a few months later, in the financial collapse of 2008, when the market went drastically down


and that was when people started logging in frantically, but it was a bit too late. So, you can think about it like this -- it's not just finance: In many different parts of our life, (Laughter) we have warning signs and bad behaviors now. And they could potentially lead to all these bad outcomes later, but not necessarily so, because there are different routs from your present to your future, right? It can go this way, it can go that way. And, as time passes, you gather more and more information about where the wind is blowing.


(Laughter) And, at any point, you can intervene and you could potentially change the outcome, but that takes energy and you might tell yourself: "What's the point about worrying about something that might happen? It might not happen." Until we reach this point, at which time you do jump into action, but sometimes it's a little bit too late. So, we wanted to know, in my lab, what type of information does leak into people. So, we conducted an experiment where we asked approximately 100 people to estimate the likelihood


of 80 different negative events that might happen to them in the future. So, for example, I might ask you: "What is the likelihood that you'll suffer hearing loss in your future?" And let's say you think it's about 50%. Then, I give you the opinion of two different experts. So, expert A tells you: "You know, for someone like you, I think it's only 40%." So, they give you a rosier view of your future. Expert B says: "You know, for someone like you, I actually think it's about 60%. It's worse." So, they give you a bleaker view of your future.


What should you do? Well, you shouldn't change your beliefs, right? Wrong. What we find is that people tend to change their beliefs towards a more desirable opinion. In other words, people listen to the positive information. Now, this study was conducted on college students, so you might say: "Well, college students are delusional, right? We all know that." (Laughter) And surely, as we grow older, we grow wiser. So we said: "OK, let's test that. Does this really generalize? Does it generalize to your kid, to your parent?


Does it generalize to your spouse?" And so, we tested people from the age of 10 until the age of 80, and the answer was yes. In all these age groups, people take in information they want to hear -- like someone telling you you're more attractive than you thought -- than information that they don't want to hear. And the ability to learn from good news remained quite stable throughout the life span, but the ability to learn from bad news, that changes as you age. So, what we found was that kids and teenagers were the worse at learning from bad news,


and the ability became better and better as people aged. But then, around the age of 40, around midlife, it started deteriorating again. So, what this means is that the most vulnerable populations, kids and teenagers on the one hand, and the elderly on the other hand, they're the least likely to accurately learn from warnings. But what you can see here is that it doesn't matter what age you are. You can be 20, 30, 40, 50 or 60; everyone takes in information they want to hear more than information that they don't.


And so, we end up with a view like this of ourselves. (Laughter) Our mistake as teachers, as mentors, as employers is that, instead of working with this positive image that people so effortfully maintain, we try and put a clear mirror in front of them. We tell them: "You know, the image is just going to get worse and worse and worse." And it doesn't work. It doesn't work because the brain will frantically try to distort the image, using Photoshop and fancy lenses, until it gets the image it's happy with. But what would happen if we went along with how our brain works


and not against it? Take handwashing, for example. We all know that handwashing is the number one way to prevent the spread of disease, and this is really important in hospitals. So, in a hospital here in the United States, a camera was installed to see how often medical staff do, in fact, sanitize their hands before and after entering a patient's room. Now, the medical staff knew a camera was installed. Nevertheless, only one in ten washed their hands before and after entering a patient's room. But then, an intervention was introduced:


an electronic board that told the medical staff how well they were doing. Every time you washed your hands, the numbers went up on the screen and it showed you your rate of your current shift and the rate of the weekly staff. And what happened? Boom. Compliance raised to 90%, which is absolutely amazing. And the research staff were amazed as well, and they made sure to replicate it in another division in the hospital. Again, the same results. So, why does this intervention work so well? It works well because, instead of using warnings


about bad things that can happen in the future, like disease, it uses three principles that we know really drive your mind and your behavior. Let me explain. The first one is social incentives. In the hospital study, the medical staff could see what other people were doing. They can see the rates of the shift, the rate of the week. We're social people, we really care what other people are doing, we want to do the same and we want to do it better. This is an image from a study that we conducted, led by PhD student Micah Edelson,


and what it's showing you is a signal in the emotional center of your brain when you hear about the opinion of others. And what we found was that this signal can predict how likely you are to conform at a later time, how likely you are to change your behavior. So, the British government are using this principle to get people to pay taxes on time. In an old letter that they sent to people who "forgot" to pay taxes on time, they simply stressed how important it was pay taxes, and that didn't help. Then, they added one sentence,


and that sentence said: "Nine out of ten people in Britain pay their taxes on time." And that one sentence enhanced compliance within that group by 15%, and it's thought to bring into the British government 5.6 billion pounds. So, highlighting what other people are doing is a really strong incentive. The other principle is immediate rewards. So, every time the staff washed their hand, they could see the numbers go up on the board and it made them feel good. And knowing that in advance made them do something that they, otherwise, may not want to do.


Now, this works because we value immediate rewards, rewards that we can get now, more than rewards that we can get in the future. And people tend to think it's because we don't care about the future, but that's completely wrong, we all care about our future, right? We want to be happy and healthy in the future, we want to be successful, but the future is so far away. I mean, maybe you'll behave badly now and you'll be fine in the future, and maybe you'll be altogether dead. (Laughter) So, the here-and-now you would rather have that tangible drink,


that tangible T-bone, rather than something that's uncertain in the future. If you think about it, it's not altogether irrational, right? You're choosing something sure now rather than something that is unsure in the future. But what will happen if you reward people now for doing actions that are good for them in the future? Studies show that giving people immediate rewards make them more likely to quit smoking, more likely to start exercising, and this effect lasts for at least six months, because not smoking becomes associated with a reward,


and exercising becomes associated with a reward, and it becomes a habit, it becomes a lifestyle. So, we can reward ourselves and others now for behaving in ways that are good for us in the future and that's a way for us to bridge the temporal gap. And the third principle is progress monitoring. So, the electronic board focused the medical staff attention on improving their performance. This is an image from a study that we conducted, that shows you brain activity suggestive of efficient coding of positive information about the future.


And what we found was that the brain does a really good job at this, but it doesn't do such a good job at processing negative information about the future. So, what does this mean? It means that, if you're trying to get people's attention, you might want to highlight the progress, not the decline. So, for example, if you take that kid with the cigarette, you might want to tell them: "You know, if you stop smoking, you'll become better at sports." Highlight the progress, not the decline. Now, before I sum up, let me just share this small anecdote with you.


A few weeks ago, I got home and I found this bill on my fridge. And was really surprised because there's never any bills on my fridge. So, I was wondering why my husband decided to put that on our fridge. And so, looking at the bill, I could see that what this bill was trying to do is get me to be more efficient with my electricity use. And how was it doing it? Social incentives, immediate rewards and progress monitoring. Let me show you. Here are the social incentives. In gray is the energy use on the average energy use of people in my neighborhood.


And in blue is my energy use, and in green is the most efficient neighbor. And my reaction to this was -- my immediate reaction was: "I'm a little bit better than average" (Laughter) -- a tiny bit, but still... and my husband had exactly the same reaction -- and "I want to get to the green bar." And then, I got a smiley face. That was my immediate reward and it was telling me, "You're doing good," and it made me want to put this on my fridge. (Laughter) And although I have this one smiley face, I can see an opportunity there to get two smiley faces.


(Laughter) So, there's an opportunity for progress and it's showing me my progress throughout the year, how my energy use changes throughout the year. And the last thing this bill gave me: it gave me a sense of control. So, it gave me a sense of I was in control of my use of electricity. And that is a really important thing, if you try to get people to change their behavior, because the brain is constantly trying to seek ways to control its environment. It's one of the principles of what the brain is actually doing.


And so, giving people a sense of control is a really important motivator. OK. So, what am I not saying? I'm not saying that we do not need to communicate risks, and I'm not saying that there's one-solution-fits-all, but I am saying that, if we want to motivate change, we might want to rethink how we do it, because fear, the fear of losing your health, the fear of losing money, induces inaction, while the thrill of a gain induces action. And so, to change behavior in ourselves and in others, we may want to try these positive strategies


hi today we're looking at fix match simplifying semi-supervised learning with consistency and confidence by cukes on David birth Berthelot and others of Google research so this paper concerns semi-supervised learning so what does semi-supervised learning mean in semi-supervised learning you have a data set of labeled samples so right you have this data set of X's and corresponding Y labels but this data set sometimes is very small now you have a much bigger data set of unlabeled examples just X's with no labels right so you don't know

FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence


what the labels of the of the unlabeled examples are but what you would like to do is you would like to use this really large data set in order to help you with learning the association between the data points and the labels so for example in this case you would have something like like an image classification data set and I'm gonna take the example here of medical data so you have a pictures of lungs let's draw a long here that is an ugly long you have pictures of lungs and whether or not they are they have like a tumor in


them right so medical data is very hard to get especially labeled medical data because you need first of all you need the data itself but then you also need like like one at least one but ideally like three radiologists to look at whether or not this is a good or a bad image and label it so it's usually very expensive to collect that data but you might have plenty of unlabeled data right you might just be able to go who you're through through some database and find like anonymized undiagnosed long scans somewhere lying around the same


with image like other images so labeling images is pretty human intensive but the internet contains like a whole bunch of unlabeled images so the task of semi-supervised learning is how do you use this unlabeled data set in order to make your classification on the label data set easier and fix match combines two approaches to this in a smart way namely the consistency and confidence approach right so what does what does well it will jump right into into the method so basically what you want to do is you want to say my loss


that I optimized right this is my loss consists of two parts namely a supervised loss which is your classic classification loss right plus an unsupervised loss right and then you have like some sort of a trade-off parameter in front now your supervised loss here this is where this is just the the cross-entropy let's call it h between your predicted labels and your the actual true labels right and the predicted labels say they can be you know kind of a distribution over labels now the magic of course is here in the unsupervised loss and this


unsupervised loss this is what's described here in this part right so the unsupervised loss is going to be this age between P and Q and we'll see what P and Q is so if for the unsupervised loss you two of course want to start with an unlabeled example then you have the same sample go into two different pipelines in the first pipeline up here what you do is you so-called weekly augmented and here we're dealing with images so we have to talk about image augmentation so image augmentation has long been used in supervised learning to kind of give you


more it's kind of a cheat to give you more training data so if you have an image right of let's say famous cat you can obtain met more training data if you for example by random cropping so you can random crop let's say we just take this bottom right corner here and then we enlarge it to the original size right then it is still sort of a cat but it's just a part of a cat right but usually that helps because you you say okay um my image data set is just pictures of animals right it's entirely conceivable that someone held


the camera like this or like this right so technically in terms of generalizing to a test set these both data points should be valid so I'm just gonna add both to my training data so you can see how from one training data point you can get many training data points just by doing this cropping what you can also do is you can flip it left right right you just in swap the pixels left right and usually the these kind of um so a a a cat that has a little dark spot here is still a cat when it has too little dark spot over there right but to your


classifier those are two different samples so you can do many of those things and they have to kind of augmentations they have what they call weakly augmented and strongly augmented right so in the weakly augmented pipeline I think they just they crop and they they shift and they rotate or something like this so you can see here this this horsey here it is something like it's cropped here about then it is turned slightly to the left and then yeah I think that's it so they crop they rotate and then they also flip


horizontally at random in like 50 percent of the time so these are what's called weekly augmented the goal here is just to kind of obtain a bit more training data alright so you run this through your model through your classification model as you would a regular sample and you get a prediction now from your prediction you can take the highest prediction here and that is going to be your pseudo label so this is P of Y this is your distribution that you estimate right so and this and this if you just take the max this is


going to be your Y hat right and this is what they call a pseudo label sorry you'll see why it is called a pseudo label so the other pipeline here is the strong augmentation pipeline now in weak augmentation we just wanted to get somewhere training it in strong augmentation now the goal is to really screw up that picture to the point where it's still you know you could recognize it in the same class but you can see here the augmentations they go wild so you play around with the color with the hue you play around with the light


intensity right with the contrast you can do many many things you can see this this image looks basically nothing like this image buddied you can still kind of recognize it as a horse but the strongly augmented data is much more distorted than the weakly augmented data and that's the point so also you send the strongly augmented data through the model and again you get a prediction right and now is that the trick is you take the label from here and you you take that as if it were the true label right you take that as if it were the


true label and you form a loss from this prediction being the model prediction as if this thing here that also comes from the model as if that was the true label right that's why it's called a pseudo label because it is a label that you produce from the model itself now of course if these were to be the same picture it would be kind of pointless right that's why you see there needs to be a weekly and a strongly augmented pipeline I am pretty sure ammo if you want a more basic version of this make this just clean


so no augmentation and make this augment it right that's that's how you can think of it the fact that there is weak and here strong augmentation I think is just a your classic trick to get more training data but in essence you can think of it as this is here the clean thing you just want to produce a label and then you want the that an Augmented version of the image has the same label now you can think of it shortly what does this model learn if you just have this you remember I think the important thing is always to remember that there


are two components here right there is first the supervised loss this is the important one ultimately because we have the true labels right and then second there is the unsupervised loss which is just an auxiliary loss that is supposed to just kind of tune our model to the nature of the data right so don't forget that this this down here just concerns the unsupervised part of that loss so if you think what does the model actually learn when whenever you train it like this it basically learns to revert this strong augmentation right say basically


sells hey model whenever I give you a week augmented image and I distort it heavily right whenever I give you an image and that distort it heavily I want the label to be the same so the model basically learns that whatever the image the whatever the image the model at the end of the trend will be able to basically map any strongly augmented picture to the same class as a weekly augmented picture if it comes from the same source right so the model basically learned to ignore these kinds of augmentations that's what


this loss over here does it basically says these sorts of augmentations these sorts of distortions of images please ignore those because I always want you to output the same label here in the prediction here as if I had not distorted or just weakly distorted the image so that's that's what you have to keep in mind that this this loss is designed to make the model not distinguish between differently augmented versions of the same image and interestingly that really seems to help with the with the supervised loss right


my kind of hypothesis is is that all these methods what they're kind of trying to do is to just tune the neural network to the let's say the orders of magnitude of the of the input data and also to the kinds of augmentations that the humans come up with and that's a very important point so the Ottoman tations and here we said you know it's it's kind of a rotation and the crop the kind of augmentation really seemed to play a role so this paper finds that on C 410 where the state of the art I believe is something like ninety six ninety seven


percent accuracy on C for ten with just two hundred and fifty labeled examples right now the usual data set size is about fifty thousand it goes to ninety four point three four point nine percent so almost 95 percent accuracy with the state of the art being like ninety seven this is incredible with just two hundred and fifty labeled examples crazy right and it with only four labels per class it gets eighty eight point six percent so that's just forty images with labels they get a 8.6 percent of of the of accuracy


compared to the 97 percent that you get with like 50,000 images that is pretty pretty cool right simply by having all other images not labeled but pseudo labeled and consistency regularized right so the the two to two things that are combined by fixed match again or consistency regularization which basically it means that the model should output similar predictions when fed perturbed versions of the same image right this they they're really forthcoming that they are not the ones who invented this they just combine the


consistency regularization with the pseudo labeling now the pseudo labeling they have also not invented the pseudo labeling leverages the idea that we should use the model itself to obtain artificial labels for unlabeled data we've seen a lot of papers in the last few months or years where it's like the teacher teaches the student and then the student teaches the teacher model again and so on so that they simply combine the two methods in a clever way they have one last thing that is not in this drawing namely they only use the pseudo


label they have a break right here and they only use the pseudo label if if the confidence if this P of Y here is above a certain threshold so they don't take all the pseudo labels but they only take the labels where the model is fairly sure about right so they haven't actually an ablation study where they show that this is reasonably reasonably important and if you go down here where they say ablation or is it ablation ablation study oh yeah something I also find cool if you just give one image per class this one image per class ten