images that are labeled it still gets like 78 percent accuracy I think the images are chosen as good representations of their class but still one image per class pretty pretty cool an important part of this is the ablation study where they say okay we want to tease apart why this algorithm why this on semi-supervised learning technique works so well and they find several important factors they find for example that they're all mentation strategy is extremely important so how they augment the images is very important you see here the error of this


4.8% and the 250 label split if you change up the if you change up the the augmentation strategies your error gets higher right and so they say we use this cutout and we measure the effect of cut out we find that both cut out and seek T augment are required to obtain the best performance removing either results in in a comparable increase in error rate we've seen before for example they went they went from there some 93 sorry 93 point something percent to ninety four point something percent from the previous state-of-the-art

FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence


semi-supervised learning and here they find that simply changing the augmentation strategy changes the error by more than a percent so you can just see this in context of of what's important here right they say again the ratio of unlabeled data seems pretty important we observe a significant decrease in error rates by losing using a large amounts of unlabeled data right then the optimizer and learning schedule seems to be very important as well in that they use this they say STD with momentum works much better than


Adam and then they use this decreasing learning rate schedule this cosine learning rate schedule so there seem to be a lot of things a lot of hyper parameters that are fairly important here and you can see that the gains are substantial sometimes but they aren't like through the roof substantial where you can make a good argument that it is unclear how much really comes from this clever combination that fit fix match proposes and how much also just comes from whether or not you set the hyper parameters correctly and exactly how


much computation are you able to throw at selecting your selecting your hyper parameters so that that seems to be a bit of a a bit of a pain point for me they also say we find that tuning the weight decay is exceptionally important for low label regimes right choosing a value that is just one order of magnitude larger or smaller than optimal can cost ten percentage points or more and so that all of that seems to me that this this kind of research where you're you're nibbling for half or single percentage points in accuracy while a


this is Earl Nightingale the purpose of this recording is to tell you about and try to condense one of the most amazing books ever written Think and Grow Rich by Napoleon Hill without question this single book has had a greater influence on the lives accomplishments and fortunes of more individuals than any other work of its kind all over the free world there are literally thousands of successful men and all lines of work who are where they are today because they once picked up and bought a copy of Think and Grow Rich and they'll be quick

Napoleon Hill's Think & Grow Rich  Condensed and Narrated by Earl Nightingale


to tell you so I first discovered this remarkable book in the fall of 1949 it was an enormous help to me it helped me decide once and for all how I was to accomplish my goal it unified my thinking and gave me a straight clear Road to the point I had decided to reach one of my closest friends found the book several years ago and stayed home for three days reading and digesting its material and he then went on to reach the top in his industry i sat in richly paneled carpeted executive offices and listened to world


famous business leaders some of them old enough to be my father tell me that everything worked out fine after they had read Think and Grow Rich now what's the secret of this amazing book why has this book out of all the thousands of self-help books remain the one towering giant I think to understand this you have to know Napoleon Hill as I do he certainly was not the first man to be appalled at the poverty and seemingly endless struggle and lack of direction he saw about him as a boy and as a young man nor was he the first to write on the


subject but he possessed two unique highly developed abilities seldom found in one man the first was in the manner in which he approached his subject Napoleon Hill went after the answers to achievement in the same way a scientist seeks to open to the light of Reason a secret of nature he went after the solution to accomplishment in the same way Thomas Edison discovered the solution to the electric light relentlessly indefatigably implacably until the truth which had been there all the time was revealed to him his second


important ability was the knack or skill of writing about his findings in such a way that it was instantly understood intellectually but what is perhaps even more important for this particular subject understood emotionally as well on the last page of Think and Grow Rich was read the hand which put the book down on the table was a different hand the man who then stood and walked out into the world was a different a changed man the suffocating entangling webs of self-imposed frustration and interaction had fallen


away and now the way was clear the man was now the possessor of the unique unseen talent for turning dreams into reality thoughts into things so called fate of the idle effects of exterior circumstances were no longer in command he who had been a passenger was now suddenly the captain to begin we have to understand the simple truth the principle or philosophy which lies is the supporting structure of this work unless whatever it is you build is based on truth you will end with the entire structure fallen and scattered about you


like the armor of Homer's ancient warriors it simply cannot stand it cannot withstand the test of time the reason thinking Grow Rich has withstood the test of time is because it rests on the broad clean foundation upon which may also be found every accomplishment of man the clear unchallengeable fact that everything begins with an idea a philosophy based on the fact that riches of every kind begin with the State of Mind that one may start with nothing but thoughts ideas and organized plans thoughts are sings incredibly powerful things when


mixed with definiteness of purpose persistence and the burning desire for their translation into material objects or riches riches being whatever it is you happen to want wise men have been saying this for centuries and just recently Charles a sarami wrote the truth is that the human mind is as real and organism as any muscle in the body but far greater in potential power and like muscle fiber it can be strengthened to lead on to unimaginable is based if you know what you want and if you want it strongly enough to muster the kind of


persistence that simply cannot be stopped you most certainly achieve it by controlling your mind you can control your destiny here on earth with this as our foundation let's talk about napoleon hill's famous 13 proven steps to riches as found in his book Think and Grow Rich remembering of course that riches are whatever it is you happen to want and right here let me make two important points the first is that whenever you listen to this record have a notebook handy and make notes as we go along the second is that this record was produced


for your own personal use to go with you on your own exciting journey to play this record for a group will prove to be of only temporary help make sure you have your own personal copy to play again and again particularly at those times when you may feel yourself getting off the track and now napoleon hill's famous 13 principles you will notice that we have separated each principles by bending them on the record in this way you're given quick access to any particular principle to which you may wish to return for reference the first


principle desire here is the starting point for all achievement that first step toward riches but it's right here that we so often run into a stumbling block a person will say I know what I desire but can I get it we'll get into this business of doubt later but once and for all let's clear up this point this point of whether or not you can accomplish that which you desire with all your heart I think it was best expressed by Emerson who wrote there is nothing capricious in nature and the implanting of a desire indicates that


its gratification is in the constitution of the creature that feels it in other words you would not have the desire unless you were capable of its achievement each of us has a built-in governor and our desires are modified by our abilities and leanings whatever it is that you desire with all your heart understand once and for all that it can and should be yours in Think and Grow Rich Napoleon Hill cites example after example of why you're burning desire is nothing more than an accurate picture of what you will one day become


so right here firmly establish in your mind that which you desire more than anything else for as hell vicious put it by annihilating the desires you annihilate the mind every man without passions has within him no principle of action nor motive to act a good way to determine whether or not you really have a burning desire is to examine the way you go after it if you go after that which you think you desire tentatively timidly in an attempt to play it safe you don't have a burning desire at all you can't get the second base if you


keep one foot on first but if you're willing to burn your bridges behind you and say once and for all this is it this is what I will do and I will never retreat I'll never go back then you have the sort of desire that can only end in success it takes that kind of resolve to be able to keep picking yourself up after there falls you're bound to take the only people who don't make mistakes are those who never try anything the timid feeders in the lagoon who never venture into the broad deep sea beyond well these principles


will work for anything you may want a more harmonious home life a more successful career for our example let's say your desire happens to be more money to better care for your family and provide for your future years to get your share of the prosperity that lies ahead Napoleon Hill gives us six definite practical steps to follow 1 fix in your mind the exact amount of money you desire it is not sufficient merely to say I want plenty of money be definite as to the amount there's a psychological reason for definiteness


which will be described in a subsequent principle number 2 determine exactly what you intend to give in return for the money you desire there's no such reality as something for nothing 3 establish a definite date when you intend to possess the money you desire and 4 create a definite plan for carrying out your desire and begin at once whether you're ready or not to put this plan into action 5 write out a clear concise statement of the amount of money you intend to acquire name the time limit for its acquisition state what you


intend to give and return for the money and describe clearly the plan through which you intend to accumulate it 6 read your written statement aloud twice daily once just before retiring at night and once after a rising in the morning as you read see and feel and believe yourself already in possession of the money or whatever your goal happens to be it's important that you follow these instructions to the letter play this part of the record over until you have it down to your satisfaction for this is by far the most important of


the 13 principles and this chapter of the book ends with these words through some strange and powerful principle of mental chemistry which she has never divulged nature wraps up in the impulse of strong desire that's something which recognizes no such word is impossible and accepts no such reality as failure the second principle is faith you never would have even thought of your main desire unless faith were tugging at your mind and if you find it difficult at times to have faith in yourself you can be certain that you can have faith in


these principles Napoleon Hill writes faith is a state of mind which may be induced or created by affirmation or repeated instructions to the subconscious mind through the principle of conscious Auto suggestion conscious auto suggestion simply means a suggestion by yourself to yourself just as an autobiography is a biography written by the person it's about by getting a mental image of yourself already having accomplished your main desire over and over again you will muster the faith you need faith is vital to accomplishment the emperor napoleon


once said all the scholastic scaffolding falls as a ruined edifice before one single word faith Pascal said faith affirms many things respecting which the senses are silent but nothing which they deny it is superior to their testimony but never opposed to it Goethe said epochs of faith are epics of fruitfulness but epochs of unbelief however glittering a barren of all permanent good and a Schlegel put it in actual life every great Enterprise begins with and takes its first forward step in faith have faith that you can accomplish that which


you seek for you would never have decided upon it unless it was meant for you to accomplish in his chapter on faith Napoleon Hill gives us a self-confidence formula first I know that I have the ability to achieve the object of my definite purpose in life therefore I demand of myself persistent continuous action toward its attainment and I hear and now promise to render such action second I realize that dominant thoughts of my mind will eventually reproduce themselves in outward physical action and gradually transform themselves into physical


reality therefore I will concentrate my thoughts for 30 minutes daily upon the task of thinking of the person I intend to become thereby creating in my mind a clear mental picture of that person third I know through the principle of Auto suggestion any desire that I persistently hold in my mind will eventually seek expression through some practical means of attaining the object back of it therefore I will devote ten minutes daily to demanding of myself the development of self-confidence fourth I have clearly written down a description


of my definite chief aim in life and I will never stop trying until I have developed sufficient self-confidence for its attainment fifth I fully realize that no wealth or position can long endure unless built upon truth and justice therefore I will engage in no transaction which does not benefit all human effects I will succeed by attracting to myself the forces I wish to use and the cooperation of other people I will induce others to serve me because of my willingness to serve others I will eliminate hatred envy jealousy


selfishness and cynicism by developing love for all humanity because I know that a negative attitude toward others can never bring me success I will cause others to believe in me because I will believe in them and in myself in rereading thinking Grow Rich so that I could write this condensation for recording I was forcibly struck all again by this great chapter on faith particularly the examples of how some of the world's greatest men have accomplished what appeared to be impossible through faith the third principle is Auto suggestion


now we've already touched on this this chapter the book tells us how through repeated suggestion the subconscious mind can be put to work for us it is the Faculty of being able to concentrate your mind on your burning desire until your subconscious mind accepts it as fact and begins to devise ways of bringing it about here is where hunches come from sudden flashes of thought or inspiration guidance the instructions given in connection with the six steps in the second chapter will now be summarized and blended with the


principles covered by napoleon hill's chapter on Auto suggestion first go into some quiet spot perhaps in bed at night close your eyes and repeat aloud so you may hear your own words the written statement of the amount of money you intend to accumulate or a careful reaffirmation of whatever your goal happens to be the time limit for its accumulation and a description of the service or merchandise you intend to give in return for the money as you carry out these instructions see yourself already in possession of your


goal for example suppose that you intend to accumulate fifty thousand dollars by the 1st of January five years from now that you intend to give personal services in return for the money in the capacity of a Salesman your written statement of your purpose should be similar to the following by the first day of January 19 whatever it happens to be I will have in my possession fifty thousand dollars which will come to me in various amounts from time to time during the interim in return for this money I will give the most efficient


service of which I'm capable rendering the fullest possible quantity and the best possible quality of service in the capacity of salesmen of and here describe the product or service you intend to sell or whatever it is you do for a living it goes on I believe that I will have this money in my possession my faith is so strong that I can now see this money before my eyes I can touch it with my hands it is now awaiting transfer to me at the time and in the proportion that I deliver the service I intend to Ren in return for it I am awaiting a plan by


which to accumulate this money and I will follow that plan when it is received second repeat this program night and morning until you can see in your imagination the money you intend to accumulate third place a written copy of your statement where you can see at night morning and read it just before retiring and upon arising until it's been memorized as you carry out these instructions you are applying the principle of auto suggestion the fourth principle is specialized knowledge it is here that I think Napoleon Hill makes a


very important point knowledge is power only to the extent that it is organized into a definite plan of action and directed to a definite end to quote from the book before you can be sure of your ability to transmute desire into its monetary equivalent you will require specialized knowledge of the service merchandise or profession which you intend to offer in return for fortune perhaps you may need much more specialized knowledge than you have the ability or the information require and if this should be true you


may bridge your weakness through the aid of your mastermind group more on this later but for now realize that you must learn all you can about your specialty set aside a definite time every day for learning more about what it is you do for a living take the courses that are offered on your subject and associate with men who know your business well the fifth principle is imagination the imagination is literally the workshop where in our fashion all plans created by man the impulse the desire is given shape form and action through the aid of


the imaginative Faculty of the mind it has been said that man can create anything he can imagine as Napoleon Hill says and teaches whatever the mind of man can conceive and believe it can achieve man's only limitation within reason lies in his development and use of his imagination and subsequent motivation to action the great leaders of business industry finance and the great artists musicians poets and writers became great because they developed the power of self motivation incidentally one of best books ever put together on this


subject if not the greatest is success through positive mental attitude by Napoleon Hill and W Clement stone I suggest you get a copy from your bookstore at your earliest convenience if your bookstore happens to be out of the book you may obtain a copy by writing to the address on the label of this record as you go about your daily work think constantly of ways in which it could be done better more efficiently think of the changes that are inevitable can they be made now and if you feel limited remember the words of the late


Frank Lloyd Wright he said the human race built most nobly when limitations were greatest and therefore when most was required of imagination in order to build it all limitations seemed to have always been the best friends of architecture as you build your future from this point onward don't concern yourself with limitations but remember that they may be your best friends since they require imagination if were to rise above them and this Beecher said the soul without imagination is what an observatory would be without a telescope


now if you turn this record over we'll get to the sixth principle the sixth principle is organized planning you have decided on your desire your goal now let's organize the plan for its accomplishment right on schedule let me quote again from Think and Grow Rich you have learned that everything man creates or acquires begins in the form of desire the desire is taken on the first lap of its journey from the abstract to the concrete in the workshop of the imagination for plans for its transition are created and organized


earlier you were instructed to take six definite practical steps as your first move in translating the desire for whatever you want into its physical equivalent one of these steps is the formation of a definite practical plan or plans through which this transformation may be made one ally yourself with one or more persons a group of as many people as you may need for the creation and carrying out of your plan or plans for the accumulation of the money you've established as your goal making use of the mastermind


principle this is important too before forming your mastermind Alliance decide what advantages and benefits you may offer the individual members of your group in return for their cooperation no one will work indefinitely without some form of compensation although this may not always be in the form of money 3 arrange to meet with the members of your mastermind group at least twice a week and more often if possible until you have jointly perfected the necessary plan or plans for the accomplishment of your goal 4 maintain perfect harmony


between yourself and every member of your mastermind group keep in mind these facts first you're engaged in an undertaking of major importance to you to be sure of success you must have plans which are faultless second you must have the advantage of the experience education native ability and imagination of other minds this is in harmony with the methods followed by every person who has risen above the average work at this until you have a well-executed formal plan for reaching your objective in this way you're never


confused or wondering what you should do next every morning you know exactly what you're going to do and why it is in this chapter of thinking Grow Rich that Napoleon Hill gives us his eleven qualities of leadership one unwavering courage two self-control 3 a keen sense of justice four definiteness of decision 5 definiteness of plans 6 the habit of doing more than paid for 7 a pleasing personality 8 sympathy and understanding 9 mastery of detail 10 willingness to assume full responsibility and 11 cooperation the


chapter on organized planning is one of the largest and most important in the book it goes without saying that a man without a plan to follow is like a ship without a course no place to go with disaster a probability the seventh principle decision the mastery of procrastination to quote accurate analysis of over twenty five thousand men and women who had experienced failure disclosed the fact that lack of decision was near the head of the last of the thirty major causes of failure this is no mayor's statement


of a theory in his effect procrastination the opposite of decision is a common enemy which every man must conquer analysis of several hundred people who had accumulated fortunes well beyond the million dollar mark disclosed the fact that every one of them had the habit of reaching decisions promptly and of changing these decisions slowly if and when they were changed people who fail to accumulate money without exception have the habit of reaching decisions if at all very slowly and of changing these decisions quickly and


often a definite objective makes reaching prompt decisions that much easier Napoleon Hill gives many examples one of which is the case of Henry Ford one of Henry Ford's most outstanding qualities was his habit of reaching decisions quickly and definitely and changing them slowly this quality was so pronounced in the late mr. Ford that it earned form the reputation of being obstinate it was this quality which prompted mr. Ford to continue to manufacture his famous Model T the world's ugliest but for the time most


practical car when all of his advisors and many of the purchases of the car were urging him to change it perhaps he delayed too long in making the change but the other side of the story is that his firmness of decision yielded a huge fortune before the change in model became necessary and the company's certainly none the worse for today when you make up your mind stay with it the majority of people who fail to make the grade are generally easily influenced by the opinions of others easily swayed they permit the newspapers and the


gossiping neighbors to do their thinking for them opinions are the cheapest commodities on earth keep your own counsel when you begin to put into practice the principles we're describing here by reaching your own decisions and following them take no one into your confidence except the members of your mastermind group and be very careful in your selection of this group that you choose only those who will be in complete sympathy and harmony with your purpose close friends and relatives while not meaning to do so often


handicapped one through a opinions and sometimes through ridicule thousands of men and women carry inferiority complexes with them all through life because some well-meaning but ignorant person destroyed their confidence through opinions or ridicule if a decision is worth anything at all it's worth sticking to until it's been completely worked the eighth principle persistence Napoleon Hill defines persistence as the power of will willpower and desire when properly combined make an irresistible pair persistence to an individual is what


carbon is to steel in uncounted thousands of cases persistence has stood as the difference between success and failure it is this quality more than any other that keeps the majority from great accomplishment they will try a thing but as soon as the going gets tough they fold experience with thousands of people has proved that lack of persistence is a weakness common to the majority of men it is a weakness which may be overcome by effort if you are to accomplish the desire you've set for yourself you must form the habit of persistence things


will get dark it will seem as though there's no longer any reason to continue everything in you will tell you to give up to quit trying and it's right here that the men are separated from the boys it's right here that if you'll go that extra mile and keep going that the skies will clear and you'll begin to see the first signs of the abundance that is to be yours because you had the courage to persist with persistence will come success persistence is a state of mind therefore it can be cultivated like all states of mind persistence is based upon


definite causes among them these 1 definiteness of purpose knowing what you want to desire 3 self-reliance 4 definiteness of plans 5 accurate knowledge knowing that your plan is sound 6 cooperation sympathy understanding and harmonious cooperation with others tend to develop persistence 7 willpower 8 Hemet persistence is the direct result of habit the ninth principle power of the mastermind it is in this section that Napoleon Hill describes the importance of forming a group of individuals sympathetic to your desire they may be


individuals with similar plans a mastermind group can be made up of two or more individuals no two Minds ever come together without thereby creating a third a third invisible intangible force which may be likened to a third mind you may have noticed many times that by discussing something with another individual you suddenly get good ideas as a result of this Association ideas you would not have gotten without this Association the same thing happens to the other person a lot of good ideas have been born in individual minds as a result of having


met in committee associating with your mastermind group is not meant as a means of letting others do your thinking for you far from it it is to stimulate your own thinking through the association with other minds no one knows everything the more sympathetic Minds you get together and by sympathetic I mean working for a common purpose the more related information is going to be available and great ideas are a combination of related information so pick the member or members of your mastermind group with care make sure


there are people you respect and who are hardworking and conscientious you have a lot of fun and you'll all reach your goals just that much sooner the tenth principle could be called enthusiasm that is the enthusiasm that comes from the channeling of all bodily drives into positive worthwhile outlets it is in this chapter that Napoleon a describes the importance of the woman the one and only woman in the achievement of a worthwhile goal it seemed quite significant to mr. Hill that practically every great leader was a man whose


achievements were largely inspired by a woman when things get tough and you can count on if they will you may be deserted by what you thought were friends but if you've got a good woman you'll never be alone she'll be willing to start over again if necessary and she'll give you the new enthusiasm that comes through her faith in you having someone to love is having someone to share your success and accomplishment to give you the praise that all of us need from time to time a man can become successful without a wife and family but


all the real joy is taken out of it take care of your wife and children as your greatest possessions the 11th principle has to do with the subconscious mind the subconscious mind consists of a field of consciousness in which every impulse of thought that reaches the objective mind through any of the five senses is classified and recorded and from which thoughts may be recalled or withdrawn as letters may be taken from a filing cabinet it receives and files sense impressions or thoughts regardless of their nature you may voluntarily plant


in your subconscious mind any plan thought or purpose which you desire to translate into its physical or monetary equivalent the subconscious acts first on the dominating desires which have been mixed with emotional feeling such as faith your subconscious mind works night and day through a method of procedure unknown to man the subconscious mind draws upon the forces of infinite intelligence for the power with which it voluntarily transmutes one's desires into their physical equivalent making use always of the most


practical media by which this end may be accomplished you cannot entirely control your subconscious mind but you can voluntarily hand over to it any plan desire or purpose which you wish transformed in a concrete form no one knows very much about what we call the subconscious or unconscious mind we do know that it is incalculably powerful and can solve our problems if we go about using it the right way and the best way is to hold in your conscious mind as often as possible a clearer picture of yourself already


having accomplished your goal you know what you want define it clearly and then project it on the motion picture screen of your mind hold it see yourself doing the things and having the things you'll have when your objective will have been reached do this as often as possible as you go about your daily work and particularly at night just before you go to sleep and the first thing upon arising as you do this your subconscious will begin to lead you in the most logical ways toward your objective don't fight it follow


your sudden hunches the ideas that come into your mind knowing that it's your subconscious trying to get through to your conscious mind if you keep at this you'll be amazed and delighted by the wonderful ideas that just seem to come from nowhere in the next principle we'll talk some more about this sixth sense that seems to control the lives of the great men and women but it comes from a systemic triggering of the subconscious mind the lives of the great men and women which seem miraculous to the average person are nothing more than the


fulfillment of their burning desires through the power of their subconscious minds time means nothing to your subconscious a man could work steadily at his job for forty years and not accomplish as much as is possible in three or four years through the proper working of this principle your subconscious mind cannot remain idle if you fail to plant desires in your subconscious mind it will feed upon the thoughts which reach it as a result of your neglect remember that you're living daily in the midst of all manner of


thought impulses which are reaching your subconscious mind without your knowledge some of these impulses are negative some are positive you are now engaged in trying to help shut off the flow of negative impulses and to aid in voluntarily influencing your subconscious mind through positive impulses of desire when you achieve this you will possess the key which unlocks the door to your subconscious mind bulwa wrote the man who succeeds above his fellows is the one who early in life clearly discerned his object and toward


that object habitually directs his powers even genius itself is but fine observation strengthened by fixity of purpose every man who observes vigilantly and resolves steadfastly grows unconsciously into genius the key word there is unconsciously know what you want decide once and for all that it will be yours remain steadfast on course propelled by faith and your subconscious or unconscious mind will do the rest the twelfth principle as outlined in thinking Grow Rich by Napoleon Hill has to do with the brain if you had access


to all the wealth in the world and took a penny you would be doing exactly what you very probably have been doing in the use of your brain nothing in the world is more pitiful than the misunderstanding by the average person of the power of his brain and the minds to which it is connected the conscious and the subconscious you own in your brain the most marvelous miraculous inconceivably powerful force the world has ever known take for example the fact that the number of lines which connect the brain cells with one another equal a figure


one followed by 15 million ciphers it has been determined that there are from ten to fourteen billion cells and the average human cerebral cortex it is inconceivable that such a network of intricate machinery should be in existence for the sole purpose of carrying on the physical functions incidental to growth and maintenance of the physical body this is the mechanism that has given us the supersonic airplane our deep rocket probes into outer space the sciences the arts all that we know and use today and will use


tomorrow have hatched from this small grey mass each of us carries around do you can you doubt even for a moment that it can bring you and yours everything you want here on earth of course it can if you will recognize your power as an individual and stop acting like those who have never even thought about it give it the job you've decided to accomplish and watch it handle it the thirteenth and final principle is called the sixth sense the sixth sense can be described as the sense through which infinite intelligence may and will


communicate voluntarily without any effort from or demands by the individual this principle is the apex of the philosophy it can be assimilated understood and applied only by first mastering the other twelve principles the sixth sense is that portion of the subconscious mind which has been referred to as the creative imagination it has also been referred to as the receiving through which ideas plans and thoughts flash into the mind the flashes are sometimes called punches or inspirations the sixth sense defies description it


cannot be described to a person who has not mastered the other principles of this philosophy because such a person has no knowledge and no experience with which the sixth sense may be compared the sixth sense is not something that one can take off and put on a twill ability to use this great power comes slowly through application of the other principles we've outlined many individuals come into a workable knowledge of the sixth sense even before the age of 40 but more often the knowledge is not available until one is


well past 50 and this for the reason that the spiritual forces with which the sixth sense is so closely related do not mature and become usable generally except through years of meditation self-examination and serious thought but begin to develop it now by applying the principles we've talked about here remember this man can create nothing which he does not first conceived in the form of an impulse of thought mens thought impulses begin immediately to translate themselves into their physical equivalent whether those thoughts are


voluntary or involuntary keep fear out of your mind by concentrating on the mental picture of your goal your greatest desire now I want to mention that Think and Grow Rich as a book carries more endorsements by great men who knew of its truth than any other book of its kind ever written I'll touch on only a few all of these great men actually endorsed Napoleon Hills principles former President of the United States William Howard Taft FW Woolworth wrote that he had built his great chain of stores by applying many


of these principles Robert dollar who built the great dollar steamship line wrote if I had had this philosophy 50 years ago I suppose I could have accomplished all that I've done in less than half the time I sincerely hope the world will discover and reward you Samuel Gompers wrote that the mastery of these principles is the equivalent of an insurance policy against failure President Woodrow Wilson John Wanamaker the merchant Prince wrote I know that your fundamentals are sound because I've been applying them in my


business for more than 30 years George Eastman the world's largest maker of cash Thomas Edison Luther Burbank Theodore Roosevelt iam Statler john d rockefeller i think you know that the endorsements of these great men could not have been bought with all the money in the world what we have been talking about here can change your life can bring you anything and everything worthwhile you want in life for yourself and your family cut yourself away from the average from the mediocre and chart your course on the dream in your heart these thirteen


principles will never let you down as long as you use them and now it is with great pleasure that i give you the author himself napoleon hill thank you era nightingale the message which you who are listening have just heard has brought you within three short steps of the supreme secret of success the same secret which has brought happiness peace of mind and financial success to countless thousands of people who have read Think and Grow Rich the same secrets which has made master salesmen out of ordinary order takers and the


same secret which has brought friendship love and marriage to men and women who have come under the spell of the thirteen principles which Earl Nightingale has just described you desire the better things of life or you wouldn't be listening to this record come with me then and I will help you chart your course so you may acquire whatever it is that you desire most in life by following these instructions one condition your own subconscious mind to work for you while you sleep as well as when you are awake you can do this by


playing this record to yourself every night just before you retire then before you go to sleep after hearing the record write out a clear statement of what you wish to accomplish the following day and request your subconscious mind to work during the night and provide you with the plan you will need to achieve your purpose to form a personal mastermind group of two or more people who are closely associated with you they can be members of your family your business or your professional associates or people who work where you are employed and play


this record in their presence at least once a week after you have done this engaging friendly roundtable discussion of the thirteen success principles and requests each person in the group to participate and to look for inspirational ideas which may help him or her to become more successful three beginning now follow the habit of rendering more service and better service than that which is expected of you and do it in a pleasing positive mental attitude this will make friends for you it will increase the value of your services and


it will attract opportunities by which you may get from life whatever it is that you most desire follow these instructions to the letter and you ensure to see the day when you can express a prayer of gratitude for having had the privilege of hearing and acting on the message which Earl Nightingale has given you through this record and now may I reach out across the space and the time which separate us and offer you a hand of friendship and a sincere prayer that you will be blessed with a richer and fuller life because of this


[Music] professor we might call this a biography of your curiosity so I want to see some of the origins of your curiosity perhaps in childhood the kinds of books you might have been reading the kind of child you were intellectually um where are we first of all in your childhood let's see I guess I'm about five let's start about five okay and when where is the location I'm on a bus coming home from school in the countryside okay my English countryside in the English country yeah my mother was a schoolteacher ah she said the

The Heidelberg Laureate Forum Foundation presents the HLF Portraits: Geoffrey E. Hinton


headmistress of a little school in the countryside about ten miles outside of Bristol in the UK okay and I went to the set of the school of where she was headmistress Oh which is the nightmare it had some disadvantages occasionally the kids would surround me in the pay playground and explained to me that I had to persuade my mother to do so it's always the kids that make it in a bit yes but it wasn't too bad but you survived that I survived that so your five on a bus and five on a bus and it the bus has velvet seats where this


seats covered with the material that's got a heavy pile and I'm sitting on the bus and I put a penny and old British pennies were big I put his name on the velvet seat and it moved and it went uphill and this was clearly impossible and I was absolutely fascinated by the fact that you put this penny down it wouldn't always do it but sometimes it would go up hill and it's a bit like those dreams you have I guess I have not everybody has but I think a lot of people have them when you suddenly discover you can fly and in the


dream yes you think this can't be real yes yes but you tried again you can fly and it's amazing and you know it can't quite be real but it's just amazing and it's wonderful that you can fly well this penny moving uphill was the same kind of thing it was sort of weird and wonderful I'm like the first time you discover magnetism and I couldn't believe what was happening and at the time I just was amazed by it and I was puzzled by it for a long long time and possibly a little experimental you tried it a few times I tried a few times and


it didn't work reliably but it did work and I remembered a lot about the circumstances on that bus I don't I remember which side of the bus we were sitting yes yes I'm wearing the bus we were sitting you for years I just didn't understand how this penny gone uphill and it was a sort of lurking memory that was this weird thing that happened that couldn't possibly have happened wonderful now did you rush to the nearest adult and say I've noticed this can you help me understand why or did you just chemo I guess I was yeah I'm


not that kind of a person I want to figure it out myself okay okay exactly and so I just couldn't figure it out did you make any sense and but it stayed in my mind and and I think might a lot of my research careers like that there's these little niggly things that don't make any sense and they just stay in my mind wonderful and with this when I was a teenager and I guess when I started physics at school although you wouldn't need to study physics for this I finally understood what must have been handling because the other thing I remember about


the bus was it vibrated a lot it had an engine a big engine at the front that was somehow loose that made the whole bus vibrate yes and so clearly what was happening was the flock on the material was at an angle and it was an angle facing upwards yes so it was slick yes the seat was kind of like this and the flock was like this and so the penny couldn't go down so when it vibrated well when it vibrated this way it will go up and he couldn't vibrate down so that was what was making that making it move up did you actually pursue the


problem eventually or it just came to me when it escaped you when I understood enough about things like vowels I guess I'm going to boringly jump to a more formal part of your education although that was that was the gem of understanding your curiosity but are you getting a good education are you in a well there's the the school your mother is head miss yourself but then the next level are you getting teaching in science are you finding teachers that interest you yes and no yes so he's I was at my mother's school in the countryside which


was mainly children of farmers yes and had a vet they had a very strong Somerset accent yes um there's actually quite a lot of German in the Somerset they use words that sound German to me and I think we must be German in origin and so I had this very strong agricultural accent and I when my mother got pregnant with my younger sister she had to leave the school so she would have been I'd being about six then yes just turning six heading and I went to a primary school in near where I lived and then my parents wanted me to go to the


grammar school which had a it had a sort of genius school I can't on board age but and ice at an exam for the grammar school when I failed it so so then my parents decided to send me to the a private school our school I never liked it was called Clifton College yeah I'm yes yes I know and they wouldn't let me in I mean they were willing to take me in the junior school but the prep the prep school it was called preparation for the big school but they said I couldn't come until something had been done about my


accident your accent my accent I wasn't allowed into the school because my accent and they said I'd have to stay in the primary school for another year until my accident be fixed may I color than only in England situation maybe I don't know no fair enough all right so you went about improving your accent or your your parents just I I mean at home the accent was fine no but you was the cool accent yeah yeah and yeah and in the local primary school my accident became more acceptable right and then I went to this prep school and


it was a British prep school I didn't like it I was completely out of place and isolated there uh because my father was a Stalinist my mother was a staunch Labour Party supporter yes the Secretary of the local leg up ah ah so you were a classy anyway I'd been taught that religion was just nonsense yes something I still believe and I mean it's just patently nonsense and this was a sort of I call it muscular Christianity that is it wasn't sort of particularly religious but it was a Christian school and they yes they


thought the things that were good for boys were things like playing playing rugger and going to church and so we had to go to a service every morning and I felt well years later at the service in the morning when I was much older when as a teenager I also had to come in on Sundays because I lived at home most of the kids in the school were boarders who lived in the school but maybe sort of 20% of the school with day boys who lived in Bristol yes and we had to come in on Sundays yeah they were service on Sunday there was no excusing your


absence you couldn't just be at home on Sunday started walking on Sunday service service and they would invite Headmaster's of other schools to come and give sermons and I remember a headmaster for another school giving a sermon about how awful Russia was and how they had forced ideological education in Marxism and people had to sort of they weren't they couldn't be a fuse they had to sit there and they had to listen to this Marxist nonsense I'm forcibly and I remember sitting there thinking you know I'm sitting here


I'm forcibly listening to this religious nonsense and I almost got up and walked out and I somehow wish I had I'd have been thrown out of the school but I wish I'd done that um few would have had the Kurds to do it I didn't I didn't no one is it's no one noticing your intelligence in this muscular school now um there were some people who some teachers who were very nice to me there was a math teacher called stp Wells and they I have no idea what his first name was Jersey school he was mr. wells um STP wells I remember he was


tall and thin and he loved math unlike many math teachers in junior schools he actually thought math was fun um he had curious ideas about discipline so I remember he did things you would never get away with nowadays so I remember a class where there were two boys talking at the back sitting across the row from each other and they were they were talking softly together well he was trying to explain something so remember he walked down the row grabbed their two heads by the hair and bang them together that was not accepted


practice now even even in such a school it seems he quite muscular to me I remember my my introduction to the school in probably in my first few weeks there there was a teacher who was the art teacher who was fairly sadistic and you won't like to run in the changing room so you changed for games you hung up your clothes she put on your sports clothes yes you came back you changed again and if you ran in the changing room you explained that was not allowed so well we were changing back into our clothes another boy stole my tie and there was a


kind of partition down the middle of the changing room and he and avoid the other side of the partition would throw the tie from one to the other if I were you I would kind of run around the petitioner try and get it and as I was trying to run around the petition to get my tie this art teacher it's curious I remember that he was Catholic I think this is how prejudice really gets going and I've never forgiven Catholics for him he came into the change room I was running so he grabbed me he marched me down the corridor to the Headmaster's


office and he beat me with a bamboo cane quite severely my trousers were still on but he really he really enjoyed it and I just remember huge sense of grievance but he wouldn't listen to explanations yes I just weren't being enormous ly aggrieved at him and at Catholics but I'm going to guess and you'll tell me if I'm wrong maybe even against Authority oh yes oh yes I never choose which is the larger our key because of my father but ah so you already had reason I read challenge authority I already had good reason it's


gonna show up later in your life that's the only reason i underscore that how are we gonna get this oppressed young man occasionally oppressed by this school and a status at the end of his time there where he is actually going to be admitted to Cambridge well how am I going to get him either intellectually or morally beyond this this stage of life well before we get there all right please I have one more story I think you would like from that school which is we had to have religious education of course it was kind of Sunday School kind


of religious education about how Jesus was good and yes when God would look after us and I remember several things about it because of course the other kids all believed in God some more than others I had a very intelligent friend there who who believed in God and I remember during religious class the teacher saying all good things come from God and I would have been about eight or nine then and I remember thinking that there was a problem I don't remember how coherent me I thought it but there's a problem that she was only saying they


came from God because they were good and she couldn't use the fact that God you couldn't sort of use this as evidence that God was good because you just assumed they came from God because they were good so it was that's what you would call a circular argument now I didn't know about circular arguments but I knew there was something deeply suspicious about what she just said and I guess I was in a mood to argue I argued a lot I was very small by the way I was a smallest boy in the school ah the smallest and lightest not an


advantage but I made up for it with arguing um so I put up my hand and ice I tried to explain to her what was wrong with what she just said yes and I think she got frustrated and she said ok Hinton where where do where do you think all good things come from a classic kind of parent move yes um build up build an assumption into the question I leave the kid with the problem of dealing with it um because of my father I um I thought for a moment and said Russia no this wasn't what was expected in an English private school in


the 1950s this would have been kind of the mid-1950s yes and when the Cold War was at its height and that kind of gave you an indication of how much I fitted in with the culture of the school I've already had the indication though can I get you to Cambridge only become at a time you can try I'll try my my parents tried haha so let's see um when I used to question my parents later on about why on earth that insisted on my staying there even though I was I'm happy yes um they said well the science teaching was good


dear and the sons teaching was good was good later on I I enjoyed the science lessons and I didn't enjoy the math much I went in when I was before I got to the senior school I enjoyed math but then there came a point when I just got confused about math when they introduced functions I didn't understand what functions were I would have been good at arithmetic and good at algebra yeah so neither were these things I didn't understand things like sine X I didn't know what sine was and I was always a very concrete thinker I think in terms of


mechanical analogies yes and I was actually I felt unhappy with functions and not at home with them until I started programming when I was a graduate student yeah as soon as I started programming a function was just a box she gave it one thing and it gave you back something else but until then I had I didn't really I wasn't friends with functions and that made math I got worse and worse at math um can I generalize and say that if you don't understand its purpose you challenge it or dismiss it maybe I'm reaching to to


deepen your childhood for the later right I certainly want to i i'm i i'm not happy with things until i understand them yes and what I mean by understanding them I think it's the same as what Fineman meant you could build one you know you understand it well enough that you could build one that's that's what I'm I'm wondering it at the X ounds like that was it as later on that's exactly what I felt about psychology in general that you know I want to know what feelings were what sensation was and yes and I felt you


never really understand that until you could build one and you as many people say you couldn't build one but if we make it to your current approach you will not be sentimental about such terms as feelings and emotions but we'll get we'll get there I'm not sure what you mean by not being sentimental I I mean the the hope that they are undefinable not reproducible human as opposed to something that you can analyze you you really I wouldn't call that sentimental okay I collect just dumb haha fair enough I still haven't gotten you into


Cambridge No so you've got a good science together several several good science teacher yes and so he's good at science I was okay at math and because my father had been to a particular College in Cambridge yes he wanted me to go there yes Ferrand ice at the entrance exams and I did quite well in the exam so I got in did you surprise your school were they assuming you were clever at this point they're not particularly no no they knew I was fairly clever by and then you're as clever but um unruly yes um also might be truth today so going on going


on oh you you sit the exams you get in uh-huh I go to Cambridge yes um it's a big shock because at school there were a few kids who were clever and interesting ideas I had a friend called Inman Harvey who was very clever he got the top math scholarship at Trinity which is the best place in math and Britain yes um but mostly the kids they weren't their primary interest was not ideas weren't they mostly influenced by the kinds of school you went to in fact I mean this is sorry Edie at my at my high school what you call a high


school a North America most of the kids were not any I went to Cambridge everybody was classy I say okay um actually not everybody there was some medics venue but but apart from that yes everybody was clever and that was a bit of a shock it was very nice yes but also quite threatening yes of course so after I've been there a month I left I found it too stressful yes and I went to London and did various odd jobs [Music] I read a lot of depressing literature like crime and punishment from The Brothers Karamazov remember sitting on


the London tube which is a fairly depressing place anyway reading these depressing Russian novels I did various jobs I ended up getting very interested in architecture and I reapplied to Cambridge to do architecture and they let me back into the same College but before I got there I'd done I've worked in an architect's office over the summer and I discovered what the practice of architecture was like yes I sort of imagined you'd sit there sketching out what wonderful buildings going to be we're going to be like or maybe new ways


of constructing buildings actually life in electrics office consists of you know are we gonna have cheap flooring or cheap door handles because one of them's got to be cheap otherwise we won't meet the budget yes and so after I've done architecture for a day I went to see my tutor and said I want to switch back to doing natural sciences and my teacher said ok which is a bit if I'm right about this one of the advantages of the Oxbridge approach I mean they they can be flexible they can be very flexible um so my first year I


did Natural Sciences and I think I was the only student in that year doing both physics and physiology Oh so I've always been interested in biology uh and I hadn't been allowed to do biology at school because my father wouldn't allow it he said they would teach me genetics and genetics was nonsense huh what he really meant by that was he was convinced that all sort of macroscopic observable traits would be caused by complex interactions of many genes yes and the sort of standard theory was you know blue eyes is caused by this gene


and intelligence is caused by that gene yes and of course that was very much against communist teaching yes and so for ideological reasons I wasn't allowed to borrow your school even though my father was a biologist he was very good biologist in his way um he got to be a member of the Royal Society yes I actually got to be a member of the world sight at age 49 and I got to be a member of the Royal sight at age 50 which was always irritating Tonica um so I was very competitive with my father anyway well the Cambridge I did physics and


physiology chemistry and physiology was really interesting I'd never done it was it was stuff I didn't know anything about yes and I remember the highlight of the course was going to be the final term where they were going to teach us about the central nervous system yes and I was I was a very very interested in had the brain work because my friend Inman Harvey at school hood was interested in how the brain worked and we had discussions about it yes and I remember how disappointed I was when they taught us about the central nervous


system because the way the central nervous system works is this those neurons and they have axons yes and electrical impulses travel down the axons and cause some chemicals to be released that get other neurons excited and they taught us a lot about how the electrical impulses travel down the neurons because that was the classic work of hodgkin-huxley and that was how the central nervous system worked and we're being immensely frustrated that they hadn't actually said anything about how it worked I mean they said how the


impulses were on propagated but how did it actually work it was just descriptive basically I mean it was interesting that that's neurons communicate yeah or another but that wasn't what I meant by how it works I wanted to know how how the brain works and how that gives rise to emotions and sensations the song oh why was that if it was such a radical thing to want to know I mean I don't think it was a radical thing hard to know okay I think lots of you want to know that it's just that in Physiology yes they didn't know


okay so after my first year I switched to philosophy because I thought philosophy would teach me more about those the things I was really interested in and that was a big mistake um I remember learning about thick and shiny and getting very depressed because I couldn't understand it later on I think he was very useful to have learned that stuff but really I think my main the main thing I got from a year of doing philosophy Cambridge was I developed antibodies against philosophy although there was one philosopher there who I


got along with extremely well who actually they're two of them there's my tutor someone called Jim Hopkins who was a very nice guy I'm also interested in how the mind actually worked although he didn't know either and then there was a philosopher called Burnham Williams yes who was a very good philosopher later went to Berkeley I never met him again which was always a regret he used to hold Monday evening a sort of open house where you would go to his room if you're interesting and just discuss philosophy for an hour - and a number of people


would turn up and I got along with him very well he's a great thing yes I know about it he was very eclectic very very fluid in his thinking yes he he wasn't he wasn't dogmatic at all and he was just very interesting to is very interesting to learn from him and he would always take the students ideas seriously and always have something interesting to say about them he was never dismissive so that was the one for philosopher who I really appreciated then I got fed up with philosophy because it wasn't telling me what I


wanted to know and so I switched again to psychology is it fair dude just to finish the philosophy apart is it that they're not asking interesting questions or that they are giving an overly to us as you're centering questions another philosopher I really like is Daniel Dennett who definitely asked interesting questions okay they just don't have the apparatus to answer them okay my sort of methodological conclusion relates to why it is that philosophers sometimes at a conference will read their paper whereas


a scientist very few scientists would do that a scientist will get up and give a talk yes and make some claims and tell you the evidence maybe philosophers will read their papers and they'll sort of the way they read them is significant so with philosophy it's it's how you say something that's important and the words you say it was that's the content and it's because there is no other content so in in philosophy there's no difference between something sounding really good and something being good yes there's no empirical test yeah in


science something can sign completely loopy like acceleration is gravity yes and yet it can turn out to be true yes and the things that science is established to be true a far crazier than anything any religious connect traumatic could dream up that outside the realm of the crazy things religious fanatics treemap huh they're always rather boring things like you know there's another world a bit like this one but above the clouds where all the good people go then they're nothing like you know there might be black holes right right so science


establishes things that are far crazier than normal people could have a thing called yes yes so this idea that scientists a kind of narrow is just crazy you know the way you can do it is because it's got something out outside of theorizing that can tell you um where these theories are right or not right and I've heard people I don't understand string theory but I've heard people claiming that um that borderline is being pushed by string theory where it's not quite clear whether it's all just mathematics or whether it really is


being verified by data the whole science has this independent testing philosophy doesn't it's in there sometimes the impossible I may be thinking only at mathematics to speak and search for the elegant solution which always surprises me is terminology because elegance is not necessarily truthful but maybe that is the the core of mathematical elegance it's a very interesting argument about why why should elegant things be true yes suddenly if you look at physics if you look at particles and things you find kind of fifteen particles which all


fit into a nice pattern except there's a missing one yes and the methodology of saying well we should search for that missing one because of all these symmetries it must be there that works and the question is why does and why does it work so it's not clear that works in biology so when I was a postdoc in San Diego I got to know Francis Crick who who was just become very interested in the brain he was a very impressive thinker but he was of the opinion that the idea that the elegant thing is going to be true did not necessarily apply to


biology if philosophy is unsatisfying to the young undergraduate I would then assume given his future career he would leap to the sciences but in fact he lept at the Social Sciences I don't know allow help me help Meg gain which psychology was a science not a session science and yeah I mean psychology has got these sort of two aspects to it but it was definitely the scientific aspect of Cambridge so we learned about rats and we learned about section theory so that's you look to the rats well I didn't want to leap to the


rats that's what psychology was there right right in fact I actually was very annoyed that it wasn't teaching us anything about people and so I went to see my shooter not my psychology tutor but my sort of general children king's college who was in charge of my welfare yes and I explained that the psychology course was not telling me anything about real psychology for example there was nothing about psychoanalysis in the psychology course yes and so what I would like to do is I'd like to go to London once a week and get a tutorial


from an existential psychoanalyst so I could learn about that because I was really really mad and I would like the college to pay for it and so my teachers my teacher said well that sounds reasonable so yes the college would pay for that ah that's a real liberal education is oh yeah King's College Cambridge which was loaded not only loaded but I had the capacity to imagine doing that yeah so the audacious undergraduate asked for it the wealthy college agreed yeah and how important was that actually that so the


only thing I really remember about those tutorials and existential psychoanalysis was psychoanalysts had a really beautiful Japanese girlfriend which makes our analyst psychoanalysis seem very good idea but he taught me about his Earl and Heidegger and I don't never really understood any of that didn't understand it couldn't be bothered no I tried I just didn't didn't really understand yes um I make the assumption that there was something there to be understood but I which now you don't make anymore which I'm not sure I make


anymore yeah I'm because we don't have very much time I'm gonna race you through your psychology course but I'll stop at any point that seems critical in your in your intellectual development so I just felt psychology was totally lacking in any idea of what a proper theory would look like they didn't they didn't sort of have the physics view that our theory should really explain something so he did have the experimental method so it's better than philosophy in that sense but what they were using experiments for was to try


and decide between theories that were sort of hopelessly inadequate and you could just dismiss out of hand without doing experiments because it was just not up to the job yes and so I had to do an experiment and I remember the experiment where you would take you take children between 3 & 5 yes and you try and decide whether they developed during that period in the sense that during that period they started paying more attention to shape and less attention to things like color and texture so is this model of little kids which is the


stimuli like shape and color and texture and they respond to stimuli so this was behaviorist kind of psychology yes I'm just beginning to get a little bit cantar and the experiment is going to be decided to decide if the strength is the response to shape increase in the strength of a response to color decreased and so you presenting with three objects during training you give them say two triangles and a square all of which are yellow and they learn to pick out the odd one Act which is the square yes and then you give them I


better get this right you give them three triangles one of which is red from the other to a yellow and they learnt it has the red triangle hands down then once they've been trained like that with various different stimulus dimensions you then give them a yellow triangle a red triangle and a yellow square so now they've got a conflict are they going to pick out the old one based on color or are they going to pick out the old one based on shape right and you look to see what they do and the hope is that as as they get old after the older kids will


use shape to pick the old one out and the younger kids will use color I'm not picking up a sense that you were impressed by this experiment well here's what actually happened the experiment was going along and then I got a bright five-year-old yes and the bright five five year old the first time I showed him one of the conflicted ones yes where there wasn't a clear up one out yes he pointed so will be a yellow triangle the yellow square in a red circle he pointed at the red circle and said you painted that one the wrong


color he thought I'd made a mistake right ah because it was the old one out game and clearly I painted that one the wrong color and I thought you know this this this organism that's meant to be modelled as responding to color or responding to shape yes this organism has just done a piece of reasoning and said you painted that one the wrong color that's just way beyond the scope of any of these theories it's just hugely complicated behavior compared with these seriously or the organism sort of figured out I had


intentions and that I made a mistake here it's just utterly out of the realm I mean if you're trying to sort of model going to the moon by climbing out of a stepladder you know yes that that the the theories they were dealing with stood no more chance of dealing with this kind of behavior than a stepladder would get you to the moon and so that had a big effect on me so you're you're disappointed I'm totally totally disenchanted with psychology because although it's got an experimental method it's using it in an incredibly naive way


to test really dumb theories yeah one man said I have to get you I'm sorry I have to get you out of Cambridge right and into your next step and just the preface for that is of course because of those who might be listening not really quite understanding the state of inquiry in artificial intelligence computers and so forth how do you then map your next step so then I became a carpenter for a year I quit I could he Mia yes and then I got back into academia by working on a project studying child language development influenced a lot by Chomsky


yes um who claimed that claimed on spurious mathematical grounds that almost all language was innate yes in tacky aspects of language were innate which is complete rubbish and the project was looking at a large section of young children in Bristol to look at their language development empirically by just measuring what happened it happened at the same time with Watergate this project uh-huh and we had little jackets with radio microphones so we put this little jacket on a kid he wear it all day and would be broadcasting everything that the kid


said and inside the house and everything that was said to the kid and inside the house it wasn't regarded as a bad thing back then inside the house we had a recorder that every 20 minutes would take a one-minute sample it would do that by having a cardboard discs that rotated very slowly and a notch in it and a little lever would fall and connect the tape recorder that's what technology was like then and we would get these little samples of child of children's language and then we would try and analyze them that was the


problem but we got very interesting the most interesting utterance we got was when we were looking at tag so a tag is like isn't here aren't they or won't we write and in english tags have a lot of syntax in them you don't just seen this part or something like that right right you have and so we were looking at tags to see whether children don't express complicated grammatical structures very early on because it's too many phonemes it was because they just don't know these structures and it tanks very good cases very few phonemes but has a lot of


grammar in it yes and we got one kid one parent who said to her child Santa don't give you no toys if you don't talk proper isn't he Wow and I just thought that was a nice example of the kind of data that children got from which they learn to speak good English yes yes but you're not gonna linger you're going to move on so then I started PhD in artificial intelligence because I thought that was such a program was possible this was 1972 yes and they just set up a couple years earlier a big center of art of intelligence in Edinburgh Edinburgh so


the science Research Council had decided to fund one big Centre in artificial intelligence ah and looking back how was it structured this doesn't impress you now in retrospect as to what they they thought they were doing I think it was a sensible thing to do okay I think was good policy to make one really good Center mmm I think nearly all the people there believed in symbolic err good ol fact what is now called good old-fashioned symbolic AI right I think they were making a huge mistake and the government um a few years later I can't remember


the exact time probably in 1974 or something like that got a very eminent mathematician called Sir James light Hill to do a report on the AI Center yes yes and he produced a very damning report saying that basically saying these guys didn't know what they were talking about there was an interchange with McCarthy who was one of the fathers of AI yes I remember the seeing the interchange years later where McCarthy was saying look anything you can compute you can compute with symbolic operations so what we doing must be right hon light Hill


was saying basically yes but you've no idea how the brain does that which is the only advice we know that can think and you've no idea whether this way of doing it is efficient now retrospect at the time everybody in Britain was outraged that AI was now going to go through hard times because Sir James like chill didn't believe in it and I think blight chill was entirely correct because both sides were assuming something and both sides were assuming that computation wouldn't get millions of times faster than it was now yes might


get thousand time plus we're not millions of times faster yes and under that assumption lecture was completely correct that this symbolic way of doing things although it theoretically you could do anything that way with the speed of computation we had there was no hope of doing things like perceptional much when I was the report and where are you at the point with undergraduate Street you are a graduate student at the time this pronouncement is made yes and as a result of the pronouncement so it must mean in 74 my advisor who's one of


the three professors who set up the AI yes the school day I remember he leaves head Umbra well he leaves for several reasons personal conflict with one of the other main protagonists was perhaps one of the reasons he left and went to Sussex and I went with him you went with him did you find your fellow graduate students bewildered by this analysis and you you know I didn't or I don't remember at the time I think I thought light Hill was too severe on a I to ah there have been a lot of paper pressure to believe that yes of course um yeah I


don't remember too much with philosophy course but you go you go to Sussex then yes this is before you've completed your degree yes oh yes so I used to have to go back to Edinburgh for one day I've returned to sign a register to say that I lived in Edinburgh I would get a return ticket from a day return from Brighton to Edinburgh um but some of the in n borough is going to have to approve a dissertation topic course yes who is that who's who's lingering there that there was a more junior guy in a I called Jim Howe who was my official


advisor I see but everybody knew it was just an arrangement how did you work out the topic of the dissertation if that was but so I wanted to work on ural networks and how they learned but I couldn't figure out how they learned from I couldn't buy this I couldn't figure out anything significantly improved over what was called the perceptron convergence theorem which was already known in the early sixties late twenties early 60s neural networks are in the air I mean ever since touring in a way no not so much when I my advisor had done neural


networks until I arrived just before I arrived in Edinburgh he switched his views so he had worked on holographic memories which I was very interested in and about when I arrived as a graduate student he was very impressed by a thesis by Terry Winograd that was using symbolic arrow methods to try and understand natural language commands like yes put the red block on the green block in the blue box and he was very impressed by that he basically switched his interest from neural Nets just symbolic AI and he'd taken on this great


student who was killing it in neural nets and he tried to convince me to switch my interest but I wasn't having any no no you're stubborn I mean I was very stunned I remembering when I look back on it and having been an adviser of graduates yes yes and having seen the various types of graduate students there are including the extremely stubborn ones I remember him coming into my office when I was a grad student and saying Geoffrey I've had this idea that you might be interested in let me explain it to you so he explained this


idea to me that seemed moderately interesting and at the end of the idea he said so Geoffrey do you think you'd like to work on that and I looked at him in an amazement and said no no I God I didn't my only need to work up um yes so he was very tolerant he was tolerant of having me as a graduate student even though he thought I was doing crazy stuff he's available to say he was charmed by your audacity or is that No we'd have arguments in the time and I kept agreeing that okay if I hadn't made it work in six months I would switch to


doing symbolic AI and then I would keep running on those arguments that's what he's tolerant of you were also unwilling as part of your temperament as we begin to understand it to just go ahead and do the union card thing which is right whatever it is that the professor says do do it and then get on with your real life you weren't prepared to do that no no I I mean I wasn't gonna work on ideas I didn't believe it yeah you're not cynical yeah I I am quite cynical about a lot of things but like that that's I still want you to to get your degree how


do you how does that manage to happen so in the end I managed to do something that wasn't learning in your all Nets but was inference in your own ads and I made it work and it had a little bit of math that justified it mmm and he was happy with that and so I got a PhD and then I got out of there and I was very disillusioned with it all by then and I took another field oh by the university or the feel academia by academia itself and so again and you've done that before yes I dropped by frequently yeah and so I took a year off I went to London and


taught in a free school that was a very different environment yes free school with a lot of disturbed children yes and then I went back I probably Trebek signal itself then I went back and got a job as a postdoc we're in Sussex in Sussex for a while and finally I saw an investment for a job in San Diego that seemed like a really nice job in cognitive science where they were gonna recruit six postdocs in various areas who were going to interact with one another and try and understand the mind one of the articles I've read


about you not necessarily was accurate but said that the question of monetary support for work was also a factor in your going to america is that not true well because of the like hill report the were no jobs in AI in vogue were there he killed it basically they killed that there was one job in AI at edinburgh and lots of very good people competing for it so you almost didn't have a choice no I couldn't get it I couldn't get an academic academic job in Britain at all those nations zero shuns I mean I couldn't even get an interview for an


academic job now we're almost at the end and the whole point of this interview is really the origins of your of your thinking so that's not a problem but I I wonder if we can get toward the end a kind of description of the academic environment in the American University and the the sort of strategies that you encountered at that point when you went to San Diego so it's a big contrast that in Britain the academic establishment in a I was monolithic there was sort of the correct view yes and there wasn't room for multiple camps ah and in America at


least you've got two coasts so in linguistics you had a salary of East Coast Chomsky camp and a West Coast film or on lake off camp and it was the same with AI that in San Diego though a group of people particularly David Rama Hart who came from psychology but was strong mathematically who basically had a completely different view of AI he was interested in how it actually happened in the mind and he thought understanding had happened in the brain would be useful for understanding that Ruth unlike many psychologists and he just


had a view that was extremely compatible with what I've been thinking so it's the first time I've been somewhere because longings had changed his mind just before I arrived yes it's first time I've been somewhere where I was working with someone who really had the same general beliefs about how to go about understanding the mind and what it was like as I did and that was wonderful working life as we're toward the end of this I may I'm gonna ask something that may be ridiculous generalization and I'm perfectly


prepared to hear you say say that but as people look at your career there are some lessons some draw about the stubborn persistence in an idea in the face of most people saying it's nonsense in America as well and then maybe even into Canada or where you later went there was still in it an academy saying what you were interested in and what you were pursuing was wrong yes but I sent him over bad things about psychology but what happened was after backpropagation you have been rediscovered by dave rama ha yes and he and i and various other


people had shown he could do interesting things in terms of learning representations right there was a surge of interest which then died out with in computer science because it didn't work as well as we'd hoped but in psychology people stayed interested in ah so it had a home in psychology and there was always support for these ideas in psychology so you had people to talk yes to anyway yeah um although on the whole i was much more interested in making it solve problems like speech recognition and object recognition which


the psychologists weren't really doing very effectively so i was interested in doing machine learning with it and the psychologists weren't really pushing that agenda right so in that sense you were relatively lonely I was relatively lonely but there was definitely it would be completely incorrect to say I was the kind of lone voice in the wilderness right there were a few other lone voices too but the wilderness with a will learning not the whole whole the whole academic scene that's going to be the last word


the following is a conversation with Ian good fellow he's the author of the popular textbook on deep learning simply titled deep learning he coined the term of generative adversarial networks otherwise known as Ganz and with his 2014 paper is responsible for launching the incredible growth of research and innovation in this subfield of deep learning he got his BS and MS at Stanford his PhD at University of Montreal with yoshua bengio and Erin Kerrville he held several research positions including an open AI Google

Ian Goodfellow: Generative Adversarial Networks (GANs) | Lex Fridman Podcast #19


brain and now at Apple as the director of machine learning this recording happened while Ian was still a Google brain but we don't talk about anything specific to Google or any other organization this conversation is part of the artificial intelligence podcast if you enjoy it subscribe on YouTube iTunes or simply connect with me on Twitter at lex friedman spelled fri d and now here's my conversation with Ian good fellow you open your popular deep learning book with a Russian doll type diagram that shows deep learning is a


subset of representation learning which in turn is a subset of machine learning and finally a subset of AI so this kind of implies that there may be limits to deep learning in the context of AI so what do you think is the current limits of deep learning and are those limits something that we can overcome with time yeah I think one of the biggest limitations of deep learning is that right now it requires really a lot of data especially labeled data there's some unsupervised and semi-supervised learning algorithms that can reduce the


amount of labeled data you need but they still require a lot of unlabeled data reinforcement learning algorithms they don't need labels but they need really a lot of experiences as human beings we don't learn to play pong by failing at pong two million times so just getting the generalization ability better is one of the most important bottlenecks and the capability of the technology today and then I guess I'd also say deep learning is like a of a bigger system so far nobody is really proposing to have only what you'd


call deep learning as the entire ingredient of intelligence you use deep learning as sub modules of other systems like alphago has a deep learning model that estimates the value function most reinforcement learning algorithms have a deep learning module that estimates which action to take next but you might have other components here basically as building a function estimator do you think it's possible you said nobody is kind of in thinking about this so far but do you think neural networks could be made to reason in the way symbolic


systems did in the 80s and 90s to do more create more like programs as opposed to functions yeah I think we already see that a little bit I already kind of think of neural nets as a kind of program I think of deep learning as basically learning programs that have more than one step so if you draw a flowchart or or if you draw a tensor flow graph describing your machine learning model I think of the depth of that graph is describing the number of steps that run in sequence and then the width of that graph is the number of


steps that run in parallel now it's been long enough that we've had deep learning working that it's a little bit silly to even discuss shallow learning anymore but back when I first got involved in AI when we used machine learning we were usually learning things like support vector machines you could have a lot of input features to the model and you could multiply each feature by a different weight but all those multiplications were done in parallel to each other there wasn't a lot done in series I think what we got with deep


learning was really the ability to have steps of a program that run in sequence and I think that we've actually started to see that what's important with deep learning is more the fact that we have a multi-step program rather than the fact that we've learned a representation if you look at things like res nuts for example they take one particular kind of representation and they update it several times back when deep learning first really took off in the academic world in 2006 when Geoff Hinton showed that you could train deep belief


networks everybody who was under ested in the idea thought of it as each layer learns a different level of abstraction but the first layer trained on images learn something like edges and the second layer learns corners and eventually you get these kind of grandmother's cell units that recognize specific objects today I think most people think of it more as a computer program where as you add more layers you can do more updates before you output your final number but I don't think anybody believes the layer 150 of the


resin it is a grand grandmother cell and you know layer 100 is contours or something like that okay so you think you're not thinking of it as a singular representation that keeps building you think of it as a program sort of almost like a state the representation is a state of understanding and yeah I think of it as a program that makes several updates and arrives it better and better understandings but it's not replacing the representation at each step its refining it and in some sense that's a little bit like reasoning it's not


reasoning in the form of deduction but it's reasoning in the form of taking a thought and refining it and refining it carefully until it's good enough to use do you think and I hope you don't mind we'll jump philosophical every once in a while do you think of you know a cognition human cognition or even consciousness as simply a result of this kind of cincuenta sequential representation learning do you think that can emerge cognition yes I think so consciousness it's really hard to even define what we mean by that I guess


there's consciousness is often defined as things like having self-awareness and that's relatively easy to turn into something actionable for a computer scientists the reason about people also defined consciousness in terms of having qualitative states of experience like qualia and there's all these philosophical problems like could you imagine jambe who does all the same information processing as a human but doesn't really have the qualitative experiences that we have that sort of thing I have no idea how to formalize or


turn it into a scientific question I don't know how you could run in experiment to tell whether a person is a zombie or not and similarly I don't know how you could run an experiment to tell whether an advanced AI system had become conscious in the sense of qualia or not but in the more practical sense like almost like self attention you think consciousness and cognition can in an impressive way emerge from current types of architectures though yes yeah or or if if you think of consciousness in terms of self-awareness


and just making plans based on the fact that the agent itself exists in the world reinforcement learning algorithms are already more or less forced to model the agents effect on the environment so that that more limited version of consciousness is already something that we get limited versions of with reinforcement learning algorithms if they're trained well but you say limited so the the big question really is how you jump from limited to human level yeah right and whether it's possible you know the even just building common-sense


reasoning seems to be exceptionally difficult so K if we scale things up forget much better on supervised learning if we get better at labeling forget bigger datasets and the more compute do you think we'll start to see really impressive things that go from limited to you know something echoes of human level cognition I think so yeah I'm optimistic about what can happen just with more computation and more data I do think it'll be important to get the right kind of data today most of the machine learning systems we train our


mostly trained on one type of data for each model but the human brain we get all of our different senses and we have many different experiences like you know riding a bike driving a car talking to people reading I think when you get that kind of integrated data set working with a machine learning model that can actually close the loop and interact we may find that algorithms not so different from what we have today learn really interesting things when you scale them up a lot and a large amount of multimodal data so


multimodal is really interesting but within like you're working adversarial examples so selecting within modal within up one mode of data selecting better at what are the difficult cases from which are most useful to learn from oh yeah like could we could you get a whole lot of mileage out of designing a model that's resistant to adverse fare examples or something like that right yeah question but my thinking on that has evolved a lot over the last few years one nice thing when I first started to really invest in studying


adversarial examples I was thinking of it mostly as that versus aryl examples reveal a big problem with machine learning and we would like to close the gap between how machine learning models respond to adversarial examples and how humans respond after studying the problem more I still think that adversarial examples are important I think of them now more of as a security liability then as an issue that necessarily shows there something uniquely wrong with machine learning as opposed to humans also do you see them


as a tool to improve the performance of the system not not on the security side but literally just accuracy I do see them as a kind of tool on that side but maybe not quite as much as I used to think we've started to find that there's a trade-off between accuracy on adversarial examples and accuracy on clean examples back in 2014 when I did the first adversary trained classifier that showed resistance to some kinds of adversarial examples it also got better at the clean data on M NIST and that's something we've replicated several times


an M NIST that when we train against weak adversarial examples Emnes classifiers get more accurate so far that hasn't really held up on other data sets and hasn't held up when we train against stronger adversaries it seems like when you confront a really strong adversary you tend to have to give something up interesting this is such a compelling idea because it feels it feels like that's how us humans learn yeah the difficult cases we we try to think of what would we screw up and then we make sure we fix that yeah


it's also in a lot of branches of engineering you do a worst case analysis and make sure that your system will work in the worst case and then that guarantees that it'll work in all of the messy average cases that happen when you go out into a really randomized world you know with driving with autonomous vehicles there seems to be a desire to just look for think I'd viscerally tried to figure out how to mess up the system and if you can be robust to all those difficult cases then you can it's a hand waving empirical way to show that your


system is yeah yes today most adverse early example research isn't really focused on a particular use case but there are a lot of different use cases where you'd like to make sure that the adversary can't interfere with the operation of your system like in finance if you have an algorithm making trades for you people go to a lot of an effort to obfuscate their algorithm that's both to protect their IP because you don't want to research and develop a profitable trading algorithm then have somebody else capture the gains but it's at least


partly because you don't want people to make adversarial examples that fool you our algorithm into making bad trades or I guess one area that's been popular in the academic literature is speech recognition if you use speech recognition to hear an audio waveform and then in turn that into a command that a phone executes for you you don't want and a malicious adversary to be able to produce audio that gets interpreted as malicious commands especially if a human in the room doesn't realize that something like that


is happening in speech recognition has there been much success in in being able to create adversarial examples that fool the system yeah actually I guess the first work that I'm aware of is a paper called hidden voice commands that came out in 2016 I believe and they were able to show that they could make sounds that are not understandable by a human but are recognized as the target phrase that the attacker wants the phone to recognize it as since then things have gotten a little bit better on the attacker side when worse on the defender