you interpret a particular token you can your context is only to the left of that token you don't know what's coming in it's like what do you when you read a sentence from left to right but then as humans unconsciously we probably go and at the end of the sentence kind of make sense of the thing as a whole but here the model is forced to make sense of what the thing only from whatever is to the left of it so that's a basic limitation of these left to right models then there is another approach which is called Elmo


which has been popular recently as a substitute for word vectors so if you know word vectors were vectors are basically the kind of first stage in most language processing tasks where for each word say the cat sat something for each word you have you had a big giant table and for each word you associate a vector of fixed size dimension right so you place every word in a vector space and these vectors you pre compute for something like word Tyvek or glove and that gives you a nice way to basically deal with these words in a canonical way

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding


and you can pre train the word vectors that's all really nice but people have realized ok words kind of multiple meanings and words can kind of slightly change meaning depending on words around them and so on so what Elmo does is Elmo uses to Alice Tian's one Elliston goes into this direction when I Austin goes into this direction and basically a single lsdm as we saw the four it takes in the input sequence one by one so here e1 and e2 if the en it produces hidden states at each step produces a hidden state that


is a result of previous hidden state and the current token and and then what it says is ok now these hidden States here basically these are now the embeddings of the token here on e and so on right these are the embeddings so the world vectors as to say Oran are no longer just one vector per word so they're not in isolation anymore but basically you need the entire sequence to compute the word vectors as a result of this of this Alice T and this is more powerful because it can give individual words multiple or each it basically each word


has kind of a unique and then depending on the surrounding words you would still hope that a given word would have similar similar embedding or similar word vector all across the language but you can kind of fine tune it to the particular sentence it is in and also you can completely change its meaning if it's if it's kind of a word that has a completely new meaning in that sentence so basically uses two LST M's one as as I said here forward one backward these also have multiple layers and so and each of these produce one such hidden


vector per token and he would simply concatenate the two from the from so from here is this leather stain on the left produces one disaster the right produces maybe here another one and you simply concatenate the two to get the the final embedding the final word vector for each token so the fundamental limitation here is that this is kind of you have information from the left end you have entered information from the right so other than here the original transformer you actually have you actually can condition on the left context and the


right context but it's very it's it's very shallow because it's simply a concatenation of the left facing Alice TM and the concatenation of the right facing Alice TM and and these ultimately intrinsically they have nothing to do with each other they so you simply concatenate the two things that the left facing Ostiense still can only see to the left and the right facing Alice team still can only see to the right so you basically have to you have blind models and then you can have concatenate so the it's still


suboptimal because what you want is you want a single model to output your word vectors or to interpret the language that can look at both the left and the right at the same time and then incorporate information from both of them simultaneously and not just at the end by concatenation this is what Birth does it's a bird here and this is kind of what they claim is the new contribution bird at each in each layer here of the model D D let's look at this and for a particular token they look at all of the context so every every other


token in the in the input they look at that and so the the basically it seems kind of it seems kind of obvious but it's it's actually there's there's reasons why these other models don't do this but so this is the entire point of Berk is at each layer in this in this transformer architecture is still an attention that can ISM by the way so that there's there's the mechanism of attention here and and here is exactly the same or almost the same they actually keep it close on purpose in order to compare but now we have


attention not only to the left but also to the right to everything right so why do these other model whether for example the opening I transformer only look to the left that's because somehow you need a task to train on right and most of the time if you especially if you want unsupervised training you going to do something like language modeling and language modeling what you have is a sentence ABCD and you're asking what comes next here alright so by by the definition of the task you can only look to the left


that's that's just how how these like how the task works so it makes sense that that these other models kind of do this because they preach right and this not Burt has a different free training because they can they can only they have to look to the left and the right and the other thing is what you want to use the model for so the good thing if you own if you go left to right is you can use the model now for generating language in the same thing if if you have a B C D and you ask and the model is trying to produce the next character


only looking to the left right then you can you can say what's the next character the model says B and then we can feed the same thing into the model and say okay what's now the next character what's now the next character G so there's pretty useful if you only look to the left you can actually use the model then for generating a language which is something you can't do with Burt or it's not it's not really obvious now how to do it with where people are I know people are investigating into language producing producing entire


sequences with Burt but as yet it's not super clear how to do this with this model that being said them all is pretty good at pretty much everything else so let's jump in to how they train they train let's see here they trained using masked basically masked language modeling so when I actually go into that first masked language modeling what they do is they basically replace some words by the mask token and they don't have a good no nice alright today I'm everyone here all right here if you just look at kind


of the top sentence here the man went to mask store right don't don't don't worry about the set and so on just this a man went to a mask store and the models simply asked to predict what's here which word is there so it needs to incorporate information from the right and from the left to do this so that's basically how you train it they simply drop out some of the words some of the time and they they have different techniques so you can clearly tell a lot of work has gone into kind of fine-tuning everything in this model


like how to Train it and so on so let's say we don't always do this sometimes we do this other thing and sometimes we do that and there's several ways of biasing this model but basically you do this mask which modeling and then because they also want to evaluate on let's say entire sequence tasks or tasks that span multiple sentences what they do is the second free training task at the same time as you can see here where they feed two sentences so that's the first sentence that's the second sentence they feed these two sentences as an input so


at first they have this token and these separate the sentences and then they ask the model to predict a label is next and is next is neck is true if the second sentence follows the first and so it's if it's like a logical continuation and the way you do this unsupervised is really easy you take a big giant corpus and you take a sentence for the first sentence and then 50% of the time you take the next sentence in the corpus and the label is true and 50% of the time you take some random sentence here you say for example the man the man mask to


the store and the next sentence is penguin mask or flightless birds and that's kind of a random sentence so the model is asked to predict well that's probably not the next and following this first sentence so you do these two tasks a pre-trained and D can do this unsupervised you don't need supervised data for that you just need a corpus and they do this for a long time with a lot of data and the model itself is giant has 24 I think of these transformer layers so it's giant and then you kind of pre-trained in this


model here is a here's an illustration of some extra things so what they do is they they first this is the input up here so the first token is this CLS token which is kind of the star token and then this is the first sentence then the step is the separator of two sentences then this is the second sentence and then again I said look up to these hashtags in a second but first they say okay first we have the token embeddings so they kind of start with with the original concept of word vectors at the very basis because you


need to start with actually going into a vector space to use these models but they don't they they then then in kind of transform these through the transform layers they also use segments embeddings and segments embeddings as you can see here is simply a kind of a binary label EI being the label for the first sentence and E being the label for today the second sentence so just the model can differentiate which one's the first which one's the second because it's kind of hard to learn for a transformer architecture that the tokens kind of


separate the sentences so you can want to help it and the last thing is positional embeddings and we've also already talked about these in attention is all you need this is where you can kind of the model since it's a transformer it doesn't go step by step it doesn't go one of them so it's kind of hard for the model to make that how far things are apart from each other how far to tokens if they're neighbors or if they're really far apart and these positional embeddings kind of help the model I decide if two tokens are close


to each other in input infer if they're right they're just neighbors or if they are actually really far apart all right so this is this is how the kind of first input is constructed out of these embeddings and then it's fed through these transformer layers as we saw with the mask that lime task and his next task I want to quickly get to these hashtags what what they mean so the input here is separated into word pieces so called work pieces and what that is is so in language processing tasks you have kind of a choice you have


you have a choice of how to tokenize your input so what let's look at a sentence here subscribe to PewDiePie so this is a sentence and the sentence is rather let's say world wise complicated so why matter language one will have problem with this so first you need to tokenize this sentence alright so what most people do is they say okay here are the word boundaries we're not tokenize this into three segments first is subscribe to piggyback okay so three things and each of these now needs a a word vector associated with it now to


the thing is the word vectors let's assume you have them pre-trained or something in any case you need a big table a big big table and this goes down here where for each word a the two I you you have a Keter associated with it right so you need to keep this in your model and as you can as you know English has a lot of words here so this table is gonna be really big and the problem is how do you make this table right okay you could make it kind of dynamically and so on but in general you're gonna create this table with all the words you know and


that's going to be too big because English has so many words and then you can say alright we'll only take the top whatever is used in 90% of the language which turns out to be it's kind of burrito distributed so it turns out to be like 5 percent of the words are used in 90 percent of the language so you just take these but then you're gonna have the problem ok here 2 2 is not a problem why not 2 is used super often that we're gonna have it at the very top somewhere and we're gonna go back to it subscribe is it's already it's not so common right


so maybe you have a word for it somewhere down but then pewdiepie is a name so there is no there's not even a word like that's not even a word it's it's just so what you what you usually do what people usually do is they have this out of vocabulary token and then they have a vector associated somewhere here without of vocabulary token is a whatever I don't know what it is I just don't like that I don't have it in my vocabulary and the model kind of deals that that's kind of it's not it's not really ideal especially if you


don't want to generate language also your model tends to generate out of tabular tokens if you allow that if you don't allow that you have a problem during training so it's all kind of messy what's the alternative the alternative is to go character level so let's look at character level in character level you say all right my words are obviously made of actors and characters I'm just gonna split it each character right here the white space can be a character too so I'm gonna split at each character and then I'm simply going to have a bone


vector for each character and there's only like 20-something six of those and um so I can keep 26 vectors but this tends to be rather problematic because a character by itself having a meaning that you know that can be encapsulated by a vector is kind of its kind of shady because the character character by itself usually doesn't mean and it doesn't have a meaning so what's the solution here the solution is to go in between the solution is to say well let's actually go forward pieces and you can kind of think of them as syllables


but you can you can split you can make them in a way that you have a fixed size vocabulary say okay I have 4,000 entry places in my big table it's I can afford 4,000 size table so first of all I'm going to have for each character ABCDE and so on I'm going to have a vector but then only have 26 have 3,000 some left I'm going to have also the most common words now a is already here but maybe I can have two and from and so the most common words they also get there and then for the other things I'm going to split the words maybe in a sub scribe


right so these are two syllables and sub can be free kind of a prefix to many things and I only need then 1 1 so I've sub here sub I only need one vector for that and then the rest if described scribe is by the way also a word so I can have that but if scribe weren't in my vocabulary I can divide scribe Len up into into characters and then describe them with the so basically I can mix and match here I can sub it that's that I have that and then scribe I don't have it I don't have any of the pieces but so I can just use


the character the character so this would would be sub and then s C or I II so these these would be the tokens that I work with now as as my input and this these tags here so this is what would happen to PewDiePie you could simply split along each character so you basic this kind of an interpolation between the token model and the character model and it's really neat and it usually works quite well the as I said the the hashtag sign here simply means that these two have original in one word and now this this


in here is just a word piece token this is a really good example where where where piece come in because play by itself is a word and that can make play yang instead of having an own vector for that I can divide it into play which already has a meaning and presumably playing and play would have similar meaning so it makes sense to have to play as a so that's the token singled-out ear and then in as is as a suffix also makes sense to have a token for that in my table and then I said we have these two tokens here and that probably


already gives me more information than simply having the word playing right by the way you should subscribe to PewDiePie just FYI alright let's go on so we we do workpiece tokenization we do the masked language model we do the next sentence prediction pre-training what do we have now we have a model that can really really well predict some masked words now how do we use it they evaluate on these I believe it's 11 tasks 11 different tasks of or is it I don't know how many it is it is a lot with the same model so this pre trend


model then how claim can be fine-tuned to do all of these tasks and it gets up it goes like state-of-the-art on everyone it's crazy so how do they fight in it so the easiest tasks are the one are the so-called sequence level tasks where you basically have the sequence and you're you're about to predict one class label for the entire sequence so here where the sentence pair classification tasks for example the task we saw before the is next task but there is more sophisticated tasks that you need kind of supervised data for and so with


supervised that you'd have a class level that you trained on so what do you do is let's look at one of them ever MN l I they had it up here nope here multi-genre natural language inference crowd-sourced entailment classification task so given a pair of sentences the goal is to predict whether the second sentence is an entailment contradiction or neutral with respect to the first one all right two sentences and you're about to predict which one of these three labels it is so you put the two cent sentences here Burke can


already take two sentences as an input as we saw right the the embeddings are the the a and b embeddings and the position of endings are left out of the picture here but they would be added to it and these these would be the embeddings for it and then you pass this through the verb monel and this is the final layer and what they do is they simply take now the the embedding the final embedding for this first one course into this starter token and they simply put a single layer of classification so basically a logistic regression on it


and that's how they then get a class label so if this is whatever let's say this is this just gives you here a hidden vector of 512 dimensions 512 and you have three labels to output here 1 2 3 you simply need a a matrix that's 512 by 3 of size and these are the these are the weights that you would then have to train in addition the vert subvert is pre trained and you have to simply only now learn these weights of course they also kind of fine-tune the entire vertol but that's really fine-tuning the only thing you have to learn from scratch is


is this these weights here that's pretty first of all it's pretty neat because you can be very quick at learning new tasks because you simply start from the pre-trade vert and then you go and learn a single class for a layer on top and astonishingly this works extremely well for these tasks a bit of a a bit of a more challenging task is this year squat is a question-answering task and we're gonna jump down here where they explain the task so you have an input question oops you have an input question and then for


question is where the water droplets collide with ice crystals to form precipitation and you have an input paragraph which is kind of a paragraph from Wikipedia page and you know that the answer is somewhere in this paragraph right the data set is constructed such that the answer is in the paragraph so the entire referees precipitation forms as small as smaller droplets call ask the collision with other raindrops ice crystals within a cloud so you the question is where do water droplets collide perform precipitation the answer


here is within a cloud so that's this this thing here so usually what squad models do is they they predict the spam they predict where it's the start of the answer and where is the end of the answer that's also what kind of birds train to do so in order to do this what you do is again you already have the ability to input two sequences so we've trained with two sentences but here they say well you say oh well our first sequence is going to be the question our second sequence is going to be the entire power paragraph from Wikipedia


and then for each output for each output for the output of each token remember there's as many outputs as there's inputs because the transformer will always transform to the same length of sequence for each token in the output we classify it is this token the start token and or is this token the end token or is this token none of all now what they do effectively is that here each each one outputs each one is a vector and they as we said at the beginning of finding out which ones the subject now here we have two queries namely query


one which is is this to start let's call it query s and query e is is this the end token so these are two queries and I'm going to just produce compute the inner product of each query with each of these outputs right and over my sequence here this is gonna give me a distribution so start for start maybe this token is not much than this token is a lot and so on the other solution there's I've tokens and for the end not so much not so probable not so probable very probable not suppose so what you get when I get is from these inner products


is a distribution over which ones start in which ones to end ok this one's probably a start and this one's probably the end so that's how you predict the span and again what you have to ultimately learn is these these queries here and so not that much and this is named entity recognition and named entity recognition you have a sentence and you're supposed to recognize named entities like up here we saw subscribe to PewDiePie and the named entity would be PewDiePie right this is a name and you're supposed to recognize that this is a name and


they do it the same same way that they do the squat basically or a similar way sorry they basically for each of the outputs here they simply classify whether or not is it's part of an M entity or not so what they have to do is they have to simply train if they you also have different labels for which kind of entity is this this is like a person and this is this is no entity so if you have ten of the labels then each for each thing you would classify it into one of ten classes so you need a classifier of input size versus number


of classes that's all you have to train in addition to pre to fine-tuning vert itself alright so they kind of evaluate on all of these tasks they get super duper numbers on all of them here burped large winds on pretty much everything and this model is big just saying and they trained it on TP use which is available in kind of Google cloud infrastructure so far they've trained it on a lot of data so - - away it's it's kind of expected that you would outperform but it's very surprising that you outperform everyone else by this much and they've


done a lot of kind of Appalachian studies where they show that it's really due to the fact that they do this left and right context they they take into account the left and right context of given token when doing the the attention that it's that that's why it's better so here for example they compare the bird base model and they say okay what if we don't do the NSP the next sentence prediction tasks then you can see the numbers they already kind of they drop on these tasks and what if we then additionally do only left-to-right


training and the numbers they drop really seriously again you see sometimes here for example you see a pretty serious drop in the number also here so there really seems to be real value in doing this kind of left and right context attention so it's not just about the model size and the amount of data that's basically what they show here and this is really cool that the paper actually shows this because usually people have an idea and they throw a lot more resources at it and they're better you never know why and this is pretty


i am sapna sharma and i am here to present a paper i am here along with ragini to present a very interesting paper published by six google researchers in may 2019 the researchers are david birthlord nicholas scarlini ian goodfellow avital oliver nicholas papernote and colin raffle the title of the paper is mix match a holistic approach to semi-supervised learning so next slide please so we will be covering the background and purpose to see why mix match was [Music] required the key terms to understand the algorithm

MixMatch: A Holistic Approach to Semi-Supervised Learning


we will be giving a brief description of the algorithm and the results of the experiments performed by the authors so we are all aware that there uh we are all aware of the three main classes of the machine learning that is the supervised machine learning the unsupervised machine learning and the semi supervised machine learning while the supervised machine learning needs the ground tooth that is the label data to build a model the unsupervised machine learning predicts unlabeled data using clustering techniques now the major


concern for most data scientists is the labeled data as we ourselves face the same problem while building a model for wound tissue classification it is very difficult to get an expert to label the whole set of data thus the scarcity of label data is the major con constraint for supervised machine learning now here is where the semi-supervised machine learning plays a major role which takes the advantages of both supervised machine learning and unsupervised machine learning to give us label data and according to the authors of the paper


mixmatch claims to have developed a technique to label the unlabeled data with much better accuracy than the present day there may supervise techniques as per the abstract of the paper mix match unifies the current dominant approaches used in semi-supervised learning to produce a new algorithm that works by guessing low entropy labels for data augmented unlabeled examples and mixing labeled and unlabeled data using mix up so before going to the actual algorithm let us just have a look at the key terms which will be used to


understand the algorithm the consistency regularization it applies uh data augmentation to semi supervised learning by leveraging the idea that a classifier should output the same class distribution for an unlabeled example even after it has been augmented in other words labels should not change when noise is added the model called pi model is used mixmatch utilizes a form of consistency regularization through the use of standard data augmentation for images such as random horizontal flips and crops now the entropy minimization


it basically means that we need to reduce the randomness in the prediction of the unlabeled data or the classified decision boundary should not pass through the high density region of the margin marginal data distribution this is done by outputting low entropy prediction on the unlabeled data so mix match also implicitly achieves entropy minimization by adding a loss function and using a sharpening function on the target distribution for unlabeled data the traditional regularization is again a method applied to avoid overfitting


of the model we have two types of regularization l one or the lasso regression and l two or the ridge regression the range regression adds squared magnitude of coefficient as penalty to loss function and mix match uses the squared or the l2 laws on prediction and guest labels and lastly the main teacher to overcome the problem of inconsistency in using the exponential moving average of label prediction on each training set on large data a mean teacher a method that averages model weights instead of label prediction is used


mean teacher improves the test accuracy and enables training with fewer models fewer labels so i will be giving a brief description of the steps involved in mix match as the four points are like the data augmentation label guessing entropy regularization and mix up so in data augmentation it is common approach to compensate the scarcity of the unlabeled data and data augmentation is done by applying transformations on the input data points such such that the label remains unchanged data augmentation is done both on


labeled and unlabeled data the individual augmentations are used for generating a guest label we'll be seeing more about it label guessing for each unlabeled example mix match produces a guess for the examples label using the model's prediction this guess is later used in unsupervised lost term the entropy regularization to enforce the fact that the classified decision boundary should not pass through the high density region of marginal data distribution the classifier outputs low entropy prediction on unlabeled data this is done by adding


loss term which minimizes the entropy for the unlabeled data mix match applies sharpening function to reduce the entropy of the label distribution and this is done by adjusting the temperature of the categorical distribution now the last is the mix-up a mix-up is a recently proposed method for training deep neural networks where additional samples are generated during training by convexly combining random pairs of images and their associated labels by doing so mixer regularizes the neural network to favor simple linear behavior


in between training examples mix-up also reduces the memorization of corrupt labels increases the robustness of adversarial example mix match uses mix-up with a slight modification as the final step of its algorithm yes okay brilliant so uh thank you sapna for um giving a brief introduction about the paper before i get into the algorithm i wanted to just stop for a bit and ask if there are any questions anybody has so far okay i guess not so i start from um talking about more on the algorithm bits where if you see this is an image taken


from the paper where it says that the algorithm basically augments the unlabeled image and tries and classifies it based on the number of augmentations and then draws an average to guess the label of this unlabeled image that it has after it has this average getting predicted or the guess label getting predicted that less label gets sharpened uh what sharpening does is it basically moves the line of prediction or the line where the decisions are made away from the the higher density association of the data points


uh because that makes it easier to give a consistent prediction and also increases the confidence of the predicted labor as you see in label guessing we only guess the label of the unlabeled data however we do augmentations on both labeled and unlabeled data to come to this um label guessing wherein the label is guest based on whatever is in the label data as well as in the unlabeled data but transformations are mostly done on the unlabeled data as many times as required but according to what the authors have seen


and gotten the results uh k that is the number of augmentations uh is two that given a result which did outperform all the standard state-of-the-art methods after this the sharpening formula or the equation that they've used where they do have another tuning parameter called st which is the temperature required to sharpen the image prediction or the label that is guessed based on the average prediction that is obtained in the previous step they average they take the decision boundary away and make the prediction more confident


so that when the loss is calculated on the unlabeled data it it is not affecting the the final prediction after sharpening what remains is two sets of data one is the augmented label data with the sharpened predictions and the label data which already has its own labels so what is done is a mix-up based on this parameter tuning which is called alpha so what basically makes up does is let's say uh you have an image of a tiger and a cheetah and you want to see how much of one image corresponds to a tiger and how much one image


corresponds to cheetah so what you do is you mix these up together saying 80 20 and have a new label saying that that image is now classified as eighty percent of a tiger and twenty percent of vegeta and that mix up is this alpha value which is again tuned based on your data sets and availability of the number of data or images that are collected i forgot to mention the sharpening temperature value that they the authors have taken as a constant for their experiments it's 0.5 yeah i think it's 0.5 after the first


pattern let me just quickly go back and see um i think it's in the next slide so we can look yeah it's 0.5 so what they they've tuned it uh to sharpen the picture sharpen the the prediction then they tune in the alpha parameter to have a proper mix up of the labeled and unlabeled data and they form in a big bunch of mixed data set which has unlabeled as well as label data set and i think we've been just talking too much of labeled and unlabeled this is the most important factor of a semi supervised learning method wherein


you have your training set that comprises of labeled as well as unlabeled data the reason being it increases the chances of getting more label data which in turn will be fed to your model to have a larger training set than what was initially available so all this is getting done to ensure that your training set increases in number than what you have earlier the last step in the algorithm is calculating the loss because we have a label set the first loss that gets calculated is the cross entropy loss for supervised


learning and the second one asapna mentioned is the l2 loss for the unlabeled learning i would stop here and ask if there were any questions okay i go forward so this slide explains what i just talked about but in a more um direct way and what all they have tuned in so the key factors to keep in mind for this algorithm to perform better or worse is tuning these hybrid parameters as the augmentations sorry k i'm sorry then the the sharpening temperature t the the parameters for mix up that's alpha and the weight of the unsupervised


consistency loss lambda which is a hundred in this case um the authors performed two sets of experiments one was a normal semi-supervised experiment set up where they compared it to the state of art uh semi-supervised learning models and um the other one is um the the method in which they cut out all the additional um transformations or sharpening or um any extra effort that they've put in to get into the efficiency or the accuracy level that the model performs to show that which uh bit in the algorithm performed the best


to get in the desired results of the desired accuracy at the end so for their semi supervised experiment set up uh they used a wide resonant model with 28 layers and a growth of factor two the data sets were our standard data sets at the zipper 10 say 400 the svhn which is the street view house numbers and the stls uh the models to compare were the main teacher as satna explained in her brief introduction about what main teacher does it is one of the semi-supervised learning methods wherein the labels are based on the exponential


moving average a virtual adversarial training again a semi-supervised method and the pseudo labeling again a semi-supervised standard model so you see from the results where the model that they used initially was the cipher 10 data set a supervised training of 50 000 examples was trained to to given the said accuracy but as comparison with mismatch it just used in 250 labels to given the desired output similarly they did it with svhn again with 70 3250 examples with no unlabeled data at all and they got in the said


with a label of labels of 250 again um it given the said efficiency so what i mean by 250 labels and 2 and 73 257 examples is that the entire data set was used for training purposes for this particular supervised uh experiment out of the 7 73 257 only 250 labels were used in to train the remainder of the unlabeled data set to get in the performance that it shows in the results and this is the abolition study set up where i said that they talked about adding or removing additional components that they used to come to this level to show which


of the exact step in the algorithm performed better than the rest so you see the first one is about the mix and match where it's 100 mix and match and it gave an error rate of 11 for 250 labels and an error rate of six percent which is quite good for 4000 labels they removed the distribution averaging or the guess labels and said the number of transformations as one it gave in a 17 error rate again no sharpening it given 27 they had an um they do mix up they did it with label data mixer they did it with unlabeled


data mix-up and you see in this that the mix up is benefiting the performance of this model um going to the end uh concluding what we have understood from the paper or the basic purpose of what the authors wanted to showcase using all the components that they have in creating the algorithm it is seen that the hyper parameters that they've used is the augmentations the sharpening the mix and match and the loss that they've calculated it seems that they are have contributed quite well in the performance of the model


but out of that in the as you seen in the previous slide that the mix-up seems to be the most important factor contributing to this a mix-up again is a tuning hyperparameter and if you tune it further you probably might get in better results if you don't tune it as much you might get in less results so these were the positives that i could get or we could see in the paper however there is a bit of a negative or that that comes out through this algorithm that the time and cost needed to generate all the transformations the mix up


and have a neural network set up on it does take a lot of time and it also requires additional gpus to be set in which perhaps could be seen as an additional overhead but it still does perform a lot better also it leaves needs a lot less data to start to train in terms of having any expert coming in signing up the training set saying that yes a label is an a label b label smb label so you see that this method does showcase that it is one of a better ways to go forward in semi-supervised space for further reading i have listed a


couple of uh papers to go ahead and understand where the thought for building up this model came through um and i think this is the last slide if you have any questions i should be glad to answer them um did you try to use the implementation that they provided on online yes i i did try it i tried on lagoon data set and i could share in my uh code base with you to give it a try so so the the the results were were really good did it improve uh well we don't have enough of um data for the wounds data sets so i would say it


did perform good uh i'm not really sure whether it will perform better uh because the data was quite less but i do see that it will go ahead and do a better job gotcha awesome thanks any further questions i don't have a question but i was wondering if you could sure um the network that you tried to run it would won't that that's one type you use or was it which category of one did you use all of them it's a semi-supervised learning space so labels are given for all and then i took in half of that as an unlabeled set and not have labels for


it okay could you maybe share that on the channel yes appreciate it sure thank you so yeah i have a question uh so what are the other like i'm not quite like uh quite familiar with semi-supervised vanny algorithms so what is the like what's the state of the art between before this algorithm what what what did they compare their girlfriends to in this paper um so he didn't quite understand your question so so before so when they introduce these algorithms so they have they should compare it to something that how were things done before this uh mix


of algorithms for semi-supervised so they did us uh if you see in here um uh they've used in a supervised method wherein they used all the 50 000 examples that were labeled so they needed labels for 50 000 items or entities and they had to train based on that but with the mix match coming in they reduced the number of training set or labeled set to 250. if you know what i mean so so let me go back and i'll explain a bit of a semi-supervised learning space so semi-supervised learning space needs both a bit of a label


set which is a trained set and a lot of unlabeled data set because what it does it takes a little bit of the label set and a bit of unlabeled set and forms in a mix of labeled and unlabeled and the reason being that it tries and labels all the unlabeled um examples that it has in this mix so that the end training sample has got more no i i know i know i know like the idea behind but i i i'm not sure what i was just asking what was the state of the art before like before me this mix of algorithms for semi-supervised it was a simple


semi-supervised approach wherein you have iterations where you take in from your unlabeled set perform some basic transformations or have a have some sharpening done for your confidence uh of your label set having say uh prediction probability or maybe have co-training methods that have two classifiers learning on the same view etc and then give your prediction so mix match as such introduces all of these methods together so they they didn't stop at just using simple uh transformations or simple decision boundary making algorithms


or a mixed match they this is a combination of all three together so state of art didn't have this the novelty of this paper is mixing all of these uh semi-supervised learning approaches together in one to come up with uh their holistic approach so that's why the name okay so basically though like uh those uh pseudo label and v80 and uh was it these are also semi supervised approaches right yes okay uh i have a question so suppose we take three examples right extending what you said cheetahs tigers and we add a


third one leopards um and uh so cheetah and tiger are in the 250 labels that are used and leopards are in the 4000. does that mean all the cheetahs and all the tigers in your data set are labeled or are some of them unlabeled it could be some of them unlabeled or they could be some of them labeled and it can be both it all depends on the percentage of the unlabeled data that you thank you hi i'm curious about one thing you say that you tested with the wound data set uh did you try perform it against fast ai platform yes yes with first ai


yeah and how how your i mean how is their performance with this method compared to plasma eye for the training uh set i did get in about a 64 64 to 69 percent uh accuracy on the training set initially and with this it did bump up to about 70 but i wasn't sure whether it was uh the right thing that i was doing so that's i'm still playing around with it but uh i know where the where the tuning needs to be done so i should be able to have a better accuracy on that this is on the training set i've not yet gone on to


testing the model on the test data they'd probably be higher than that yeah yeah okay thank you me any further questions should we say that's it then okay thank you very much uh if you have any questions um please get back to me i should be glad to answer them and i will share in the link to my notebook i am still a little not worse than not books i write it in uh prehistoric language as an editor and things like that but i will move on to notebooks soon and i'll post it there so thank you very much for attending it


[Music] thanks for watching Henry AI Labs this video will present the semi weak supervised learning framework presented by research at Facebook's AI research lab this framework is a really interesting extension to their previous work on weak supervision such as using the hashtags and Instagram images as a weak supervised signal to trip retrain imagenet classification models this research is going to extend this idea to integrate semi-supervised learning as well as weekly supervised learning and then introduce a lot of other

Semi-Weak Supervised Learning


interesting ideas like incorporating model distillation into this framework and looking at class imbalance evident in these unlabeled data sets this video will present the research paper billion scale semi-supervised learning for image classification from researchers at Facebook a research lab this animation from Facebook's blog post on billions scale semi-supervised learning shows the idea of their semi-supervised training framework before integrating week supervision so in this case their take on semi-supervised learning is different

Semi-Weak Supervised Learning


from other definitions of semi-supervised learning such as rotating an image and then predicting the rotation angle or something like word to vector wave defect where you mask out certain parts of the sequence and then train the model to predict the missing part of the sequence this idea of semi-supervised learning is to have a label data set such as the image net data set train a large capacity model like a res NEX 101 32 by 48 group convolutions res next architecture and then use this massive high capacity

Semi-Weak Supervised Learning


model to predict the labels on an unlabeled data set and then you would use these unlabeled the softmax distribution of these predictions to pre train the target model ie as in model distillation such as what powers models like hugging faces distill burped then you'll fine tune the model that's been trained with model distillation on the label data set and this is your new model so one of the interesting things we already see about this is the novel use of model distillation as semi-supervised learning it's not really

Semi-Weak Supervised Learning


common to use these two terms together semi-supervised learning and model distillation also interesting lis is we can see this kind of model compression that arises from this framework you can have a really high capacity teacher model like the res NEX 101 32 by 48 D and then you can have a lower capacity more manageable probably faster inference time lower storage costs like the res net 50 that you could deploy on mobile and IOT in these kinds of things this animation shows the extension from the semi-supervised training framework

Semi-Weak Supervised Learning


to semi weekly supervised training so in this case instead of having some massive unlabeled data set we have a weekly supervised data set so instead of just having a collection of images we have a week label such as the hashtags on Instagram images and so the thing with the weekly supervised hashtags on Instagram images is that they're really subjective they're noisy and not really like as precisely labeled as say the data from imagenet so in this model we're going to pre train the teacher model same idea of having some larger

Semi-Weak Supervised Learning


capacity teacher model smaller capacity student network so we're gonna pre-trained it with the weekly supervised data set fine tuna with the image net then we're gonna use the fine-tuned model after having been pre trained to predict the softmax distribution over the weekly supervised data set and then when you use this model distillation knowledge distillation in order to train our student network we're gonna fine tune the student Network and then we have our Train model some of the interesting issues that research paper raises is

Semi-Weak Supervised Learning


this idea of class imbalance and unlabeled weekly supervised data sets so one characteristic of machine learning models and you know deep learning convolutional image classifiers is that class imbalance can really destroy the performance so for example if you're training a cat versus dog image classifier and you have 80% of your data your training data is cats and 20% its dogs your train model is going to want to predict cats it's going to be biased towards the imbalance data so in these weekly supervised data sets such

Semi-Weak Supervised Learning


as hashtags on Instagram images and then trying to transfer that into image net classification there's going to be like a long tail distribution where you're not gonna have as many of these really specific classes and image net contained in this image net data set so another interesting idea is just incorporating model distillation in the semi-supervised learning this teacher-student model compression and then this framework is gonna achieve 81.2% imaging at top one accuracy with a ResNet 50 and they're gonna scale this

Semi-Weak Supervised Learning


up to the res next 101 32 X 16 D with the student network capacity and they're gonna achieve 80 4.8% and this is up from eighty four point two from their previous research on doing a lot of label engineering for the weekly supervised Instagram dataset and then in that previous study they had achieved eighty five point four but it is a larger capacity bottle at the 48 D and they probably didn't test the 48 D because you know it's expensive and time-consuming to train these kinds of models this research from Facebook's and

Semi-Weak Supervised Learning


research lab is in line with a lot of their other work such as using weekly supervised pre training for video action recognition using over 65 million images like from Instagram as well as using billions of images in their weekly supervised pre-training of an image net image classifier in this case they do achieve a slightly higher performance but they do have larger capacity models and also interestingly is that in this case they are manually doing the you know kind of removing some of the noise from the week supervised data set

Semi-Weak Supervised Learning


whereas this framework is gonna present an automatic way automated way of doing this an interesting characteristic of the newest semi weak supervised training framework is they're going to use an explicit algorithm to balance the classes in the predicted distribution from the weekly supervised data set so this teacher model has been trained on the image Minette classification task but the weekly supervised data set probably isn't as balanced as imagenet classification it's probably heavily skewed to sort towards some classes more

Semi-Weak Supervised Learning


than others such as dogs and things like that compared to these really specific things like I don't know a tiger shark there's things like this so this visualization shows the top k scoring of examples so as the teacher model predicts a distribution of class labels over the unlabeled data or the weekly label data the models are going to be ranked according to their probability their class probability and then they're going to be balanced in this way so that each class has an even number of training samples and in this

Semi-Weak Supervised Learning


visualization you see how as it goes from rack 1000 to being very close to the image like the leaf beetle looks like the beetle whereas by rank 10000 16000 is not really a beetle anymore but the teacher model has given some probability to beetle when it's a predicting distribution of that image another interesting issue with this framework that's raised in the paper is the idea of inference time with model distillation so in this case we're predicting over a billion unlabeled or weekly labeled images with our teacher

Semi-Weak Supervised Learning


model so we want to have fast inference time for this prediction is much more important in this case than it is for typical examples of model distillation where the data sets aren't that large when you have billions of images you want to make sure the teacher model has fast inference I think with the rising popularity of knowledge distillation model distillation techniques such as hugging faces disco Bert and now Facebook semi weekly supervised training paradigm we're gonna see these kinds of inference accelerators like in videos

Semi-Weak Supervised Learning


tensor Artie accelerator to be more and more important because usually these frameworks are developed for inference when the model has been deployed but now we're seeing the inference be a part of the training loop as well in this teacher-student Maalik model distillation paradigm from their paper the researchers at Facebook give these six guideline recommendations for large-scale semi-supervised learning so the first idea is really interesting this teacher-student model distillation paradigm also really interestingly and

Semi-Weak Supervised Learning


uniquely in this paper is that they're going to test model distillation when the student and the teacher have the same architecture or the same capacity the second idea is to fine tune the model with true labels only this is a pretty intuitive idea the weekly supervised label dataset has a ton of noise in it compared to the imagenet data set or you know other more specific label data sets a third idea is that large-scale unlabeled data is key to this performance naturally the key driver behind this algorithm is that

Semi-Weak Supervised Learning


they have a billion images from Instagram that they're using to train this model the fourth idea is really interesting that they use a large number of training iterations for their pre training with the weekly supervised learning compared to you know more pre-training iterations compared to normal supervised learning the fifth idea is a novel contribution to this paper the idea of having a balanced distribution for inferred labels so when you're doing the model distillation you want you don't want to have class and

Semi-Weak Supervised Learning


balance in the distribution of these labels and the sixth idea is that pre training the high capacity teacher model week supervision further improves the results the idea of adding the week supervised to make this the semi week supervised learning framework now we'll get into some of the results of their research report you can check out their repository on github semi-supervised image net 1k models where you have the pre trained models that you can load with torch hub and then they also present some of the results you see the

Semi-Weak Supervised Learning


semi-supervised learning framework different model architectures from ResNet 18 and 50 up to the res next group convolution architectures and then you see up to the 80 4.8% accuracy when using the semi-weekly supervised learning framework with 193 million parameters on the res next 101 32 by 16 d architecture the first set of results they present shows the success of the semi-supervised learning framework with different student models so first we're looking at the resident 18 the resident 50 and then higher versions of the

Semi-Weak Supervised Learning


Resnick's and the 50 101 higher capacity resonate variations so we see that the fine-tuned semi-supervised learning framework is always outperforming the fully supervised learning tasks when you just train the student model on imagenet classification then they present this idea of varying the complexity of the teacher model and showing how increasing the capacity of the teacher model increases the accuracy of the student model we see the gains are increasing every time as we scale up the capacity the teacher model while holding the

Semi-Weak Supervised Learning


student model with constant this plot shows the results when the teacher and the student model have the same architectural capacity interestingly we still see these gains when we're using the same capacity for each model this plot shows how the top one accuracy changes as a function of the unlabeled data site data set size used as the you know the unlabeled or the weekly supervised data in this pipeline so we see that the performance continues to increase as a data set gets larger and larger following the recommendation from

Semi-Weak Supervised Learning


their guideline that having massive unlabeled data sets is key to this framework this plot shows how the accuracy improves as a function of the number of pre training iterations so as stated in their recommendations they use a much larger amount of pre training epochs than supervised learning epochs really interesting Lee is there showing four billion training iterations it achieves the highest accuracy in this plot then they show the results of increasing the K parameter so the K parameter shown here is this idea of

Semi-Weak Supervised Learning


scoring the class label distributions when you're doing the model distillation from the teacher Network so basically the idea is if you increase the K from say eight K to 16 K and then you're looking at a specific class such as leaf beetle the as you get towards the end of the eight K and then especially from say eight thousand one up to six the top eight thousand one to the top sixteen thousand the images are going to look less and less like beetles they just have been assigned some probability as beetle and now they're a part of the

Semi-Weak Supervised Learning


balanced data set because you've increased this K parameter so they show how there is a limiting effect to how large you increase the K because naturally as you increase the K passed a certain threshold you're making your knowledge distillation data step four your student network to be really imbalanced and deep learning machine learning these kinds of decision boundary models do not respond well to class and balance although they don't show you like things like random over sampling a lot of the techniques commonly used to overcome class

Semi-Weak Supervised Learning


imbalance this table shows the evidence of the balancing of the distribution with these ablation studies such as balancing dataset or unbalanced in a dataset showing a point eight percent accuracy improvement which is very significant for imagenet classification and then also the idea of using the Instagram tags versus ranking the list of the pretty distributions and then comparing all these performances to supervised learning this table is the showing the big highlights of the paper you see they achieve 81.2% accuracy with

Semi-Weak Supervised Learning


the resident 50 architecture and how this is state VR compared to previous works trying to achieve weekly supervised learning at the same model capacity then you can see the head-to-head comparison with their previous work on you know label engineering the week supervised learning and sort of you do see how the accuracy starts to saturate at the further model capacity of the Residex architecture another really interesting characteristic of this framework is success on transfer learning so when they transfer this pre trained model

Semi-Weak Supervised Learning


from imagenet to the bird image classification model they achieve a really high level of transfer learning performance compared to previous approaches this chart shows the difference between fine-tuning just a fully connected layer at the end of the network compared to the full network and then the performance achieved after doing this they also test this model on video classification with the deep mind kinetics data set and they show a significant improvement achieving seventy five point nine percent accuracy

Semi-Weak Supervised Learning


using this technique compared to the previous research achieving seventy four point eight percent accuracy thanks for watching this presentation of semi weak supervised learning from facebook's AI research lab this research paper has presented a really interesting framework for semi-supervised learning and weekly supervised learning and integrating this model distillation paradigm there are some really interesting ideas presented in this paper such as the importance of how they having a balanced class distribution for the model distillation

Semi-Weak Supervised Learning


eighteen secrets that lie hidden in your subconscious mind as I continue to do the work of going deep within my own subconscious mind which is really a net result of the training that I put together which was released earlier this year on programming your subconscious mind I recognize that there's far more to what goes on in the subconscious mind and how we create reality based on what's on our subconscious mind so much so that I've been making a lot of videos lately on this topic and what I've done in this video is I've pulled 18 key

18 Secrets That Lie Hidden In Your Subconscious Mind (Neville Goddard, Napoleon Hill)


distinctions that we're going to discuss from the perspectives of neville goddard and napoleon hill who I believe are masters with working with the subconscious mind and these 18 distinctions are reflections insights perspectives that I've gathered as a result of working with their information and as a result of working with what I had put together in my subconscious mind training program which by the way made a huge difference in my life because I think I might have mentioned this a few times but in order to test the


effectiveness of reprogramming my subconscious mind around that time I decided to apply it towards attracting a ideal relationship and by using the exact principles in that program I found myself in my ideal relationship in fact my girlfriend is in front of me right now as we are recording this she's playing a huge integral role in what I do and together we create amazing content for sharing and I also want to add that on the list of qualities that I look for she meets every single one of them because I realized one very


important thing the vision is the identity and I made a video on this last video I did talking about the imagination being the identity and you can say the imagination and the vision is the identity both the same and also different as far as discussions go we say imagination and vision but the bottom line is this who you are who you're destined to be is your true vision and you can create it by cleansing disempowering thoughts and elements in your subconscious mind which you have learned through five sensory


based input and meaning associated to it meaning that you have created that you've associated to the five sensory input data or meaning that you have learned from others so my goal in this video is to assist you on your journey to materialize your dreams to bring forth your vision I made a number of videos helping you uncover your vision and I'll continue to do so and in this video we're talking about releasing elements in your subconscious mind working with really deep elements in your subconscious mind to bring forth


that what you truly desire because your vision is yours and you have the power within you to bring forth your vision by working with the power of your subconscious mind neville goddard says so when you know what you want remain faithful to that assumption and the assumption though at the moment is denied by your senses and denied by reason if you persist in it it will harden into fact which brings us to our first point what gets hardened into fact controls automatic behaviors of the individual if you believe that you are


not worth it as far as your vision goes if you don't believe fully within you that the vision is your birthright it is who you are then you have elements that are within your subconscious mind that are being projected outwards and materialized into form through various different ways but primarily or it could even be secondary through your behaviors the way you carry yourself the way you connect with people the way you navigate reality the way you handle different people environment and circumstances in your life reveals to


you as to what you are assuming to be true in your subconscious mind which a lot of times can be hardened into fact now when it's hardened into fact what you'll notice is that in the external you will find all kinds of evidence to support that fact now if you believed in another assumption the polar opposite of that assumption and that was hardened into fact then you're gonna find plenty of evidence to prove to you that that assumption is true by the facts that you will find in the external world now what does this mean this means that the


source is the thoughts within and the thoughts within are mostly subconscious we have a conscious mind we've got a subconscious mind and the subconscious mind is responsible for creating our reality and the subconscious mind has to become on board with our vision and the way we do this is by impressing our vision through our imagination or subconscious mind reconditioning or taking in information via our five senses that are in alignment and congruent to our vision till that hardens into fact and behaviors thoughts


and actions and whatever you do projects outwards interacts with the external world the environment people and circumstance to bring forth your vision number two what gets hardened into fact removes from an individual's consciousness that what is not related without the use of mental or emotional force what I mean by this if you believe reality to be a certain way and if that has hardened into fact you will find yourself surrounded by people environment circumstance and information that supports it and what will be


excluded from your consciousness is that what will not be related to that vision you won't be able to even see it it'll be like a blind spot now this can work to our advantage I'm not talking about being indifferent I'm talking about being in the spirit of harmony your true vision comes from who you really are your soul expression see I believe we have our conscious or subconscious and the super conscious the super conscious is the universal mind it is the one mind the single mind in which all individual expressions from that one


line contained within that one mind is our true vision and when we uncover that vision in our imagination and we honor that vision what gets included in our consciousness is that what is related to the vision which includes being in the spirit of harmony with all people environment and circumstance and what gets excluded from the consciousness is that what is in harmonious in thoughts feelings emotions and behaviors that create in harmony in the external world but the truth still remains that the source is within it is from mostly the


subconscious so as you observe what's within your consciousness with your awareness not what distracts you that what your attention goes on it reveals to you what's within yourself and what is revealed to you can be worked on if it's related to your vision then you can encourage that if it's not related to your vision if it's an emotion that creates turmoil within which reflects outwards and materialized into form as turmoil with people environment circumstance that you can change that within by adjusting your subconscious


mind but the bottom line is this is that as a person continuously does the work on themselves the environment the people and circumstance the external world becomes more harmonious number three faithful assumption is what levels up a person's thought beyond limited thinking now what Neville is saying is remain faithful to that assumption even if it's denied by your five senses five senses the sixth sense is your vision it is your connection to the superconscious it is your conversation with infinite intelligence which we're going to talk


about in a moment the five senses are earth based senses and there are empowering thoughts and feelings via those senses and there are some disempowering ones that are belonging on this earth based experience the disempowering ones are fear doubt and indecision because for many thousands of years we had a certain level of consciousness and it wasn't high enough to help us realize that abundance is the natural order of things when we are in harmony with the lovers within which I call the connection between the conscious the


subconscious and the super conscious harmony within but we're talking about here in this video is becoming harmonized between the conscious the subconscious and superconscious and even if you don't believe in the super conscious even if you don't believe that it exists then we can at least believe because it'll do wonders for you and then in the process you'll uncover this super conscious and you don't really have to be living a harmonious life it's the relationship between the conscious and the subconscious the conscious being


the Guardian the director the decision-maker on where you decide to go and where your attention goes on and guarding what goes into your subconscious and the subconscious being the creative force that expresses outwards to materialize into form that what gets inputted into the subconscious will be impressed upon the subconscious and the subconscious will bring it forth so by remaining faithful to that assumption you are working with the imagination which is the success so you are transcending the five senses you're


saying this is what I want impressed on the subconscious mind that what is in my imagination and the subconscious mind will bring it forth whether it's from the imagination or whether it's from the five senses now what is in the imagination which comes from your vision is a higher level thinking a lot of times you'll find that it transcends the thinking beyond what you have learned from the five senses and as a result of honoring that imagination that vision that assumption of the wish fulfilled in the imagination your thinking will start


to elevate what you'll notice is that you will think different thoughts that are more empowered or more empowering to yourself your self-esteem your self-confidence will go up towards others people environment and circumstance and these thoughts that you have will become a assumption that will harden into fact because you will honor the thoughts the thoughts will be projected outwards and materializing to form and you will see it and you will assume it to be fact at a subconscious level and you will see it with the repetition Rima


T realizing over and over again into form he says so if I assume that I am I don't have to I don't need any evidence to support it I assume that I am and what well i name it and having giving it a name giving it form given it definition remaining in it I resurrect and if it takes a thousand men to aid the birth of the state a thousand men will play their parts and I don't have to go out and look for them what is he talking about see there is one imagination really one mind and we are individual expressions of that mind the


super conscious mind is the universal mind and you as an individual is made up of a conscious and a subconscious mind when you're dealing with something like faith and honoring that imagination living in the imagination you are working by sending a faith-based message over to the super conscious mind and the super conscious mind the universal mind is connected to the subconscious mind of you and everybody else and that information goes over to them in via the subconscious mind and they will have hunches and inspirations to be in


connection with you now this is something that I experience on a regular basis a lot of times I get emails from individuals that say I felt I had to reach out to you and you have the answer to this question that I'm looking for and they asked me the question and it happened to be the very thing that I figured out last week and we are all connected via this invisible link the question is do we believe it's possible do we honor the connection how do you establish the connection you establish the connection through higher vibration thoughts


leveling up your thoughts thoughts that are in the spirit of harmony what is the spirit of harmony well looking look at the spirit of harmony from multiple perspectives a good relationship between the conscious the subconscious and the superconscious number one which I call the lovers within and number two a relationship and realization that your vision doesn't take away your true vision doesn't take away from the world from divine from evolution it contributes to it win for you win for those that you deal with and win for


divine or evolution that's the true vision now this might not have been happening for thousands of years but our consciousness is going up and this is going to continue to increase and we will see more and more living in the spirit of harmony look around and you'll see that there's far more people living in the spirit of harmony when they embrace this philosophy a lot of my students including myself have realized that by releasing that does not that what does not serve me that was within my subconscious mind which I've taken in


beyond five senses and given meaning to that was disempowering by releasing that programming I allow myself to go into the superconscious and get my vision and express accordingly why because fears start to go away fears break the connection between the conscious the subconscious and superconscious so number four we are individuals made up of conscious and subconscious mind connected to the super conscious mind otherwise known as the universal model and when you work in harmony in the imagination by encouraging positive uplifting nurturing


spirit of harmony thoughts that information will go and up to the super conscious and the super conscious will work with it that's how you work with the universal mind that's how you communicate with the universal mind the universal mind will not bring scarcity based thinking into the subconscious mind of others it will not be allowed to go forth that kind of information is from the five sensory-based input here on earth it does not exist in that level of vibration that level of consciousness that is the superconscious


which is the creative source of everything spirit of harmony brings forth number five those that are in the spirit of harmony now the beautiful thing is that when you work with this when you cleanse your subconscious you'll start to think more spirit of harmony based thoughts and you will notice that you'll attract the people that are also in spirit of harmony and I'll keep increasing it's almost like when they show up into your life you could say I was expecting you and they will say to you well I was expecting to


deal with you and here's what I would like to do and you would say to them here is what I like to do and you will find it to be in harmony in harmony in the spirit of harmony it is what is known as a true mastermind true mastermind see in my opinion and working with this through realization of experiencing this a mastermind is when two minds come together to a more come together and they create something that is greater than than they could have done by themselves so they're working on a creative solution to a problem and as


a results of the two Minds working in sync they come up with an idea a concept a way of solving the problem a way of going about doing things that is greater then they are actually combining together to be in the spirit of harmony and as a result of that infinite intelligence shows up and creates the third mind so when you cleanse your subconscious mind you'll notice that you'll be in more of a greater degree of spirit of harmony within yourself between the conscious the subconscious and superconscious and you will attract


those that are also like that because you don't attract what you want you attract who you are and part of this journey of cleansing the subconscious mind is becoming who we are being one with the vision the vision being the true identity number six your brain knows your imagination as reality see this is very interesting if you consistency and persistency from that place imagine believe have faith in that what you hold in your imagination your body will start to act different your thoughts will be different your posture will be different the way


you communicate will be different I know because I've been working with this a lot of people ask me how did I improve my communication skills how can I get on a microphone and just keep expressing like this so precisely well it's because harmoniously within I've developed a really good relationship between my conscious mind my subconscious mind in my super conscious mind I am not overly in my head yet I am consciously aware my subconscious mind is a repository of a lot of experience that I've gathered in my life


in business entrepreneurship on these topics which expresses through me and as that connection and facilitation happens the superconscious gets involved and projects outwards now what happens is my brain is working there's no doubt about it it is involved in the equation and all of these behaviors this way that I'm communicating and everything that I am is a net result of what I imagined myself to be and for many years I would imagine myself being this way and that applies for everything else and as a result of that by behaviors my actions


my words how I communicate expresses accordingly the training happens in the mind and the visualization now this is not to say that you can't do the hands-on training like practice public speaking but you can absolutely work in your imagination to become a better communicator your imagination can be made so vivid that you can't tell the difference between a real act in an imaginary act and if you've done it right you'll notice your behaviors have changed he says so if you proceed your visit by an imaginal act they will see you as you


see yourself if you walk in knowing that you are no good they will see you exactly that way but if you walk in the assumption that things are as you desire them to be they are going to see you that way and this is life so while in the presence of another practice putting no label on them to allow infinite intelligence to express out from you and materialize them into form in the spirit of harmony see what happens is we have our imagina lacked we believe ourselves to be a certain way and that's either a conscious imaginal act or it's a


subconscious from past program but regardless we have that in our imagination and what is in our imagination impresses on the subconscious mind and projects outwards to materialize into form of how we interpret people environment and circumstance see all people are the same we just believed them to be different now that might sound like a bold statement but practice it imagine in your mind how you want to be received imagine in your mind how you want to receive others and when you show up in the presence of others release the


conscious judging release the categorising allow the subconscious to express that what was impressed on the subconscious and allowed superconscious to also do its thing and you won't start to notice that they will be that if you imagine them to be a certain way a positive way and you show up and they're not that way there is programming that's in the subconscious mind that's still projecting outwards to materialize them into form why because all is actually one mind if you believe them to be their greatest self that what is actually


their true vision then that message will go into the super conscious and the super conscious will communicate to their subconscious and they will start behaving that way around you their behaviors will be different around you what you might even notice is they behave totally different to you versus others they'll be more harmonious around you they'll be more pleasant around you this will be more gracious around you now I'll speak from experience on this because everybody that I'm around with now presents themself in this higher


level of version of themselves whereas in a past time in my life I would walk around with scarcity thinking and negative programming in my subconscious mind and the world would seem really dark it would seem really harsh and people would respond to me in kind of this dark energy and I I would think it was them failing to realize that I was REME aterial izing them or projecting outwards them into form and attracting those that want to play that theater with me but after I realized that I'm the cause and I started working on this


by working with my subconscious mind by doing my imaginal acts by cleansing my subconscious mind using subconscious audios and other modalities what I then realized is that by assuming them to be a certain way they show up that way even if it's just for that moment they are that way and I've watched people transform in front of me people who I haven't seen in years who were a certain way but now they're this different way number 8 mood materializes into people environment and circumstance to reveal our mood within how we believe reality


to be will be reflected in the external world to be so that's one of the deep secrets of the subconscious mind a deep understanding and grasp of this gives you an enormous amount of power now we're taking this down into a nuanced level of emotion mood how you feel they will feel if you're angry they will be angry in front of you I know it seems kind of far out like how could it be that way but just try and remember actually remember a time in your life when you had an amazing uplifting powerful source based mood


abundance and you were doing your thing whatever it is you were out and about notice how harmonious the world was to that mood and I gave you my own personal example a time in my life where I felt a lot of darkness within negativity with it I experienced it without in the external world now I've seen this happen many times in my life a really high peak positive state reflected outwards by surrounded as far as the senses can see by people environment and circumstances that are harmonious to that and I've seen it in darker times in my life again


surrounded by people environment and circumstance to reflect it accordingly I was the source within I am the source within my mood materializes into people environment circumstance to reveal what is going on within me in a way they're helping me they're helping you telling you what's inside of you when you adjust that within yourself via the subconscious it starts to change because see we're not trying to consciously will and force this the subconscious mind projects outwards and creates into form that's it and it's it does so


automatically we don't even have to think about it consciously it's always happening all the time and it can be programmed in our imagination and I like a dual process working with the imagined imagination the imaginary act and journaling taking notes of what is revealed to me in the external world as I go about reality doing whatever I do every day for me it's the world of entrepreneurship it's one of the best worlds to reveal about myself because it's a world filled with higher level of challenges that I choose to rise up to


to reveal more about myself and as I rise after those challenges I create more success more returns and this has been the process I've been following for years and I will continue to do so and each time I noticed something that is inharmonious revealed to me in the external world I will write that down I will take a note of it and I will address it by in my imagination or my subconscious mind audios affirmations now unlocking the power of affirmations is very important I got a specific process and I created a video about that


and I'll put a link in the description thoughts become things and affirmations in the external world which may be assumed as fact the origin is still thought and thus that assumed fact no matter how valid it may appear can and will be changed by thought now by the way these quotes that I'm reading here are the actual 18 elements are pulled for my Instagram and every day I post to my Instagram so I recommend you go over there and follow me and consume this information because these are thoughts that I have as a result of working with


my own subconscious mind and working with my clients and helping them in areas of their life whether it be entrepreneurship and business and personal development working with a lot of the identity elements in the subconscious mind so I took them and infused them in here they were my inspiration so thoughts become things and affirmations in the external world which may be assumed as fact what is being revealed to you in the external world because it's there because you can pick it up within your five senses you


can taste it you can smell it you can feel it you can see it you can hear it it's easily assumed as fact because you see it you experience it beyond the five senses and what has experienced the we are the five senses can be assumed as fact but that's a choice when you realize that that was a projection from the subconscious materialized in the external world and you begin to question that and say wait a second I can now choose to assume that to be fact if I change the cause within what happens here's a circumstance here's a person


here's an environment here's how I feel about it here's the the chain of events leading up to it here's what happened what if I change the cause within about this what if I change the way I look at it what happens well when you change the way you look at things the things you look at change try it take a current circumstance take your current problem that you have and shift your perspective around see one of the things that I've learned on this journey of entrepreneurship is I have learned to be able to see opportunity where others can


only see problems and roadblocks and impossible I've trained myself to do it now I've been on this journey for 10 years 10 years as a full-time entrepreneur the repetition of being exposed through so many problems in my businesses in my clients businesses and working as a consultant and coach in working with many entrepreneurial organizations and companies of all different levels I've been exposed to so many problems and issues and solutions to that that my subconscious has found so many ways to solve problems to the point of


repetition where someone who is starting this journey brings to me their scope of problems and I can easily see solutions easily see the solutions what does that mean it means the solutions exist now these are valid viable solutions these are concrete solutions or things that I'm just making up but the difference between that entrepreneur and me is a perspective they don't see it as an opportunity they see it as a problem they see it as an impossibility the moment you see it as an impossibility it becomes so what you believe as far as


reality goals will be projected outside of you and materialized into form so one of the things we have to always remember is that the word impossible can limit us so Napoleon Hill had one set in thinking Grow Rich when my favorite quotes he said a great many years ago I purchased a fine dictionary the first thing I did with it was to turn to the word impossible and neatly clip it out of the book it would not be an unwise thing for you to do now why would he suggest something like this because the moment we have this impossibility idea show up


it is reflected in the external world on the based on what we're thinking about dealing with a person an environment or circumstance it will be reflected in the external world as fact of impossibility but in order to and think about your unlike don't just take my word for it you have experienced this is a beautiful thing I'm not talking about things that you don't know about you I've already experienced these things I'm talking about reflect upon a time in your life where others saw a situation as an impossibility and you saw it as a


possibility and you were the one who figured it out you learn how to create that where others could not see it it was because you first believed in your mind that it was possible and that projected out and reflected and materialized as the possibility that door showed up and you held the key with the thought the thought is the key that opens up the door so Napoleon Hill talks about the concept of the sixth sense and I refer to this as the superconscious the infinite intelligence conversation the universal mind now some might agree


with this and some might not agree with it I have a lot of reference experiences personally that I work with and I've talked to many others who work with this and I agree with what he says here he says the sixth sense is the portion of the subconscious mind which has been referred to as the creative imagination it has also been referred to as the receiving set through which ideas plans and thoughts flash into the mind the flashes are sometimes called hunches or inspirations the sixth sense defies description it cannot be described to a


person who has not mastered the other principles in the philosophy because such a person has no knowledge and no experience with which sixth sense may be compared understanding of the sixth sense comes only by meditating through mind development from within now this is why I've been pointing out a lot of videos about developing your mind in the Kabbalah and we say all is mine though all is mine and the universe is mental you have the solution to everything via your mind within the subconscious within the superconscious within infinite


intelligence we are the sixth sense we are the connection to the subconscious mind of other individuals via the mastermind again the mastermind individuals coming together in the spirit of harmony I'm not talking about groupthink where people come together and try to be right rather than focus on what is right rather than focusing on what is right they focus on who is right that would be groupthink there'll be egoic we're looking to believe in the possibility of the solution and saying as a collective coming together in a


mastermind we will figure it out together that thought goes into the minds of all individuals impresses itself in the subconscious mind goes to the superconscious mind and is brought forth adds the idea that hunch the inspiration to solve the problem or create the solution whatever it is now he says the sixth sense defies description it cannot be described to a person who has not mastered the other principles of the philosophy and I'll speak from experiences because I've been reading thinking Grow Rich since 2004


and I've used it many times to create multiple definite chief aims and I've achieved every single one of them and I have one right now and I will achieve it and each time I achieve my definite chief aim I realize the power of the sixth sense how so I'll tell you exactly how because during the time of in between of generating the definite chief aim and the manifestation of the definite chief aim I had no idea how I was gonna be brought forth but I will tell you this ideas hunches and inspiration showed up that redefine all


the things that I've ever learned logic and reason even and I followed those hunches and inspirations in some way somehow it brought forth and Steve Jobs sanity says you cannot connect the dots looking forward you can only connect them looking backwards even talks about trusting your intuition that's why he really recommends the book by Yogananda and so what I do so do I the autobiography of a yogi' you read a book like that and you will automatically realize the power that you have within you and it is through the


repetition of achieving your definite chief aims that you build a deeper connection with the sixth sense perhaps that's why a few people when you look at the grand scheme of things have a deep connection with the sixth sense maybe they have not honored the voice within and created a result of the definite chief aim which is a fragment of their true identity see I believe that the definite chief aim is part of a grand vision I believe that the definite chief aim is part of your true vision if by nurturing the definite chief aim that


you have right now and bringing it forth you have a greater sense of understanding and awareness of your overall mission and purpose in life one of the things that I found to be true is that by following the definite chief I realized that all is in harmony to contribute to the definite chief aim and that each element of the definite chief aim or each definite chief aim when completed builds my connection with the sixth sense and then reveals to me my next definite chief aim and each definite chief aim it's like a little


pixelated vision in each pixel shows up what the definite chief aim accomplished to reveal to me what my purpose is here on this planet and that's why I made that video the last video I made when I said your imagination is your vision it is your identity because I've I've realized this is that every time I had a definite chief aimed more and more of my mission my life purpose is revealed to itself revealed to me now I believe life has an interesting pattern when you accomplish something infinite intelligence $0.06 shows up with a


greater responsibility for you and this responsibility is to bring it forth heaven on earth Devine is always looking to manifest on earth that's why it's said as above so below and it is through this process that it's manifested you you are the one that brings it forth it is through you and I and everyone here that the vision of where we are going divine evolution is being brought forth if you look back humans have evolved and it is a net result of paying attention and honoring the vision each individual that has contributed had a definite


chief aim and by constantly identifying and honoring and seeing the definite chief aim all the way till completion you uncover more of your vision and you start to understand the sixth sense even more so number 10 all is part of the super conscious mind impressing the super conscious subconscious mind is done through the imagination so when you get your vision from the super conscious mind your definite chief aim you are to follow it all the way till completion and you can work with bringing forth you're definitely Fame via your


imagination it is via your imagination that the conscious the subconscious and the super conscious work in harmony the love within the subconscious mind is impressed and brought forth the conscious mind continues to facilitated guard and nurtured the subconscious mind and the ways that define or defy possibility are figured out by the super conscious number 11 optimal behaviors manifest automatically because we continuously support and encourage our imagination we also realize that our five senses are there to support our


sixth sense the imagination so for many years we could have been living in a world where we were run by our five senses and you could live that way if you want but when you work with this philosophy when you work through this process of identifying your vision and making this a continuum it's a continuous journey you realize that the five senses are here to support the vision of the sixth sense information pours through you like a conduit via the sixth sense through your imagination into the subconscious mind impresses the


subconscious mind the subconscious mind projects outwards and materializes into form that information is picked up we are the five senses and interpreted by the conscious mind and given meaning to to tune the like an instrument the subconscious mind to be in alignment with the vision and then send specific instructions back to the superconscious to repeat the process again and again over and over again that's the relationship of the lovers within but the primary living is through the imagination the sixth sense now that is


a person that is projecting outwards releasing that is a person that is said to have bring who brings the goods the table arrives with the full cup they're not looking to take because all is within they are looking to express and share and contribute and that's how a mastermind is created and that's how you had the button's consciousness and you can feel when you're around a person like that and my goal in this video is to help you unlock that because we all have that potentiality within us and it continuously gets


released more and more by doing the work by cleansing the subconscious mind from that what breaks the connection between the conscious and the subconscious and superconscious and when the superconscious is expressed through the suit through the subconscious in the external world that's where you get beautiful works of art that's where you get beautiful information insights that's where you get breakthroughs in technology breakthrough a breakthroughs in medical science breakthroughs in everything because you're working with


the spirit of harmony because you're working with imagination because you're releasing fears doubts and indecision on that what is your purpose in your vision see Napoleon Hill said this and I'm a huge fan of this in that chapter of outwitting the six ghosts of fear he said this and I'll pull it up here he said indecision is the seedling of fear remember this as you read indecision crystallizes into doubt and the to blend and become fear see in the last video when I talked about mental chemistry I recommend you watch that video and watch


it again and again and again because what I was really getting at over there is you don't want to wait till you fear you want to catch it at an indecision and doubt level before it crystallizes interfere you have the awareness and the power to capture it at that point because as soon as fear hits you that's when you risk drifting that's when you break the connection between the lovers within that's when you get in your head about it that's when you're gonna act from a scarcity perspective that is when you're not working in harmony to release


that's what's on your subconscious mind to help you solve the problem so catch it at an indecision and doubt level the moment you feel indecision you trust that the answers are within you you release it and you ask the subconscious mind you say subconscious mind you are the source of creation of all that exists because you are connected with the superconscious mind you are also a repository of the experiences I have had in my life and I realize consciously that you have far more experiences than I could consciously think of at this