Wrapping that oh, okay add wrapping with style whoops add wrapping with stuff run that again


Decorators - Intermediate Python Programming p.18

And now what you get is a beautifully wrapped up box of a new tesla p100 GPU


And then you could say like this now


horribly


And now it's a horribly wrapped up box of a beautifully wrapped up box of a new tesla p100 GPU


Okay, so I think that's enough on


Decorators, but there's obviously a lot of really cool things that you can do and it kind of gives you a lot more dynamic


functionality to your


functions


So it's actually pretty nice and also methods too


But I do like the added functionality basically if the do not repeat yourself that you can do with this and still the dynamicness


Because you can pass parameters


Obviously you don't have to just use strings you can use variables and all kinds of cool stuff


This is just meant to be a super simple example


But yeah pretty cool, so if you have questions comments concern something cool with decorators


Convolutional Neural Networks Basics - Deep Learning withTensorFlow 12

What is going on everybody and welcome to part 12 of our deep learning with neural networks,


tensorflow and python tutorial series.


In this tutorial what we're going to be talking about is the convolutional neural network


or CNN or ConvNet if you're a cool kid.


So the convolutional neural network is basically state of the art for recognizing what an image


is, what's in images or even playing roles in doing things like assigning captions for


images.


So the convolutional neural network breaks down into basically the following structure.


You have some input data and with that input data you're going to do some convolutions


or some convolving and that generally is just creating feature maps and we'll talk a little


bit more about that in a little bit but for now we'll just stick with convolve.


And then you're going to do some pooling, usually max pooling, again we're going to


talk a little bit more about that later on.


And each one of these you do is going to be a layer or a hidden layer better put actually.


So each one of those is your hidden layer and then generally you will add one fully


connected layer which would also be a hidden layer as well.


And then finally you will output whatever your output ought to be.


So that's kind of the idea.


So again you take your input data, you convolve it, you pool it, you usually will convolve


and pool again just because you're going to have two likely hidden layers at least to


start real basic two hidden layers.


Then you'll have a fully connected layer, a fully connected layer is identical to the


hidden layers of our typical just simple neural network that multilayer perceptron model.


And then finally the output again same as the output from a multilayer perceptron model.


So now let's talk about what convolve is and what pool is.


So when you do a convolution this is generally just any form of mapping or creating a feature


map from an original data set.


So generally what this is going to involve is it's just going to be this moving region,


this moving window over your image that is looking for something or in some way is going


to create a feature out of a window of pixels and some sort of end by end of pixels.


It's going to look for something or do something basically to take that region of pixels and


classify it or call it something.


So sometimes you might have this searching through an image looking for horizontal lines


or vertical lines or edges and so on.


So you might use something like OpenCV for example.


So to visualize this let's take an image and this will be a photo realistic image of a


cat.


Okay and the image in general you're going to break it down by pixel but I can't break


this image down by pixel so let's just pretend we're breaking this one down by pixel.


Good enough.


So each of these little squares is a pixel let's pretend.


So in the convolution step let's say you're going to take a 3 by 3 convolution.


So you might start with a window that is like this and that window is going to basically


pertain to a new, we'll call it cell and it will be that.


And then we're going to shift that window and we could shift it two spots over so we


could make that window here.


We could make that next window right here.


So there's a little bit of overlap there so we'll make that there just for kicks for now


and that would be here.


And now we've reached the edge and we're all set.


So then we would move the window down to that left where now the 3 by 3 is here and that


would correspond to this window.


I didn't draw the line of the blue one but I'm sure you get it.


And then finally our last 3 by 3 shifted over would be like this.


And that would correspond to this one.


Okay so we just keep doing that just to the point where we're not actually shifting any


pixels or rather not shifting to the point where we're not skipping any pixels.


So if this is your convolution what is pooling?


So let's say you've done some convolution and convolution has just produced values for


us.


So again when we convolved if you remember from a few seconds ago, if you can remember


that far long ago, we created new boxes and each of these new boxes or each of these new


kind of regions of the image, they might actually have values in them like 2 and 1 and 4 and


Something like that.


So let's imagine this is a new basically feature map and we'll just add some values here.


Okay so let's say this is that new feature map.


Now what we're going to do is we're going to move, we're going to do pooling and we'll


do a 3 by 3 pooling.


That'll extremely simplify this and make this quick.


So the pooling that we're going to do is a 3 by 3 and when we pool it's a little different


from when we created the feature map.


Yes when we created the feature map we did what pooling is actually going to do for us


but we also kind of extracted value whereas pooling is simply simplifying.


So let's take a 3 by 3 pool and this is going to be a max pool.


So we're looking for the max value.


That max value is a 4.


So in this case it would be a 4 and we'll just make a box around it I guess and then


we could shift it over 2 and then we've got this box.


The max value is a 3 and then we come down and here the max value is a 4 and then again


here the max value is a 3.


So it's an extremely simplified version of what pooling is but and we never had a value


here.


This is the great unknown value.


Anyway it's an extremely simplified version of what pooling does but that is what pooling


does and together convolution plus the pool equals a hidden layer.


And then we had that fully connected term at the end and that just means a fully connected


is just you've got neurons and they're just fully connected to each other.


We've kind of already spoken on that so hopefully you understood that already.


And then you finally have your output which we've also already covered and this is exactly


the same as before.


Let's say the first one's cat, dog, then horse.


So these might all connect to each other again.


So you've got cat, dog, horse.


Maybe this outputs a 5, this is a 2, and this is a 1.


So then of course the winner is cat and our hypothetical convolutional neural network


we are successful.


So again we start with input data.


We do our convolutions and pooling and together with convolution pooling you've got a hidden


layer.


You'll probably have two hidden layers at least in our basic example that we are actually


going to do in tensor flow you'll have two hidden layers.


So convolution pooling.


Then you'll have a fully connected layer and it's arguable whether or not this is actually


of use but people do it anyway so we're going to continue doing it.


And then you have the output which again is just the same as before.


So basically from fully connected to output it's the same as any other neural network.


So doing this in tensor flow is actually really simple.


We can slightly modify our initial code to work with this.


It's actually pretty impressive how easy it is given how different the model itself is


actually especially when you have something like sparsely connected layers and then just


pooling which is not quite something we've done before.


So anyways we're going to do that in tensor flow and see how just how simple that is and


we're going to do it against the MNIST dataset and using that script that we've been basically


using all the way up to this point.


The same one that we did with the simple multi-layer perceptron model.


The same one we did with the recurrent network, recurrent neural network that is and then


here with the con, the CONVENET.


So anyways questions comments leave them below otherwise as always thanks for watching thanks


for all the support and subscriptions.


$5 MILLION AI for FREE

Imagine an AI where all in the same model you could translate languages, write code, solve crossword puzzles, be a chatbot, and do a whole bunch of other crazy things.


$5 MILLION AI for FREE

This sort of an AI would certainly require a supercomputer of hundreds of A100 GPUs and months of training, even on all that power.


$5 MILLION AI for FREE

We would need a team of researchers, the best of the best.


$5 MILLION AI for FREE

We're talking about a project in the realm of say, 5 plus million dollars.


$5 MILLION AI for FREE

So when you're done, you probably want to keep it to yourself.


$5 MILLION AI for FREE

Maybe you'll sell access via an API, but definitely you can't share the actual model because of AI safety or something like that.


$5 MILLION AI for FREE

What if I told you though, for the last year, a group of over a thousand researchers has been quietly working on their own version of a


$5 MILLION AI for FREE

176 billion parameter model trained on the nuclear powered supercomputer, the Jonset, and is available now for you to download free of charge.


$5 MILLION AI for FREE

You can download multiple size variants all the way up to 176 billion parameters for free.


$5 MILLION AI for FREE

I'm talking about the BLUM model.


$5 MILLION AI for FREE

BLUM is an acronym that stands for Big Science, Large, Open Science, Open Access, Multilingual.


$5 MILLION AI for FREE

But for all intents and purposes, it's more like big language out of memory.


$5 MILLION AI for FREE

So just how much memory are we talking here?


$5 MILLION AI for FREE

Well, it's going to cost you about 680 gigabyte-ish of memory.


$5 MILLION AI for FREE

You can also run at half precision and use more like 350 gigabytes of memory with likely minimal performance difference in terms of the output quality.


$5 MILLION AI for FREE

But either way, a lot of memory.


$5 MILLION AI for FREE

So I know there are probably like a couple of you guys out there who don't have a Puget workstation with a terabyte of RAM.


$5 MILLION AI for FREE

So what can you do?


$5 MILLION AI for FREE

Well, it turns out that Hugging Face has your back.


$5 MILLION AI for FREE

Not only have they facilitated to a large extent this model's very training and existence, as well as making the model available for you to download for free,


$5 MILLION AI for FREE

they're also hosting the inference of this model via their API at the very fair price of free.


$5 MILLION AI for FREE

This API is also GPU hosted.


$5 MILLION AI for FREE

It's on A100s.


$5 MILLION AI for FREE

So your inference times should be 50 to 100 times faster than what you'd be seeing inferencing this model locally with RAM anyways.


$5 MILLION AI for FREE

There is a queue, however.


$5 MILLION AI for FREE

So if you do have your local A100 compute cluster available, you could use that instead.


$5 MILLION AI for FREE

Besides all of these options, you don't actually have to run the full 176 billion parameter model.


$5 MILLION AI for FREE

Bloom has many sizes from a mere 350 million parameters to one bill, two bill and six billion.


$5 MILLION AI for FREE

For many tasks, you'll probably find that these smaller models are sufficient.


$5 MILLION AI for FREE

All of the examples mentioned in the intro are real, actual, non-cherry picked outputs from the Bloom model,


$5 MILLION AI for FREE

which is what we would call a large language model or an LLM.


$5 MILLION AI for FREE

This style model, while not entirely new, is fairly novel and very unique in how we as deep learning and just programming engineers might interact with it.


$5 MILLION AI for FREE

GPT-3 is another large language model with a very similar size, 175 billion parameters, which came out about two years ago.


$5 MILLION AI for FREE

We're still learning new things that GPT-3 and large language models in general can do, but we've learned quite a bit.


$5 MILLION AI for FREE

It's also just been the case that these really massive models have only been available and accessible to a pretty small subset of developers


$5 MILLION AI for FREE

and developers already make up a tiny subset of people in general who might come up with like unique ways to leverage technology like this.


$5 MILLION AI for FREE

Your typical free range human has absolutely no idea that models like this exist, nor how they really work at all or could be used.


$5 MILLION AI for FREE

But even deep learning engineers struggle with this new sort of paradigm of prompt engineering.


$5 MILLION AI for FREE

And part of what makes this hard is many of the tasks that someone might try to get this model to do and that it can do are not what this model actually is doing.


$5 MILLION AI for FREE

So, for example, you could use this model for a regression type task and generate a scalar sentiment score for text.


$5 MILLION AI for FREE

But large language models aren't regressors, at least not this one.


$5 MILLION AI for FREE

You could use this model to be a chatbot.


$5 MILLION AI for FREE

But these models are not sequence to sequence models that generate a response to some input.


$5 MILLION AI for FREE

In very simplistic words, large language models just generate continued text based on the input text.


$5 MILLION AI for FREE

They're predicting basically what words will come next based on the words that they've seen so far.


$5 MILLION AI for FREE

Based on this capability, we can what's called prompt the model to generate outputs that actually are doing the task that we want them to do.


$5 MILLION AI for FREE

But again, this is all just the model attempting to continue a string of text based on probabilities of what it thinks should come next.


$5 MILLION AI for FREE

And this distinct difference in how we interact with these models and get them to do what we actually want to do is what makes prompt engineering kind of its own field entirely.


$5 MILLION AI for FREE

In fact, I kind of think especially if things keep going the way that they're going, you will have an entire subsection and field of people who aren't even programmers


$5 MILLION AI for FREE

who are prompt engineers, and that's their job.


$5 MILLION AI for FREE

But what this means is if you want Bloom or a large language model to behave like a chatbot,


$5 MILLION AI for FREE

your prompt needs to contain structure that suggests it's something like a chat dialogue.


$5 MILLION AI for FREE

So maybe you have something like a speaker name, colon, some text, a new line, and then another speaker, a colon, and then nothing.


$5 MILLION AI for FREE

This will almost certainly elicit a continuation from the large language model that mimics chatbot text.


$5 MILLION AI for FREE

But the large language model, it's not a chatbot.


$5 MILLION AI for FREE

It's just generating a continuation of the text that you put in, which also means it can, in general, generate more than just one speaker's response.


$5 MILLION AI for FREE

You can almost expect that it will.


$5 MILLION AI for FREE

This means you'll often see in an example like this that it might actually just generate quite a bit more of that conversation rather than just a simple response.


$5 MILLION AI for FREE

If you wanted to make an actual chatbot application from here, you'd wind up coding some logic to stop at a new line or the next speaker and so on.


$5 MILLION AI for FREE

So how does the model actually know what's next?


$5 MILLION AI for FREE

Basically, it's seen all of the text.


$5 MILLION AI for FREE

Bloom specifically has been trained on a total of 1.6 terabytes of text data, which contains 46 natural languages and 13 programming languages.


$5 MILLION AI for FREE

The distribution of which you can see here.


$5 MILLION AI for FREE

And for programming languages specifically, here's their distribution.


$5 MILLION AI for FREE

We can peruse this data set by visiting one of the hugging face spaces that illustrates the corpus in a sort of map and interactive form.


$5 MILLION AI for FREE

You can click through and then find any specific data set on the corpus page.


$5 MILLION AI for FREE

From here, you can usually click through and see what that corpus contains and even kind of see some of the text examples.


$5 MILLION AI for FREE

In this case, we have topics ranging from medicine, chemistry, engineering, psychology, economics to art, history, philosophy and more.


$5 MILLION AI for FREE

And this is just one of the data sets in one of the categories under one of the languages of the entire data set that this Bloom model was trained on.


$5 MILLION AI for FREE

Like I said, basically all the text.


$5 MILLION AI for FREE

But it's a good idea to have an understanding of this text to some extent so you understand because your prompts should look something like some of the text that the model has seen or a combination of them.


$5 MILLION AI for FREE

So, for example, within the code category, we have both GitHub as well as Stack Exchange.


$5 MILLION AI for FREE

And this can influence how we might interact with this model if the model thinks we're producing text that looks more like Stack Overflow versus maybe GitHub.


$5 MILLION AI for FREE

So on GitHub, the text that you might find is generally going to be finished, working, acceptable code and it will be mostly code.


$5 MILLION AI for FREE

On Stack Exchange, often the first snippet of code is actually, it has some sort of flaw or issue with it.


$5 MILLION AI for FREE

And there can be as much as 50% or more is actually natural language text, not actual code.


$5 MILLION AI for FREE

This can either bite us in the foot or we can use this to our advantage when we're prompting to get the type of output that we're actually wanting.


$5 MILLION AI for FREE

All of these large language models also have parameters that can be tweaked.


$5 MILLION AI for FREE

Things like how long of a generation do you want to make?


$5 MILLION AI for FREE

Temperature, which is a degree of creativity and diversity in token choice.


$5 MILLION AI for FREE

There's also top P, another parameter to influence the pool of possible tokens to consider and so on.


$5 MILLION AI for FREE

You might be wondering, what are tokens?


$5 MILLION AI for FREE

Essentially, tokens are what you get when you convert some string of text to an array of values.


$5 MILLION AI for FREE

This blue model uses a byte level byte pair encoder for tokenization.


$5 MILLION AI for FREE

What this means essentially is every possible character is covered.


$5 MILLION AI for FREE

But the total vocabulary size is 250,680, which is way more tokens than you would need for full vocab coverage of a just character level model.


$5 MILLION AI for FREE

So not only do we have character only, but we also have many combinations, including words, words plus punctuation and so on.


$5 MILLION AI for FREE

So all of that is included here.


$5 MILLION AI for FREE

The benefit here is no matter what, you'll never see what was once the dreaded unk or unknown token.


$5 MILLION AI for FREE

You will always get valid tokens because the actual vocab that the model has, there's no word that you could produce that is not covered already.


$5 MILLION AI for FREE

So what this means is you can do things like make up new words to feed to the model.


$5 MILLION AI for FREE

And the model can actually not only understand those new words, that model can plausibly create new words to return back to you.


$5 MILLION AI for FREE

So when we see the word token, just think of this as a numerical ID that is assigned to a bit of text.


$5 MILLION AI for FREE

And these are numbers that the machine learns to represent as that bit of text.


$5 MILLION AI for FREE

To further solidify this, let's peek at this just very briefly.


$5 MILLION AI for FREE

So with the example text, the blue model is very interesting.


$5 MILLION AI for FREE

We can see how this is tokenized by passing this to the tokenizer for the bloom model.


$5 MILLION AI for FREE

Here we can see that we get returned a vector of input IDs.


$5 MILLION AI for FREE

Sometimes these IDs are just a word.


$5 MILLION AI for FREE

Sometimes they're some of the letters of the word.


$5 MILLION AI for FREE

And sometimes they're a combination of words, spaces, and other grammatical elements.


$5 MILLION AI for FREE

For example, notice ID 34495.


$5 MILLION AI for FREE

This word is interesting with a leading space.


$5 MILLION AI for FREE

This means the interesting in this case is probably not the beginning of a quote, probably not the beginning of a sentence, etc.


$5 MILLION AI for FREE

We can even take this a step further and just tokenize the lower case word interesting.


$5 MILLION AI for FREE

We can see here that actually this tokenizes into two tokens, one being enter and the other being esting.


$5 MILLION AI for FREE

We can also try tokenizing some random sequence of characters.


$5 MILLION AI for FREE

In this case, we find that the first ID 208032 is the lowercase s with an asterisk.


$5 MILLION AI for FREE

And then we can see ID 42 is a capital G.


$5 MILLION AI for FREE

Here's the tokenization and their representations for the bloom acronym.


$5 MILLION AI for FREE

So these IDs and tokens, these are what models like bloom are generating and what they're fed.


$5 MILLION AI for FREE

And all they really do is just predict the next best token given some input sequence.


$5 MILLION AI for FREE

If we know we want bloom to finish a sentence, for example, we might just ask for only 10 or 20 more tokens and then trim the end if there's extra.


$5 MILLION AI for FREE

If we want an entire paragraph, then maybe we ask more like for 500 tokens.


$5 MILLION AI for FREE

So to begin, let's imagine we want bloom to show us how to do some coding in open CV in Python.


$5 MILLION AI for FREE

We might try a prompt like use open CV in Python, but the output of that is just a continuation of natural language.


$5 MILLION AI for FREE

It looks kind of like maybe it came from Stack Overflow.


$5 MILLION AI for FREE

It sounds like someone who is just outlining their goals and dreams of using open CV, but that's not what we wanted.


$5 MILLION AI for FREE

We can sympathize with that, but we want the code as programmers and probably as many of you have done with GitHub Copilot.


$5 MILLION AI for FREE

You probably know what needs to be done.


$5 MILLION AI for FREE

We need to add a hashtag or a pound sign and a space to the beginning of this text.


$5 MILLION AI for FREE

In this case, we can see that bloom produces code and well commented code at that that illustrates to us some of the basics of using open CV, at least to start.


$5 MILLION AI for FREE

Now, why did this work?


$5 MILLION AI for FREE

We wanted bloom to generate some code for us, but to do that, we needed to structure the prompt to be like some of the code bloom has seen,


$5 MILLION AI for FREE

rather than like the text bloom has seen where people are simply talking about code or maybe about modules that might be used with that code.


$5 MILLION AI for FREE

So bloom doesn't code, but bloom can continue to generate text that looks like code that it's seen before, and it's seen a lot of code.


$5 MILLION AI for FREE

Bloom also isn't a chatbot, so treating bloom immediately like it's a chatbot and issuing a prompt like what should I do today merely produces text that continues this,


$5 MILLION AI for FREE

and it really appears more like the original speaker is just continuing their speech, but that's not what we wanted.


$5 MILLION AI for FREE

We wanted a reply.


$5 MILLION AI for FREE

So to make bloom behave more like a chatbot, what we need to do is rethink that prompt.


$5 MILLION AI for FREE

What we need to do is prompt bloom with text that's structured like a chat transcript, something like person colon, what should I do today, new line, bot, colon, then nothing.


$5 MILLION AI for FREE

This will very strongly encourage bloom to produce a continuation of essentially a transcript, and it does.


$5 MILLION AI for FREE

So again, this isn't a chatbot.


$5 MILLION AI for FREE

Instead, it's more like a transcript generator, but if you wanted to, you could make it a chatbot.


$5 MILLION AI for FREE

All you would need to do is just add some logic on top.


$5 MILLION AI for FREE

So for example, you might just go until the next new line character, or maybe you use a regular expression to identify a name, colon, space, you know, identification for the next speaker.


$5 MILLION AI for FREE

So again, so far we've seen that this is just a generative model, but through prompt engineering, we can influence this model to do and produce things in a desired way,


$5 MILLION AI for FREE

and then we've also seen how we can incorrectly attempt to prompt it and how we can go about trying to fix that.


$5 MILLION AI for FREE

It turns out that we can actually influence the output even more.


$5 MILLION AI for FREE

Let's continue with this chatbot idea.


$5 MILLION AI for FREE

So what if we wanted to give the chatbot a bit of character?


$5 MILLION AI for FREE

So we might try a prompt like this, where we prime it a bit by suggesting this is an argumentative chatbot, and then we'll call it arguably.


$5 MILLION AI for FREE

Upon us asking how arguably is doing, we see arguably responds with, I'm fine. Thank you. How are you?


$5 MILLION AI for FREE

This is absolutely unacceptable.


$5 MILLION AI for FREE

In this case, it appears the zero shot example isn't going to work, and we might need to prime it a bit with an example reply.


$5 MILLION AI for FREE

In this case, we include a sassy reply example in the prompt before feeding in the real question about hobbies and prompting the argue bot to reply.


$5 MILLION AI for FREE

In this case, when we ask if argue bot has any hobbies, it's at least a negative reply, but I'd like something better than this.


$5 MILLION AI for FREE

Maybe argumentative and argue aren't the right words.


$5 MILLION AI for FREE

We could try mean, for example, and this seems to be the same as argue bot.


$5 MILLION AI for FREE

Let's try rude bot.


$5 MILLION AI for FREE

I like this response a whole lot more.


$5 MILLION AI for FREE

This is more in line with what I was going for, and let's see if we can keep it going.


$5 MILLION AI for FREE

And now I like it even more.


$5 MILLION AI for FREE

And the reason why this worked, again, is these models continue generating new tokens based on probabilities.


$5 MILLION AI for FREE

Sometimes all you need to do is maybe change a word to another synonym of that word, and now you've opened up a new set of probabilities.


$5 MILLION AI for FREE

So all of this so far is really cool, and I would like to dive more deeply into the chat bot aspect of things specifically,


$5 MILLION AI for FREE

as well as other attributes and capabilities of this model much more specifically.


$5 MILLION AI for FREE

And that's probably its own entire video on the side or multiple videos, so maybe more on that later.


$5 MILLION AI for FREE

Comment below if you want to see more of that or some specific example with me diving a little deeper.


$5 MILLION AI for FREE

But instead, let's focus on some other interesting examples of what these generative models can do.


$5 MILLION AI for FREE

Because so far we've seen it code, we've seen it be a chat bot, but these models can actually do quite a bit more advanced things.


$5 MILLION AI for FREE

So far we've just seen things in a very linear fashion as far as how it's generating its output, but let's see something else.


$5 MILLION AI for FREE

So imagine you're balling out of control, you're dismantling historical bridges to move your mega yacht through,


$5 MILLION AI for FREE

and you're launching into space for tourism, but on the side you also run the largest online marketplace,


$5 MILLION AI for FREE

and you want to do analytics on user reviews of products on your website.


$5 MILLION AI for FREE

So I've grabbed here a random review from a product on your website, and what I want to do is boil this review down to the meaningful things,


$5 MILLION AI for FREE

because this person talked way too much, I don't have time for this.


$5 MILLION AI for FREE

So maybe what we want to do is categorize it or figure out what is it a review of.


$5 MILLION AI for FREE

We probably want to know if it's a good or bad review, and then maybe we want to briefly summarize it.


$5 MILLION AI for FREE

Time is money.


$5 MILLION AI for FREE

So you might think you need to do that like one at a time, one prompt at a time,


$5 MILLION AI for FREE

but you can actually prompt multiple responses from a large language model like Bloom, and it can be done fairly nonlinearly.


$5 MILLION AI for FREE

So for example, you can form the prompt to contain a review, a numbered list of questions about that review,


$5 MILLION AI for FREE

and then start the answers with the one.


$5 MILLION AI for FREE

So you're prompting Bloom to go up to that number one question, answer it, and then what would be the most likely next thing that you would generate?


$5 MILLION AI for FREE

Well, it's going to be an answer to the second question.


$5 MILLION AI for FREE

And running this, we can see that Bloom understands this is a review of a recliner.


$5 MILLION AI for FREE

The review is overall positive, and it can even pull out the meaningful aspects of the review and shorten it considerably.


$5 MILLION AI for FREE

And it did all that in one fell swoop, essentially performing three tasks for us.


$5 MILLION AI for FREE

And so not only did it know how to answer those questions,


$5 MILLION AI for FREE

it also knew how to continue that kind of structure of our prompt where it was like this kind of questionnaire almost structure.


$5 MILLION AI for FREE

And again, this model is not trained to do this.


$5 MILLION AI for FREE

I'm sure there are some samples like this in the entire data set,


$5 MILLION AI for FREE

but this is by no means even, you know, mostly what this model was trained to do.


$5 MILLION AI for FREE

All it does is predict the next N tokens.


$5 MILLION AI for FREE

It's through prompt engineering that we can elicit such powerful behaviors.


$5 MILLION AI for FREE

And this is thanks to the large language models, just massive and general understanding of language.


$5 MILLION AI for FREE

And that's not just the English language.


$5 MILLION AI for FREE

Like I showed before, this is all language, just language.


$5 MILLION AI for FREE

It's just a large language model.


$5 MILLION AI for FREE

Another example of this sort of behavior could be something like code error diagnosis and proposed solutions.


$5 MILLION AI for FREE

So for example, let's take an error that most of you will be seeing if you attempt to run Bloom.


$5 MILLION AI for FREE

We can then prompt Bloom like we did before to answer two questions.


$5 MILLION AI for FREE

What causes this error? What does it mean? And how might we fix this?


$5 MILLION AI for FREE

The output is accurate and we could probably engineer this prompt even further to be more specific


$5 MILLION AI for FREE

and maybe make multiple suggestions for what might work and how to fix it.


$5 MILLION AI for FREE

But one thing I really want to stress is all of the examples,


$5 MILLION AI for FREE

except for the chatbot example, but I showed you how I worked through that.


$5 MILLION AI for FREE

None of these are cherry picked examples.


$5 MILLION AI for FREE

These are all examples that I did on my first attempt.


$5 MILLION AI for FREE

And like, not gonna lie, the review one really just blew my mind.


$5 MILLION AI for FREE

I just sometimes you do stuff with AI and like you see the result


$5 MILLION AI for FREE

and you just got to get up and walk away from the computer.


$5 MILLION AI for FREE

Like it's just so mind blowing.


$5 MILLION AI for FREE

And that is that's what I just had to walk away when I saw the review, the review one.


$5 MILLION AI for FREE

And again, these aren't cherry picked.


$5 MILLION AI for FREE

Like this model is just insanely powerful.


$5 MILLION AI for FREE

So how about something a bit more abstract?


$5 MILLION AI for FREE

What if we not only want to have a conversation like a chatbot,


$5 MILLION AI for FREE

but we want one person speaking one language and another speaking another language.


$5 MILLION AI for FREE

Here we have person one speaking English and person two replying and speaking Spanish.


$5 MILLION AI for FREE

Despite the two different languages, when translated, at least for me,


$5 MILLION AI for FREE

the conversation makes total sense and is correct as far as I can tell.


$5 MILLION AI for FREE

This is a pretty abstract task that requires some understanding of both conversation


$5 MILLION AI for FREE

and a sort of like global language understanding,


$5 MILLION AI for FREE

which is really interesting to see it actually doing it so well.


$5 MILLION AI for FREE

I might be wrong, but I really doubt something like this exists in the data set.


$5 MILLION AI for FREE

And if it did, it would be unbelievably rare and it couldn't possibly have influenced the model


$5 MILLION AI for FREE

to have some sort of talk in two different languages aspect of some neurons saved in the back pocket.


$5 MILLION AI for FREE

Like that's just it can't be.


$5 MILLION AI for FREE

It's just incredible.


$5 MILLION AI for FREE

OK, so we've seen a lot of examples of what large language models can do.


$5 MILLION AI for FREE

But it's my best guess that we've only just begun to scratch the surface,


$5 MILLION AI for FREE

especially with layering large language models with a bit of logic,


$5 MILLION AI for FREE

doing some more of these like nonlinear tasks or even multiple large language model tasks


$5 MILLION AI for FREE

on top of each other or combining large language models and other models.


$5 MILLION AI for FREE

So there have even been examples of attaching a frozen large language model to an image generation model


$5 MILLION AI for FREE

and then doing things like text to image with pretty good results.


$5 MILLION AI for FREE

I think we'll wind up seeing way more possibilities as more people who aren't even necessarily programmers


$5 MILLION AI for FREE

getting their hands on models like this.


$5 MILLION AI for FREE

It's no surprise that one of the first real tools in the wild powered by large language models


$5 MILLION AI for FREE

is a code assistant GitHub Copilot.


$5 MILLION AI for FREE

Don't get me wrong, it's an amazing application.


$5 MILLION AI for FREE

I love it.


$5 MILLION AI for FREE

It has made my life way better.


$5 MILLION AI for FREE

But we as programmers are really biased when it comes to trying to think of ways to use technology like this.


$5 MILLION AI for FREE

It was always the first thing I tried when dealing with smaller transformers.


$5 MILLION AI for FREE

And even before transformers, some of these earlier sequence to sequence models,


$5 MILLION AI for FREE

you can find the videos on my channel.


$5 MILLION AI for FREE

That was always the thing.


$5 MILLION AI for FREE

I was always like, oh, let's let's get it to do this thing.


$5 MILLION AI for FREE

And I know I wasn't alone.


$5 MILLION AI for FREE

It was just the thing I wanted in my life.


$5 MILLION AI for FREE

It was the obvious thing.


$5 MILLION AI for FREE

It was the first thing I could think of.


$5 MILLION AI for FREE

And just imagine what happens when doctors, lawyers, copy editors, authors, et cetera,


$5 MILLION AI for FREE

all the careers get their hands and get their hands on this kind of technology


$5 MILLION AI for FREE

and start thinking of ways that it could help them and make their life better.


$5 MILLION AI for FREE

In this way, I acknowledge you have to acknowledge open AI for leading the way in the R&D of models like this


$5 MILLION AI for FREE

and taking those first steps and leaps in developing large language models.


$5 MILLION AI for FREE

And then other companies like a Luther AI who have historically contributed the largest,


$5 MILLION AI for FREE

truly open source language models and now hugging face and the entire big science team just pulling a project like this off,


$5 MILLION AI for FREE

along, of course, with the French government supplying their Jansay supercomputer through grants


$5 MILLION AI for FREE

and just everyone else involved here.


$5 MILLION AI for FREE

This was a massive project.


$5 MILLION AI for FREE

And just like what an incredible time, you know, to be alive where anyone like anyone,


$5 MILLION AI for FREE

first of all, can access that free API and just tinker around with this incredible large language model.


$5 MILLION AI for FREE

And not only that, you can just you could just download it.


$5 MILLION AI for FREE

You can just download a five million dollar AI.


$5 MILLION AI for FREE

It's just incredible.


$5 MILLION AI for FREE

So I encourage you if you can run it or you have a way to run it, go ahead and download it, play with it locally.


$5 MILLION AI for FREE

It's so cool with the full context.


$5 MILLION AI for FREE

And if not, you can use that free API.


$5 MILLION AI for FREE

And then also, if not, there's lots of options in the cloud.


$5 MILLION AI for FREE

I wouldn't be surprised if before long, if not already when I'm recording this,


$5 MILLION AI for FREE

somebody also has it running live in the cloud somewhere.


$5 MILLION AI for FREE

So that's the brilliant thing of somebody really just training and releasing this model.


$5 MILLION AI for FREE

Anybody can share it and continue to share it and play with it and learn with it.


$5 MILLION AI for FREE

And I can't wait to see the cool things that will come out from models like this just being free and open like this.


$5 MILLION AI for FREE

I mean, it's just it really is incredible.


$5 MILLION AI for FREE

And I don't know of many fields where things like this are really, truly happening,


$5 MILLION AI for FREE

where that technology that someone has poured all this time, energy and money into is then just released.


$5 MILLION AI for FREE

So very exciting times.


$5 MILLION AI for FREE

Thank you to everyone involved in releasing this model.


$5 MILLION AI for FREE

I'm having a blast playing with it.


$5 MILLION AI for FREE

If you've ever wondered and wanted to know more about how neural networks actually work,


$5 MILLION AI for FREE

including the optimization and fitment rather than just simply calling some method,


$5 MILLION AI for FREE

then you might be interested in checking out the Neural Networks From Scratch book by myself and Daniel Kukewa.


$5 MILLION AI for FREE

The book can be had in the form of an ebook PDF, softcover or hardcover and we ship for free worldwide.


$5 MILLION AI for FREE

Also, the physical books just come with an ebook copy.


$5 MILLION AI for FREE

All copies are in full color, which helps because there's a lot of code syntax highlighting


$5 MILLION AI for FREE

and lots of charts and diagrams to help convey the principles.


$5 MILLION AI for FREE

Also, almost all of those charts and diagrams have QR codes and links that take you to animations to help further illustrate the concepts.


$5 MILLION AI for FREE

This is truly a real neural networks from scratch,


$5 MILLION AI for FREE

teaching everything from the forward pass, calculation of loss, back propagation and optimization.


$5 MILLION AI for FREE

The only math that you're expected to know coming in is basic algebra.


$5 MILLION AI for FREE

The rest is taught by us in the book step by step.


$5 MILLION AI for FREE

Everything is shown and explained in the book first logically, then mathematically,


$5 MILLION AI for FREE

then via raw Python code, no third-party libraries, and then finally optimized via NumPy.


$5 MILLION AI for FREE

And this is for every step of the way, building and training actual neural networks


$5 MILLION AI for FREE

for a fully fundamental understanding of neural networks and how they work from scratch.


$5 MILLION AI for FREE

If at any point you're lost or confused, all copies of the book also grant access to a Google Docs version of the book


$5 MILLION AI for FREE

where you can ask your questions in line with the actual text itself.


$5 MILLION AI for FREE

This is an incredible project that I'm extremely proud of to share with you all.


$5 MILLION AI for FREE

We've sold over 13,000 copies to students all over the world.


$5 MILLION AI for FREE

If you're looking to take your knowledge of deep learning to the next level,


$5 MILLION AI for FREE

or if you're just looking to start that journey, I can't imagine a better way.


$5 MILLION AI for FREE

So to learn more and buy yourself a copy, you can head to nnfs.io.


$5 MILLION AI for FREE


Game Development in Python 3 With PyGame - 20 - Sounds and Music

Hey everybody, what is going on? Welcome to another Pygame tutorial video. In this video


we're going to be talking about sounds and music with Pygame. So what we want to do is


add maybe some background music to our game and then also some sounds, especially when


we crash our car there should be some sort of sound that sounds like a crash. So there's


a lot of places where you can get free music either online or in sound effects, stuff like


that. Also YouTube just has an audio library that you can use to put stuff in your videos.


So I'm just going to use that but just keep in mind you probably can't download this and


then distribute a game with that, like with this audio. So just keep that in mind, especially


if you're charging for the game. So anyway, we're just going to use this as an example.


So for music we can just search around but personally I'm just going to pick jazz and


then I just went ahead with jazz and pairs. So we get that and then also I already have


that. And then sounds I would go just type in crash for example, that's what we're looking


for. And this is what we went with, was this crash right here. Can't hear it very well,


let me turn off my sound. There we go, hopefully that came through okay. So that's the other


one I grabbed. Now these are in MP3 format and Pygame does support MP3 as well as like


OGG sometimes but not the best. The best solution is to put it in a WAV format. So if you have


say any sort of movie maker generally also will let you convert into MP3, WAV, anything


like that. There's probably some sort of audio converter too if you happen to have one. Also


if you just record on your desktop, like a lot of like just desktop recording applications


that come with your computer default into WAV format. I use Sony Vegas to convert it


to WAV so if you have Vegas you can use that too. Anyway you can also just download some


WAV formats offline. So once you have them, you'll want them at least for the purpose


of this tutorial in the same directory as your script otherwise you'll need to give


the full path. We're going to assume it's in the directory of this script which it is


so I'm not going to specify the full path. So you can basically load in these sounds


anywhere. I'm going to go ahead and load it in for right here in Pygame under this Pygame


init and for example we can load in things like crash underscore sound and crash sound


is going to equal Pygame dot mixer dot sound and actually that's capital S which is really


weird. I don't understand why this is a capital S because everything else is under case but


sound is capital S. Anyway, sound and then we'll do this sound is crash dot WAV for me.


So that's the sound file I have. So now we have this crash sound in our memory and then


there's another one for example if you want to do music you would do something like let's


just say we're going to call this well with music it's a little different so with sounds


you'll load in like ten different sounds maybe for your game. For music as far as I know


you have to like you probably could save it to like an object or something but for the


most part music is just one thing. Now you can have like a queue of music so you could


have a different tune per level or something like that. I'm not going to get into that


now so for now we're just going to load one thing into the music and so that's going to


be Pygame dot mixer dot music dot load and then we're going to load jazz dot WAV. Anybody


has an idea on why this is specifically a capital S but like this isn't a capital M


or this isn't let me know I just want to know the thinking like what I'm missing from pep


eight is why we would capitalize that. Anyway that loads the music but it doesn't play it


no different than this loads the sound but it does not play it. So where do we want the


crash sound to actually occur? Well chances are we want that to occur when we actually


have a crash so we'll come down to crash and in here we would say we want two things right


we want the music to stop playing if it's playing so we're going to play music in the


background but if we crash we stop the music so we would do Pygame dot mixer dot music


dot stop again it's not a capital S and then we would want to play Pygame dot mixer dot


sound dot play and then what do we want to play? We want to play the crash sound so here


we're actually going to play it and the last thing that we need to do is we need to go


ahead and have we need to have the music playing in our game loop so we'll come down to the


game loop here and we need to bring in the music bring in the music so Pygame dot mixer


dot music dot play and then here you can put anything like you could put like a one and


this will play the music once and then again once so if you put five it's going to play


the music six times and if you put negative one it will play indefinitely like it'll just


keep looping so that's what we want we just want it to loop we're not going to make it


very far anyways this game is a slow game but each time the game loop runs it's going


to start over the music and then when you crash it stops it which here so it stops the


music now there's another thing that we might want to add and that is in pause so if we


come to our pause function when we pause the game generally you pause because maybe someone


comes in the room and asks you a question or the phone rings or whatever you want to


press pause and you probably don't want the music still playing while you've pressed pause


so we can stop the music or actually we can pause the music so it doesn't start over but


it pauses it so we can do pygame.mixer.music.pause and then when we're done we need to stop the


pause right so when do we stop the pause well whenever we call button and the function this


button runs is unpause so now we need to find our unpause function which is right here and


we global pause and then we are turning it to pause equals false well now we just do


pygame.mixer.music. and it just so happens this function is also called unpause so that


just worked out so that should be it so let's go ahead and save and run this see what we


get so here's our game awesome we hit go and here comes our music wow it's really loud


okay so we're playing i'm gonna try hit a high score here and you crash you die so anyways


you guys get the point you got the music sound effects and all that obviously you can add


a lot more sound effects you can maybe pick a better track or something but i thought


that was actually an okay track for the game so anyway hope you guys enjoyed if you have


any questions or comments please feel free to leave them below otherwise as always thanks


Topographic VAEs learn Equivariant Capsules (Machine Learning Research Paper Explained)

Hello there, today we'll look at topographic VAE's learn equivariant capsules by T. Anderson


Keller and Max Welling. On a high level, this paper proposes a new type of variational autoencoder,


where the latent variables aren't independent, but are organized in a topographic way. Now what


that means, we're going to look at that, but in essence, it means that it can, it can, it can


represent transformations in the real world of a certain kind, as transformations inside of the


latent space of the model. So the whole question is here, how do we build a latent space and a model


where this naturally happens as we train it. So we want the real world to somehow correspond to the


latent space in a way such that if the real world moves, the latent space moves it equivalently,


or equivariantly, that's where this word is going to come in. So we're going to go through the paper,


I have to say, I don't understand this fully as well, these variational frameworks, they are


always kind of, I feel kind of math heavy, and they take a very different approach than the papers I


might be used to. So I'm going to tell you what I think is going on here. And if I'm completely


wrong, this is entirely possible, please let me know. Alright, let's dive into the paper.


This is the first graphic right here that shows kind of an overview over the system. So what do


they want to achieve? What they say is we're not going to consider, we're going to try to build a


generative model like a variational autoencoder, but we're not going to consider any kind of data,


we're going to consider data, essentially, essentially frames of a video. So we're going to


assume that what we're looking at is kind of a video. And the transition in the transitions


inside the video are sort of continuous, sort of monotonic, and, and, and, and slow. So here,


you can see the seven rotates slowly, and also changes its color slowly, relatively monotonously


over this sequence. So what they're going to say is, we're gonna, our model is going to take


this entire sequence, one of picture is going to be kind of the focus here. So this green one is


the focus, but we're going to take in this entire sequence right here into the model. And we want


the model to come up with a latent representation of the focus image. In this case, it's going to be


we'll jump a step here is going to be this thing right here. Let's call that I don't even remember


how they call it. Let's call it like Z hat. Okay, this is a latent representation of the focus image.


And now, obviously, in a regular variational auto encoder, I could now push this again into the


decoder and get back the same image and I can do so here as well. However, we want something else


as well. We also want that if I now transform my latent space in a certain way, and this way is


going to be this role operation in this paper, if I transform my latent space in this way, I want


this to correspond to moving forward in this sequence, right? So I have a sequence as an input.


And I say, well, my latent space should be such that if I perform certain operations,


right here, in this case, I roll by 10. That that corresponds not to the picture that I've input,


but to the picture that would be if I were to observe this transition 10 steps into the future.


So roll by 10. And roll in this case means you can see here, they have two of these


what they call, you know, capsules, I think they call them capsules, the left one and the right one.


And the role simply means that I take every variable latent variable, and I simply roll them


forward. So this is over the latent dimension, I just roll them forward by one step, I do that 10


times this is as you can see, this is arranged in sort of a torus here in a 1d torus. So I can just


roll this around. And also this capsule, I can just roll it around 10 times. And that hopefully,


if we train the model correctly, should correspond to not the input image, but the image that is 10


steps into the future. Okay, so that is the goal. Now, we don't want to train a model explicitly to


predict 10 steps into the future, that will be will be a valid task. But it's not what this model


wants. What this model wants is, say, can we build a model architecture and the latent space


architecture such that this is kind of happens automatically. And let's see, well, you can


already see kind of how this latent space comes to be I said this Z hat here is going to be the


latent representation, you can see that is not the thing that is directly output by the encoder,


the encoder in this case outputs many things. So it outputs a Z variable. So the Z hat is what I


call kind of Z normalized, the Z variable is kind of Z on normalized, so outputs a Z variable for


the focus image, but it also outputs these u squared variable or it outputs the u variables,


which we then square. So these u variables right here, are output I'm going to guess this is from


this image, and this is from this image, and this is from this image and also kind of look into


the future right here. And yeah, so I have these u variables, and I define sort of a a window,


a context window around which I look. And I also predict them, I square them. And then I sum them


all up, but pull the square root right here, and they divide. So this is why I say kind of a


normalized Z is what comes out of this, but it's fairly, fairly complicated, right. But this is


going to, in a way encourage this behavior. So let's see why that is. And for that, I want to


just draw back a little bit to like a regular VAE, a regular variational auto encoder. So if


in a regular VAE, you have like an image, this is encoded, decoded, and you get back


an image, right. So in a regular VAE, what you assume is you assume that the latent space is


sort of made up out of these independent latent variables, latent random variables,


they're Gaussian distributed. And yeah, they're already said they're independent from each other.


And you claim, if I know the latent variables, so essentially, if I know the mean and variance of


these, then you know, producing an image is, is easy, right. You can simply train a neural network,


I input, you know, which, which, I input what values my latent variables are, or how the


Gaussians are parameterized. Alternatively, I input that, and I train the decoder to produce


a picture from that. That is easy. The question is, if I have a picture, trusty cat right here.


If I have a picture, what are the corresponding latent variables? You know, what are the values


of the latent variables? That makes sense right here. And of course, in a VAE, we train the encoder


and the decoder jointly such that they cooperatively can construct this latent space,


like, okay, how, how should, how should the latent space look from which the decoder decodes? But I


just want to turn your attention to the question of the encoder's job is essentially to take the


encoder's job is essentially to take in an image and produce what values the latent variables are.


And the latent variables are assumed to be independent from each other and Gaussian distributed.


Now, this is where this model right here differs. Okay, so this model says, well, we're going to


assume we have observed and latent variables, observed variables x and latent variables t


observed are, I guess, the images, or the image sequences, and t are the latent variables.


So this, I guess, would this would be equivalent to z hat to what I call z hat, they call t.


Alright, so they say will formulate the joint distribution. Note that in this framework,


in these variational frameworks, I don't, it's not my thing, either. But what you do is you always


you propose a mechanism by which the data and by which the variables are generated. So you,


as a designer of the algorithm, propose the structure of how the latent variables


work together. And then you have some small parts in there that you say, well, these things,


I don't know, I'm going to let a neural network do these things. But essentially, you come and you


impose a structure upon the world, right? And you know, if you get the structure correct, your model


will work fine. If you don't get the structure correct, your model won't work fine. But this is


a bit of a different way of working than you know, saying, well, I train a convnet to predict. So


we're going to propose our structure, we're going to say, the joint distribution of observed latent


variables factorizes into these two, it factorizes into this conditional. So if I have the latent


variables, right, then what are the images, and times the prior across the latent variables.


Now, we already seen this distribution, it's the first one is listed here again.


And this conditional distribution, that's simply your decoder in the VAE framework. And that's


written here, it essentially says, well, to produce an image, I'm going to put t the latent


variable into this neural network G right here. And that will give me the distribution of my


output image. So this is your decoder in the VAE. Now, the interesting part and where it differs


from a regular VAE is right here, where they say, well, how do our latent how does our latent space


look? Well, this is zooming around. Our latent space isn't a independent Gaussians, it's actually


this TPOT distribution, this topographic product, no, where, where does it, I forgot what it I


forgot what it's what it's called a topographic product of student T's model, the TPOT topographic


product of student T, that's going to be our distribution. And that distribution is going


to encourage this topographically organized latent space, right? So we can ask, how does it how does


it do that note that the encoder isn't here yet, because we've only we've defined we've imposed


the generative process of the data. The generative process starts at the latent space, I said, if I


know what the latent variables are, I can just ask my decoder to produce an image. So this


distribution here tells us, you know, the latent variables are distributed like this. And then


there we go. Now, obviously, what we want is we want our encoder to produce the variables,


the latent variables, but we also want what the encoder produces to follow this distribution


right here. And that's going to be the sort of difficulty right here, because what we know what


we can train with backpropagation is pretty much Gaussians, you know, like we can train things


where we can apply the reparameterization trick. That's stuff we can backprop through stuff we can


Gaussians we can sample from efficiently and so on, we have closed form solution for the KL


divergences in the objectives. So essentially, what we can do in these variational frameworks is


Gaussians, not topographic product of student teas. However, here, they show, okay, we can, in fact,


construct a product of student teas, this is no, this is not yet a topographic product is just a


product of student teas distribution from Gaussians. And that is, I take one z variable, and I take a


bunch of u variables, and they're all distributed, like Gaussians. And I square the use, I sum them


up, I average them, and then I take the square root and divide z by dot. And this variable right


here, that's going to be a univariate student T random variable. This should be kind of known if


you've ever taken statistics or like, use the T test for anything. Okay. And you know, this is


already quite familiar. And I can extend this now to the multi dimensional case. So if T is a multi


dimensional student, he's random variable, composed of independent z's and us, then we can construct


T as a vector. And that is going to be distributed according to a product of student teas variable.


And this should connect to what we've seen before, right? We said that this model's organization of


the latent space is pretty much of this form that we saw right here, we have the z variable divided


by the square root of the sum of the squared u variables. And now we learn how we can construct


the product of student T's latent space, given z and u independent Gaussians. And that is,


you know, now it should connect for you. In deep learning variational frameworks, we can work


pretty much only with Gaussian random variables. In this model, we want to work with product of


student T random variables. And here is the way how we can construct the product of student T


random errors from Gaussian random variables. So that's why here, we, the neural networks will


output the z and the u. That's what they will output. That's those are those are Gaussians,


or supposed to be Gaussians. And then we transform them by dividing them and summing them up in this


way to the latent variable that the decoder receives, which is this z hat or T, I guess,


to this is what the decoder receives. So we know that if the encoder outputs Gaussian random


variables, the decoder will receive a product of student T random variable. Now, why is the product


of student T random variable special in any way? Because it enables us to what they call here,


introduce topography. In essence, and they formulate this a little bit, what it does is


it it lets if if some of the use in this sum and some of the u in this sum are the same,


which you can see by the indices in this case, they are not. But if some are shared, that means


that the two were the two T variables, not the two z, the two T. So this is one T, and this is


another T, right? This is T one, this is T two, lots of T. These two variables will no longer be


independent, they will actually be dependent on each other. So this is a way how we can


construct latent spaces where some of the variables are actually correlated or in some other way have


have higher order correlations with each other, meaning that the value of one is not independent


from the value of the other one. And that is pretty much a basis for what we want for constructing


these topographic latent spaces. So here they say introducing topography, essentially, what we're


going to do is we're not, we're going to define neighborhoods across our u variables. And we're


going to share the u variables according to these neighborhoods. And that's going to make the in the


components of T dependent on each other. And this sounds complicated, but essentially, you can


imagine instead of having like four latent random variable, which are all Gaussians, now we have


simply one set of z variables, and one set of u variables. And we're going to consider an entire


sequence and not just one, one image, right? So we're going to consider an entire sequence of


images like this right here. Every image produces one z and one u variable. And then when we


consider an image, let's say this is the focus right now, we consider it's z. And we're going


and we consider a neighborhood of us. And that's just going to amount sort of like a convolution,


like this is maybe a neighborhood of three. So we're going to consider this u this u and this u.


So we're going to construct the z on top of the fraction divided by this thing squared,


this bubble here squared, this bubble here squared, square root of top on top of that.


And that's going to be our T. So the T for this image right here,


that's going to be this whole fraction. So when we train the VAE, we input the whole sequence,


we focus on for example, this picture, we construct its T by looking at its z and its


neighborhood of use, then we put that T into the decoder, the decoder is going to produce an image,


and then we can apply a loss function between those two. Okay, so that is the loss, that's the


loss function, right? The loss function, note that the loss function doesn't say


you need if you roll 10 times, then it needs to be the picture that's 10 times ahead. That is not


the case at all, we actually don't have the role function in here. But even now, even once we


introduce the role function in the in the latent space, we're not going to explicitly train the


model to predict the future. We're simply going to construct as we did here, the latent space,


such that it such that this naturally happens. So how are we going to do this?


Almost the same and here you have they talk about capsules. So you can see that they divide this


neighborhood structure. So the W defines the neighborhood structure, you can see here some


of the use, they are connected, and then other ones are connected. But these users are not connected


with those. They kind of talk about capsules, essentially, it's just that they make some of


the variables dependent on each other and some not. Or, or when they do these neighborhood things,


they just have two sets of variables, like to have two sets of z's and us, and they only


yeah, they construct two t variables. And that that's what they call capsules that I don't,


I don't know why the capsule terminology enters this paper necessarily. But, you know, they,


they want to draw a connection here. So temporal coherence, now we get to how do we organize this


latent space such that the role operation now also gets in. And this is pretty simple, it's actually


just an extension of this right here. So here, if you consider these images here, as images of a


sequence, we always said, well, you need to be connected to sort of your your neighboring variables.


And now, sorry, your neighboring u variables, as they are, right. And now we're going to say


the same thing. But, but I'm going to draw the critical path here again. So this, we have a z


variable right here, we have u variables from the neighborhood, okay. And we're going to take


the z variable on top of the fraction. And we're going to take the u variables below the fraction


right here. Like so, like so, like so. Now, before we do this, before we take the u variables here


below the fraction, we're going to roll the u variables according to their distance from


according to their distance from the focus. So in this case, this would be simply one roll back,


this will be simply one roll forward. So in the language of this paper, what this means is that


we don't want we, we don't want this image, or it given a particular position in this image,


right, this position right here. If we simply apply the classic neighborhood structure, we say,


we want this position in this image to be correlated with the same position,


a step back and a step forward. Now, if we construct the role like this, what we're saying


is no, no, no, no, I don't want I want, I want this position to be correlated with maybe this


position here and this position there, like slightly behind and slightly ahead. But I'm


obviously not going to tell the model what I expect, I simply say, please, this image is one


time step, this image is one time step back from me, please roll the latent space by one. And


that's going to be your relevant variable. And in this case, it's please roll the latent space of


this thing, one forward, and that's going to be your relevant latent variable. So it's not that


we train, we train rolling this T variable here, because the T is what finally comes out, we're


not training this T to roll forward or back, and then predict 10 steps ahead, we're simply saying,


how you are influenced you as a focus, how you are influenced by pictures before and after you,


you're not simply taking into account their latent variables, you want to take into account


rolled versions of their latent variables, in order for you to reconstruct yourself in the


training objective. And it turns out, at least that's how I understand it, right. And it turns


out, so here you can see the whole process, we're going to take images, we're going to produce mean


and variance of late of Gaussian variables for the Z and the U variables. So if you had just the VAE,


it would just be this right here. And those will be your latent variables, but not here,


we produce two sets Z's and U's. Then we're going to construct the T variables, I don't know why


this is on the bottom here, but then we're going to construct the T variables according to this


formula. W here is the neighborhood structure, you define it, U and Z are the variables you


produced from your encoder or you sampled from what your encoder produced. And mu here is also


a learnable parameter, a learnable mean parameter. And then you want to stick this these T's into,


you're going to stick these T's into this neural network. Now here it says Z and ZL and UL, but


essentially, this here, this here, these create T. Oh, here, it's here, you're going to stick the T


into your decoder neural network, remember the G, how do we get the picture from the latent variable,


that's the decoder, and stick that into the decoder and out you get an image. And then you


train it with the classic elbow, the evidence lower bound, which says, okay, what I want is,


I want to reconstruct the picture accurately, right? That's this term right here to reconstruct


the picture accurately. But I also want that my Z, well, essentially, what I want is that my T


variables are distributed according to this TPOT distribution, I want to enforce that,


but I can't, right, I can work with Gaussians. So what about what I can do is I can say, well,


the Z variables and the U variables, they must be as Gaussian as possible. So I penalize the KL


divergence between what I produce, which is this right here, and the Gaussian, like a a pure Gaussian,


this has a closed form, I can, I can calculate KL divergences from what I produce with Gaussians,


no problem. Okay, and that's the training loss. And I simply average that over the input sequence.


And there, there you go. Now, the evaluation of these things, I have to say, after reading through


the experiments in the evaluations, this is this is a paper, kind of an idea, at least I feel so,


right? Correct me if I'm wrong, but I feel that this is sort of an idea paper, it's like, here's


an idea, it works if we you know, specifically construct a data set for it. And if we specifically


also the experiments are appeared to be kind of fiddly, like you have to really, you know,


get your parameters right to make this work. But if you do, then you know, the model behaves as you


as you expect. And so they measure things like, is the rolled version of the latent variables really


equal to the latent variables a couple of time steps ahead, and things like this, and they


produce these these maps. So here is one where the latent space isn't a 1d torus like we looked at.


So 1d torus is this right, so you go around, around, around, sorry. This is a 2d torus. So a


2d torus is like a plane. And if you leave here, you come back here. And if you leave here, you


come back here. So if you if you roll this up, and then you you have a pipe and if you close the pipe,


you have like a donut. So that's a torus. So if they have a topographic space, like a torus,


they and they simply apply that to MNIST, the test set sort of looks like this. I don't know if you


want to read something into this, like, feel free. I'm not sure. But in when they go with the


sequences. So here, you see like the sequences, I think on top is what they input. And then this is


the continuation that the model doesn't see on the bottom is what the model produces. You can see the


model does get to a point where it understands how these sequences go here. Right, it goes larger,


larger, larger, and then it kind of flips around to the smallest. This is an expected behavior.


Here as well, the rotation, it model continues the rotation. And it turns out, even if the model is


just trained with, they have these these these experiments, even if the model is just trained


with single transformations, so either a roll, sorry, either a rotation, or a scale transformation,


or a color change, it can generalize to multiple transformations at once, as you can see right here,


colors and rotations can the model can generalize to that fairly, fairly well. Okay, I don't want to


get too much into the experiments, because I'm not sure how important the numbers here are. I'm not


sure how important the numbers here are, I'm safe to say, if you construct this model, and if you


apply to the, you know, problems where exactly this is needed, and if you get the hyper parameters


right, then this model actually works, it's better, whereas a regular neural network, it could not


easily incorporate the concept of these slow changing transitions, it would sort of have to


learn, okay, what color comes after red, orange, okay, what color comes after orange, yellow, okay,


what color comes after yellow, green, I guess the other model has to learn that as well. But


this model, it cannot represent the transition in a sequence as sort of as it has to learn it


as a parameterized function, rather than being able to map it to an internal transformation of


the rate of the latent space, like the topographic VAE can do. Okay, that was it for me, I'm not


competent enough to tell you how big of a step this is, it feels to me like a little step.


It might be a giant step, I don't know, okay. It feels to me like it's kind of an idea paper


to show something neat that you could do in an idealized case, it might be that this is a


much bigger deal than I think I thought it was a cool paper, I thought it was a neat idea. It's


written even though it's I think, kinda, you know, more high level, sorry, more, more. So I'm not as


competent at it, but I could still make sense of it. So if you enjoy this, give it a read. Yeah,


NNFS Update #2: Content done

What's going on everybody? Welcome to an update to the Neural Networks from Scratch project.


NNFS Update #2: Content done

The content for Neural Networks from Scratch is done.


NNFS Update #2: Content done

So the only thing we're kind of doing right now is going through and reviewing for like grammar issues and stuff like that.


NNFS Update #2: Content done

If you read the book like a month or so ago or more


NNFS Update #2: Content done

and you found certain chapters such as the derivatives or backpropagation confusing,


NNFS Update #2: Content done

I strongly encourage and suggest you to go back and read those sections or wait for your book if you ordered a physical copy


NNFS Update #2: Content done

because those chapters specifically, but all of the chapters have been revamped a lot.


NNFS Update #2: Content done

So much so that I really would call Neural Networks from Scratch already at like a V2 right now


NNFS Update #2: Content done

just simply because of all of the feedback that we've gotten.


NNFS Update #2: Content done

So we're over 4,000 sales for ebook, softcover, hardcover.


NNFS Update #2: Content done

So we've had a lot of people come through this document


NNFS Update #2: Content done

and part of what has taken so long is arguably we probably, I mean, we could have shipped sooner


NNFS Update #2: Content done

but it would have been a lesser product, honestly.


NNFS Update #2: Content done

The changes that we have made, especially recently with this kind of like revamp when we went through again


NNFS Update #2: Content done

have been pretty significant. Some of these chapters have changed like more than 50%.


NNFS Update #2: Content done

So huge, huge updates. So I definitely recommend everyone goes back and reads if you were confused anyways.


NNFS Update #2: Content done

Also, huge shout out and thanks to everyone who both supported the document as well as the contributors.


NNFS Update #2: Content done

Special shout out to both Thomas and Spencer who have been helping with the editing process


NNFS Update #2: Content done

especially Spencer most recently even like right now as I film this


NNFS Update #2: Content done

helping us find out all our little grammar issues and stuff like that.


NNFS Update #2: Content done

So thank you to everybody.


NNFS Update #2: Content done

But we've had, I don't know the exact number, but we've had a huge number of contributors.


NNFS Update #2: Content done

So thank you to everybody who's made contributions both in suggestions but also just asking questions.


NNFS Update #2: Content done

So that's what's given us the ability to see like, okay, a lot of people are having a hard time with this specific section.


NNFS Update #2: Content done

Let's update that section. Let's try to word this in a different way and make it easier to understand and so on.


NNFS Update #2: Content done

So overall, I'm super happy with everything.


NNFS Update #2: Content done

The whole format of us doing it this way has been absolutely phenomenal.


NNFS Update #2: Content done

So now the only delay is with the hard copies of the books.


NNFS Update #2: Content done

So the hardcover and the softcover. We are, I've been shipping myself demos and Daniel demos.


NNFS Update #2: Content done

And kind of the problem is it takes me two weeks to receive my demo.


NNFS Update #2: Content done

It only takes Daniel six days to receive his. So a lot of times we know about certain issues really quick and we can kind of have a sort of rolling demo.


NNFS Update #2: Content done

So I'm actually still waiting on a demo book that Daniel already got and I actually already ordered another demo book that actually I just sent to Daniel


NNFS Update #2: Content done

just because I wanted this process to go a little quicker.


NNFS Update #2: Content done

But it still takes a significant amount of time. And just to give you an idea, I mean, there's some stuff that is like, okay, we want a demo book just so we can see in physical form.


NNFS Update #2: Content done

Things like margins and stuff like that. I am using Adobe InDesign for the book.


NNFS Update #2: Content done

But still things look different in physical form compared to like on a computer screen when you're editing,


NNFS Update #2: Content done

even though you can make it look kind of like a book and stuff like that in InDesign.


NNFS Update #2: Content done

It just doesn't compare to seeing it in physical form. It really does look different.


NNFS Update #2: Content done

So you don't want to see it, want to get margins right, especially like that inside margin and stuff like that.


NNFS Update #2: Content done

So definitely to get those things done. But also just to give an example, the latest demo that I sent Daniel and myself,


NNFS Update #2: Content done

I updated the main kind of neural network image on the cover that I sent to the publishing company.


NNFS Update #2: Content done

And for some reason, like our version of the PDF looks great. It looks like you would expect.


NNFS Update #2: Content done

It's nice, transparent image. The image itself was 100 percent. It was transparent.


NNFS Update #2: Content done

But somehow I send it in as PDF. It looks normal on the file that I sent in.


NNFS Update #2: Content done

But then once they do their conversion on their end, suddenly this image has a background.


NNFS Update #2: Content done

So it looks pretty silly. And I definitely would not want thousands of books with that problem.


NNFS Update #2: Content done

So there's really no question in my mind. I got to see an ideally bold me and Daniel have to see the physical version of the book


NNFS Update #2: Content done

before we send it out to thousands of people, because we're talking about I don't know the exact number.


NNFS Update #2: Content done

All I can say is it's over 50, as in five zero, fifty thousand dollars worth of books.


NNFS Update #2: Content done

So I'd rather not have some mega issue with that many books. I can't afford to make that mistake.


NNFS Update #2: Content done

And plus, I wouldn't I just don't want to let you guys down with something like that. That would be a stupid problem.


NNFS Update #2: Content done

So. So, yeah, I have to see it. And I really think both Daniel and I have to see it.


NNFS Update #2: Content done

Like some stuff, obviously, like the book cover. We both don't have to see it to know that's a problem.


NNFS Update #2: Content done

But anyway, stuff like that. I just don't want that to happen. I can't order the book for everyone's sight unseen.


NNFS Update #2: Content done

So that's kind of in the works as we speak.


NNFS Update #2: Content done

My hope is the next big demo that we send will be the full book to both myself and Daniel.


NNFS Update #2: Content done

And then hopefully within hopefully the first week of October. And if not the first week, by the second week of October, we start shipping out the books.


NNFS Update #2: Content done

But again, the content is all done. If you bought any version of the book that includes the physical versions and then obviously the e-book,


NNFS Update #2: Content done

you get access to the e-book. So so if you're not in that group, that Google group, contact me.


NNFS Update #2: Content done

Let me know. We'll get you in that group. You should have access to all of the chapters, not some people think it's just chapter one.


NNFS Update #2: Content done

It's all the chapters. The links are all there. Also, once we do the kind of final pass of all the grammar checks,


NNFS Update #2: Content done

I'll convert it to a single file PDF download. That will be pretty quick for me to do.


NNFS Update #2: Content done

I also want to convert it to an e-pub so you could read it on like an e-reader.


NNFS Update #2: Content done

But I suspect I'm I don't suspect I know it's going to require some different formatting.


NNFS Update #2: Content done

So that might take a little longer to get done. I mean, I can put out one that's just like a converted version,


NNFS Update #2: Content done

but it won't be as clean as a formatted specifically for version.


NNFS Update #2: Content done

So anyway, that might take a little longer, but the PDF version will be put up within pretty short order.


NNFS Update #2: Content done

So you can just download that. But again, if you have questions or something on the material,


NNFS Update #2: Content done

you can go to the Google Doc and kind of ask them exactly in line with that Google Doc.


NNFS Update #2: Content done

And I think that's a little more useful. So anyways, I think that's everything I want to cover.


NNFS Update #2: Content done

Thank you so much to everybody for your support and your patience with all of this.


NNFS Update #2: Content done

I haven't had anybody complain that we've had to push this deadline a couple of times.


NNFS Update #2: Content done

I think it's because everyone has access to the e-book, but still I am surprised.


NNFS Update #2: Content done

I'll probably get some emails just because I said that. But but still, thank you guys so much for, like I said, the support and the patience.


NNFS Update #2: Content done

This is this is just such cool material. I'm so excited to be a part of it.


NNFS Update #2: Content done

And I know Daniel is super excited about it as well. And I just I can't wait to send everybody their books.


NNFS Update #2: Content done

I can't wait to get my own version. I've got some demo books, but none of the demo books so far have only been like 150 pages.


NNFS Update #2: Content done

So they're not quite the the Bible that I expect it to be. So anyways, stay tuned.


NNFS Update #2: Content done

Hopefully the next time you get an update on this, it will be we have shipped the books.


NNFS Update #2: Content done

So very, very exciting. So anyways, that's all for now. Questions, comments, concerns, whatever.


NNFS Update #2: Content done

Feel free to leave below again. If you're having any problem accessing the material that you've already purchased, please email me.


NNFS Update #2: Content done

That's Harrison at Python Programming.net. And of course, if you want to buy the book, you can get it at NNFS.io.


NNFS Update #2: Content done


ReBeL - Combining Deep Reinforcement Learning and Search for Imperfect-Information Games (Explained)

Hi there, take a look at this variant of the game Rock Paper Scissors. It's like usual Rock Paper


Scissors, except with the added complexity that when either player chooses scissors, then the


rewards and the losses are doubled. So for example, you see right here, player one chooses rock,


and player two chooses scissors. So both the reward for player one and the loss for player


two are double the size. Now, you might know that in original Rock Paper Scissors, the optimal


strategy is to play one third of each of the three choices at any time. So you basically take a fair


three-sided coin dice, does that exist? I'm not sure. And you throw it and whatever side is up,


that's what you play. However, here, since one of the options is different, the sort of optimal


strategy shifts and interestingly, it shifts as follows. What you want to do is you want to play


rock and paper both with a 0.4 probability. And you want to play scissors with only 0.2 probability.


That is pretty interesting. You might intuitively conclude that you want to go more where there are


more rewards to be had. But of course, also you lose more. So you might also conclude, well,


it doesn't make a difference ultimately. But why does the why does the sort of optimal strategy


shift such that you want to decrease your likelihood of playing scissors? Let's just


quickly analyze this game before we jump into the paper because this game is sort of a microcosm


of what the paper of today is about. So the paper of today is called Combining Deep Reinforcement


Learning and Search for Imperfect Information Games by Noam Brown, Anton Bakhtin, Adam Lehrer


and Qi Chenggong of Facebook AI research. So this paper brings basically what AlphaGo or AlphaZero


has done for perfect information games. It brings this to the domain of imperfect information games,


and we'll see what the difficulties are in this and what can be done to solve it. And not only


do they have an algorithm, but they have interesting theoretical results that under


some conditions, namely under the condition that neural networks do something useful,


will actually converge to Nash equilibriums in these games. So that is pretty cool. So


practical and theoretical paper right here. As always, if you like content like this,


don't hesitate to share it out. And tell me what you think in the comments. This is not my field,


so I might get quite a bit of stuff wrong right here. Also, if you haven't seen the the Negrano


poker challenge, so it's I think it's the last video I did, be sure to check that out just to


see how you have to think about situations like this. Alright, let's get back to this


this rock paper scissors example right here. Interestingly to note is that these these dash


lines here means that player two cannot decide which of these states they're in. So player two


doesn't know what states are in. For player two, this is all basically the same state. It'd be


really easy, right? If player one plays first, and then player two sees what player one does,


and then they they just act that they always win. However, player two doesn't. So they have to sort


of decide what to do, independent of which state they're in. Especially this is a this is a symmetric


game, right? This is a two player game. Because as two players, it's zero sum, because whenever


one player wins a reward, the other player loses the same reward. And it is also it is that makes


it symmetric. So all the both players play at the same time, though that is not necessary in general,


but here it's the case. Alright, so this means in this particular case, whatever strategy player one


has player two must have as well. So we'll just do the analysis for player one. So let's say you


deviate from this optimal strategy, right? We claim that this here is the optimal strategy, playing 20%


of scissors. Let's say player one doesn't believe it player one deviates from it and says, nah,


there is so much reward there, I'm going to get some more of that. So they up this right,


they up this to like, let's say point, I don't know, point three, three, like doing the classic


one third or even higher, right? They up this, go more scissors, okay. And they probably want to take


this mass because they have to take it from somewhere, they probably want to take this from


rock and paper. Let's say they just take it equally from rock and paper towards scissors to


up the to up the probability that they play scissors. So from paper and from rock, they go


towards scissors. Now, player two observes this, right, they can just play against player one for


a while. Or what we're going to assume is that everyone announces their strategy publicly. It's


the same thing, you can just observe someone for a while, or they can just announce their strategy.


It's what we'll treat this equally. So player two observes player one playing scissors too often. So


player two knows they are very often in this situation right here in this right state, they


can't directly observe, but they infer I must be very often in this right, rightmost state where


player one chooses scissors. And therefore, you see player two's payoffs, it's zero here, minus


two here and two here. So they'll say, well, I also have this optimal strategy of point four,


point four, point two, what I can do is I can simply knowing that I'm a lot in this state,


I can simply take some mass from paper and put it on rock. So I play rock way more often.


And I reduce the amount I play paper, right? Scissors doesn't matter. But now I lose two


less often, and I win two much more often. And player one in turn loses two much more often and


wins much less often, right? So player one wanted to get more reward, but they're sort of being


punished by player two for playing this too often. Now you can say, well, it player one can do the


same thing, knowing that player two plays rock too often now, right? They've taken away mass from


paper towards rock, knowing that player two has taken rock. Player one knows that either they're


here, or they're here, right? And in this case, player one can say, all right, you play rock too


often. Obviously, if I play scissors, then I'm going to, I'm going to lose, but I've already


decided I want to play scissors much more. So they're trying to make it up right here. So what


they can do in this case is they can say, when I play paper, I win one instead of if I play rock


too, I win zero. So I know player two is playing rock way more often than they should. So I'm going


to punish player two by playing paper more often. So let's erase this arrow. Let's say we play


scissors. Sorry, we play scissors. No, let's not erase this, we play scissors by moving from rock.


And we also move from rock to paper, like we're almost never playing rock, we're just playing


scissors more often, because that's what we started with. And we're playing also now paper more often.


So now, we basically do the same thing that player two did to us, we are upping the likelihood of


this thing happening and decreasing the likelihood of this thing happening. So now we can say,


ah, now I also, I play paper more often. Now I also win more often here, and you lose more often.


But you see, because the rewards are doubled over here, the fact that player two can achieve this


is much more meaningful than the fact that player one can achieve this. Okay. And that's why player


one will be punished harder for deviating here. So that's sort of how you reason about these


strategies. So if player one will play this point two too often, they will be punished harder than


player two for deviating in response to that. And the same counts for the symmetric part. This is a


very important concept right here. Namely, you can see player two strategy depends on player one's


strategy, even though you could conceptualize this game of player one plays a move. And then


they play a move, but they don't show it yet, right? They play a move, they take like a picture


of their hands doing rock, paper, scissors, and they just don't show the picture yet. And then


player two plays a move. So now we're basically back in we're in this game, where it's sequential


in nature. And usually in a sequential game, you can just do a sub game analysis. So you can just


say, okay, and do a sub game analysis. But the sub game analysis depends on the strategy of player


one because you don't know the situation. This is different than a full information game.


And this is illustrated right here. So they say, usually, what something like alpha zero does is


your game starts here, right? And then you have two actions to take, you maybe take this action,


okay, now your opponent has two action, maybe they take this action, all right. And now you have two


actions again, which one do you take? What, what something like deep Q learning or something like


deep Q learning or actor critic learning would do is they would simply put a neural network here,


they would look at this state, and they would simply tell you which action to pick,


like this action right here sounds good to the neural network. In contrast to that, alpha zero,


if I draw the same situation right here, alpha zero, what it will do is it will say, well,


I could do this, or I could do this. If I do the left thing, then I'm going to have my opponent's


gonna have two options, they could do this, or they could do that. If they do the left thing


again, and so you get the idea, it sort of goes down the tree and it does this over here, right?


Sorry, this should be so it goes down the tree, I'm stupid. And it evaluates, it kind of calculates


ahead, it uses its internal simulator to look ahead. And it could technically do this until


it reaches the end. And then it would know if it reaches the end state every time here,


it wouldn't know it could simply backwards calculate which one is the best option for me to


do right now. However, this game is often very, very deep. So the tree, the depth here is often


so deep that you can't solve the whole game. So what alpha zero does instead is it says,


I'm not going to play until the end, I'm going to play a certain amount ahead, right, I'm going to


think some limited depth ahead. And I know alpha zero does this adaptively. But bear with me, I'm


going to think some limited depth D ahead. So here, in this case, D is equal to two because we think


two layers ahead. And then at the end, I'm going to replace everything that comes after with a


single value that indicates how good this is for me. Okay, so and this thing right here is very


hard to get. Of course, if you knew how good anything is for you, then you have solved the


game. But alpha zero at this point, the neural network comes in, right? It this is a neural


network, it's a black box. So it simply asks for each one of these states, how valuable do you


think that is? Okay, how valuable do you think that is? Okay, and so on. So it asks for each state


the neural network, how valuable that particular node is. And then it does the same backwards


calculation. So we've sort of substituted going to the end of the game by the neural network.


But it is still more powerful than asking the neural network at the very beginning,


like we do here, okay, the power comes from combining the learning, this is this is the


learning and the search, this here is the search. Right, so this is what alpha zero does. And this


is what this paper does for imperfect information games. So imperfect information games is when you


don't know a particular thing about the game at any point. So there is hidden information,


like in poker. And the problem is right here, if you do the same thing for this game right here,


and you look from player one's perspective, and you say, okay, this game is very deep, actually,


it's just too deep, right? But let's assume that's too deep for you. And you want to replace you want


to say, okay, I'm just going to look ahead, D equals one, that's all I can afford. I go ahead,


and at the end, I'm going to ask my neural network, what the value here is. And the neural network


will tell you accurately that the value at each of these nodes is zero. So the average value,


if you can see right here, the average value of each of these nodes is zero,


depending, of course, on how player two acts, but in this case, it's zero. So as player one,


this information will not lead you to the correct optimal conclusion, the correct optimal conclusion


being this point 4.4.2. Okay, player one, like it's indifferent, any strategy could work here,


right? If there is some regularization, it will probably come to the point, the one third, one


third, one third, right? Since all the values are equal, it might conclude it's probably best if I


distribute my actions or something. So you can see the problem right here. And the problem is that


this value right here, it depends on the strategy of player one. Okay, the and this is something


that alpha zero has no concept on for alpha zero, the value of a node only ever depends on what


comes downstream. In imperfect information game, the value of a node also depends on what has


happened upstream. So on the strategy of the upstream events. And that is, as I said, that is


that is quite important. Also for alpha zero, once I have evaluated a game tree and determined


the value of a node like this, I can evaluate the same game tree again, and the value is going to


be the same but for the same reason because the value depends upstream the value of this node


right here, depending on upstream, if I change my strategy, so if here I determine either action one


or action two with a certain probability, if this search process results in a result that tells me


this is how often you should pick action one, and that's different from what I searched with, right,


then all of these values down here are going to change. And I can basically search again. So these