Hey, so in this video what we're going to do is we're going to watch as a computer learns to play the visual doom defend the center scenario. So in this scenario the computer can only turn left or right and fire and then enemies come from all around it so it has to learn to turn to find the enemies and then shoot those enemies. It has a limited supply of ammunition once it runs out of ammunition it can't kill enemies anymore. So we're going to frame this for 75 hours which costs about 25 dollars on an ammo's and an instance and then every 10 thousand episodes which is about five hours of training time more or less. We're going to make a copy of the model as it is and what we're doing here is we're playing back, we're taking each of those models and we're making it play one episode and seeing how it looks. So we're going to look at the model after zero hours, after five hours and so on for the rest of the training time. So let's begin. Alright so this is right at the start and so initially it's just playing randomly and then here after five hours it's learned okay I should turn and shoot. I mean it's turning a lot and it's not shooting very accurately. Here it's learning to shoot a little bit more sparsely maybe. So we're now up to 30 thousand episodes. It's been training for 17 hours it's cost about seven dollars. So it's getting a reward of like two so it gets a reward of plus one every time it shoots a monster. We're still on two and basically what it needs to learn to do is to kind of turn and then aim at the monsters and then shoot them. Installing all this quite self-legged doesn't actually know what it's doing. What it does is the network has the what we see on the screen is the input. This goes into a convolutional neural network and then the output of this network is the decision should I go left? Should I turn right or should I click fire? But it doesn't know that any of these buttons actually means left or right to fire. All it knows is what it can see happening on the screen. Right so now we're up to 45 hours which is about two days. A got a reward of 12 is actually learning to save ammunition and it's learning to fire basically when there's a monster in front of it more or less. But it's kind of missing a lot so it kind of vaguely shoots and it actually is getting quite a few. So we're up to a reward of six now. We're on episode a hundred thousand. It's been training for two and a half days. So it's not doing badly. It could be a little bit more accurate. What's happening here is it ran out of ammunition so it couldn't do anything. All it could do is watch just the monster kill there. So now we're up to 62 hours and it's gradually becoming more accurate. And then out of ammunition. What it needs to learn to do is to conserve it ammunition and shoot accurately. So it's shooting fairly accurately but it could be a little bit more accurate. I think it's doing a fairly good job of serving ammunition like it's realizing that it shouldn't be shooting when there's not a monster in front of it. And now we're up to 67 hours. Out of ammunition. And this is the second to last episode. This is about as good as it gets. It's getting rewarded about 21. It's shooting 21 months before it dies which is not bad. I mean it's pretty good. Yeah and then out of ammunition. And then this is the last episode. So there's sort of some variance in the game. So this particular episode is I think if this is not its best game. I think the second to last game was its best game. So I'm usually at this point the game crashed on my instance. So there's no more training. I think if we carried on training for another couple of days it would have likely become a little bit more accurate. But there we go. So this is this is using the reinforced algorithm otherwise called policy gradient. There are other better algorithms called PPO. This is relatively straightforward algorithm term. If you're interested in reinforcement learning and you're interested in building this system yourself I have some videos in my playlist where I build this system from scratch. So maybe I'll see you there.