start out with a completely random network you feed that network a frame from the game engine it produces a random up with action you know either up or down you send that action back to the game engine and the game engine produces the next frame and this is how the loop continues and the network in this case it could be a fully connected network but you can obviously apply convolutions there as well and now in reality the output of your network is going to consist of two numbers the probability of going up and the probability of going