Handle scrolling the game panel manually, since using a JScrollPane didn't work