Got polygon collision detection working when moving forward.  If you collide with a wall going backwards it just ignores the collision to resolve issues with getting stuck in the wall when colliding with it.