Order My Steps: Learning to Walk the Path that&#39;s Ordered