Only do the search if the position is not a wall/traversed