Don't match twice when moving inside a fence