Add small optimization to Lat/Long device filter.

This improves time when configurations has not the mentioned filter
set.