Avoid hitting all segments for TimeBoundary queries