Simplify the logic deciding when we're finished