update and improved the logic for stage file selection
in the indexer