Add directory scanning to indexer class