added a second full scan to track if the first metadata attempt at 500K fails