added skip bad documents logic for IA