check file existence before indexing and add more warning log when write to hdfs failed