An asynchronous mechanism for ingesting files into HDFS