apache/spark — pull request #3203 by maji2014 (2014-11-11 06:35:16+00:00)
Title: [SPARK-4314][Streaming] Exception when textFileStream attempts to read deleted _COPYING_ file

The ephemeral file(_COPYING_) is caught by FileInputDStream interface.
On one hand, the file could be counted two times including file itself and file._COPYING_ if _COPYING_ file is not deleted when spark handle this file.
On the other hand, exception appears when textFileStream attempts to read deleted _COPYING_ file if _COPYING is deleted when spark handle this file.
Therefore, add filter for intermediate partly written state