[SPARK-4518][SPARK-4519][Streaming] Refactored file stream to prevent files from being processed multiple times

Because of a corner case, a file already selected for batch t can get considered again for batch t+2. This refactoring fixes it by remembering all the files selected in the last 1 minute, so that this corner case does not arise. Also uses spark context's hadoop configuration to access the file system API for listing directories.
Would you mind adding a comment that this is a cache? Or maybe you could call it `modTimeCache`? If you keep the current title, it should probably be `fileToModTime` with no 's'.