[spark-3586][streaming]Support nested directories in Spark Streaming

For text files, the method streamingContext.textFileStream(dataDirectory). 
The improvement of the streaming to Support subdirectories,spark streaming can  monitor the subdirectories dataDirectory and process any files created in that directory.
eg:
streamingContext.textFileStream(/test). 
Look at the direction contents:
/test/file1
/test/file2
/test/dr/file1
if the directory "/test/dr/" have new file "file2" ,spark streaming can process  the file


Calling `getFileStatus` for each path here can be really slow for S3 files. It often takes over 100ms or even hundreds of ms to accomplish 1 call.