[spark-3586][streaming]Support nested directories in Spark Streaming

For text files, the method streamingContext.textFileStream(dataDirectory). 
The improvement of the streaming to Support subdirectories,spark streaming can  monitor the subdirectories dataDirectory and process any files created in that directory.
eg:
streamingContext.textFileStream(/test). 
Look at the direction contents:
/test/file1
/test/file2
/test/dr/file1
if the directory "/test/dr/" have new file "file2" ,spark streaming can process  the file


I'd suggest to refactor `getPathList` and `getSubPathList` into the following version, which also addresses the performance issue [mentioned above](https://github.com/apache/spark/pull/2765/files#r19327246) by replacing individual `getFileStatus` calls with batch style `listStatus` calls:

```scala
  def getPathList(path: Path, fs: FileSystem): List[Path] = {
    def dfs(status: FileStatus, currentDepth: Int): List[FileStatus] = status match {
      case _ if currentDepth < 0 => Nil
      case _ if !status.isDirectory => status :: Nil
      case _ => fs.listStatus(status.getPath).toList.flatMap(dfs(_, currentDepth - 1))
    }

    dfs(fs.getFileStatus(path), depth).filter { status =>
      val isNewlyModified = status.getModificationTime >= ignoreTime
      logDebug("Newly modified file detected, modification time is " + status.getModificationTime)
      isNewlyModified
    }.map(_.getPath)
  }
```

Also, the `fs.getFileStatus(path).isDirectory()` call can be removed with this version.