[SPARK-1860] More conservative app directory cleanup.

First contribution to the project, so apologize for any significant errors.

This PR addresses [SPARK-1860]. The application directories are now cleaned up in a more conservative manner.

Previously, app-* directories were cleaned up if the directory's timestamp was older than a given time. However, the timestamp on a directory does not reflect the modification times of the files in that directory. Therefore, app-* directories were wiped out even if the files inside them were created recently and possibly being used by Executor tasks.

The solution is to change the cleanup logic to inspect all files within the app-* directory and only eliminate the app-* directory if all files in the directory are stale.

I think the key to the executors is appId/execId, which is not the same as the dir's path, which contains only appId. Additionally I would clarify this logic, either by extracting it out to a variable or adding a comment, or both. Something like

```scala
val appId = dir.getPath.getName
// Attempt to filter out running applications based on the existence of executors for that appId.
val isAppStillRunning = executors.values.map(_.appId).contains(dir.getPath.getName)
```

I would also add a higher level comment that describes the overall logic (i.e., only delete directories for apps that are not running anymore and whose files are sufficiently old).