[WIP][SPARK-1720][SPARK-1719] use LD_LIBRARY_PATH instead of -Djava.library.path

- [X] Standalone
- [X] YARN 
- [X] Mesos
- [X]  Mac OS X
- [X] Linux 
- [ ]  Windows
 
This is another implementation about #1031
I would add a comment here that explains why we have to set the environment this way:
```
// Set the environment variable through the prefix because we need to preserve the existing value
val prefixEnv = ...
```