Revert "[HOT FIX] [YARN] Check whether `/lib` exists before listing its files"

This reverts commit 90ec643e9af4c8bbb9000edca08c07afb17939c7.
Revert "SPARK-2624 add datanucleus jars to the container in yarn-cluster"

This reverts commit a975dc32799bb8a14f9e1c76defaaa7cfbaf8b53.