Revert "[HOT FIX] [YARN] Check whether `/lib` exists before listing its files"

This reverts commit 38cb2c3a36a5c9ead4494cbc3dde008c2f0698ce.
Revert "SPARK-2624 add datanucleus jars to the container in yarn-cluster"

This reverts commit a975dc32799bb8a14f9e1c76defaaa7cfbaf8b53.