SPARK-1588 Re-add support for SPARK_YARN_USER_ENV and SPARK_JAVA_OPTS

This is what I did to unblock our jobs - please feel free to modify or create a new PR based on this in case this wont suffice.
Unfortunately, I wont be able to shepard this patch actively ...