[SPARK-4082] remove unnecessary broadcast for conf

We already broadcast the task (RDD and closure) itself, so some small data used in RDD or closure do not needed to be broadcasted explicitly any more.