modify to match new Spark APIs