added spark functions and email alert