updated for batch processing, utilities and analytics