improved response handling, calculate stats, and optimize seperations