major refactor for manual data collection

major refactor...