major refactor and extraction of shared functionality to services