major refactoring to define types and flow control for async url processing