some refactoring and support for using an http proxy