major cleanup and support for a smart user agent