various refactoring of data collection, added caching of calls, new functions