major refactoring WRT threading and other issues discovered when doing sync