various important refactors and improvements