refactoring and cleanup for term generation and cleaning