More eagerly set and clear the thread context class loader.