Changelog and cleanup for [JENKINS-8334]