Regression test and changelog for [JENKINS-16573]