Port test from cf1904f to avoid future regression

Related issue: #11939, #12084

This is for `master`