Fix up issue 12144 test case

Cherry-pick the fix from #3271 into master branch.