[1.5.x] Changed testing strategy used in 6b03179e.

Avoid polluting the app cache as it causes unrelated test failures.

Refs #19688.

Backport of 7b49da1 from master.