[1.6.x] Fixed #20760 -- Reduced timing variation in ModelBackend.

Thanks jpaglier and erikr.

Backport of 5dbca13f3baa2e1bafd77e84a80ad6d8a074712e from master.