Correctly bypass the cache if `bypassCache` is set (see #5872)