Fix flaky tests

[#75286170]

Signed-off-by: Johannes Tuchscherer <jtuchscherer@pivotallabs.com>