Set upper memory limits for the perf tests.

Hopefully this stabilises CI.