Add weird case where max_replays=5 produces different interreplay_*
directories