Made cancel after X responses configurable