Paper: Learning Optimal Contracts: How to Exploit Small Action Spaces (ICLR-2024)

Reviewer comment (Theory):
The paper studies the setting without prior distribution of the agent's type, but the sample complexity problem is not well-motivated in this setting. If acting as the principal, when an agent comes, I would not try to find my optimal contract merely for this agent by signing the agent multiple times with different contracts and observing the outcomes. I would only use this approach when there is a large candidate pool and I do not know the prior distribution of the candidates' skills.

Author response:
We disagree with the Reviewer on this point, as we think that the sample complexity problem that we study in our paper is well motivated. Indeed, we believe that, even in the case where the principal repeatedly interacts with the same agent, if the principal has no prior knowledge about agent's features, then it is reasonable that the principal would try to learn them by repeatedly signing contracts with the agent. Honestly, we are not able to find a better approach to tackle the problem. At the same time, the problem is arguably interesting and of practical relevance even with a single type.
   
Moreover, let us also remark that similar learning problems have also been addressed for the (much simpler) Stackelberg problems, where the leader repeatedly interacts with the same (unknown) follower in order to learn an optimal strategy to commit to (Letchford et al. (2009); Peng et al. (2019)). Followers do not have types in these settings as well.
    
Another way of looking at the sample complexity problem studied in our paper is to imagine a scenario in which the principal has access to some ``simulated model'' of the agent that can be used to learn agent's features and in turn an optimal contract to commit to when the actual interaction with the agent takes place. Thus, we believe that studying learning in principal-agent problems in which the agent has a single type is of interest and well motivated.

Reviewer comment (Novelty):
The previous work of [Zhu et al. 2022] studies the setting with prior distribution of the agent's type. Their open question is also posed in that setting, so the claim in this paper that it "solves an open problem by Zhu et al." is an overclaim.

Author response:
We agree with the Reviewer. In the final version of the paper, we will make explicit that our work provides an answer to the open question posed by Zhu et al. (2022) in the specific case in which there is a single agent's types.  Nonetheless, the lower bound by Zhu et al. (2022) holds for the setting with a single type, proving that the setting with arbitrary number of actions is intractable even in single-type instances. This suggests that at least some of the main difficulties of the problem are present in instances with a single type, motivating our study. We believe that our result is a first milestone towards answering the open question by Zhu et al. (2022) in more general multi-type settings. Indeed, when seeking for solutions to open problems, it is reasonable to make the first attempts in settings that are more specific than those in which the question was originally posed.

Reviewer comment (Novelty):
Theorem 1 should rather be an observation or proposition than a theorem, since it is a quite obvious fact in learning optimal contracts, even though it might be mentioned explicitly in previous work.

Author response:
We agree with the Reviewer that the result in Theorem 1 is somehow expected. Nevertheless, to the best of our knowledge, no previous works have formally proved it. Moreover, the proof is not so trivial to be done formally. That said, we agree with the Reviewer that turning Theorem 1 into a proposition is the right choice.

Reviewer comment (Theory):
Definition 1 should rather be an assumption than a definition. I also do not see any benefit of treating this condition as a high probability event. I think you are essentially restricting the learning problem to a subclass of instances where the approximately optimal contract has bounded payment.

Author response:
We believe that the Reviewer is not correctly interpreting Definition 1. Indeed, the definition formalizes the problem solved in our paper, which is the one of learning an approximately-optimal bounded contract, adopting a PAC-learning-inspired perspective. We remark that this problem is meaningful in any instance (also those in which the optimal contract sets payments above the bound).
 Moreover, notice that the `with-high-probability' requirement in Definition 1 is necessary in order to meaningfully define the learning task. This is different from what we believe the Reviewer is considering, i.e., assuming to restrict the attention to problem instances where there exists an approximately-optimal contract with bounded payments. As a result, we think that Definition 1 should remain a definition (and not turning into an assumption). Moreover, let us also remark that our problem (Definition 1) is consistent with what has been studied in previous works (see, e.g., the work by Zhu et al. (2022)).

Reviewer comment (Theory):
Do we know any lower bounds for the sample complexity of learning optimal contracts when the action size is small? Do you think it is possible to improve the current method beyond the constant action size case (possibly under other assumptions)?

Author response:
When the number of actions is constant, our algorithm provides a bound that is polynomial in the instance size. We think that this is to be considered as satisfactory and tight (meaning that there exists a trivial lower bound polynomial in the instance size). It is known that without additional assumptions the sample complexity is exponential (Zhu et al., 2022).
 Nevertheless, we believe that this lower bound can be circumvented under some assumptions.
 For instance, assuming that the best-response regions either have a finite constant volume or are empty, we conjecture that it is possible to design an algorithm that depends on the smallest of such volumes and do not require an exponential dependence on the number of agent's actions.