Paper: Rethinking the Buyer’s Inspection Paradox in Information Markets with Language Agents (ICLR-2024)

Reviewer comment (Experiments):
The paper only compares the performance between agents powered by different LLMs. Though the "Evaluating the Evaluator’’ experiment made some comparison between GPT4 and humans as evaluators, what about baselines based on algorithmic approaches, e.g., a buyer agent that designs quoting strategy based on keyword-matching? The ability to forget the information from rejected quotes can also be artificially planted in these algorithms. Hence, without these algorithmic or heuristic baselines, it is unclear to me how it is necessary to use LLM based agents in this task.

Author response:
As you suggested, we added a new baseline experiment with a keyword matching baseline (BM25) and found that 95% of the time it was outperformed by LLama-2-70B. 

1. For 95% of the questions, the LLM-powered buyer agent’s answers (based on Llama-2-70b) are preferred to the BM25 agent’s answers by the evaluator. This verifies that leveraging LLMs can significantly boost the quality of the generated answers.
2. Increasing the budget for the BM25 heuristic yields better results — the GPT-4 evaluator prefers answers from the high-budget simulations for 67% of all questions (vs. low-budget simulations). This result serves to verify that the simulated marketplace functions as expected. 

We have updated the manuscript accordingly (Appendix C).

Reviewer comment (Theory):
The paper focuses on the inspection paradox, but the procedure of inspection, especially in the context of information market, is oversimplified — it is basically whether to allow the agent to read all the text content. However, there is a rich line of econ/cs/ml literature (see e.g., [1, 2, 3]) on the information market that concerns the power of information design for buyer inspection. For example, the seller could only give out a summary of the content, or only answer certain query questions from the buyer agent — because it is unclear whether it is reasonable or enforceable to trust the buyer to forget all the information from rejected quotes in reality.

Author response:
In our design, we operate under the assumption that sellers trust the marketplace but not necessarily the buyers. This is because the marketplace is controlled by software that implements checks to ensure legal actions by the buyer agents. For instance, the software guarantees that the buyer stays within budget and cannot leak information. This means that the buyer agent can try to buy information outside its budgetary constraints, but the software will not permit that behavior. Therefore, the need to trust buyers is eliminated as the marketplace software provides data security, facing the same set of vulnerabilities as other online businesses. This risk can be reduced through standard cybersecurity practices such as regular security audits, threat modeling, implementation of security protocols, and the addition of compliance layers.

Crucially, the buyer agents can decide whether to buy information after they have received it. They will never commit to buying information before having seen all of it. This approach provides a mechanism for vendors to safely increase the amount of inspection they allow, trusting only the marketplace. While we did not investigate all possible inspection mechanisms, we demonstrated the benefit of allowing more inspection in our study. Specifically, we showed the advantage of allowing content inspection versus metadata-only inspection. This eliminates the need for sellers to strategically withhold or present information, which is one of the key benefits of our approach. We hope that future work builds on the information bazaar concept to evaluate the efficacy of these alternative vendor strategies.

Reviewer comment (Experiments):
To my knowledge, there is no clear evidence so far that an LLM based agent has any reasonable rationality or strategic reasoning skill — GPT4 cannot play tic-tac-toe, or even reliably compare two numbers. This is also verified in the paper's experiment. Hence, I am not sure whether it is meaningful for the paper to give LLM based agents the ability to make economic decisions — these decisions could be left for the human or some simple algorithmic procedures while only asking the LLM to provide its reasoning or evaluation scores on the value of the text.

Author response:
We agree that all the LLMs we evaluated display a degree of irrational behavior, and one contribution of this paper is to highlight these. We developed this debate prompting methodology to ameliorate some of those behaviors. To our knowledge this style of method was first published in a blogpost titled **[The Socratic Method for Self-Discovery in Large Language Models](https://princeton-nlp.github.io/SocraticAI/)** on May 5, 2023 by Runzhe Yang and Karthik Narasimhan — **we cite this in our work** (Section 3.4, paragraph on “Debate Prompting”). In contrast to their contribution, our method underscores the importance of adaptable character shaping within the debate, providing the opportunity to balance the debate dynamics by offering tactical hints to the respective characters.

The key conceptual idea in this work is that of a buyer agent that is guaranteed to forget (by the marketplace) and can therefore be safely granted access to all information. Such buyer agents can be implemented in different ways, we focus on an LLM-based proof of concept because they are currently by far the most powerful agents for operating on textual data (besides humans, but erasing their memory would be ethically problematic). We completely agree that LLMs fail to act rationally in many cases, but they are rapidly becoming more powerful and there is a growing body of literature on LLMs as economic agents (see e.g. [Large Language Models as Simulated Economic Agents: What Can We Learn from Homo Silicus?](https://arxiv.org/abs/2301.07543)). Any advancements in LLMs will directly translate to improved performance of our approach.

Reviewer comment (Theory):
The idea of protecting the information lies in the belief in LLM. However, LLM may face data leakage risks. The authors need to clarify whether it is secure for LLMs to access the information, even if only metadata is accessed.

Author response:
Thank you for asking this question. To clarify, we do not rely on the LLM to avoid data leakage. Data security is provided by classical software running the marketplace and faces the same set of vulnerabilities as other online businesses. The buyer agent only finds and buys information. The marketplace software guarantees that the buyer stays within budget and cannot leak information. Here’s how this works:

1. **The buyer agent is entirely controlled by the marketplace**. The software that runs the marketplace implements checks that constrain the buyer agent to acting “legally”. It is this static code that ensures that only purchased information can leave the marketplace. This means, for example, that the buyer agent can try to buy information outside its budgetary constraints, but the software will not permit that behavior.
2. **Cybersecurity Measures**: The cybersecurity risk can be reduced through standard practices, such as regular security audits, threat modeling, implementation of security protocols, and the addition of compliance layers.

Reviewer comment (Novelty):
This paper primarily focuses on explaining how to transform LLM into an agent capable of performing rational actions in an information bazaar, but it lacks detailed explanations regarding using LLM. In other words, replacing LLM with any intelligent agent possessing information evaluation capabilities and trading intelligence would still be able to effectively address the problem of determining the value of information.

Author response:
We appreciate your insightful question. Our framework necessitates buyer agents to meet two primary conditions:

1. They must possess the capacity to accurately estimate the value of the information for the buyer. 
2. They must have the capability for their memory to be audited and manipulated in a hard-coded manner. 

The key ideas we present indeed apply to any such agent. LLMs are the most promising implementation of such an agent due to their unprecedented ability to operate on textual data. We added an experiment with a simpler, keyword-based agent to highlight this point.