Sponsored Question Answering

Venue: ACM_org_SIGIR_ICTIR_2024_Conference
Published: 2024-04-25 21:33:17

Abstract: The potential move from search to question answering (QA) ignited the question of how should the move from sponsored search to sponsored QA look like. We present the first formal analysis of a sponsored QA platform. The platform fuses an organic answer to a question with an ad to produce a so called sponsored answer. Advertisers then bid on their sponsored answers. Inspired by Generalized Second Price Auctions (GSPs), the QA platform selects the winning advertiser, sets the payment she pays, and shows the user the sponsored answer. We prove an array of results. For example, advertisers are incentivized to be truthful in their bids; i.e., set them to their true value of the sponsored answer. The resultant setting is stable with properties of VCG auctions.

Main paper: 
 introduction: 
 In sponsored search [9] , advertisements (henceforth referred to as ads) are ranked in response to a query. A click on an ad usually leads to a landing page on the Web. In contrast to organic search, the ranking is not based on relevance estimation but rather on monetization criteria. For example, by the foundational General Second Price (GSP) Auction [7, 18] , ads are ranked by bids posted by the advertisers 1 where the payment of an advertiser is the bid posted by the advertiser ranked just below her. More evolved versions include measures of ad quality (e.g., predicted click through rate) in the ranking criterion [7, 18] .Recently, there is an on-going discussion, and some implementations, of using question answering systems as potential alternatives to search engines [2, 4, 12] . This is due to the dramatic progress with large language models (LLMs) [21] . An emerging question is then how sponsored search will evolve with the transition from search to question answering. A few recent suggestions include showing ads during a QA conversation (chat), providing the ad as an answer in a QA session, and integrating an answer with an ad [1, 4, 8] .However, to the best of our knowledge, there is still no theoretical treatment of potential mechanisms for sponsored QA. In this paper, we make a first step towards this end. We present a formal auction mechanism for a sponsored QA platform. The use of ads in the QA platform we study is inspired by Feizi's et al. [8] conceptual proposal and by Microsoft's alleged approach [15] to fuse (integrate) answers with ads. For example, suppose the question "Which book is used in most CS programs in the U.S. to teach Python?" has an answer "Book X" generated by some LLM. Suppose there is an ad about a specific bookstore Y: "Bookstore Y has all the programming books one might need to thoroughly learn how to program. For example, we highly recommend book X which consistently receives excellent reviews.". The result of fusing the answer and the ad can be: "In bookstore Y you can find book X which is used by most CS programs in the U.S. to teach Python. The book consistently receives excellent reviews. "Our suggested QA platform operates as follows. Given a user question, the platform generates an organic answer which is independent of monetization considerations, e.g., using an LLM. The platform fuses the answer with each of the candidate ads to produce so called sponsored answers. Then, each advertiser places a bid for her sponsored answer. Inspired by the GSP auctions used in sponsored search [18] , we devise a criterion for selecting the sponsored answer and the payment the selected advertiser should pay. We prove that a dominant strategy of advertisers is to place a bid equal to the value of the sponsored answer for them. This result which is a property of VCG auctions [11] is important since the auction setting is in a stable (equilibrium) state: advertisers are incentivized to be truthful in their bids rather than engage in shading [14] -i.e., continuously changing bids to improve gains.We further define a notion of social welfare which is an aggregate of the value of a sponsored answer to the advertiser and the user utility attained by this answer. Then, in the spirit of VCG auctions [11] , we show that under the dominant strategy, the utility of the winning advertiser (defined as the difference between the value and the payment) is the difference between the social welfare given her sponsored answer and the social welfare given the sponsored answer of the second best advertiser whose bid is used to determine the winner's payment.Our formal analysis is agnostic to the fusion approach employed to integrate the organic answer and an ad. We discuss a few potential fusion methods using large language models (LLMs). However, formally coupling the result of LLM-based fusion and the auction mechanism is extremely difficult to impossible. Hence, to provide an end-to-end formal analysis of the process that starts with fusion and continues with the auction, we use unigram language models. We formally show that the advertiser with the highest bid is not necessarily the winner and that the winning sponsored result does not necessarily maximize user utility. The latter result can be attributed to the fact that we devise the platform to maximize both users' and advertisers' satisfaction.To summarize, our main contributions are (i) presenting the first -to the best of our knowledge -formal proposal of a sponsored QA system which is based on an auction mechanism, and (ii) proving a few important theoretical results about the platform.We assume that the QA platform generates an organic answer to a question, e.g., using a large language model (LLM). Indeed, question answering is a prominent application of using large language models [21] . We note that answer generation is not the focus of our work here. Any LLM can be used to generate the organic answer in our suggested platform.Sponsored search is central to the monetization of search engines [3] . In sponsored search, ads are ranked in response to a query. In contrast, following recent proposals [8, 15] , we assume that an ad is fused with an organic answer to a question.Two prevalent auctions on the Web are VCG [11] and Generalized Second Price (GSP) auctions [7, 18] . These auctions are equivalent when a single advertisement is selected [7, 19] . In this case, the highest bidding advertiser wins, and the payment is the minimum bid necessary to secure the win, specifically, the bid of the advertiser with the second highest bid. Our suggested auction is adaptation of a GSP auction to the sponsored QA setting. Since our QA platform strives to satisfy both users and advertisers, the winning advertiser is not necessarily the one who posted the highest bid as we show.There is work on content generation using multiple LLMs where an incentive-compatible auction2 is held for each generated token [6] . Our platform is fundamentally different as bids are posted for a given question and a sponsored answer created from an ad and an organic answer. Furthermore, in contrast to our work, there is strong coupling between the content generation approach and the auction [6] . Our platform is agnostic to the QA algorithm used to generate an organic answer and the approach used to fuse it with an ad. Furthermore, in contrast to Duetting et al. [6] , we define and analyze social welfare.Dubei et al. [5] devise an auction mechanism where ads of advertisers are summarized using an LLM. The advertisers essentially bid on the prominence of their ad in the summary. In contrast, in our QA platform a single sponsored answer created from an ad and an organic answer is presented to the user. Hence, the auction mechanisms are completely different and so is the corresponding analysis.Recently, Feizi et al. [8] proposed a conceptual framework for online advertising in LLM-based QA systems. They discuss important considerations including privacy, latency, reliability, users' and advertisers satisfaction. However, no formal/algorithmic framework is proposed in their work. We adopt Feizi et al. 's [8] proposal of fusing an organic answer with an ad which is also allegedly the case in some implementations [15] . Our focus in this paper is on devising an auction mechanism and analyzing it using several perspectives. The important issues discussed by Feizi et al. [8] are outside the scope of this paper, but certainly deserve in-depth future exploration. A case in point, the reliability of sponsored answers generated from organic answers and ads is highly important. 
sponsored question answering platform: 
 We now turn to present a question answering (QA) platform which responds to users' questions with answers which include sponsored information.Suppose that a user posts a question q and the QA platform generates an organic (textual) answer d o which is not affected by monetization considerations. There are n advertisers (content providers) interested in presenting their textual advertisements (ads in short) in response to q. 3 We assume that each advertiser i (∈ N := {1, 2, ..., n}) has a single ad d i she wants to present. The platform fuses for every advertiser i her ad (d i ) with the organic answer d o to produce a sponsored answer d s i which will be shown to the user in case advertiser i is selected. The fusion process can be advertiser-specific and be based on commercial terms between the platform and the advertiser. A case in point, the relative emphasis on the organic answer versus that on the ad can be the result of a commercial agreement. The platform then runs an auction in which the advertisers bid on the corresponding sponsored answers. The platform uses the bids with additional information to determine the winner of the auction and her payment.Since auctions are essentially games composed of players (bidders) and their strategies (bids) we start by describing in Section 3.1 some basic game theory concepts. In Section 3.2 we formalize the auction mechanism and present some theoretical results. The formalism and results are agnostic to the actual fusion approach employed to fuse an organic answer and an ad. In Section 4 we discuss two options of fusion using large language models, and in addition provide a theoretical analysis of the results of using unigram language models for fusion.We now briefly review some basic concepts in game theory. Definition 1. An n-players game is a tuple G = ({S i } i ∈N , {U i } i ∈N ). S i is the set of strategies of player i. S = S 1 × S 2 × ... × S n is the set of strategy profiles in the game. U i : S → R + is the utility function of player i. 4Each player in a game aims to maximize her utility. Note that the utility depends not only on her strategy but also on the strategies of other players. Consider the following 2-players game:S 1 = S 2 = [0, 1], U 1 (s 1 , s 2 ) = 1 + s 2 1 + s 2 2 , U 2 (s 1 , s 2 ) = 2 + s 2 1 + s 2 2 .If the players select s 1 = 1 and s 2 = 1, then their utilities are U 1 (1, 1) = 3 andU 2 (1, 1) = 4.A fundamental characterization of games is their stability or lack thereof. A stable strategy profile of a game is a profile where no player has an incentive to deviate from her strategy. A well known example of a stable profile is Nash equilibrium. LetS -i = S 1 × ... × S i-1 × S i+1 × ... × S n denote the strategy profile of all the players except i. Then, Definition 2. A strategy profile s = (s i , s -i ) where s i ∈ S i , s -i ∈ S -i is a Nash equilibrium if ∀i ∈ N and ∀s ′ i ∈ S i , U i (s ′ i , s -i ) ≤ U i (s i , s -i ). 5Considering the game example from above, it is easy to show that the strategy profile s 1 = 1, s 2 = 1 is a Nash equilibrium.A dominant strategy of a player is a strategy that is better than any other strategy regardless of the strategies of other players:Definition 3. s i ∈ S i is a dominant strategy of player i if ∀s -i ∈ S -i , ∀s ′ i ∈ S i , U i (s ′ i , s -i ) ≤ U i (s i , s -i ).Note that players' strategies in a Nash equilibrium (Definition 2) need not necessarily be dominant strategies. In our example game,s 1 = 1 is a dominant strategy of the first player, since U 1 (1, s 2 ) ≥ U 1 (s 1 , s 2 ) ∀s 1 , s 2 ∈ [0, 1].A common measure of the "goodness" of a strategy profile s is social welfare, often defined as the sum of the players' utilities:SW (s) = i ∈N U i (s).(1)We next turn to formalize the auction employed by the QA platform.Recall that prior to the auction, each ad d i is fused with the organic answer d o to yield a sponsored answer d s i . Advertiser i then places a bid for d s i to be shown. The platform selects the sponsored answer to show and determines the payment of the winning advertiser. Definition 4. A question-specific setup of a question-answering (QA) platform with sponsored answers is a tupleQAS := (q, d o , N , {d i } i ∈N , {d s i } i ∈N , {v i (d s i )} i ∈N ). q and d o are a question and the organic answer generated for it, respectively. N := {1, 2, ..., n} is a set of advertisers where n ≥ 2. {d i } i ∈N are the advertisers' ads for q. d s i is the sponsored answer generated by fusing the organic answer d o with ad d i . v i (d s i ) ∈ R + is the value for advertiser i of displaying the sponsored answer d s i to the user.We assume the QA platform has to select a single sponsored answer from {d s i } i ∈N to show the user. This practice is aligned with a setting where the platform shows a single organic answer. As in classical sponsored search [9] , the platform is incentivized to receive a payment based on the content it presents to users. We now turn to define the utilities of the stakeholders -users, advertisers and the platform itself -and propose an auction mechanism to determine the winner and her payment.We focus on a setup where the (game) strategy s i of every advertiser i is a bid b i (d s i ) (∈ R + ) for showing the sponsored answer d s i to the user. The set of strategy profiles of the advertisers is S = S 1 × S 2 × ... × S n where S i is the set of all possible bids of advertiser i; namely, R + . We adapt GSP (generalized second price auction) [18] to our setting for the allocation and payment procedures.The platform's value for a sponsored answer d s i is an aggregation of the user utility and the advertiser's bid on d s i :Definition 5. The prospect Platform Value (henceforth, platform value in short) from displaying the sponsored answerd s i is defined as PV (d s i , q, b i (d s i )) = U (d s i , q) + b i (d s i ), where U (d s i , q) is the user utility attained by seeing the sponsored answer d s i in response to her question (q), and b i (d s i ) is the bid of the advertiser i on the sponsored answer d s i .We use the term "prospect" to emphasize that the payment received by the platform can be lower than b i (d s i ) as explained below. We next define social welfare with respect to a sponsored answer as the sum of the user utility and the value for the advertiser whose sponsored answer was selected. In standard GSPs for sponsored search [18] , the social welfare is a weighted sum of advertisers' values (cf., Equation 1) where the weights represent click probabilities 6 . Here we do not account for click probabilities. We do take into account, in contrast to standard GSPs, the user's utility. Definition 6. The Social Welfare from displaying the sponsored answerd s i is SW (d s i , q, b i (d s i )) = U (d s i , q) + v i (d s i ).Note that the social welfare is composed of a user part (utility) and advertiser part (value). Indeed, the platform aims at satisfying both users and advertisers.The advertiser selected as the winner of the auction, i w , is the one whose sponsored answer and bid maximize the platform value:EQUATIONThe platform shows the sponsored answer d s i w . In GSP auctions [18] , the winner pays the minimal paymentp i w (d s i w ) ≤ b i w (d s i w )needed to surpass the second best platform value, attained by advertiser i snd . Formally,PV (d s i s nd , q, b i s nd (d s i s nd )) ≤ PV (d s i w , q, b i w (d s i w )) and PV (d s i s nd , q, b i s nd (d s i s nd )) = PV (d s i w , q, p i w (d s i w )).Based on these definitions, we arrive to the following result:Lemma 1. The payment p i w (d s i w ) of the winner i w is b i s nd (d s i s nd )+ U (d s i s nd , q) -U (d s i w , q).Proof. The winner i w pays the minimal p i w (d s i w ) s.t. the resultant platform value is the second best. SincePV (d s i s nd , q, b i s nd (d s i s nd )) = PV (d s i w , q, p i w (d s i w )), then U (d s i s nd , q)+ b i s nd (d s i s nd ) = U (d s i w , q) + p i w (d s i w ) and therefore p i w (d s i w ) = b i s nd (d s i s nd ) + U (d s i s nd , q) -U (d s i w , q). □We now define the utilities of advertisers. Note that the utility of an advertiser depends on the bids of all other advertisers, not only her own; specifically, the payment of the winner depends on the bid of another advertiser. Indeed, recall from Section 3.1 that the utility of a player in a game depends on the strategies employed by all players.Following common practice in auction theory [13] , we assume quasi-linear utility functions of the advertisers: we substract the payment from the value to determine the advertiser utility if she is the winner, and set the utility to 0 otherwise. Definition 7. The utility of advertiser i is:U a i (d s i , v i (d s i ), {d s j } j i , {v j (d s j )} j i ) =      v i (d s i ) -p i (d s i ) if i wins, 0 otherwise; (3) p i (d s i ) is i's payment in case she wins.The auction setting we defined can be analyzed as a game where the advertisers' strategies are their bids and they strive to maximize their utility. We next show that advertisers have an incentive to behave truthfully, that is, bid the true value of their sponsored answer. More specifically, this bidding strategy is a dominant strategy. As a result, the game (auction) has a Nash equilibrium (i.e., the game is stable). This stability is highly important. A case in point, in auctions which are not incentive-compatible (i.e, bidders are not incentived to bid their true value) shading phenomena [14] are prevalent: bidders continuously addapt their bids (specifically, lower than the true value) to maximize gain.Lemma 2. b i (d s i ) = v i (d s i) is a dominant strategy of advertiser i.Proof. We show that the utility of player i is maximized when bi (d s i ) = v i (d s i ). Let M = max j i PV (d s j , q, b j (d s j)) be the maximal platform value attained by selecting a player i ′ i. There are two options for the utility of advertiser i:• i is not the winner:PV (b i (d s i )) ≤ M. In this case, the utility of player i is U i (b i (d s i )) = 0 and b i (d s i ) = v i (d s i ) is a dominant strategy. • i is the winner: PV (b i (d s i )) > M. Hence, j = i snd and M = b i s nd (d s i snd ) + U (d s i snd , q). By Lemma 1, p i (d s i ) = M -U (d s i , q). The utility of i is then v i (d s i ) -p i (d s i ) = v i (d s i ) -M + U (d s i , q).We analyze two cases :-if v i (d s i ) -M + U (d s i , q) < 0 then v i (d s i ) < U (d s i s nd , q) + b i s nd (d s i s nd )-U (d s i , q) and v i (d s i )+U (d s i , q) < b i s nd (d s i s nd )+ U (d s i s nd , q). If b i (d s i ) ≤ v i (d s i ), we get b i (d s i ) +U (d s i , q) < b i s nd (d s i s nd ) + U (d s i snd , q).Consequently, i snd is the winner and we get a contradiction.-if v i (d s i )-M +U (d s i , q) ≥ 0,v i (d s i ) -p i (d s i ) ≥ 0; to this end, i can bid b i (d s i ) = v i (d s i ) since v i (d s i ) +U (d s i , q) ≥ M. □It is easy to show that the utility of the winner is the difference between the social welfare if her sponsored answer is selected and the social welfare if the sponsored answer of the second best advertiser (in terms of platform value) is selected. This is a property of VCG auctions [11] . Formally, Proposition 1. Assume all advertisers play their the dominant strategy: b i (d s i ) = v i (d s i ). The utility of the winner i w is:v i w (d s i w ) - v i s nd (d s i s nd ) + U (d s i w , q) -U (d s i snd , q) = SW (d s i w , q, b i w (d s i w )) - SW (d s i s nd , q, b i s nd (d s i snd )). 
from ads to sponsored answers: 
 Heretofore, we treated the fusion of the organic answer d o and the ad d i to yield a sponsored result, d s i , as a black box. That is, advertiser i bids on d s i based on the value v i (d s i ) it attributes to it. Then, the QA platform selects the sponsored answer that yields the highest platform value (see Definition 5) . We now turn to describe possible fusion methods and discuss their implications.A possible approach to fusing the organic answer d o and an ad d i is to request (via a prompt) a large language model (LLM; e.g., GPT) to perform the fusion. However, it is impossible to make theoretical statements about the resultant sponsored answer, d s i . Furthermore, it is highly difficult, to impossible, to quantitatively control the relative emphasis on d o versus d i in a prompting-based fusion approach as just described. This aspect can be quite important for both the advertiser and the user of the QA platform: the advertiser strives to have the sponsored answer d s i similar to d i and the user probably wants it to be similar to the organic answer, d o .We re-visit this point below. We now consider an alternative approach to fusion inspired by a recent proposal of integrating multiple LLMs for content generation [6] . Suppose advertiser i generates the (textual) ad d i using an LLM, denoted θ ad . Suppose also that the QA platform uses an LLM, θ or д , to generate the organic answer d o for the question q which is used as the prompt. Assume that both LLMs are autoregressive. That is, given a sequence of tokens x (composed of the prompt and tokens already generated), there is a probability distribution over the token vocabulary V from which the next token is sampled. Formally, p(t |θ, x ) is the probability of token t given the LLM θ and the sequence x already generated. Then, we can fuse the organic answer d o and the ad d i by (i) defining a next-token generation distribution: λp(t |θ or д , x ) + (1-λ)p(t |θ ad , x ), where λ is a parameter controlling the relative emphasis on the organic answer and the ad; and (ii) using the original prompts in each of the two LLMs: the request to generate the ad (θ ad ) and the question (θ or д ).There are three main challenges embodied in the fusion process just described. First, there is no guarantee about the quality of the generated sponsored answer, d s i , since we mix distributions at the token level. Second, we assumed that an ad was generated using an LLM which is not necessarily the case. Third, we still cannot theoretically reason about the generated sponsored answer and formally use it in the advertiser value function and user utility function defined above.To facilitate a first step towards a formal end-to-end (theoretical) analysis of the process of fusing d o and d i to produce d s i , and using d s i in the auction defined in Section 3.2, we subscribe to the language modeling framework to retrieval [10, 16] . Specifically, we use unigram language models based on a multinomial distribution [10, 16, 17] .Let x be a text and t a term (token) in the vocabulary V . p x (t ) := t f (t,x ) t ′ ∈x t f (t ′ ,x ) is the maximum likelihood estimate of t with respect to x assuming a multinomial distribution; t f (t, x ) is the number of times t appears in x. p x (•) is the unsmoothed unigram language model induced from x. The probability assigned to a term sequence t 1 , . . . , t n is p x (t 1 , . . . , t n ) := n i=1 p x (t i ). Often, unigram language models are smoothed to avoid the zero probability problem [20] . For our formal analysis herein, smoothing is not required. We hasten to point out however that our findings also hold for smoothed language models.Given an organic answer d o and an ad d i , we can fuse the unigram language models induced from them using a linear mixture to produce a unigram language model from which a sponsored answer will be generated:p spon i (t ) := λ i p d o (t ) + (1 -λ i )p d i (t ).Note that λ i is advertiser-specific and can potentially be negotiated between the advertiser and the QA platform before the auction takes place. We can now generate a specific sponsored answer d s i of length k by sampling k times from p spon i (•). The mean number of occurrences of token t in a document of length n generated using p spon i (•), i.e., in a sponsored answer, is np spon i (t ). 7 Herein, we use d s i to denote the "mean" document which is composed of these mean number of occurrences of tokens, henceforth simply referred to as the sponsored answer. As was the case of fusing LLMs at the next-token generation level, there is no guarantee about the quality of the generated sponsored answer. We hasten to point out that our use of unigram language models is intended to facilitate the formal analysis of the auction mechanism given a specific fusion method.Advertiser's value, user's utility and text similarity. We recall that advertiser i is interested in having her ad d i presented for the question q. Instead, the platform suggests showing the sponsored answer d s i which is the result of fusing d i with the organic answer d o . Naturally, the advertiser strives to have the sponsored answer as similar as possible to her ad. Hence, we define the advertiser's value function used in Definition 5 asEQUATIONwhere S is an inter-text similarity measure.Similarly, we assume that the goal of the user who posted the question q so as to (presumably) receive an organic answer is to have the sponsored answer d s i as similar as possible to the organic answer d o . Accordingly, we define the user utility function used in Definition 5 asEQUATIONWe define the similarity of texts x and y based on the cross entropy measure (cf. [10] ): EQUATIONCE(p x (•)||p y (•)) = t ∈V p x (t ): A := max j 1 , j 2 CE(p d j 1 (•)||p d j 2 (•)) + CE(p d j 2 (•)||p d j 1 (•)) where d j 1 , d j 2 ∈ {d o } ∪ {d i } i ∈N .Auction Analysis with Unigram Language Models. We now turn to show that in the unigram-language-model setting described above, the advertiser i who wins the auction is not necessarily the one with the maximal value v i (d s i ). Since by Lemma 2 the dominant strategy is to set the bid as the value, we arrive to an interesting result: the advertiser who placed the highest bid is not necessarily the one who wins the auction. This result is in contrast to the stateof-affairs in many other auction mechanisms, and is due to the QA platform's goal to satisfy both users and advertisers. Proposition 2. The winner of the auction is not necessarily advertiser i whose value v i (d s i ) is the maximum with respect to all advertisers' values. Accordingly, the winner is not necessarily the advertiser who placed the highest bid. 8 We use cross entropy rather than KL divergence because the cross entropy is linear in its left argument. Note that the resultant similarity function is concave. 9 As noted above, usually language models are smoothed so as to avoid zero probabilities (e.g., in cross entropy computation). For our purposes (constructive proofs presented below), smoothing is not needed as there are no cases of zero probabilities. Following the fusion approach described above, p sd i (t ) = λ i p d 0 (t )+ (1 -λ i )p d i (t ) for i ∈ {1, 2} and t ∈ {a, b}. We set λ 1 = ϵ and λ 2 = 1 -ϵ. It can be shown that for ϵ → 0, S (d s 1 , d 1 ) = v 1 (d s 1 ) > S (d s 2 , d 2 ) = v 2 (d s 2 ) and v 2 (d s 2 ) + S (d s 2 , d 0 ) > v 1 (d s 1 ) + S (d s 1 , d 0 ); i.e., v 2 (d s 2 ) + U (d s 2 , q) > v 1 (d s 1 ) + U (d s 1 , q). Thus, d 2 wins the auction but the value for d 1 is higher. The full proof is provided in Appendix A.1. Since by Lemma 2 bidding the value is a dominant strategy, we get that the winner of the auction is not the one who placed the highest bid. □We next show that the user utility is not necessarily the maximal possible. This is, again, due to the fact that the QA platform aims to satisfy both users and advertisers. Proposition 3. If advertiser i won the auction, it is not necessarily the case that the resultant user utility, U (d s i , q), is the maximal with respect to that attained by selecting other advertisers (sponsored answers).Proof. We consider the same 2-advertisers setting as in Proposition 2 with the same ads and organic answer. The difference is that now we set λ 1 = 1 -ϵ and λ 2 = 0.5.It can be shown that for ϵ → 0, U (d s 1 , q) = S (d s 1 , d o ) > S (d s 2 , d o ) = U (d s 2 , q) and v 2 (d s 2 ) + U (d s 2 , q) > v 1 (d s 1 ) + U (d s 1 , q). That is, i = 2 is the winner of the auction but the resultant user utility is lower than that for advertiser 1. The full proof is provided in Appendix A.2. □ 
conclusions and future work: 
 We presented a novel formal platform for sponsored question answering (QA). The platform is based on fusing an organic answer to a question with an ad so as to produce a sponsored answer. Advertisers bid on their corresponding sponsored answers. Inspired by principles of Generalized Second Price Auctions (GSPs) [7, 18] , the platform selects the sponsored answer to show the user and sets the payment for the selected advertiser. We prove that a dominant strategy of advertisers is to bid on their true value of the sponsored answer which is a property of VCG [11] auctions. The result is that the QA setting reaches a stable state (equilibrium) where advertisers have no incentive to continuously change their bids (a.k.a., shading). We also formalize the notion of social welfare and show that the utility of the advertiser who wins the auction is the difference between the social welfare attained when presenting her sponsored answer and the social welfare of presenting the sponsored answer of the second best advertiser whose bid is the payment the winner has to pay.Our general analysis of the auction is not committed to a specific approach to fusing an organic answer and an ad. To theoretically analyze end-to-end the process of fusing an organic answer with an ad and apply the auction, we use unigram language models. We prove that the auction winner is not necessarily that with the highest bid. We also prove that the attained user utility is not the maximal possible with respect to selecting other advertisers. This result is due to the fact that we design the QA platform to satisfy both users and advertisers. 
 
 Appendix: Advertisers bid on keywords. If a keyword appears in a query then the ad is candidate for ranking.In these auctions, bidders are incentivized to bid their true value.The advertisements can include images and video captions, but henceforth we focus on the textual part,R + is the set of non-negative real numbers.We focus on pure strategies and pure Nash equilibrium. Mixed strategies which are distributions over pure strategies are outside the scope of this paper.In standard GSPs for sponsored search, ads are ranked. In our setting, a single ad is presented, and only its advertiser has potential gain. 


Decision: Motivated by the move away from search toward question answering, the submission models and theoretically analyzes a platform for sponsored question answering. The proposed platform fuses an organic answer with an advertisement using, for instance, a unigram language model or a LLM. The model considers platform value, advertiser value as well as user utility. Advertisers bid on a fused answer based on a generalized second-price auction mechanism. One key finding of the submission is that a dominant strategy for advertisers is to bid based on their true value, which avoids effects such as shading. In a subsequent analysis, the submission considers a concrete fusion mechanism, namely a unigram language model which linearly interpolates token probabilities estimated from the organic answer and the ad. An interesting finding here --under the assumed fusion mechanism-- is that the winning advertiser is not necessarily the one having the maximal value.

All reviewers agreed that the submission addresses an interesting and timely problem. It is clear from their reviews, that the reviewers regard the submission as a first formal step with potential for follow-up work and analyses. They also pointed out several issues, which the authors should address. This includes the ethics and desirability of such a platform in general, several fixable presentations issues, and a more complete discussion of related work.

Reviews: review: main_review: main_review: Minor: 

- A detailed (running) example would greatly help digest the content of the paper...
main_review: main_review: I quite liked the paper, but am not an expert in advertising for online services, so it could just be novelty for me. The authors could do a better job at situating their contribution in the literature (the related work is quite good though - so it may also just be my superficial degree of knowledge from this research niche).
main_review: main_review: The approach appears to be sound, but I admit that I haven't checked in detail all the proofs.

- The introduction and related work seems to mix a couple of different types of systems, including search engines, conversational agents, and QA platforms. It might be better to simply focus on a QA platform throughout, pointing out that sponsored search is relevant across all these systems.

- The notion of social welfare could perhaps be better explained. As I understand this is something that the QA platform optimizes for, but it is actually up to the platform how they balance the trade-off between advertiser value and user utility. Currently, these two seem to be considered with equal weights. If the platform chose, say, to place more emphasis on satisfying advertisers, would that change the findings?
main_review: main_review: I find the work very interesting as a discussion and analysis
However, I find it difficult to relate to real-life aspects due to lack of implementation and evaluation

Moreover, the theory part presented is not motivated much
It is not clear why different definitions and lemmas needed
at some point one may question, whether we are doing any better than already existing methods like VCG

having said this, I found the idea of social welfare very interesting