# Analyzing the Dynamic Data Sponsoring in the Case of Competing Internet Service Providers and Content Providers

Authors: Mohamed El Amrani; Hamid Garmani; Driss Ait Omar; Mohamed Baslam; Brahim Minaoui
Journal: Mobile Information Systems
 (2021)

---

## Abstract

With a sponsored content plan on the Internet market, a content provider (CP) negotiates with the Internet service providers (ISPs) on behalf of the end-users to remove the network subscription fees. In this work, we have studied the impact of data sponsoring plans on the decision-making strategies of the ISPs and the CPs in the telecommunications market. We develop game-theoretic models to study the interaction between providers (CPs and ISPs), where the CPs sponsor content. We formulate the interactions between the ISPs and between the CPs as a noncooperative game. We have shown the existence and uniqueness of the Nash equilibrium. We used the best response dynamic algorithm for learning the Nash equilibrium. Finally, extensive simulations show the convergence of a proposed schema to the Nash equilibrium and show the effect of the sponsoring content on providers’ policies.


---

## Body

## 1. Introduction

One of the principal trends on the Internet in the last few years is the explosion in demand for cellular data usage. Therefore, one of the main challenges for CP is how to motivate end-users to access their content to achieve a higher profit. In addition, this increase in cellular data usage needs higher investments in wireless capacity. ISPs launched a new type of data pricing called data sponsoring to get additional revenue and to increase the capacity of their existing network architectures [1]. The critical idea of data sponsoring is to allow the CPs to pay the ISP on behalf of the end-users the network access fees. Data sponsoring plans benefit all entities in the network; the ISPs can generate more revenue with CP’s subsidies, and end-users can enjoy free network access to the CPs, which increases the demand and attracts more traffic, resulting in higher income of the CP. As a real-world example, AT&T allows advertisers to sponsor video to attract more end-users to watch advertisements [2, 3], and GS Shop (Korea TV) has cooperated with SK Telecom to sponsor the traffic of its application [4].Sponsoring data have recently been subject to modeling and analysis in the literature. The authors in [5] proposed a novel joint optimization approach of a Stackelberg contract game to characterize the market-oriented model for sponsored content market and to capture the interactions among the ISPs, CPs, and end-users. They have developed a Stackelberg game, where the ISP acts as the leader and the CPs and end-users act as the followers. In [6], the authors developed a new model to study the competition among CPs under sponsored data plans. The authors in [7] analyzed the interactions of three network entities, i.e., the end-users, the ISPs, and the CP, based on the game theory. The authors designed an effective data sponsoring control scheme using a novel dual-leader Stackelberg game model. The authors in [8, 9] investigated joint sponsored and caching content under the noncooperative game. The interactions among ISP, CP, and end-users are modeled as a three-stage Stackelberg game. In [10], the authors studied the sponsored data as the noncooperative game among ISPs. The authors derived the best response of the CP and the ISP and analyzed their implications for the sponsoring strategy. The authors in [11, 12] analyzed content sponsoring data from an economic point of view. They examined the implications of sponsored data on the CPs and the end-users and identified how the sponsored data influence the CP inequality. In [13], the authors studied the sponsorship competition among CPs in the Internet content market and demonstrated that the competitions improve the welfare of the ISP and the CP. The authors in [3] considered a sponsored data market with one ISP, one CP, and a set of end-users. They have modeled the interactions among three entities as a two-stage Stackelberg game, where the ISP and the CP act as the leaders determining the pricing and sponsoring strategies, respectively, in the first stage and the end-users act as the followers deciding on their data demand in the second stage. In [14], the authors analyzed the interaction among ISP and CPs and proposed a pricing mechanism for sponsored content that is truthful in CP’s valuation.The sponsored content plan has been extensively investigated during the past few years; most papers focus on a simple model with a single ISP and a single CP interacting in a game-theoretic setting; few works study the completion between multiple CPs and multiple ISPs with content sponsoring plans. However, to the best of our knowledge, none of the current works includes the time constraint that makes sponsoring content more dynamic. The present paper moves toward this less explored direction. The main objective of this paper is to study a sponsored content market consists of multiple CPs, multiple ISPs, and end-users, with the time constraint using game theory.Game theory has been used to solve many problems in communication networks [15–23]. It has been used to propose new pricing strategies for Internet services [24, 25]. Many other issues relating to wireless networks have been modeled and analyzed using game theory, such as resource allocation [26, 27], power control [28, 29], network routing [30], network caching [31], and security [32].The contributions of this paper are as follows:(i)
We present new features in the mathematical modeling that include sponsoring content, CPs revenues, and ISPs revenues with the time constraint.(ii)
We model the interplay among ISPs as a function of two market parameters network access prices and quality of service; each ISP wants to maximize its utility. We formulate a competitive problem between ISP as a noncooperative game.(iii)
We model the interplay between CPs as a function of three market parameters content access price, the credibility of content, and the number of sponsored content. The number of sponsored content is modeled as a function of time. We formulate a competitive problem between CPs as a noncooperative game.(iv)
We analytically prove the existence and uniqueness of the Nash equilibrium in the noncooperative game between ISPs and between CPs, which means that there exists a stable state where all providers do not have an incentive to change their strategies. The best response algorithm is used to find the Nash equilibrium point.(v)
Numerical analysis shows the effect of sponsoring content on providers’ policies.This paper is organized as follows. Section2 discusses the system model with temporality constraint. We prove the existence and uniqueness of a Nash equilibrium point in Section 3. Then, we present a numerical investigation in Section 4, and we conclude this paper in Section 5.


## 2. Problem Modeling

In our setting, we consider a telecommunication network with three types of actors: ISPs, CPs, and end-users. The ISP provides the network infrastructure to the end-users. The CPs provideN content for end-users and sponsor a fraction of content on behalf of the end-users to lower the network access price. The ISPk sets two decision parameter network access price psk and quality of service (QoS) qsk. Let pcf and cf, respectively, be the content access price and the credibility of content decided by CPf. End-users behavior is a function of CP and ISP policies (see (1)).


### 2.1. Demand Model

Dfk is the demand of end-users for the content provided by CPf and transferred by ISPk which is a function of content access price pcf, credibility of content cf, network access price psk, and QoS qsk (see [33, 34]). This demand function is also a function of prices pc−f, credibilities c−f, prices ps−k, and QoSs qs−k set by the opponents. The demand Dfk is decreasing with respect to pcf and psk and increasing with respect to pcg, g≠f and psj, j≠k, whereas it is increasing with respect to cf and qsk and decreasing with respect to cg, g≠f and qsj, j≠k. Then, the demand functions Dij can be written as follows:(1)Dfk=dfk−σffpcf+ςffcf−τkkpsk+ϱkkqsk+∑g=1,f≠gFσfgpcg−ςfgcg+∑j=1,k≠jKτkjpsk−ϱkjqsk.The parameterdfk is the potential demand of end-users. σfg and ςfg are two positive constants representing, respectively, the responsiveness of demand Dfk to content access price and credibility of CPg. Moreover, τgj and ϱgj are two positive constants representing, respectively, the responsiveness of demand Dfk to network access price and QoS of ISPj.Assumption 1.
The sensitivityσ verifies the following:(2)σff≥∑g=1,g≠fFρfg,∀f=1,…,F.
The sensitivityς verifies the following:(3)ςff≥∑g=1,g≠fFςfg,∀f=1,…,F.
The sensitivityτ verifies the following:(4)τkk≥∑j=1,j≠kKτkj,∀k=1,…,K.Assumption1 means that the effect of provider policies on its demand is greater than the effect of the policies of its opponent on its demand function [24, 35]. Assumption 1 will be used to prove the uniqueness of the Nash equilibrium.


### 2.2. Utility Model of the CP

The utility ofCPf can be modeled as follows:(5)UCPf=∑k=1KpcfNf1+χfSfkDfk−∑k=1KpufSfkNf1+χfSfkDfk−∑k=1KptkcfNf1+χfSfkDfk−θfcf,where Sfk is the fraction of content sponsored by CPf for each ISPk subscriber. Let Sfk=1 if full sponsoring is decided and Sfk=0 if CPf decides not to sponsor any content. Recall that sponsoring could be an incentive to consume more CPf content. 1+χfSfkDfk is the new demand for contents provided by CPf and distributed by ISPk, which is a function of the proportion of sponsored content Sfk. The quantity χfSfk reflects the change in the demand for the contents of the CP. χf is a nonnegative constant. θi is the cost to produce a unit of the credibility of content ci. ptk is the transmission price of ISPk. puf is the sponsoring price of ISPf. The first term ∑k=1KpcfNf1+χfSfkDfk is the revenue of CPf. The second term ∑k=1KpufSfkNf1+χfSfkDfk denotes the cost due to sponsorship. The third term ∑k=1KptkcfNf1+χfSfkDfk is transmission fee results when the CPf forwards to the end-users the demand with credibility of content cf. The fourth term θfcf is the cost to produce the credibility of content cf.Credibility of contentcf of CPf is a linear function of the quality of service (QoS) qssf and the quality content (QoC) qcf, which is written as follows [16, 34, 36, 37]:(6)cf=λqssf+μqcf,where λ and μ are nonnegative constants. The QoS is defined as the expected delay (see [17, 34]). The QoC can be specified for a specific domain of content (e.g., video streaming).Then, the utility ofCPf is expressed as follows:(7)UCPf=∑k=1KpcfNf1+χfSfkDfk−∑k=1KpufSfkNf1+χfSfkDfk−∑k=1KptkNf1+χfSfkDfkλqssf+μqcf−θfλqssf+μqcf.


### 2.3. Utility Model of the ISP

The utility function ofISPk is the difference between the revenue and the fee:(8)UISPk=∑f=1Fpsk1−SfkNfDfk+∑f=1FpukSfkNfDfk+∑f=1FptkcfNfDfk−vkBk.The first term∑f=1Fpsk1−SfkNfDfk is the revenue of network access. The second term ∑f=1FpukSfkNfDfk is the revenue of sponsorship. The third term ∑f=1FptkcfNfDfk is the revenue of ISPf by forwarding the amount of content requests to the end-users. The fourth term υkBk is the investment of ISPk, where υk is a cost per unit of requested bandwidth and Bk is the backhaul bandwidth. The QoS qsf is defined as the expected delay computed by the Kleinrock function (see [38, 39]):(9)qsk=1Delayk=Bk−∑f=1FDfk,this means that(10)Bf=∑f=1FDfk+qsf2.Then, the utility ofISPf is given as follows:(11)UISPk=∑f=1Fpsk1−SfkNfDfk+∑f=1FpukSfkNfDfk+∑f=1FptkcfNfDfk−vk∑f=1FDfk+qsf2.


### 2.4. Adding Temporality to the Model

We study in this section the impact of time on the number of sponsored content. We model the proportion of sponsored content by attaching it with the time parameter. This proportion is expressed in the following form:(12)Sfk=1−e−wfkt,where wfk represents the speed at which CPf sponsors content for each ISPk subscriber. We notice that when t=0, Sfk=0 and when t=∞, Sfk=1.The temporal analysis of the number of sponsored content can be performed in the networks; we considerξ as a discount factor, so that a monetary unit in t years is worth e−ξt monetary units of today. CPf with a profit UCPf at time t can predict this profit over a period ranging from 0,T as the average of the discounted revenue in this period as follows:(13)U¯CPf=1∫0Te−ξt∫0TUCPfte−ξt=ξ1−e−ξT∫0TUCPfte−ξt=∑k=1KpcfNfDfk1+χf+χfξe−ωfk+ξT−11−e−ξTω+ξ−∑k=1KptkNfcfDfk1+χf+χfξe−ωfk+ξT−11−e−ξTω+ξ−θfcf−∑k=1K1+ξe−ωfk+ξT−11−e−ξTω+ξ+χf1+2ξe−ωfk+ξT−11−e−ξTω+ξ−χfξe−2ωfk+ξT−11−e−ξT2ω+ξpukDfkNf.Similarly, we have(14)U¯ISPk=1∫0Te−ξt∫0TUISPkte−ξt=ξ1−e−ξT∫0TUISPkte−ξt=∑f=1Fξ1−e−ωfk+ξT1−e−ξTω+ξpskNfDfk+∑k=1K1+ξe−ωfk+ξT−11−e−ξTω+ξpukNfDfk+∑f=1FptkNfcfDfk−vkBk.


## 2.1. Demand Model

Dfk is the demand of end-users for the content provided by CPf and transferred by ISPk which is a function of content access price pcf, credibility of content cf, network access price psk, and QoS qsk (see [33, 34]). This demand function is also a function of prices pc−f, credibilities c−f, prices ps−k, and QoSs qs−k set by the opponents. The demand Dfk is decreasing with respect to pcf and psk and increasing with respect to pcg, g≠f and psj, j≠k, whereas it is increasing with respect to cf and qsk and decreasing with respect to cg, g≠f and qsj, j≠k. Then, the demand functions Dij can be written as follows:(1)Dfk=dfk−σffpcf+ςffcf−τkkpsk+ϱkkqsk+∑g=1,f≠gFσfgpcg−ςfgcg+∑j=1,k≠jKτkjpsk−ϱkjqsk.The parameterdfk is the potential demand of end-users. σfg and ςfg are two positive constants representing, respectively, the responsiveness of demand Dfk to content access price and credibility of CPg. Moreover, τgj and ϱgj are two positive constants representing, respectively, the responsiveness of demand Dfk to network access price and QoS of ISPj.Assumption 1.
The sensitivityσ verifies the following:(2)σff≥∑g=1,g≠fFρfg,∀f=1,…,F.
The sensitivityς verifies the following:(3)ςff≥∑g=1,g≠fFςfg,∀f=1,…,F.
The sensitivityτ verifies the following:(4)τkk≥∑j=1,j≠kKτkj,∀k=1,…,K.Assumption1 means that the effect of provider policies on its demand is greater than the effect of the policies of its opponent on its demand function [24, 35]. Assumption 1 will be used to prove the uniqueness of the Nash equilibrium.


## 2.2. Utility Model of the CP

The utility ofCPf can be modeled as follows:(5)UCPf=∑k=1KpcfNf1+χfSfkDfk−∑k=1KpufSfkNf1+χfSfkDfk−∑k=1KptkcfNf1+χfSfkDfk−θfcf,where Sfk is the fraction of content sponsored by CPf for each ISPk subscriber. Let Sfk=1 if full sponsoring is decided and Sfk=0 if CPf decides not to sponsor any content. Recall that sponsoring could be an incentive to consume more CPf content. 1+χfSfkDfk is the new demand for contents provided by CPf and distributed by ISPk, which is a function of the proportion of sponsored content Sfk. The quantity χfSfk reflects the change in the demand for the contents of the CP. χf is a nonnegative constant. θi is the cost to produce a unit of the credibility of content ci. ptk is the transmission price of ISPk. puf is the sponsoring price of ISPf. The first term ∑k=1KpcfNf1+χfSfkDfk is the revenue of CPf. The second term ∑k=1KpufSfkNf1+χfSfkDfk denotes the cost due to sponsorship. The third term ∑k=1KptkcfNf1+χfSfkDfk is transmission fee results when the CPf forwards to the end-users the demand with credibility of content cf. The fourth term θfcf is the cost to produce the credibility of content cf.Credibility of contentcf of CPf is a linear function of the quality of service (QoS) qssf and the quality content (QoC) qcf, which is written as follows [16, 34, 36, 37]:(6)cf=λqssf+μqcf,where λ and μ are nonnegative constants. The QoS is defined as the expected delay (see [17, 34]). The QoC can be specified for a specific domain of content (e.g., video streaming).Then, the utility ofCPf is expressed as follows:(7)UCPf=∑k=1KpcfNf1+χfSfkDfk−∑k=1KpufSfkNf1+χfSfkDfk−∑k=1KptkNf1+χfSfkDfkλqssf+μqcf−θfλqssf+μqcf.


## 2.3. Utility Model of the ISP

The utility function ofISPk is the difference between the revenue and the fee:(8)UISPk=∑f=1Fpsk1−SfkNfDfk+∑f=1FpukSfkNfDfk+∑f=1FptkcfNfDfk−vkBk.The first term∑f=1Fpsk1−SfkNfDfk is the revenue of network access. The second term ∑f=1FpukSfkNfDfk is the revenue of sponsorship. The third term ∑f=1FptkcfNfDfk is the revenue of ISPf by forwarding the amount of content requests to the end-users. The fourth term υkBk is the investment of ISPk, where υk is a cost per unit of requested bandwidth and Bk is the backhaul bandwidth. The QoS qsf is defined as the expected delay computed by the Kleinrock function (see [38, 39]):(9)qsk=1Delayk=Bk−∑f=1FDfk,this means that(10)Bf=∑f=1FDfk+qsf2.Then, the utility ofISPf is given as follows:(11)UISPk=∑f=1Fpsk1−SfkNfDfk+∑f=1FpukSfkNfDfk+∑f=1FptkcfNfDfk−vk∑f=1FDfk+qsf2.


## 2.4. Adding Temporality to the Model

We study in this section the impact of time on the number of sponsored content. We model the proportion of sponsored content by attaching it with the time parameter. This proportion is expressed in the following form:(12)Sfk=1−e−wfkt,where wfk represents the speed at which CPf sponsors content for each ISPk subscriber. We notice that when t=0, Sfk=0 and when t=∞, Sfk=1.The temporal analysis of the number of sponsored content can be performed in the networks; we considerξ as a discount factor, so that a monetary unit in t years is worth e−ξt monetary units of today. CPf with a profit UCPf at time t can predict this profit over a period ranging from 0,T as the average of the discounted revenue in this period as follows:(13)U¯CPf=1∫0Te−ξt∫0TUCPfte−ξt=ξ1−e−ξT∫0TUCPfte−ξt=∑k=1KpcfNfDfk1+χf+χfξe−ωfk+ξT−11−e−ξTω+ξ−∑k=1KptkNfcfDfk1+χf+χfξe−ωfk+ξT−11−e−ξTω+ξ−θfcf−∑k=1K1+ξe−ωfk+ξT−11−e−ξTω+ξ+χf1+2ξe−ωfk+ξT−11−e−ξTω+ξ−χfξe−2ωfk+ξT−11−e−ξT2ω+ξpukDfkNf.Similarly, we have(14)U¯ISPk=1∫0Te−ξt∫0TUISPkte−ξt=ξ1−e−ξT∫0TUISPkte−ξt=∑f=1Fξ1−e−ωfk+ξT1−e−ξTω+ξpskNfDfk+∑k=1K1+ξe−ωfk+ξT−11−e−ξTω+ξpukNfDfk+∑f=1FptkNfcfDfk−vkBk.


## 3. Game Analysis

LetG1=ℱ,Pcf,Qssf,Qcf,U¯CPfpc,qss,qc denote the noncooperative QoC price QoS game (NQPQG), where ℱ=1,…,F is the index set identifying the CPs, Pcf is the content access price strategy set of CPf, Qssf is the QoS strategy set of CPf, and Qcf is the QoC strategy set of CPf. We assume that the strategy spaces Pcf, Qssf, and Qcf of each CPf are compact and convex sets with maximum and minimum constraints; for any given CPf, we consider as strategy spaces the closed intervals Pcf=p¯cf,p¯cf, Qssf=q¯ssf,q¯ssf, and Qcf=q¯cf,q¯cf. Let the price vector pc=pc1,…,pcFT∈PcF=Pc1×Pc2×⋯×PcF, QoS vector qss=qss1,…,qssFT∈QssF=Qss1×Qss2×⋯×QssF, and QoC vector qc=qc1,…,qcFT∈QcF=Qc1×Qc2×⋯×QcF.LetG2=K,Psk,Qsk,U¯ISPkps,qs denote the noncooperative price QoS game (NPQG), where K=1,…,K is the set of the ISPs, Psk is the price strategy set of ISPk, and Qsk is the QoS strategy set of ISPk. We assume that the strategy spaces Psk and Qsk of each ISPk are compact and convex sets with maximum and minimum constraints; for any given ISPk, we consider as strategy spaces the closed intervals Psk=p¯sk,p¯sk and Qsk=q¯sk,q¯sk. Let the price vector ps=ps1,…,psKT∈PsK=Ps1×Ps2×⋯×PsK and QoS vector qs=qs1,…,qsKT∈QsK=Qs1×Qs2×⋯×QsK.


### 3.1. PricePs Game

A NPQG in network access price is defined for fixedqs∈Qs as G2qs=K,Psk,U¯ISPk.,qs.Definition 1.
A price vectorps∗=ps1∗,…,psK∗ is a Nash equilibrium of the NPQPG G2qs if(15)∀k,psk∈K,Psk,U¯ISPkpsk∗,ps−k∗,qs≥U¯ISPkpsk,ps−k∗,qs.Theorem 1.
For eachqs∈Qs, the game K,Psk,U¯ISPk.,qs admits a unique Nash equilibrium.Appendix A gives a proof of the above theorem.


### 3.2. QoSQs Game

A NPQG in QoS is defined for fixedps∈Ps as G2ps=K,Qsk,U¯ISPkps,..Definition 2.
A QoS vectorqs∗=qs1∗,…,qsK∗ is a Nash equilibrium of the NPQG G2ps if(16)∀k,qsk∈K,Qsk,U¯ISPkps,qsk∗,qs−k∗≥U¯ISPkps,qsk,qs−k∗.Theorem 2.
For eachps∈Ps, the game K,Qsk,U¯ISPkps,. admits a unique Nash equilibrium.Appendix B gives a proof of the above theorem.


### 3.3. PricePc Game

A NQPQG in pricepc is defined for fixed qss∈Qss and qc∈Qc as G1qss,qc=ℱ,Pcf,U¯CPf.,qss,qc.Definition 3.
A price vectorpc∗=pc1∗,…,pcF∗ is a Nash equilibrium of the NQPQG G1qss,qc if(17)∀f,pcf∈ℱ,Pcf,U¯CPfpcf∗,pc−f∗,qs,qc≥U¯CPfpcf,pc−f∗,qs,qc.Theorem 3.
For eachqss∈Qss and qc∈Qc, the game ℱ,Pcf,U¯CPf.,qss,qc admits a unique Nash equilibrium.Appendix C gives a proof of the above theorem.


### 3.4. QoCQc Game

A NQPQG in QoC is defined for a fixedpc∈Pc and qss∈Qss as G1pc,qss=ℱ,Qcf,U¯CPfpc,qss,..Definition 4.
A QoC vectorqc∗=qc1∗,…,qcF∗ is a Nash equilibrium of the NQPQG G1pc,qss if(18)∀f,qcf∈ℱ,Qcf,U¯CPfpc,qss,qcf∗,qc−f∗≥U¯CPfpc,qss,qcf,qc−f∗.Theorem 4.
For eachpc∈Pc and qss∈Qss, the game ℱ,Qcf,U¯CPfpc,qss,. admits a unique Nash equilibrium.Appendix D gives a proof of the above theorem.


### 3.5. QoSQss Game

A NQPQG in QoS is defined for a fixedpc∈Pc and qc∈Qc as G1pc,qc=ℱ,Qssf,U¯CPfpc,.,qc.Definition 5.
A QoS vectorqss∗=qss1∗,…,qssF∗ is a Nash equilibrium of the NQPQG G1pc,qc if(19)∀f,qssf∈ℱ,Qssf,U¯CPfpc,qssf∗,qss−f∗,qc≥U¯CPfpc,qssf,qss−f∗,qc.Theorem 5.
For eachpc∈Pc and qc∈Qc, the game ℱ,Qssf,U¯CPfpc,.,qc admits a unique Nash equilibrium.Appendix E gives a proof of the above theorem.


### 3.6. Learning Nash Equilibrium

In this section, based on our previous analysis, we introduce two distributed and iterative learning processes that convergence toward the Nash equilibrium point. In this algorithm, each provider observes the policy taken by its competitors in previous rounds and inputs them in its decision process to update its policy. Therefore, the best response and Nash seeking algorithms will converge to the unique equilibrium point.The best response (BR) algorithm is known to reach equilibria for S-modular games, by exploiting the monotonicity of the best response functions. Each player fixes its desirable strategies to maximize its profit. Then, each player can observe the policy taken by its competitors in previous rounds and input them in its decision process to update its policy. Then, it becomes natural to accept the Nash equilibrium as the attractive point of the game. Yet, the best response algorithm requires perfect rationality and complete information, which is not practical for real-world applications and may increase the signaling load as well. Therefore, we propose an adaptive distributed learning framework to discover equilibria for the activation game based on the “Nash seeking algorithm” (NSA) with stochastic state-dependent payoffs for continuous actions.The equilibrium-learning framework is an iterative process. At each iterationt, the player I chooses its policy and obtains from the environment the realization of its payoff. The improvement of the strategy is based on the current observation of the realized payoff and previously chosen strategies. Hence, we say players learn to play an equilibrium if after a given number of iterations; the strategy profile converges to an equilibrium strategy.The proposed learning framework has the following parameters:φf is the perturbation phase, zf is the growth rate, bf is the perturbation amplitude, and Ωf is the perturbation frequency. This procedure is repeated for the window T.Algorithms1 and 2 summarize the best response learning and Nash seeking algorithm steps that each player has to perform to discover its Nash equilibrium strategy.Algorithm 1: Best response algorithm.
(1)
Initialize vectorsx0=x10,…,xF0 randomly;(2)
For eachEf, f∈ℒ at time instant t compute:(i)xft+1=argmaxxf∈XfUEfxt.(3)
If∀f∈ℒ, xft+1−xft<ɛ, then STOP.(4)
Else,t⟵t+1 and go to step (2).Algorithm 2: Nash seeking algorithm.
(1)
Data(i)φf∈0;2π: perturbation phase;(ii)bf>0: perturbation amplitude;(iii)Ωf: perturbation phase;(iv)zf: the growth rate;(2)
Result. Equilibrium xft(3)
Initialize vectorsx∗0=x1∗0,…,xF∗0 and τ∗0=τ1∗0,…,τF∗0 randomly;(4)
Learning Pattern. For each iteration t:(5)
Observes the payoffUEfxt and estimates τ∗t+1 using(v)τf∗t+1=τf∗t+t∗zfbfsinΩft∗+φfUEfxt;(6)
Updatexft using the following rules(vi)xf∗t+1=τf∗t+1+bfsinΩft∗+φf;such as(i)
E denotes a CP or ISP(ii)
ℒ refers to ℱ or K(iii)
x refers to the vector price pc, vector price ps, vector qs, vector qc, or vector qss(iv)Xf refers to the policy profile price, QoS, or QoC


## 3.1. PricePs Game

A NPQG in network access price is defined for fixedqs∈Qs as G2qs=K,Psk,U¯ISPk.,qs.Definition 1.
A price vectorps∗=ps1∗,…,psK∗ is a Nash equilibrium of the NPQPG G2qs if(15)∀k,psk∈K,Psk,U¯ISPkpsk∗,ps−k∗,qs≥U¯ISPkpsk,ps−k∗,qs.Theorem 1.
For eachqs∈Qs, the game K,Psk,U¯ISPk.,qs admits a unique Nash equilibrium.Appendix A gives a proof of the above theorem.


## 3.2. QoSQs Game

A NPQG in QoS is defined for fixedps∈Ps as G2ps=K,Qsk,U¯ISPkps,..Definition 2.
A QoS vectorqs∗=qs1∗,…,qsK∗ is a Nash equilibrium of the NPQG G2ps if(16)∀k,qsk∈K,Qsk,U¯ISPkps,qsk∗,qs−k∗≥U¯ISPkps,qsk,qs−k∗.Theorem 2.
For eachps∈Ps, the game K,Qsk,U¯ISPkps,. admits a unique Nash equilibrium.Appendix B gives a proof of the above theorem.


## 3.3. PricePc Game

A NQPQG in pricepc is defined for fixed qss∈Qss and qc∈Qc as G1qss,qc=ℱ,Pcf,U¯CPf.,qss,qc.Definition 3.
A price vectorpc∗=pc1∗,…,pcF∗ is a Nash equilibrium of the NQPQG G1qss,qc if(17)∀f,pcf∈ℱ,Pcf,U¯CPfpcf∗,pc−f∗,qs,qc≥U¯CPfpcf,pc−f∗,qs,qc.Theorem 3.
For eachqss∈Qss and qc∈Qc, the game ℱ,Pcf,U¯CPf.,qss,qc admits a unique Nash equilibrium.Appendix C gives a proof of the above theorem.


## 3.4. QoCQc Game

A NQPQG in QoC is defined for a fixedpc∈Pc and qss∈Qss as G1pc,qss=ℱ,Qcf,U¯CPfpc,qss,..Definition 4.
A QoC vectorqc∗=qc1∗,…,qcF∗ is a Nash equilibrium of the NQPQG G1pc,qss if(18)∀f,qcf∈ℱ,Qcf,U¯CPfpc,qss,qcf∗,qc−f∗≥U¯CPfpc,qss,qcf,qc−f∗.Theorem 4.
For eachpc∈Pc and qss∈Qss, the game ℱ,Qcf,U¯CPfpc,qss,. admits a unique Nash equilibrium.Appendix D gives a proof of the above theorem.


## 3.5. QoSQss Game

A NQPQG in QoS is defined for a fixedpc∈Pc and qc∈Qc as G1pc,qc=ℱ,Qssf,U¯CPfpc,.,qc.Definition 5.
A QoS vectorqss∗=qss1∗,…,qssF∗ is a Nash equilibrium of the NQPQG G1pc,qc if(19)∀f,qssf∈ℱ,Qssf,U¯CPfpc,qssf∗,qss−f∗,qc≥U¯CPfpc,qssf,qss−f∗,qc.Theorem 5.
For eachpc∈Pc and qc∈Qc, the game ℱ,Qssf,U¯CPfpc,.,qc admits a unique Nash equilibrium.Appendix E gives a proof of the above theorem.


## 3.6. Learning Nash Equilibrium

In this section, based on our previous analysis, we introduce two distributed and iterative learning processes that convergence toward the Nash equilibrium point. In this algorithm, each provider observes the policy taken by its competitors in previous rounds and inputs them in its decision process to update its policy. Therefore, the best response and Nash seeking algorithms will converge to the unique equilibrium point.The best response (BR) algorithm is known to reach equilibria for S-modular games, by exploiting the monotonicity of the best response functions. Each player fixes its desirable strategies to maximize its profit. Then, each player can observe the policy taken by its competitors in previous rounds and input them in its decision process to update its policy. Then, it becomes natural to accept the Nash equilibrium as the attractive point of the game. Yet, the best response algorithm requires perfect rationality and complete information, which is not practical for real-world applications and may increase the signaling load as well. Therefore, we propose an adaptive distributed learning framework to discover equilibria for the activation game based on the “Nash seeking algorithm” (NSA) with stochastic state-dependent payoffs for continuous actions.The equilibrium-learning framework is an iterative process. At each iterationt, the player I chooses its policy and obtains from the environment the realization of its payoff. The improvement of the strategy is based on the current observation of the realized payoff and previously chosen strategies. Hence, we say players learn to play an equilibrium if after a given number of iterations; the strategy profile converges to an equilibrium strategy.The proposed learning framework has the following parameters:φf is the perturbation phase, zf is the growth rate, bf is the perturbation amplitude, and Ωf is the perturbation frequency. This procedure is repeated for the window T.Algorithms1 and 2 summarize the best response learning and Nash seeking algorithm steps that each player has to perform to discover its Nash equilibrium strategy.Algorithm 1: Best response algorithm.
(1)
Initialize vectorsx0=x10,…,xF0 randomly;(2)
For eachEf, f∈ℒ at time instant t compute:(i)xft+1=argmaxxf∈XfUEfxt.(3)
If∀f∈ℒ, xft+1−xft<ɛ, then STOP.(4)
Else,t⟵t+1 and go to step (2).Algorithm 2: Nash seeking algorithm.
(1)
Data(i)φf∈0;2π: perturbation phase;(ii)bf>0: perturbation amplitude;(iii)Ωf: perturbation phase;(iv)zf: the growth rate;(2)
Result. Equilibrium xft(3)
Initialize vectorsx∗0=x1∗0,…,xF∗0 and τ∗0=τ1∗0,…,τF∗0 randomly;(4)
Learning Pattern. For each iteration t:(5)
Observes the payoffUEfxt and estimates τ∗t+1 using(v)τf∗t+1=τf∗t+t∗zfbfsinΩft∗+φfUEfxt;(6)
Updatexft using the following rules(vi)xf∗t+1=τf∗t+1+bfsinΩft∗+φf;such as(i)
E denotes a CP or ISP(ii)
ℒ refers to ℱ or K(iii)
x refers to the vector price pc, vector price ps, vector qs, vector qc, or vector qss(iv)Xf refers to the policy profile price, QoS, or QoC


## 4. Numerical Investigations

In this section, we study the telecommunication network numerically as a noncooperative game while considering the expressions of the utility functions and using the best response algorithm. We consider a network scenario that includes two ISPs and two CPs.Figures1–5 show the convergence toward Nash equilibrium price, Nash equilibrium QoS, and Nash equilibrium QoC of all providers. Figures 1–5 demonstrate the existence and uniqueness of a Nash equilibrium point at which no providers can profitably deviate given the strategies of another opponent. So, our model ensures the existence of an equilibrium for keeping the economy stable and achieving economic growth.Figure 1
Nash equilibrium price under the BR and NSA algorithms: (a) best response algorithm; (b) Nash seeking algorithm.
(a)(b)Figure 2
Nash equilibrium QoS under the BR and NSA algorithms: (a) best response algorithm; (b) Nash seeking algorithm.
(a)(b)Figure 3
Nash equilibrium price under the BR and NSA algorithms: (a) best response algorithm; (b) Nash seeking algorithm.
(a)(b)Figure 4
Nash equilibrium QoS under the BR and NSA algorithms: (a) best response algorithm; (b) Nash seeking algorithm.
(a)(b)Figure 5
Nash equilibrium QoC under the BR and NSA algorithms: (a) best response algorithm; (b) Nash seeking algorithm.
(a)(b)Table1 gives a comparison between the two algorithms proposed for the learning of the numerical results; we notice that the algorithm of best response gives the same results as the Nash seeking algorithm but in fewer iterations and in a very small time compared with a Nash seeking algorithm.Table 1
Best response algorithm versus Nash seeking algorithm.
Time (s)IterationsBest response algorithmNash seeking algorithmBest response algorithmNash seeking algorithmps∗28.710555.6239216pc∗2.83075.1134816Qc∗26.522932.734768qss∗28.4598122.8715627qs∗1.00034.437328The effect of the parameterχ on the QoS and the QoC is shown in Figures 6 and 7. The QoS and the QoC of the proposed model are growing as χ increases. When χ increases, the demand of end-users increases, and then, the revenue of CPs increases. As a result, the CPs increase their QoS and QoC to attract more end-users.Figure 6
Nash equilibrium QoCqc evolution with respect to parameter χ.Figure 7
Nash equilibrium QoSqss evolution with respect to parameter χ.Figures8–10 represent the impact of sponsoring price pu on the content access price, the QoS, and the QoC of the two CPs. As the sponsoring price increases, the content access price pc increases, the QoC decreases, and the QoS decreases. When the sponsoring price is low, the CP invests more to offer better QoS, better QoC, and low content access price to induce increased demand from end-users. On the other hand, when the sponsoring price is high, the CP increases their content access price and decreases their QoS and QoC to compensate the increase in the sponsoring price.Figure 8
Nash equilibrium pricepc evolution as a function of sponsoring price pu.Figure 9
Nash equilibrium QoCqc evolution as a function of sponsoring price pu.Figure 10
Nash equilibrium QoSqss evolution as a function of sponsoring price pu.The impact of sponsoring pricepu on the network access price and QoS of the two ISPs is illustrated in Figures 11 and 12. Figures 11 and 12 show that the network access price decreases and the QoS increases when sponsoring price increases. The reason is that as sponsoring price increases, the revenue of sponsoring increases, which leads to a rise in the income of the ISPs. Therefore, the ISPs decrease their network access price and invest for more bandwidth to increase their QoS to attract more end-users.Figure 11
Nash equilibrium QoSqs evolution as a function of sponsoring price pu.Figure 12
Nash equilibrium priceps evolution as a function of sponsoring price pu.Figures13 and 14 show the influence of sponsoring content speed w, respectively, on network access price and QoS equilibrium. From the two figures, we notice that when the speed of sponsoring content increases, the number of sponsored content increases and then the revenue of ISPs increases. Therefore, the ISP needs to lower the price and improve the QoS to induce increased demand from end-users.Figure 13
Nash equilibrium QoSqs evolution as a function of sponsoring content speed w.Figure 14
Nash equilibrium priceps evolution as a function of sponsoring content speed w.We plot in Figures15–17, respectively, the interplay of the speed of sponsoring content w on the content access price, the QoC, and the QoS at Nash equilibrium, for both CPs that we consider in this example. On the one hand, we note that the equilibrium content access price for both CPs is increasing with respect to the speed of sponsored content. On the other hand, we indicate that the equilibrium QoS and QoC for all CPs is decreasing with the speed of sponsored content. When the speed of sponsoring content is low, the CPs invest more to offer better QoS, QoC, and an attractive content access price. However, as the speed of sponsoring content increases, the CPs choose to raise their content access price and decrease their QoS and QoC to compensate the rise in the sponsoring price.Figure 15
Nash equilibrium pricepc evolution as a function of sponsoring content speed w.Figure 16
Nash equilibrium QoCqc evolution as a function of sponsoring content speed w.Figure 17
Nash equilibrium QoSqss evolution as a function of sponsoring content speed w.


## 5. Conclusion

In this paper, we study the data sponsoring problem with time constraint in the Internet market with multiple ISPs, multiple CPs, and a set of end-users. The interaction among ISPs and among CPs is investigated by using the noncooperative game. Then, we have proved the existence and uniqueness of the Nash equilibrium. This result is significant because it implies that a stable solution with suitable economic incentives in collaborative data sponsoring is feasible in the Internet paradigm. In addition, we describe a learning mechanism that allows each provider to discover accurately and rapidly its equilibrium policies. At last, we have presented a numerical investigation to validate the proposed approach, and we found that the sponsoring content has a negative effect on the strategies of CPs and positive one on the strategies of ISPs and to motivate the CPs to sponsor more content to reduce the cost of sponsorship in a long term.



---

*Source: 6629020-2021-01-19.xml*