# Applications of Game Theory and Advanced Machine Learning Methods for Adaptive Cyberdefense Strategies in the Digital Music Industry

Authors: Jing Jing
Journal: Computational Intelligence and Neuroscience
 (2022)

---

## Abstract

As the likelihood and impact of cyber-attacks continue to grow, organizations realize the need to invest in specialized methods to protect their digital data and the information they circulate or manage. Due to its broad use, game theory has evolved into a concept that can be applied practically while analyzing and modifying existing cyber protection methods to arrive at the best possible conclusions. This study presents an innovative hybrid model that combines game theory and advanced machine learning methods for adaptive cyber defense strategies. Specifically, a repetitive game methodology is implemented to analyze cyber-attacks and model behaviors and study how defenders and attackers make decisions in a competing field. Based on Bayesian inference, the proposed method can predict the next steps in the game to produce the appropriate countermeasures and implement the best cyber defense strategies that govern an organization. The suggested system introduced to the academic literature for the first time was successfully tested in a particular application scenario involving the digital music industry and coping with impending cyber-attacks.


---

## Body

## 1. Introduction

As the cyberspace landscape evolves rapidly, a dynamic framework emerges in which very delicate balances are observed to make optimal decisions. The asset, information, and data values that modern organizations must manage constantly increase. New cyber-attack techniques are continually developing while existing technological defense systems age, making active defense strategies resilient. Furthermore, economic changes, institutional reorganizations, consumer trends, and legal and regulatory compliance requirements all impact decision-making and the overall development of sound defense strategies for an organization or company. To maintain its competitive advantage, the organization in question will need to continuously improve its defense strategies, which will be based on ongoing knowledge and utilization of the cyber threat landscape [1]. An accurate inventory of all assets classified by their value to the organization is critical in assessing the severity of the risks they face and, as a result, the decisions that must be made about them.Game theory [2] can be used in cybersecurity to create tangible solutions that will allow the maximum utilization of existing strategies and optimize them to create a robust and long-term security environment at the organization level [3–5]. Using game theory principles, cyber security professionals can implement a network of controls that specialize and, as a result, reduce the risk to their valuable assets. They can also apply areas with a low level of risk, maximizing their return on investment. As a result, using specialized scenarios based on game theory, it is possible to predict the attackers’ strategy at each stage of the attack cycle, assisting in developing intelligent models to improve cyber security and the development of new intelligent systems to deter attackers.This study presents applications of game theory and advanced machine learning methods for adaptive cyber defense strategies in the digital music industry. An innovative application of a repetitive game methodology for cyber-attack analysis is proposed; using Bayesian inference [6], the next steps in the game can be predicted so that the best cyber defense strategies can be implemented to shield an organization from cyber-attacks.


## 2. Literature Review

The literature on game theory for adaptive cyber defense approaches is extensive, whether at the network level, where we must deal with massive amounts of raw data or at the strategy level.In 2015, Laszka et al. [7] developed a strategy for reducing spear-phishing assaults by targeted per-user filtering criteria. They framed the task of screening harmful e-mails, both targeted and untargeted, as a security game. These defined optimum filtering techniques and demonstrated how they might be computed in practice. They put their theoretical hypotheses to the test by comparing them to two datasets taken from the actual world. Using two different sets of real data, they demonstrated that the recommended baselines result in less harm than the nonstrategic restrictions. In addition, they found that the improvement over nonstrategic criteria was more substantial for the comprehensive information. This improvement was unaffected by an increase in the number of targeted customers. This indicated that their technique scaled effectively, both analytically and in terms of how well it performed.Schlenker et al. [8] explored the critical issue of allocating cyber alarms to a restricted number of professionals in cyber security activities. They proposed the cyber-alert allocation game to investigate this issue and demonstrated how to compute the defender’s best options. They offered a unique technique for addressing implement ability concerns in determining the defender’s best marginal tactic to resolve this game. Finally, they presented heuristics for solving big games like the one described and an objective assessment of the suggested framework and solution methods.Nguyen et al. [9] conducted a study of deep reinforcement learning (DRL) techniques used in cyber protection. They discussed various critical topics, such as DRL-based security approaches for cyber-physical infrastructure, independent intrusion detection approaches, and multiagent DRL-based game theory simulations for cyber-attack defensive tactics. Additionally, extensive debates and potential study paths on cyber security focusing on DRLs are provided. They hoped that this exhaustive assessment would provide the groundwork for and support future research into the ability to develop DRL to address more sophisticated digital privacy concerns.Alpcan and Basar [10], in 2010, in their book, about network security and game theoretic approaches, aimed to present a theoretical foundation for making resource allocation decisions that balance available capabilities and perceived security risks in a principled manner. They focused on analytical models based on game, information, communication, optimization, decision, and control theories applied to diverse security topics. At the same time, connections between theoretical models and real-world security problems are highlighted to establish the critical feedback loop between theory and practice.Hemberg et al. [11] presented an architecture for adversarial AI called RIVALS that abstractly replicated the hostile, competing for a coevolutionary mechanism in security contexts. The purpose was to develop a system capable of pro-active cyber security against dynamic automated attackers. They reviewed its present uses and how it is used to develop defensive measures. Further work will involve expanding it to enable more cyber defense purposes, creating more effective or genuine reality methods, and applying other Nash equilibrium-finding algorithms to other cyber security challenges with established Nash equilibria [12] and analyzing efficiency.


## 3. Proposed Game Strategy

Many cyber-attacks follow a pattern built on repeating tactics or procedures over time. Infrastructure vulnerability control strategies, for example, can compete with current defensive systems and change their applications over time. In this sense, attackers and defenders engage regularly, and these interactions may be depicted using repetitive games, which are a type of dynamic game [13]. The concept described here is using a repeated game to evaluate cyber-attacks and simulating some cooperative behaviors without a clear endpoint. A repetitive game begins with a static game repeated infinitely or intermittently many times. A reward is given to each player who completes a specific action in this strategic stage game. The sum of each player’s gains throughout the game constitutes their final reward. In addition, Bayesian inference [14] can predict the next steps in the game to produce the appropriate countermeasures and implement the best cyber defense strategies that govern a complex system with high uncertainty [15].The starting point for modeling the proposed system is a static game of the following format:(1)Γ=N,Xi,uii∈N,where Ν = {1, 2,. .., n} is the set of players, Xi is the set of the player’s ί pure strategies, and u is its performance function. Assume that this stage game is repeated T times, where T is finite or infinite. Each repetition takes place over a period. A typical time (or stage) is denoted by ί, where ί = 1, 2, ..., Τ. The interaction evolves as follows [16, 17]:(1)
In period 1, players simultaneously select actions, which we symbolize asx1=x11,x21,…,xn1, where the pointer symbolizes the player and the exponent symbolizes the stage (time period). The action x11 belongs to the set Xi,i∈N. Each player has been informed afterward of the choices of the other players. The performance of player i in this period is(2)uix1,i∈N.(2)
In period 2, players choose at the same time actionsx2=x12,x22,…,xn2. Each player has been informed afterward of the choices of the other players. Player’s i odds are(3)uix2,i∈N.(3)
In periodT, players choose actions xT=x1T,x2T,…,xnT. The player’s odds are(4)uixT,i∈N.(4)
IfT is finite, the interaction is complete (at the end of period T). Otherwise, the game continues in perpetuityIn the repetitive game, we suggest each player earns a sequence of payoffs (one payoff for each period). This yield sequence is assumed to be valued using the discounted sum of the sequence terms. The term discount expresses the assumption that a person does not value current and future returns equally. Parameterδ is the discount rate of an individual [18]. The closer δ is to 0, the less the individual values a future versus a present performance. In other words, the smaller the δ is, the less the person is interested in the future or the more impatient he is. Conversely, the higher the δ (the closer it is to 1) is, the more a person values a future performance or is more patient [19–21].Let a finite terminal storyhT=x1,x2,…,xT. If the discount rate of player i is given by the parameter δi∈0,1, then the discounted sum of the payoffs of player i is(5)Vi≡uix1+δiuix2+δi2uix3+⋯+δiT−1uixT=∑t=1Tδit−1uixt.Let now an infinite terminal storyh∞=x1,x2,…. In this case, the discounted sum of player’s payoffs is(6)Vi≡uix1+δiuix2+δi2uix3+⋯=∑t=1∞δit−1uixt.In the case of infinityT, instead of the discounted sum of the odds, we use the discounted average of the player’s odds as a valuation function:(7)uit=uixt,t=1,2,….Based on the relation, playeri obtains the sequence of odds ui1,ui2,…. We define a number c such that i is indifferent between the series of yields ui1,ui2,… and the sequence (c, c1, ..., cn). So, we take the following relation [22]:(8)Vi=∑t=1∞δit−1uit=∑t=1∞δit−1c.However, it is true that(9)∑t=1∞δit−1c=c1−δi.Consequently, we have(10)Vi=∑t=1∞δit−1uit=c1−δi,so that(11)1−δiVi=1−δi∑t=1∞δit−1uit=c.The term1−δiVi is the discounted average of the yield sequence ui1,ui2,…. The functions Vi and 1−δiVi express the same preferences since one is a positive monotonic transformation. Thus, the set of all terminal stories is the set of all sequences [3, 23]:(12)h∞=x1,x2,….So, playeri evaluates the terminal history h∞ based on the function:(13)Vi=∑t=1∞δt−1uixt,or it is equivalent:(14)1−δVi=1−δ∑t=1∞δt−1uixt.So, we have the repetitive gameG∞δ.A rule that dictates the player’s behavior in response to any and all scenarios is an example of a strategy. Consequently, a single-player strategy is a guideline for picking actions at each stage (repetition) of the stage game as a function of previous decisions when playing a game that features observable actions and is repetitive. To be more exact, the energy that the player will pick at each narrative stage is determined by the strategy used in a game with just one player and repeated steps. The action must be feasible; that is, it must belong to all the available options of the gaming stage [24, 25].Odds pairs corresponding to the four pairs of clear strategies are feasible (in the sense that there are strategies that generate specific returns). It should also be noted that all convex combinations of two or more yield pairs of pure methods constitute achievable yields (using mixed strategies) [5, 26].In general, the total possible odds of the stage game are given by the quadrilateral in Figure1, which has as vertices the four pairs of payoffs corresponding to the clear strategies described by the set {(5, 5), (0, 6), (6, 0), (1,1)}.Figure 1
Possible payoffs in the form of a diagram.


## 4. Application Testing

In order to model the proposed system, a specialized threat scenario was implemented with an application study in the music industry. This was done because the large number of visitors combined with the enormous amounts of music content spent on a daily basis creates a new landscape of threats, in which the clear strategies for cybersecurity need to be rearranged on an ongoing basis. According to this logic, they frequently employ advanced techniques, including zero-day attacks, to launch attacks on music streaming platforms that modern cybercriminals have targeted [24, 27, 28].For the implementation of the proposed system, we consider a static gameΓ=N,Xi,uii∈N, such that(1)
u1∗,u2∗,…,un∗ is the payoff vector of a Nash equilibrium of Γ (the Nash equilibrium is a solution technique for non-cooperative games with two or more participants in game theory. We assume that each player is aware of the equilibrium strategies that the other players would employ and that no player stands to gain by simply changing his strategy [2] if each participant has decided on a strategy and no one has the option of modifying their strategy. On the contrary, the other players maintain their strategy constant; their set of plans and results are the Nash equilibrium.(2)
v^1,v^2,…,v^n is a vector of possible returns of Γ such that v^i>ui∗ for each i.(3)
The discount rateδ is quite large.Then, there is a perfect subgame balance for the gameG∞δ with average odds v^1,v^2,…,v^n.To prove the above scenario, we will assume that the vectorv^1,v^2,…,v^n can be achieved through a vector of pure strategies x^=x^1,x^2,…,x^n. That is, uix^=u^i,i∈N. It should be noted that it is not necessary to assume clean strategies. If this is not the case, vector V is achieved through mixed strategies [5, 10].Suppose that the equilibrium returnsu1∗,u2∗,…,un∗ of Γ are derived from the strategy vector x1∗,x2∗,…,xn∗.We consider the following firing strategy:(1)
Stagei = 1:(a)
playeri selects an action x^i(2)
Stagei > 1:(a)
If the result in the previous stages 1, 2, ..., t-1 isx^, player i selects an action x^i(b)
In all other cases, playeri selects an action xi∗We will examine the players’ motivations for adopting or not the above strategy.Let us look at playeri if all other players adopt the firing strategy. We must determine the optimal response of i in each period where it has observed a result x^ and its optimal response in each period where it has observed a result different from x^.It should be recorded the actions thati will take if it observes in a period t that in the previous period t-1 a result other than x^ was outcome. Player i knows that, in period t, the other players will choose the actions x−i∗ and that this will be done in perpetuity (because they follow the firing strategy). Therefore, the optimal reaction of 1 is to choose xi∗ in perpetuity.Let us now see what playeri will do in those periods for which history contains only the result x^. Suppose that, in some of them, i chooses to deviate from the energy x^i. The optimal deviation, which we denote by xidd, is what solves the problem.In particular,(15)maxxiuixi,x^−i.The yield ofi during the deviation period is(16)uid≡uixid,x^−i,where it is true that(17)uid≥u^i.From the next period, the other players will start choosingx−i∗, after noticing the choice xid of i. The punishment period of i will therefore begin. Given this, the optimal response of i is to select the energy xi∗. Thus, from the period following the deviation to perpetuity, i will have a payoff of ui∗ per period. Its average discounted return from the breach is, therefore,(18)1−δuid+δui∗+δ2ui∗+…=1−δuid+δui∗.If, on the contrary,i, after each story that contains only x^, chooses the action x^i; its return will be u^i because after such a story, the other players choose x^−i. For each of the following periods, the same result will be repeated, and i will continuously gain u^i per period. The average payoff of i in this case is(19)1−δu^i+δu^i+δ2u^i+…=1−δ11−δu^i=u^i.Therefore, playeri will select x^i after each story that contains only x^ if(20)u^i≥1−δuid+δui∗⟺δuid−ui∗≥uid−u^i.If the following inequality relations are valid,(21)uid≥u^i>ui∗,so it applies(22)δ≥uid−u^iuid−ui∗.And so(23)uid−u^iuid−ui∗≤1.The possible payoffs that exceed the Nash payoffs are shown in Figure2.Figure 2
Possible payoffs that exceed Nash payoffs.Next, to determine which value can best estimate the random variable, we apply Bayesian inference to create a loss function, which will measure how wrong the estimate is, that is, how different the estimation of the parameter is from its actual price. This means that the goal is for the value of the estimate to minimize the loss function. For example, in the absolute error loss function, underestimation and overestimation of the parameter are punished in the same way, depending on the deviation of the estimate. In contrast to the preceding statement, the linear loss function penalizes underestimation with a different weight than overestimation.The evaluation of the decision rules is done through the risk function, which is defined as the average value of the loss function. IfLδ,θ is the loss function and δ = δ(x) is the decision rule, the hazard function of the decision rule δ is mathematically expressed as(24)Rδ,θ=EθLδ,θ=∫Lδx,θfx|θdx.In the Bayesian approach, it is possible to balance the risk function of each decision rule, based on ex-ante personal opinion, throughu(θ). So, the Bayesian risk is defined as [29–31](25)BRδ=∫ΘRδ,θuθdθ=∫Θ∫XLδx,θfx|θuθdxdθ.The decision rule that minimizes Bayesian risk is the Bayesian rule. In particular, the Bayesian risk of decision ruleδ, as a function of the loss of a square error, is equal to(26)Eδ−θ2=∫Θ∫Xδx−θ2fx|θdxuθdθ=∫Θ∫Xδx−θ2fx|θuθdxdθ=∫Θ∫Xδx−θ2fx,θdxdθ=∫X∫Θδx−θ2pθ|xgxdθdx=∫X∫Θδx−θ2pθ|xdθgxdx.Since the margin function g(x) is a nonnegative function, we are interested in minimizing the(27)Ix=∫Θδx−θ2pθ|xdθ,which becomes(28)Ix=∫Θδx2−2θδx+θ2pθ|xdθ=δx2∫Θpθ|xdθ−2δx∫Θθpθ|xdθ+∫Θθ2pθ|xdθ=∫Θpθ|xdθδx2−2δx∫Θθpθ|xdθ∫Θpθ|xdθ+∫Θθ2pθ|xdθ=∫Θpθ|xdθδx−∫Θθpθ|xdθ∫Θpθ|xdθ2+∫Θθ2pθ|xdθ−∫Θθpθ|xdθ2∫Θpθ|xdθ.The above function is minimized forδ(x) when(29)δx=∫Θθpθ|xdθ∫Θpθ|xdθ=Eθ|x.So, through a hypothesis check, we can determine the correctness of a hypothesis that concerns an action without dealing with estimating the possible values of the strategy. Based on this view, it is possible to predict the future prices of the process as, through the ex-post distribution, the forecasting process is immediate and accurate.Assuming that, for a random sample of observationsx = x1, x2, …, xn, it is desirable to predict the future observation y, and it is necessary to calculate the a posteriori distribution of prediction πy|x. For the mathematical calculation of the ex-post distribution, the mathematical analysis of the ex-prediction distribution gx is required:(30)gx=∫θfx,θdθ=∫θfx|θuθdθ.Observing the equation, it is easy to see that the integral contains the product of the probability functionfx|θ, with the ex-ante distribution u(θ). Thus, based on the above procedure, the mathematical definition of the ex-post prediction distribution πy|x will be made:(31)πy|x=∫θfy,θ|xdθ=∫θfy|θ,xpθ|xdθ=∫θfy|θpθ|xdθ.The probability function of the following observations is equal to the integral of the joint probability functionfy,θ|x. It should be noted that this time the shared distribution function is bound to the observed data. Re-applying the Bayesian theorem, we conclude that the ex-post distribution of prediction is again equal to the product of probability fy|θ, but this time, with the ex-post distribution pθ|x, this is completed for the variable θ.The effect of the ex-post information is therefore apparent. It should be noted that, from the probability functionfy,θ|x, we came to the probability function if all a posteriori information originates from the parameter θ.To prove the validity of the methodology described above, we will make the assumption that we will count the number of privilege escalation attacks made against a randomly chosen account on the music streaming platform that is the focus of the current example.Suppose that the number of specific attacks over a period follows the “Poisson” distributionθ,θ∈Θ= (0, ∞), i.e.,(32)fxi|θ=e−θθxixi!,i=1,2,…,n.Forn observation time, the probability of the sample is(33)Lθ|x=∏i=1nfxı|θ=∏i=1ne−θθxixi!=e−nθθ∑xi∏xi!.Suppose that the ex-ante distribution ofθ follows the Gamma distribution (a β), where a, β > 0; quantities are known as a function of probability density:(34)uθ=θα−1e−θ/ββαΓα.Therefore, the distribution margin is(35)gx=∫0∞Lθ|xuθdθ=∫0∞e−nθθ∑xi∏xi!θα−1e−θ/ββαΓαdθ=1βαΓα∏xi!∫0∞e−nθ−θ/βθ∑xi+α−1dθ=β/nβ+1∑xi+αΓ∑xi+αβαΓα∏xi!∫0∞e−θnβ+1/βθ∑xi+α−1β/nβ+1∑​xi+αΓ∑xi+αdθ=β/nβ+1∑xi+αΓ∑xi+αβαΓα∏xi!.Therefore, the a posteriori distribution ofθ is(36)pθ|x=Lθ|xuθgx=e−nθθ∑xi/∏xi!θα−1e−θ/β/βαΓαgx=e−θnβ+1/βθ∑xi+α−1/∏xi!βαΓαβ/nβ+1∑xi+αΓ∑xi+α/βαΓα∏xi!=enβ+1/βθ∑xi+α−1β/nβ+1∑​xi+αΓ∑xi+α.From the above relation, it becomes clear that the a posteriori distribution ofθ is [32](37)Gamma∑xi+α,βnβ+1.The ex-post distribution following theGamma distribution was expected since the Gamma distribution family, where the ex-distribution belongs, is conjugated to the Poisson distribution.The ex-post average value for the aboveGamma distribution is [30, 32, 33](38)Eθ|x=∑i=1nxi+αβnβ+1=β∑xinβ+1+αβnβ+1=nβnβ+1x¯+1nβ+1αβ.Letr∼Poissonθ be a new independent observation. The equation gives the forecast distribution:(39)πy|x=∫Θfy|θpθ|xdθ,where for the specific statistical model it is [34, 35](40)πy|x=∫θe−θθyy!e−θnβ+1/βθ∑xi+α−1β/nβ+1∑xi+αΓ∑xi+αdθ=1y!β/nβ+1∑xi+αΓ∑xi+α∫θe−θnβ+β+1/βθ∑xi+α+y−1dθ=β/nβ+β+1∑xi+α+yΓ∑xi+α+yβ/nβ+1∑xi+αΓ∑xi+αΓy+1y=Γ∑xi+α+yΓ∑xi+αΓy+1nβ+1nβ+β+1∑xi+αβnβ+β+1,which is a negative binomial with parameters [36–38]:(41)∑xi+α,βnβ+β+1.


## 5. Conclusions

This research offered an original and forward-thinking application of game theory and cutting-edge machine learning methods for adaptive cyber protection measures. It suggested a methodology based on game theory, which concerns the study of elements that characterize situations of competitive interdependence with an emphasis on the decision-making process of more than one decision maker. We make the assumption that every player is informed of the equilibrium tactics that the other players will use and that no player has anything to gain by merely altering the strategy that he uses. Therefore, in order to figure out which value can provide the most accurate estimation of the random variable, we use Bayesian inference to devise a loss function. This function will measure how inaccurate the estimate is, or more specifically, how far off the estimation of the parameter is from the actual price. This indicates that the objective is to have the value of the estimate be as low as possible concerning the loss function.Specifically, the study of features that characterizes conditions of competitive interdependence includes the opponents. To be more specific, a methodology based on a repeated game is used to examine cyber-attacks and model behaviors and research how defenders and attackers make decisions in an environment where they compete. The unique approach developed can forecast the future moves in the game to generate the proper countermeasures and apply the most acceptable cyber defense tactics that govern a company. This ability is based on the use of Bayesian inference, which is a type of statistical inference. The suggested system was tested with great success in a particular application scenario in the digital music sector and how to cope with upcoming cyber-attacks. The testing was successful on both fronts.A crucial step in further developing the proposed model is the further investigation of how they can adapt the method parameters to processes of modern and asynchronous change of the initial parameters of the evaluators. This is one of the processes that will further the development of the proposed model. As a result, the expansion and empirical exploration of the characteristics of the method estimators in finite samples, which call for the use of Monte Carlo simulations, is also a vital component of the proposed system’s evolutionary parameter.



---

*Source: 2266171-2022-06-17.xml*