—This paper proposes a cooperative game theory- based approach for reserve optimization to enable distributed energy resources (DERs) participate in tertiary frequency regu- lation. Tertiary frequency regulation schemes ensure that reserve requirements of primary and secondary frequency regulation are fulﬁlled with a minimum cost. While the available reserve from a single distribution system may not sufﬁce tertiary frequency regulation, stacked reserve from several distribution systems can enable them participate in tertiary frequency regulation at scale. In this paper, a two-stage strategy is proposed to effectively and precisely allocate spinning reserve requirement from each DER in distribution systems. In the ﬁrst stage, two types of characteristic functions are computed: worthiness index (WI) and power loss reduction (PLR). In the second stage, the equivalent Shapley values are computed based on the characteristic functions, which are used to determine distribution factors for reserve allocation among DERs. The effectiveness of the proposed method for allocating reserves among DERs is demonstrated through several case studies on modiﬁed versions of the IEEE 13-node and 33- node distribution systems. Index Terms —Co-operative game theory, distributed energy resources, Shapley value, tertiary frequency regulation. I. I NTRODUCTION Integration of distributed energy resources (DERs) has brought several challenges and beneﬁts to power grid oper- ation including frequency control and regulation. Frequency deviation occurs when there is an imbalance between the generation and load, which can happen due to several factors including faults, large load changes, generating unit tripping and islanding parts of the grid [1]. In such scenarios, frequency regulating schemes come into play to compensate frequency deviations. For tertiary frequency regulation, under vertically integrated monopolistic structure of utilities, system operators set operating points of individual generators based on an opti- mal power ﬂow (OPF) solution, which minimizes the overall operating cost of generation subjected to network and reserve constraints. On the other hand, in deregulated power systems, the main function of tertiary frequency regulation schemes is to maximize the net social welfare through allocating adequate spinning reserve from generators or DERs participating in primary and secondary frequency regulation [2]. Although the contribution of a single DER in frequency control and regulation is not signiﬁcant, the accumulated contribution from a ﬂeet of DERs can enable them to collectively participate in frequency control and regulation. However, allocating impact-ful reserves from DERs is a challenging task and requires ﬂexible and efﬁcient solutions. Various methods have been presented in the literature for tertiary frequency regulation and control in transmission and distribution systems. An approach for optimal tertiary fre- quency control has been proposed in [3], which also considers regulation based on electricity market. A model predictive control (MPC)-based approach has been proposed in [4] for the activation of tertiary frequency control reserves. A mixed integer linear programming-based optimization tool has been proposed in [5] for the activation of tertiary frequency control reserves at transmission level. In [6], the co-optimization of energy and reserve has been performed in a standalone microgrid considering uncertainties associated with renewable energy sources and loads. In [7], a data-driven approach for the estimation of secondary and tertiary reserve has been presented and tested on a real-life case study. Although several methods and algorithms have been developed and employed for tertiary frequency regulation and control of transmission systems and microgrids, allocating reserves from active distribution sys- tems for the tertiary frequency regulation is still a challenge. Cooperative game theory-based approaches have been suc- cessfully applied in various ﬁelds of power systems. A coop- erative game theory-based approach has been implemented in [8] for loss reduction allocation of distributed generation using Shapley values. A cooperative game theory-based approach has been proposed in [9] for under frequency load shed- ding control. A cooperative game theory-based approach for computing participation factors of distributed slack buses has been proposed in [10]. For tertiary frequency regulation, the cooperative game theoretic approaches based on the Shapley value can ensure that the total available reserve is fairly distributed among different DERs taking into account their marginal contributions. This paper proposes a cooperative game theoretic two-stage approach for fair allocation of reserves among DERs. In the ﬁrst stage, two types of characteristic functions, viz., worthi- ness index (WI) and power loss reduction (PLR), are computed for each set of possible coalitions of participating DERs. The equivalent Shapley values and hence distribution factors of DERs are determined in the second stage, which are utilized for the allocation of reserves among DERs. The effectiveness of the proposed approach is demonstrated through case studies G GGGGrid G Distributed energy resource (DER) Communication between DERs and substationSubstationDist. System Operator (DSO) Communication between DSO and substationFig. 1. The Layout of the Proposed Cooperative Game Theoretic Approach on several test systems such as the IEEE 13-node and the 33- node distribution test systems. The rest of the paper is organized as follows. Section II describes the cooperative game theory including the Shapley value. Section III explains the proposed approach for tertiary frequency regulation. Section IV presents case studies on the modiﬁed IEEE 13-node and 33-node distribution systems. Section V provides concluding remarks. II. C OOPERATIVE GAME THEORY AND SHAPLEY VALUE A cooperative game (or coalitional game) is a special class of games in which each player forms alliances with other players to maximize its incentives. In each cooperative game, there are three components as follows: (a) a ﬁnite set of players N, and (b) a real-valued set function V, called characteristic function, deﬁned on all sub-sets of Nand satisﬁes V( ) = 0 . In game theory terms, Nis deﬁned as the player set, and V(S) : 2N!Ris deﬁned as the “worth” or “value” of coalitionS, i.e., the total utility that members of Scan acquire if a coalition is formed among themselves and the game is played without assistance from players in other coalitions. A. The Core of a Cooperative Game The set of feasible allocations that cannot be further im- proved through any other coalitions is referred to as the core. Generally, outcomes of a cooperative game are expressed as n-tuples of utility: =f i:i2Ng, called payoff vectors that are measured in some common unit of money [11]. The core is the set of imputations under which all sets of coalitions have values less than or equal to the sum of its members’ payoffs. Thus, is core if and only if, :eSV(S);8SN (1) :eN=V(N) (2) whereeSdenotes the n-vector having eS i= 1 ifi2Sand eS i= 0ifi2NS. Equation (1) is the stability (or coalitional rationality) criterion and equation (2) is the efﬁciency criterion.B. The Shapley Value The Shapley value fairly allocates the payoff among the players of the cooperative game. The Shapley value of a cooperative game is given as follows [12]. j(V) =X S22N;j2S(jSj1)!(njSj)! n![V(S)V(Snfjg)](3) wheren=jNjis the total number of players. The Shapley value satisﬁes the following axioms: 1)Efﬁciency:P j2N j(V) =V(N). 2)Individual Rationality: j(V)V(fjg);8j2N. 3)Symmetry: Ifjandkare such that V(S[fjg) =V(S[ fkg)for every coalition Snot containing jandk, then j(V) = k(V). 4)Dummy Axiom: Ifjis such that V(S) =V(S[fjg) for every coalition Snot containing j, then j(V) = 0 . 5)Additivity: IfVandWare characteristic functions, then (V+W) = (V) + (W). III. T HEPROPOSED APPROACH In this section, an approach for the fair allocation of reserves among participating DERs is developed using a cooperative game theoretic approach. The task of allocation of reserves among DERs is regarded as a cooperative game and the participating DERs are regarded as players of the game. As a motivation for DERs to participate in tertiary frequency regulation, DERs are allowed to send their bid prices for reserves. DERs also send information related to their sellable capacity to a virtual aggregator at the substation, which then sends this information to system operators. Fig. 1 shows the layout of the proposed cooperative game theoretic approach to allocate reserves in an active distribution system for the tertiary frequency regulation. The proposed cooperative game theoretic approach consists of two stages. In the ﬁrst stage, the two types of characteristic functions are computed: WI and PLR. The consideration of two types of characteristic functions, here, is equivalent to consideration of two types of objective functions in an optimization problem. In the second stage, Shapley values are computed and then reserve allocations of DERs are deter- mined. WI of each DER indicates the worth or value of each DER for the allocation of reserve so as to maximize the social welfare or beneﬁt. The two factors are taken into consideration for computing WI of each DER, which are available capacity for reserve (ACR) and reserve bid price (RBP). The ACR of the ithDER available after energy market clearance is calculated based on the total capacity of the ith DER available for selling, Pci, and the accepted capacity of theithDER after clearing energy market, Pei, as follows. ACR i=PciPei, (4) The total available capacity for reserve (TACR) from n players (here, DERs) is calculated as follows. TACR =nX i=1ACR i, (5) If the reserve command received from the system operator, PR, is equal to or higher than TACR, then all reserves are allocated without any optimization. But when PRis less than TACR, the tertiary frequency controller or regulator allocates PRoptimally among the participating DERs. The WI of the ith DER is deﬁned as follows. WIi=ACR i RBP i, (6) whereACR iis ACR of the ithDER andRBP iis reserve bid price of the ithDER. WI of each DER acts as the ﬁrst characteristic function of the proposed cooperative game and PLR acts as the second characteristic function. PLR is the difference between the active power loss of the system with DERs of a particular coalition and that without any DER. Shapley values 1;iand 2;iare calculated using (3) based on each type of characteristic function of participating DERs and their coalitions. The normalized Shapley values corre- sponding to each characteristic function are then calculated as follows. norm 1;i= 1;i.nX k=1 1;k, (7) norm 2;i= 2;i.nX k=1 2;k. (8) Using normalized Shapley values calculated in (7) and (8), the equivalent Shapley value of the ithDER is computed as follows. eqv i= norm 1;i+ norm 2;i 2(9) The distribution factor (DF) of the ithDER is then calculated using (10). The distribution factors are utilized to distribute PR among the participating DERs. DFi= eqv i.nX k=1 eqv k(10) The allocated reserve of the ith DER is then determined as follows. Ri=PRDFi (11) The proposed approach or the solution algorithm to de- termine reserve allocated for DERs for tertiary frequency regulation can be summarized as follows. 1) Read system data related to lines, loads, transformers, and DERs. 2) ReadPc,Pe, andRBP of each DER. 3) Determine available capacity for reserve (ACR) and worthiness index (WI) of each DER. 4) Enumerate all possible coalitions of DERs and compute two types of characteristic functions, viz., WI and PLR, of each coalition. 5) Compute Shapley values using (3) and normalized Shap- ley values using (7) and (8). 6) Compute the equivalent Shapley values using (9) and the distribution factors using (10).TABLE I DIFFERENT PARAMETERS OF DER S IN THE CASE OF THE MODIFIED IEEE 13-NODE SYSTEM DER 1 DER 2 DER 3 Pc(kW) 250 350 450 Pe(kW) 220 340 400 RBP ($/kW) 10 20 12 7) Determine the allocated reserves each DER using (11). The ﬂowchart of the proposed cooperative game theoretic approach is shown in Fig. 2. Start YesNoRead Pc, Pe, and RBP of each DERDetermine ACR and WI of each DER Compute power loss of the system without DERsEnumerate all possible coalitions of DERs Select a coalition of DERsCompute WI of the coalition Compute PLR of the coalitionAll coalitions evaluated? Compute the equivalent Shapley valuesCompute the distribution factor (DF) of each DER Determine the allocated reserves of DERsRead system data Stop Fig. 2. Flowchart of the proposed approach IV. C ASE STUDIES AND DISCUSSIONS The proposed approach is implemented on the modiﬁed IEEE 13-node and the modiﬁed 33-node distribution systems. The IEEE 13-node system is a 4:16kV distribution test system characterized by having overhead and underground lines, transformers, a voltage regulator, shunt capacitor banks, and unbalanced loading with constant current, power, and impedance models. The total real and reactive loads of this system are, respectively, 3577 kW and 1725 kV Ar. For the detailed data of the IEEE 13-node system, the readers are re- ferred to reference [13]. This system is modiﬁed by including three DERs at phase 1 of node 652, phase 2 of node 645, and phase 1 of node 675 as shown in Fig. 3. The33-node distribution test system is a 12:66kV radial distribution system with 33nodes and 32branches [14]. The total active and reactive power loads are 3715 kW and 2300 kV Ar, respectively. In this paper, the 33-node system is modiﬁed by placing four DERs of sellable capacity of 300 kW,200 kW,400 kW, and 200 kW at nodes 7,14,24, and 32, respectively, as shown in Fig. 4. G 646 645 632 633 634650 692 675 611 684 652671 680 G GDER-2 DER-1DER-3Fig. 3. The modiﬁed IEEE 13-node distribution system 12 34 5 67 89 10 11 12 13 14 15 16 17 2223 24 181920 2126 27 28 29 30 31 32 S/S25 33 GGG G DER -1DER -2 DER -4DER -3 Fig. 4. The modiﬁed 33-node distribution system TABLE II DIFFERENT PARAMETERS OF DER S IN THE CASE OF THE MODIFIED 33-NODE SYSTEM DER 1 DER 2 DER 3 DER 4 Pc(kW) 300 200 400 200 Pe(kW) 280 190 390 170 RBP ($/kW) 10 15 12 10 TABLE III CHARACTERISTIC FUNCTIONS : W ORTHINESS INDEX (WI) AND POWER LOSSREDUCTION FOR THE MODIFIED IEEE 13- NODE SYSTEM Coalitions Worthiness Power Loss of DERs Index (WI) Reduction (kW) 1 3.00 2.60 2 0.50 0.29 3 4.17 4.44 1,2 3.50 2.89 1,3 7.17 6.87 2,3 4.67 4.73 1,2,3 7.67 7.17 The proposed approach starts by by collecting information about sellable capacity ( Pc), energy market clearing capacity (Pe), and RBP of each DER. The ACR of each DER is then calculated. For the case of the modiﬁed IEEE 13-node system, the sellable capacities of DERs are 250 kW, 350 kW, and 450 kW. The energy market clearing capacities DERs for a particular timestamp under consideration are 220 kW, 340 kW, and 400 kW. Reserve bid prices of DERs are $10/kW, $20/kW, and $12/kW. These values are shown in Table I. Similarly, the different parameters of DERs in the case of the modiﬁed 33- node system are shown in Table II.TABLE IV CHARACTERISTIC FUNCTIONS : W ORTHINESS INDEX (WI) AND POWER LOSSREDUCTION FOR THE MODIFIED 33-NODE SYSTEM Coalitions of DERsWorthiness Index (WI)Power Loss Reduction (kW) 1 2.00 1.66 2 0.67 1.36 3 0.83 0.44 4 3.00 3.73 1, 2 2.67 3.01 1, 3 2.83 2.10 1, 4 5.00 5.37 2, 3 1.50 1.80 2, 4 3.67 5.08 3, 4 3.83 4.17 1, 2, 3 3.50 3.45 1, 2, 4 5.67 6.71 1, 3, 4 5.83 5.80 2, 3, 4 4.50 5.51 1, 2, 3, 4 6.50 7.14 TABLE V DISTRIBUTION FACTORS AND ALLOCATED RESERVES OF DER S FOR THE MODIFIED IEEE 13- NODE SYSTEM DERsProposed Approach Capacity-based Approach Dist. Allocated Dist. Allocated Factors Reserves (kW) Factors Reserves (kW) 1 0.3712 18.56 0.2381 11.90 2 0.0530 2.65 0.3333 16.67 3 0.5758 28.79 0.4286 21.43 As explained in Section II, a cooperative game is expressed in terms of a ﬁnite set and characteristic functions. In this paper, two types of characteristic functions, viz., WI and PLR, are considered. These characteristic functions are deﬁned for all possible sets of coalitions. PLR for a coalition is the difference between power loss of the system without DERs and that with DERs of that coalition. For the case of the modiﬁed IEEE 13-node system, the possible sets of coalitions of DERs and the corresponding values of WI and PLR are shown in Table III. As shown in the table, WI is 3.00 and PLR is 2.60 kW for DER-1. For DER-2, WI is 0.50 and PRL is 0.29 kW. For the coalition of DER 1 and 2, WI is 3.50 and PLR is 2.89 kW. Similarly, for the case of the modiﬁed 33-node system, the possible sets of coalitions and the corresponding values of WI and PLR are shown in Table IV. TABLE VI DISTRIBUTION FACTORS AND ALLOCATED RESERVES OF DER S FOR THE MODIFIED 33-NODE SYSTEM DERsProposed Approach Capacity-based Approach Dist. Allocated Dist. Allocated Factors Reserves (kW) Factors Reserves (kW) 1 0.2689 13.45 0.2727 13.64 2 0.1457 7.28 0.1818 9.09 3 0.0948 4.74 0.3636 18.18 4 0.4907 24.53 0.1818 9.09 The characteristic functions shown in Table III and IV are used to calculate Shapley values using (3), and the Shapley values are normalized based on (7) and (8). The equivalent Shapley values are then computed using (9) and ﬁnally the distribution factor ( DF) of each DER is calculated using (10). For the case of the modiﬁed IEEE 13-node system, the distribution factors of DERs 1, 2, and 3, respectively, are 0.3712, 0.0530, and 0.5758; and the allocated reserves of DERs (for PR= 50 kW) are 18.56 kW, 2.65 kW, and 28.79 kW as shown in Table V. Similarly, for the case of the modiﬁed 33-node system, the distribution factors of DERs 1, 2, 3, and 4, respectively, are 0.2689, 0.1457, 0.0948, and 0.4907; and the allocated reserves of DERs (for PR= 50 kW) are 13.45 kW, 7.28 kW, 4.74 kW, and 24.53 kW as shown in Table VI. Fig. 5. Bar graph showing the comparison of total reserve costs For comparison purpose, the allocated reserves computed using the proposed approach are compared with that using a DER-capacity-based approach in terms of total reserve cost. In the DER-capacity-based approach, the distribution factors are computed based on sellable capacities of DERs. The distribution factors and the allocated reserves of DERs obtained using DER-capacity-based approach for the modiﬁed IEEE 13-node and the modiﬁed 33-node systems are shown in Table V and Table VI respectively. In case of the modiﬁed IEEE 13-node system, the total reserve costs obtained using the proposed approach and the DER-capacity-based approach, respectively, are $584.06 and $709.52. Similarly, in case of the modiﬁed 33-node system, the total reserve costs obtained using the proposed approach and the DER-capacity-based approach, respectively, are $545.89 and $581.82. Fig. 5 shows the comparison of total reserve costs obtained using the proposed approach and the DER-capacity-based approach for both 13- node and 33-node systems. The result shows that the total reserve cost can be lowered when the reserves are allocated using the proposed approach. This is because of the use of Shapley values, which take marginal contribution of each player into account while allocating the reserves. When the study is performed on a PC with 64-bit Intel i5 core, 3.15 GHz processor, 8 GB RAM, and Windows OS, the execution time of the proposed approach is around 0.25 seconds for the 13-node and 33-node systems. V. C ONCLUSION In this paper, a cooperative game theoretic two-stage ap- proach for tertiary frequency regulation of active distribution systems has been proposed. In the ﬁrst stage, the two types ofcharacteristic functions, viz., worthiness index (WI) and power loss reduction (PLR), of each set of coalition of DERs were computed. In the second stage, the equivalent Shapley values and distribution factors were computed for fair allocation of reserves among DERs. In developing the proposed approach, the following variables/parameters of DERs have been taken into consideration: sellable capacity, market clearing capacity, and reserve bid price. The case studies were performed on the modiﬁed IEEE 13-node and the modiﬁed 33-node distribution test systems. The results demonstrate the effectiveness of the proposed approach for tertiary frequency regulation compared to a DER-capacity-based approach. ACKNOWLEDGEMENT This paper is based upon work supported by the U.S. Department of Energy’s Ofﬁce of Energy Efﬁciency and Re- newable Energy (EERE) under the Solar Energy Technologies Ofﬁce Award Number DE-EE0009022. The views expressed herein do not necessarily represent the views of the U.S. Department of Energy or the United States Government.