## 1 Introduction  
The fast-growing electricity demand driven by the development of social economy has made the current power grid face serious challenges, such as grid security and power supply reliability [1, 2]. Since electricity cannot be largely stored, power grid has to constantly balance energy supply and demand, reducing supply when generation exceeds consumption and increasing supply when consumption exceeds generation. Once the supply is incapable of meeting the demand, blackout will happen which can be solved by building new power plants to promote the energy supply ability or reducing the energy consumption of consumers to alleviate the demand pressure [3]. Considering energy shortage periods only occur in a few time slots, it may be not very economically feasible to build a new power plant. Therefore, for this reason, demand response (DR) project has been applied extensively in demand side, including price-directed DR and incentive-directed DR [4, 5]. Because there exists a lot of flexible resource in demand side, especially in residential side, hence, a well-designed DR framework via introducing market mechanism can effectively yield peak shaving capability under emergent circumstances [6]. However, since single residential consumer has limited DR resource, residential consumers are usually not eligible to directly participate in DR market [7; 8]. Under the background, DR aggregator gradually appears, which is referred to as a cooperative consumer, virtual power plant and load aggregator in different contexts [7; 9]. Such aggregator, who is the intermediary institution between DR purchaser and consumers, will participate in the DR bidding market with other aggregators by aggregating consumers' DR resource. With the assistance of DR aggregator, more flexible resource in consumer side can participate in the DR project, and power grid can also economically relieve the supply pressure from consumer side [10]. In recent years, the bidding problem of DR aggregators for participation in the market has been widely investigated in the literature. Authors in [11; 12] presented optimal bidding strategy in the day-ahead market via load aggregator aggregating flexible DR resource. A price-based self-scheduling mixed-integer linear programming model for an aggregator was proposed in [13], with the aim to maximize the aggregator's payoff for participation in the day-ahead market. While in [14], an optimization model was proposed for planning and operating DR resource managed by a price-maker aggregator participating in the market, through both a deterministic and stochastic approach. Additionally, some examples on the bidding strategies of virtual power plants consisting of distributed generation and energy storage were also found in the literature [15; 16; 17; 18]. Concretely, study [15] concentrated on a novel DR scheme that avoids the need to predict the price elasticity of demand by ranking candidate load profiles of consumers in the preference order. The bidding problem was formulated in [17; 18] with reference to the day-ahead energy and spinning reserve markets. The presented problem was a nonlinear mixed-integer linear programming problem, which was solved by using a genetic algorithm. Different from above referred research, some literatures grasped the dynamic interaction characteristics of market participants in the process of strategy optimization and then introduced game theory into the DR mechanism [19; 20; 21; 22]. Specifically, some literatures focused on the bidding strategy optimization of DR aggregator with game theory [23; 24; 25; 26]. For example, research [24] employed non-cooperative game approach to analyze the bidding strategies of load aggregators in the day-ahead market. Authors in [25] proposed a game-theoretic approach to develop a decentralized aggregated control algorithm to seek for an optimal energy usage plan for a population of heterogeneous loads by employing potential game theory.