## 1 Introduction  
The use and development of renewables has grown continuously in the power sector owing to climate change, with renewable power generation becoming second in the electricity mix in 2018 [1]. Renewables installed more than 200 gigawatts in 2019, which is the largest increase to date [2]. It is expected that renewable-based power capacity will grow by 50% between 2019 and 2024 [3]. In particular, forecasts predict that wind capacity installations will triple by 2024 [3]. Wind power generation (WPG) is subject to high fluctuations and intermittent properties. The characteristics of WPG make it difficult to ensure power system reliability [4; 5]. Although various wind power forecasting methods such as the ensemble method [6], aggregated probabilistic method [7], and machine learning-based method [8] have been researched, uncertainty cannot be completely eliminated owing to the nature of wind-resource phenomena. An energy storage system (ESS) plays an essential role in managing the uncertainty of WPG [9]. ESSs for WPG are used in various applications such as frequency regulation [10], ramp rate mitigation [11], and demand response [12]. The basic role of an ESS is to charge the surplus energy and discharge the stored energy according to the operational objective. Therefore, the primary issue of the usage of ESS is the effective operation of the ESS. [PERSON] et al. proposed a stochastic mixed-integer linear programming approach to manage the mismatching of the renewable power generation uncertainty using ESS operation in the day-ahead market [13]. [PERSON] and [PERSON] presented stochastic ESS scheduling over an extended planning horizon [14]. [PERSON] et al. proposed stochastic ESS scheduling, considering a demand response [15]. [PERSON] et al. suggested a revised genetic algorithm (GA) to solve the ESS operation optimization problem for economic dispatch [16]. [PERSON] proposed particle swarm optimization and chaotic particle swarm optimization algorithms to minimize energy cost in renewable systems with ESS [17]. [PERSON] et al. suggested a chicken swarm optimization algorithm to improve the reliability of ESS for renewables [18]. [PERSON] and [PERSON] presented a frequency domain analysis-based ESS operation algorithm to reduce the uncertainty of WPG [19]. These studies show that ESSs effectively manage uncertainty to enhance the utilization of WPG. However, there are two constraints: First, the performance of the ESS is strongly related to the ESS capacity [19; 20; 21]. Second, for effective ESS operation, environment modeling of the characteristics of WPG is required in conventional stochastic and meta-heuristic approaches. This study focuses on reinforcement learning (RL)-based virtual ESS (VESS) operation to manage the uncertainty of WPG. VESS (also named cloud ESS or shared ESS) is an ESS that is linked to different end-units and promotes coordination [22]. VESS virtually operates the physical ESS for serving several units. Virtualization reduces the investment cost of ESS and increases its utilization [23; 24]. By applying VESS, the limitation of the ESS capacity can be addressed. Moreover, RL is a model-free approach [25]. In recent years, various studies have been conducted using RL for demand response [26], micro-grid systems [27; 28], frequency oscillation control [29], and energy trading [30]. The RL-based, model-free approach reduces difficulties related to modeling. The goal of this study is to design an RL-based VESS operation strategy to manage WPG forecast uncertainty. The VESS logically shares the physical ESS. The suggested VESS operation system considers not only its own operation but also the operation of other units. Under the system, the proposed RL approach to determine VESS operation is presented. Most previous studies used Q-learning-based RL approaches. However, in WPG environments, Q-learning is not efficient owing to the high penetration of uncertainty [31].