Abstract-The vehicle-to-grid (V2G) system enables energy flow from the electric vehicles (EVs) to the grid. The distributed power of the EVs can either be sold to the grid or be used to provide frequency regulation service when V2G is implemented. A V2G control algorithm is necessary to decide whether the EV should be charged, discharged, or provide frequency regulation service in each hour. The V2G control problem is further complicated by the price uncertainty, where the electricity price is determined dynamically every hour. In this paper, we study the real-time V2G control problem under price uncertainty. We model the electricity price as a Markov chain with unknown transition probabilities and formulate the problem as a Markov decision process (MDP). This model features implicit estimation of the impact of future electricity prices and current control operation on long-term profits. The Q-learning algorithm is then used to adapt the control operation to the hourly available price in order to maximize the profit for the EV owner during the whole parking time. We evaluate our proposed V2G control algorithm using both the simulated price and the actual price from PJM in 2010. Simulation results show that our proposed algorithm can work effectively in the real electricity market and it is able to increase the profit significantly compared with the conventional EV charging scheme. Vehicle-to-grid (V2G) system enables the delivery of energy from the future electric vehicles (EVs) to the grid [1]- [3] . The batteries of the EVs in the V2G system can either provide power to the grid when parked or take power from the grid to charge the batteries. With V2G abilities, the EVs have a dual role in the electricity market. On one hand, they are power consumers when the batteries are being charged. On the other hand, they are power suppliers when they sell excessive energy from the batteries. As most of the vehicles are parked on an average of 96% of the time [4], they can be utilized as a power source besides transportation. However, each EV has a limited power capacity (10-20 kW) [2] and most of the services in the electricity market are carried out on a MW basis [5] . Therefore, an intermediate system called the V2G aggregator is needed to collect the small scale power from a large number of EVs in order to enter the electricity market. The primary objective of the EVs when parked is to charge batteries before their next departure. The conventional approach would start charging at the maximum rate once plugged in until the expected state-of-charge (SOC) is reached [6] . With the development of V2G technology, bulk power selling and frequency regulation can be provided via the V2G aggregator in order to bring revenues for the EV drivers.
However, REF ) discuss the same issue in the context of V2G control.
Real-time vehicle-to-grid control algorithm under price uncertainty
2011 IEEE International Conference on Smart Grid Communications (SmartGridComm)
2011 IEEE International Conference on Smart Grid Communications (SmartGridComm)
Engineering Computer Science