Blockchain Based Smart-Grid Stackelberg Model for Electricity Trading and Price Forecasting Using Reinforcement Learning  
[PERSON]  
1 Department of Computer Science and Engineering Brac University, Dhaka 1212, Bangladesh;  
[EMAIL_ADDRESS] (M.M.M.A.M. ); [EMAIL_ADDRESS] (R.S.U. );  
[EMAIL_ADDRESS] (M.A.H. );  
[EMAIL_ADDRESS] (M.G.R.A);  
1  
[PERSON]  
1 Department of Computer Science and Engineering Brac University, Dhaka 1212, Bangladesh;  
[EMAIL_ADDRESS] (M.M.M.A.M. ); [EMAIL_ADDRESS] (R.S.U. );  
[EMAIL_ADDRESS] (M.A.H. );  
[EMAIL_ADDRESS] (M.G.R.A);  
[EMAIL_ADDRESS] (M.G.R.A);  
1  
[PERSON]  
1 Department of Computer Science and Engineering Brac University, Dhaka 1212, Bangladesh;  
[EMAIL_ADDRESS] (M.M.A.M. ); [EMAIL_ADDRESS] (R.S.U. );  
[EMAIL_ADDRESS] (M.A.H. );  
[EMAIL_ADDRESS] (M.G.R.A);  
[EMAIL_ADDRESS] (M.G.R.A);  
1  
[PERSON]  
1 Department of Computer Science and Engineering Brac University, Dhaka 1212, Bangladesh;  
[EMAIL_ADDRESS] (M.M.A.M. ); [EMAIL_ADDRESS] (R.S.U. );  
[EMAIL_ADDRESS] (M.A.H. );  
[EMAIL_ADDRESS] (M.G.R.A);  
[EMAIL_ADDRESS] (M.G.R.A);  
1  
[PERSON]  
2 Information Systems Department, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia 22* 2  
Demand response (DR) is an efficient method to use in a smart grid system to reduce costs and improve grid efficiency. Price-based DR refers to influencing the customers' electricity usage with the variable electricity prices. To help with dynamic pricing in a hierarchical energy market, this paper proposes a DR algorithm that keeps up with the dynamic pricing while also reducing the service provider's (SP) and customer's (CU) costs. With the help of reinforcement learning (RL) and Q-Learning, we are predicting the price of electricity. The SPs possess the flexibility to set the price dynamically in accordance with demand and level of dissatisfaction. Furthermore, blockchain integration of every customer's profile will help to secure the decentralized transaction of electricity between SP and CU. In DR efficiency, we can see several works regarding DR models that help to subsidize customer usage by minimizing costs. There is one such example in [6; 7; 8], where electricity consumption of different home appliances was monitored, and time-of-use (TOU) pricing helped to minimize customers' costs. In [9; 10; 11; 12; 13; 14], we can see the benefit of predetermined next-day electricity prices and efficient scheduling to help keep the costs of CU in check. Though the contributions of the papers mentioned above helped the electricity demand response field to a good extent, it still lacks the dynamic market where demand is ever-changing. Thus, a DR strategy with dynamic pricing compatibility is apt for modern usage. Another smart way to provide a variable amount of service to customers is by using a dynamic pricing model, which changes the price of energy from time to time for perfect resource allocation [15]. One of the implementations can be found where retailer profit has been maximized with a quadratic programming problem [16]. The deterministic dynamic pricing approach and abstract linear models are sometimes unable to provide optimal performance due to any variable change in demand, resulting in loss of money, and secondly, abstract models visualize an approximation of energy distribution and are more dependent on the modeler's experience. This is where reinforcement is a viable solution, as it is model-free and can efficiently react to the ever-changing demand for CUs and benefit both CUs and SPs. Therefore, the key contributions of this research are as follows:  
* This research introduced a dynamic Stackelberg model-based retail price forecasting of electricity in a smart grid. The Stackelberg model considered two-stage pricing between electricity producers to retailers and retailers to customers. To enable adaptive and dynamic price forecasting, reinforcement learning is used.