## 4 Proposed Q-Learning Method for Smart Grid Price Forecasting
### Q-Learning-Based Electricity Price Forecasting
#### 4.2.3 Using \(Q\)-Learning for Dynamic Pricing Problem
Thus, it is beneficial for both the SP and CU, ensuring a win-win strategy. Furthermore, a secured blockchain transaction is paramount for the security of both the SP and CU, which is also discussed in this paper. In this paper, a blockchain based transaction

Figure 7:1 Sum of \(Q\) values. Figure 7:2 Convergence total reward for best policy. DApp has been created where it works as a virtual electricity market place for CU. SPs will list their selected prices for a particular amount of load in the virtual market, and CUs can buy it using Ether. As the transaction is decentralized and performed in cryptocurrency, the maximum security of the user's transaction activity is secured. The blockchain environment is created using a personal blockchain named Ganache. The frontend is constructed using react.js and the backend with smart contracts, which are a cumulation of transaction functions. Web3.js is used to connect the backend and front end. Metamask works as an online wallet to connect the blockchain with a browser. In the future, the current dynamic pricing model will be further improved upon by deeply analyzing the weighting factor and further scrutinizing the optimal between the provider, producer and customers. The blockchain system can be improved by implementing advanced storage options such as the interplanetary file sharing system (IPFS), which acts as decentralized online storage to store the information of users. User data in IPFS can have an encryption facility for more security and convenience. Although the blockchain transaction is highly secured, the PoW needed for authentication is relatively energy consuming. It takes a lot of computational power; therefore, in cases where a fast transaction is needed, efficiency can sometimes be an issue. This efficiency can be highly negligible in most cases, given the secured, reliable transaction method the blockchain system provides. Therefore, even though the computational power required for such a system is very high, the trade-off here is that security is heavily enhanced as a result. Conceptualization, M.M.M.A.M., R.S.U. and M.G.R.A. ; methodology, M.M.M.A.M., R.S.U., M.A.H., T.B.S., M.G.R.A., M.M.H. and M.R.H. ; software, M.A.H., T.B.S. ; validation, M.M.M.A.M., R.S.U., M.A.H. and M.G.R.A. ; formal analysis, R.S.U., M.M.H. and M.R.H. ; investigation, M.M.M.A.M., R.S.U., M.A.H., T.B.S., M.G.R.A. and M.M.H. ; resources, M.M.M.A.M., and M.M.M.A.M., R.S.U., M.A.H., T.B.S., M.G.R.A. and M.M.H. ; writing--original draft preparation, M.M.M.A.M., R.S.U., M.A.H., T.B.S., M.G.R.A. and M.M.H. ; writing--review and editing, M.M.M.A.M., R.S.U., M.A.H., T.B.S., M.G.R.A., M.M.H. and M.R.H. ; supervision, M.G.R.A. ; funding acquisition, M.M.H. All authors have read and agreed to the published version of the manuscript. This work was supported by King Saud University, Riyadh, Saudi Arabia, through Researchers Supporting Project number RSP-2021/18. This work was also supported in part by the MEIF-SCI FY2021 grant of the University of Maine System. Not applicable. The authors declare no conflict of interest. ## References

* [PERSON] et al. (2021) [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON] Smart anomaly detection in sensor systems: A multi-perspective review. _Inf. Fusion_**2021**, _67_, 64-79. [CrossRef]
* [PERSON] et al. (2022) [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON] [PERSON]; [PERSON] Energy Theft Detection in Smart Grids: Taxonomy, Comparative Analysis, Challenges, and Future Research Directions. _IEEE/CAA J. Autom. Sin._**2022**, _99_, 578-600. [CrossRef]
* [PERSON] et al. (2020) [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON] Evaluating smart grid renewable energy accommodation capability with uncertain generation using deep reinforcement learning. _Future Gener. Comput. Syst._**2020**, _110_, 647-657. [CrossRef]
* [PERSON] (2022) [PERSON] 5 Countries Investing Heavily in Smart Grid Tech.