## 4 Proposed Q-Learning Method for Smart Grid Price Forecasting
### Q-Learning-Based Electricity Price Forecasting
#### 4.2.3 Using \(Q\)-Learning for Dynamic Pricing Problem
These functions need accurate and correct values in the parameters in order to run. After that, the events are executed. #### 5.2.3 Web3.js

Web3.js is a set of libraries that allows a system to communicate with an Ethereum node, either locally or remotely, using an HTTP or IPC connection. It establishes a connection between the smart contract and the blockchain. The Ethereum blockchain is accessed with the web3 JavaScript library. It can retrieve user accounts, send transactions, and communicate with smart contracts among other things. Several utility functions are also provided by using the web3.js, which makes development easier. Furthermore, the smart contract is loaded into the blockchain using web3.js. It is an API that connects every component of the whole application. #### 5.2.4 MetaMask Wallet

MetaMask Wallet is a cryptocurrency wallet that permits users to communicate with a blockchain with the help of a browser extension. The wallets provided by Ganache must be imported into the MetaMask Wallet using the private key of the accounts. A custom RPC must be created, which holds the IP [[http://127.0.0.1:7545](http://127.0.0.1:7545)]([http://127.0.0.1:7545](http://127.0.0.1:7545)) (accessed on 24 January 2022), the same as the Ganache RPC. These accounts confirm transactions and pay for the gas price required for buying energy. One of the imported accounts is named 'Seller', which can add selling items to the list, and another account is named 'Buyer', which can buy the listed

Figure 6: Ganache UI blockchain server. items and be the owner of that particular product. After that, they perform transactions. The user interface of the MetaMask wallet browser extension is displayed in Figure 7.

Table 1 contains the information on created blocks in the blockchain. It also stores the timestamps of the mined blocks and the amount of gas used to conduct that transaction. #### 5.2.5 React.js

The front end of the application is made with React.js. React is a front-end JavaScript library for creating user interfaces and UI designs. The front end connects to the blockchain using the web3 functions. Every time the seller adds an item to the list by providing the parameters (name, price), an item is added to the' Buy Electricity' list. It will also show the owner's address of the item, which will be the seller while the items are listed. Then the buyer can buy those particular items by pressing the' Buy' button and confirming the transactions with MetaMask Wallet. As a result, the Ether will be transferred to the seller \begin{table}
\begin{tabular}{c c c} \hline \hline
**Block No** & **Mined Date** & **Gas Used** \\ \hline
82 & 2 June 2021 3:53:33 & 52,511 \\
81 & 2 June 2021 3:53:01 & 117,930 \\
80 & 2 June 2021 3:49:43 & 117,954 \\
79 & 2 June 2021 3:48:47 & 117,918 \\
78 & 2 June 2021 3:48:17 & 117,906 \\
77 & 2 June 2021 3:48:02 & 132,894 \\
76 & 2 June 2021 3:45:42 & 745,906 \\
75 & 2 June 2021 3:45:42 & 244,636 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Gas used for every block. Figure 7: MetaMask wallet for seller and buyer. from the buyer, and the ownership of the product will be swapped, where the buyer's address will then be presented. Figure 8 depicts the user-friendly interface of the web application where buyers and sellers can make transactions. ### Implementation of Q-Learning-Based Electricity Price Forecasting

Input Data

* **Producer Input** Part of the input has five different electricity producers in the form of coal, nuclear, wind, water, and air. Since each of the producers produces different quantities of electricity at different times of the day and prices them accordingly, the simulation uses a 0/1 knapsack as the algorithm to choose the best producer with the best price for retailers. The algorithm takes the maximum capacity of weight as list W for each hour of the day [0,1,2,3], the list that contains the electricity production weight in list wt [C1, C2, C3,...], and the prices for production in values V [V1, V2, V3, V4].