# A Trusted Routing Scheme Using Blockchain and Reinforcement Learning for Wireless Sensor Networks
## 3 Approach
### Blockchain-Based Network Architecture  
To enhance the trustworthiness and robustness of the routing information, we introduce the blockchain which is essentially a distributed ledger with tamper-proof, decentralization, and information traceability characteristics into the wireless sensor network and use the blockchain token transactions to record related information of each node, as shown in Figure 3. The main framework is divided into two parts: the actual routing network and the blockchain network. In the framework, there are three kinds of entities: server node \(S\), routing node \(R\), and terminal device. In Figure 3, the actual routing network consists of routing nodes and terminals. Each routing node \(R\) has its own LAN to connect several terminals and is responsible for receiving packets from other routing nodes or terminals, as well as forwarding the received packets to the target nodes. We briefly describe the routing flow; packets from the source terminal to the destination terminal are transmitted to a routing node \(R_{i}\). \(R_{i}\) then selects the next-hop routing node \(R_{\pi}\) via the routing policy \(\pi\) obtained by the local learning model. The local learning model constantly queries and collects the relevant routing network state information from the blockchain network. After continuous transmission, the packets will be delivered to the target routing node \(R_{t}\) then to the destination terminal. Each blockchain system has a specific consensus algorithm to ensure the fairness of the blockchain transaction. In our blockchain network, we choose the PoA consensus algorithm which can process transactions more efficiently. In Figure 3, the red lines represent the PoA blockchain network, which is constituted by server nodes and routing nodes. They respectively represent two kinds of entities of the PoA blockchain network with different identities:  
Figure 3: The framework of the blockchain-based routing scheme. * Validator: The validators are the pre-authenticated nodes of the blockchain, which have advanced authorization and are responsible for the verification work in the PoA blockchain. In our system, each server node is a validator with higher rights in the PoA blockchain and has a unique blockchain address. Their specific tasks include executing smart contracts, verifying blockchain transactions, and releasing blocks on the blockchain. A new validator can be added by the authenticated validators election with more than 50% of the votes. Even if there is a malicious validator, it can only attack one of the contiguous blocks at most, during which time the malicious validator can be kicked out by other validator votes. * Minion: The minions are less-privileged nodes and cannot perform the verification work as validators in the PoA blockchain. In our system, each routing node is also a minion and has fewer rights in the PoA blockchain and it has a unique blockchain address, too. They can initiate token contracts, trigger some contract functions, and query the transaction information on the blockchain. In our scheme, these nodes in WSN can be static or dynamic. For example, the server nodes in our solution are often static, while the routing nodes can be dynamic. However, the entry and exit of nodes does not affect our scheme, because the status information of our blockchain-based system is also updated dynamically. On the blockchain network, we use different blockchain tokens to represent the different packets to be delivered to the target nodes that \(n\) unit tokens represent \(n\) unit corresponding packets. The essence of a token is the representation of the digitized information of the corresponding packets stored in the smart contract. The routing nodes can initiate token contracts to generate tokens and map the state information of related packets. They will make token transactions with each other via the token contract to transmit the tokens based on the sent and received packets.