# A Trusted Routing Scheme Using Blockchain and Reinforcement Learning for Wireless Sensor Networks
Compared with other consensus mechanisms, the resource consumption of the PoW blockchain is high and the supervision is weak. At the same time, every time a PoW consensus is reached, the whole network needs to participate in the operation, which has low performance and efficiency. * PoS: The main idea of the PoS consensus mechanism is that the difficulty of obtaining the accounting right of a node is inversely proportional to the stake held by the node. According to the proportion and time of coins taken by each node, the difficulty of mining coins can be reduced in the same proportion so as to speed up the speed of finding random numbers. The greater the stake, the greater the privilege, the greater the responsibility to generate the block and the power to generate more revenue. * PoA: PoA is an improved algorithm of PoS that uses the verified identity of the nodes to replace the role of the stake rather than the monetary value. In a PoA blockchain, the transaction and the block are validated by an approved node (called a validator) without a huge computational overhead of a mining process. The validator must authenticate on the blockchain and the qualification is hard to acquire which means the validator will not have a motive for acting on their own interests. Even if there is a malicious validator, it will be kicked out by other validators' votes. In this way, the PoA blockchain becomes safer and cheaper. 4. The last technical element is the smart contract, which is based on the trusted and non-tampering data and can automatically execute the predefined codes by a blockchain miner [29]. The execution result of the smart contract updates the ledger status on the blockchain network. These changes cannot be falsified or tampered with once they are confirmed by a specific consensus mechanism because the content has been agreed upon in the blockchain network. In an open, reliable and distributed blockchain network, a routing node can acquire routing information including but not limited to its neighbor routing nodes. The efficiency of the routing can be improved if this routing information is properly used. Some routing schemes have introduced the reinforcement learning into the dynamically networks [30; 31; 32; 33; 34]. Reinforcement learning is a kind of machine learning algorithms represented by Q-learning, which gives the feedbacks into the selection of each step through the reward and punishment mechanism. A standard reinforcement learning algorithm consists of five parts: environment, agent, state, action and reward [35]. An agent interacts with the environment by performing actions. A state is an indicator of the situation of the agent, and each state has a corresponding set of actions for the agent to choose. The agent can only perform one action per state and get the reward which is the feedback on the success or failure of the action. In this work, we introduce a novel trusted routing scheme based on blockchain and reinforcement learning for WSNs. In particular, we use the blockchain technology to provide a distributed routing information management platform that all the routing information is recorded on the blockchain through the blockchain token transactions. The scheme takes advantage of the decentralized, tamper-proof and traceable characteristics of the blockchain transactions to improve the trustworthiness of the routing information between the routing nodes. We exploit the reinforcement learning to learn the dynamic, reliable and extensive routing information from the blockchain network. A dynamically updated reinforcement learning model is generated in each routing node through the dynamically updated reward value brought by the action (scheduling) of each state (packet location), so as to help the routing nodes make better routing decisions and select the more reliable and efficient routing links.