# A Trusted Routing Scheme Using Blockchain and Reinforcement Learning for Wireless Sensor Networks
The existing routing schemes find it difficult to identify such malicious nodes, because the real-time change of the routing information between two routing nodes are difficult to be accurately distinguished. When a malicious node receives the packets of data from a neighbor node, it directly discard the packets and does not forward the packets of data to its next-hop neighbor node. This creates a data \"black hole\" in the network, hence it is named as a black hole attack which is hard to be perceived for routing nodes in WSNs [11; 12]. These malicious nodes may be attackers of the external intrusion or internal legitimate nodes captured by external attackers. Recently, trust management is a pervasive means to ensure the security of the routing network [13; 14; 15; 16; 17; 18; 19]. Its core approach is for each node to maintain and exploit a trust model that records the trust values of the neighbor routing nodes and make routing decisions. This method can effectively make the routing node choose the relatively reliable routing links according to the trust values. However, its application is limited since a single routing node can only get the trust values of the neighbor routing nodes, which is not completely compatible with the multi-hop distributed WSN. In view of the above security issues, a third-party intermediary is proposed to solve the trust problem between the routing nodes, but the intermediary is obviously not suitable for multi-hop distributed wireless sensor networks. Meanwhile, a third-party trust management center is likely to be attacked and controlled by malicious nodes, and therefore the security and fairness of the system cannot be guaranteed. As a trusted, decentralized, self-organizing ledger system, the blockchain is very suitable for multi-hop distributed wireless sensor networks [20; 21; 22; 23]. A lot researches on applying the blockchain to the routing algorithms are carried out in the past few years [24; 25; 26]. The blockchain is essentially a decentralized database maintained by multiple nodes, and it mainly deals with trust and security problems. We've summarized four core technical elements that enable the blockchain to provide trusted and secure services:  
1. The first is the distributed ledger which contains all the transactions on the blockchain. The contents of these transactions include the address of the receiver of the transaction, the amount of the transfer, the timestamp, the smart contract code, the execution result of the smart contract, etc. The transaction ledger is completed jointly by multiple nodes in different places. Each node in the blockchain keeps a complete ledger, so that no ledger information can be tampered. While all of the nodes can participate in monitoring the legality of transactions. Figure 1: The malicious node in a backpressure (BP) routing algorithm. 2. The second is the asymmetric encryption and authorization technology. The transaction information stored on the blockchain is public, but the account identity information is highly encrypted and can only be accessed under the authorization of the data owner, thus ensuring the security of data and personal privacy. 3. The third part, called consensus mechanism, is how all accounting nodes reach consensus to determine the effectiveness of a blockchain transaction, which is a means of preventing tampering. Some common consensus algorithms including proof of work (PoW), proof of stake (PoS), proof of authority (PoA), delegated proof of stake (DPoS) and proof of capacity (PoC) are discussed in [27; 28]. We introduce three mainstream blockchain consensus mechanisms relevant to our work:  
* PoW: Bitcoin, Dogecoin and Litecoin are among the digital currencies based on the PoW consensus mechanism. PoW algorithm relies on the node to carry out mathematical operations to find a random number and obtain the accounting right. A malicious node needs more than 51 percent of the network's computing power to take control of the blockchain network.