# An On-Demand Charging for Connected Target Coverage in WSRNs Using Fuzzy Logic and Q-Learning +
## 4 Fuzzy Q-Charging Algorithm
### Reward Function  
Our objective is to maximize the network lifetime. To achieve this goal, we need to guarantee that every target is monitored by at least one sensor. Note that a target is considered to be monitored by a sensor if it is covered by a sensor (i.e., remaining within the sensor's sensing range) and the sensor is connected to the base station (i.e., a routing path exists between the sensor and the base station). Hence, the next charging location of MC should be selected to prioritize the following sensors:  
* The sensors with either high energy consumption rate or low level of remaining energy;
* The sensors either cover many targets or participate in many routing paths from target-covering sensors to the base station. We emphasize that our design can be applied to any routing protocol. However, in the experiments in Section 5, we adopt the geographic greedy routing protocol [32]. The geographic greedy routing protocol is widely accepted in WSNs due to its simplicity and efficiency. In this routing protocol, each node chooses the next hop to be the neighboring node closest to the destination. It is worth noting that in geographic routing, it is usually assumed that every node knows its location, and the location of its one-hop neighbors. This assumption can be realized by using positioning services [33] and the neighbor notification packets, respectively. Besides, the source node knows the position of the destination node. This assumption is legitimate in geographic routing [34; 35; 36; 37; 38; 39; 40; 41; 42]. To determine the next node, the current node only needs to look up in its neighbor table and find the one with the smallest distance to the destination. The computational complexity for determining such the next node is only \(O(m)\), where \(m\) is the number of the current node's one-hop neighbors. For each sensor \(S_{j}\), we define an _energy severity index_ (denoted as \(\xi_{j}\)) which is calculated by the ratio of \(S_{j}\)'s energy consumption rate \(e_{j}\) to its remaining energy \(E_{j}\) (\(\xi_{j}=\frac{e_{j}}{E_{j}}\)). We can see that the higher the energy severity index, the more critical the sensor is. Hence, the MC should charge a larger amount of energy to the sensor. We name a _priority index_ foreach sensor, which indicates the sensor's importance in covering targets and transferring sensory data to the base station. Specifically, a sensor \(S_{j}\) has the priority index \(w_{j}\), defined as a sum of its covered target number and the routing-path number. The routing path is from a target-covering sensor to the base station through \(S_{j}\). Similarly, the priority index is proportional to the significant effects on the target coverage and connectivity. The sensor with a higher priority index needs more energy from the MC than a lower-index one does. We reflect the observation mentioned above in the design of the reward function, which considers three factors _energy factor_, _sensor priority factor_, and _target monitoring factor_. The first two factors depict the relationship between the sensors' energy severity indexes and priority indexes with the energy they will be charged. The last factor estimates the number of monitored targets. We denote a charging location \(D_{i}\)'s energy factor, sensor priority factor, and target monitoring factor as \(\mathbf{E}(D_{i}),\mathbf{P}(D_{i})\), and \(\mathbf{T}(D_{i})\), respectively. The first two factors are designed based on a well-known fact that given two sorted number sets \(a_{1}\geq a_{2}\geq\ldots\geq a_{n}\) and \(b_{1}\geq b_{2}\geq\ldots\geq b_{n}\). Let \(\pi_{1}\) and \(\pi_{2}\) be two permutations of \(\{1,2,\ldots,n\}\), then \(\sum\limits_{i=1}^{n}a_{\pi_{1}(i)}b_{\pi_{2}(i)}\) attains the greatest value when \(\pi_{1}=\pi_{2}=\{1,\ldots,n\}\). Intuitively, \(\sum\limits_{i=1}^{n}a_{\pi_{1}(i)}b_{\pi_{2}(i)}\) is maximized when \(a_{\pi_{1}(i)}\) is proportional with \(b_{\pi_{2}(i)}\).