## 3 Preliminaries
### Problem Formulation  
Using the MDP, the problem is derived based on the values collected from different resources, including environmental, internal, and system history. Based on these parameters, the states and actions are defined and used. The states are defined as \(s\in S\), while actions are denoted as \(n\in N_{i}\). The state-changing probabilities are defined by \(Pr(s^{\prime}/s,n)=Pr(S_{t+1})=s\), \(A_{t}=n\). Based on these values, an expected reward can be obtained as \(r(S,s,n)\). The required reward depends on the state that changes from \(s\) to \(S\) on the expected action \(n\). All these values are provided to the agent for persistent action. The reward \(R_{t+1}\) is in \(R\), and this reward is the indication of a better or worse prediction. The reward is the amount of energy that is consumed by the sensor in any transaction. All sensors are in \(S_{Act}\), and after some cycles, the network adjusts itself using traffic conditions. Other parameters can also help calculate these values. These values are defined over a range \(1-N_{i}\) for each sensor. Other factors are the accuracy parameter \((A_{P})\) and the energy  
Figure 3: Generic model of ASC-RL.  
parameter \((E_{P})\) for each sensor. Both are used to balance accuracy and energy. When a sensor is in \(S_{Act}\), the agent gains a level of accuracy in the form of an event detection ratio. Four states with their reward functions are depicted in the equations below:  
\[R_{t}=\left\{A_{P}\sum_{n=1}^{n+1}k_{i}^{<i>}-NE_{P},::IfS_{n}=S_{Act}\right. \tag{12}\]  
\[R_{t}=\left\{N.E_{P}+A_{P}.\sum_{n=1}^{n+1}(k_{i}^{<i>}\times k_{i}^{<i>})-A_{P }.\sum_{n=1}^{n+1}(k_{i}^{<i>}-k_{i}^{<i>})::IfS_{n}=S_{Wt}\right. \tag{13}\]  
\[R_{t}=\left\{N.E_{P}+A_{P}.\sum_{n=1}^{n+1}(k_{i}^{<i>}\times k_{i}^{<i>})-A_{P }.\sum_{n=1}^{n+1}(k_{i}^{<i>}+k_{i}^{<i>})::IfS_{n}=S_{Rot}\right. \tag{14}\]  
\[R_{t}=\left\{N.E_{P}+A_{P}.\sum_{n=1}^{n+1}(k_{i}^{<i>}\times k_{i}^{<i>})-A_{ P}.\sum_{n=1}^{n+1}(k_{i}^{<i>}\times k_{i}^{<i>})::IfS_{n}=S_{Slp}\right. \tag{15}\]  
In Equations (12)-(15), the agent obtains accuracy based on the sum of the detected values, \(\mu\), which starts from any value \(n=1\) and can end at \(n+1\). For \(k_{i}^{<i>}=1\), \(k_{i}^{<i>}\) is the ith element of vector \(k^{i}\). Here, \(N\) is the total number of sensors at the time of state transition in a specific state \(S_{n}\). When all sensors are in the \(S_{Slp}\) state, the agent receives all the gains in \(E_{p}\). But in other cases, the gain in precision depends on the prediction of the \(\mu\) terms of events, which is \(k^{\prime<i>}\), in contrast to \(k^{i}=1\). The states are \(S_{n}=S_{Act},S_{Wt},S_{Rot}\), and \(S_{Slp}\). These equations ensure that the four states are based on the different parameters that are used in the state-changing policy. The three \(S_{Act}\), \(S_{Rot}\), and \(S_{Wt}\) ensure that the probability is 1, while in the case of \(S_{Slp}\), \(Pr(s/S,n)\), it will be 0 as given in the following equation:  
\[Pr(s/S,n)=\begin{cases}1&\text{if }Pr(j_{i}/j_{i-1};h_{i})::S_{n}=S_{Act}\\ 1&\text{if }s=S_{st}\text{ and }S_{n}=S_{Rot}\\ 1&\text{if }Pr(k_{i}/k_{i-1};h_{i})::S_{n}=S_{Wt}\\ 0&\text{otherwise}\end{cases} \tag{16}\]