## 1 Introduction
For this reason, to execute an optimum offloading policy as a learning agent for an IoT system, we propose a Dynamic Online/Offline Q-learning-based approach for IoT-Edge Offloading called DO2 QIEO. In addition, we use a Post-Decision State (PDS)-based learning approach to learn the best joint offloading for online scenarios. To overcome this problem, the PDS-based learning algorithm uses the unique structure of the considered EH's state transitions, greatly boosting run-time performance and learning convergence speed. Hence, we utilize the EH module to extend the life of the battery. The current state and the Q-function value are used to determine the offloading policy, including the edge device and the offloading rate of IoT objects to determine the incentive or efficiency benefit based on energy consumption, task failure rate, and computation delay within, and time slot. This procedure, in turn, generates the most recent policy by updating the Q-function, the efficiency-based iterative Bellman equation in offline mode, and the PDS in online mode. Once the system is online, the circumstance must be changed. An online deep Q-learning algorithm has been presented to deliver the best policy on the fly. To expedite learning, the PDS method employs partially known knowledge about dynamic systems and allows the edge node to incorporate such knowledge into its learning experience. The DO2 QIEO method is based on the PDS, which can handle dynamic offloading in real-time. Thus, to enhance long-term offloading performance, we use our DL method to handle highly dynamic settings while also addressing computing complexity. We use Transfer Learning (TL) during exploration to accelerate convergence by minimizing unnecessary exploration. As a result, the TL is utilized in the technique to launch the system. Effective TL is required to maximize learning outcomes. Furthermore, some of the primary benefits include reduced training data, improved model generalization, and increased DL usability. Further, because the system can transition from offline to online, it could be used in real-world scenarios. The key contributions of this study are listed below. * Proposing a secure framework for integrating edge and blockchain technologies into IoT networks to ensure data protection and energy efficiency;
* Taking into account the computing servers and the status of the controllers, we design the optimization problem as an MDP by specifying state, action, and reward function simultaneously, as well as the dynamic features of IoT systems;
* Providing a platform for dynamic online/offline offloading for various IoT-edge applications called DO2 QIEO;
* Using the TL method can assist in achieving the best offloading strategy;
* Using the EH module to increase the battery's life and improve offloading performance;
* Increasing system performance by lowering energy consumption, reducing computational latency, increasing device efficiency, and minimizing the task failure rate. The rest of this paper is structured in the following manner. Section 2 deals with related work. Section 3 describes the system model and problem statements. Section 4 presents the proposed method. Section 5 deals with performance evaluation. Section 6 discusses the conclusions and future scope.