## 3 Materials and Methods
### Model Architectures
#### 3.3.1 Deep Q-Network
The findings of this study offer valuable insights that may encourage scholars and developers to explore the potential of reinforcement learning techniques for improving smart contract security and mitigating financial risks for DeFi stakeholders. The code is provided at [[https://github.com/jidlg361/SmartContractRL](https://github.com/jidlg361/SmartContractRL)]([https://github.com/jidlg361/SmartContractRL](https://github.com/jidlg361/SmartContractRL)). URL accessed on 19 May 2025

Conceptualization, R.; methodology, C.Z., J.J.d.L. and C.-S.K. ; software, J.J.d.L., C.-S.K. and C.Z. ; validation, J.J.d.L., C.-S.K. ; investigation, J.J.d.L., C.-S.K., C.Z., F.M., and R.; resources, J.J.d.L. ; data curation, J.J.d.L. ; writing--original draft preparation, J.J.d.L., C.-S.K., C.Z., F.M., and R.; writing--review and editing, J.J.d.L., C.-S.K., C.Z., F.M., and R.; visualization, J.J.d.L., C.-S.K. and C.Z. ; supervision, F.M. and R.; project administration, F.M. All authors have read and agreed to the published version of the manuscript. This research received no external funding. Not applicable. The dataset adopted in this study was obtained from the public dataset released by [PERSON] et al. [32] on HuggingFace [29]. Combined with other open-source datasets sorted in earlier research, this dataset is extensive, high-quality, and includes over 100,000 entries from active contracts on the Ethereum main net, where the source code of every contract was obtained via Etherscan API [30] and bytecode was retrieved using the 'web3' Python package. Author [PERSON] was employed by the company Honda R&D Europe (UK) Ltd. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest. ## Abbreviations

The following abbreviations are used in this manuscript: \begin{tabular}{l l} CFG & Control Flow Graph \\ CFGs & Control Flow Graphs \\ DeFi & Decentralized Finance \\ DQN & Deep Q-Network \\ FPR & False Positive Rate \\ MLP & Multi-Layer Perceptron \\ PPO & Proximal Policy Optimization \\ PR & Precision--Recall \\ RL & Reinforcement Learning \\ RLF & Reinforcement Learning Fuzzer \\ ROC & Receiver Operating Characteristic \\ SVM & Support Vector Machine \\ TPR & True Positive Rate \\ \end{tabular} 

## References

* [1] [PERSON] Ethereum: A Secure Decentralized Generalised Transaction Ledger. _Ethereum Proj. Yellow Pap._**2014**, _151_, 1-32. * [2] [PERSON] A Risk Assessment Method for Decentralized Finance(DeFi) with Fermatean Fuzzy AHP Approach. _Adv. Transdiscipl. Eng._**2023**, _42_, 1215-1223. [CrossRef]
* [3] [PERSON] [PERSON]; [PERSON]; [PERSON]; [PERSON] Vulnerability Detection in Smart Contracts: A Comprehensive Survey. _arXiv_**2024**, arXiv:2407.07922. [CrossRef]
* [4] [PERSON]; [PERSON]; [PERSON]; [PERSON] [PERSON]; [PERSON] [PERSON]; [PERSON]; [PERSON]; [PERSON] Understanding a revolutionary and flawed grand experiment in blockchain: The DAO attack. _J. Cases Inf. Technol._**2019**, _21_, 19-32. [CrossRef]* _[PERSON] et al. (2017)_ _[PERSON]; [PERSON]; [PERSON]; [PERSON] An in-depth look at the parity multisig bug._ Hacking_ **2017**_._
* _Explained: The BNB Chain Hack. 2022. Available online: [[https://www.halborn.com/blog/post/explained-the-bnb-chain-hack-october-2022](https://www.halborn.com/blog/post/explained-the-bnb-chain-hack-october-2022)]([https://www.halborn.com/blog/post/explained-the-bnb-chain-hack-october-2022](https://www.halborn.com/blog/post/explained-the-bnb-chain-hack-october-2022)) (accessed on 15 March 2025)._
* _SlowMist. Available online: [[https://hacked.slowmist.io/](https://hacked.slowmist.io/)]([https://hacked.slowmist.io/](https://hacked.slowmist.io/)) (accessed on 12 March 2025)._
* _[PERSON] et al. (2025)_ _[PERSON]; [PERSON]; [PERSON]; [PERSON] A detection model for false data injection attacks in smart grids based on graph spatial features using temporal convolutional neural networks._ Electr. Power Syst. Res._ **2025**_, 238, 111126.