## 3 Materials and Methods
### Control Flow Graphs  
The key concept in the feature engineering process is the generation of control flow graphs. Rather than utilizing decompiled raw opcode at a sequence level to retain semantic information of how the contract operates within Ethereum Virtual Machine, this paper leveraged a graph representation approach for bytecode parsing. CFGs are often preferred over the raw opcode of bytecode as input representations for reinforcement learning (RL) methods because CFGs provide a structured and high-level abstraction of the program's execution flow. A CFG captures the possible paths that might be traversed through a program during its execution, which allows for a more comprehensive understanding of the logical flow and control dependencies within the code. This is particularly important in detecting vulnerabilities in smart contracts, where understanding the control flow can reveal critical insights into how different parts of the contract interact and where potential security flaws might be located. Also, RL algorithms like PPO rely on the environment's state representation to learn optimal policies. When the state space is well-structured, as with CFGs, the learning process becomes more efficient and effective. However, the inherent complexity of Ethereum bytecode, particularly in jump resolution, presents substantial challenges for static analysis and reduces the accuracy of extraction of CFGs by existing automated tools. EtherSolve, introduced by [PERSON] et al. [35], provided a novel static analysis tool leveraging symbolic execution of the Ethereum operand stack. This approach enables the resolution of jumps within the Ethereum bytecode, facilitating the construction of accurate CFGs for compiled smart contracts. The CFG is a directed graph that illustrates the execution flow within a program, where nodes represent the program's basic blocks--sequences of opcodes that lack jumps except in the final opcode--while edges connect potential successive basic blocks. The process of generating a CFG is detailed through the series of steps given below. 1. [leftmargin=*]
2. Bytecode parsing. It begins by identifying and removing the metadata section of the raw bytecode, followed by parsing the remaining bytes into opcodes. 3. Basic block identification. A basic block is a sequence of opcodes executed consecutively without any other instructions altering the flow of control. Specific opcodes JUMP (unconditional jumps), JUMPI (conditional jumps), STOP, REVERT, RETURN, INVALID and SELFDESTRUCT, mark the end of a basic block, while JUMPDEST marks the start. Figure 1: Distribution of smart contract by vulnerability type in training, validation and testing sets. Symbolic stack execution. This step is used to resolve the destinations of orphan jumps during CFG construction. Orphan jumps are common in smart contracts, especially when a function call returns, and their destinations need to be determined. The approach involves symbolically executing the stack by focusing only on opcodes that interact with jump addresses (PUSH, DUP, SWAP, AND and POP), while treating other opcodes as unknown. 4. Static data separation. It involves removing static data from the CFG by identifying and excluding sections of the code that are not executable. This is achieved by detecting the 0 xFE opcode, which marks the transition from executable code to static data with a representation for invalid instructions, and subsequently removes any unconnected basic blocks from the graph. 5. CFG decoration. It involves adding additional information to the control flow graph to make it more useful for analysts and subsequent static analysis tasks (e.g., for vulnerability detection). Specifically, EtherSolve highlights important components of the CFG, such as the dispatcher, fallback function, and final basic block. This paper utilized EtherSolve.jar, sourced from Github [36], to generate control flow graphs from the bytecode of smart contracts.