## 1 Introduction  
The study of effective learning for enhancing long-term memory has been a focal point in both the educational and academic fields. One such strategy, spaced repetition, has been proven to effectively enhance long-term memory by strategically scheduling review intervals [1, 2]. Quantitative experiments conducted by [PERSON] [3] illustrate that human memory decays with a delay since the last review of the content but can immediately be reinforced by reviewing it. Traditional approaches have explored applying rule-based strategies for spaced repetition scheduling, such as the Pimsliev [4] and Leitner [5] systems. These strategies are intuitive to understand but lack the flexibility to adopt individual learning patterns. In real-world applications, spaced repetition scheduling is currently widely used in practice, especially in online learning platforms, such as SuperMemo [6], Anki [7], Duolingo [8], and MaiMemo [9]. With the large number of online learners on e-learning platforms, it is feasible to collect large-scale training data [9, 10]. In addition, with the development of deep learning, the field has witnessed a paradigm shift towards data-driven methods, particularly deep reinforcement learning (DRL) [11, 12, 13]. Since review scheduling of spaced repetition can be viewed as a sequential decision-making process, it is natural to adopt DRL for spaced repetition optimization [14; 15; 16; 17]. [PERSON]'s study is a pioneer work that formulates instruction as a partially observable Markov decision process (POMDP) planning problem [14]. TADS [17] employs a time-interval prediction module and Dyna-style planning [18] for spaced repetition optimization. However, these methods can suffer several drawbacks in real-world applications. Firstly, the simulation environments used in these works are ruled-based without real-world data, which may deviate from real-world situations. Secondly, previous works formulated the reinforcement learning problem as deciding the next item, which implies only one item will be learned in each learning session. However, in realistic situations, the learning process contains multiple days while a number of items should be learned in a day. The main purpose of the spaced repetition algorithm is to determine the optimized interval to review an item. To address the above challenges, we propose the DRL-SRS framework, which stands for the deep reinforcement learning approach for optimizing spaced repetition scheduling framework. The framework contains two modules. Our work first introduces the Transformer-based Half-Life Regression (THLR) model, which predicts the half-life of learning items with greater accuracy than traditional models based on real-world data. By capturing the temporal dynamics of recall likelihood, THLR provides a more nuanced understanding of the process of forgetting. Furthermore, following the problem definition of SSP-MMC [9], we develop a simulation environment that emulates the learner's daily review process, considering both inter-day and intra-day time dynamics. Although previous works have tried simulating student interactions for educational systems [19], they are not feasible for spaced repetition optimization. Lastly, we propose a deep reinforcement learning model for spaced repetition optimization based on a Deep Q-Network (DQN) augmented with a Long Short-term Memory (LSTM) mechanism. Through extensive experiments, we demonstrate that our proposed framework significantly improves upon existing spaced repetition optimization methods. The results indicate that our approach not only enhances recall probabilities but also optimizes the scheduling process, leading to more efficient use of the learner's time and cognitive resources. The rest of this paper is organized as follows: Section 2 reviews related work in spaced repetition optimization and reinforcement learning. Section 3 discusses the background knowledge like models for human memory used in our framework and basics of reinforcement learning.