## 3 Methodology
### Problem Formulation  
In this section, we investigate the core elements of the teaching process and formulate them to facilitate subsequent research. First, the teaching process is typically defined as a partially observable Markov process, as shown in Figure 1--where \(S_{t}\) represents student knowledge stated in time \(t\), which cannot be directly observed; \(O_{t}\) represents student observation results in time \(t\), which is usually the score of text; and \(a_{t}\) represents student actions in time \(t\); which usually means the next knowledge point that the student will learn. The Markov property is also called no aftereffect, which means that the next state is only related to the current state and has nothing to do with the previous state. For example, in the process of playing Go, the next stage of the game is only related to the current state and the next move. How the game comes to be in its current state is not important. A Markov process is a random process that satisfies the Markov property and describes a series of changing processes of the state. The sequence of states generated in this process is referred to as a Markov chain. Fitting the law of state change to a functional relationship, we call it the state transition function. If the state transition function includes the action taken in the current state, that is, the state change will be affected by the current action, which is called the Markov decision process. Taking Go as an example. The next stage of the game is also affected by the next move, while the next state of a clock is only affected by the current state. In the process of Markov decision-making, we can usually obtain a complete picture of the exact state, or make it clear what the current state is. In a chess game, we can know the current state. In the process of student learning, we cannot know the current student's accurate state. We can only infer the student's state through testing and other methods. A Markov process like this is referred to as a partially observable Markov process, which means that only one sequence of observation states can be observed. Teaching path planning is a sequential decision-making problem under uncertain conditions, as shown in Figure 2. Through continuous teaching, a chain of knowledge points, a chain of student states, and a chain of test results are gradually formed. The constraints and assumptions we need to meet are presented as follows:  
* The total class time is limited, that is, the teacher can only have a limited number of classes for teaching, and each class can only teach one knowledge point;
* There is only one teacher, and one or more students, and all students participate in the course at the same time;
* There are three teaching objectives as the optimization target of teaching path planning: the ratio of students passing the final exam; the ratio of students achieving excellent results in the final exam; and the average score of students in the final exam. This poses an exam-oriented optimization target for our teaching path planning scheme;
* The knowledge points are directly related to each other and the learning of the current knowledge point will be affected by the learning of its anterior knowledge points. Figure 1: Partially observable Markov decision process [8]. Formally, we use \(S\) to represent the set of students and \(K\) to represent the set of knowledge points. In addition, \(preK_{i}\) represents the pre-knowledge point of the \(i\)-th knowledge point, and \(Seq\) represents the knowledge point sequence that will be taught by the teacher. More specifically, \(S_{i,k}\) represents the probability that the \(i\)-th student has mastered the \(k\)-th knowledge point. \(n\) represents the total number of class times, \(m\) represents the total number of students, \(p\) represents the requirement of passing rate, and \(e\) represents the requirement of an excellence rate.