# Analysis of College Students’ Ideological and Political Dynamics and Communication Path Based on Reinforcement Learning

Authors: Wenbin Wu; Hongwei Liu
Journal: Journal of Sensors
 (2022)

---

## Abstract

Contemporary college students are the main force of future national construction. Their ideological political dynamics are related to the development of the party and the country. Some students have some problems with study concepts and study habits. For a long time, the ideological political education of university students has not been paid attention to, resulting in the inability to accurately analyze the ideological political dynamics of university students. Grasping the ideological political dynamics of university students in the new era is the top priority of current educational work and an important guarantee for the development of ideological political education in universities. With the development of the times, communication channels are also constantly updated. The focus of this article is to analyze the ideological political dynamics and communication channels of university students. To some extent, traditional analysis methods cannot satisfy current research. This paper constructs an analysis model of university students’ ideological political dynamics and communication paths based on reinforcement learning. The Markov decision process and Monte Carlo method are used to analyze the ideological political dynamics and communication paths of college students. The results show the following: (1) the highest accuracy of reinforcement learning is 99.7%, and the lowest is 96.2%; the highest accuracy is 99.7%, and the lowest is 97.4%; the highest recall is 99.6%, and the lowest is 97.6%. (2) The average accuracy rate of reinforcement learning is 98.16%, the average accuracy rate is 98.75%, and the average recall rate is 98.65%. (3) In the ideological political dynamics of college students, the score of value orientation is 6.975, the score of learning status is 8.025, the score of consumption concept is 7.7, and the score of employment is 7.45. (4) In the communication path analysis, there are 12 people in interpersonal communication, 15 people in organizational communication, 21 people in mass communication, 28 people in network communication, and 24 people in Internet communication.


---

## Body

## 1. Introduction

As an important part of the youth group, the ideological political dynamics of college students cannot be ignored. Comprehensively analyze the ideological political dynamics and communication channels of university students to improve the effectiveness of ideological and political education for university students. This paper constructs an analysis model of university students’ ideological political dynamics and communication paths based on reinforcement learning and analyzes the ideological and political dynamics of university students. This paper provides a lot of support on the basis of previous results. Reinforcement learning is a popular model for analyzing problems [1]. Analyze behavior through trial-and-error interactions with dynamic environments. Reinforcement learning describes an algorithm similar to Q-learning for finding optimal policies [2]. Popular Q-learning algorithms overestimate action values under certain conditions [3]. A common model for reinforcement learning is the standard Markov decision process [4]. Reinforcement learning is developed from theories such as animal learning and parameter perturbation adaptive control [5]. The goal of reinforcement learning is to dynamically adjust parameters [6]. For maximum signal enhancement, the trend is known as an important content of the ideological political education of university students [7]. It is also an effective way to carry out ideological political education for university students. Understand the ideological dynamics of college students and apply the Internet to ideological political education [8]. Ideological political education must conform to the changes and trends of the form and keep innovating [9]. At present, ideological political education in universities should be combined with art education [10]. The continuous progress of information technology has broadened the dissemination path of university students’ ideological political dynamics [11]. Ideological politics teachers are one of the important ways to optimize the dissemination of ideological political education [12]. Reinforcement learning acquires learning information and updates parameters by receiving action rewards from the environment [13]. Reinforcement learning is mainly manifested in reinforcement signals [14]. Reinforcement learning focuses on online learning [15].


## 2. Theoretical Bases

### 2.1. Reinforcement Learning

#### 2.1.1. Overview

Reinforcement learning (RL) [16] is a type of goal-oriented learning. The reinforcement learning process is the continuous interaction between the agent and the environment. In this process, the agent continuously observes the characteristics of the environment state and takes actions on the current environment according to certain policy rules. The environment gives feedback on actions taken in the form of rewards. The agent updates the policy based on the reward value to get a better reward for the next action it takes.The basic framework of reinforcement learning is shown in Figure1.Figure 1
The basic framework of reinforcement learning.


#### 2.1.2. Markov Decision Process

The Markov decision process (MDP) [17] is a mathematical description that can be provided for reinforcement learning, and most reinforcement learning problems can be modeled as an MDP. MDP adds action elements to the transition probability from one state to another state, enriching the Markov feature, and can be expressed as
(1)psi+1si,ai,⋯,s0,a0=psi+1si,ai.For the answer, MDP consists of 5 basic elements, namely,S,A,R,P,γ. Among them, S is the state space, which can reflect all the state sets of the complete information of the system; s is the current state, s∈S; A is the limited action space, which is composed of all possible actions; a is the currently taken action, a∈A; Rs,a is the reward function, which represents the expectation of the reward value that the agent can get from the current state s to the next state s′; Ps′s represents the probability of transitioning from state s to state s′; and λ represents the discount factor, which is a random float in the range of 0 to 1. Points can be used to determine whether the total reward is discounted or not.To find the optimal strategy, that is, to find the optimal state, the action value is(2)Q∗s,a=Rs,a+λ∑s′∈SPs′smaxa∈AQs′,a′,where Rs,a is the average of instant reward rs,a.


#### 2.1.3. Exploration and Utilization

The purpose of reinforcement learning is to obtain the optimal result; that is, the agent gets the maximum reward. Therefore, during the training process, it is necessary for the agent to perform actions according to the behavior that can obtain the maximum reward value. At the same time, considering that the “trial-and-error” experience experienced by the agent is not necessarily rich, only the local optimal solution can be obtained, so the agent cannot blindly use the existing experience to make actions, but it is necessary to improve the agent’s exploration to find new and latest ability to solve. With limited time, we need to rely on strategies to find a balance between exploration and exploitation.


#### 2.1.4. Strategy

Policy refers to the operation policy of the agent in MDP, which is a function that can calculate the output. In reinforcement learning, policies can be defined as deterministic policies and stochastic policies. Deterministic strategy means that in the same state, the action output by the agent is deterministic and unique; on the contrary, in the same state under the stochastic strategy, the output behavior of the agent is not unique but follows a specific probability distribution, but the sum of all possible output behavior probabilities in the same state should be equal to 1.


#### 2.1.5. Value Function

During the interaction between the agent and the environment, actions need to be evaluated to ensure that the final action set can obtain the maximum reward. There are two evaluation mechanisms here, namely, the value function and theQ function. The value function refers to the value function of the state, which measures the pros and cons of the agent’s state under the policy. A value function can be defined as follows:
(3)Vπs=EπRtst=s.The above formula expresses the expected reward that can be obtained by following policyπ in state s. Among them, Rt represents the reward obtained by the agent from the environment at time t, which can be expressed as follows:
(4)Rt=rt+1+rt+2+⋯+rT.And if it is in a continuous situation, there may be no final state, namely,(5)Rt=rt+1+rt+2+⋯.A discount factor is required to reward the discount, which can be expressed as(6)Rt=rt+1+γrt+2+γ2rt+3⋯,Rt=∑k=0∞γkrt+k+1.Among them, ifγ is 0, the reward is an immediate reward, and if γ is 1, the reward is mainly reflected in the future reward. Therefore, the value function can be expressed as
(7)Vπs=Eπ∑k=0∞γrt+k+1st=s.TheQ function, also known as the state action value function, is used to measure the pros and cons of the agent following the policy and performing the actions in the state. The Q function can be defined as follows:
(8)Qπs,a=EπRtst=s,at=a.The above formula represents the expected reward that can be obtained by following policyπ and taking action a in state s. This formula can be expressed as
(9)Qπs,a=Eπγkrt+k+1st=s,at=a.The value function is used to evaluate the state, and theQ function is used to evaluate the action [18]. Further derivation of the value function can be obtained:
(10)Vπs=Eπrt+1+γrt+2+γ2rt+3+⋯st=s,Vπs=Eπrt+1+γrt+2+γrt+3+⋯st=s,Vπs=Eπrt+1+γVπst+1st=s.Similarly, theQ function can also be derived as
(11)Qπs,a=Eπrt+1+γQπst+1,at+1st=s,at=a.According to the derivation of the above value function andQ function [19]. This can be further extended to the Bellman equations of both:
(12)Vπs=∑aπas∑s′ps′s,aRs′s,a+γVπs′,Qπs,a=∑s′ps′s,aRs′s,a+γ∑a′Qπs′,a′.The value function that produces the maximum value should satisfy(13)V∗s=maxπVπs.Likewise, the optimal strategy should be better than or equal to any other strategy. The optimal policy can produce the optimal value function [20]. That is, the maximum value of the Q function is the optimal cost function:
(14)V∗s=maxaQ∗s,a.Combining the above formula, the optimal cost equation can be obtained:(15)V∗s=maxa∑s′ps′s,aRs′s,a+γ∑a′Qπs′,a′.


### 2.2. Commonly Used Reinforcement Learning Algorithms

#### 2.2.1. Monte Carlo Method

For the Monte Carlo method [21], a very important advantage is that it does not need to know the environment, only needs to get the experience represented by the Markov quadruple interacting with the environment, and then solves the reinforcement learning problem by averaging the returns of the samples. The state value function at this time can be written as
(16)Qπst,at=1m∑i=1mRτi.Among them,τi indicates that in state st, action at has always used the trajectory data generated by strategy π, and Rτi indicates the sum of all rewards on this trajectory. When updating the action value function, an incremental method can be used to implement the Monte Carlo method.


#### 2.2.2. Timing Differential Method

Sutton proposed the temporal difference algorithm, which combines Monte Carlo and dynamic programming methods [22]. It is an important learning algorithm in reinforcement learning. This method can learn in some continuous state.The standard temporal difference method is a model-free algorithm that learns directly from experience and estimates the current state value after one or more steps of action. The most basic one-step update is the TD(0) algorithm [23]. When using a table of values, the iterative formula for the TD(0) algorithm is
(17)Vst⟵Vst+αrt+1+γVst+1−Vst,where Vst is the value function of state st at time t.The TD method is also called the TD(0) method, because this method updates the value function with the corresponding subsequent state after one step. We can define the general form of step return as(18)Gtn=Rt+1+λRt+2+λ2Rt+3+⋯+λn−1Rt+n.At this time, the update of the value function becomes(19)VSt⟵VSt+αGTn−VSt.


#### 2.2.3. Sarsa Learning

The name of the Sarsa algorithm comes from the 5 variables used when the value function is updated, which are the current states, the action a in the current state, the reward r of the current action, the next state s′ reached, and the assumed next state. The action consists of a′.In the current states and action a, after the state transitions to another state s′, the current action cost function Qs,a must be updated. Then, after reaching the next state, update the next action cost function until the end. This cost is updated as follows:
(20)Qs,a⟵Qs,a+αr+γQs′,s′−Qs,a,where α the learning is the rate and γ is the decay factor.


#### 2.2.4.Q-Learning

Q-learning is a temporal difference algorithm under the off-track strategy. The off-track strategy means that the strategy for determining the current behavior is different from the strategy for updating the value function. The agent chooses the action in the current state through a strategy and interacts with the environment, but then, when the value function is updated, it uses another strategy. The action-value function update formula for Q-learning is as follows:
(21)Qst,at⟵Qst,at+αrt+1+γmaxaQst+1,a−Qst,at.


## 2.1. Reinforcement Learning

### 2.1.1. Overview

Reinforcement learning (RL) [16] is a type of goal-oriented learning. The reinforcement learning process is the continuous interaction between the agent and the environment. In this process, the agent continuously observes the characteristics of the environment state and takes actions on the current environment according to certain policy rules. The environment gives feedback on actions taken in the form of rewards. The agent updates the policy based on the reward value to get a better reward for the next action it takes.The basic framework of reinforcement learning is shown in Figure1.Figure 1
The basic framework of reinforcement learning.


### 2.1.2. Markov Decision Process

The Markov decision process (MDP) [17] is a mathematical description that can be provided for reinforcement learning, and most reinforcement learning problems can be modeled as an MDP. MDP adds action elements to the transition probability from one state to another state, enriching the Markov feature, and can be expressed as
(1)psi+1si,ai,⋯,s0,a0=psi+1si,ai.For the answer, MDP consists of 5 basic elements, namely,S,A,R,P,γ. Among them, S is the state space, which can reflect all the state sets of the complete information of the system; s is the current state, s∈S; A is the limited action space, which is composed of all possible actions; a is the currently taken action, a∈A; Rs,a is the reward function, which represents the expectation of the reward value that the agent can get from the current state s to the next state s′; Ps′s represents the probability of transitioning from state s to state s′; and λ represents the discount factor, which is a random float in the range of 0 to 1. Points can be used to determine whether the total reward is discounted or not.To find the optimal strategy, that is, to find the optimal state, the action value is(2)Q∗s,a=Rs,a+λ∑s′∈SPs′smaxa∈AQs′,a′,where Rs,a is the average of instant reward rs,a.


### 2.1.3. Exploration and Utilization

The purpose of reinforcement learning is to obtain the optimal result; that is, the agent gets the maximum reward. Therefore, during the training process, it is necessary for the agent to perform actions according to the behavior that can obtain the maximum reward value. At the same time, considering that the “trial-and-error” experience experienced by the agent is not necessarily rich, only the local optimal solution can be obtained, so the agent cannot blindly use the existing experience to make actions, but it is necessary to improve the agent’s exploration to find new and latest ability to solve. With limited time, we need to rely on strategies to find a balance between exploration and exploitation.


### 2.1.4. Strategy

Policy refers to the operation policy of the agent in MDP, which is a function that can calculate the output. In reinforcement learning, policies can be defined as deterministic policies and stochastic policies. Deterministic strategy means that in the same state, the action output by the agent is deterministic and unique; on the contrary, in the same state under the stochastic strategy, the output behavior of the agent is not unique but follows a specific probability distribution, but the sum of all possible output behavior probabilities in the same state should be equal to 1.


### 2.1.5. Value Function

During the interaction between the agent and the environment, actions need to be evaluated to ensure that the final action set can obtain the maximum reward. There are two evaluation mechanisms here, namely, the value function and theQ function. The value function refers to the value function of the state, which measures the pros and cons of the agent’s state under the policy. A value function can be defined as follows:
(3)Vπs=EπRtst=s.The above formula expresses the expected reward that can be obtained by following policyπ in state s. Among them, Rt represents the reward obtained by the agent from the environment at time t, which can be expressed as follows:
(4)Rt=rt+1+rt+2+⋯+rT.And if it is in a continuous situation, there may be no final state, namely,(5)Rt=rt+1+rt+2+⋯.A discount factor is required to reward the discount, which can be expressed as(6)Rt=rt+1+γrt+2+γ2rt+3⋯,Rt=∑k=0∞γkrt+k+1.Among them, ifγ is 0, the reward is an immediate reward, and if γ is 1, the reward is mainly reflected in the future reward. Therefore, the value function can be expressed as
(7)Vπs=Eπ∑k=0∞γrt+k+1st=s.TheQ function, also known as the state action value function, is used to measure the pros and cons of the agent following the policy and performing the actions in the state. The Q function can be defined as follows:
(8)Qπs,a=EπRtst=s,at=a.The above formula represents the expected reward that can be obtained by following policyπ and taking action a in state s. This formula can be expressed as
(9)Qπs,a=Eπγkrt+k+1st=s,at=a.The value function is used to evaluate the state, and theQ function is used to evaluate the action [18]. Further derivation of the value function can be obtained:
(10)Vπs=Eπrt+1+γrt+2+γ2rt+3+⋯st=s,Vπs=Eπrt+1+γrt+2+γrt+3+⋯st=s,Vπs=Eπrt+1+γVπst+1st=s.Similarly, theQ function can also be derived as
(11)Qπs,a=Eπrt+1+γQπst+1,at+1st=s,at=a.According to the derivation of the above value function andQ function [19]. This can be further extended to the Bellman equations of both:
(12)Vπs=∑aπas∑s′ps′s,aRs′s,a+γVπs′,Qπs,a=∑s′ps′s,aRs′s,a+γ∑a′Qπs′,a′.The value function that produces the maximum value should satisfy(13)V∗s=maxπVπs.Likewise, the optimal strategy should be better than or equal to any other strategy. The optimal policy can produce the optimal value function [20]. That is, the maximum value of the Q function is the optimal cost function:
(14)V∗s=maxaQ∗s,a.Combining the above formula, the optimal cost equation can be obtained:(15)V∗s=maxa∑s′ps′s,aRs′s,a+γ∑a′Qπs′,a′.


## 2.1.1. Overview

Reinforcement learning (RL) [16] is a type of goal-oriented learning. The reinforcement learning process is the continuous interaction between the agent and the environment. In this process, the agent continuously observes the characteristics of the environment state and takes actions on the current environment according to certain policy rules. The environment gives feedback on actions taken in the form of rewards. The agent updates the policy based on the reward value to get a better reward for the next action it takes.The basic framework of reinforcement learning is shown in Figure1.Figure 1
The basic framework of reinforcement learning.


## 2.1.2. Markov Decision Process

The Markov decision process (MDP) [17] is a mathematical description that can be provided for reinforcement learning, and most reinforcement learning problems can be modeled as an MDP. MDP adds action elements to the transition probability from one state to another state, enriching the Markov feature, and can be expressed as
(1)psi+1si,ai,⋯,s0,a0=psi+1si,ai.For the answer, MDP consists of 5 basic elements, namely,S,A,R,P,γ. Among them, S is the state space, which can reflect all the state sets of the complete information of the system; s is the current state, s∈S; A is the limited action space, which is composed of all possible actions; a is the currently taken action, a∈A; Rs,a is the reward function, which represents the expectation of the reward value that the agent can get from the current state s to the next state s′; Ps′s represents the probability of transitioning from state s to state s′; and λ represents the discount factor, which is a random float in the range of 0 to 1. Points can be used to determine whether the total reward is discounted or not.To find the optimal strategy, that is, to find the optimal state, the action value is(2)Q∗s,a=Rs,a+λ∑s′∈SPs′smaxa∈AQs′,a′,where Rs,a is the average of instant reward rs,a.


## 2.1.3. Exploration and Utilization

The purpose of reinforcement learning is to obtain the optimal result; that is, the agent gets the maximum reward. Therefore, during the training process, it is necessary for the agent to perform actions according to the behavior that can obtain the maximum reward value. At the same time, considering that the “trial-and-error” experience experienced by the agent is not necessarily rich, only the local optimal solution can be obtained, so the agent cannot blindly use the existing experience to make actions, but it is necessary to improve the agent’s exploration to find new and latest ability to solve. With limited time, we need to rely on strategies to find a balance between exploration and exploitation.


## 2.1.4. Strategy

Policy refers to the operation policy of the agent in MDP, which is a function that can calculate the output. In reinforcement learning, policies can be defined as deterministic policies and stochastic policies. Deterministic strategy means that in the same state, the action output by the agent is deterministic and unique; on the contrary, in the same state under the stochastic strategy, the output behavior of the agent is not unique but follows a specific probability distribution, but the sum of all possible output behavior probabilities in the same state should be equal to 1.


## 2.1.5. Value Function

During the interaction between the agent and the environment, actions need to be evaluated to ensure that the final action set can obtain the maximum reward. There are two evaluation mechanisms here, namely, the value function and theQ function. The value function refers to the value function of the state, which measures the pros and cons of the agent’s state under the policy. A value function can be defined as follows:
(3)Vπs=EπRtst=s.The above formula expresses the expected reward that can be obtained by following policyπ in state s. Among them, Rt represents the reward obtained by the agent from the environment at time t, which can be expressed as follows:
(4)Rt=rt+1+rt+2+⋯+rT.And if it is in a continuous situation, there may be no final state, namely,(5)Rt=rt+1+rt+2+⋯.A discount factor is required to reward the discount, which can be expressed as(6)Rt=rt+1+γrt+2+γ2rt+3⋯,Rt=∑k=0∞γkrt+k+1.Among them, ifγ is 0, the reward is an immediate reward, and if γ is 1, the reward is mainly reflected in the future reward. Therefore, the value function can be expressed as
(7)Vπs=Eπ∑k=0∞γrt+k+1st=s.TheQ function, also known as the state action value function, is used to measure the pros and cons of the agent following the policy and performing the actions in the state. The Q function can be defined as follows:
(8)Qπs,a=EπRtst=s,at=a.The above formula represents the expected reward that can be obtained by following policyπ and taking action a in state s. This formula can be expressed as
(9)Qπs,a=Eπγkrt+k+1st=s,at=a.The value function is used to evaluate the state, and theQ function is used to evaluate the action [18]. Further derivation of the value function can be obtained:
(10)Vπs=Eπrt+1+γrt+2+γ2rt+3+⋯st=s,Vπs=Eπrt+1+γrt+2+γrt+3+⋯st=s,Vπs=Eπrt+1+γVπst+1st=s.Similarly, theQ function can also be derived as
(11)Qπs,a=Eπrt+1+γQπst+1,at+1st=s,at=a.According to the derivation of the above value function andQ function [19]. This can be further extended to the Bellman equations of both:
(12)Vπs=∑aπas∑s′ps′s,aRs′s,a+γVπs′,Qπs,a=∑s′ps′s,aRs′s,a+γ∑a′Qπs′,a′.The value function that produces the maximum value should satisfy(13)V∗s=maxπVπs.Likewise, the optimal strategy should be better than or equal to any other strategy. The optimal policy can produce the optimal value function [20]. That is, the maximum value of the Q function is the optimal cost function:
(14)V∗s=maxaQ∗s,a.Combining the above formula, the optimal cost equation can be obtained:(15)V∗s=maxa∑s′ps′s,aRs′s,a+γ∑a′Qπs′,a′.


## 2.2. Commonly Used Reinforcement Learning Algorithms

### 2.2.1. Monte Carlo Method

For the Monte Carlo method [21], a very important advantage is that it does not need to know the environment, only needs to get the experience represented by the Markov quadruple interacting with the environment, and then solves the reinforcement learning problem by averaging the returns of the samples. The state value function at this time can be written as
(16)Qπst,at=1m∑i=1mRτi.Among them,τi indicates that in state st, action at has always used the trajectory data generated by strategy π, and Rτi indicates the sum of all rewards on this trajectory. When updating the action value function, an incremental method can be used to implement the Monte Carlo method.


### 2.2.2. Timing Differential Method

Sutton proposed the temporal difference algorithm, which combines Monte Carlo and dynamic programming methods [22]. It is an important learning algorithm in reinforcement learning. This method can learn in some continuous state.The standard temporal difference method is a model-free algorithm that learns directly from experience and estimates the current state value after one or more steps of action. The most basic one-step update is the TD(0) algorithm [23]. When using a table of values, the iterative formula for the TD(0) algorithm is
(17)Vst⟵Vst+αrt+1+γVst+1−Vst,where Vst is the value function of state st at time t.The TD method is also called the TD(0) method, because this method updates the value function with the corresponding subsequent state after one step. We can define the general form of step return as(18)Gtn=Rt+1+λRt+2+λ2Rt+3+⋯+λn−1Rt+n.At this time, the update of the value function becomes(19)VSt⟵VSt+αGTn−VSt.


### 2.2.3. Sarsa Learning

The name of the Sarsa algorithm comes from the 5 variables used when the value function is updated, which are the current states, the action a in the current state, the reward r of the current action, the next state s′ reached, and the assumed next state. The action consists of a′.In the current states and action a, after the state transitions to another state s′, the current action cost function Qs,a must be updated. Then, after reaching the next state, update the next action cost function until the end. This cost is updated as follows:
(20)Qs,a⟵Qs,a+αr+γQs′,s′−Qs,a,where α the learning is the rate and γ is the decay factor.


### 2.2.4.Q-Learning

Q-learning is a temporal difference algorithm under the off-track strategy. The off-track strategy means that the strategy for determining the current behavior is different from the strategy for updating the value function. The agent chooses the action in the current state through a strategy and interacts with the environment, but then, when the value function is updated, it uses another strategy. The action-value function update formula for Q-learning is as follows:
(21)Qst,at⟵Qst,at+αrt+1+γmaxaQst+1,a−Qst,at.


## 2.2.1. Monte Carlo Method

For the Monte Carlo method [21], a very important advantage is that it does not need to know the environment, only needs to get the experience represented by the Markov quadruple interacting with the environment, and then solves the reinforcement learning problem by averaging the returns of the samples. The state value function at this time can be written as
(16)Qπst,at=1m∑i=1mRτi.Among them,τi indicates that in state st, action at has always used the trajectory data generated by strategy π, and Rτi indicates the sum of all rewards on this trajectory. When updating the action value function, an incremental method can be used to implement the Monte Carlo method.


## 2.2.2. Timing Differential Method

Sutton proposed the temporal difference algorithm, which combines Monte Carlo and dynamic programming methods [22]. It is an important learning algorithm in reinforcement learning. This method can learn in some continuous state.The standard temporal difference method is a model-free algorithm that learns directly from experience and estimates the current state value after one or more steps of action. The most basic one-step update is the TD(0) algorithm [23]. When using a table of values, the iterative formula for the TD(0) algorithm is
(17)Vst⟵Vst+αrt+1+γVst+1−Vst,where Vst is the value function of state st at time t.The TD method is also called the TD(0) method, because this method updates the value function with the corresponding subsequent state after one step. We can define the general form of step return as(18)Gtn=Rt+1+λRt+2+λ2Rt+3+⋯+λn−1Rt+n.At this time, the update of the value function becomes(19)VSt⟵VSt+αGTn−VSt.


## 2.2.3. Sarsa Learning

The name of the Sarsa algorithm comes from the 5 variables used when the value function is updated, which are the current states, the action a in the current state, the reward r of the current action, the next state s′ reached, and the assumed next state. The action consists of a′.In the current states and action a, after the state transitions to another state s′, the current action cost function Qs,a must be updated. Then, after reaching the next state, update the next action cost function until the end. This cost is updated as follows:
(20)Qs,a⟵Qs,a+αr+γQs′,s′−Qs,a,where α the learning is the rate and γ is the decay factor.


## 2.2.4.Q-Learning

Q-learning is a temporal difference algorithm under the off-track strategy. The off-track strategy means that the strategy for determining the current behavior is different from the strategy for updating the value function. The agent chooses the action in the current state through a strategy and interacts with the environment, but then, when the value function is updated, it uses another strategy. The action-value function update formula for Q-learning is as follows:
(21)Qst,at⟵Qst,at+αrt+1+γmaxaQst+1,a−Qst,at.


## 3. Analysis of University Students’ Ideological Political Dynamics and Communication Paths

### 3.1. Dynamic Analysis of University Students’ Ideological Politics

Facing the complex and changing social environment, to carry out the ideological political education work in universities and grasp the ideological dynamics of university students, it is necessary to analyze the current ideological political dynamics of university students. It analyzes four aspects: value orientation, learning status, consumption concept, and employment, as shown in Table1.Table 1
Dynamic analysis of university students’ ideological politics.
Ideological political trendsDetailed analysisValue orientationModern university students strongly support the leadership of the CPC Central Committee with Comrade Xi Jinping as the core, highly identify with the “Chinese dream of the great rejuvenation of the Chinese nation” and “the goal of building a modern socialist country,” and are able to understand and grasp the ideas of socialism with Chinese characteristics in the new era. The fundamental core pays close attention to the party’s latest strategies and measures.Learning statusThe learning status of contemporary university students presents a positive and good development trend, and their learning initiative is also significantly enhanced. Most of the students have a correct and serious learning attitude, have clear learning goals and plans, and actively expand their knowledge reserves through online courses, lectures, and other methods. When many students encounter difficulties in learning, they will take the initiative to consult relevant materials to solve the problem.Consumer attitudesWith the rapid development of my country’s economy, people’s living standards have gradually improved, and the living consumption level of modern college students has also gradually improved. Modern college students already have strong independence and show a strong sense of self in daily consumption. Most of the students do not have the habit of keeping consumption records, and their daily consumption is mainly for meals, shopping for clothes, and class reunions.EmploymentOne of the focuses of university students in the new era is employment. Most students have a clear sense of employment and are optimistic about the future. Most of the students showed a positive attitude towards future employment.


### 3.2. Propagation Path Analysis

#### 3.2.1. Original Propagation Path

The original communication paths of college students’ ideological and political dynamics are divided into three categories: interpersonal communication, organizational communication, and mass communication, as shown in Table2.Table 2
Original propagation path.
Propagation pathContentAdvantages and scope of applicationHuman-to-human communicationInterpersonal communication, that is, the communicator communicates the ideological and political morality requirements of a certain society to the educational object face to face.The face-to-face communication between the communicator and the educational object can more accurately understand the current situation of the educational object’s ideological and political morality and its formation and development law, so as to effectively carry out the ideological political education dissemination activities.Organizational communicationOrganizational communication means that a certain organization spreads the ideological and political moral requirements of a certain society to members of the organization by holding various meetings with the members of the organization.The spread of influence is wider than human-to-human transmission and easier to control.Mass mediaMass communication is the ideological and political education and dissemination carried out by a communication organization to the whole society by means of mass media, such as newspapers, radio, and television.Spread the widest and timely and effective. For example, television communication also has the advantages of image and intuition.


#### 3.2.2. New Propagation Paths

Although the original communication path of college students’ ideological and political dynamics has its own advantages, its influence on interpersonal communication is not extensive, and it is limited by time and place. At the same time, it is also restricted to a large extent by the quality of the communicator. The scope of organizational communication is still limited to local areas, and it is difficult to solve the problem of timely and effective communication. Mass communication is only one-way communication, not interactive communication [24]. Therefore, in the process of ideological and political dynamic dissemination, while adopting and improving the original dissemination path, a new ideological and political dynamic dissemination path should also be opened up.
(1)
Network communication is based on the computer communication network to transmit, exchange, and utilize information, so as to achieve the purpose of social and cultural exchange. On the Internet, people can freely browse almost all the information on the Internet [25](2)
Opening up the Internet is a new way for college students to exchange ideological and political dynamics. It is not simply to publish some information on the Internet for ideological and political dynamic exchanges. The key is to use the various advantages of the Internet and computers to realize the dynamic exchange of ideological and political dynamics from postevent to preevent through the ideological and political dynamic database and scientifically use this series of databases in practice, from qualitative communication to quantitative communication and edge propagation to multidirectional propagation


### 3.3. Model Construction

This paper builds an analysis model of college students’ ideological political dynamics and propagation path based on reinforcement learning. The model first collects college students’ ideological political dynamics and then summarizes the ideological political dynamics and propagation paths through call requests. If there is no call request, the call request will continue, until there is a call request. Ideological and political dynamics and propagation paths can only be analyzed after the signal is felt until the end. Similarly, if no signal is sensed, the propagation path analysis will be repeated until a signal is sensed, as shown in Figure2.Figure 2
Model construction.


## 3.1. Dynamic Analysis of University Students’ Ideological Politics

Facing the complex and changing social environment, to carry out the ideological political education work in universities and grasp the ideological dynamics of university students, it is necessary to analyze the current ideological political dynamics of university students. It analyzes four aspects: value orientation, learning status, consumption concept, and employment, as shown in Table1.Table 1
Dynamic analysis of university students’ ideological politics.
Ideological political trendsDetailed analysisValue orientationModern university students strongly support the leadership of the CPC Central Committee with Comrade Xi Jinping as the core, highly identify with the “Chinese dream of the great rejuvenation of the Chinese nation” and “the goal of building a modern socialist country,” and are able to understand and grasp the ideas of socialism with Chinese characteristics in the new era. The fundamental core pays close attention to the party’s latest strategies and measures.Learning statusThe learning status of contemporary university students presents a positive and good development trend, and their learning initiative is also significantly enhanced. Most of the students have a correct and serious learning attitude, have clear learning goals and plans, and actively expand their knowledge reserves through online courses, lectures, and other methods. When many students encounter difficulties in learning, they will take the initiative to consult relevant materials to solve the problem.Consumer attitudesWith the rapid development of my country’s economy, people’s living standards have gradually improved, and the living consumption level of modern college students has also gradually improved. Modern college students already have strong independence and show a strong sense of self in daily consumption. Most of the students do not have the habit of keeping consumption records, and their daily consumption is mainly for meals, shopping for clothes, and class reunions.EmploymentOne of the focuses of university students in the new era is employment. Most students have a clear sense of employment and are optimistic about the future. Most of the students showed a positive attitude towards future employment.


## 3.2. Propagation Path Analysis

### 3.2.1. Original Propagation Path

The original communication paths of college students’ ideological and political dynamics are divided into three categories: interpersonal communication, organizational communication, and mass communication, as shown in Table2.Table 2
Original propagation path.
Propagation pathContentAdvantages and scope of applicationHuman-to-human communicationInterpersonal communication, that is, the communicator communicates the ideological and political morality requirements of a certain society to the educational object face to face.The face-to-face communication between the communicator and the educational object can more accurately understand the current situation of the educational object’s ideological and political morality and its formation and development law, so as to effectively carry out the ideological political education dissemination activities.Organizational communicationOrganizational communication means that a certain organization spreads the ideological and political moral requirements of a certain society to members of the organization by holding various meetings with the members of the organization.The spread of influence is wider than human-to-human transmission and easier to control.Mass mediaMass communication is the ideological and political education and dissemination carried out by a communication organization to the whole society by means of mass media, such as newspapers, radio, and television.Spread the widest and timely and effective. For example, television communication also has the advantages of image and intuition.


### 3.2.2. New Propagation Paths

Although the original communication path of college students’ ideological and political dynamics has its own advantages, its influence on interpersonal communication is not extensive, and it is limited by time and place. At the same time, it is also restricted to a large extent by the quality of the communicator. The scope of organizational communication is still limited to local areas, and it is difficult to solve the problem of timely and effective communication. Mass communication is only one-way communication, not interactive communication [24]. Therefore, in the process of ideological and political dynamic dissemination, while adopting and improving the original dissemination path, a new ideological and political dynamic dissemination path should also be opened up.
(1)
Network communication is based on the computer communication network to transmit, exchange, and utilize information, so as to achieve the purpose of social and cultural exchange. On the Internet, people can freely browse almost all the information on the Internet [25](2)
Opening up the Internet is a new way for college students to exchange ideological and political dynamics. It is not simply to publish some information on the Internet for ideological and political dynamic exchanges. The key is to use the various advantages of the Internet and computers to realize the dynamic exchange of ideological and political dynamics from postevent to preevent through the ideological and political dynamic database and scientifically use this series of databases in practice, from qualitative communication to quantitative communication and edge propagation to multidirectional propagation


## 3.2.1. Original Propagation Path

The original communication paths of college students’ ideological and political dynamics are divided into three categories: interpersonal communication, organizational communication, and mass communication, as shown in Table2.Table 2
Original propagation path.
Propagation pathContentAdvantages and scope of applicationHuman-to-human communicationInterpersonal communication, that is, the communicator communicates the ideological and political morality requirements of a certain society to the educational object face to face.The face-to-face communication between the communicator and the educational object can more accurately understand the current situation of the educational object’s ideological and political morality and its formation and development law, so as to effectively carry out the ideological political education dissemination activities.Organizational communicationOrganizational communication means that a certain organization spreads the ideological and political moral requirements of a certain society to members of the organization by holding various meetings with the members of the organization.The spread of influence is wider than human-to-human transmission and easier to control.Mass mediaMass communication is the ideological and political education and dissemination carried out by a communication organization to the whole society by means of mass media, such as newspapers, radio, and television.Spread the widest and timely and effective. For example, television communication also has the advantages of image and intuition.


## 3.2.2. New Propagation Paths

Although the original communication path of college students’ ideological and political dynamics has its own advantages, its influence on interpersonal communication is not extensive, and it is limited by time and place. At the same time, it is also restricted to a large extent by the quality of the communicator. The scope of organizational communication is still limited to local areas, and it is difficult to solve the problem of timely and effective communication. Mass communication is only one-way communication, not interactive communication [24]. Therefore, in the process of ideological and political dynamic dissemination, while adopting and improving the original dissemination path, a new ideological and political dynamic dissemination path should also be opened up.
(1)
Network communication is based on the computer communication network to transmit, exchange, and utilize information, so as to achieve the purpose of social and cultural exchange. On the Internet, people can freely browse almost all the information on the Internet [25](2)
Opening up the Internet is a new way for college students to exchange ideological and political dynamics. It is not simply to publish some information on the Internet for ideological and political dynamic exchanges. The key is to use the various advantages of the Internet and computers to realize the dynamic exchange of ideological and political dynamics from postevent to preevent through the ideological and political dynamic database and scientifically use this series of databases in practice, from qualitative communication to quantitative communication and edge propagation to multidirectional propagation


## 3.3. Model Construction

This paper builds an analysis model of college students’ ideological political dynamics and propagation path based on reinforcement learning. The model first collects college students’ ideological political dynamics and then summarizes the ideological political dynamics and propagation paths through call requests. If there is no call request, the call request will continue, until there is a call request. Ideological and political dynamics and propagation paths can only be analyzed after the signal is felt until the end. Similarly, if no signal is sensed, the propagation path analysis will be repeated until a signal is sensed, as shown in Figure2.Figure 2
Model construction.


## 4. Experimental Analysis

### 4.1. Model Testing

Based on reinforcement learning, this paper constructs an analysis model of university students’ ideological political dynamics and communication paths. The model needs to be tested first. 100 college students were randomly selected as experimental subjects, and 10 groups were divided into 10 groups. Reinforcement learning is compared to deep learning, machine learning, structural equation modeling, and traditional methods. In the model test comparison, this paper uses the most common accuracy rate, precision rate, and recall rate as the comparison indicators. The experimental result data are shown in Tables3–5.Table 3
Precision comparison.
Model12345678910Reinforcement learning98.7%99.4%97.6%96.5%98.3%99.1%96.2%99.7%97.8%98.3%Deep learning94.1%89.6%90.4%92.4%90.6%88.7%86.4%89.2%85.3%92.6%Machine learning85.4%86.2%89.3%90.1%83.6%80.5%79.3%83.4%78.4%80.4%Structural equation79.5%76.1%80.3%77.2%81.3%70.4%72.6%73.4%76.1%74.9%Traditional method69.4%65.2%70.4%67.8%69.2%62.1%63.8%65.9%64.3%69.1%Table 4
Accuracy comparison.
Model12345678910Reinforcement learning99.7%98.9%97.4%98.7%98.3%99.5%98.5%98.6%98.3%99.6%Deep learning92.4%87.4%90.4%90.8%91.4%89.8%87.3%87.3%88.2%90.5%Machine learning86.4%83.7%87.6%87.6%81.8%80.3%78.7%81.8%79.1%83.9%Structural equation76.5%74.8%81.3%71.4%77.4%71.8%70.8%73.7%75.4%75.1%Traditional method68.4%66.4%68.9%67.3%68.3%61.9%68.9%63.9%63.8%68.9%Table 5
Comparison of recall rates.
Model12345678910Reinforcement learning97.6%99.4%98.6%99.5%97.9%98.4%99.6%97.8%98.8%98.9%Deep learning90.2%89.2%91.2%93.9%87.6%87.2%88.3%90.7%86.2%91.3%Machine learning85.8%80.7%83.2%83.8%80.1%81.8%79.8%87.3%76.8%81.5%Structural equation79.1%76.3%80.8%74.6%78.6%79.1%73.7%76.1%73.2%78.4%Traditional method66.7%68.7%69.5%69.3%64.3%60.3%66.3%67.4%61.4%65.9%It can be seen from the data results that reinforcement learning is higher than other models in the comparison of accuracy, precision, and recall, with obvious advantages, indicating that reinforcement learning is more suitable for this study.The precision of reinforcement learning is 99.7% and 96.2%, which is 37.6% higher than other methods. The highest accuracy was 99.7%, and the lowest was 97.4%. Compared with other methods, it is 37.8% higher than the lowest accuracy. The highest recall rate is 99.6%, and the lowest is 97.6%. Compared with other methods, it is 39.3% higher than the lowest recall. In order to see the advantages of this model more intuitively, it is shown in Figures3–5.Figure 3
Precision comparison.Figure 4
Accuracy comparison.Figure 5
Comparison of recall rates.Through the comprehensive comparison of the accuracy, precision, and recall of the five methods, the average value of each index of the five methods indicates that the reinforcement learning method has more obvious advantages, as shown in Figure6.Figure 6
Comprehensive comparison.From Figure6, we know that reinforcement learning has the highest average accuracy, precision, and recall, with an average precision of 98.16%, an average precision of 98.75%, and an average recall of 98.65%. Therefore, this model is most suitable for the research analysis of this article.


### 4.2. Dynamic Analysis of College Students’ Ideology and Politics

After passing the test, the model will be applied to the research of this paper. First, analyze the ideological and political dynamics of college students. Randomly selected 100 college students were divided into four groups: freshmen, sophomores, juniors, and seniors. It analyzes four aspects: value orientation, learning status, consumption concept, and employment. Through the questionnaire survey, students scored four aspects according to their own situation, with a total score of 10 points. The result is shown in Figure7.Figure 7
Dynamic analysis of college students’ ideology and politics.According to Figure6, the value orientation score in the ideological political dynamics of university students is 6.975, the learning status is 8.025, the consumption concept is 7.7, and the employment aspect is 7.45. Among them, freshman students are more concerned about the state of study, while senior students are most concerned about employment issues and have the highest score among all the scoring results, reaching 10 points.


### 4.3. Propagation Path Analysis

This article lists five communication paths, interpersonal communication, organizational communication, mass communication, network communication, and the Internet. In order to more accurately analyze the ideological political dynamic communication paths of university students, this experiment made statistics on the ideological political dynamic propagation paths of 100 university students. The results are shown in Figure8.Figure 8
Propagation path analysis.The experimental results showed that 12 people communicated through people, 15 people communicated through organizations, 21 people communicated through mass communication, 28 people communicated through the Internet, and 24 people communicated through the Internet. It shows that the communication path of college students’ ideological dynamics is mainly based on network communication, and the number of first-year students through interpersonal communication and senior students through organizational communication is only 2.


## 4.1. Model Testing

Based on reinforcement learning, this paper constructs an analysis model of university students’ ideological political dynamics and communication paths. The model needs to be tested first. 100 college students were randomly selected as experimental subjects, and 10 groups were divided into 10 groups. Reinforcement learning is compared to deep learning, machine learning, structural equation modeling, and traditional methods. In the model test comparison, this paper uses the most common accuracy rate, precision rate, and recall rate as the comparison indicators. The experimental result data are shown in Tables3–5.Table 3
Precision comparison.
Model12345678910Reinforcement learning98.7%99.4%97.6%96.5%98.3%99.1%96.2%99.7%97.8%98.3%Deep learning94.1%89.6%90.4%92.4%90.6%88.7%86.4%89.2%85.3%92.6%Machine learning85.4%86.2%89.3%90.1%83.6%80.5%79.3%83.4%78.4%80.4%Structural equation79.5%76.1%80.3%77.2%81.3%70.4%72.6%73.4%76.1%74.9%Traditional method69.4%65.2%70.4%67.8%69.2%62.1%63.8%65.9%64.3%69.1%Table 4
Accuracy comparison.
Model12345678910Reinforcement learning99.7%98.9%97.4%98.7%98.3%99.5%98.5%98.6%98.3%99.6%Deep learning92.4%87.4%90.4%90.8%91.4%89.8%87.3%87.3%88.2%90.5%Machine learning86.4%83.7%87.6%87.6%81.8%80.3%78.7%81.8%79.1%83.9%Structural equation76.5%74.8%81.3%71.4%77.4%71.8%70.8%73.7%75.4%75.1%Traditional method68.4%66.4%68.9%67.3%68.3%61.9%68.9%63.9%63.8%68.9%Table 5
Comparison of recall rates.
Model12345678910Reinforcement learning97.6%99.4%98.6%99.5%97.9%98.4%99.6%97.8%98.8%98.9%Deep learning90.2%89.2%91.2%93.9%87.6%87.2%88.3%90.7%86.2%91.3%Machine learning85.8%80.7%83.2%83.8%80.1%81.8%79.8%87.3%76.8%81.5%Structural equation79.1%76.3%80.8%74.6%78.6%79.1%73.7%76.1%73.2%78.4%Traditional method66.7%68.7%69.5%69.3%64.3%60.3%66.3%67.4%61.4%65.9%It can be seen from the data results that reinforcement learning is higher than other models in the comparison of accuracy, precision, and recall, with obvious advantages, indicating that reinforcement learning is more suitable for this study.The precision of reinforcement learning is 99.7% and 96.2%, which is 37.6% higher than other methods. The highest accuracy was 99.7%, and the lowest was 97.4%. Compared with other methods, it is 37.8% higher than the lowest accuracy. The highest recall rate is 99.6%, and the lowest is 97.6%. Compared with other methods, it is 39.3% higher than the lowest recall. In order to see the advantages of this model more intuitively, it is shown in Figures3–5.Figure 3
Precision comparison.Figure 4
Accuracy comparison.Figure 5
Comparison of recall rates.Through the comprehensive comparison of the accuracy, precision, and recall of the five methods, the average value of each index of the five methods indicates that the reinforcement learning method has more obvious advantages, as shown in Figure6.Figure 6
Comprehensive comparison.From Figure6, we know that reinforcement learning has the highest average accuracy, precision, and recall, with an average precision of 98.16%, an average precision of 98.75%, and an average recall of 98.65%. Therefore, this model is most suitable for the research analysis of this article.


## 4.2. Dynamic Analysis of College Students’ Ideology and Politics

After passing the test, the model will be applied to the research of this paper. First, analyze the ideological and political dynamics of college students. Randomly selected 100 college students were divided into four groups: freshmen, sophomores, juniors, and seniors. It analyzes four aspects: value orientation, learning status, consumption concept, and employment. Through the questionnaire survey, students scored four aspects according to their own situation, with a total score of 10 points. The result is shown in Figure7.Figure 7
Dynamic analysis of college students’ ideology and politics.According to Figure6, the value orientation score in the ideological political dynamics of university students is 6.975, the learning status is 8.025, the consumption concept is 7.7, and the employment aspect is 7.45. Among them, freshman students are more concerned about the state of study, while senior students are most concerned about employment issues and have the highest score among all the scoring results, reaching 10 points.


## 4.3. Propagation Path Analysis

This article lists five communication paths, interpersonal communication, organizational communication, mass communication, network communication, and the Internet. In order to more accurately analyze the ideological political dynamic communication paths of university students, this experiment made statistics on the ideological political dynamic propagation paths of 100 university students. The results are shown in Figure8.Figure 8
Propagation path analysis.The experimental results showed that 12 people communicated through people, 15 people communicated through organizations, 21 people communicated through mass communication, 28 people communicated through the Internet, and 24 people communicated through the Internet. It shows that the communication path of college students’ ideological dynamics is mainly based on network communication, and the number of first-year students through interpersonal communication and senior students through organizational communication is only 2.


## 5. Conclusion

The ideological political trend of university students is related to the future and destiny of the country and the nation, and the communication path is also very important. Based on reinforcement learning, this paper constructs an analysis model of university students’ ideological political dynamics and communication paths and improves the accuracy, precision, and recall rate on the basis of traditional methods, which is helpful to analyze the ideological and political dynamics and communication paths of college students.The findings of this article show that(1)
by comparing reinforcement learning with deep learning, machine learning, structural equations, and traditional methods, the accuracy of reinforcement learning is 99.7% and 96.2%, respectively, which is 37.6% higher than other methods. The highest accuracy was 99.7%, and the lowest was 97.4%. Compared with other methods, it is 37.8% higher than the lowest accuracy. The highest recall rate is 99.6%, and the lowest is 97.6%. Compared with other methods, it is 39.3% higher than the lowest recall(2)
reinforcement learning has the highest average accuracy, precision, and recall, with an average accuracy of 98.16%, an average precision of 98.75%, and an average recall of 98.65%(3)
freshman students pay more attention to the state of study, while senior students are most concerned about employment issues, and they have the highest score among all scoring results, reaching 10 points(4)
the communication path of college students’ ideological dynamics is mainly based on network communication. The number of first-year students through interpersonal communication and the number of senior students through organizational communication is the least, only 2 people, respectivelyBased on the analysis of the experimental results, it is concluded that in order to guide the positive development of the ideological political dynamics of university students, (1) increase ideological and political education, (2) improve curriculum and mental health monitoring mechanism, (3) improve school employment guidance, (4) strengthen the management of online public opinion, and (5) strengthen home-school cooperation. Although the model constructed in this article has obvious advantages in terms of accuracy, precision, and recall, it still has certain limitations. This model is limited to the research on the ideological political dynamics of university students. Further research on the generality of the model is needed in the future to increase the generality of the model and enable the model to be applied to a wider range of studies.



---

*Source: 9704315-2022-09-12.xml*