# Intelligent System Application in Clinical Management of Medical Teaching Based on Deep Reinforcement Learning

Authors: Min Zhu; Ju Zhou; Liang Chen; Xueping Zhao; Chunhui Li
Journal: Mobile Information Systems
 (2022)

---

## Abstract

When dealing with engineering projects, there are many kinds of planning schemes. There are many problems to be solved in the performance and economic conditions of these indicators and engineering contents we study. Recently, many researchers have made great achievements in optimizing various subject projects. One of its purposes is to optimize and improve intelligent multi-objectives to make it more effective. Therefore, it is of great significance to develop intelligent multi-objective projects in academia and more engineering fields. The purpose of this study is to make efforts to strengthen the learning and construction of rapidly developing multi-objective optimization programs, and closely link these programs with neural network, fuzzy technology, interactive technology, and give a lot of examples of multi-objective optimization improvement program methods, which can eliminate the uncertainty in the multi-objective project plan, The purpose is to solve some difficult problems of multi-objective optimization and give the best suggestions. In this study, the strengthening of learning construction is combined with the clinical experiment, and most of this teaching method is put in the clinical management experiment of medical teaching. At the end of this study, we compared the performance of deep reinforcement learning with its dual structure version and interleaved structure version in different medical teaching clinical management environments and make a detailed analysis. Deep reinforcement learning provides a new idea for the improvement of intelligent multi-objective optimization and clinical management of medical teaching.


---

## Body

## 1. Introduction

There are three model learning methods in machine learning: one is enhanced learning, the other is supervised learning, and the other is unsupervised learning. Enhanced learning and supervised learning are not the same as unsupervised learning [1]. The enhancement of learning construction is not a professional guidance label with intelligence like supervised learning, but is a stage in which learning and absorbing knowledge and nutrition to expand its own advantages in the environment [2]. Enhancing learning construction is a stage of constantly enhancing its own experience. Unsupervised learning construction is to find hidden structures in data without label. In this respect, it is different from enhancing learning construction [3]. The purpose of enhancing learning construction is to enlarge the reward. If a hidden structure of environment is found, it can not maximize reward, because unsupervised learning structure is not the same as enhancing learning construction. One of the important fields in machine learning is deep learning. In recent years, it has developed rapidly, and has been favored by many scholars at home and abroad [4]. With the rapid development of the computer age, deep learning has also been widely used in many realities. In recent years, researchers have put the reinforcement of learning construction and deep learning together. Deep enhancement learning combines the two closely related advantages, and has no shortcomings [5]. For example, it is good at analyzing high-intensity data and the best way to contact the environment.In this study, we use the advantage of deep enhancement learning to apply it to intelligent multi-objective optimization to gradually improve the medical teaching and clinical management experiment. In practice, in this paper, most neural network structures like structure types are designed [6]. The input mode is model parameters, and the output method is the value of the Pareto solution of model parameters. After several rounds of mutual absorption, the final result value Pareto solution is obtained [7]. Then, the data model that designers like is used as the optimization goal of the ANN model. It is better to find a new plan to improve the target and take the intelligent optimization objective as the improvement development guidance scheme [8]. The deep enhancement learning scheme is used to study the teaching improvement of medical clinical experiment in detail, which can enhance the overall ability and teaching level of the doctor team [9]. The knowledge system learned should be improved, the learned knowledge should be consolidated, the interest and hobby of learning should be cultivated, and the students’ practical ability should be strengthened to adapt to the educational learning mode in modern medical field in line with the development of globalization [10].


## 2. Related Work

The important viewpoints about reinforcement learning: we study two aspects of bias and variance analysis, and compare the results of reinforcement learning structure stability. The bias variances are positive bias, negative bias, delusional bias, random variance, and importance sampling variance. These are derived from the examples of the formula and the calculation of the equation, which can directly get the specific causes of the deviation and variance and the impact on the stability, and provide effective evidence for future solutions [11]. There are positive and negative deviations in the deviation, and the method adopted in this study intuitively and clearly states the factors and influences to solve the deviation. A novel access method is interleaving method, which can solve the problem of positive and negative deviation quickly [7]. There are three kinds of interleaving access methods: coupling estimator, adaptive coupling rate device, and interleaving access method. These three methods together constitute the interleaved access method. The formula in this paper can prove the method of interleaving access. Literature interleaving methods can be applied to many fields, such as experiments in reinforcement learning and deep reinforcement learning [12]. In this study, we can see a new computing method. Finally, the experimental results show that the performance of the new algorithm is the best. The design scheme of intelligent multi-objective project interaction is proposed in the literature, which makes the optimization of many objectives solved and improved [13]. In the scheme of model optimization, the designer needs to provide the value of parameter vector. In real life, designers can design their own favorite model, and take the input as the parameter, and the Pareto solution as the output. These conclusions are obtained by several rounds of training. The double beam crane is proposed as the object of multi-objective project optimization [14]. Their weight and safety are taken as the optimization index. Designers add these two indicators to the trade-off of goals, and they can get more satisfactory answers within the allowed time [15].


## 3. Intelligent Multi-Objective Optimization Improvement of Deep Reinforcement Learning

### 3.1. Deep Reinforcement Learning

First of all, we need to understand the background knowledge and related symbols of enhancing learning stability, such as the calculation method and definition of Markov, and then we can judge the reasons for enhancing learning stability. Markov decision process can enhance learning. Markov decision-making process is based on mathematical model, which can simulate the decision-making in the intelligent environment of the system. It has nothing to do with any state previously arrived and any action performed. The strategy is the action probability distribution of performing a certain action in a certain state, and the expression formula is:(1)πs|a=PAt=a|St=s.Expectation function of Markov decision process:(2)Vπs=E∑t=0∞γtRt+1|St=s.The correlation function equation obtained by the Markov decision process under the condition of following the strategy:(3)Qπs,a=E∑t=0∞γtRt+1|St=s,At=a.The true value of the highest expected value inX is:(4)μ∗X=maxi∫−∞+∞fixdx.Bias is often different from real data. Based on the above, we estimate a definition of bias:(5)Biasμ^=Eμ^−μ∗.Use the largest estimated value to estimate the value of the random variable, the expression is:(6)μ^∗MES=maxμ^iSi.Knowing that the maximization function is a convex function, for any set of samples and any unbiased estimator, we can get:(7)Eμ^∗MES=Emaxμ^iSi.Randomly select a sample, and then draw a random number of 100 as the mean. The statistical table is shown in Table1:Table 1
The maximum estimator’s sampling estimation table for four random variables.
Variable nameX1X2X3X4Maximum estimator valueSample 1 mean1.541.902.122.152.15Sample 2 mean1.432.042.151.672.16Sample 3 mean0.761.732.161.362.14Sample 4 mean0.981.282.222.742.72Sample 5 mean1.262.332.171.932.33Sample 6 mean1.042.152.292.452.44Sample 7 mean1.071.572.152.612.66Sample 8 mean1.412.252.182.752.70Sample 9 mean1.182.012.231.842.27Sample 10 mean1.022.242.222.222.27Final sample average1.161.932.242.032.38The upper bound of the estimated variance of the dual estimator and the lower bound of the estimated negative deviation are both proved in this study. The negative deviation is always less than 0, so its upper bound is 0, and its lower bound expression:(8)Biasμ^∗DES=Eμ^∗DES−μ∗.The upper bound expression of the estimated variance of the double estimator:(9)j=argmaxμ^jS1,μ^∗DES=μ^jS2i.The double estimator finds the subscript of the maximum sample mean variable in the estimator 1. This is because the samples used by the estimators 1 and 2 are independent of each other. The subscript of the maximum value in the estimator 1 is equivalent to a random Variables, as shown in Table2.Table 2
Estimator 1 sampling estimation table for four random variables.
Variable nameX1X2X3X4Estimator 1 takes the subscriptSample 1 mean0.842.032.541.283Sample 2 mean1.702.772.272.252Sample 3 mean1.142.142.152.452Sample 4 mean1.331.062.261.953Sample 5 mean1 251.862.172.233Sample 6 mean0.991.972.171 283Sample 7 mean1.271.862.202.233Sample 8 mean1 201.572.231.523Sample 9 mean1.781.492.211.564Sample 10 mean0.44.2.442.152.262Final sample average1.191.932.181.97—As shown in Table3, the largest subscript in estimator 1 may be the largest or the smallest if it is checked in estimator 2. Therefore, if the sampling mean selected by estimator 2 is divided into two categories, one is the sampling mean with the true value variable selected, and the other is the sampling mean without the true value variable selected, then the expectation of the first category is the maximum expectation. The second kind of expectation must be less than the maximum expectation, so the combination of the two must be less than the maximum expectation, and the double estimator will cause underestimation.Table 3
Estimator 2 sampling estimation table for four random variables.
Variable nameX1X2X3X4Estimator 2 takes the valueSample 1 mean1.122.242.232.542.24Sample 2 mean0.532.352.251.812.35Sample 3 mean0.252.432.252.062.05Sample 4 mean0.841.702.211 952.21Sample 5 mean1.162.252.151.771.76Sample 6 mean1.152.162.222.112.22Sample 7 mean0.642.042.231.831.87Sample 8 mean0.842.042.190.992.19Sample 9 mean1.061.692.161.972.18Sample 10 mean1.661.732.271.961.72Final sample average0.972.042.211.842.08The learning methods of reinforcement learning can be roughly divided into two categories, one is the Monte Carlo method, and the other is the temporal difference method. The calculation formula of the expected value of all state action values:(10)Eμ^∗DES=Eμ^jS2,EQs′,a′=∑a′∈Aπs′|a′Qs′,a′.This article introduces the n-step timing difference algorithm. After performing then action steps, it is updated according to the return value obtained during this period and the estimated value of the previous round of value function. The n-step return formula is expressed as:(11)Gt:t+n=Rt+1+γRt+2+⋯+γn−1Rt+n+γnRt+n−1St+n.The return is a special case of the conforming return. It weights all possible n-step updates in proportion, and finally multiplies it by the regular term to guarantee the weight and the sum is 1. Its formula is expressed as:(12)Gtλ=1−λ∑n=1∞λn−1Gt:t+n.The calculation items after these terminal states can be separated from the main summation items:(13)Gtλ=1−λ∑n=1T−t−1λn−1Gt:t+n+λT−t−1Gt.On the basis of Q-learning, through the calculation of convolution, multi-dimensional data can be processed well, and features can be extracted from high-latitude information such as pictures, which makes it possible to use continuous pictures as output. The calculation used by the Q′ network to predict the target:(14)YtDQN=Rt+γmaxa′Q′St+1,a′;θt′.The calculation of the update target of deep Q-learning is the same as that of Q-learning, which uses the maximum Q value in the next state as the target. The difference is that in deep Q-learning, the fixed past nerve is used to estimate the maximum Q value. The algorithm process diagram of deep Q-learning is shown in Figure1:Figure 1
Algorithm structure diagram of deep Q-learning.Deep double Q-learning can effectively reduce the overestimation of deep Q-learning through such a simple small change. Its modification formula:(15)YtdoubleDQN=Rt+γQ′St+1+argmaxa′QSt+1,a′;θt′,θt′.The algorithm structure diagram of deep double Q-learning is shown in Figure2:Figure 2
Structure diagram of deep double Q-learning algorithm.Hasselt described the depth of this version in the study. In the deep double Q-learning paper experiment, we can see that the deep double Q-learning under this version does reduce the overestimation compared to the deep Q-learning. However, due to the high correlation between the two Q-value networks, deep double Q-learning still has a certain overestimation, as shown in Figure3:Figure 3
Graph of experimental results in Hasselt's study.
(a)(b)(c)(d)Figure3 is from Hasselt’s deep double Q-learning paper. It can be seen that in the performance of the four games given, deep double Q-learning greatly reduces the overestimation of deep Q-learning, but deep double Q -Studying in Space Invaders and Zaxxon games still has high overestimations, satisfying the principle of independent cross-estimation in the double Q-learning thought. The updated objective function is the formula:(16)YtdoubleDQN=Rt+γQ′St+1+argmaxa′QSt+1,a′;θ1′,θ1′.The specific update target is calculated as:(17)YtdoubleDQN=Rt+γQ′St+1+argmaxa′QSt+1,a′;θ1′,θ2′.The classical reinforcement learning algorithm Sarsa and Q-learning algorithm both use theϵ-greedy algorithm in the action strategy, but in the update strategy, the Q-learning uses the greedy algorithm and the Sarsa uses the ϵ-greedy algorithm, the expression is:(18)YtinterleaveDQN=Rt+12γQ1′St+1+argmaxa′Q2′St+1,a′;θ2′,θ2′.


### 3.2. Intelligent Multi-Objective Optimization and Improvement

In the case of solving multiple target projects, you can find a variable to optimize all project goals. Requirements for each small goal:(19)minimizefx=f1x,f2x,…,fmxSubjectx∈X.The general model of goal planning can be expressed as:(20)min∑i=1LPi∑k∈Jiwk+dk+_+wk−dk−.The linear problem of the weight coefficient:(21)min∑i=1mλifix.Commonly used functions to describe deviation are:P modulus function:(22)hfx=∑k=1mλk|fkx−fk¯|p1/p.Maximum deviation function:(23)hfx=max1≤k≤mλkfkx−fk¯|.Geometric mean function:(24)hfx=∏k=1mfkx−fk¯1/m.Because the extremely large deviation function cannot be calculated, the calculation is very difficult. Generally, it can be calculated with the same value as its properties:(25)mints.tλkfkx−fk¯=tk=1,…,m.The interval value range of each objective function is shown in Table4.Table 4
The interval boundary and parameterσ of each design objective preference function.
Design goalsPreference function typeGi5Gi4Gi3Gi2Gi1Natural frequency (Hz)2-S100110120150200+∞0.460Expenses (￥/m3)1-S160001560015200144008000−∞22.23Width (m)2-S0.340.450.430.450.54+∞0.0027Length(m)2-S3.03.33.54.06.1−∞0.017Mass (kg)1-S29002700300025002700−∞5.570Height (m)1-S0.600.550.600.450.54+∞0.0028Not feasibleFeasibleThickness of the first layer (m)2-H————0.01+∞—Thickness of the second layer (m)2-H————0.01+∞—Thickness of the third layer (m)2-H————0.01+∞—In the case of specific problems in the design of project goals in reality, every designer wants to add his own love and dislike to achieve the design criteria for each project under the premise of getting the latest version of the scheme. Improvements and optimizations will not be until the final stage of the project to design a pleasing design plan. Therefore, interlaced design methods have received widespread attention and application in today’s society. And because every designer’s favorites and preferences are not the same, and each structure has its own outline structure, the situation when meeting the optimization and improvement target schemes is also different from each other, including large-scale. The problem of how to optimize the optimization of multi-objective projects such as small and medium-sized projects can create a satisfactory, mature, effective, intelligent, and convenient method of optimizing project objectives that people can quickly accept. Pareto’s curve function icon shows that the height of the surface is not a linear function, and the surface of the surface is not smooth enough. One possibility is that there is no continuity. The characteristic of the well-known neural network system is that there is no linear model function and there may be discontinuous curved surfaces. We put the Pareto surface to be designed in the range that we can see and show it to the designers in the form of graphs or charts. The purpose is to let the designer grasp the optimized scale in a certain situation, so as to improve the design efficiency more effectively and quickly, and obtain the improvement results faster after the design. The shape of Pareto is shown to us in the form of a histogram, and its form of existence is an environmental space with a high degree of satisfaction, as shown in Figure4.Figure 4
Visualization of pareto solution.In dealing with the problem of neural network system, we use the interlaced processing methods to solve the complex problems existing in the calculation method and find the most suitable solution under the effective situation, so as to make the problems we encounter more practical, more common and more effective.


## 3.1. Deep Reinforcement Learning

First of all, we need to understand the background knowledge and related symbols of enhancing learning stability, such as the calculation method and definition of Markov, and then we can judge the reasons for enhancing learning stability. Markov decision process can enhance learning. Markov decision-making process is based on mathematical model, which can simulate the decision-making in the intelligent environment of the system. It has nothing to do with any state previously arrived and any action performed. The strategy is the action probability distribution of performing a certain action in a certain state, and the expression formula is:(1)πs|a=PAt=a|St=s.Expectation function of Markov decision process:(2)Vπs=E∑t=0∞γtRt+1|St=s.The correlation function equation obtained by the Markov decision process under the condition of following the strategy:(3)Qπs,a=E∑t=0∞γtRt+1|St=s,At=a.The true value of the highest expected value inX is:(4)μ∗X=maxi∫−∞+∞fixdx.Bias is often different from real data. Based on the above, we estimate a definition of bias:(5)Biasμ^=Eμ^−μ∗.Use the largest estimated value to estimate the value of the random variable, the expression is:(6)μ^∗MES=maxμ^iSi.Knowing that the maximization function is a convex function, for any set of samples and any unbiased estimator, we can get:(7)Eμ^∗MES=Emaxμ^iSi.Randomly select a sample, and then draw a random number of 100 as the mean. The statistical table is shown in Table1:Table 1
The maximum estimator’s sampling estimation table for four random variables.
Variable nameX1X2X3X4Maximum estimator valueSample 1 mean1.541.902.122.152.15Sample 2 mean1.432.042.151.672.16Sample 3 mean0.761.732.161.362.14Sample 4 mean0.981.282.222.742.72Sample 5 mean1.262.332.171.932.33Sample 6 mean1.042.152.292.452.44Sample 7 mean1.071.572.152.612.66Sample 8 mean1.412.252.182.752.70Sample 9 mean1.182.012.231.842.27Sample 10 mean1.022.242.222.222.27Final sample average1.161.932.242.032.38The upper bound of the estimated variance of the dual estimator and the lower bound of the estimated negative deviation are both proved in this study. The negative deviation is always less than 0, so its upper bound is 0, and its lower bound expression:(8)Biasμ^∗DES=Eμ^∗DES−μ∗.The upper bound expression of the estimated variance of the double estimator:(9)j=argmaxμ^jS1,μ^∗DES=μ^jS2i.The double estimator finds the subscript of the maximum sample mean variable in the estimator 1. This is because the samples used by the estimators 1 and 2 are independent of each other. The subscript of the maximum value in the estimator 1 is equivalent to a random Variables, as shown in Table2.Table 2
Estimator 1 sampling estimation table for four random variables.
Variable nameX1X2X3X4Estimator 1 takes the subscriptSample 1 mean0.842.032.541.283Sample 2 mean1.702.772.272.252Sample 3 mean1.142.142.152.452Sample 4 mean1.331.062.261.953Sample 5 mean1 251.862.172.233Sample 6 mean0.991.972.171 283Sample 7 mean1.271.862.202.233Sample 8 mean1 201.572.231.523Sample 9 mean1.781.492.211.564Sample 10 mean0.44.2.442.152.262Final sample average1.191.932.181.97—As shown in Table3, the largest subscript in estimator 1 may be the largest or the smallest if it is checked in estimator 2. Therefore, if the sampling mean selected by estimator 2 is divided into two categories, one is the sampling mean with the true value variable selected, and the other is the sampling mean without the true value variable selected, then the expectation of the first category is the maximum expectation. The second kind of expectation must be less than the maximum expectation, so the combination of the two must be less than the maximum expectation, and the double estimator will cause underestimation.Table 3
Estimator 2 sampling estimation table for four random variables.
Variable nameX1X2X3X4Estimator 2 takes the valueSample 1 mean1.122.242.232.542.24Sample 2 mean0.532.352.251.812.35Sample 3 mean0.252.432.252.062.05Sample 4 mean0.841.702.211 952.21Sample 5 mean1.162.252.151.771.76Sample 6 mean1.152.162.222.112.22Sample 7 mean0.642.042.231.831.87Sample 8 mean0.842.042.190.992.19Sample 9 mean1.061.692.161.972.18Sample 10 mean1.661.732.271.961.72Final sample average0.972.042.211.842.08The learning methods of reinforcement learning can be roughly divided into two categories, one is the Monte Carlo method, and the other is the temporal difference method. The calculation formula of the expected value of all state action values:(10)Eμ^∗DES=Eμ^jS2,EQs′,a′=∑a′∈Aπs′|a′Qs′,a′.This article introduces the n-step timing difference algorithm. After performing then action steps, it is updated according to the return value obtained during this period and the estimated value of the previous round of value function. The n-step return formula is expressed as:(11)Gt:t+n=Rt+1+γRt+2+⋯+γn−1Rt+n+γnRt+n−1St+n.The return is a special case of the conforming return. It weights all possible n-step updates in proportion, and finally multiplies it by the regular term to guarantee the weight and the sum is 1. Its formula is expressed as:(12)Gtλ=1−λ∑n=1∞λn−1Gt:t+n.The calculation items after these terminal states can be separated from the main summation items:(13)Gtλ=1−λ∑n=1T−t−1λn−1Gt:t+n+λT−t−1Gt.On the basis of Q-learning, through the calculation of convolution, multi-dimensional data can be processed well, and features can be extracted from high-latitude information such as pictures, which makes it possible to use continuous pictures as output. The calculation used by the Q′ network to predict the target:(14)YtDQN=Rt+γmaxa′Q′St+1,a′;θt′.The calculation of the update target of deep Q-learning is the same as that of Q-learning, which uses the maximum Q value in the next state as the target. The difference is that in deep Q-learning, the fixed past nerve is used to estimate the maximum Q value. The algorithm process diagram of deep Q-learning is shown in Figure1:Figure 1
Algorithm structure diagram of deep Q-learning.Deep double Q-learning can effectively reduce the overestimation of deep Q-learning through such a simple small change. Its modification formula:(15)YtdoubleDQN=Rt+γQ′St+1+argmaxa′QSt+1,a′;θt′,θt′.The algorithm structure diagram of deep double Q-learning is shown in Figure2:Figure 2
Structure diagram of deep double Q-learning algorithm.Hasselt described the depth of this version in the study. In the deep double Q-learning paper experiment, we can see that the deep double Q-learning under this version does reduce the overestimation compared to the deep Q-learning. However, due to the high correlation between the two Q-value networks, deep double Q-learning still has a certain overestimation, as shown in Figure3:Figure 3
Graph of experimental results in Hasselt's study.
(a)(b)(c)(d)Figure3 is from Hasselt’s deep double Q-learning paper. It can be seen that in the performance of the four games given, deep double Q-learning greatly reduces the overestimation of deep Q-learning, but deep double Q -Studying in Space Invaders and Zaxxon games still has high overestimations, satisfying the principle of independent cross-estimation in the double Q-learning thought. The updated objective function is the formula:(16)YtdoubleDQN=Rt+γQ′St+1+argmaxa′QSt+1,a′;θ1′,θ1′.The specific update target is calculated as:(17)YtdoubleDQN=Rt+γQ′St+1+argmaxa′QSt+1,a′;θ1′,θ2′.The classical reinforcement learning algorithm Sarsa and Q-learning algorithm both use theϵ-greedy algorithm in the action strategy, but in the update strategy, the Q-learning uses the greedy algorithm and the Sarsa uses the ϵ-greedy algorithm, the expression is:(18)YtinterleaveDQN=Rt+12γQ1′St+1+argmaxa′Q2′St+1,a′;θ2′,θ2′.


## 3.2. Intelligent Multi-Objective Optimization and Improvement

In the case of solving multiple target projects, you can find a variable to optimize all project goals. Requirements for each small goal:(19)minimizefx=f1x,f2x,…,fmxSubjectx∈X.The general model of goal planning can be expressed as:(20)min∑i=1LPi∑k∈Jiwk+dk+_+wk−dk−.The linear problem of the weight coefficient:(21)min∑i=1mλifix.Commonly used functions to describe deviation are:P modulus function:(22)hfx=∑k=1mλk|fkx−fk¯|p1/p.Maximum deviation function:(23)hfx=max1≤k≤mλkfkx−fk¯|.Geometric mean function:(24)hfx=∏k=1mfkx−fk¯1/m.Because the extremely large deviation function cannot be calculated, the calculation is very difficult. Generally, it can be calculated with the same value as its properties:(25)mints.tλkfkx−fk¯=tk=1,…,m.The interval value range of each objective function is shown in Table4.Table 4
The interval boundary and parameterσ of each design objective preference function.
Design goalsPreference function typeGi5Gi4Gi3Gi2Gi1Natural frequency (Hz)2-S100110120150200+∞0.460Expenses (￥/m3)1-S160001560015200144008000−∞22.23Width (m)2-S0.340.450.430.450.54+∞0.0027Length(m)2-S3.03.33.54.06.1−∞0.017Mass (kg)1-S29002700300025002700−∞5.570Height (m)1-S0.600.550.600.450.54+∞0.0028Not feasibleFeasibleThickness of the first layer (m)2-H————0.01+∞—Thickness of the second layer (m)2-H————0.01+∞—Thickness of the third layer (m)2-H————0.01+∞—In the case of specific problems in the design of project goals in reality, every designer wants to add his own love and dislike to achieve the design criteria for each project under the premise of getting the latest version of the scheme. Improvements and optimizations will not be until the final stage of the project to design a pleasing design plan. Therefore, interlaced design methods have received widespread attention and application in today’s society. And because every designer’s favorites and preferences are not the same, and each structure has its own outline structure, the situation when meeting the optimization and improvement target schemes is also different from each other, including large-scale. The problem of how to optimize the optimization of multi-objective projects such as small and medium-sized projects can create a satisfactory, mature, effective, intelligent, and convenient method of optimizing project objectives that people can quickly accept. Pareto’s curve function icon shows that the height of the surface is not a linear function, and the surface of the surface is not smooth enough. One possibility is that there is no continuity. The characteristic of the well-known neural network system is that there is no linear model function and there may be discontinuous curved surfaces. We put the Pareto surface to be designed in the range that we can see and show it to the designers in the form of graphs or charts. The purpose is to let the designer grasp the optimized scale in a certain situation, so as to improve the design efficiency more effectively and quickly, and obtain the improvement results faster after the design. The shape of Pareto is shown to us in the form of a histogram, and its form of existence is an environmental space with a high degree of satisfaction, as shown in Figure4.Figure 4
Visualization of pareto solution.In dealing with the problem of neural network system, we use the interlaced processing methods to solve the complex problems existing in the calculation method and find the most suitable solution under the effective situation, so as to make the problems we encounter more practical, more common and more effective.


## 4. Establishment and Improvement of Clinical Management of Medical Teaching

### 4.1. Current Situation of Clinical Teaching Management System

Nowadays, in the field of medicine, the teaching method of clinical medicine is the embodiment of a medical student’s basic medical literacy, and it is also one of the key points of innovation in the teaching characteristics of medical colleges. In recent years, with the continuous reform and innovation and development of higher education, the cheers of reform and education innovation in many medical colleges are increasing. In the environment of this reform, the education management center and core content of traditional Chinese medicine are the top priority and the focus of the reform. In this journey of reform and innovation, we have absorbed the development mode and operation system of higher education at home and abroad to supplement the educational problems and loopholes existing in China’s educational background, which has a far-reaching impact on the future development of China’s education, It plays an important role in the training of talents and the development of education, and also has a guiding role in the future development of young people.There are many problems in the management system of Chinese higher medical education, such as whether the laboratory teaching management is centralized or decentralized. The centralized education management system is to establish the medical department with the University as the center. The management department is composed of all the research institutes, medical colleges, departments and laboratories related to the hospital in the University. And they are responsible for all the medical education management, research and development experiments, medical facilities, and so on. The Institute of medicine has an independent management organization, which can be established independently without being attached to the University. This characteristic is very similar to the Japanese education system. The characteristic requirement of the decentralized education management system is to be named as the second unit of the same school as the hospital directly under it. In addition, there is no direct relationship between the Affiliated Hospital and the Affiliated Hospital in China’s Medical University. According to the above problems, they will appear in medical clinical experiments, and the Affiliated Hospital and the medical university often have differences on some matters.In the hospital management system, the clinical medicine and Affiliated Hospital of medical university can be merged into one and two different institutions, but the nature is the same, which is of great significance in the situation of rescuing the wounded and rescuing the dying. There is also a situation where the clinical medicine of the Medical University and the affiliated hospital have no relationship and influence with each other. There is a legend that an American educator named Miller once drew a pyramid in medicine. It mainly tells about the basic literacy of a medical staff and his ability to solve clinical problems. As shown in Figure5, Miller pyramid, which can be called classic, was also very famous at that time, Miller pyramid shows that a qualified medical staff should have their own learning and growth process at each stage, and constantly absorb and train their clinical medical ability at each stage to achieve their goals step by step.Figure 5
Miller pyramid model.There are some problems in the implementation of the system of “hospital integration”, some medical teaching processes have many different places and differences, and we need to seriously deal with them: in the complex social relations, there are differences in the communication content, such as medical university, Medical University College, Affiliated Hospital, etc. In this pyramid system, there are many conflicts among different classes, the groups are different leadership classes, different employees in different posts, and the concept of teaching management level, which are the conflicts and contradictions existing in solving problems in three units. At the same time, there are also great differences in management teaching, but the leadership cannot organize and communicate in time to solve them. Thus, the conflicts between the three parties cannot be reduced and the contradictions can only be intensified constantly.The hospital has no independence, which makes it impossible for the hospital to have its own management power, and others can intervene in any matter. Especially in financial accounting affairs, there are also personnel exchanges and personnel relationship handling relationships in hospitals. Hospitals cannot make decisions on their own. If they cannot say anything important, the development in the next stage has great limitations. They cannot realize their ideas and keep their hands tied. Even with good suggestions, they can not practice. These are all the consequences of the inability to meet the conditions of the external environment. Basically, it’s all in vain.There are also some common and serious problems in the teaching management of the supervision of the prosecution system is not complete. The purpose of the regulatory prosecution system is to improve the management level and measure system of teaching medicine. Even if there are many facilities and equipment in Medical University and relevant systems, these systems can not be completely convincing. There is also a situation where the implementation of the problems will be in a bad way.In contrast, hospitals will pay attention to the cost of the economy, and it is difficult to invest too much in teaching. There are also some medical hospitals with formal but small sizes. They only pay attention to short-term profit and increase the utilization rate of cost, but ignore the medical teaching and scientific research innovation, especially not. At this time, the proportion of teaching management and research and development experiments is less and the status will fall. The number of teachers’ tasks in clinical experimental Department of learning teaching management is increasing. After the hospital is integrated, doctors have multiple roles, that is, when doctors and teachers, of course, the tasks of doctors are also diverse. They should do not only clinical medical experiments but also teaching management. The contradiction is that the workload in both directions cannot be well arranged to continue, which will cause trouble to the work, and then delay the progress and slow down the efficiency.


### 4.2. Operation Mechanism of Higher Medical Education

The education management of colleges and universities is to deal with relevant matters according to the rules and regulations stipulated by the Education Bureau, which is also a standard that the education department must abide by. Among the types of education, the development of medical education is the most characteristic, and it is also a very special type of education. Therefore, its characteristics are similar to those of ordinary colleges and universities, but also different from those of colleges and universities. In the educational purpose of colleges and universities, their main responsibility is to train high-tech medical talents for the country, develop better medical products and service equipment, strengthen community construction and improve the medical and health security of urban people. The most fundamental characteristics are as follows: it has the social function of human beings, in essence, the complexity of the content is inconsistent with the facts, the practical activities related to medical clinical experiments, and the inconsistency of teaching management. All of these prove that the management of medical education is a combination of practice and theory. The proportion of practice is a little more than that of education. This requires educators to pay more attention to practical teaching than theoretical output while cultivating students. Affiliated hospitals can be a channel for students to practice, medical knowledge in education, and an important part of higher medical education. Interns can learn professional clinical experiment opportunities in class, and learn new knowledge that is not in textbooks through clinical experiments in practice. They need to explain clinical professional theoretical knowledge through classroom teaching. In addition, we must teach students clinical practice skills through the medical practice of outpatient and inpatient departments.In today’s society, with the arrival of the new era, it also brings many new challenges and opportunities. Along with it comes the doctor’s moral standards and the complex relationship between doctors and patients’ families, which are hot topics in today’s society. All of them are what the society needs to face, so many solutions have been put forward according to these problems, that is, the requirements of the society are higher and higher on the professional quality of doctors, and the patients and nurses in hospitals also need to know a lot of theoretical basis and rich social practice. Patients need more care and care from doctors and nurses when they are facing diseases, It is the commonwealth of all patients and even their families to have excellent psychological quality, experience from long-term surgical experiments, and a professional doctor with love and responsibility.To build an institution different from the previous education system can not only improve the establishment and improvement of clinical teaching management system and mechanism, but also help to ensure and improve the quality of medical personnel training, as well as the self-development of medical colleges and affiliated hospitals, as shown in Table5.Table 5
The basic elements of cognitive apprenticeship.
ContentDomain knowledge, strategic knowledge (heuristic strategy, control strategy, learning strategy)MethodDemonstration, guidance, construction of scaffolding, dismantling of scaffolding (core technology), clear expression, reflection and explorationSequenceIncreasing complexity and diversity of knowledge and skills, global skills ahead of local skills strategiesSocialityEmotional learning, sociality, internal motivation, cooperation, and competition


## 4.1. Current Situation of Clinical Teaching Management System

Nowadays, in the field of medicine, the teaching method of clinical medicine is the embodiment of a medical student’s basic medical literacy, and it is also one of the key points of innovation in the teaching characteristics of medical colleges. In recent years, with the continuous reform and innovation and development of higher education, the cheers of reform and education innovation in many medical colleges are increasing. In the environment of this reform, the education management center and core content of traditional Chinese medicine are the top priority and the focus of the reform. In this journey of reform and innovation, we have absorbed the development mode and operation system of higher education at home and abroad to supplement the educational problems and loopholes existing in China’s educational background, which has a far-reaching impact on the future development of China’s education, It plays an important role in the training of talents and the development of education, and also has a guiding role in the future development of young people.There are many problems in the management system of Chinese higher medical education, such as whether the laboratory teaching management is centralized or decentralized. The centralized education management system is to establish the medical department with the University as the center. The management department is composed of all the research institutes, medical colleges, departments and laboratories related to the hospital in the University. And they are responsible for all the medical education management, research and development experiments, medical facilities, and so on. The Institute of medicine has an independent management organization, which can be established independently without being attached to the University. This characteristic is very similar to the Japanese education system. The characteristic requirement of the decentralized education management system is to be named as the second unit of the same school as the hospital directly under it. In addition, there is no direct relationship between the Affiliated Hospital and the Affiliated Hospital in China’s Medical University. According to the above problems, they will appear in medical clinical experiments, and the Affiliated Hospital and the medical university often have differences on some matters.In the hospital management system, the clinical medicine and Affiliated Hospital of medical university can be merged into one and two different institutions, but the nature is the same, which is of great significance in the situation of rescuing the wounded and rescuing the dying. There is also a situation where the clinical medicine of the Medical University and the affiliated hospital have no relationship and influence with each other. There is a legend that an American educator named Miller once drew a pyramid in medicine. It mainly tells about the basic literacy of a medical staff and his ability to solve clinical problems. As shown in Figure5, Miller pyramid, which can be called classic, was also very famous at that time, Miller pyramid shows that a qualified medical staff should have their own learning and growth process at each stage, and constantly absorb and train their clinical medical ability at each stage to achieve their goals step by step.Figure 5
Miller pyramid model.There are some problems in the implementation of the system of “hospital integration”, some medical teaching processes have many different places and differences, and we need to seriously deal with them: in the complex social relations, there are differences in the communication content, such as medical university, Medical University College, Affiliated Hospital, etc. In this pyramid system, there are many conflicts among different classes, the groups are different leadership classes, different employees in different posts, and the concept of teaching management level, which are the conflicts and contradictions existing in solving problems in three units. At the same time, there are also great differences in management teaching, but the leadership cannot organize and communicate in time to solve them. Thus, the conflicts between the three parties cannot be reduced and the contradictions can only be intensified constantly.The hospital has no independence, which makes it impossible for the hospital to have its own management power, and others can intervene in any matter. Especially in financial accounting affairs, there are also personnel exchanges and personnel relationship handling relationships in hospitals. Hospitals cannot make decisions on their own. If they cannot say anything important, the development in the next stage has great limitations. They cannot realize their ideas and keep their hands tied. Even with good suggestions, they can not practice. These are all the consequences of the inability to meet the conditions of the external environment. Basically, it’s all in vain.There are also some common and serious problems in the teaching management of the supervision of the prosecution system is not complete. The purpose of the regulatory prosecution system is to improve the management level and measure system of teaching medicine. Even if there are many facilities and equipment in Medical University and relevant systems, these systems can not be completely convincing. There is also a situation where the implementation of the problems will be in a bad way.In contrast, hospitals will pay attention to the cost of the economy, and it is difficult to invest too much in teaching. There are also some medical hospitals with formal but small sizes. They only pay attention to short-term profit and increase the utilization rate of cost, but ignore the medical teaching and scientific research innovation, especially not. At this time, the proportion of teaching management and research and development experiments is less and the status will fall. The number of teachers’ tasks in clinical experimental Department of learning teaching management is increasing. After the hospital is integrated, doctors have multiple roles, that is, when doctors and teachers, of course, the tasks of doctors are also diverse. They should do not only clinical medical experiments but also teaching management. The contradiction is that the workload in both directions cannot be well arranged to continue, which will cause trouble to the work, and then delay the progress and slow down the efficiency.


## 4.2. Operation Mechanism of Higher Medical Education

The education management of colleges and universities is to deal with relevant matters according to the rules and regulations stipulated by the Education Bureau, which is also a standard that the education department must abide by. Among the types of education, the development of medical education is the most characteristic, and it is also a very special type of education. Therefore, its characteristics are similar to those of ordinary colleges and universities, but also different from those of colleges and universities. In the educational purpose of colleges and universities, their main responsibility is to train high-tech medical talents for the country, develop better medical products and service equipment, strengthen community construction and improve the medical and health security of urban people. The most fundamental characteristics are as follows: it has the social function of human beings, in essence, the complexity of the content is inconsistent with the facts, the practical activities related to medical clinical experiments, and the inconsistency of teaching management. All of these prove that the management of medical education is a combination of practice and theory. The proportion of practice is a little more than that of education. This requires educators to pay more attention to practical teaching than theoretical output while cultivating students. Affiliated hospitals can be a channel for students to practice, medical knowledge in education, and an important part of higher medical education. Interns can learn professional clinical experiment opportunities in class, and learn new knowledge that is not in textbooks through clinical experiments in practice. They need to explain clinical professional theoretical knowledge through classroom teaching. In addition, we must teach students clinical practice skills through the medical practice of outpatient and inpatient departments.In today’s society, with the arrival of the new era, it also brings many new challenges and opportunities. Along with it comes the doctor’s moral standards and the complex relationship between doctors and patients’ families, which are hot topics in today’s society. All of them are what the society needs to face, so many solutions have been put forward according to these problems, that is, the requirements of the society are higher and higher on the professional quality of doctors, and the patients and nurses in hospitals also need to know a lot of theoretical basis and rich social practice. Patients need more care and care from doctors and nurses when they are facing diseases, It is the commonwealth of all patients and even their families to have excellent psychological quality, experience from long-term surgical experiments, and a professional doctor with love and responsibility.To build an institution different from the previous education system can not only improve the establishment and improvement of clinical teaching management system and mechanism, but also help to ensure and improve the quality of medical personnel training, as well as the self-development of medical colleges and affiliated hospitals, as shown in Table5.Table 5
The basic elements of cognitive apprenticeship.
ContentDomain knowledge, strategic knowledge (heuristic strategy, control strategy, learning strategy)MethodDemonstration, guidance, construction of scaffolding, dismantling of scaffolding (core technology), clear expression, reflection and explorationSequenceIncreasing complexity and diversity of knowledge and skills, global skills ahead of local skills strategiesSocialityEmotional learning, sociality, internal motivation, cooperation, and competition


## 5. Conclusion

The performance of deep reinforcement learning and its dual structure version and interleaved structure version are compared in different medical teaching and clinical management environments and analyzed in detail. Deep reinforcement learning provides new ideas for the improvement of intelligent multi-objective optimization and clinical management of medical teaching, while retaining some of the problems they face, such as unstable factors in reinforcement learning, which will also be inherited in deep learning. Unstable algorithms will have adverse effects on the learned results and strategies. So this article has launched an in-depth study on this and proposed a series of stable reinforcement learning and deep reinforcement learning algorithms. This article applies deep reinforcement learning to intelligent multi-objective optimization and improvement and clinical management of medical teaching. In this article, we use the advantages of deep reinforcement learning and apply it to intelligent multi-objective optimization to gradually improve medical teaching and clinical management experiments. In practical applications, most neural network structures that like structure types are designed in this article. The input method is model parameters, and the output method is the value of the model parameters corresponding to the Pareto solution. After several rounds of mutual absorption of each other’s advantages, the final result is a Pareto solution. Then, use the data model that the designer himself likes as the optimization target of the ANN model. It is best to find a new plan to improve the target and use the intelligent optimization target as the improvement guidance plan. The deep-enhanced learning program is used to meticulously study the teaching improvement of medical clinical experiments, which can enhance the overall ability and teaching level of the doctor team. Improve the knowledge system learned, consolidate the knowledge learned, cultivate the interest and hobby of learning, and strengthen the students’ practical ability to adapt to the modern medical education and learning mode in line with the development of globalization. This article aims at the characteristics of clinical medical education in China, combined with the actual needs of medical education in medical schools, and systematically analyzes modern distance medical education, medical education models, and clinical skills training, and combs relevant pedagogy theories from different theoretical perspectives. Constructed a remote medical education clinical skills training model and developed a remote clinical skills training platform prototype based on the model design. It provides a set of effective clinical skills training methods for the vast number of medical workers in China, and also makes useful explorations on how to effectively use the Internet to carry out distance medical education.



---

*Source: 4881321-2022-08-30.xml*