esa-sceva/satcom-chunk-collection huggingface dataset:
# Design and Analysis of Novel Low-Cost Linear Vibration Motor for an Electronic Cigarette
## 3 Analysis Methods
### Experimental Setup  
The experimental setup for measuring the acceleration on the testing jig includes a computer, a monitor, a mouse, and a keyboard, a vibration tester (Type BK 2120C2), a testing jig, and a vibration motor, as shown in Figure 13. The testing jig consists of a base, 4 spring foundations, a 100 g dummy jig, and an accelerometer. The detailed components of the testing are shown in Figure 14. When the sinusoidal voltage is applied to the vibration motor, the motor vibration and the acceleration on the dummy jig can be detected by an accelerometer. The acceleration is converted from a time-domain signal to a frequency-domain signal through a preamplifier and fast Fourier transform (FFT) analysis inside the vibration tester, and the results are displayed on the screen.  
Figure 11: Each component of the linear vibration motor sample prototype.  
Figure 12: Sample and 3D model of the prototype spring (thickness = 0.20 mm).  
Figure 10: The coin-type vibration motor sample.  
Figure 13: The acceleration measurement setup under standard test conditions.  
A vibration motor is designed to provide haptic feedback to a human by vibrating human skin. According to mechanoreceptors [11], the Pacinian corpuscles in the skin and fascia can detect rapid vibrations of around 200-300 Hz. In addition, the most sensitive frequency of the human finger is around 250 Hz [12; 13].  
The coin-type vibration motor has been used for E-cigarettes for a long time, so its acceleration was measured as reference data. Two input voltages were used for the prototype linear vibration experiment. One is 0.5 V to check the resonance frequency of the vibration motor. The other is 4.35 V to measure the acceleration performance from 100 to 300 Hz. Furthermore, the final excited frequency range is from 240 to 260 Hz at 4.35 V.  
A comparison between the experiment results of the acceleration on the dummy jig of the prototype with different voltages is shown in Figure 15. The analysis results are in good agreement with the experiment results. The acceleration of the prototype (0.021 g) is 86.0% lower than that of the coin-type vibration motor (0.15 g) at 250 Hz, which is a significant problem that needs to be solved. Based on the same analysis methods, there are several designs listed in the next section, and the analysis results of the novel design were verified by experiment results.  
Figure 14: The detailed components of the acceleration testing jig.  
Figure 15: Acceleration comparison between coin-type vibration motor and prototype linear vibration motor.
# Tri-Partition Alphabet-Based State Prediction for Multivariate Time-Series
## 3 Models and Problem Statement
### Data Model
Notably, \(g=2\) means that \(D=\{0\}\)._  
Figure 1: The original numerical MTS and PAA-MTS. **Example 3**. : _Figure 2 shows an example of the transition from PAA-MTS to SAX-MTS. Here, we let \(D=\{-1.07,-0.57,-0.18,0.18,0.57,1.07\}\), \(g=6\), and its alphabet \(\Sigma=\{a,b,c,d,e,f,g\}\)._  
**Example 4**. : _Table 3 shows an example of SAX-MTS with three variables (i.e., \(A=\{SO_{2}\) (\(a_{1}\)), \(NO_{2}\) (\(a_{2}\)), and PM2.5 (\(a_{3}\))), and 10 time stamps (i.e., \(T=\{t_{1},t_{2},\ldots,t_{10}\}\)). For variable \(SO_{2}\), symbols a and f are missing. For variable \(NO_{2}\), symbols b and e are missing. For variable PM2.5, symbols e, d, and e are missing. This phenomenon is temporary until the data are big enough._ \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline \multirow{2}{*}{\(\mathbf{D}\)} & \multicolumn{8}{c}{\(\mathbf{g}\)} \\ \cline{2-10}  & **3** & **4** & **5** & **6** & **7** & **8** & **9** & **10** \\ \hline \(\delta_{1}\) & \(-0.43\) & \(-0.67\) & \(-0.84\) & \(-0.97\) & \(-1.07\) & \(-1.15\) & \(-1.22\) & \(-1.28\) \\ \(\delta_{2}\) & \(0.43\) & \(0\) & \(-0.25\) & \(-0.43\) & \(-0.57\) & \(-0.67\) & \(-0.76\) & \(-0.84\) \\ \(\delta_{3}\) & & \(0.67\) & \(0.25\) & \(0\) & \(-0.18\) & \(-0.32\) & \(-0.43\) & \(-0.52\) \\ \(\delta_{4}\) & & & \(0.84\) & \(0.43\) & \(0.18\) & \(0\) & \(-0.14\) & \(-0.25\) \\ \(\delta_{5}\) & & & & \(0.97\) & \(0.57\) & \(0.32\) & \(0.14\) & \(0\) \\ \(\delta_{6}\) & & & & & \(1.07\) & \(0.67\) & \(0.43\) & \(0.25\) \\ \(\delta_{7}\) & & & & & & \(1.15\) & \(0.76\) & \(0.52\) \\ \(\delta_{8}\) & & & & & & & \(1.22\) & \(0.84\) \\ \(\delta_{9}\) & & & & & & & & \(1.28\) \\ \hline \hline \end{tabular}
\end{table} 
Table 2: The breakpoints for the \(\mathcal{N}(0,1)\) distribution [52]. \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{\(\mathbf{T}\)} & \multicolumn{4}{c}{\(\mathbf{A}\)} \\ \cline{2-5}  & **SO\({}_{\mathbf{2}}\) (\(a_{1}\))** & **NO\({}_{\mathbf{2}}\) (\(a_{2}\))** & **PM2.5 (\(a_{3}\))** \\ \hline \(20(t_{1})\) & b (\(-0.989\)) & a (\(-1.422\)) & b (\(-0.857\)) \\ \(21(t_{2})\) & b (\(-0.966\)) & a (\(-1.460\)) & b (\(-0.770\)) \\ \(22(t_{3})\) & b (\(-0.615\)) & c (\(-0.318\)) & b (\(-0.752\)) \\ \(23(t_{4})\) & d (\(-0.106\)) & d (\(0.095\)) & b (\(-0.681\)) \\ \(24(t_{5})\) & g (\(1.173\)) & f (\(1.007\)) & f (\(0.922\)) \\ \(25(t_{6})\) & g (\(1.496\)) & f (\(0.842\)) & g (\(1.490\)) \\ \(26(t_{7})\) & e (\(0.272\)) & f (\(0.609\)) & f (\(0.959\)) \\ \(27(t_{8})\) & c (\(-0.203\)) & d (\(0.016\)) & c (\(-0.465\)) \\ \(28(t_{9})\) & c (\(-0.508\)) & c (\(-0.453\)) & b (\(-0.691\)) \\ \(29(t_{10})\) & e (\(0.447\)) & g (\(1.083\)) & f (\(0.846\)) \\ \hline \hline \end{tabular}
\end{table} 
Table 3: An example of SAX-MTS and PAA-MTS. Figure 2: The PAA-MTS and SAX-MTS for NO\({}_{2}\).
# Resistance to Cybersecurity Attacks in a Novel Network for Autonomous Vehicles
## 7 Implementation
### Simulators
#### 7.1.1 Opportunistic Networking Environment Simulator  
The opportunistic networking environment (ONE) is a specifically developed simulation tool designed for opportunistic networks [45]. It is designed to allow a direct comparison between routing protocols based on how they would react to real world scenarios. On the initial start-up of the ONE simulator a default simulation is loaded, which makes it easy to understand what the simulator is capable of straight away. It includes a map of Helsinki in order to run its simulations. A user can provide their own maps, however they have to be in multiple layers in a Well-Known Text (.wkt) file type. Due to the variety of features ONE offers, it was used to perform the experiments making the evaluation metrics easier to collect.
###### Abstract  
Unmanned aerial vehicles have been widely employed in recent years owing to their remarkable features such as low environmental requirements and high survivability, and a new tendency towards networking, intelligence, and collaboration has emerged. The realization of these novel capabilities requires a secure and efficient wireless communication channel; however, it is vulnerable to eavesdropping, forgery, and manipulation by attackers. Therefore, ensuring the security of the wireless communication between unmanned aerial vehicles and ground stations is an urgent issue. The traditional solution to this problem is to design an authenticated key-agreement protocol between unmanned aerial vehicles and ground stations. However, an analysis of existing representative methods has shown that these methods are computationally expensive and difficult to implement in resource-intensive aerial vehicles. Furthermore, existing key-agreement systems are highly dependent on the security of temporary session information. When the temporary session information is stolen, the attacker can obtain the session key for the current communication and perform information theft attacks. Therefore, a security-enhanced lightweight authenticated key-agreement protocol for unmanned aerial vehicles' communication is proposed in this study. We present a low-computational-cost agreement method that can achieve secure key agreement in cases of temporary session information leakage. Both theoretical analysis and experimental verification show that our proposed protocol has superior security properties and lower computational costs than representative protocols.  
unmanned aerial vehicles; key-agreement protocol; security analysis +
Footnote †: journal: Applied sciences
## 3 Some Statistical Properties
### Order Statistics
The nonlinear following score equations are used to obtain the maximum likelihood estimators (MLE) \(\hat{\alpha}\), \(\hat{\beta}\), and \(\hat{\theta}\) of the unknown parameters \(\alpha\), \(\beta\), and \(\theta\), as  
\[\frac{3L}{\hat{\alpha}}=\sum_{i=1}^{n}\delta_{i}\Big{[}\frac{16 \alpha^{3}}{\omega_{i}}-\tfrac{1}{\alpha}-\tfrac{1}{\alpha_{i}}\big{(}16 \alpha^{3}+4\alpha^{2}\theta^{2}+2\theta^{3}x_{i}^{\beta}\big{)}\Big{]}-\sum_ {i=1}^{n}\tfrac{1}{S(x_{i},\gamma)}\bigg{(}\tfrac{\gamma}{\alpha}\big{(}1- \tfrac{1}{2}u_{i}e^{-\upsilon_{i}}\big{)}-e^{-\upsilon_{i}}\bigg{(}\tfrac{ \alpha x_{i}^{\beta}x_{i}^{\beta}}{\alpha^{2}(2\alpha^{2}+\theta^{2})}+\tfrac {\upsilon_{i}}{\alpha}+2 uu_{i}\Big{)}\bigg{)},\] \[\frac{3L}{\hat{\alpha}}=\sum_{i=1}^{n}\delta_{i}\Big{[}\tfrac{1}{ \theta}+\frac{4\alpha^{3}x_{i}^{2\beta}}{\omega_{i}}-\tfrac{1}{\alpha_{i}} \big{(}4\alpha^{2}\theta+4\theta^{3}x_{i}^{2\beta}+6\alpha\theta^{2}x_{i}^{ \beta}\big{)}\Big{]}+\sum_{i=1}^{n}\tfrac{1}{S(x_{i},\gamma)}\bigg{(}\tfrac{ \alpha^{3}}{\alpha}\big{(}1-\tfrac{1}{2}u_{i}e^{-\upsilon_{i}}\big{)}-e^{- \upsilon_{i}}\bigg{(}\tfrac{\alpha x_{i}^{\beta}x_{i}^{\beta}}{\alpha}-\tfrac{ 2\theta^{3}x_{i}^{\beta}+3\alpha\theta x_{i}^{\beta}}{\alpha^{2}(2\alpha^{2}+ \theta^{2})}+\theta u_{i}\Big{)}\bigg{)}.\]  
\[\frac{3L}{\hat{\alpha}\beta}=\sum_{i=1}^{n}\delta_{i}\bigg{[}\tfrac{1}{\theta}+ \ln(x_{i})+\tfrac{2\theta^{4}x_{i}^{2\beta}\ln(x_{i})}{\omega_{i}}-\tfrac{1}{ \theta_{i}}\Big{(}2\theta^{4}x_{i}^{2\beta}\ln(x_{i})+2\alpha\theta^{3}x_{i}^ {\beta}\ln(x_{i})\Big{)}\bigg{]}-\]  
\[\sum_{i=1}^{n}\tfrac{1}{S(x_{i},\gamma)}\bigg{(}v_{i}\ln(x_{i})\Big{(}1- \tfrac{1}{2}u_{i}e^{-\upsilon_{i}}\Big{)}-e^{-\upsilon_{i}}\bigg{(}v_{i}\ln(x _{i})e^{\upsilon_{i}}-\tfrac{\theta^{4}x_{i}^{2\beta}\ln(x_{i})+\alpha\theta^{3 }x_{i}^{\beta}\ln(x_{i})}{\alpha^{2}(2\alpha^{2}+\theta^{2})}\bigg{)}\bigg{)}\]  
The explicit forms of \(\hat{\alpha}\), \(\hat{\beta}\), and \(\hat{\theta}\) cannot be gained, so we may use some numerical methods to obtain them.
# Power Optimization in Multi-Tier Heterogeneous Networks Using Genetic Algorithm
## 4 System Model
### Channel Model  
The access point \(AP_{i}\) transmits the information signal \(x_{AP_{i}}\), which is connected via relay node \(R_{i}\). It is fed with multiple signals \(y_{R}\), including the desired signal (DS) from access points through a channel with path loss \(P_{A,R}\) between the access point and relay nodes, the inter-relay interference (IRI) from \(R_{j}\) through the channel \(|H_{R_{j},R_{i}}|\) with path loss \(P_{R,R}\) between relay nodes, the relay self-interference (RSI) through the channel \(|H_{R_{i},R_{i}}|\), and the received noise \(n_{R}\). To acquire channel estimates, a total of \(l_{p}\) pilots are transmitted in the uplink, which are mutually orthogonal. The channel coefficients for the downlink are obtained by assuming channel reciprocity.
## 2 Literature Review
### Environmental Effects on Resin-Printed Part Performance  
#### 2.3.1. Hygroscopic Behavior and Moisture-Induced Swelling  
The hygroscopic behavior of photopolymers and its impact on 3D-printed parts is a critical concern, particularly for resin materials utilized in water-exposed environments. Water absorption tends to lead to dimensional changes and compromised mechanical properties, potentially undermining the performance of printed structures. Recent studies have highlighted that 3D-printed resins exhibit higher water sorption rates compared to their traditionally processed counterparts, which consequently affects their integrity and service longevity [28; 42].  
Moisture-induced swelling is often evaluated through experimental methods that focus on dimensional stability and physical properties over time spans under varying humidity conditions. [PERSON] et al. [28; 33] report that the water absorption behavior of acrylic-based resins has significant implications for their application in dental restoratives, where reliability under moist conditions is crucial.  
Furthermore, attention to moisture impacts results in an increased requirement for hybrid resin formulations that combine traditional and innovative polymers, effectively mitigating swelling through tailored structural reinforcements [34; 43].
## 4 Conclusions  
In this work, a deep belief network (DBN) combined with modified grey wolf optimization (MGWO) algorithm for PM2.5 concentration prediction is proposed. The proposed model is successfully applied to the PM2.5 concentration prediction of Baoding city in China. Compared with the other prediction models, the proposed model has higher prediction accuracy.  
The use of DBN, which is an advanced deep learning model, is proposed to establish the close relationship between the influencing factors and pollutant. The use of MGWO algorithm, which is an advanced optimization algorithm, is proposed to determine the DBN structure parameters. The basic grey wolf optimization (GWO) algorithm is modified by using the nonlinear convergence and position update strategies, so that the MGWO algorithm can get a better solution.  
Considering that the selection of DBN structure parameters still has no theoretical results, this paper only obtains a possible optimal solution from the perspective of optimization. Future work will focus on the selection theory of DBN structure parameters to further improve the prediction accuracy of PM2.5 concentration.  
[PERSON]. ([PERSON]) and [PERSON] conceived and designed the experiments; [PERSON]. ([PERSON]) performed the experiments; [PERSON] and [PERSON] analyzed the data; [PERSON] carried out the data acquisition and manuscript editing; [PERSON]. ([PERSON]) wrote the paper. All authors have read and approved the final manuscript.  
This research was funded by National Key R&D Program of China, grant number 2018 YFC1508603.  
The authors declare no conflict of interest.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Model** & **MAE (\(\upmu\)g/m\({}^{3}\))** & **MSE (\(\upmu\)g\({}^{2}\)/m\({}^{6}\))** & **R\({}^{2}\)** \\ \hline GABP & 20.318 & 551.859 & 0.708 \\ DESVM & 18.623 & 492.428 & 0.758 \\ Random Forest & 18.957 & 498.668 & 0.863 \\ MGWODBN & 17.604 & 410.266 & 0.884 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Prediction errors of PM2.5 concentration using different models  
Figure 7: Prediction curves of PM2.5 concentration using different models.
## 2 Materials and Methods
### Data Analysis
#### 2.3.2 \(p\)-Value Interpretation  
A Neo-Fisherian significance assessment (NFSA) was used instead of the paleo-Fisherian and Neyman-Pearson paradigms (i.e., null hypothesis significance tests) [41]. The NFSA paradigm includes the following definitions: (1) the paradigm does not fix \(\alpha\) (i.e., the level of significance); (2) the paradigm does not describe \(p\)-values as significant or nonsignificant; (3) the paradigm does not accept null hypotheses based on high \(p\)-values, but only suspends judgment; (4) the paradigm presents effect size information in conjunction with \(p\)-values; and finally, (5) the NFSA paradigm interprets \(p\)-values based on three-valued logic; namely, either they seem to be positive (i.e., there seems to be a difference between Finland and Spain), seem to be negative (i.e., there does not seem to be a difference between Finland and Spain), or judgment is suspended (regarding the difference between Finland and Spain) [41].  
[PERSON] and [PERSON] cited [PERSON]'s philosophical proposal that \"no scientific worker has a fixed level of significance at which from year to year, and in all circumstances, he rejects [null] hypotheses; he rather gives his mind to each particular case in light of his evidence and ideas.\" [41] (p. 316). In addition, [PERSON] et al. noted that the logic of Occam's razor should not be used for universal interpretation of the \(p\)-value [42]. Therefore, the neo-Fisherian paradigm was used to interpret the signs and magnitudes of the statistical effects [41].  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Effect Size Estimation Procedure** & **Small** & **Medium** & **Large** & **Reference** \\ \hline The absolute natural log odds ratio \(|\ln\theta|\) & 0.51 & 1.24 & 1.90 & [36] \\ The absolute Cliff’s \(\delta\)\(|\delta|\) & 0.147 & 0.33 & 0.474 & [37] \\ \hline \hline \end{tabular}
\end{table}
Table 3: The absolute effect size thresholds.
# Analytical Model for the Duty Cycle in Solar-Based EH-WSN for Environmental Monitoring
## 4 Analytical Energy Consumption Model
### Assumptions  
In order to preserve the analytical tractability, the energy model developed in this section relies on some simplifying assumptions, which will be further validated via the simulation tests described in the next section. These assumptions are as follows:  
* The sensor network supports a monitoring application, which means that nodes periodically sense the environment and send the corresponding data towards a sink or base station. So, the network behaves as a data-gathering tree where the overall data flow from nodes to base station is  
Figure 1: State transition diagrams of one CC2420 radio activation with no channel activity (**left**) and with channel activity (**right**). RX and Power Off states include also their durations, \(T_{l}\) and \(T_{slp}\), respectively, on the figure on the left.  
organized into communication rounds whose duration is known as reporting period. With no loss of generality, we assume that every node generates a single packet per communication round (which may result from aggregating multiple samples of the environment).
* Power control is enabled and hence every node can tune its transmission power according to the receiver distance (in fact, this is why we define forward and backward transmission distances).
* For simplicity purposes, the proposed model does not reflect the transitory states described in Figure 1 (Power Down and Idle), which means that it only captures the alternation between RX-TX (ON) and Power Off (OFF) states. However, given the small durations and current draws that correspond to transitory states, this assumption can be adopted without penalizing the realistic nature of the model. In fact, simulation results shown in Section 5 reveal the validity of this assumption.
* There are no packet collisions. Collisions are quite unpredictable, as they depend on the spatial and temporal distribution of the traffic load; however, given the fact that nodes are not synchronized and typically reporting periods in data-gathering applications are very large (on the order of minutes or more), we can reasonably assume that, after an initial period, transmissions become sufficiently randomized so as to neglect collisions. Again, the experiments performed with Avrora (reported in the next section) allowed us to definitely validate this assumption.
* Since transmissions are randomized, we can also assume that all packets processed by any intermediate node during a communication round (either generated or received and forwarded from any other node) occupy different duty cycles. This assumption is based again on the fact that reporting periods in monitoring applications are very large compared to the length of duty cycles. The experiments performed with Avrora showed that, even if this is not the case for some nodes, it only affects the way energy consumption is distributed over time, but it has very little impact on the overall balance of energy consumption per node and per communication round.
# Frequency-Dependent Acoustic Reflection for Soil Classification in a Controlled Aquatic Environment
## 2 Materials and Methods--Acoustic Spectrum Characterization
### The Parameters Under Study  
To quantify the acoustic reflection spectrum, the following energy parameters were calculated:  
Incident Energy \(I_{1}(f)\)--the energy reaching the soil surface at a specific frequency.  
Figure 2: The rating curve (**a**) and the GP soil acceptance result table (**b**).  
Reflected Energy \(I_{0}(f)\)--the energy leaving the soil surface post-interaction at a specific frequency.  
It can be assumed that the ratio between these values describes the soil's frequency-dependent reflection. To accurately determine the acoustic reflection spectrum, the incident energy reaching the soil surface \(I_{1}\), must first be quantified. This energy represents the total acoustic energy propagating through the water before interacting with the soil. The incident energy at each frequency is computed by integrating the squared measured amplitude of the incident signal over time, following the fundamental relation that appears in Equation (1) [23]:  
\[I_{1}(f)=\int_{t_{start}}^{t_{end}}A(f,t)^{2}\cdot dt \tag{1}\]  
where  
\(I_{1}(f)\) is the total acoustic energy impacting the soil surface at a given frequency;  
\(A(f,t)\) represents the normalized amplitude of the transmitted wave; and  
\(t_{start}\) and \(t_{end}\) denote the time window corresponding to the transmission period at the selected measurement location.
## 4 Methods
### Learning Algorithm  
In this paper, we employ the PPO algorithm to train our agent. PPO is an Actor-Critic algorithm. Detailed information about the PPO algorithm is provided in pseudocode in Algorithm 1. The Actor refers to the policy network \(\pi_{\theta}\) described above, while the Critic and Actor utilize the same GIN.  
We present the scheduling process of our proposed reinforcement learning method in the ship panel block assembly line in Figure 5. This reinforcement learning model consists of an Agent that determines the input order of panel blocks and an environment that captures the current state of the assembly line and the processing information of each station using a disjunctive graph. The environment provides feedback to the agent regarding the current processing status of the assembly line. Subsequently, the agent selects a block to input and makes decisions on the next block to input when certain operations on the block are completed.  
```
Input: update epoch \(k\); PPO steps \(M\); number of actors to compute reward and perform update \(N\); actor network \(\tau_{\theta}\); behavior actor network \(\pi_{\theta,a}\); trainable parameters of actor network \(\theta\); trainable parameters of behavior actor network \(\theta_{old}\); critic network \(V_{O}\); trainable parameters of critic network \(\varnothing\); clipping ratio \(\varepsilon\); policy loss coefficient \(C_{p}\); value function loss coefficient \(C_{0}\); entropy loss coefficient \(C_{c}\);
1 Initialization: initialize parameter sets of \(\pi_{\theta}\), \(\pi_{\theta_{old}}\) and \(V_{O}\);
2 for\(m=1,\cdots,M\), do;
3 Pick \(N\) independent scheduling instances from distribution \(D\);
4 for\(n\)=1, \(\cdots,N\), do;
5 for\(t\)=0, 1, 2, \(\cdots\), do
6 sample \(a_{n,t}\) based on \(\pi_{\theta_{old}}(a_{n,t}|S_{n,t})\);
7 Receive reward \(r_{n,t}\) and next state \(S_{n,t+1}\);
8\(\hat{A}_{n,t}=\sum_{0}^{t}\gamma^{t}r_{n,t}-V_{O}(S_{n},t)\); \(r_{n,t}(\theta)\)=\(\frac{\pi_{\theta}(a_{n,t}|S_{n,t})}{\pi_{\theta_{old}}(a_{n,t}|S_{n,t})}\)
9 if\(S_{n,t}\) is terminal then
10 break;
11 end if
12
13 end for
14\(L^{CLIP}(\theta)=\sum_{0}^{t}min\left(r_{n,t}(\theta)\hat{A}_{n,t},\ clip(r_{n,t}(\theta),\ 1- \varepsilon,\ 1+\varepsilon)\hat{A}_{n,t}\right)\)
15\(L^{E}(\theta)=\sum_{0}^{t}Entropy(\pi_{\theta}(a_{n,t}|S_{n,t}))\)
16\(L^{VF}(\varnothing)=\sum_{0}^{t}\left(V_{O}(S_{n,t})-\hat{A}_{n,t}\right)^{2}\)
17\(L(\theta,\varnothing)=C_{p}L^{CLIP}(\theta)-C_{p}L^{VF}(\varnothing)+C_{q}L^{E}\theta\)
18 end for
19 for\(k=1,2,\cdots,K\)do;
20 update actor and critic parameters \(\theta,\varnothing\) by Adam optimizer
21\(\theta,\varnothing=argmax\left(\sum_{n=1}^{N}L_{n}(\theta,\varnothing)\right)\)
22 end for
23\(\theta_{old}\leftarrow\theta\ (update,\ \theta_{old})\)
24 end
```  
**Algorithm 1**PPO Algorithm for training our model  
Figure 5: The algorithm structure of the panel block assembly line scheduling process.