esa-sceva/satcom-chunk-collection huggingface dataset:
# Multi-Agent Deep-Q Network-Based Cache Replacement Policy for Content Delivery Networks
## 4 Formulation of Markov Decision Process Problem  
The content replacement problem can initially be defined as a Markov decision process (MDP) and then decomposed into sub-problems. These sub-problems can be implemented individually on edge servers and solved as single-agent reinforcement learning problems [18]. In traditional single-agent deep reinforcement learning, an agent aims to make effective decisions by considering its past experiences and the rewards it receives. The MDP consists of a finite state space \(\mathcal{S}\), an action space \(\mathcal{A}\), a reward function \(R(.)\), and a transition probability \(T(.)\). The problem starts with an initial state \(\mathbf{s}_{1}\in\mathcal{S}\). At every \(t\)th decision epoch, the agent takes action \(\mathbf{a}_{t}\) and transitions its state \(\mathbf{s}_{t}\) to \(\mathbf{s}_{t+1}\). For each action taken by the agent, the agent receives a reward \(r\). Hence, the transition probability \(T(\mathbf{s}_{t},\mathbf{a}_{t},\mathbf{s}_{t+1})\) and the reward \(R(\mathbf{s}_{t},\mathbf{a}_{t},\mathbf{s}_{t+1})\) are functions of the current state, next state, and the taken action. In our model, the decision epoch is defined when a cache miss occurs. Every cache miss is handled on the decision epoch, and action is taken only on a cache miss.
## 2 Background
### Informed Rapidly Exploring Random Tree-Star (Informed-RRT*) Algorithm  
As a well-known algorithm for random path planning, the RRT in dynamic construction sites can tackle the problems of high-degree nonholonomic freedom, such as moving equipment path planning. The RRT algorithm family has been successfully implemented in different research contexts, such as the motion planning techniques for automated vehicles and the autonomous mobile robot path planning algorithms [31].  
The original version of the RRT algorithm progresses with a specific type of graph in which each node is connected to one parent, and the tree's root (the point of origin) is specified in a place. The algorithm progresses by randomly sampling the nodes and connecting them to the current graph. The system creates a random node (\(\mathbf{q}_{\text{rand}}\)) in the basic configuration in each iteration. If the random node does not collide with any obstacles, it is connected to the nearest available node in the tree (q\({}_{\text{near}}\)), checking that the space is free (unconstrained). Then, the distance between the two nodes is investigated. If the distance between the random node and the nearest available node in the tree is less than the default value (\(\varepsilon\)), the algorithm connects these two nodes directly. However, if the connection length is higher than the default value, a line with the default value length is created from the nearest node to the random node in the tree, and a new node (q\({}_{\text{new}}\)) is generated (Figure 1). In the initiation of random branch creation, the tree only includes one node (i.e., the path initiation node). This iterative method of environment exploration extends the tree until one branch reaches the finishing point [32].  
The Informed-RRT* algorithm version performs similarly to the RRT until a solution is provided. This algorithm only generates new random nodes in an area where the previously found path may be improved. Its pseudocode is shown in Algorithm 1. Random nodes are only placed in an ellipse when a path becomes smooth. The Informed-RRT* algorithm decides where the new nodes should be sampled rather than sampling the whole process domain. The Informed-RRT* generates an n-dimensional hyper ellipsoid that surrounds the path based on the best available solution. Due to the properties of a hyper ellipsoid, the Informed-RRT* restrains the production of random nodes with zero probability for improvement of the path [33].  
```
\(V\leftarrow\left\{x_{\text{start}},\ x_{\text{goal}}\right\}\); \(E\leftarrow\varnothing\); \(X_{\text{sol}}\leftarrow\varnothing\); \(T=(V,\ E)\); for iteration = 1... \(N\)do if\(X_{\text{sol}}\sim=\varnothing\); then \(c_{\text{min}}\gets min(Cost(x|x\in x_{\text{sol}}))\); \(x_{r}\leftarrow\textit{Sample}\Big{(}x_{\text{start}},\ x_{\text{goal}},\ \epsilon_{\text{min}}\Big{)}\); \(T\leftarrow\textit{OptimalRRT}(T,\ x_{r})\); else \(x_{r}\leftarrow\textit{RandomState}()\); \(T\leftarrow\textit{OptimalRRT}(T,\ x_{r})\); endif endfor
```  
**Algorithm 1**Informed RRT* \(\Big{(}x_{\text{start}},\ x_{\text{goal}}\Big{)}\)
# Influence of Suction on Capacity of Pile Foundation
## 1 Introduction
This study is intended to provide the bridge between past studies and future research on the suction effect on pile capacity. This study aims to investigate the effect of suction on the increase in the pile capacity and the effect of rainwater infiltration on changes in the pile capacity based on the data obtained from in situ testing. The three modified methods (i.e., modified \(\alpha\), \(\beta\), and \(\lambda\)) were implemented in order to calculate the shaft capacity of reinforced concrete bored pile foundations incorporating the unsaturated soil mechanics principle. The total cohesion method was implemented to investigate the possible effect of varying matric suction, and the resulting capacity from the ground water table level change was also calculated. The importance of this research is to improve design calculations of the foundation in areas under unsaturated conditions since in conventional practice the mechanism of unsaturated soils is neglected. This is needed for more accurate calculations with justified assumptions in structures such as highways and dams [24]. Possible rain infiltration and the resulting increase in saturation levels can lead to a decrease in the pile capacity and the following failure of the structure.
## 3 Results and Discussion
### Antimicrobial Activity  
The gelatin/CuSNP nanocomposite films' antimicrobial activity is shown in Figure 6. As anticipated, the gelatin film had no antibacterial action. Conversely, the CuSNP-integrated nanocomposite film drastically decreased the cell viability of _E. coli_ but did not significantly affect the growth of _L. monocytogenes_. The antimicrobial effect of CuSNP varied with concentration. A low concentration (0.5 wt %) of CuSNP was not sufficient to inhibit microbial growth. However, at a high concentration (2.0 wt %), it exhibited solid antibacterial activity enough to entirely impede the growth of _E. coli_ after 9 h of contact. These results clearly showed that the gelatin/CuSNP film showed more vigorous antibacterial activity against Gram-negative compared with Gram-positive bacteria, presumably owing to the bacterial cell wall structure, which consists of a thick cell wall composed of several layers of peptidoglycans. In addition, CuSNP has a lower antimicrobial activity than CuNP due to the sulfidation process, but CuSNP is more advantageous because it has much less cytotoxicity than CuNP [40; 49; 50]. Similar antimicrobial activity behavior has been reported for the agar films integrated with CuSNP [40]. The antimicrobial action of CuSNP has yet to be discovered. However, copper ions are assumed to be released in contact with water and interact with negatively charged membrane proteins, destroying the cell wall and causing cell death [51]. Another probable cause is the interaction between bacteria and nanoparticles that activate reactive oxygen species in the cell, causing oxidative harm to the antioxidant defenses and damaging the cell membrane, thus directing cell death [51].  
Figure 6: Antimicrobial activity of Gel/CuSNP films against (**a**) _E. coli_ and (**b**) _L. monocytogenes_.
# Studies on a Piezoelectric Cylindrical Transducer for Borehole Dipole Acoustic Measurements
## 1 Introduction  
Borehole acoustic wave logging or acoustic wave measurement is used to predict formation lithology, evaluate the formation porosity, and indicate the pore fluid properties in an open hole by measuring the compressional and shear velocities of the formation in the vicinity of a borehole [1]. In addition, these measurements could be used to characterize the geo-stress field in horizontal directions [2; 3; 4].  
In a slow formation in which the shear wave velocity is smaller than that of borehole fluid, borehole dipole acoustic measurement has been used for generating flexural mode waves propagating along the borehole, and these mode waves, during propagation, carry the shear wave events of the formation outside the borehole, and can be used for picking up the shear velocity of the formation [5]. In order to procure the accurate shear velocity and obtain the dispersion of flexural mode waves effectively, dipole transducers are required with a wide frequency bandwidth and a high radiation power. Moreover, the transducer should be able to work against high temperature and pressure in a borehole, with temperatures of 175 \({}^{\circ}\)C and pressures of 140 MPa.  
Since 1968, various dipole acoustic transducers have been proposed and applied. One of the most conventional, piezoelectric bender-type transducers [6; 7; 8; 9], are extensively used in the borehole dipole acoustic measurement tools [10]. However, they have the disadvantage of limited bandwidth in low frequency [10; 11]. Moreover, the bender-type transducers easily produce hexapole aliasing because of the limited azimuthal angle of the opening window of the tool body in high frequency [10]. Although there are some other reports on borehole dipole acoustic transducers, which are not widely used in practice, adipole transducer that has a wide bandwidth of operation and a high modal purity is still in urgent need [12].  
Based on the piezoelectric bender bars [8], a new dipole transducer for borehole acoustic measurement, effective to produce dipole signals in the frequency range of 0.4-6 kHz, is proposed and studied. In our article, four sections are included: principles and numerical analyses of the actuator and the whole transducer; comparison between the test and simulation results of the new transducer; comparison between the simulation results of traditional and new transducers; and conclusions.
## 3 Results
### Physical Properties of Matrices
#### 3.1.2 Swelling Ratio and Degradation Weight-Loss Analysis  
Swelling behavior of dry porous matrices is shown in Figure 5.  
Water uptake capacity is strongly affected by polymer concentration and morphology of the porous network of the matrices. Due to higher concentration of gelatin, these matrices were tighter and displayed poor water absorption. When a plasticizer was added, CMC and HEC films were able to absorb higher amounts of water and the variation trend was similar and correlated to the pore size.  
Figure 4: Optical image of CMC2 matrix directly before application (picture taken with Nikon S9300 camera using macro lens).  
Figure 5: Swelling behavior of CMC and HEC porous matrices: Data are presented as the mean \(\pm\) SE, n = 6.  
The highest swelling value was achieved in systems containing HEC in its structure due to the presence of a high number of hydroxyl groups. Inside of gelatin macromolecules, numerous short-range interactions occur (including structure-stabilizing hydrogen bonds) that limit polymer's swelling and solubility [17; 40]. Therefore, the addition of gelatin to the dispersion resulted in a reduction of absorbency.  
Dried HEC matrices absorbed water corresponding to 1-3 times (100, 170, and 297%) their mass, while in the case of CMC matrices, the water uptake was significantly higher and reached 413, 893, and 999% of dried matrix mass, respectively. The swelling or water uptake ability defines the rate of body fluid (e.g., blood and saliva) uptake into the matrix and is one of the key factors determining its applicability in periodontal surgical treatment. We observed that the CMC matrix degradation ratio is lower than that of the HEC matrix, possibly due to the differing content of hydrophilic groups in these cellulose derivatives. Moreover, matrices with a higher CMC and HEC content displayed higher resistance to dissolution and higher swelling ability (Table 3). It is important to note that, in a periodontal setting, degradable matrices are of high applicability as their use eliminates the necessity of removal by surgical means and allows new tissue to grow and proliferate. A statistically significant difference was observed between all formulations with CMC and HEC (parametric ANOVA test, \(p<0.05\)) with regard to all parameters presented above.
## 6 Simulation Analysis of Thrust Allocation
### Objective Function and Constraint Conditions of Thrust Allocation  
The objective function of thrust allocation can be set to [36; 37; 38]  
\[J(\alpha,u,s)=u^{T}Wu+s^{T}Qs+\frac{\rho}{\varepsilon+\det(B(\alpha)B(\alpha)^{T })} \tag{39}\]  
In the formula, \(\alpha\) is the azimuth angle of each propeller; \(u\) is the thrust generated by each propeller; and, as a relaxation variable, \(s\) is a column matrix, which represents the difference between the thrust input by the controller and the thrust after the thrust synthesis output so as to ensure that the thrust allocation problem always has a feasible solution.  
The definition of the right end of the above equation is as follows:  
* The first term is used to optimize energy consumption. \(W\) is the energy consumption weight matrix, which is a diagonal matrix. The weight of energy consumption in each optimization objective can be changed by changing its size. Optimizing this item can also reduce the wear degree of the propeller and make the propeller work more safely and stably.
* The second term belongs to the penalty term. \(Q\) is also a diagonal matrix, which can determine the weight of the error value of input and output in the objective function by changing the value of \(Q\).
* The third item is set to avoid the singular structure of the thruster, where \(\varepsilon\) is the number greater than 0 and a is the weight of the optimization item in order to avoid the denominator being 0.  
According to the analysis of the actual situation of the ROV, the constraints of thrust distribution are divided into equality constraints and inequality constraints.  
The equality constraint is  
\[s=\tau-B(\alpha)u \tag{40}\]  
In the formula, \(\tau\) is the input thrust of the controller and \(B(\alpha)\) is the vector layout matrix of the propulsion system.  
The constraint condition represents the relationship between the error between the input of the controller and the output of the thrust synthesis and the relaxation variable, allowing the existence of the error, in order to better complete the task of thrust distribution.  
Figure 17: The basic process of the sequential quadratic programming method.  
The inequality constraint is  
\[\begin{split} u_{\text{min}}\leq u\leq u_{\text{max}}\\ \Delta u_{\text{min}}\leq\Delta u\leq\Delta u_{\text{max}}\end{split} \tag{41}\]  
In the formula, \(u_{\text{min}}\) is the minimum thrust of the propeller, \(u_{\text{max}}\) is the maximum thrust of the propeller, and \(u_{0}\) is the thrust at the previous moment.  
This constraint limits the output range of the propeller, which can prevent the loss of the propeller and the low efficiency of the underwater operation caused by exceeding the maximum thrust.
## 2 Materials and Methods
### Data Collection and Analysis  
The force exerted by the Exoband was measured by means of a load cell (ISB S-Beam Force Sensor, Flintec, Hudson, MA, USA), inserted between the belt and the elastic components (Figure 2). Spatio-temporal parameters were recorded using the OptoGait system (Microgate, Italy) placing its two optical bars on each side of the treadmill. Load cells and optical measurement system were synchronized by means of a trigger box sending a 5V square pulse. An overview of the experimental setup is shown in Figure 2. Only the last two-minutes of each five-minute bout were recorded and then used for the analysis. The average between the last two-minutes of the two bouts of each condition was considered for analysis and is reported in the results section. Force data relative to the right leg were segmented and normalised to 0-100% of the gait cycle using the heel-strike event detected by the OptoGait system. To obtain an estimate of the torque applied by the Exoband to the users the moment arm was measured on five participants. The obtained average value (0.08 \(\pm\) 0.01 m) was then multiplied by force data collected by the load cells. All the above analysis was conducted by means of a customized software (MATLAB, The MathWorks Inc., Natick, MA, USA). Metabolic cost was assessed by indirect calorimetry by means of a gas analysis machine (Vmax, SensorMedics, US). Carbon dioxide and oxygen rate were averaged across the last two minutes (3-5 min) of each walking condition and then used to calculate metabolic power using the Brockway equation Brockway (1955). Net metabolic power of the two walking conditions was obtained by subtracting the standing metabolic power from the walking metabolic power of each condition and then normalising it by the body mass of each participant. FreeWalk and EXO conditions presented in the results were obtained as an average of the respective two bouts of walking.  
Figure 2: Overview of the experimental instrumentation adopted in the study.
# Implementing and Testing a U-Space System: Lessons Learnt
## 3 The U-Space Services
### Monitoring
AIT also developed a methodology to analyze the information exchange in the environment, identify weaknesses and weigh the impact of a possible cyber-attack [32].
## 4 Simulation and Experimental Results
Figure 7 depicts waveforms of input (\(i_{g}\)) and output (\(i_{L}\)) currents, as well as input (\(v_{o}\)) and output (\(V_{g}\)) voltages in the same cases previously shown in Figure 5.  
In Figure 7, waveforms of the experimental results show higher QIP in relation to the simulation results due to noise in the experimental tests. In the same way, Figure 6(a),b corresponds to Test 1 using the conventional approach, where the DPWM was configured for 8.96-bit. Output voltage and current ADC resolution are set to 12 bits, and the input ADC input voltage range goes from 0 V to 3 V. Figure 6(a),b shows the simulated and experimental results when the proportional gain of the voltage loop is selected as \(K_{pv}=0.7\). In this case, the current waveforms present perturbations with high current overshoot and undershoot values. Results when the gain \(K_{pv}\) is reduced to the more conservative value of 0.35 in Test 2 are plotted in Figure 6(c),d, showing that limiting the voltage loop bandwidth using the conventional error-calculation method reduces QIP in both currents improving the closed-loop stability. The current waveforms in Test 3 with the proposed improved error measurement approach in Figure 6(e),f, show that there are no significant current perturbations when the proportional gain is again selected to \(K_{pv}=0.7\), so that a wide bandwidth voltage loop is obtained with a phase margin larger than \(50^{\circ}\). In this case, the DPWM has been configured with 8.96-bit resolution, while the ADCs sampling the output current and the voltage error have been configured with resolutions of 12-bit and 16-bit differential mode, respectively. Figure 6(g),h shows the simulated and experimental results for the Test 4, when the proportional gain is \(K_{pv}=0.7\) and both conditions (11) and (14) are fulfilled. Time domain current waveforms of the prototype for the different tests when it works in boost mode are shown in Figure 8. Figure 7(a) current waveforms for Test 1 present high current undershoot because conditions (11) and (14) are not fulfilled. The rest of test results show that the QIP is reduced, obtaining better results for the Test 4 with a \(f_{c}=4\) kHz (Figure 7(d)). It is important to remark that fulfilling condition (11) for the external loop is enough to reduce QIP and  
Figure 6: Voltage loop-gain Bode plots: (**a**) simulated, (**b**) experimental. LCO at the current waveforms in steady-state when the control of the converter is a two-loop with an integrator due to the external loop not cause induced perturbations in the internal loop. Figure 7: Simulated (**a**,**c**,**e**,**g**) and experimental (**b**,**d**,**f**,**h**) start-up waveforms: (**a**,**b**) Test 1, (**c**,**d**) Test 2, (**e**,**f**) Test 3, and (**g**,**h**) Test 4 (\(V_{g}=200\) V, \(v_{o}=300\) V, and \(R_{o}=200\) \(\Omega\)). CH1: \(v_{o}\) (100 V/div). CH2: \(V_{g}\) (100 V/div) CH3: \(i_{g}\) (2 A/div), CH4: \(i_{L}\) (2 A/div).
## 2 Materials and Methods
### Data Sources and Processing
#### 2.2.2 Other Data and Processing  
Temperature and precipitation data, with a 1 km spatial resolution, were acquired from the \"monthly precipitation and monthly average temperature for China\" from the National Tibetan Plateau Data Center. This dataset was created by downscaling global climate data from the Climatic Research Unit (CRU) and WorldClim using the Delta spatial downscaling approach [29]. In this study, we used the annual mean temperature and the total annual precipitation.  
\begin{table}
\begin{tabular}{l l l l l} \hline \hline
**Site Name** & **Lon (\({}^{\circ}\)N)** & **Lat (\({}^{\circ}\)E)** & **Time Periods** & **Vegetation Type** \\ \hline Changbai mountain & 128.10 & 42.40 & 2003–2010 & Forestland \\ Haibei shrubland & 101.33 & 37.67 & 2003–2010 & Shrubland \\ Inner Mongolia & 116.40 & 43.33 & 2004–2010 & Grassland \\ Haibei wetland & 101.32 & 37.60 & 2004–2009 & Wetland \\ \hline \hline \end{tabular}
\end{table}
Table 1: Site descriptions of the eddy covariance flux observation network.  
Figure 1: Geographical distribution of key ecological restoration programs in Northern China. The upper panel (**a**) displays land cover types and the shaded area is the boundary of key ecological restoration programs; the lower panel (**b**) represents digital elevation data. Abbreviations: TNSP, Three-North Shelterbelt Program; APTM, Afforestation Program for Taihang Mountain; SPLR, Shelterbelt Program for Liaohe River; SPMRYR, Shelterbelt Program for Middle Reaches of Yellow River).  
Radiation data were sourced from the Breathing Earth System Simulator (BESS) Radiation v1 dataset, which is based on radiation transfer models and artificial neural networks. This dataset integrates various MODIS atmospheric and land products as input layers and has a temporal resolution of 4 days. It has been validated for reliability across instantaneous, 4-day, and interannual scales [30]. For this study, the radiation data were synthesized on an annual scale. Tree cover data, covering the period from 2001 to 2020, were derived from the MOD44 BV006 dataset, which was also synthesized annually and adjusted to a 1 km spatial resolution to match the study's parameters.