esa-sceva/satcom-chunk-collection huggingface dataset:
# A Review of Current Trends, Techniques, and Challenges in Large Language Models (LLMs)
## 4 Pretraining--Strategies and Objectives
### Learning Strategies
#### 4.2.5. Mixture of Denoisers (MoD)-Based Pretraining  
Typically, the objectives used during pretraining differ in the context in which the model is conditioned. For example, span correction objectives use bidirectional context and are helpful for language understanding and fact completion tasks. In contrast, prefix-LM objectives use unidirectional context (previous tokens) and are helpful for more open-ended and generative tasks. To enable strong performance across all the different tasks, UL2 [58] proposed mixture of denoisers (MoD) objectives that uniformly combine several paradigms to achieve hybrid self-supervised objectives. UL2 distinguishes between these different denoiser modes during pretraining and adaptively switches modes while finetuning the downstream tasks using discrete prompting. This is achieved by introducing an additional paradigm token ([R], [S], or [X]) during pretraining so that the model can select a mode that is more appropriate for the task at hand. This helps bind or associate the downstream finetuning behavior with the specific mode used during pretraining. MoD consists of the following denoising objectives:  
It considers extreme span lengths to have a corruption rate of up to 50%. Therefore, given a small or moderate part of the input, the model is supposed to recover or predict a large chunk of the sequence. The pretraining objective is considered to be highly denoising if it has a long span (for example, equal to or greater than 12) or has a large corruption rate (for example, more significant or more than 30%). So, it covers scenarios with long spans and low corruption, long spans and high corruption, and short spans and high corruption, where it generates long targets based on relatively limited information from memory.  
Sequential Denoising  
This objective strictly follows sequence order, i.e., the prefix language modeling. The target tokens cannot attend to the future context tokens, but the prefix context does use bidirectional architecture.  
Regular Denoising  
This denoising approach has short spans, a range of two to five tokens, and a low corruption rate that masks up to 15% of the sequence. Because of the short span length, they are not fit for generating text but are preferred for acquiring knowledge and understanding tasks.  
With the MoD approach, UL2 outperformed GPT-3 on the SuperGLUE benchmark in the zero-shot setting, and, in the one-shot setting, it tripled the performance of T5-XXL on the summarizing task. In the zero-shot setting, UL2-20B also outperformed T0 and T5 on the massive multitask language understanding (MMLU) benchmark and performed well with chain-of-thought processes using prompting and reasoning steps. UL2-20B, when tested with FLAN instruction tuning, achieved a competitive score compared to FLAN-PaLM 62B on MMLU and Big-Bench benchmarks. After using the MoD objective, U-PaLM [76] achieved the same performance as PaLM-540B but with only half of its computational budget.
# Action Recognition via Adaptive Semi-Supervised Feature Analysis
## 4 Proposed Approach
### Manifold Learning  
In contrast to [10], which utilizes a graph model to estimate data distribution on the manifold, we model the local geometrical structure by generating between-class similarity graph \(G_{b}\) and within-class similarity graph \(G_{w}\), where \(G_{w}(i,j)=1\), if \(x_{i}\in N_{w}(x_{j})\) or \(x_{j}\in N_{w}(x_{i})\); otherwise, \(G_{w}(i,j)=0\). \(G_{b}(i,j)\) applies the same method, although it selects \(x_{i}\in N_{b}(x_{j})\) or \(x_{j}\in N_{b}(x_{i})\), where \(N_{b}(x_{i})\) contains neighbors with different labels, and \(N_{w}(x_{j})\) is the set of neighbors \(x_{j}\) sharing the same label as \(x_{i}\). Notably, the intra-class and inter-class distances can be mapped on a manifold by similarity graphs [33].  
Inspired by manifold learning [12; 31; 33], we maximize inter-class separability and minimize intra-class compactness simultaneously. An ideal transform pushes the connected points of \(A_{b}\) to the extent possible while moving the connected points of \(A_{w}\) closer. The discriminative information can be represented as follows:  
\[\begin{split} f&=\frac{1}{2}\sum_{i,j=1}^{n}(F_{i}- F_{j})^{2}G_{w}(i,j)-\frac{1}{2}\beta\sum_{i,j=1}^{n}(F_{i}-F_{j})^{2}G_{b}(i,j) \\ &=tr(\mathbb{F}^{T}(L_{w}-\beta L_{b})\mathbb{F}),\end{split} \tag{2}\]  
where \(\beta\) is a regularization parameter, which controls the trade-off between inter-class separability and intra-class compactness. \(tr(\cdot)\) denotes the trace operator, and \(L_{w}=D_{w}-G_{w}\) denotes the Laplacian matrix. Furthermore, \(D_{b}\) is a diagonal matrix with \(D_{b}(i,i)=\sum_{j=1}^{n}G_{b}(i,j)\), and \(D_{w}\) is a diagonal matrix with \(D_{w}(i,i)=\sum_{j=1}^{n}G_{w}(i,j)\).
## 4 Experimental Results and Discussion
### Experimental Settings
#### 4.1.1 Transfer Learning Settings  
To fit the architecture's input size, each image was resized to the required value. For VGG16, ResNet50, and both MobileNet architectures, images were fixed to a size of 224 \(\times\) 224. On the Xception, ResNet152v2, and Inception-based architectures, they were scaled to 299 \(\times\) 299, while NasNetLarge used a size of 331 \(\times\) 331. We fed the resized images to these pre-trained networks, extracted the features, and trained our classifiers using them. Both the logistic regression classifier and the SVM classifier were implemented using the Scikit-learn Python library [42].  
Since there is a difference between the available images per category, we implemented five-fold cross-validation, generating five models per architecture. This technique helps reduce model bias when compared to other approaches, such as the training-testing split. The dataset was divided into five folds, four of which were used to fit the model, whereas the last one is used for validation. This process was repeated until every fold was used to test the proposed model.
## 3 Results and Discussion
### TGA Analysis  
TGA analysis was also used to investigate the influence of the chemical modification on thermal stability of PVA. The results from TGA are presented in Figure 5. The deflection temperatures are shown in Table 2.  
Figure 4: High-resolution XPS N\({}_{\text{1s}}\) core level spectra of **(a)** PVA-N\({}_{\text{3}}\) and **(b)** PVA-NH\({}_{\text{2}}\).  
Figure 5: Thermal stability analysis (TGA) curves of **(a)** unmodified PVA; **(b)** PVA-Ts; **(c)** PVA-N\({}_{\text{3}}\) and **(d)** PVA-NH\({}_{\text{2}}\).  
From TGA results, we noticed initial weight loss until 5% for all of PVA samples when the temperature achieved 105 \({}^{\circ}\)C. This loss is due to the evaporation of the adsorbed water. After 105 \({}^{\circ}\)C, we clearly observed a difference between the PVA-Ts and the other samples. For the unmodified PVA, two inflexions were observed with a maximum at 351 \({}^{\circ}\)C and 423 \({}^{\circ}\)C corresponding to two distinct decomposition steps. The first temperature can be assigned to the side chain of the PVA while the second temperature can be related to a decomposition of the main chain of PVA [25]. The loss of 10% of PVA was recorded at 295 \({}^{\circ}\)C. PVA-Ts showed the presence of three inflexions at 161 \({}^{\circ}\)C, 320 \({}^{\circ}\)C and 434 \({}^{\circ}\)C correspond to three distinct decomposition steps. The appearance of the first temperature of degradation and the shift of the other deflections temperature should be related to the presence of the tosyl groups. The loss of 10% of PVA-Ts was recorded early at 139 \({}^{\circ}\)C. The tosyl can modify the intermolecular force between PVA chains. After aidation (Figure 5c), the inflexion at 161 \({}^{\circ}\)C was disappeared due to the consumption of the tosyl group. Comparing to unmodified PVA, the thermal decomposition of PVA-N\({}_{3}\) started at a lower temperature (\(T_{\text{reight loss (10\%)}}\) = 244 \({}^{\circ}\)C comparing to \(T_{\text{reight loss (10\%)}}\) = 295 \({}^{\circ}\)C for PVA). Finally for PVA-NH\({}_{2}\), TGA analysis (Figure 5d) showed an increase of the thermal stability compared to PVA-N\({}_{3}\). The shape of the curve became very close to unmodified PVA. The \(T_{\text{reight loss (10\%)}}\) increased to 275 \({}^{\circ}\)C. This increase can be related to intermolecular interactions due to the incorporation of the amine group.  
These results showed that the incorporation of azide and amine groups slightly modified the thermal stability of PVA. The thermal stability was more influenced in the case of tosylated PVA which was less thermally stable when compared to the other modified PVA.
## 2 Proposed CPM-GFDM Transceiver
### CPM-GFDM Receiver
#### 2.2.2. GFDM Demodulator  
Next, FFT is taken. This is followed by one of the three types of GFDM receivers--matched filter (MF), zero forcing (ZF), or minimum mean square error (MMSE). In this work, we explore MF and ZF. Pre-multiplying (8) by a square matrix \(Z\) having \(KM\times KM\) dimensions, we obtain  
\[Zr_{eq}=Z\mathbf{A}\mathbf{D}+Z\mathbf{\bar{n}} \tag{13}\]  
The matrix \(Z\) is computed such that the estimated transmitted data symbols \(\mathbf{\hat{D}}\) can be obtained as  
\[\mathbf{\hat{D}}=Zr_{eq} \tag{14}\]  
The method of computing the \(Z\) matrix determines whether a matched filter or a zero-forcing receiver is being employed, which is discussed next.
# Extending Battery Lifetime by Avoiding High SOC
## 2 Methods
### Test Procedure  
The capacity as well as the impedance of the battery cells were investigated on a regular basis during the lifetime, using an RPT (Reference Performance Test) procedure, at approximately each  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**SOC Interval (\%)**} & \multicolumn{3}{c}{**25 \({}^{\circ}\)C**} & \multicolumn{3}{c}{**35 \({}^{\circ}\)C**} \\ \cline{2-7}  & **1C** & **2C** & **4C** & **1C** & **2C** & **4C** \\ \hline
0–10 & - & x & - & - & x & - \\
10–20 & x & x & - & x & x & x \\
20–30 & - & x & - & - & - & - \\
40–50 & - & x & - & - & x & - \\
60–70 & x & x & x & x & x & x \\
70–80 & - & x & - & - & - & - \\
80–90 & - & x & - & - & - & - \\
0–30 & - & - & - & - & x & - \\ \hline \hline \end{tabular}
\end{table}
Table 1: Cycling test matrix for developing and verifying the empirical model for various State of Charge (SOC) intervals, temperatures, and C-rates.  
Figure 1: Test cell with holder.  
200 Full Cycle Equivalent (FCE). The RPT procedure contains a 1C capacity measurement, followed by discharge and charge pulses at 5C and 1C at every 10% SOC level.
# A Numerical Study of the Performance of Point Absorber Wave Energy Converters
## 3 Results and Discussion
### Effect of Wave Height  
The influence of the wave height on the power absorbed by the WECs is now investigated. A similar power absorption and efficiency of the WEC (\(\eta\)) expression is provided in [28]:  
\[\eta=\frac{P_{absolute}}{P_{wave}} \tag{4}\]  
\[P_{absolute}=\ \frac{1}{T}\int_{0}^{T}F_{z}\ x\ V_{z}\ dt \tag{5}\]  
\[P_{wave}=\rho g^{2}H^{2}T\frac{L}{32\pi} \tag{6}\]  
where \(P_{absolute}\) is the absorbed power estimated from the simulations, \(F_{z}\) is the instantaneous lift force, and \(V_{z}\) is the instantaneous heave velocity. \(P_{wave}\) is the incidence power, \(T\) in Equation (6) is the total time of the simulation, and \(L\) is the characteristic dimension of the WEC. The lift force (\(F_{L}\)) is dependent on the region where the vertical fluid velocity increases and lowers the pressure. It is calculated as follows:  
\[F_{L}=\frac{1}{2}\rho C_{L}A_{Z}v_{z}^{2} \tag{7}\]  
where \(C_{L}\) is the lift coefficient, \(v_{z}\) is the velocity of the SWEC relative to the fluid, \(A_{z}\) is the cross-sectional area perpendicular to the \(z\)-axis, and \(\rho\) is the fluid density.  
Figure 12a,b illustrate the absolute power absorbed and the efficiency for W1 for different wave heights (\(H\)) at \(d_{s0}\) with \(k_{2}\). Figure 12b shows that for W1, \(\eta\) decreases with an increase in wave height. This is attributed to the higher value of \(P_{wave}\) when \(H\) increases. It can be concluded that with an increase in \(H\), the power absorption increases. A similar  
Figure 11: **(a,b)** Vorticity contours of W1 and W3. **(c,d)** Dynamic pressure of W1 and W3 at \(H\) = 0.04 m, submergence depth \(d_{s}\) = 0.2 m, and \(k_{2}\) = 50 N/m.  
observation was made by [PERSON] et al. [16]. Figure 12c shows the absolute power absorption of W1 for the studied submergence depths. It can be noticed that the power absorption decreases with an increase in WEC depth for W1. Figure 12d,e show the effect of the WEC shape on power absorption. It can be seen from Figure 12f that the efficiency is higher for W3 than for W1 and W2 with an increase in depth, and the efficiency decreases with an increase in depth for W1. This is attributed to the power absorption, which is higher for W3 than for W1 and W2. However, as shown in Figure 12f, the efficiency of all three WECs seems to be identical at the \(d_{s0}\) submergence depth.
## 4 Experimental Results
### Results
#### 2.2.1 Adversarial Text Detection
This result demonstrates that the abstraction technique in PSO4 RNN ensures high detection accuracy. It has practical significance in mitigating adversarial text attack scenarios, helping to detect adversarial patterns efficiently. The model learning approach not only provides a way to understand how the RNN works but also has the potential to open the door to applying software analysis techniques (such as model-based testing, model checking, runtime monitoring, and validation) to real-world RNN models.
## 4 Discussion
### Influence of Environmental Factors  
#### 4.2.1 Temperature and Relative Humidity  
The positive correlation between tick abundance and temperature is widely recognized by most of the studies projecting climate change [26; 55; 72; 73; 74]. Our micro-scale data and regression results are consistent with these theories as higher temperature generally stimulates more question activities, while extremely low or high temperatures discourage such activities. Examining our raw field data, the surface temperature alone may not be a significant driving factor for the quantity of question black-legged ticks at the micro scale [70]. However, significant changes in temperature that vary with seasons may function as a threshold determining the states of the tick host-seeking activity being either \"occur\" or \"not occur\" [75; 76]. While a binary variable of season that differentiates the spring from the fall is statistically significant in one of our regression models, the temperature performs better in correlating with the quantity of question blacklegged ticks because season is just a broad and coarse non-numerical proxy.  
Our study also highlights the critical role that relative humidity plays in black-legged ticks' questioning activity, which is widely recognized in tick ecology [16]. At the local scale,  
Figure 8: Landscape of hardwood forest (CSP, **left**) and seashore (FINS, **right**). The two parks have drastically different land-cover types in the sampling sites. CSP has large areas covered by hardwood trees, whereas FINS is mainly covered by short shrubs in a very dry condition.  
relative humidity would outperform most meteorological variables (i.e., precipitation) in modeling tick queuing activities because it is a straightforward indicator for tick water stress in the microclimate [42]. Although all relative humidity data measured in this study are below the 82% threshold of black-legged tick survival suggested in [20], they were the microclimatic relative humidity experienced by questing black-legged ticks in the natural habitat. In addition, ticks are adaptive vectors, and they actively change questing behaviors (e.g., questing height) in response to water stress in their surroundings [70]. Interestingly, we also observed the nonlinearity of humidity. Extremely low and high humidity may present a challenge for tick question activity, which is also found in [47]. The fitted smooth curve suggested that higher moisture itself does increase the number of questing ticks but only when the relative humidity is above 40. As suggested by the generalized additive model, however, the relationship could be rather complex when considering other environmental factors.
# DQN-Based Shaped Reward Function Mold for UAV Emergency Communication
## 2 Materials and Methods
### Deep Q-Learning Network  
In the professional realm, research on deep reinforcement learning algorithms is primarily bifurcated into value function-based and policy function-based methodologies. The DQN algorithm falls under the category of value function-based deep reinforcement learning. By incorporating an experience replay mechanism, an experience pool is established to store training samples. Subsequently, a small batch of these samples is selected for iterative updating. This technique effectively mitigates the correlation between samples [10].  
At the same time, the target network and the evaluation network are separated to improve the stability of the algorithm. The DQN algorithm sets up two neural networks, namely a target network and action network. Firstly, the neural network is used to fit the Q value, and the F function is designed to reset the reward function according to the agent collision effect. The parameter value of the network is updated in real time, and it can be assigned to the target network after N iterations. By separating two networks, the stability and convergence of the algorithm can be improved [11]. Its optimal value function can be expressed as  
\[Q^{*}(s,a)=_{\Pi}^{max}E\Big{(}r_{t}+\gamma r_{t+1}+\gamma^{2}r_{t+2}+...\Big{|} s_{t}=s,a_{t}=a,\Pi\Big{)}. \tag{3}\]  
The basic DQN algorithm uses the \(\varepsilon\)-greedy strategy to balance exploration and utilization to improve the agent's exploration of the environment. The strategy can be expressed as  
\[\Pi^{*}=\left\{\begin{array}{ll}argmaxV^{\Pi}(s)&p=1-\varepsilon\\ rand(a)&p=\varepsilon\end{array}\right.. \tag{4}\]  
The agent randomly selects the action in the action space with the probability of \(\varepsilon\), and then it selects the known optimal action with the probability of \(1-\varepsilon\). The traditional DQN algorithm can solve the problem of dimensionality disaster when the Q-learning algorithm deals with a complex environment. However, the traditional DQN algorithm still cannot overcome the sparse reward problem, which requires a lot of extra training time.  
The following is the frame of DQN, as shown in Figure 1.
# An Efficient and Robust Topology Optimization Method for Thermoelastically Damped Microresonators
## 2 Statement of the Topology Optimization Problem
### State-Space Method for Large-Scale Microresonators with Phase-Lag Heat Conduction Law
With the help of the state-space method, for the free vibration considered herein, the solution is assumed to take the form:  
\[\mathbf{\Theta}=\mathbf{\tilde{\Theta}}e^{\lambda_{i}t},\mathbf{U}=\mathbf{ \tilde{U}}e^{\lambda_{i}t} \tag{14}\]  
where \(\lambda_{i}\) is the \(i\)-th eigenvalue of the thermoelastic system, \(\left[\mathbf{\tilde{\Theta}}\right]_{N_{n}\times 1}\) and \(\left[\mathbf{\tilde{U}}\right]_{2N_{n}\times 1}\) or \(\left[\mathbf{\tilde{U}}\right]_{3N_{n}\times 1}\) are the amplitudes of temperature and 2D/3D displacement, respectively. Then, Equation (13) can be expressed as  
\[(\lambda_{i}\mathbf{B}-\mathbf{A})\mathbf{\Phi}_{i}=\mathbf{0} \tag{15}\]  
where, for the 2D non-Fourier dual-phase lag heat conduction model, \(\left[\mathbf{\Phi}_{i}\right]_{6N_{n}\times 1}=\left[\mathbf{\ddot{\Theta}}, \mathbf{\dot{\Theta}},\mathbf{\dot{U}},\mathbf{\dot{U}}\right]^{\mathsf{T}}\), and in the case of 3D, \(\left[\mathbf{\Phi}_{i}\right]_{8N_{n}\times 1}=\left[\mathbf{\ddot{\Theta}}, \mathbf{\dot{\Theta}},\mathbf{\dot{U}},\mathbf{\dot{U}}\right]^{\mathsf{T}}\). Since the degrees of freedom of the system matrix reach \(6N_{n}\) and \(8N_{n}\) for the 2D and 3D thermoelastic problems, respectively, solving the system typically incurs heavy computational costs. Moreover, for large-scale design problems, the intermediate variables of the expanded system may exceed the storage capacity of computing devices. Once the thermoelastic system matrices are constructed, an optimal design method must be employed to solve the quality factor design problem.
# A Review of Computer Vision and Deep Learning Applications in Crop Growth Management
## 3 Deep Learning  
Deep learning (DL) is a subfield of machine learning inspired by the structure of neurons in the human brain. It automatically learns multi-level feature representations of data by simulating the neural network structure of the human brain. Deep learning uses multiple layers of nonlinear transformations to progressively abstract raw data into high-level feature representations, enabling it to effectively perform tasks such as classification, prediction, and generation. The ability to automatically extract high-level features from large datasets without relying on manually designed features has led to the increasing adoption of deep learning across various industries [60; 61]. DL involves several basic processes. First, data is propagated layer by layer from the input layer to the output layer to compute the prediction results (forward propagation). Then, based on the prediction error, the chain rule is used to adjust the network parameters (weights and biases) from the output layer backward (backpropagation). By iteratively optimizing the loss function, the prediction error is gradually reduced until convergence is achieved. The potential of DL has led to significant achievements in the agricultural field [62]. In [63], recurrent neural networks (RNNs) are used to process time-series data collected by sensors, proposing a deep learning-based IoT module to enable smart agriculture under various environmental conditions. Convolutional neural networks (CNNs) are a crucial component of deep learning, offering enhanced feature extraction capabilities as the number of layers increases, though this also leads to the issue of gradient explosion [64]. In [65], researchers introduced deep residual networks (ResNet) into the field of agricultural pest identification to address the issue of gradient explosion in deep layers. This approach is used for identifying agricultural pests in complex farmland environments. Deep learning emphasizes the importance of feature learning and has proposed numerous classic mechanisms, network structures, and models to learn features more efficiently and rapidly [66].