esa-sceva/satcom-chunk-collection huggingface dataset:
## 3 Message Format
### Summary  
The related work surveyed above extracts the message structures used in a protocol. Some of them such as [PERSON] et al. [12; 21; 23], [PERSON] et al. [19] and [PERSON] et al. [24] cluster the messages based on their type in one cluster, and keep one common message as the template for the type. Then, they try to find the most important and dynamic parts of the messages that need to be changed in a new message using iterative clustering and/or statistical analysis such as frequency/entropy, TF/IDF, server's behaviour such as executed code fragments, library, system calls, output and file system activity, etc.  
Others such as [PERSON] et al. [25], [PERSON] et al. [26], [PERSON] et al. [28], [PERSON] et al. [32], [PERSON] et al. [34] and [PERSON] et al. [30; 31] go further and try to infer the whole structure of the messages, field by field, using iterative clustering and statistical analysis of the messages to find the structures for each cluster. In addition, [PERSON] et al. [32] and [PERSON] et al. [34] infer one FSM for each cluster. The existing binary message format extraction methods rarely extract the hierarchical structure of messages where the dependencies between different fields in a message are identified. In addition, some of them are designed to extract the structure of messages that have fixed-length fields and cannot work on protocols with the variable length fields or complex structures.
# A Review on the Performance of Linear and Mixed Integer Two-Stage Stochastic Programming Software
## 4 Benders Decomposition Enhancement Strategies
### Reducing the Cost of Each Iteration  
Cheaper iterations are achieved by reducing the time spent solving the MP and subproblems. The MP is often the most time-consuming part of the BD algorithm (more than 90% of the execution time) in the case of mixed-integer problems [27]. The overall process can be accelerated by relaxing the integrality of the MP in most of the iterations to rapidly compute a large number of cuts [32]. A variation of this method was proposed by [PERSON] and [PERSON] [28], in which the MP is solved to a non-zero optimality gap. The integrality gap is continuously reduced to ensure global convergence.  
Alternatively, the MP might be solved via (meta) heuristics [33; 34], which provide good approximate solutions in short time; however, it is still required to solve the MP to optimality to guarantee convergence. The application of heuristics or the LP relaxation of the MP often yields worse bounds and lack of controllability, reducing the ability of BD to generate the necessary cuts [35].  
Similarly, suboptimal solutions of the dual subproblems yield valid cuts, known as inexact cuts. Algorithm convergence can still be guaranteed under the conditions described by [PERSON] et al. [36]. Additional subproblem acceleration schemes comprise synchronous parallelization and re-optimization. The latter exploits structural similarities between scenarios to solve the subproblems in fewer solver iterations.
###### Abstract  
Visible and near-infrared (Vis-NIR) spectroscopy enables the rapid prediction of soil properties but faces three limitations with conventional machine learning: information loss and overfitting from high-dimensional spectral features; inadequate modeling of nonlinear soil-spectra relationships; and failure to integrate multi-scale spatial features. To address these challenges, we propose ReSE-AP Net, a multi-scale attention residual network with spatial pyramid pooling. Built on convolutional residual blocks, the model incorporates a squeeze-and-excitation channel attention mechanism to recalibrate feature weights and an atrous spatial pyramid pooling (ASPP) module to extract multi-resolution spectral features. This architecture synergistically represents weak absorption peaks (400-1000 nm) and broad spectral bands (1000-2500 nm), overcoming single-scale modeling limitations. Validation on the LUCAS2009 dataset demonstrated that ReSE-AP Net outperformed conventional machine learning by improving the R\({}^{2}\) by 2.8-36.5% and reducing the RMSE by 14.2-69.2%. Compared with existing deep learning methods, it increased the R\({}^{2}\) by 0.4-25.5% for clay, silt, sand, organic carbon, calcium carbonate, and phosphorus predictions, and decreased the RMSE by 0.7-39.0%. Our contributions include statistical analysis of LUCAS2009 spectra, identification of conventional method limitations, development of the ReSE-AP Net model, ablation studies, and comprehensive comparisons with alternative approaches.  
a Article
Further research and validation are needed to assess the model's performance, generalizability across diverse accident scenarios, and its interpretability in the context of safety-critical applications. Additionally, addressing domain-specific challenges and tailoring the model to the unique characteristics of construction-related text data via the fine-tuning method is crucial for maximizing the effectiveness of GPT-based approaches in accident classification within the construction industry. In this regard, the authors developed a model to predict construction accidents and visualize accident precursors using a fine-tuned GPT model from raw construction accident reports. The contributions of this paper are as follows: (1) The authors developed an approach to automatically classify construction accidents from a dataset of raw construction accident reports using a fine-tuned GPT with various hyperparameter optimizations (e.g., max token and temperature parameters). Such information holds unique value, as it can enhance the ability to understand, predict, and prevent accidents; (2) This paper proposes a method to identify word patterns that, following the fine-tuning process, consistently demonstrate high predictiveness for each safety outcome; (3) The suggested methods can also serve to visualize and comprehend the models' prediction sensitivity; and (4) The authors experimented with four state-of-the-art machine learning [Term Frequency-Inverse Document Frequency (TF-IDF)], deep learning [Convolutional Neural Network (CNN)], and two LLMs [fine-tuned GPT 2.0/3.0 and Korean Bidirectional Encoder Representations from Transformers (BERT)] using structured and unstructured text from construction accident reports. This paper is structured as follows to demonstrate the aforementioned contributions. First, the authors offer a background regarding the utilization of machine learning, deep learning, and LLMs in NLP in Section 2. Subsequently, in Section 3, the authors outline the structure of the proposed model, introducing the saliency visualization of accident attributes in unstructured free-text data. In Section 4, the authors test the developed method with the actual construction accident dataset and the preprocessing procedures applied and elaborate on the experimental configuration alongside comparative benchmark models, and report the results. Then, Section 5 includes a comprehensive analysis and interpretation of the findings. Lastly, Section 6 provides a summary and conclusion of the study's key contributions. The findings of this paper underscore the improved generalization performance of construction accident prediction models based on large language processing, leading to a proactive approach in preventing accidents at construction sites.
## 3 Review of Methods
### Convolutional Autoencoders  
When useful features of images form the basis of an application, _Convolutional Autoencoders_ (_CAEs_) are better suited than SAEs due to their capability of retaining spatial information. Their high-level structure is equal to that of a simple autoencoder, namely the sequence of an encoder and a decoder, but in this case, both parts contain CNNs (see Figure 9).  
[PERSON] and [PERSON] [56] pursued an augmented approach, where a Convolutional Autoencoder was used for a classification task using local feature maps for the convolutional part and the whole signature for the multi-head attention (MHA) part. MHA is an aggregation of single-attention heads, where each head is a function of three parameters: query, key, and value. The dataset was established using a 6 GHz Doppler radar to collect data from five subjects based on coarse-grained activities (_falling_, _bending_, _sitting_, _and walking_[56]), where each activity was repeated six times. The study was carried out for different training and test split sizes. From the results, it was observed that the attention-based CAE required less data for training compared to the standard CAE with up to three layers, achieving an accuracy of 91.1% for the multi-head attention using a multi-filter approach.  
Figure 8: Example of a Stacked Autoencoder.  
A comprehensive overview of key articles with regard to the radar technology domain, data, classification method, and achieved results is provided in Table A1, which can be found in the Appendix A.
## 5 Discussion
### Visualization
#### 5.2.2 Visualization of Attention Weights  
In stage one, we obtained a 64-dimensional visual feature vector, a 64-dimensional acoustic feature vector, and a 128-dimensional textual feature vector. In stage two, we concatenated these three vectors into a 256-dimensional multimodal vector and fed it into the fusion network. To explore the differences in fusion methods, we visualized the attention weights of three fusion strategies: multi-head attention (see Figure 7a), Transformer (see Figure 7b), and Conformer (see Figure 7c). In Figure 7a-c, we constructed an attention weight matrix for the aforementioned 256-dimensional multimodal feature vectors. The horizontal axis positions correspond to visual, acoustic, and textual feature components from left to right, and the vertical axis positions correspond to them from top to bottom. The numerical values (different colors) of each position in the matrix represent the degree of correlation of the corresponding positions, and the diagonal represents autocorrelation. The higher the value, the brighter the color. The weights of different heads were averaged.  
In Figure 7a, we observed a few scattered regions with relatively high weights in the distribution, indicating that the multi-head attention fusion model successfully identified important emotional features. In Figure 7b, the weights were predominantly higher along the main diagonal, suggesting that the Transformer model discovered significant intra-modal relationships. Although there were a few high weight regions near the bottom left corner, their weights were lower compared to those along the main diagonal. This implies that the model had less proficiency in fusing inter-modal information and primarily focused on intra-modal relationships. Moving on to Figure 7c, we notice that the main diagonal had a wider area of prominence compared to Figure 7b. Additionally, some vertical highlight regions appeared at the bottom left of the main diagonal. These observations indicate that the Conformer model, benefiting from its convolutional kernel, assigns higher weights to intra-modal relationships and exhibits greater confidence in inter-modal relationships. These findings align with our expectations and suggest that the Conformer network serves as an effective fusion model.  
Figure 7: Attention weights of three fusion methods: (**a**) for “multi-head attention”, (**b**) for “Transformer”, and (**c**) for “Conformer”. The lighter the color, the higher the weight.
## 4 Discussion  
### Data Analysis of \"Rough-Smooth\" Perception Words and Material Surface Roughness  
In SPSS 27.0 software, the mean values of the perceptual scores for the perception words \"rough-smooth\" were correlated with the corresponding surface roughness of the material, as shown in Table 9, resulting in a [PERSON]'s correlation of \(r=-0.777\). The two perception words are highly correlated, with a [PERSON]'s correlation value \(|r|\) of 0.70-0.89 [26]. The results showed a significant correlation between the \"rough-smooth\" perceptual scores and the subject material.  
Figure 5 shows a polynomial curve fitting of the \"rough-smooth\" perception words, with an R\({}^{2}\) of 0.6166 to the material's surface roughness.  
According to the fitting data, when the roughness of the material's surface was 0.459 \(\upmu\)m, the children's feeling of \"roughness\" was more prominent.  
The data show that the surface roughness of P1 (plastic PP) was 0.331 \(\upmu\)m and the surface roughness of P2 (plastic PP) was 0.459 \(\upmu\)m. Objectively, these two materials are made of the same polymer with different surface processes, and children's perception of P1 (plastic PP) tended to be more for smooth than P2 (Plastic PP).  
The data also show that the 10- to 16-year-old children were more sensitive to the perception of roughness of material surfaces. They still had high sensitivity to the perception of material variability of surfaces resulting from different surface treatment processes for the same material. This is consistent with the conclusion of [PERSON] et al. (2012), who explored the surface of a material by touch [27].  
Figure 4: The surface gloss of different materials.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline  & & **Rough–Smooth** & **Roughness** \\ \hline \multirow{3}{*}{**Rough–Smooth**} & Pearson correlation & 1 & \(-0.777\) * \\  & Significance (2-tailed) & & 0.014 \\  & Number of cases & 9 & 9 \\  & Pearson correlation & \(-0.777\) * & 1 \\
**Roughness** & Significance (2-tailed) & 0.014 & \\  & Number of cases & 9 & 9 \\ \hline \hline \end{tabular}  
* Correlation is significant at the 0.05 level (2-tailed).  
\end{table}
Table 9: Correlation analysis between “rough–smooth” and material surface roughness.
## 3 Materials and Methods
### Baselines
#### 3.3.3 Pre-Trained 3D-CNN  
* 3D ResNet: The 3D version of ResNet-34 [19] was chosen as one of the baselines in this paper. The input size, optimizer, and learning rate value were the same as that used in the previous approaches. The network was built with squeeze-and-excitation[38] layers, which improves the network resources with small computation overhead but without increasing the network depth.
* C3D: It is a generic 3D-CNN proposed in [21] to learn video features by modeling temporal information using pre-trained 3D convolution and 3D pooling operations. It has 8 convolution layers (3D) and 5 pooling layers. The C3D network was originally designed for action recognition; however, it is also very effective in other tasks such as anomaly detection in surveillance videos [34]. In this work, the same hyperparameters employed for the previous methods were used with this model: input size = 224 \(\times\) 224; learning rate = \(10^{-4}\); and Adam as the optimizer.
* MoViNet-A2: It is a member of a family of computation- and memory-efficient 3D CNNs (from A0 to A5) recently proposed in [20]. The three first models (MoViNets A0, A1, and A2) are lightweight methods that can be used on mobile devices. The model employed in this work is the MoviNet-A2, since the input dimension used  
Figure 5: Architecture of the customized 3D-CNN designed to detect anomalies in video data.  
with the previous baseline models matches its training resolution (224 \(\times\) 224). Even focusing on a trade-off between efficiency and accuracy, it is expected that MoviNets-A2 achieves lower accuracy when compared to the other supervised CNN investigated in this paper.
## 2 Problem Description
### Problem Description  
In Section 2.1, a preliminary model has been established for measuring signals of the ADS. The model is primarily derived based on the inertial navigation system measurements and the missile's kinematic relationships. In practice, there is a wide variety of air data systems, including pitot tubes, total temperature probes, angle of attack/sideslip angle sensors, and other components. These systems measure corresponding information based on their unique operating principles and input it into the air data computer, which calculates a series of atmospheric parameters for use in control and guidance processes. Among these parameters, airspeed, angle of attack, and sideslip angle are crucial for missile control. However, due to the different mechanism of sensors and approximation equations  
Figure 2: Diagram of equation derivation.  
employed by the air data computer, the estimation process is prone to deviations and may vary in different environments, thus posing challenges for fault diagnosis.  
Currently, the research on missile fault diagnosis is mainly focused on actuators, and studies on sensors mostly target integrated navigation systems typical of INSs/GPSs, and relatively less on ADSs. Specifically speaking, there are the following problems regarding fault diagnosis methods for ADSs:  
1. Compared to integrated navigation systems, like INS/GPS, the ADS model is difficult to describe analytically, and conventional modeling methods cannot accurately capture the characteristics of the system. Furthermore, due to the high speed and high maneuverability of missiles, there are many uncertainties, like unstable vortices during flight, which reduce the reliability of hypothesis testing-based methods in the diagnostic stage.
2. The sensors of an ADS are located on the surface of the missile and confronted with harsh environmental conditions, which exacerbates the interference and makes it difficult to diagnose minor faults. Additionally, missiles operate in uncertain conditions during different combat missions, such as multiple trajectories. The data from various flight states often exhibit significant variations. Conventional data-based fault diagnosis strategies may not be able to differentiate between changes in atmospheric parameters caused by variations in flight states and those caused by other factors, like faults, and obtaining data under all possible scenarios in advance is impractical, which limits their applicability to ADSs.  
According to the analysis above, it can be observed that fully data-based fault diagnosis methods have low applicability in ADSs, while model-based methods lack precision in diagnosis. Therefore, in cases where models can be established, a model-based approach combined with data, especially neural networks, is employed to improve the diagnostic results.
## 3 Results and Discussion  
In the first step, the challenge was to understand the initial WQMPs and to integrate the data into the database EnkiTM connected to the IDSS.  
The fact that the data were already entered in the prototype of EnkiTM facilitated the process of integrating the data from W1. However, the purpose of the questions was to understand the rationale behind the selection of the lake sampling sites, the decisions taken on the sampling strategy at each site, as well as understand the sampling objectives which contributed greatly to documenting the rationale and making it transferable to WQMP managers and organizations that would like to increase the value of the data for other studies. This process also contributed to documenting the limitations of some sampling sites, such as shoreline sampling sites. This information is also crucial to optimize the network, as some of the deficits of the sampling site distribution might already be known and can be justified.  
The data integration for W2 was more of a challenge, as the data had to be validated and completed from various sources. The choice of the sampling sites, their specific justification, especially for river sections, was not well documented at the outset and the information was retrieved from old paperwork. Also, changes in sampling frequency and problems in the data sets were a challenge for both the optimization of the data management and the application of the optimization approach. However, the onboarding (settings) steps and the retroactive integration of field observations, specifications on laboratories, sampling site justifications, etc. greatly contributed to obtaining a holistic view of the WQMP, as well as the specific rationale of decisions taken regarding the W2 WQMP. Data and metadata could now be efficiently consulted, visualized and retrieved. This contributed greatly to the choice of optimization methods: not only must they be based on specific optimization objectives (elicited by a participative approach), but also on the available data sets.  
The data integration steps showed that the data management question is not always clearly addressed at the outset of a WQMP. This may be due to the fact that challenges related to data management are underestimated and are only addressed when the data needs to be analyzed. In addition, for lack of resources, solutions, and knowledge on data management questions, data management is not always addressed adequately. For the W1 watershed, this question had been addressed at the outset which led to a data management strategy which was adapted to the data management needs of a WQMP. W2 watershed managers have adopted the same strategy as W1 after this experience.
## 4 System Model and Assumptions  
Our proposed mobility schemes are based on the following network assumptions:  
Figure 1: Possible mobility scenarios.  
1. A PAN consists of several SNs, which are Full Functional Devices (FFDs) and a PAN coordinator, i.e., the GW. Multiple GWs/PANs can also be considered to improve the reliability and scalability. However, in this paper, we limit our scope for a network having single PAN coordinator.
2. The nodes at the periphery of the PAN are known as Border Nodes (BNs). These are FFDs which are mostly in quasi-sleep state. In quasi-sleep state, the node's sensor is turned on, whereas its transceiver is turned off.
3. A MN is an FFD or RFD which moves within a PAN. A MN is assumed to transmit or receive data packets periodically, i.e., if a MN does not transmit or receive data packet within MST (Maximum Sleep Time), it is assumed to be dead.
4. The SNs spend most of their time in the sleep state.
5. A MN is an active device which moves within a PAN. It can be a FFD or a RFD.
6. The SNs are densely deployed to an extent that their transmission ranges overlap.
7. Each 6 loWPAN device has the same transmission signal strength and receiver sensitivity.
8. Each SN measures the distance between itself and the MN based on the Received Signal Strength Indication (RSSI) [18].
9. Each SN is equipped with a radio-triggered hardware component that activates sensors from/to the sleep state, by sending a special wake up radio signal [19].
10. 6 LoWPAN networks provide two types of addresses, IEEE EUI 64-bits extended address, and 16-bits short addresses. The SNs are assigned with IPv6 and 16 bits short address [20].
11. The MN is assigned with a 16 bits short address, which is unique within a PAN, and remains fixed irrespective of its location within the PAN. The IP address of the MN also remains fixed irrespective of its mobility within the PAN.
12. The interoperability between IPv6 domain and the IEEE802.15.4 device is handled by the adaptation layer [20].
13. Each SN is equipped with an antenna array in order to obtain the Angle of Arrival (AoA) measurements.
14. Any of the MAC protocols, synchronous or asynchronous, can be used to ensure the reliability and routing of packets in duty cycle WSN.
# Resource Allocation in Uplink NOMA-IoT Based UAV for URLLC Applications
###### Abstract
* Unlike the previous works, the novelty of the presented algorithm lies in its ability to consider both the strict delay requirements of IoT devices and the system throughput while ensuring high reliability and fairness, where simulations are performed to evaluate the proposed scheduling algorithm performance. The results demonstrate the effectiveness of the proposed algorithm to serve URLLC traffic with restricted delay limits, due to the significant enhancement in delay, reliability and fairness, in addition to maximizing the sum data rate and spectral efficiency while achieving the same system complexity when compared to the maximum channel quality indicator (max CQI) algorithm.