esa-sceva/satcom-chunk-collection huggingface dataset:
## 2 Related Works
### PCA-Based Approaches  
PCA [2; 3] is a classical dimensionality reduction technique and has been applied in various situations, such as data visualization and preprocessing. PCA aims to derive a linear projection onto a subspace that maximizes the variance of projected high-dimensional data. This formulation yields a closed-form expression of the embedding, improving the applicability and scalability of PCA. Probabilistic PCA (PPCA) [42] introduces probabilistic formulation into PCA. Specifically, PPCA defines latent variables in a low-dimensional space and derives the projection by maximizing the likelihood after marginalizing these latent variables. Although PCA and PPCA have produced successful results, they are both linear methods, and data generally contain nonlinear correlations. GP-LVMs [15] are a nonlinear extension of the PCA framework and realize nonlinear embedding with two differences: optimization of latent variables instead of projection and introduction of the kernel method [43] for nonlinear expression. These differences are achieved by the Gaussian-process-based formulation, and GP-LVMs assume that high-dimensional data are generated from the Gaussian process [44] input using latent variables. GP-LVMs can derive nonlinear latent variables according to the nonlinearity of the kernel, and several extensions introduce prior distributions, including discriminative [28; 32], dynamic [24; 25], and hierarchical GP-LVMs [45]. Recently, GP-LVMs have been developed for scalable training by inducing methods [46; 47; 48], for the Bayesian inference of latent variables [16; 23], and for deep modeling [49; 50]. Although GP-LVMs and PCA-based approaches preserve the global structures of data, they do not consider the local structures of data beneficial for visualization-aided dimensionality reduction. Furthermore, the kernel functions used in GP-LVMs are selected from homogeneous kernels, which have limited expression. In this study, we design a GP-LVM for visualization with a stationary kernel and local estimation of data.
# Social Media Opinion Analysis Model Based on Fusion of Text and Structural Features
## 4 Experimental Section
### Baseline Method  
A comparison of our proposed model with other text vectorization methods including RoBERTa, Ernie, BERT, CPT, GPT2, and so on. The description of these models is as follows.  
RoBERTa [34]: RoBERTa is an enhanced version of BERT, achieving better performance by improving training tasks as well as data generation methods, by training for a longer period of time, using larger batch sizes, and by using more data.  
Ernie [35]: ERNIE is a large scale knowledge enhanced model published by Baidu, covering NLP and cross-modal models. ERNIE performed further optimisations based on the BERT model and performed better on Chinese NLP tasks. The main improvement of this technique is the masking mechanism.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline  & **Toutiao** & **Toutiao\#1** & **Toutiao\#2** \\ \hline Number of news & 511 & 11 & 11 \\ Number of users & 71,579 & 3496 & 5940 \\ Number of comments & 103,787 & 5570 & 10,580 \\ Positive comments & 54,994 & 2224 & 3622 \\ Neutral comments & 23,236 & 1647 & 4095 \\ Negative comments & 25,557 & 1699 & 2863 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Toutiao and three subsets Dataset Statics.  
BERT [17]: BERT is a pre-trained language representation model that emphasizes not using traditional unidirectional language models or shallowly concatenating two unidirectional language models for pre-training, but by using a novel masked language model (MLM) to generate a bidirectional deep language representation.  
CPT [36]: The CPT model primarily makes changes to the structure of the encoding and decoding portions of the Transformer and adds three portions: the shared encoder (S-Enc), the decoder comprehension (U-Dec), and the decoder generation (G-Dec). It improves language comprehension and generation skills by using multi-task articulatory training.  
GPT2 [37]: The GPT2 model is a new pre-trained model published by the OpenAI organisation on the basis of the GPT model in 2018. Unlike the BERT model, GPT2 is built using the transformer decoder module, while BERT is built using the transformer encoder module. The auto-regression mechanism allows GPT2 to better capture information about the context content.  
GCN [29]: GCN was first proposed by [PERSON] et al. Each node in the network structure updates its own node state by exchanging information with one another, and uses convolutional computations to extract spatial features to learn the node representation, thus performing classification tasks.
# SumMER: Structural Summarization for RDF/S KGs
## 4 Constructing Schema Summaries
### Selecting the Top-k Nodes as a Regression Problem  
In this work, we argue that the various centralities offer complementary perspectives on assessing a node's importance. As such we move beyond approaches that try to identify a single optimal measure (e.g., [2,4]). In order to combine multiple measures for selecting the k most important schema nodes we model the problem as a regression problem. The features of each schema node are the various centrality measures. In addition, we also add in the feature vector the normalized number of instances belonging to each schema node, as we believe that the more populated with instances a schema node is the most important should be. We explore eight ML regressor algorithms to this purpose: Adaboost (AB), Gradient Boosting (GB), Extreme Random Forest (ERF), Random Forest (RF), Decision Tree (DT), Bayesian Ridge (BR) and ElasticNet (EN).  
**Example 2**.: _For our running example, the following table (Table 2) is constructed, including the features that will be given as input to the machine learning algorithms._  
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Class** & **BE** & **DE** & \(\ldots\) & **HT** & **PR** & **RA** & **Instances** \\ \hline [[http://dbpedia.org/ontology/Work](http://dbpedia.org/ontology/Work)]([http://dbpedia.org/ontology/Work](http://dbpedia.org/ontology/Work)) & 0.73 & 1.18 & \(\ldots\) & 1.26 & 0.50 & 0.35 & 0.96 \\ \hline [[http://dbpedia.org/ontology/Film](http://dbpedia.org/ontology/Film)]([http://dbpedia.org/ontology/Film](http://dbpedia.org/ontology/Film)) & 0.18 & 0.7 & \(\ldots\) & 0.83 & 0.12 & 0.07 & 0.12 \\ \hline \(\ldots\) & \(\ldots\) & \(\ldots\) & \(\ldots\) & \(\ldots\) & \(\ldots\) & \(\ldots\) & \(\ldots\) \\ \hline [[http://dbpedia.org/ontology/Settlement](http://dbpedia.org/ontology/Settlement)]([http://dbpedia.org/ontology/Settlement](http://dbpedia.org/ontology/Settlement)) & 0.75 & 1.24 & \(\ldots\) & 1.29 & 0.78 & 0.36 & 0.90 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Example Input Data.  
All algorithms are quite efficient, in their test phase. The complexities of the algorithms are shown in Table 3[16; 17]. Note that complexities depend on many factors, such as number of features, samples, number of trees, depth of the trees (e.g., for Random Forest), etc. Note also that Bayesian Ridge regressor and Elastic Net regressor are Linear regressors as well.
# Explainable MRI-Based Ensemble Learnable Architecture for Alzheimer's Disease Detection
## 4 Results and Discussion
### Comparative Analysis of Ensemble Explainability Techniques
#### 4.6.1. LIME Results Comparison  
_Secondly, the LIME visualizations_ in Figure 15 provide additional insights into the ensemble techniques capabilities to highlight areas linked to brain atrophy.  
* **Binary Feature Attribution (Green vs. Red):** Unlike Grad-CAM's continuous spectrum, LIME provides binary classification of regions as either positively contributing (green) or negatively contributing (red) to the diagnosis. In AD cases, both ensembles show green regions in areas associated with known pathology, but stacking demonstrates more precise positive contribution boundaries, particularly in medial temporal structures. This binary delineation offers clearer interpretability of which specific regions influence diagnostic decisions, supporting stacking's higher precision (98.0%) reported in Table 2.
* **Color Distribution in CN vs. AD:** LIME visualizations reveal a diagnostic color inversion between CN and AD cases. CN cases predominantly display red coloration in medial temporal regions, indicating these areas negatively contribute to an AD diagnosis when healthy. Conversely, AD cases show green coloration in these same regions, reflecting positive contribution to diagnosis. This color-based contrastive explanation provides intuitive interpretability that aligns with the clinical understanding of Alzheimer's progression.
* **MCI Contribution Patterns:** MCI cases reveal a distinctive mixed pattern of green and red regions. Stacking demonstrates more distributed green areas across cortical regions compared to hard voting, suggesting better sensitivity to the subtle, widespread changes in early neurodegeneration. This distribution difference correlates with the overlap coefficient metrics in Table 9, where higher values indicate better sensitivity  
Figure 14. Grad-CAM visualization for (**a**) hard voting and (**b**) stacking ensemble classifiers.  
to early biomarkers. The balanced color distribution in stacking's LIME visualizations provides a potential explanation for its superior early detection capabilities.
# Robot Delay-Tolerant Sensor Network for Overhead Transmission Line Monitoring
## 3 RDTSN Model and Data Delivery Strategy  
This section introduces a new type of Delay-Tolerant Sensor Network (DTSN) model and proposes a novel data delivery strategy. In order to make RDTSN work more efficiently, this research analyzes the robot's movement along the ground wire and summarizes two types of movement models. Furthermore, the Discrete-Time DTSN mathematical model was established based on the knowledge of graph theory and the characteristics of RDTSN. Finally, the message forwarding algorithm (DTMA) is put forward for practical application.
## 7 Rationale for ka-band selection
### Technical aspects  
Selecting the Ka-band allows a larger bandwidth (500 MHz), which provides a vertical resolution of 0.3 m instead of 0.5 m in Ku-band. With such a resolution, the altimeter is close to a beam limited one: there is no 'plateau' in the echo, since it strongly attenuates shortly after the leading edge, due to the small antenna aperture. This will greatly reduce the pollution of 'land gates' into 'ocean gates', when considering land-sea transition areas.  
The shorter decorrelation time of sea echoes at Ka-band also enables to double the number of independent echoes per second compared with Ku-band altimeters. The instrument is thus designed for a high Pulse Repetition Frequency (PRF) around 4000 Hz, adjustable along the orbit.  
Moreover, due to the smaller antenna beamwidth, the Brown echo has a sharper shape in Ka-band than what is obtained with conventional Ku-band altimeters. This point is illustrated on Figure 6. The antenna footprint is thus reduced: at 800 km altitude, the 3-dB footprint radius is about 4 km versus 15 km for Poseidon 2 (Jason-1). This also contributes to improve spatial resolution versus classical Ku-band altimetry and to discriminate types of surface in transition zones (coastal areas, sea ice boundaries, and so on).  
All these above aspects, associated with a high performance on-board algorithm, entails a higher expected performance using altimetry in Ka-band.  
Figure 6: Waveform shapes in Ka- and Ku-bands for a 2 m significant waveheight (SWH). (Horizontal axis: Time; Vertical axis: Return Power)
# Absolute Distance Measurement Based on Self-Mixing Interferometry Using Compressed Sensing
## 2 Theoretical Analysis
### Compressed Sensing  
A signal \(\mathbf{x}\in\mathds{R}^{N}\) can be expressed as a weighted sum of the columns of an orthonormal basis matrix \(\mathbf{\
abla}\in\mathds{R}^{N\times N}\), via  
\[\mathbf{x}=\mathbf{\
abla}\mathbf{s}, \tag{6}\]  
where \(\mathbf{s}\in\mathds{R}^{N}\) is the weight coefficient. This signal can be called \(K\)-sparse on a sparse basis \(\mathbf{\
abla}\) if there are no more than \(K\) (\(K<N\)) nonzero elements in \(\mathbf{s}\). The sparse basis can be Fourier basis, wavelet basis, cosine basis, etc. A suitable sparse basis should be selected to minimize the number of nonzero elements to obtain the best sparse representation. In CS theory, a sparse signal can be sampled by a measurement matrix \(\mathbf{\Phi}\in\mathds{R}^{M\times N}\) where \(M<N\) to obtain the signal \(\mathbf{y}\in\mathds{R}^{M}\). The sampling process can be denoted with  
\[\mathbf{y}=\mathbf{\Phi}\mathbf{x}=\mathbf{\Phi}\mathbf{\
abla}\mathbf{s}= \Theta\mathbf{s}, \tag{7}\]  
where matrix \(\mathbf{\Theta}\in\mathds{R}^{M\times N}\) denotes the sensing matrix. The matrix \(\mathbf{\Theta}\) needs to satisfy the Restricted Isometry Property (RIP) [22] so that the sampled vector \(\mathbf{y}\) can preserve sufficient information to fully recover the original signal \(\mathbf{x}\). It has been proven that the sensing matrix \(\mathbf{\Theta}\) can satisfy the RIP when the measurement matrix \(\mathbf{\Phi}\) is a Gaussian matrix, Bernoulli matrix, Fourier random matrix, or Hadamar matrix [23].  
Signal reconstruction will be conducted after measurement and is formulated as an \(\ell_{0}\) norm minimization problem. The unknown \(K\)-sparse vector can be reconstructed via  
\[\min_{s}\lVert\mathbf{s}\rVert_{0}\text{ s.t. }\mathbf{y}=\mathbf{\Phi}\mathbf{ \
abla}\mathbf{s}, \tag{8}\]  
where \(\lVert\bullet\rVert_{p}=\left(\sum_{1}^{m}\lVert\bullet\rVert\right)^{1/p}\) denotes the \(\ell_{p}\)-norm. The \(\ell_{0}\)-minimization is NP-hard, in general. There are usually two methods for resolving this issue. The first is \(\ell_{1}\)-minimization and the second is greedy pursuit algorithms. It has been demonstrated that \(\ell_{1}\)-minimization can replace \(\ell_{0}\)-minimization when the sensing matrix \(\mathbf{\Theta}\) satisfies the RIP [24]. The main advantage of \(\ell_{1}\)-minimization is that it is a convex optimization problem, and therefore, liner programming technologies can be used to solve the function. However, \(\ell_{1}\)-minimization has high computational complexity. In comparison, greedy pursuit algorithms have a favorable computational profile. The main idea of greedy pursuit is to select the most valuable columns, called atoms, from the sensing matrix and then gradually reconstruct the signal. Orthogonal matching pursuit (OMP) [25] is a widely used greedy pursuit algorithm.
# Secure and Efficient Federated Gradient Boosting Decision Trees
## 2 Related Work  
This paper mainly studies horizontal FL schemes for GBDTs. Each party has its own data, and these data have the same features. The existing horizontal FL schemes for GBDTs has some limitations.  
[PERSON] et al. [33] designed a distributed GBDTs scheme (TFL), in which all parties take turns to train a differential privacy decision tree. This method only uses the local datasets of data participants to update the tree, which leads to low model accuracy. [PERSON] et al. [34] put forward a federated GBDTs model (eFL-Boost) which focuses on properly allocating local computing (executed by each data owner alone) and global computing (executed by all owners in cooperation). Although it can improve the prediction performance to a certain extent, the accuracy of the model is greatly affected by non-iid datasets. [PERSON] et al. [35] proposed a GBDTs of private FL (FederBoost) to construct accurate trees through gradient histograms. However, this method brings problems such as information leakage. [PERSON] et al. [36] put forward SimFL, which uses a weighted gradient boosting method to model a single tree, which is a new direction of GBDTs algorithm under the FL framework. Unfortunately, when training large-scale datasets, the communication overhead of each iteration is very large. Therefore, the existing horizontal FL schemes of GBDTs has some problems, such as insufficient prediction accuracy, model accuracy greatly affected by non-iid datasets, information leakage and high communication cost.  
To sum up, this paper proposes a Secure and efficient horizontal FL schemes for GBDTs (SeFB). The main contributions of this paper are as follows: (1) Aiming at the influence of non-iid datasets on the model accuracy, this paper uses Locality Sensitive Hash (LSH) in the GBDTs of FL, and uses coordinator to count the similar instances of participants, fully absorbing the gradient information of each participant, and measuring the importance of the gradient of each participant to updating the tree model, so as to stimulate and motivate participants with high-quality datasets to make greater contributions. (2) Aiming at the existing GBDTs schemes under secure training in federated environment has the problems of complicated calculation and high communication cost. This paper mainly adopts the master-slave architecture, and the high-performance server is responsible for the global complex operations, such as calculating the weighted gradients and global leaf weights. Participants are responsible for simple operations, such as calculating hash values, building trees and updating models. Reduce the calculation burden of participants, and at the same time reduce the number of communications. (3) Experimental analysis shows that the algorithm can protect the privacy of the original data, and the communication cost is low. At the same time, the performance of imbalanced binary data sets is evaluated (ROC AUC and F1-score as indicators). The results show that our SeFB algorithm has higher prediction accuracy than the current horizontal federated GBDTs scheme.
## 3 The Proposed Model
### 1d Cnn  
The role of the convolution process is to extract features from the input text. Convolutional layers are used to extract low-level semantic features from the original text and to reduce the number of dimensions as opposed to Bi-LSTM, which process the text as sequential data. In this study, we use several 1D convolution kernels to perform convolution over the input vectors.  
Equation (1) defines a vector of sequential text obtained by concatenating embedding vectors of the component words:  
\[X_{1:T}=[x_{1},x_{2},x_{3},x_{4},\cdots x_{T}], \tag{1}\]  
where \(T\) is the number of tokens in the text. To capture the inherent features using a 1D CNN, convolution kernels with varying sizes are applied to \(X_{1:T}\) to capture the unigram, bigram, and trigram features of the text. During the \(t\)th convolution, when a window of \(d\) words that stretches from \(t:t+d\) is taken as an input, the convolution process generates features for that window as follows:  
\[h_{d,t}=\tan\hh(W_{d}x_{t+d-1}+b_{d}), \tag{2}\]  
where \(x_{t+d-1}\) are the embedding vectors of the words in the window, \(W_{d}\) is the learnable weights matrix, and \(b_{d}\) is the bias. Since each filter must be applied to various regions of the text, the feature map of the filter with convolution size \(d\) is:  
\[h_{d}=[h_{d1},h_{d2},h_{d3},h_{44},\cdots x_{T-d+1}]. \tag{3}\]  
The key advantage of using convolution kernels with diverse widths is that the hidden correlations between several adjacent words can be captured. The most important aspect of using a CNN for textual feature extraction is reducing the number of trainable parameters during feature learning, which is  
Figure 4: The proposed model architecture.  
achieved using max-overtime pooling [18]. The input is acted upon by numerous convolution channels, and each distinct channel consists of values on diverse timesteps. Hence, during max-overtime pooling, the output from each convolution channel will be the largest value of all timesteps in that channel.  
For each convolution kernel, we apply max-overtime pooling to the feature maps with convolution size d to obtain  
\[p_{d}=Max^{t}(h_{d1},h_{d2},h_{d3},h_{d4},\cdots x_{T-d+1}), \tag{4}\]  
To obtain the final feature map of the window, we concatenate \(p_{d}\) for each filter size d = 1, 2, 3 and extract the unigram, bigram, and trigram hidden features:  
\[h_{d}=[p_{1},p_{2},p_{3}]. \tag{5}\]  
The advantage of using the CNN over the LSTM is that it reduces the number of dimensions in the input features to be fed to a sentiment classifier or a natural inference prediction model that must follow the feature extraction step.
# Research on Price Prediction of Construction Materials Based on VMD-SSA-LSTM
## 4 Experimental Data Processing and Analysis
### Data Pre-Processing  
The collected and organized dataset may contain missing values due to human errors or force majeure factors. Additionally, missing data can have a certain impact on the prediction accuracy of the model. Therefore, addressing the missing values in the dataset helps improve data quality, thereby enhancing the model's predictive accuracy. The specific steps for data preprocessing are as follows.  
1. Missing data  
Missing data can interfere with statistical calculations and model training, increasing errors, and potentially leading to overfitting or underfitting, thereby affecting prediction accuracy. In addition, missing data result in incomplete data, reducing data quality, which in turn affects the accuracy of analysis results and further weakens the model's performance. To improve data quality and minimize the impact of missing data on the prediction model, the missing data should be filled using the average of the non-missing values from the previous and subsequent periods, based on the characteristics of the experimental data. The calculation formula is as follows Equation (19):  
\[D(x,t)=\frac{D(x-1,t)+D(x+1,t)}{2} \tag{19}\]  
where \(D(x,t)\) represents the filled data value, \(x\) is the missing data, \(t\) is the corresponding collection point of the missing data, \(D(x-1,t)\) are the data at the same time as the previous period of the data point, and \(D(x+1,t)\) represent the data at the same time as the next period of the data point.  
2. Normalization data  
Normalization, also known as data standardization, is the removal of magnitudes from data. At the same time, normalization helps the data distribution align more closely with the assumptions of the algorithm, speeds up model training, and improves the model's accuracy and stability. The method used in this paper is Min-Max Scaling. The specific calculation formula is given as Equation (20).  
\[x_{i}=\frac{x-x_{\min}}{x_{\max}-x_{\min}} \tag{20}\]  
where \(x_{\min}\) and \(x_{\max}\) are the minimum and maximum values of the dataset, respectively.