esa-sceva/satcom-chunk-collection huggingface dataset:
## 3 Materials and Methods
### Transformer Encoder  
The Transformer Encoder structure is the key of the multi-modal _Rclub. f._ quality grading model, and its construction was shown in Figure 1a. Self-attention is the core of the Transformer Encoder layer, which enables the model to dynamically weigh the importance of different input features. The Transformer Encoder consists of a bunch of identical layers, each of which consists of a multi-head self-attention and a fully connected feedforward network as its main components. Each major component incorporates normalization and residual concatenation to mitigate the vanishing gradient problem. Multiple Transformer Encoder layers are connected back and forth to form a deep Transformer Encoder structure [17; 18].  
The self-attention allows the encoder to consider other parts of the input sequence when encoding a particular piece. It computed the attention scores based on the query (Q), key (\(K\)), and value (\(V\)) matrices derived from the input embedding layer. The attention function can be described as  
\[\text{Attention}(Q,K,V)=\text{Softmax}\left(\frac{QK^{T}}{\sqrt{d_{k}}}\right)V \tag{1}\]  
where \(d_{k}\) is the dimensionality of the key vectors, which helps in stabilizing the gradients. Multi-head attention extends this by parallelizing multiple attention heads, each focusing on different parts of the input sequence, thereby capturing a richer representation of the input data. Each Transformer Encoder layer includes a feed-forward network applied to each position separately and identically. Since the order of seven features was irrelevant to grading results, the positional encoding in Transformer Encoder was removed in this study. This adjustment allows the model to focus on the key features itself, adapting to the disorder of features.  
\begin{table}
\begin{tabular}{c c c c c c} \hline
**Label** & **Reference Price Level** & **Numbers of Samples** & **Size of Training Set** & **Size of Validation Set** & **Size of Test Set** \\ \hline
1 & 0–100 & 722 & 587 & 73 & 62 \\
2 & 100–300 & 696 & 556 & 70 & 70 \\
3 & 300–600 & 673 & 538 & 66 & 69 \\
4 & 600–1500 & 668 & 534 & 65 & 69 \\
5 & 1500–3000 & 624 & 499 & 63 & 62 \\
6 & 3000–5000 & 593 & 474 & 60 & 59 \\
7 & \(>\)5000 & 580 & 464 & 58 & 58 \\ Total & & 4556 & 3652 & 455 & 449 \\ \hline \end{tabular}
\end{table}
Table 2: Label, reference price, and sample numbers for training, validation, and test sets.
## 1 Introduction
Despite a long period of study, the bubble sizes used for analysis have been limited, and there is no rigorous evaluation of the accuracy and robustness of the numerical methods used. As bubble shape parameters play an important role in the theory describing various dynamic phenomena related to floating bubbles [36], accurate prediction of the bubble parameter is crucial, especially in a wide range of Bond numbers. For instance, the lifetimes of floating bubbles are related to the surface area of the bubble film [15]. The jet velocity generated when the bubble bursts is related to the depth of the cavity, and the amount of aerosol emitted depends on the radius and height of the cap [23; 27]. [PERSON] et al. [5] used the correlation between the dimensionless cap area and the dimensionless diameter to calculate the number of film droplets generated in the bubble bursting process, and the correlation between the dimensionless base radius and the dimensionless diameter to obtain the critical film thickness. [PERSON] et al. [37] employed a perturbation method for large bubbles to obtain analytical expressions for bubble parameters as a function of the Bond number, and compared their results with experimental data. [PERSON] et al. [36] also derived approximate analytical solutions for bubble parameters over a moderate range of Bond number (\(0<Bo<1\)) and compared these with their experimental results. Moreover, to the best of the present author's knowledge, no study has yet been published detailing the  
Figure 1: The schematic configuration of a floating bubble. correlations between the Bond number and various bubble parameters over an extensive range of Bond numbers. In the present work, an improved numerical procedure is proposed to calculate the shapes of floating bubbles over a much wider range of Bond numbers than that examined in the previous studies. The proposed method is validated against the earlier numerical and experimental results to prove its accuracy and robustness. New correlations between the Bond number and several important bubble parameters, along with their asymptotic relations, are then provided. The novelty of this study is as follows. A new method is developed by introducing specific functions for accurately finding the location of the connection points on the bubble surface regardless of the ODE solvers and their boundary points. An improved algorithm to find the junction point is developed, based on a novel function to determine the type of meniscus.
# Experimental and Numerical Dynamic Identification of an Aerostatic Electro-Spindle
## 4 Numerical Results
### Damped Natural Frequencies and Damping-Factor Identification  
The damped natural frequencies and the damping factors were evaluated for the displacements along the x and y directions by the rotor in correspondence of planes at \(z=0\) (at the right edge of the front journal bearing) and at \(z=z_{L}=126\) mm (at the left edge of the rear journal bearing). Moreover, to better identify both the cylindrical and the conical modes, the center-of-mass displacement, \(y_{G}\), and the tilting angles, \(\vartheta_{x}\) and \(\vartheta_{y}\), were considered. Two different initial conditions were simulated since the frequency content of the spectra of the system response may depend on how the system is excited. Condition 1 is defined by:  
\[\begin{array}{c}x(z=0,t=0)\ =\ 0\,\ y(z=0,t=0)\ =0\\ x(z=z_{L},t=0)\ =0,\ \ y(z=z_{L},t=0)\ =0\\ \dot{x}(z=0,t=0)\ =0\,\ \dot{y}(z=0,t=0)\ =-0.01\ \text{m/s}\\ \dot{x}(z=z_{L},t=0)\ =0\,\ \dot{y}(z=z_{L},t=0)\ =0\end{array}\]  
Condition 1 was adopted in an attempt to reproduce the performed experimental tests. Vertical impulses were applied to the spindle nose, and the rotor oscillations were measured on the rear and front plane. Since the results obtained with the initial conditions (condition 1) were mainly governed by the conical mode, a second condition (condition 2) was simulated in an attempt to better figure out the cylindrical mode-shape of the rotor.  
\[\begin{array}{c}x(z=z_{G},t=0)\ =1\ \text{\mu m},\ \ \ y(z=z_{G},t=0)\ =0\\ \dot{\vartheta}_{x}(t=0)\ =0,\ \ \vartheta_{y}(t=0)\ =0\\ \dot{x}(z=z_{G},t=0)\ =0\,\ \dot{y}(z=z_{G},t=0)\ =-0.01\ \text{m/s}\\ \dot{\vartheta}_{x}(t=0)\ =0\,\ \dot{\vartheta}_{y}(t=0)\ =0\end{array}\]  
Condition 2 was chosen to simultaneously produce a whirl and a translation of the rotor. Due to the non-symmetry of the rotor-bearings system, both modes were visible both with initial condition 1 and with condition 2. The frequency content of rotor displacements puts in evidence the general presence of two natural frequencies, as seen in Figure 13. The prominence of one with respect to the other depends on the initial condition.  
Figure 12: Comparison of vibrations obtained on front and rear bearings with 31 \(\times\) 96 and \(31\times 48\) grids; the two curves match fairly well in case the time step is halved when doubling the grid points.
## 4 Results
### Dataset X1
#### 4.1.3 Feature Contribution for Dataset X1  
Absolute loadings of the first principal component reveal the significance of each feature. Notable feature contributions for Dataset X1 are depicted in Figure 9.  
The absolute loadings of the first principal component in Dataset X1 offer valuable insights into the relative importance of each feature requirement for optimal cargo logistics center locations. The following detailed analysis highlights the significance of each feature.  
Among the features examined, FR9 emerges as the most critical factor, with its highest loading indicating its pivotal role in assessing suitability. Additionally, FR2, FR3, FR5, and FR7 exhibit substantial loadings, highlighting their significant influence on decision-making. Conversely, FR4, FR8, and FR1 demonstrate moderate loadings, suggesting their relevance but lesser importance compared to the aforementioned factors. Finally, FR6, focusing on the improvement district's work, emerges as the least significant, with the lowest loadings indicating its relatively minor impact on cargo logistics center assessments. This comprehensive analysis enables stakeholders to prioritize factors that most significantly impact decision outcomes, thereby facilitating more informed and effective decision-making processes in cargo logistics center assessments.  
Figure 8: Scree plot of PCA result: Dataset X1.
## 4 Empirical Study
### Rank Comparison  
The evaluation of performance and stability is carried out by examining the average ranking and standard deviation of the 60 outcomes derived from varying the \"feature selection percentile\", \"look-back size\", and \"window size\" across the 16 feature selection algorithms. This approach is designed to ascertain the optimal feature selection results. The results garnered from this process facilitate the identification of the most suitable feature selection algorithm for the current sector, allowing for an examination of the feature class percentages within the optimal feature selection combination. Likewise, calculating the average rank of feature importance aids in assessing the significance of each feature, thereby augmenting the interpretability and predictive accuracy of the model. The insights obtained from the Dynamic Feature Selection System (DFSS) enhance the understanding of effective feature selection algorithms and the relevance of each feature for the specified sector at the present time. Moreover, by determining which feature class has a greater prevalence in predictions and comparing the outcomes across various combinations, it becomes possible to draw stable and reliable conclusions.
## 4 Experiments
### Results on HELEN  
#### 4.2.1 Comparison with Mainstream Methods  
To verify the practical effectiveness of the proposed model in face parsing, we first compared our method with several representative state-of-the-art approaches on the HELEN dataset. Table 1 presents the overall F1-score performance of each method. It can be observed that, although our model performs slightly lower than that of [PERSON] et al. [38] on the upper lip, it achieves comparable or better results on other facial components. Compared with the method proposed by [PERSON] et al. [39], our model demonstrates significant improvements in the segmentation of facial organs. These results indicate that the proposed method exhibits stronger adaptability in structural recognition and boundary preservation of facial regions, especially in fine-grained areas such as eyes and lips.
## 1 Introduction  
Transformer language modeling with pretraining and word representations combined with transfer learning has significantly improved many natural language understanding (NLU) tasks, such as text classification and natural language inference [1, 2, 3, 4, 5, 6]. Deep-contextualized language models pretrained through masked language modeling, such as bidirectional encoder representations from transformers (BERT) [3], have demonstrated state-of-the-art performances in NLU tasks. However, these BERT-based models are primarily tested on text data in English; hence, they may not perform well on text written in other languages. In particular, the Korean language, which is an agglutinative language that requires sophisticated processing, may not be managed effectively by pretrained language models (PLMs) developed for the English language. Korean-specific PLMs such as KoBERT [7], KLUE-BERT [8], and KR-BERT [9], pretrained on numerous novel Korean-centric datasets, have shown significant improvements in Korean-based NLU tasks. This indicates that PLMs are sensitive to the language of the text data used for pretraining. Most studies regarding pretraining with Korean-specific corpora are typically conducted using datasets obtained based on different criteria; however, the performance is not comprehensively verified for each NLU benchmark, rendering the selection of pretrained specifications difficult.  
In this study, we investigated the manner in which the current PLMs classify climate technology in research proposals written in the Korean language, which is a multiclasstext classification task. We fine-tuned seven PLMs using a dataset and compared the classification performance across PLMs while matching all hyperparameters. The main contributions/findings of this study are as follows:  
* We compared the performance of a fine-tuned model with those of seven BERT-based models pretrained with different pretraining corpora.
* We evaluated the performance of each model when performing Korean-based multi-class text classification using a climate technology classification dataset that contained more than 200,000 research proposals in Korean spanning 45 different categories.
* The pretrained model, which was fine-tuned using the most recent novel Korean corpora, showed up to a 7% performance improvement as compared with the model with different Korean corpora.
# Recent Advancements in Non-Destructive Testing Techniques for Structural Health Monitoring
## 5 Fiber Optics
### Distributed Optical Fiber Sensors
The proposed method was tested by embedding DOFSs in reinforced concrete specimens to measure the expansion strain due to the corrosion of the reinforcement. The experimental results have shown that the proposed method, based on DOFSs can accurately evaluate reinforcement corrosion in RC structures. [PERSON] et al. [PERSON] et al. (2018) used DOFSs based on Brillouin optical domain reflectometry to monitor strain and location and width of resulting crack openings of a historic masonry vault due to the pile construction of the London Bridge Station Redevelopment. The results of this research indicated that DOFSs could accurately estimate the location and width of new radial cracks that formed during piling. [PERSON] et al. [PERSON] et al. (2018) monitored the response of a reinforced concrete pump lock structure during the construction process. The variations of temperature and stress/strain were measured using OTDR sensors based on Raman and Brillouin scattering. The results showed that the integrated distributed optical fiber sensing technology is a powerful tool to monitor the construction quality of a pump lock structure. DOFSs have also been used to measure strain fibre-reinforced polymer (FRP) strengthened reinforced concrete shear walls tested under cyclic loading by [PERSON] et al. [PERSON] et al. (2018). The DOFSs were utilised instead of strain gauges and could capture strain at any location along the length of a fiber, without knowing the specific measurement locations before the test. The DOFSs provided accurate estimations of the strain distribution and were not damaged during large reversals of the cyclic loading. Figure 7: OTDR method-Rayleigh backscattering ([PERSON] et al., 2018). [PERSON] and [PERSON] [52] used DOFSs to monitor strain in the members of the steel model truss. The truss model was tested under both static and dynamic conditions, and the results indicated that DOFSs could detect localized strain variations due to damage. [PERSON] et al. [53] used the Brillouin OFDR to monitor the tunnel lining segment joint of the Suzhou Metro Line 1 tunnel. The results demonstrated the accuracy of the distributed fiber sensing method. Finally, [PERSON] et al. [54] installed DOFSs to measure the deformation of the cross-section of a non-circular PVC pipe due to the dead weight of its carrying water. The deformation was displayed as oscillations in strain measurements made by the DOFSs.
# An Evolutionary Algorithm to Optimise a Distributed UAV Swarm Formation System
## 3 Distributed Formation Algorithm3 (DFA3)
In this case, depending on the distance from the UAV to the rogue drone and the value of \(D_{min}\), an extra intensity \(F_{RD}\) can be applied as a repelling force (\(\omega\)) with respect to the rogue drone. Finally, having calculated the 3D components of \(r\), the inclination \(\theta\) and azimuth \(\phi\) are calculated as the new moving direction (in 3D space) to be returned to the UAV's controller. Figure 2 shows a flow chart of the DFA3 depicting the beacons being received by the RADIO module to be processed in order to calculate the vector \(r\) by using the optimised parameter set. Finally, the inclination \(\theta\), azimuth \(\phi\), and \(Spd\) are sent to the UAV to modify its moving direction. ```
functionDFA\({}^{3}(D_{RD},D_{th},D_{min},F_{RD})\) \(r_{x}\gets 0,r_{y}\gets 0,r_{z}\gets 0\) for\(uav\in BEACONS\)do \(range,vBearing,hBearing\gets RangeAndBearing(uav)\)\(\triangleright\) From the other UAVs \(r_{x}\gets r_{x}+(range-D_{th})\times\cos(hBearing)\times\sin(vBearing)\)\(\triangleright\) Force \(r_{y}\gets r_{y}+(range-D_{th})\times\sin(hBearing)\times\sin(vBearing)\) \(r_{z}\gets r_{z}+(range-D_{th})\times\cos(vBearing)\) endfor \(range,vBearing,hBearing\gets RangeAndBearing(rogue_drone)\)\(\triangleright\) Rogue drone \(\omega\gets 1.0\) if\(range<D_{min}\)then \(\omega\gets F_{RD}\)\(\triangleright\) Force intensity endif \(r_{x}\gets r_{x}+\omega(range-D_{RD})\times\cos(hBearing)\times\sin(vBearing)\) \(r_{y}\gets r_{y}+\omega(range-D_{RD})\times\sin(hBearing)\times\sin(vBearing)\) \(r_{z}\gets r_{z}+\omega(range-D_{RD})\times\cos(vBearing)\) \(\theta\leftarrow\arctan\frac{r_{y}}{r_{x}},\phi\leftarrow\arccos\frac{r_{z}}{ \sqrt{r_{z}^{2}+r_{y}^{2}+r_{z}^{2}}}\)\(\triangleright\) Next moving direction (angles) return\(\theta,\phi\)\(\triangleright\) Inclination and azimuth endfunction
```  
**Algorithm 1** Distributed formation algorithm\({}^{3}\) (DFA\({}^{3}\)). Table 1 shows the parameters of UAVs running DFA\({}^{3}\). It can be seen that the range of the first three parameters depends on the desired distance \(D_{RD}\), whereas the robots' speed is in a fixed range. All these parameter constraints were experimentally calculated. In the following sections, we describe the optimisation approach including the problem representation, the proposed optimisation algorithm and operators, and its parameterisation. After that, a random search algorithm is proposed as a sanity check, followed by the description of our case studies and our experimental results. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Parameter** & **Name** & **Value Range** \\ \hline Distance threshold & \(D_{th}\) & \(\{\frac{1}{2}D_{RD}-3D_{RD}\}\) \\ Minimum distance & \(D_{min}\) & \(\{\frac{1}{2}D_{RD}-3D_{RD}\}\) \\ Attracting/Repelling force & \(F_{RD}\) & \(\{\frac{1}{2}D_{RD}-3D_{RD}\}\) \\ Speed & \(Spd\) & \([1-200]\) \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Proposed parameters for the UAVs in the formation. Figure 2: Flow chart of the DFA\({}^{3}\).
# Federated Learning for Surface Roughness
## 4 Results
### Model Results for Surface Roughness Prediction in a Federated Learning Environment  
In the preliminary experiments of federated learning, we directly utilized the existing distributed datasets for training without performing any data augmentation or expansion. Based on the raw Absolute Error results shown in Table 1, the application of federated learning led to an approximate 10% reduction in the overall Absolute Error, indicating potential benefits. However, there remains significant room for improvement. Figure 10 illustrates the comparison between the predicted values and actual labels on the test dataset using the federated learning model without any data balancing strategies. From the figure, it can be observed that although the model was capable of capturing the overall trend of surface roughness, the prediction errors increased noticeably in certain data intervals. This is particularly evident in regions with relatively high or low roughness values, where the predictions appear more dispersed and less accurate. This issue mainly stems from the uneven data distribution across clients in the federated learning setup. Some clients have limited data volume, or the data within specific roughness ranges are underrepresented. As a result, the global model's learning performance in those regions is constrained. To address this, future experiments will focus on adjusting data distribution strategies to improve model robustness. The detailed Absolute Error distribution of the federated learning model is summarized in Table 2, and the distribution of sample proportions across different Absolute Error intervals is shown in Figure 11. Additionally, the RMSE during this phase increased to 0.109, indicating reduced prediction accuracy in the initial federated setup.  
As shown in Table 2, the proportion of samples falling within the high-accuracy range (Absolute Error < 0.1) was 62%, indicating that federated learning can effectively enhance the model's learning capability for certain portions of the data. However, 38.21% of the samples exhibited prediction errors exceeding 0.1, highlighting that data imbalance can significantly impact the model's stability and generalization performance. To further address this issue, the study will conduct an in-depth analysis of the underlying causes and propose effective data processing strategies in subsequent phases, with the goal of improving the overall predictive performance of the model.