esa-sceva/satcom-chunk-collection huggingface dataset:
# Compressing BERT for Binary Text Classification via Adaptive Truncation before Fine-Tuning
## 3 Full BERT Does Not Always Perform the Best
### A Brief Introduction to BERT  
A BERT model [4] consists of multiple stacked Transformers [47]. Its size is mainly determined by three hyper-parameters: the number of Transformer layers, the dimensionality of hidden state vectors and the number of self-attention heads in each Transformer. Typical settings include (12, 768, 12) and (24, 1024, 16). The corresponding models are called \(\text{BERT}_{\text{base}}\) and \(\text{BERT}_{\text{large}}\), which contain 110M and 360M parameters, respectively, and hence need huge computation resources and enormous corpora for training. To mitigate this, BERT models are pre-trained in a self-supervised manner using large-scale unlabeled texts, such as those collected from Wikipedia, on two tasks, i.e., masked language modeling (MLM) and next sentence prediction (NSP). In addition, the model parameters learned via pre-training have been made publicly available. Thus, a BERT-based model for a downstream task such as BTC (see Figure 2 for example) can be easily obtained by (i) appending a task layer to the top of BERT, (ii) loading the pre-trained parameter values as initialization for BERT, (iii) randomly initializing the parameters in the task layer and (iv) training those model parameters, i.e., those both in BERT and the task layer, using a set of labeled samples of the downstream task. As the dataset involved in step (iv) commonly contains no more than tens of thousands of samples, which is quite small compared with the number of model parameters, the training process can often slightly change the parameter values, particularly those in BERT, and hence is called fine-tuning. In short, the training process of a BERT-based model consists of two stages: _pre-training_ and _fine-tuning_. Subsequently, the trained model could be applied to unseen test samples to predict their labels. The latter process is called _inference_.  
As pre-trained BERTs have been shared online, the resource issues that need to be considered are mainly regarding those consumed during fine-tuning and inference. However, as previously mentioned, existing studies chiefly focus on the problem involved in inference, while ignoring those involved in fine-tuning. Even worse, they usually reduce the resources needed for inference at the cost of much larger investments required for the training process.  
Figure 2: An illustration of typical BERT-based text classification models. (**a**) BERT-based model for single text classification, which takes a single sequence of tokens as input. (**b**) BERT-based model for text-pair classification, which converts two input sequences of tokens into one by inserting a separator token [SEP] between them. Experiments presented in this article adopt one of these two models according to the input format.
# Multi-Head Self-Attention-Based Fully Convolutional Network for RUL Prediction of Turbofan Engines
## 5 Performance Comparison and Discussion
### Comparison with Other Models
In summary, compared with some of the existing deep neural networks used for predicting the RUL of turbofan engines, MSA-FCN can achieve competitive advantages in terms of accuracy and timeliness. However, it does not yet exhibit outstanding performance in robustness and computational cost. Therefore, one of focuses of our future research work is to achieve a trade-off among performance, stability, and computational time for the prediction model.
# A Node Placement Algorithm Utilizing Mobile Nodes in WSN and IoT Networks +
## 6 Evaluation  
To verify the effectiveness of our algorithms, we run three different scenarios. In the first scenario, we evaluate the performance of the Node Placement Algorithms, as presented in Section 4. In particular, we compare the performance of the Dynamic and Direct Mobile CC algorithms between them, as well as with a \"resource control\" [18] congestion control algorithm. In the second scenario, we evaluate the performance of the Energy Node Placement Algorithm through different scenarios based on the mobile node's actions (see Section 6.3.2). Finally, in the third scenario, we again evaluate the performance of the Energy Node Placement algorithm but when different energy models are employed, based on different mobile robots characteristics (see Section 6.3.3).
## 1 Introduction  
Computer vision is an interdisciplinary field that deals with how computers can be made for gaining high-level understanding from digital images or videos. From the perspective of engineering, it seeks to automate assignments that the human eyes can do. As a sub-domain of computer vision, visual measurement is employed for some applications involving dimensional survey tasks, and it always utilizes one or more cameras with exactly known intrinsic parameters for addressing perspective-n-point problem [1] and, therefore, camera calibration is pivotal for ensuring the system accuracy [2].  
Most camera calibration approaches require a certain number of correspondences between world and image frames, which are also known as control points, and they usually are called \"targets\" in photogrammetry [3]. These approaches are performed with planar or non-planar targets with exactly known geometries. After the targets are photographed, their corresponding image points need to be localized for solving intrinsic and extrinsic parameters based on bundle adjustment or other optimization models [4]. As a result, the accuracy of camera calibration is largely dependent on the localization of image points, and usually evaluated by re-projection errors [5].  
Circular dots and chessboards are the most common target types. Without a loss of generality, projecting the center of a circle yields an image point that is not necessary to be the center of a pattern projected from the circle, unless the pattern is still circular. Contrarily, the corner of a chessboard is scarcely subjected to projective transformations. For that reason, chessboards are more convenient for achieving targets in visual measurements [6]. In addition, since [PERSON] [2] proposed a flexible calibration approach employing a planar rig with chessboard patterns, this approach has been cited more than ten thousand times, and made chessboards the most frequently used targets for camera calibration.  
Chessboard corners can be detected at pixel level using conventional detectors, such as Harris Harris and Kanade-Lucas-Tomasi (KLT) [8]. These detectors usually extract a set of redundant points from one corner, due to user-defined parameters. Some modified means [9; 10; 11] are contributed to make arbitrary points converge into nearby corners, but their operational accuracy is still at pixel level. In many application scenarios [12; 13; 14], however, pixel resolution is not yet accurate enough and, therefore, mathematical techniques, such as interpolations or approximations, are used to localize sub-pixel corners. In this paper, a novel approach for sub-pixel corner localization is proposed, and experiments are conducted to verify the new approach.
## 4 Model Applied to Polyp Segmentation
### Ablation Study  
We also evaluated the effectiveness of FuzzyTransNet in colon polyp segmentation through an ablation study. In Table 7, when comparing F.1 and F.2, we can observe that the Swin Transformer V2 outperformed DeiT-Small [36] with the same CNN branch configuration. Furthermore, when comparing F.3 and F.4, we note that the parallel model demonstrated superior performance over the sequential model, underscoring the efficacy of our fusion module. Additionally, when comparing F.4 and F.5, we can observe a significant  
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline \multirow{2}{*}{**Method**} & \multirow{2}{*}{**Year**} & \multicolumn{2}{c}{**Kvasir**} & \multicolumn{2}{c}{**ClinicDB**} & \multicolumn{2}{c}{**ColonDB**} \\ \cline{3-8}  & & **Dice** & **mIoU** & **Dice** & **mIoU** & **Dice** & **mIoU** \\ \hline U-Net [6] & 2015 & 0.818 & 0.746 & 0.823 & 0.750 & 0.512 & 0.444 \\ U-Net++ [7] & 2019 & 0.821 & 0.743 & 0.794 & 0.729 & 0.483 & 0.410 \\ ResUNet++ [41] & 2019 & 0.813 & 0.793 & 0.796 & 0.796 & - & - \\ PraNet [42] & 2020 & 0.813 & 0.793 & 0.796 & 0.796 & - & - \\ HarDNetMSEG [43] & 2021 & 0.912 & 0.857 & 0.932 & 0.882 & 0.731 & 0.660 \\ TransUnet [44] & 2021 & 0.913 & 0.857 & 0.932 & 0.883 & **0.780** & **0.697** \\ TransFuse [32] & 2021 & 0.918 & 0.868 & 0.934 & 0.886 & 0.773 & 0.696 \\ Ms RED [33] & 2022 & 0.911 & 0.854 & 0.934 & 0.885 & 0.779 & 0.690 \\
**Our method** & - & **0.929** & **0.875** & **0.942** & **0.891** & 0.778 & 0.689 \\ \hline \hline \end{tabular}
\end{table}
Table 6: Quantitative results on polyp segmentation datasets of our method compared to other segmentation algorithms. ‘-’ means results not available.  
improvement in the Dice coefficient after the incorporation of the fuzzy block. F.5 achieved a 0.6% improvement over F.4 on both the Kvasir and ClinicDB datasets.
## 3 Throughput Performance Limits of Multimode Underwater Acoustic Networks
### Modeling of Throughput Performance Limits in Multimode Networks
#### 3.1.4 Reward Function Modeling  
The reward function \(R_{a}(\hat{s},\hat{s}^{\prime})\) is represented by the ability of the current state to successfully receive data in the future. Specifically, under the decoding conflict model, whether each node can successfully receive data is analyzed according to the current state \(\hat{s}\) and action \(a\).  
(1) When the node receives the signal in the sending state, the amount of data successfully received is 0.  
(2) If the node does not receive the signal that the destination node is its own, the amount of data received at this time is 0.  
(3) In other cases, the amount of successfully received data can be analyzed by the decoding conflict model.  
It can be expressed by Formula (12):  
\[r_{ik}(\hat{s},a_{ik})=\left\{\begin{array}{l}0,\;a_{ik}\
eq 0\\ 0,\;\exists j\;s.t.\;\hat{s}_{j,G+k-1-D_{ij}}\
eq i,\;a_{ik}=0\\ f\big{(}w_{j},w_{i},d_{ji},SINR_{ji}\big{)},\;\exists j\;s.t.\;\hat{s}_{j,G+k-1 -D_{ij}}=i,\;a_{ik}=0\\ f\big{(}w_{j},w_{i},d_{ji},SINR_{ji}\big{)}=\left\{\begin{array}{l}v_{1}\times \tau,\;if\;SINR_{ij}\geq SNR_{th}\big{(}w_{i},w_{j},d_{ij},ber_{th},f_{1} \big{)}=SNR_{th1}\;and\;\hat{s}_{j,G+k-1-D_{ij}}=i\\ \cdots\\ v_{n}\times\tau,\;elseif\;SINR_{ij}\geq SNR_{th}\big{(}w_{i},w_{j},d_{ij},ber_{ th},f_{n}\big{)}=SNR_{thn}\;and\;\hat{s}_{j,G+k-1-D_{ij}}=i\\ 0,\;others\end{array}\right. \tag{12}\] \[SINR_{ji}=\frac{\begin{array}{l}P_{i}\\ \sum\limits_{i=1}^{N-1}\;\;P_{m}+Noise\end{array}}{\exists m\;s.t.\;\hat{s}_{m,G+k-1-D_{mi}}\
eq 0}{\begin{array}{l}m=1\\ m\
eq i\end{array}}\]  
where \(\hat{s}_{j,G+k-D_{ij}}=i\) means that the signal sent by node \(j\) in slot \(t+k-1-D_{ij}\) reaches node \(i\) in slot \(t+k-1\), \(n\) is the number of supported communication systems, \(f_{i}\) is the processing algorithm of the \(i\)-th communication mode and \(v_{i}\) is the rate of the \(i\)-th communication system. If \(\hat{s}_{m,G+k-D_{mi}}\
eq 0\), it indicates that the signal sent by node \(m\) in slot \(t+k-1-D_{mi}\) reaches node \(i\) in slot \(t+k-1\), which is the interference signal of node \(i\), \(P_{m}\) is the interference signal energy, and \(\sum\limits_{i=1}^{N-1}\;P_{m}\) is the total interference signal energy.  
After that, the amount of data received by each node at each moment is added up as the reward. The transmission scale of the signal in the network is \(G\), so it is necessary to consider the benefits of the time scale of \(G\).  
\[R(\hat{s},a)=\sum\limits_{i=1}^{N}\sum\limits_{k=1}^{G}r_{ik}(\hat{s},a_{ik}) \tag{13}\]The damping coefficient \(\gamma\) is set to the constant 1.
## 2 Multi-Classification LS-SVM Survey
### Proposed WP-SVM  
Once the hyper-plane slopes have been defined, incorporation of recently acquired image data into a traditional LS-SVM model necessitates a full retraining of the system in order to calculate the new model parameters:  
\[\begin{bmatrix}W\\ B\end{bmatrix}_{new}= \begin{bmatrix}(C_{new}-G_{new})&(D_{new}-H_{new})\\ (D_{new}-H_{new})^{T}&(R_{new}-Q_{new})\end{bmatrix}\begin{bmatrix}E_{new} \\ U_{new}\end{bmatrix} \tag{17}\]  
For large data sets, such retraining is not efficient. It is expensive in terms of memory and computation time requirements. To maintain an acceptable balance between storage, accuracy and computation time, we propose the _WP-SVM_, a dynamic Weighted Proximal SVM approach. Whenever the model needs to be updated, each incremental sequence will alter matrices C, G, D, H, E, R, Q and U as defined in Eqs. (13) and (14) by the amounts of \(\Delta\)C, \(\Delta\)G, \(\Delta\)D, \(\Delta\)H, \(\Delta\)E, \(\Delta\)R, \(\Delta\)Q and \(\Delta\)U respectively. As an example, let us consider a recently acquired data set \(x_{N+1}\) that belongs to class \(t\). Eq. (17) then becomes:  
\[\begin{bmatrix}W\\ B\end{bmatrix}_{new}= \begin{bmatrix}(C+\Delta C)-(G+\Delta G)&(D+\Delta D)-(H\ +\Delta H)\\ (D+\Delta D)-(H+\Delta H)&(R\ +\Delta R)-(Q+\Delta Q)\end{bmatrix}^{-1} \begin{bmatrix}E+\Delta E\\ U+\Delta U\end{bmatrix}\]  
In order to adequately capture the effect of the newly acquired sequences, and to ensure that their impact on the hyper-plane orientations \(W\) and \(B\) is accounted for despite the unbalanced classes, we scale the incremental changes in \(\Delta\)C, \(\Delta\)G, \(\Delta\)D, \(\Delta\)H, \(\Delta\)E, \(\Delta\)R, \(\Delta\)Q and \(\Delta\)U by some weight factors (\(\Psi\)). The basic idea of the _WP-SVM_ is to assign an entropy measure to each incremental data point. These weight factors are determined based on the misclassification rate, the relative importance of a dynamic data point with respect to its class and its variance with respect to the other classes. The proposed weight factors are defined as:  
\[\psi_{\ell}=(\
u.\log\frac{N}{\
u})/\sqrt{\min\arg(s_{\ell}^{2}\zeta^{\ast})} \tag{18}\]  
We define \(\
u\) as the frequency of the incremental data sequence acquired, and \(N\) is the total number of sequence data that was used to determine the initial hyper-plane parameters of the model. The \(s_{\ell}^{2}\) factor is the Mahalanobis distance between an incremental data feature \(f\) and the hyper-plane parameters for class \(c\), scaled by \(\zeta^{\mathrm{c}}\) that represents the error rate observed in class \(c\) before the introduction of new incremental data. Eq. (18) ensures that different data points have different impact on the classifier parameters, and that data points which have low probability of occurrence but that are nevertheless important with respect to the hyper-plane position are not outnumbered and neglected in the dynamic model update process.
# Improvement of 5G Transportation Services with SDN-Based Security Solutions and beyond 5G
## 2 Literature Review and Related Work
### Transportation with Future Technologies  
Emerging technologies of transportation facilities have grown in many directions because users and service providers expect to adapt to new systems quickly. Legacy systems and adopting a new system without significant changes in infrastructures are a big challenge in transportation systems. All future infrastructures rely on 5G and beyond, but their network will be emerging with other technologies such as SDN, IoT, and photonics. Using these emerging technologies, transportation service provisions can minimize costs and energy consumption. When the appropriate technique identifies or removes the unwanted energy for the interferences from the 5G systems, energy-saving is possible. As shown in Figure 1, we illustrate the possible services of transportation that influence 5G cellular connectivity (5G-CC). According to [6], connectivity is one of the essential technical concepts for improving transportation services and infrastructure. The study of 5G-CC provides the best solutions with emerging technologies. It creates many challenges for analyzing Autonomous Driving Vehicles (ADV) and the future transportation system with an efficient framework.  
Infotainment (reports safety news such as an accident) services depend on efficient and secure data or text transmission from road service units to vehicle users. Here, all reliable transmission systems enhance the facilities of handling transportation services such as sending and delivering quick messages. Reliable transmission means that connectivity should provide the connections between the user and the service provider quickly and efficiently.  
Future transportation systems need to absorb the energy dynamically with the necessary absorbing 5G-based sensors, including photonic sensors that have emerged with 5G technology [7]. This research delivers the EE and harvests the dynamic energy from the electrojet, which influences the ionosphere and photons. Further, EE with photonic technology will increase the reliability and lifetime of future communications systems.  
Table 1 shows the different multiple accessing methods for enhancing EE, which depends on the latency. Thus, transportation service providers can improve the network traffic of the transportation service with minimum latency and delay.  
Figure 1: Transportation services with 5G cellular connectivity.  
In the 5G transportation service and infrastructure, multiple access provides some benefits for managing the data traffic of the transportation system.
# Architectural Design for Digital Twin Networks
## 3 Digital Twin Network
### Reference Architecture  
Based on the definitions provided in previous section, a Digital Twin Network architecture is outlined in Figure 1, consisting of three main layers, each described below.  
\begin{table}
\begin{tabular}{p{42.7 pt} p{284.5 pt}} \hline \hline
**Element** & **Description** \\ \hline Data & Historical and/or real-time data about the real network, including configuration, operational state, topology, and process data. This data repository serves as the authoritative source for model development. \\ \hline Models & Representations of the network, created using data collected from real-world sources. These models are essential for emulation, diagnosis, and understanding network operations. Examples include service models, data models, and knowledge graphs. \\ \hline Interfaces & Standardized connections that ensure interoperability. One interface links the DTN platform with the real network for real-time data collection and control. Another interface connects the platform with applications, facilitating service requests and access to DTN features. \\ \hline Mapping & Establishes real-time interactive relationships between the real network and its digital twin, or among multiple DTNs. This can involve continuous one-to-one synchronization or occasional one-to-many data exchanges. \\ \hline \hline \end{tabular}
\end{table}
Table 1: Core Elements of a Digital Twin Network, inspired by the draft and adapted for a more practical connection.
## 3 Geological Structure and Fault Blocks Onshore and Offshore Samos  
### Geological Units  
The island of Samos is composed of four tectonometamorphic units, which are the following from the base to top [5; 6; 23; 24] (Figure 5):  
1. The Kerketeas unit in western Samos is considered to be a relatively autochthonous unit of the island. It contains dolomitic marbles and local schists on top, which crop out in a tectonic window below the Ampelos nappe.
2. The Ampelos nappe comprises mainly schists and marbles as well as metabasites in small occurrences. The lower part contains slices of Carboniferous and Triassic gneiss.
3. The Selcuk nappe includes metabasites, which outcrop discontinuously in central Samos.
4. The Vourliotes nappe is composed of schists and marbles in eastern Samos. The upper formation of the Vourliotes nappe is of Upper Cretaceous age.
5. The Kallithean nappe comprises sandstones, serpentinized peridotite, spilite, and diabase with red radiolarites and fossiliferous limestone of ammonitico roos type. Upper Triassic to Jurassic carbonates are observed in the top.
6. The Katavasis complex comprises marbles, amphibolites, and quartzites.  
As regards the contact between these units, the Pythagoras thrust separates the Kerketeas unit from the Ampelos nappe in the western part of Samos, the Vourliotes thrust puts the Vourliotes nappe on the Ampelos nappe in central Samos, and the Kallithea detachment is a low-angle extensional fault at the western end of the island [24; 25; 26] (Figure 5).  
The geological structure of Samos also comprises Neogene (Micocene-Pliocene) formations and Quaternary deposits, which filled in the Karlovasi, Pyrgos, Mytilinii, and Paleokastro basins [4; 5; 7] (Figure 5). The Karlovasi and Mytilinii sedimentary basins, in the western and eastern part of Samos, respectively, are connected by a narrow passage, which corresponds to the Pyrgos Basin, while the Paleokastro basin is a second-order basin structure formed in the eastern part of Samos [4; 5] (Figure 5). The sedimentary basin fill comprises non-clastic and clastic series of Neogene age [4; 5; 7]. The Quaternary is represented mainly by torrential deposits within the aforementioned basins and in karst depressions, alluvial deposits along the coastal plains, as well as recent scree and tal  
Figure 5: Neotectonic map illustrating the geological formations, the active faults onshore and offshore Samos, the major inactive faults, the main fault blocks of Samos Island based on onshore and offshore studies, the 2020 Samos earthquake epicenter, and its largest aftershocks. Data from onshore studies on the active tectonics of Samos [4; 5; 7; 8; 9; 15; 17; 23; 24; 27] and offshore studies based on continuous seismic reflection profiles [6]. ONSF: Offshore Northern Samos Fault; KF: Karlovasi fault; MF: Marathokampos Fault; PF: Pythagorgio fault; VF: Vathy Fault; PT: Pythagoras Thrust; VT: Vourliotes Thrust; KD: Kallithea Detachment; WSFB: Western Samos Fault Block; KB: Karlovasi Basin; CSFB: Central Samos Fault Block; PYB: Pyrgos Basin; SSFB: Southern Samos Fault Block; MB: Mytilinii Basin; PAB: Paleokastro Basin, ESFB: Eastern Samos Fault Block.
## 4 Experimental Results
### Data and Evaluation Indicators
The model is trained using the Adam optimizer with a multi-learning strategy, starting with an initial learning rate of 0.0002 and decaying by 0.9 at each iteration. For the **evaluation indicators**, the segmentation accuracy of the brain tumor was measured using the dice evaluation function and Hausdorff distance (95%) index, respectively, for the core tumor region (TC, label 1 and label 4), enhanced tumor region (ET, label 1), and entire tumor region (WT, label 1, label 2, and label 4). 1. The dice evaluation function is a commonly used index to measure the segmentation accuracy of brain tumors. This index measures the accuracy of the model by calculating the overlap between the predicted results of the model and the real label. When the dice coefficient is close to one, the higher the overlap and the better the performance. \[\text{Dice}=\frac{2\times\text{TP}}{2\times\text{TP}+\text{FP}+\text{FN}}\] (8) where true positive (TP) represents the number of pixels that the model correctly predicts to be positive, false positive (FP) indicates the number of pixels that the model incorrectly predicts to be positive, and false negative (FN) indicates the number of pixels that the model incorrectly predicts as a negative class. 2. The Hausdorff distance (95%) represents the surface distance between the prediction and ground truth. The 95% quantized value of the maximum distance is different from the dice coefficient, which is sensitive to the inner filling of the mask. The Hausdorff distance is sensitive to the segmentation boundary of the brain tumor. It can effectively identify the boundary of the enhanced tumor region and tumor core region. The Hausdorff distance measures the distance between two subsets in the space, where d represents the element-by-element distance closest to the voxels from the first set of voxels to the second set of identical labels, X represents the true value label of the voxel, and Y represents the predicted value label of the voxel. \[\text{Hausdorff distance }(95\%)\ =95\%(\text{d}(\text{X},\text{Y})||\text{d}( \text{Y},\text{X})) \tag{9}\]