esa-sceva/satcom-chunk-collection huggingface dataset:
# Does Two-Class Training Extract Real Features? A COVID-19 Case Study
## 2 Materials and Methods
### Works and Datasets  
First, a study of the different publicly accessible datasets was carried out. It should be noted that, on some occasions, the datasets collected to prepare a work are not published, and therefore, we cannot evaluate them to corroborate their content. However, thanks to the published articles, we can indicate their most relevant characteristics. The results of this study are shown in Table 1.  
As can be observed in the datasets shown in Table 1, some of them only provide information about two classes: COVID-19 positive and COVID-19 negative. Due to this fact, on some occasions, the election of using a 2-class classification system in some works depends on the dataset used. Moreover, some datasets combine images from other COVID-19 datasets (like COVID-CRX, which includes some images from other datasets of the previous table), and finally, in some cases, other classic pneumonia datasets are used to complete the dataset itself. \begin{table}
\begin{tabular}{l l l l l l} \hline \hline
**Dataset** & **Data Type** & **Classes** & **Images** & **COVID-19 1** & **Perfect Balance 2** \\ \hline COVID-chestXray [11; 12] & CXR, CT & 4: COVID-19, healthy, other viral pneumonia & 960 & 37.4\% & 25\% \\ BIMVCCVOD-19+ [13] & CXR, CT & 3: COVID-19, non-COVID-19, others & 1354 & 90.5\% & 33.3\% \\ COVID-CXR [8] & CXR & 3: COVID-19, pneumonia, normal & 13,975 & 12.8\% & 33.3\% \\ COVID-cT [14] & CT & 3: COVID-19, pneumonia, normal & 104,009 & 19.2\% & 33.3\% \\ COVID-CT [15] & CT & 2: COVID-19, non-COVID-19 & 746 & 46.8\% & 50\% \\ COVID-CS[16] & CT & 2: COVID-19, non-COVID-19 & 144,167 & 0.5\% & 50\% \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Dataset summary. 1 Percentage of COVID-19 positive images. 2 Calculated from the number of classes in the dataset. However, as can be observed in Table 2, the most widely used dataset is COVID-chestXray, which does provide information on more than two classes. This dataset uses its own images and distinguishes between several illnesses that attack lungs and produce different types of pneumonia; thus, this is the most complete dataset. Furthermore, as can be observed in the last two columns of Table 1 (and based on the explanation included in the caption), this dataset is the most balanced among the studied datasets that distinguish between more than two classes. Other important points shown in Table 2 are described next:  
* All the works used Deep Learning (DL) as the mechanism to detect COVID-19 positives. However, four of them used a variant based on transfer learning, obtaining a Deep Transfer Learning (DTL) system. * Sixty-six-point-seven percent of the works detailed only classified between COVID-19 positive or COVID-19 negative. * ResNet is the most used CNN model. It is important to mention that other works like [9] used very simple and computationally very light training models (such as pretrained VGG16), but despite this, very promising results were obtained.
# A Historical Survey of Advances in Transformer Architectures
## 1 Introduction
### Self Attention  
The first and most important component of transformer architecture is the self-attention mechanism seen in Figure 2 which allows the model to learn the relationships between the elements of a sequence [13]. In the context of an LLM such as BERT, this would mean that in a sentence such as \"The bank of the river is overflowing\", the model would  
Figure 1: A depiction of transformer architecture.  
use self-attention to conclude that the \"bank\" in this case refers to the side of a river as opposed to a financial organization.  
In the encoder version of this layer, the inputs consist of queries and keys. The attention function is then applied to these vectors as seen in (1).  
\[Attention\ (Q,K,V)=\left(\frac{Q\cdot K^{\top}}{\sqrt{d_{k}}}\right)\ \cdot V \tag{1}\]  
where  
- \(Q\) is the matrix of the queries;  
- \(K\) is the matrix of the keys;  
- \(V\) is the matrix of the values.  
The equation is applied in a way that the dot product between the query and the key is first computed to form the score \(S=Q\cdot K^{\top}\) These scores are important as they determine how much attention is given to other words when encoding words at the current position. These scores are then normalized in order to ensure the stability of the gradient to enhance training, thereby giving the normalized score \(S_{n}=\frac{S}{\sqrt{d_{k}}}\). The softmax function is then applied to the normalized scores in order to translate them into probabilities \(P=(S_{n})\). These probabilities can then be applied to the value matrix to obtain \(Z=V\cdot P\). This would mean that vectors with larger probabilities would receive a greater focus from the consequent layers [5]. In transformers, a multi-head attention system is used wherein the original queries, keys, and values are projected into H different sets of learned projections. For each projection, the attention equation from (1) is applied to formulate the output. The output across the H projections is then concatenated to form the multi-head output. The formulation for this process can be found in (2).  
\[\begin{split} MultiHeadAttn\ (Q,K,V)=(\ head\,\iota,\cdots,\ head\ _{H})W^{O}\\ \ \ \ \  
complex relationships among different elements in a sequence to effectively be captured by the model [13] and therefore enhances the diversity of the subspace. The original transformer model proposed uses eight different heads; however, consequential works have experimented with optimizing the heads to retain the ones which provide the most important information [14].
## 3 Results and Discussion
### Carbonation Test  
The CO\({}_{2}\) uptake capacities of all the prepared samples are presented in Table 2. It was observed that, although Ca11 Mg2 Ce1 and CaO had high calcium weight percentages (72 wt.% and 100 wt.% respectively), their calcium utilizations were low (30 wt.% and 18 wt.% respectively). The Ca1 Mg2 Ce1 sample had a low calcium percentage uptake capacity, but its calcium utilization was very high at 69%. These results suggest that the high calcium conversion in the sorbent is not related solely to its calcium content but also to its porous structure, as discussed in Section 3.3.3. The thermogravimetric analyzer (TGA), the SDT-Q600-TA instrument, provides a balance sensitivity of 0.1 ug and calorimetric accuracy of \(\pm\)2%. The CO\({}_{2}\) uptake tests were repeated to investigate the accuracy of the results, and the error difference was found to be \(\pm\)1%.  
According to their CO\({}_{2}\) uptake capacities, two groups were selected for further characterization and carbonation/de-carbonation cycling tests. The prepared CaO was used as the benchmark. The first group had higher CO\({}_{2}\) uptake capacities: Ca6 Mg2 Ce1 and Ca4 Mg2 Ce1; the second group had relatively low CO\({}_{2}\) uptake capacities: Ca11 Mg2 Ce1 and Ca1 Mg2 Ce1 (see Table 2). Ca11 Mg2 Ce1 had high CaO molar content, whereas Ca1 Mg2 Ce1 had low CaO molar content (see Table 1). It was observed that the CO\({}_{2}\) uptake capacities of Ca4 Mg2 Ce1 and Ca6 Mg2 Ce1 were 25 wt.% and 29 wt.%, respectively, and were in the acceptable range for sorption-enhanced steam reforming (SESR) processes, i.e., 20-40 wt.% [15].
# Performance Analysis of Cooperative Non-Orthogonal Multiple Access in Visible Light Communication
## 5 Conclusions  
This paper investigates the performance of non-cooperative and cooperative PD-NOMA-VLC downlink systems under perfect CSI. BER analytical expressions are matched with the Monte Carlo simulation results. The obtained results show that the cooperative PD-NOMA enhanced \(U_{1}\) BER performance by 8.2 dB at \(10^{-6}\) BER in a two-user scenario. Furthermore, in a three-user scenario, BERs of \(U_{2}\) and \(U_{1}\) are enhanced by 2 dB and 9 dB at \(10^{-6}\) BER, respectively. Achievable rate performance of non-cooperative and cooperative PD-NOMA are evaluated. Simulation results show that in a two-user scenario the cooperative PD-NOMA effectively enhances \(U_{1}\) achievable rate by 14.1 bit/s/Hz at 45 dB SNR. Moreover, in a three-user scenario cooperative PD-NOMA improves \(U_{2}\) and \(U_{1}\) achievable rates by 12.4 bit/s/Hz and 14.3 bit/s/Hz at 45 dB SNR, respectively. The recommendation for future work is to study the maximum number of users that can cooperate and achieve undisruptive communication in the proposed system. Also, we suggest extending the system for multiple LEDs and apply multiple-input multiple-output techniques to the proposed downlink cooperative PD-NOMA-VLC system and study its effect on both the BER and achievable rate performance.  
Conceptualization, M.A.; Validation, H.S., M.A., S.M.G. and H.E.; Formal analysis, H.S. and H.E.; Investigation, H.S.; Data curation, M.A.; Writing--original draft, H.S.; Writing--review and editing, M.A., S.M.G. and H.E.; Supervision, M.A., S.M.G. and H.E.  
This research received no external funding.  
The authors declare no conflict of interest.
# AI-Assisted Security Alert Data Analysis with Imbalanced Learning Methods
## 3 Methodology
### Classification and Analysis
The algorithm comprises the gradient-based one-side sampling (GOSS) and exclusive feature bundling (EFB) techniques. In GOSS, different samples play different roles in calculating the information gain, in which an instance with greater gradients can add more to the information gain. GOSS retains instances with high gradients and eliminates those with limited gradients at random to maintain the precision of information gain estimation [69; 70]. The mathematical analysis of GOSS is shown in Equation (4):  
\[\hat{V}_{j}(d)=\frac{1}{n}\Bigg{(}\frac{(\sum_{x_{i}\in A_{i}}g_{i}+\frac{1-a} {b}\sum_{x_{i}\in B_{i}}g_{i})^{2}}{n_{l}^{j}(d)}+\frac{(\sum_{x_{i}\in A_{i} }g_{i}+\frac{1-a}{b}\sum_{x_{i}\in B_{i}}g_{i})^{2}}{n_{l}^{j}(d)}\Bigg{)} \tag{4}\]  
where \(\hat{V}_{j}(d)\) is the estimated variance gain over the subset. \(A\cup B\), \(A_{l}=\{x_{i}\in A:x_{ij}\leq d\}\); \(A_{r}=\{x_{i}\in A:x_{ij}>d\}\); \(B_{l}=\{x_{i}\in B:x_{ij}\leq d\}\); \(B_{r}=\{x_{i}\in B:x_{ij}>d\}\); the coefficient \(\frac{1-a}{b}\) was used to normalize the sum of the gradients over \(B\) to the size of \(A^{c}\). Thus, the estimated \(\hat{V}_{j}(d)\) was used over a smaller instance subset instead of the accurate \(V_{j}(d)\) over all instances to determine the split point. Simultaneously, the EFB technique is used by LightGBM to minimize the model complexity by bundling exclusive features into a single feature. XGBoost is a supervised learning algorithm that implements gradient-boosted DTs. Gradient boosting is a technique for predicting a target variable by combining the estimates of a set of simpler and weaker models. The weak models are DTs trained sequentially to minimize the loss function. New models are created that predict the residuals or errors of prior models and are subsequently added together to make the final prediction. The gradient descent algorithm minimizes the loss when adding new models. XGBoost minimizes a regularized (L1 and L2) objective function that combines a convex loss function and penalty term for model complexity. The loss function is based on the difference between the predicted and the target outputs. Gradient boosting is a powerful technique for achieving state-of-the-art results for several ML tasks. For a dataset with \(n\) examples, where \(i\) represents each sample, XGBoost uses a loss function to build trees by minimizing Equation (5):  
\[\mathcal{L}(\phi)=\sum_{i}l(\hat{y}_{i},y_{i})+\sum_{t}\Omega(f_{t}) \tag{5}\]  
where the first part represents the loss function that calculates the pseudo-residuals of the predicted \(\hat{y}_{i}\) and the true value \(y_{i}\) in each leaf, and the second part \(\Omega(f)=\gamma T+\frac{1}{2}\lambda\parallel w\parallel^{2}\). \(\gamma\) represents the user-definable penalty, which encourages pruning, and \(T\) represents the number of terminal nodes or leaves in a tree. \(\lambda\) is the regularization term intended to reduce the insensitivity of the prediction to individual observations, and \(w\) represents the leaf weights and output value for the leaf.
## Materials and Methods
### Improved K&I  
After obtaining the CI of the bi-temporal PolSAR data, an automatic method of threshold selection is introduced. The K&I algorithm for the automatic estimation of the optimal threshold is suitable for SAR data [4,14,15,26] and has been widely used to distinguish the changed and unchanged classes in the CI. The K&I thresholding method is an extension of [PERSON] minimum-error probability theory and can be shown as follows:  
\[J(T)=\sum_{d_{l}=0}^{L-1}h(d_{l})c(d_{l},T)\text{ where }c(d_{l},T)=\left\{ \begin{array}{l}-2\ln P(\omega_{u}|d_{l},T),d_{l}\leq T\\ -2\ln P(\omega_{c}|d_{l},T),d_{l}>T\end{array}\right. \tag{11}\]  
where \(h(d_{l})\) and \(L\) represent the histogram and the number of possible gray levels of the CI, respectively, and \(c(d_{l},T)\) denotes the cost of classifying pixels by comparing the corresponding gray level and threshold \(T\). \(P(\omega_{i}|d_{l},T)\)\((i=u,c)\) is the posterior probability, which represents the unchanged (or changed) class under the condition of gray level \(d_{l}\) and a specific value of the threshold \(T\).  
The optimal threshold corresponding to minimizing the classification error is the following cost function [37]:  
\[T^{*}=\underset{T=0,1,\ldots,L-1}{\arg\min}J(T) \tag{12}\]  
Using [PERSON] theorem, the posterior probability can be expressed by the prior probability and class-conditional PDF, i.e.,  
\[P(\omega_{i}|d_{l},T)=\frac{P(\omega_{i})\cdot P(d_{l}|\omega_{i},T)}{\sum \limits_{j\in\{u,c\}}P(\omega_{j})\cdot P\big{(}d_{l}|\omega_{j},T\big{)}}. \tag{13}\]  
The traditional K&I method assumes that the class-conditional distribution follows a Gaussian distribution, but this assumption cannot accurately reflect the distribution of the CI. In order to improve the threshold selection, an improved model based on the GGM describing the statistical behaviors of the changed and unchanged classes in the CI can be used. The model of the class-conditional PDF is shown as follows [14]:  
\[P(d_{l}|\omega_{i},T)=a_{i}(T)e^{-[b_{l}(T)|d_{l}-m_{l}(T)|]^{\beta_{l}(T)}}, i=u,c\text{ where }a_{i}(T)=\frac{b_{i}(T)\beta_{i}(T)}{2\Gamma(1/\beta_{i}(T))},
# Machine Learning-Based Classification of Mushrooms Using a Smartphone Application
## 2 Materials and Methods
### Machine Learning Algorithm  
While recognizing and classifying all dangerous mushroom species is the objective, we began by constructing databases for three distinct scenarios. The first model was able to classify species of the morphologically similar genera of _Gyromitra_ (poisonous species) and _Morchella_ (highly sought edible species). The second model was able to classify genera  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline  & & **Total** & **Training** & **Testing** \\ \hline \multirow{2}{*}{2-class model} & _Gyromitra_ & 238 & 190 & 48 \\ \cline{2-5}  & _Morchella_ & 238 & 190 & 48 \\ \hline \multirow{3}{*}{3-class model} & _Clavulina_ & 497 & 369 & 128 \\ \cline{2-5}  & _Inocybe_ & 497 & 369 & 128 \\ \cline{2-5}  & _Marsmius_ & 497 & 369 & 128 \\ \hline \multirow{3}{*}{5-class model} & _Agaricus_ & 257 & 215 & 42 \\ \cline{2-5}  & _Amanita_ & 328 & 288 & 40 \\ \cline{2-5}  & _Cantharellus_ & 453 & 409 & 44 \\ \cline{2-5}  & _Pleurrotus_ & 399 & 349 & 50 \\ \cline{2-5}  & _Tricholoma_ & 456 & 412 & 44 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Total number of images used for machine learning training and testing.  
of _Clavulina_, _Inocybe_, and _Marasmius_, and the third model was able to classify genera of _Agaricus_ (some edible species), _Amanita_ (includes deadly species), _Cantharellus_ (prized eidibles), _Pleurotus_ (prized eidibles), and _Tricholoma_ (some toxic species). The first two models relied on the same neural network architecture (Figure 1), but the third model employed a different one. A convolutional neural network and a feed-forward neural network comprised the architecture of the first two models. For the convolutional neural network, input variables were 224 \(\times\) 224-pixel images in three-channel RGB color and six convolutional layer blocks. Max-pooling was applied after 1 st, 3 rd, 4 th, and 5 th convolutional layers. A rectified linear unit (ReLU) activation function was used after the six convolutional layers [25]. The feed-forward network consisted of three fully connected layers; the number of features in the last layer was equivalent to the number of classes. After each fully connected layer, exponential linear unit (ELU) [26] activation was used with 70% dropout rates. This neural network architecture was unable to obtain satisfactory classification results for the 5-class model. Therefore, we decided to utilize the fine-tuned ResNet system [23] with 152 layers model and added an output layer with a number of neurons equivalent to the number of mushroom species classes. The hardware specification of the server was Intel(r) Xeon(r) CPU @ 2.30 GHz, 34 GB RAM, and Tesla P100-PCIE-16 GB.
## 2 Methodology
### Negative Selection Algorithm Using Spherical Detectors with Variable Radii
#### 2.2.7 Notes  
* The radius of each detector is computed dynamically based on the minimum distance to the nearest data point.
* Detectors are designed to be non-overlapping the data points and with centers not detected by other detectors, which promotes more efficient coverage of the feature space.
* The parameter \(r_{\min}\) ensures that detectors cannot be smaller than a minimum size, preventing very small detectors from being added to the set.
* The number of generated detectors and the ability to cover the feature space depend on the dataset distribution and the parameters \(r_{\min}\) and \(T_{\max}\).
* The minimum value of the parameter \(r_{\min}\) depends on the data being analyzed and can be selected through trial and error. It can also be based on the average distances between nearest neighbors present in the analyzed data, as described in Section 1.1. An example of a specific approach is presented in the Results section.  
An example result of this algorithm applied to 2D data is shown in Figure 2.  
Figure 2: Example result of negative selection algorithm on two-dimensional data. Spherical detectors (red circles) with different radii were used.  
Negative Selection Algorithm Using Spherical Detectors with Variable Radii and Decreasing Radius Strategy  
The third version of the negative selection algorithm (for semi-supervised anomaly detection) is a modified version of the previous one. This time, detectors with large radii are more strongly preferred, and the generation of detectors begins with them. In contrast to the previous algorithm, where the detector, with its center at a random point, maximized its radius based on self-data but was accepted even if the calculated radius was small (as long as it was not smaller than the minimum allowable), the new version prioritizes larger radii from the start.
## 4 Enhancement of Brillouin Stokes Signal (I): Induction of Stimulated Scattering
### Experimental Setup  
The FUT used in the experiment was a 1-m PFGI-POF (Asahi Glass Co., Ltd., 2009) with a core diameter of 120 \(\upmu\)m, NA of 0.185, a core refractive index of \(\sim\)1.35, and propagation loss of \(\sim\)250 dB/km at 1.55 \(\upmu\)m (manufacturer's specifications). The experimental setup is schematically shown in Figure 1, which is similar to that of Brillouin optical correlation-domain analysis (BOCDA) [16,17].  
The light beam from a 1550-nm three-electrode laser diode (LD) was divided into two. One was used as the pump light, after being chopped with an intensity modulator (IM) for lock-in detection and being amplified with an erbium-doped fiber amplifier (EDFA). The other was used as the probe light, after passing two EDFAs, a single-sideband modulator (SSBM), and a polarization scrambler (PSCR). The SSBM was employed with a microwave generator (MG) and a proper DC bias control to suppress the carrier (pump) and the anti-Stokes component of the two first-order sidebands and to maintain a stable frequency downshift from the pump light. This frequency downshift was swept from 2.5 GHz to 3.5 GHz with a period of 300 ms to obtain the BGS of the PFGI-POF, which is observed approximately at 2.8 GHz. The suppression ratio of the other frequency components was kept higher than 25 dB, as shown in Figure 2.  
The PSCR, which can modulate the polarization state at 1 MHz, was inserted to suppress the polarization-dependent fluctuations of the signal. The PFGI-POF and the silica SMFs were butt-coupled [27] with the gaps filled with index matching oil (\(n=1.46\)) to minimize the Fresnel reflection. The Stokes  
Figure 1: Experimental setup for observing stimulated Brillouin scattering (SBS) in perfluorinated graded-index polymer optical fibers (PFGI-POFs) with pump-probe technique: LD, laser diode; EDFA, erbium-doped fiber amplifier; FG, function generator; FUT, fiber under test; IM, intensity modulator; ISO, isolator; LI-A, lock-in amplifier; MG, microwave generator; OSC, oscilloscope; PD, photo-detector; POF, polymer optical fiber; PSCR: polarization scrambler; SSBM, single-sideband modulator; VOA, variable optical attenuator.  
Figure 2: Measured optical spectrum of the SSBM output when the frequency of the MG was set to 2.83 GHz.  
light was adjusted in power with a variable optical attenuator (VOA), and converted to an electrical signal with a 125-MHz photo-detector (PD). After passing a lock-in amplifier (LI-A) with a chopping frequency of 5.018 MHz and a time constant of 10 ms, the electrical signal was observed as a BGS with an oscilloscope (OSC) synchronized with the frequency sweep of the SSBM.
## 3 Experimental Approach
### Voltage to Meter Conversion  
Reflected and transmitted signals received by the transducer from the interface are in units of voltage, therefore, the second order nonlinear parameter for the reflected ultrasound from the interface \(\gamma\) is in V\({}^{-1}\). To compute the nonlinear stiffness \(K_{2}\), the nonlinear parameter for the reflected ultrasound amplitude from the interface \(\gamma\) must be in units of meters. A method for determining the amplitude (in meters) from the piezo voltage was required. A Laser Doppler vibrometer (Polytec with laser sensor head OFV 354 and the vibrometer controller OFV 2500) was employed, as shown in Figure 6. A laser beam was incident on the free surface of the aluminium block with a transducer connected to the back face. While the transducer generated longitudinal waves in the aluminium block at different excitation voltages from 70 V to 350 V at center frequency 2 MHz, the laser beam was reflected from the free surface and captured by the laser sensor head. The captured laser beam returns velocity information. A numerical integral with respect to time results in the displacement of the surface in meters. The same signal processing procedure as explained in Section 3.3 was used to measure the displacement of the surface at different piezo voltages and hence incident amplitudes.  
Figure 7 presents the linear variation of amplitude of the reflected signal from the interface in voltage unit against the displacement of solid-air contact in meter. The slope of the curve \(\delta\) (V/m) gives voltage to meter conversion [47]:  
\[\gamma\left(\text{m}^{-1}\right)=\delta\left(\frac{\text{V}}{\text{m}}\right) \gamma\left(\text{V}^{-1}\right) \tag{41}\]