esa-sceva/satcom-chunk-collection huggingface dataset:
## 4 Model Architecture Design
### Model Architecture Parameter Configuration
#### 4.1.2 Computational Power Requirements  
To calculate the FLOPs (floating-point operations) required for a Transformer model, the focus is on multiplication and addition operations. Each layer of self-attention and FFN generates a significant number of multiplications and additions. For self-attention, the complexity of each head is approximately O(seq_len\({}^{2}\)\(\times\) d_k), where d_k = d_model/n_head [37]. For 8 heads, each layer is roughly 8 \(\times\) seq_len\({}^{2}\)\(\times\) d_model. For 8 layers, it is 8 \(\times\) seq_len\({}^{2}\)\(\times\) d_model. The complexity of FFN is 4 \(\times\) d_model\({}^{2}\). The total complexity is (8 \(\backslash\) times seq_len\({}^{2}\)\(\backslash\) times d_model + 8 \(\backslash\) times d_model\({}^{2}\)\(\backslash\) times n_(layer)). With this approximate FLOPs requirement, a suitable GPU can be chosen based on its FLOPs performance. To calculate the required GPU memory size more precisely, note that actual memory requirements are influenced by various factors, including framework implementation details, mixed-precision training optimization techniques, batch size, etc. We provide estimates for the model.  
* [rightmargin=*]
* [rightmargin=*]
* Backward Propagation: Estimated as twice the forward pass, approximately 469.76 MB.
* Additional Overhead: Including gradients, optimizer states, etc., assuming additional occupancy is similar to the model parameters, but without specific parameter quantities, it generally accounts for a small portion.  
Comprehensively considering, the memory requirement would fall within the range of 150 MB to 100 MB (forward + backward), which translates to approximately 0.15 GB to 0.1 GB in GB units. In reality, especially considering the additional overhead of deep learning frameworks, batch normalization, storage of model parameters, optimizer states, and dynamic allocation during actual training, the required memory is often much larger than the directly calculated value. Therefore, we consider having at least 16 GB to 8 GB of GPU memory to ensure stability and flexibility during training, while for larger scales or batch sizes, a GPU with 12 GB or higher memory would be more suitable.  
Based on the above assessment of hardware requirements, we adopted a single NVIDIA GeForce RTX 4090 graphics card, which has 16,384 CUDA cores and 24 GB of memory, providing the parallel computing power required for our experiments. Therefore, we chose it as the core computing power hardware for model training.
# Particle Swarm Optimisation in Practice: Multiple Applications in a Digital Microscope System
## 6 Image Distortion Correction
### Affine and Perspective Distortion  
Affine (linear) and perspective (non-linear) distortions are shown in Figure 19, including a 3 \(\times\) 3 matrix that can be used to carry out that transformation. The new location (x\({}^{\prime}\), y\({}^{\prime}\)) of each pixel is attained by multiplying the matrix by the current location of each pixel. Equation (19) depicts a pixel rotation operation. A 1 is added to the (x, y) and (x\({}^{\prime}\), y\({}^{\prime}\)) pixel locations to allow matrix multiplication.  
\[\begin{bmatrix}x^{\prime}\\ y^{\prime}\\ 1\end{bmatrix}=\begin{bmatrix}cos\theta&sin\theta&0\\ sin\theta&cos\theta&0\\ 0&0&1\end{bmatrix}*\begin{bmatrix}x\\ y\\ 1\end{bmatrix} \tag{19}\]  
Figure 19: Affine and perspective distortions.  
Figure 18: Optical decentring.  
All of these matrix operations can be combined into a single 3 \(\times\) 3 matrix known as a homography matrix Equation (20). This single matrix with 8 degrees of freedom can fix all the following distortions:  
* Rotation.
* Shearing.
* Translation.
* Scaling.
* Perspective.  
\[\begin{bmatrix}x^{{}^{\prime}}\\ y^{\prime}\\ 1\end{bmatrix}=\begin{bmatrix}h_{11}&h_{12}&h_{13}\\ h_{21}&h_{22}&h_{23}\\ h_{31}&h_{32}&h_{33}\end{bmatrix}*\begin{bmatrix}x\\ y\\ 1\end{bmatrix} \tag{20}\]
## 5 Experiments
### Ablation Experiment  
To validate the necessity of each key module in SFPT, we performed ablation experiments, with the results presented in Table 1. As shown in Table 1, supervised learning, Zernike modes, and the pixel and channel self-attention mechanism in the transformer network all contribute to varying degrees of improvement in the reconstruction results.  
Figure 8: The visual comparison of the different unwrapping results in bullfrog erythrocytes dataset. (**a**) FPM reconstruction results. (**b**) FPM reconstruction unwrapping results. (**c**) SFPT wrapping reconstruction results. (**d**) SFPT unwrapping reconstruction results.  
Figure 7: The visual comparison of the reconstruction results for different phase ranges between FPM and SFPM.
# Design and Investigation of Capsule Networks for Sentence Classification
## 4 Experiments, Results and Discussion
### Baseline Models  
The CNN model was implemented with: Adam optimizer with learning rate of 0.001, batch size of 100, dropout of 0.8 and 2500 iterations. Similarly, the LSTM model was implemented with: Adam optimizer with learning rate of 0.001, batch size of 150, dropout of 0.8 and 100 epochs. These hyperparameters were chosen after performing exhaustive experiments to generate best results.  
In addition to normal testing, to test whether the networks learn the sequential order of words or not, we test both networks with sentences in test dataset where words have been randomly shuffled. If the network learns the sequential order of words, we expect the accuracy of the network with word-shuffling to be lower than the accuracy of the network with no shuffling. The results of our experiment on these two models are summarized in Table 2.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Dataset** & **No. of Class** & **Dataset Size** & **Max. Sent. Length** & **Vocab. Size** \\ \hline MR & 2 & 10662 & 56 & 18758 \\ TREC & 6 & 5952 & 37 & 8760 \\ SST-2 & 2 & 9613 & 53 & 16185 \\ Subj & 2 & 10000 & 120 & 21323 \\ MPQA & 2 & 10606 & 36 & 6236 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Details of datasets used in the experiments.  
Figure 4: Baseline CNN model.  
As can be seen from Table 2, for most datasets, the difference in accuracy between normal and shuffled test data for the CNN network is not much compared to that of the LSTM network. From this, we argue that the CNN model didn't learn the spatial information (the exact order of words) of the input data very well. This is expected as max-pooling in CNN is problematic in keeping the spatial order. In most cases, the LSTM network shows a superior performance to the CNN model. SST-2 dataset is presented in both phrase-level and sentence-level representations. The training is done on both phrase and sentence-levels while the test is performed on sentence-level inputs only.
## 3 DG-Darts  
### Vote Dispersion Problem  
For architecture search, DARTS [3] generates a search network with a stack of eight cells in which every operation in the search space is combined, and this combination of operations is the mixed operation. There can be multiple instances of mixed operations in DARTS. Using the search network, the architecture parameter \(\alpha\) of the mixed operations changes as the training of the dataset progresses. When the training is complete, the operation with the largest \(\alpha\) is selected among \(k\) operations in the search space, where \(k\) is the number of predefined operations; \(k=8\) in this study. In other words, the operation of the cells is decided with the largest \(\alpha\).  
\[o_{i,j}=arg\max_{o\in\mathcal{O},\mu\
eq none}\alpha_{o}^{i,j} \tag{1}\]  
For the sake of the reader's convenience, Algorithm 1 for DARTS is cited from [3].  
```
1:Create operation search space \(\delta^{(i,j)}\) parameterized by \(\alpha^{(i,j)}\) for each edge \((i,j)\)
2:whilenot convergeddo
3: Update architecture \(\alpha\) by descending \(\
abla_{\alpha}\mathcal{L}_{val}(w-\
abla_{\alpha}\mathcal{L}_{train}(w, \alpha),\alpha)\)
4: Update weights \(w\) by descending \(\
abla_{w}\mathcal{L}_{train}(w,\alpha)\)
5:endwhile
6: Derive the final architecture based on learned \(\alpha\).
```  
**Algorithm 1** DARTS--Differentiable Architecture Search  
A negative phenomenon can occur: the weights of the appropriate operation could be dispersed, and irrelevant operations for an edge can be selected. This occurs because an invalid operation can have a higher weight than an adequate operation. The weights of several similar, adequate operations are divided once all such operations are considered important. In this paper, we refer to these phenomena as the vote dispersion problem.  
**Definition 1**.: _Vote Dispersion: Votes for the weight of meaningful but similar operations happen to be dispersed, and the weight of a meaningful operation becomes lower than that of a meaningless operation._  
Thus, the possibility exists that meaningless operations can be selected because of the vote dispersion problem.  
Many NAS algorithms, including NASNet [1], search the search spaces to determine appropriate operations composing cells. Examples of such operations are convolution, pooling, and skip connections. There are groups of operations with similar computation outputs in the search space, such as {max pool and average pool}. The possibility exists of a vote dispersion problem under such conditions, and DARTS cannot avoid this problem because it uses the same search space as NASNet. In this study, our goals were to solve the vote dispersion problem experienced by DARTS, to increase the performance of the final architecture, and to decrease the search cost.
## 4 Results and Analysis
### Results
\begin{table}
\begin{tabular}{c c c c c c c c c c c c} \hline \hline  & \multicolumn{3}{c}{**Numerical Simulation**} & \multicolumn{3}{c}{**Theoretical Results**} & \multicolumn{3}{c}{**Results**} & \multicolumn{3}{c}{**Numerical Simulation**} & \multicolumn{3}{c}{**Theoretical Results**} & \multicolumn{3}{c}{**Results**} \\ \cline{2-10}
**Numbering** & \multicolumn{3}{c}{**Text Results**} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} \\ \cline{2-10}
**Table** & \begin{tabular}{c} **Maximum** \\ **Height/m** \\ **Ratio** \\ \end{tabular} & \begin{tabular}{c} **Crack** \\ **Height/m** \\ **Ratio** \\ \end{tabular} & \begin{tabular}{c} **Average** \\ **Relative** \\ **Ratio** \\ \end{tabular} & \begin{tabular}{c} **Medium** \\ **Height/m** \\ **Ratio** \\ \end{tabular} & \begin{tabular}{c} **Predication** \\ **Error/\%** \\ \end{tabular} & \begin{tabular}{c} **Medium** \\ **Height/m** \\ **Ratio** \\ \end{tabular} & \begin{tabular}{c} **Predication** \\ **Ratio** \\ \end{tabular} & \
\begin{tabular}{c} **Medium** \\ **Ratio** \\ \end{tabular} &
\begin{tabular}{c} **Predication** \\ **Error/\%** \\ \end{tabular} \\ \hline M1 & 78.5 & 157 & 82.7 & 16.5 & & M13 & 138.0 & 27.6 & 135.5 & 27.1 & \\ M2 & 97.6 & 19.5 & 81.6 & 16.3 & & M14 & 148.3 & 29.6 & 158.7 & 31.7 & \\ M3 & 99.1 & 19.8 & 97.7 & 19.5 & & M15 & 150.0 & 30.0 & 139.4 & 27.9 & 1.9 \\ M4 & 117.5 & 23.5 & 128.1 & 25.6 & & M16 & 157.2 & 31.4 & 164.7 & 32.9 & 1.9 \\ M5 & 128.4 & 25.7 & & & M17 & 170.5 & 34.1 & 175.8 & 35.4 & \\ M6 & 133.2 & 26.6 & 147.9 & 29.6 & & M18 & 177.9 & 35.6 & 182.9 & 36.6 & \\ M7 & 48.0 & 16.0 & 57.1 & 19.0 & & M19 & 64.6 & 21.5 & 90.6 & 30.2 & \\ M8 & 74.1 & 24.7 & 81.5 & 27.2 & & M20 & 98.8 & 32.9 & 112.6 & 37.5 & \\ M9 & 99.0 & 33.0 & 57.6 & 32.5 & & M21 & 118.1 & 39.4 & 142.1 & 47.4 & 17.0 \\ M10 & 118.0 & 39.3 & 127.8 & 42.6 & & M22 & 157.9 & 52.6 & 164.7 & 54.9 & 17.0 \\ M11 & 129.0 & 43.0 & 138.4 & 46.1 & & M23 & 171.3 & 57.1 & 176.9 & 59.0 & \\ M12 & 134.2 & 44.7 & 147.5 & 49.2 & & M24 & 178.2 & 59.4 & 183.1 & 61.0 & \\ \hline \hline \end{tabular}
\end{table} 
Table 4: The maximum development height of water-flowing fracture in 24 numerical models of coal seam overburden. According to Table 4 the relative error rate of the maximum height of water-flowing fracture calculated by the two methods used in this paper is 1.9-17.0%, and the average absolute error rate is only 8.15%. It can be seen that the accuracy of the two results is high.
## 2 Study Area and Site Investigations  
The study area is within the Perth metro area, the capital of Western Australia, located in the south west of this state. This area is in Perth basin in the western part of the Darling fault with its eastern part extended to the Darling Plateau (Yilgarm Craton) [37]. Figure 1 illustrates an overview of the study area. Also, Figure 1 illustrates an overview of pest control depots as potential areas for atrazine contamination. The site investigations in the observed atrazine-contaminated area (Dianella site) show that the soil in this area is mainly of a coarse-grained nature (mainly sand).  
A review of current literature on the effects of contamination on the mechanical characteristics of soil showed that the studies in contaminated soil have mainly been conducted on clayey soils ([28; 30; 31] amongst others). This huge body of literature indicates that contamination has almost no impact on coarse-grained soils due to their relatively high hydraulic conductivity characteristic. In fact, a permeable soil causes contamination to infiltrate to groundwater and no interactions occur. Also, the recorded contaminated groundwater in this area is another proof for this fact. Therefore, the sites contained clayey soils were considered for the field investigations. According to [38], the Guildford formation is a clayey formation that is mainly located in the eastern and southern parts of Perth. Therefore, field investigations were mainly focused in this region.  
The field investigations in the southern suburbs showed that some parts of the two areas: Armadale and Rockingham contain the clayey soil. Also, further studies showed that due to the rural nature of a large part of these two areas, many pest control companies currently provide services for the farms, gardens, and parks in these areas. Therefore, these two areas were selected as the most prone to atrazine contamination and the soils were collected from these areas. Figure 2 illustrates the locations of the investigated sites.  
Figure 1: Depot pest control locations in Perth (idea from [39]) and contaminated groundwater spots by atrazine [40].
# Numerical Analysis of Reinforcing Effect for Scissors-Type Bridge with Strut Members
## 3 Application of Strut Reinforcement
### Reinforcing Pattern  
Reinforcing effects of the strut member are examined numerically on the basis of the nonreinforcing state without any strut members. Table 1 shows the detail of reinforcing patterns. We focus on the number of horizontal and vertical reinforcing members and their setting position. The point is to find key horizontal and vertical reinforcing members.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Case** & **Horizontal mem.** & **Vertical mem.** & **Configuration** \\ \hline
0 & No & No & \\ \hline
1-1 & & No & \\
1-2 & Only & Center place & \\
1-3 & Lower place & Both side place & \\
1-4 & & All place & \\ \hline
2-1 & Lower place & No & - \\
2-2 & and & Center place & - \\
2-3 & Central & Both side place & - \\
2-4 & upper place & All place & - \\ \hline
3-1 & All & No & \\
3-2 & Lower place & Center place & \\
3-3 & and & Both side place & \\
3-4 & Upper place & All place & \\ \hline \hline \end{tabular}
\end{table}
Table 1: Numerical cases with different reinforcing patterns.  
Figure 6: Outline of the FE numerical model based on full-scaled MB: (**a**) two scissors unit based on MB1.0; (**b**) three scissors unit based on MB1.1.  
In the case of the two scissors units model, there are two different setting places for the horizontal reinforcing members. First, it is considered that there are only deck boards on the frame member. Hence, the horizontal members are applied to the bottom hinge parts (Case 1) and the upper and bottom hinge parts (Case 3). Then, the vertical members are given four patterns as: (1) nonmember, (2) center place, (3) both side places and (4) all hinge places.  
In the case of the three scissors units model, there are three different setting places for the horizontal reinforcing members. The horizontal members are applied to the bottom hinge parts (Case 1), the central upper and bottom hinge parts (Case 2), and the upper and bottom hinge parts (Case 3). Then, the vertical members are given four patterns as well as in the case of the two scissors units model. According to these different configurations, the reductions of the sectional forces and displacement are evaluated.
# A Time Bank System Design on the Basis of Hyperledger Fabric Blockchain
## 4 The Proposed Time Bank System
### Phase 2: Token Transferring  
This phase focuses mainly on token transferring, that is, transfer of the token between SR and SP after their services are exchanged.  
_(1) Token Transferring:_ The whole token transferring-related process in TC is shown in Figure 4. At first, a user executes an application to access TC and invokes the token transferring function embedded on token Chaincodes. In this case, after SR and SP exchange their services, SP wants to get the time credit for the service he or she provided. Due to the passivity of blockchain, a token will be transferred only when a user accesses it and invokes a smart contract on it. Since only SP has the motivation to get time coins, a token transfer must be activated by an SP. He or she can directly invoke the token transferring function in TC through an application and gets the time credit as a return. However, there are several rules to obey for preventing some possible fraud generated by SPs.  
Firstly, the system needs to check whether the time an SP requests the token transfer is later than the time that the corresponding services were exchanged. This means that a token transferring function should not be activated until the corresponding service's exchange time passes. Otherwise, the system must check whether SP and SR matched their services exactly. Secondly, at the time an SP is executing the token transferring function in TC, the application-related function defined in token Chaincodes should already have successfully obtained the service-related data, which are recorded on SC. Finally, the response to users' queries about SC should indicate that both service states of SP and SR are successful, and then the token transferring function transfers the time credit from SR's wallet to SP's wallet.  
_(2) Reward Function:_ An important goal of the time bank is to promote the willingness of supplying services; thus, the system needs to incentivize the participants to provide services; therefore, a reward function is designed in our system. Once a service-exchange is completed, the reward function will automatically count the service time that SP provided. In our system, every hour of service-providing will gain 0.1 units of time credit as a reward. Of course, the above two numbers are adjustable parameters. The process of our reward function is also shown in Figure 4.
# Recording a Hologram Transmitted over a Communication Channel on One Sideband
###### Abstract  
The paper presents experimental results on the recording and restoration of 3D holographic frames suitable for transmitting 3D holographic images with the frame rate required for TV images and a resolution of the Full HD standard and higher. The Patent RF No. 270582 proposed a method for compressing holographic information and transmitting it using a procedure similar to SSB over conventional communication channels. In this work, the holographic information of a 3D portrait of a person, transmitted and received via the Wi-Fi communication channel, was restored in the form of a rainbow hologram, as one of a variety of holograms, by the computer addition of the carrier spatial frequency, and then hologram was actually produced on a photoresist. This technology can be used to create a holographic phototelegraph or, if there is a dynamic holographic display, to create a holographic television and 3D augmented reality.  
Keywords:optics; holography; rainbow hologram; holographic television; 3D augmented reality +
Footnote †: journal: Applied
## 2 Our Proposal
### Static MSL Signs Classifier  
Once the data have been preprocessed, the machine learning algorithms shown in Figure 1 are trained to identify the most suitable model for classifying static MSL signs. Among the algorithms tested, the fully connected neural network (FC-NN) demonstrated the best performance. Leveraging its architecture, we utilized the output from the softmax layer as the classification result, where each output corresponds to a confidence value representing the probability associated with each class. This confidence value is used in step 4 of Figure 8.  
We used the Grid Search algorithm to obtain the final architecture of the FC-NN, together with its hyperparameters. Table 4 summarizes the tested models, their learning rate, and several epochs. We used ADAM as an optimizer for all the models, categorical cross-entropy as the loss function, and ReLU as an activation function. At the same time, the output layer always has 26 neurons with a softmax activation function. The best model is indicated in bold, which is selected in step 4 of Figure 1.  
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**FC-NN** & **Layer 1** & **Layer 2** & **Layer 3** & **Layer 4** & **LR** & **Epochs** & **Accuracy** & **F1** \\ \hline
1 & 20 & 30 & 40 & - & \(1\times 10^{-1}\) & 30 & 88.70\% & 87.93\% \\
2 & 20 & 30 & 40 & - & \(1\times 10^{-2}\) & 30 & 88.73\% & 88.18\% \\
3 & 40 & 50 & 40 & - & \(1\times 10^{-2}\) & 90 & 90.39\% & 90.02\% \\
4 & 40 & 40 & 40 & - & \(1\times 10^{-3}\) & 90 & 90.09\% & 89.67\% \\
5 & 30 & 40 & 40 & 50 & \(1\times 10^{-3}\) & 120 & 90.89\% & 90.74\% \\
6 & 30 & 40 & 50 & 60 & \(1\times 10^{-3}\) & 120 & 90.78\% & 90.46\% \\
7 & **70** & **80** & **70** & **26** & \(\textbf{1}\times\textbf{10}^{-3}\) & **120** & **91.09\%** & **90.91\%** \\ \hline \hline \end{tabular}
\end{table}
Table 4: Summary of FC-NN models.  
In contrast, the convolutional neural network (CNN) takes the cropped hand image as input, with the bounding box calculated during preprocessing. The performance of the CNN is evaluated to determine which type of input data yields the best classification results: MediaPipe's vector of key points or the cropped hand image.  
Table 5 outlines the architecture of the CNN, which consists of two main types of layers: convolutional and fully connected. Additionally, max pooling and flatten layers are included in the network design. The table specifies the number of neurons in each layer, the activation functions used (with ReLU applied in all hidden layers and softmax for the output layer), and the kernel sizes for the convolutional layers. These architectural choices are aimed at effectively capturing the spatial features in the input data to enhance classification accuracy.  
Table 6 shows the hyperparameters used during the CNN training. These values are similar to the FC-NN's, but we reduced the number of epochs because this model requires more computational resources than FC-NN.