esa-sceva/satcom-chunk-collection huggingface dataset:
# A Review of Current Trends, Techniques, and Challenges in Large Language Models (LLMs)
## 7 Scalability
### Model Depth (Network Layers)  
In LLMs, network width is captured by the parameter size (hidden representation dimension), whereas network depth is the number of self-attention layers. Previous studies have indicated that increasing the network depth is the same as increasing the network representation. However, recent studies, such as [98], confirm the contrary. For instance, deepening is not favorable over widening for smaller network sizes. That is, when the width of the deeper network is not large enough, it cannot use its excess layers efficiently. Meanwhile, the transition into depth efficiency is clearly demonstrated when the network width is increased. It was shown in [98] that the transition between depth-efficiency and depth-inefficiency regimes exponentially depended on the network's depth. From a certain network width onwards, increasing the network depth does help improve efficiency. However, if the depth is increased with the network width, then it leads to efficiency. So, first, the width of the network must be chosen appropriately to leverage the full extent of the power brought by the depth of the network. For a given parameter budget, there is an optimal depth. So, for the same parameter budget, the deeper network performs better.  
As per the proposed theory in [98], the optimal depth for GPT3's 175 B parameters should have been 80 layers instead of 96. As per [98], Jurassic-1 [31] used 76 layers for the 178 B parameter model and found a significant gain in runtime performance. Using the same hardware configuration compared to GPT3, Jurassic-1 had 1.5% speedup per iteration and 7% and 23% gain in batch inference and text generation. Also, by shifting computation resources from depth to width, more operations can be performed in parallel (width) rather than sequentially (depth).  
Additionally, Ref. [97] mainly focused on the upstream (pretraining) loss, whereas [62] found that scaling laws differ in upstream and downstream environments. For instance, in addition to model size, model shape matters for downstream tasks, and it is recommended to increase the model's depth (DeepNarrow strategy) before uniform scaling of any other dimensions. This demonstrates that such redesigned models were able to achieve a similar performance to the T5-base model but with 50% fewer parameters and 40% faster training.
## 6 Building Mode Availability Layer  
On a particular section of parallel roads, there are only a certain number of modes available and ANNs can utilize this additional input which can further aid the ANNs classifying capabilities. In this paper, a newly-developed type of GIS map layer in raster format, which contains binary (1 for available and 0 for not-available state) information is created. Mode-specific vector map layers are generated by selecting a particular subset of road networks corresponding to the availability of the mode throughout the map. Then, the newly-generated vector map is transformed into a raster map with 10 m \(\times\) 10 m cells, where cells within a 100-m buffer of the road networks available with the particular mode are turned on. Once the newly-formed mode availability (MA) layers are prepared and ready to be accessed for each mode, raw data associated with any particular coordinates of GPS data from the field can be matched against those MA layers for checking the availability of each mode at that location (i.e., geocentric coordinates of \(x\), \(y\), \(z\)). This would form a new set of binary input factors to further aid the mode detection processes with ANNs. Figure 4 illustrates the concept of MA layers for each mode in a raster format. An auto mode is enabled in all sections in Figure 4 while the transit mode is available via points A-B-D-C and the metro mode is available via points C-D-E.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Modes** & **Average Number of Satellites in View** & **Average Magnitude of Magnetic Field (\(\mu\)T)** \\ \hline Auto & 8 & 49 \\ Bus & 6 & 55 \\ Truck & 9 & 45 \\ Tram & 6 & 138 \\ Walk & 8 & 38 \\ Bike & 7 & 33 \\ Boat & 10 & 25 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Distinctive input variables for each mode.  
Figure 4: Mode availability (MA) layers in a raster format.  
ANNs are generally categorized as a \"black box\" approach, referring to the fact that there are no well-specified analytical relationships between input and output as mathematical functions. The method is also data-driven. To some, those facts may be viewed as major flaws as the approach is missing the understanding of phenomenon from the perspectives of intuitive mathematical reasoning. However, this very black-box approach indeed offers pattern recognition power that explorers search spaces that analytically approaching minds would not have even recognized they existed. In brief, the proposed MA approach produces binary (1 or 0) values for each cell on a raster map. When raw data are collected from within those \"enabled\" regions for a certain available mode, it gives additional stimulation to the ANNs and forces its mode classifying estimations towards the actual mode during the training phase. In other words, the extra stimulation would still help ANNs to estimate the actual mode more often than the case without the additional input, even when the output layer lists other non-available dummy modes among its output mode options.
## 4 Trusted Network Secure Routing Algorithm
### The Specific Process of the MPASR Algorithm  
For the calculation of the reputation values and the application of ant-colony optimization, it is necessary to release many ants onto each path to record the individual attribute information for each node. Then, the optimal path is selected based on the pheromone values. The specific algorithm for implementing this process is shown below, and the specific flow chart is shown in Figure 5.  
1. Initialize the parameters. The maximum number of iterations is \(C_{max}\). Use the variable \(c\) to record the number of iterations performed during the execution of the algorithm; its initial value is 1. \(m\) ants will eventually be randomly placed at the \(n\) sensor nodes. Use the variable \(k\) to record the cumulative number of ants; its initial value is 0.
2. Increase the number of ants: \(k=k+1\). When \(k>m\), begin a new round of path selection.
3. Based on the node attribute information (node reputation value, residual energy and transmission delay), calculate the pheromone value and insert it into Equation (1) to calculate the random proportion rule to be used to select the next-hop node.
4. After selecting the next-hop node, record the corresponding node attribute information needed to calculate the comprehensive pheromone value. If the next-hop node is the destination node, then terminate the query process and begin the traversal of the next ant; otherwise, continue to choose the next-hop node as described in Step 3.
5. According to the pheromone update Equation (2), update the amount of pheromone on the path, and return to Step 2.
6. For each sensor node, calculate the credibility, the path length, and the remaining energy, and select the optimal path after the current iteration.
7. Update the number of iterations: \(c=c+1\). If \(c>C_{max}\), go directly to Step 8 and terminate the algorithm; otherwise, go to Step 2 to start a new round of path selection.
8. Compare the paths selected over all rounds and select the optimal one as the output. The algorithm ends.  
Figure 5: Flow chart of the proposed wireless sensor network algorithm based on ant-colony reputation values.
## 5 Sensitivity Analysis  
Sensitivity analysis evaluates the extent to which a model responds to variations in its parameters. For a given model \(P\), \(P=F[x_{1},\,x_{2},\,\ldots,\,x_{n}]\), the sensitivity function \(S_{i}(x_{i})\) of a parameter \(x_{i}\) can be determined as  
\[S_{i}(x_{i})=\frac{|\Delta P|/P}{|\Delta x_{i}|/x_{i}}=\left|\frac{\Delta P}{ \Delta x_{i}}\right|\frac{x_{i}}{P}\;i=1,2,\ldots,n \tag{12}\]  
If \(\Delta x_{i}/x_{i}\) is small, Equation (12) can be written as  
\[S_{i}(x_{i})=\left|\frac{\partial P}{\partial x_{i}}\right|\frac{x_{i}}{P}\;i= 1,2,\ldots,n \tag{13}\]  
Therefore, based on Equations (12) and (13), the sensitivity of Model 1 and Model 2 to parameters can be analyzed. For a given slope model, the yield acceleration is constant and can be determined using Equation (2). Hence, this section examines the sensitivity of Model 1 and Model 2 to \(PGA\) and \(I_{a}\) under different yield accelerations. As shown in Figure 9, under a specific state \(P^{\star}\) (Figure 8), the sensitivity factor \(S_{i}^{\star}\) of Model 1 remains constant, indicating \(S_{i}^{\star}(PGA)>S_{i}^{\star}(I_{a})\). Similarly, in Model 2, \(S_{i}^{\star}(I_{a})\) is also a constant. However, as \(a_{y}\) increases, \(S_{i}^{\star}(PGA)\) exhibits a nonlinear trend. Notably, when \(a_{y}/PGA\) approaches 1, \(S_{i}^{\star}(PGA)\) increases rapidly, indicating that Model 2 becomes highly sensitive to PGA in this range. Additionally, Figure 9 shows that if \(a_{y}/PGA\) is relatively small, \(PGA\) and \(I_{a}\) have a similar influence on the predictive results of Model 2. However, as \(a_{y}/PGA\) increases beyond a certain threshold, the impact of \(PGA\) on Model 2's predictions significantly exceeds that of \(I_{a}\).
## 2 Numerical Modeling
### Governing Equations  
The first assumption of the current work is steady-state conditions for fluid flow and heat transfer, while the second assumption is that the fluid has constant properties. Hence, the fluid is considered to be incompressible. Three equations govern the current problem's  
Figure 1: The physical domain of a ribbed tube with a zigzag configuration: (**a**) isometric view; (**b**) front view.  
The average heat transfer coefficient over the distance from \(\mathrm{x_{1}}\) to \(\mathrm{x_{2}}\) can be calculated as [35]:  
\[\mathrm{h_{av}}=\frac{1}{\left(\mathrm{x_{2}}-\mathrm{x_{1}}\right)}\int\limits_{ \mathrm{x_{1}}}^{\mathrm{x_{2}}}\mathrm{h_{x}}\mathrm{d}\mathrm{x} \tag{8}\]  
The average Nusselt number can be calculated from the average heat transfer coefficient as [35]:  
\[\mathrm{Nu_{av}}=\frac{\mathrm{h_{av}}\mathrm{d}}{\mathrm{k}} \tag{9}\]  
where k is the thermal conductivity of the flowing fluid. The friction factor is calculated over the distance \(\mathrm{x_{2}}-\mathrm{x_{1}}\), as [36]:  
\[\mathrm{f}=\frac{2\mathrm{d}(\mathrm{P_{1}}-\mathrm{P_{2}})}{\rho\mathrm{u^{2} }(\mathrm{x_{2}}-\mathrm{x_{1}})} \tag{10}\]  
In order to evaluate the performance of the zigzag configuration in comparison to the traditional ribbed tube configuration, where the rib heights are the same, the performance evaluation criterion index, PEC, was applied. The performance evaluation criterion is a method for weighing the thermal enhancement of different cases at the same pumping power. [PERSON] [37] and [PERSON] [38] coined an expression for the PEC. The PEC has been adapted in previous work [8; 24], and in [PERSON] et al. [39], which mentions many references for studies that applied the PEC. The PEC can be evaluated as:  
\[\mathrm{PEC}=\frac{\left(\mathrm{Nu}/\mathrm{Nu_{s}}\right)}{\left(\mathrm{f }/\mathrm{f_{s}}\right)^{1/3}} \tag{11}\]  
where \(\mathrm{Nu_{s}}\) is the Nusselt number of a fully developed smooth tube, and \(\mathrm{f_{s}}\) is the corresponding friction factor. The friction factor for fully developed smooth tubes is a function of the Reynolds number and can be evaluated based on the Petukhov correlation in the range \(3000\ \leq\mathrm{Re}\leq 10^{6}\)[40] as:  
\[\mathrm{f_{s}}=\left(0.79\ \mathrm{ln}\ \mathrm{Re}-1.64\right)^{-2} \tag{12}\]  
The Nusselt number value of a smooth tube depends on the Reynolds number, the Prandtl number, Pr, and the friction factor, as suggested by the Gnielinski correlation, such as [40]:  
\[\mathrm{Nu_{s}}=\frac{\left(\mathrm{f_{s}}/8\right)\left(\mathrm{Re}-1000 \right)\ \mathrm{Pr}}{1.07+12.7\left(\mathrm{f_{s}}/8\right)^{1/2}\left(\mathrm{Pr}^{2 /3}-1\right)} \tag{13}\]  
Another widely used correlation to find the Nusselt number for smooth tubes and turbulent fully developed flow is the Dittus-Boelter correlation. When the working fluid is heated, the correlation can be presented as [35; 41]:  
\[\mathrm{Nu_{s}}=0.023\ \mathrm{Re}^{0.8}\ \mathrm{Pr}^{0.4} \tag{14}\]
# Robust and Accurate Recognition of Carriage Linear Array Images for Train Fault Detection
## 3 Methodology
### Template Matching-Based Image Recognition Method  
This paper proposes a template matching-based image recognition algorithm, consisting of four main steps: image-to-point conversion, point set registration, matching quality evaluation, and score ranking. As shown in Figure 4c, the process begins by converting the target detection results of the line-scan carriage images into point sets. In Figure 3, different colored rectangles represent different detection targets. Based on predefined criteria, only the center coordinates of the bounding boxes are retained and no labels are assigned to the points, forming a two-dimensional point set. This step corresponds to Figure 4b. The same procedure is applied to both the image to be recognized and the template image, yielding a scene point set and a target point set, respectively. The image to be recognized may exhibit local nonlinear scale distortions, while the template image is a standard scale image for each category.  
Figure 4: Framework of the image recognition algorithm. (**a**) Shows four different types of images corresponding to the same carriage model. (**b**) Illustrates the transformation from the image to a point set. (**c**) Depicts the process of retrieving the matching image category from the template image library.  
The algorithm then performs point set registration for the specified scene point set and the template point sets of each category, aiming to align the horizontal coordinates of the point sets. Subsequently, each registration result undergoes a quality evaluation to verify the matching status between point sets and determine the success of the match, scoring each accordingly. Finally, all matching scores are ranked and the template category with the highest score exceeding a set threshold is selected as the recognition result.  
The key to the recognition process is accurately finding the target point set that matches the scene point set. The core challenge is addressing the point set registration problem under local nonlinear scale distortions in the horizontal direction of the line-scan carriage images. Due to the unique nature of line-scan carriage images, the initial vertical coordinates of correctly paired target and scene point sets are generally aligned. To improve point set registration efficiency, only horizontal alignment needs to be addressed, using an appropriate matching quality evaluation method to ensure accurate carriage category recognition. To achieve this, a coordinate transformation equation based on a weighted radial basis function is proposed, along with a numerical solution. Additionally, a specific point set matching quality evaluation method is provided. The following sections will detail these core components.
# Finding Optimal Manipulator Arm Shapes to Avoid Collisions in a Static Environment
## 4 Optimization Algorithms  
Given the robot's kinematic structure and simulation model described above, the goal is to find the shape of the lower and upper arms of the robot with the following conditions:  
* The trajectory and the layout of the workplace, including the location of the robot base, are fixed;
* The robot must be able to reach all points of the trajectory without collisions;
* The dimensions of the robot should be as small as possible.  
The shape of each the lower and upper arm is described by the following parameters:  
* The linear distance between the boundary joints (\(|AB|\) in Figure 3);
* The amount of bending (\(|CD|\) in Figure 3);
* The direction of bending (vector \(\vec{e}\) in Figure 3).  
Two methods were proposed, implemented, and tested: an iterative algorithm that relies on an analysis of the character of the collisions to choose the next step of shape modification, and an algorithm based on the particle swarm optimization (PSO) method that does not require any additional information about the collisions.
## 1 Introduction
Other works have been focused on the enhancement of deep acoustic models, where a sequence of local feature vectors is squeezed into a single global context vector [22], representing both speaker and environment information. In addition, model agnostic meta-learning has also been applied to rapidly adapt ASR models on cross-accented speech [23]. Other recent deep-learning-based techniques, such as Domain Adversarial Training (DAT) [24], have demonstrated that the model is able to reuse a latent space to improve performance on unseen input domains. Acoustic features must be robust to model the wide variety of speaker characteristics [25] and can play a relevant role in avoiding the bias in ASR systems with regard to diversity in gender, age, regional accents, and non-native accents, as was reported in Ref. [26]. To this end, DAT has been applied to ASR tasks by learning features invariant to different conditions, such as acoustic variabilities [27; 28], accented speech [29], and inter-speaker feature variability [30]. In this paper, our aim was to contribute to the comparison of both hybrid and end-to-end ASR systems under the conditions of the IberSpeech-RTVE 2020 Speech-to-Text Transcription Challenge [31]. This can be considered one of the aforementioned complex scenarios containing a variety of TV shows and broadcast news, in different noisy environments and challenging scenarios, such as TV debates. For this purpose:  
* We firstly studied state-of-the-art techniques for hybrid and end-to-end ASR systems;
* We report the use of data augmentation techniques to improve our Kaldi-based hybrid ASR system presented in the IberSpeech-RTVE 2018 edition [32];
* Then, we evaluated a baseline end-to-end system on a real TV content dataset. We chose PyChain because it is based on the state-of-the-art LF-MMI approach, for which good results have been previously reported;
* Finally, looking to improve the end-to-end ASR system, we propose the use of DAT to learn features invariant to the environmental conditions and TV show format. Thus, we developed a novel improved version of the PyChain baseline including DAT. Thisimplementation allowed us to compare the performance of both end-to-end systems in the case of having low-computational or -speech data resources. The rest of the paper is structured as follows. In Section 2, we describe the architecture of the ASR systems: a Kaldi-based hybrid ASR and two PyChain-based end-to-end systems. Section 3 explains the experimental protocols we followed under the IberSpeech-RTVE 2020 Challenge. Results are shown and discussed in Section 4. Section 5 presents related works comparing our systems with those submitted to IberSpeech-RTVE 2020 and other approaches in prior work. Finally, we present our conclusions in Section 6.
# A Multipath Data-Scheduling Strategy Based on Path Correlation for Information-Centric Networking
## 5 Evaluation
### Comprehensive Evaluation  
As shown in Figure 11, to bring the simulation closer to the real network environment, we use two real-world backbone topologies from the dataset provided by Topology Zoo [40], namely, BT North America and GEANT. The default bandwidth between nodes in the topology is uniformly distributed between 25 Mbps and 125 Mbps, the default delay between nodes is set to 1 ms and default loss rate is set to 0%. The default packet size is set to 1250 bytes, and each router can accommodate 200 packets. In addition, we select 10 nodes with the highest BC in these two topologies as relay nodes and randomly select 20 groups of sending and receiving node pairs. Each group of node pairs uses a multipath method to transmit 400 M data, among which the first 200 M of data is used to collect the path correlation information, and the last 200 M of data is used to collect the experimental results. In addition, we set the chunk size to 128 KB, 256 KB, 512 KB, 1 MB and 2 MB to evaluate the performance of each method under different scheduling granularities.  
Figure 12 shows the average throughput of each group of node pairs under different scheduling granularities in these two topologies. The results in Figure 12a,b are consistent. We can see that the average throughput of the RRDS is always lower than the other methods under different scheduling granularities. This is because RRDS mechanically distributes data to each sub-path in a polling manner, which leads to the problem of an unbalanced load. In addition, compared with the LRFDS and FDPS, our proposed PSDS can maintain the highest average throughput, which also reflects the effectiveness of our proposal.  
Figure 11: Simulation topology 2: (**a**) BT North America; (**b**) GEANT.  
Figure 10: Proportion of out-of-order chunks under different bandwidths.  
Figure 13 shows the proportion of the out-of-order chunks of the four scheduling strategies under different scheduling granularities in these two topologies. Similarly, the results of Figure 13a,b are consistent. It can be observed that under different scheduling granularities, the proportion of out-of-order chunks of the RRDS is always higher than that of the other three methods. Compared with RRDS, the LRFDS and FDPS alleviate the out-of-order problem to a certain extent by perceiving the path state through the RTT and forward delay. Additionally, our proposed PSDS is able to maintain a relatively low level of out-of-order proportion compared to other methods, indicating that our proposal can achieve significant performance in real network topologies.