esa-sceva/satcom-chunk-collection huggingface dataset:
# A Parameter Communication Optimization Strategy for Distributed Machine Learning in Sensors
## 2 Related Work
### Parameter Server System  
Smola proposed in 2010 a parallel topic model architecture [29], which used the idea of a Parameter Server System, which we call the first-generation parameter server system. The parameter server system is essentially a distributed shared memory system, where each node can access the shared global model parameters through the key value interface, as shown in Figure 1.  
Figure 1: The first-generation parameter server system.  
The model uses a distributed Memcached to store the parameters, where each worker node only retains part of the parameters which are required in computing, and they can synchronize global model parameters with each other in this model. However, this parameter server is only a prototype design, the communication overhead is not optimized, and it is not suitable for distributed machine learning.  
Industry has done a lot of work in improving the Parameter Server System. [PERSON] et al. proposed a second-generation parameter server system in 2012, and developed a deep learning system called DistBelief [12] based on the Parameter Server System. As shown in Figure 2, the system sets up a global parameter server. The deep learning model is distributed stored on worker nodes, the communication between worker nodes is not allowed, and the PS is responsible for the transfer of all parameters. The second-generation parameter server system can solve the problem that the machine learning algorithms are very difficult to be distributed, but because it does not consider the performance differences of each worker node, the utilization of each worker node in the distributed system is still not high.  
[PERSON] proposed a third-generation parameter server system in 2014 [30; 31]. As shown in Figure 3, the Parameter Server System provides a more general design, including a parameter server group and multiple worker nodes.  
Figure 3: The third-generation parameter server system.  
Figure 2: The second-generation parameter server system.  
Among them, each parameter server stores part of the global model parameters, and the server management node manages the entire parameter server group. Multiple worker nodes can run one or more different machine learning algorithms. Like the previous generation of the parameter server system, PS is responsible for the transfer of all parameters. Each worker nodes group sets up a scheduler to assign tasks to worker nodes and monitor the running status. If worker nodes are not responding, the dispatcher performs redistribution of the remaining tasks without re-starting the model training. The Parameter Server System solves the problem of low computing efficiency through scheduling worker nodes, but it can only add or remove the worker nodes. The strategy is too simple and does not consider the situation that the training process of the machine learning model could be disturbed by any unrelated tasks in the cluster.
# Optimization Design of Centrifugal Fan Blades Based on Bezier Curve Method
## 3 Multi-Objective Optimization Task  
### Blade Profile Parameter Design with Bezier Curve Method  
In this study, the voluteless centrifugal fan had arc-shaped, equal-thickness blades with the same profiles on both the pressure and suction surfaces. A fourth-order Bezier curve was used to fit the blade profile, and the coordinates of five Bezier curve control points were obtained. By changing the coordinates or the relative positions of these control points, the Bezier curve could be controlled, to change the shape of the blade.  
The Bezier curve is a vector-drawing method that uses \(n\) control points to determine the shape of the curve. The curve passes through the initial point \(A_{0}\) and the final point \(A_{n}\), while it only approximates the intermediate points \(A_{2}\) through \(A_{n-1}\) without actually passing through them. The expression for the fourth-order Bezier curve is  
\[B(t)=\sum_{i=0}^{4}A_{i}\cdot B_{4,i}(t),t\in[0,1] \tag{3}\]  
\[B_{4,i}(t)=\begin{pmatrix}4\\ i\end{pmatrix}t^{i}(1-t)^{4-i},i=0,1,2,3,4 \tag{4}\]  
The core idea of blade profile-fitting is to use the known curve points to deduce the control points of the Bezier curve. Suppose there are \(m\) discrete planar points \(Q_{m}\) on the target curve corresponding to parameters \(t_{m}\), and these discrete points are assumed to lie on the Bezier curve. That is, they satisfy  
\[Q_{m}=\sum_{i=0}^{4}A_{i}\cdot B_{4,i}(t_{m}),t\in[0,1] \tag{5}\]  
A matrix \(B\) is constructed for given parameters \(t_{0}\)-\(t_{m}\), where each row represents the computed Bezier basis functions for a specific parameter \(t_{i}\). The basis functions for a fourth-order Bezier curve are as follows:  
\[B_{i}(t)=\begin{bmatrix}(1-t)^{4}&4(1-t)^{3}t&6(1-t)^{2}t^{2}&4(1-t)t^{3}&t^{4 }\end{bmatrix} \tag{6}\]  
The Bessel function matrix \(B\) is constructed as follows:  
\[B=\begin{bmatrix}(1-t_{0})^{4}&4(1-t_{0})^{3}t_{0}&6(1-t_{0})^{2}{t_{0}}^{2}& 4(1-t_{0}){t_{0}}^{3}&{t_{0}}^{4}\\ (1-t_{1})^{4}&4(1-t_{1})^{3}t_{1}&6(1-t_{1})^{2}{t_{1}}^{2}&4(1-t_{1}){t_{1}} ^{3}&{t_{1}}^{4}\\ \cdot&\cdot&\cdot&\cdot&\cdot\\ \cdot&\cdot&\cdot&\cdot&\cdot\\ \cdot&\cdot&\cdot&\cdot&\cdot\\ (1-t_{m})^{4}&4(1-t_{m})^{3}t_{m}&6(1-t_{m})^{2}{t_{m}}^{2}&4(1-t_{m}){t_{m}} ^{3}&{t_{m}}^{4}\end{bmatrix} \tag{7}\]  
The control points through matrix operations are solved by  
\[BA=Q \tag{8}\]  
\[A={(B^{T}B)}^{-1}B^{T}Q \tag{9}\]  
where \(Q\) is the matrix of known data points:  
\[Q=\begin{bmatrix}A(t_{0})&A(t_{1})&\cdot&\cdot&\cdot&A(t_{m})\end{bmatrix}^ {T} \tag{10}\]and \(A\) is the matrix of control point:  
\[A=\begin{bmatrix}A_{0}&A_{1}&A_{2}&A_{3}&A_{4}\end{bmatrix}^{T} \tag{11}\]  
The suction surface profile was selected as the fitting target (red line in Figure 6), and several two-dimensional point coordinates were obtained from the original profile. The coordinates of the control points \(A_{j}\) (\(x_{j}\), \(y_{j}\)) were calculated based on Equations (6)-(11), and the 3D model could be rebuilt. The goodness of fit reached 99.6%, making it suitable for subsequent parametric design.
# Outlier Reconstruction of NDVI for Vegetation-Cover Dynamic Analyses
## 4 Discussion  
The NDVI contains important data for providing vegetation cover information and supporting environmental analyses. This article highlighted that accurately reconstructing NDVI value for cloud-covered pixels in optical remote-sensing imagery is an urgent issue in widening NDVI-based applications. The proposed approach is reliable and suitable for reconstructing outliers of NDVI from multitemporal optical remote-sensing images. This facilitates vegetation dynamic change detection and environmental analyses in cloudy areas. However, there are some limitations in this study that need to be further improved and optimized.  
First, optical images used in experiments were all acquired by the Landsat series satellites. The influence of data acquired by sensors of other Earth observation plans, such as MODIS, Sentinel, and SPOT on reconstruction results needs further experimental research and analysis. Moreover, wavelengths of infrared and near-infrared bands of sensors from different periods of the Landsat series were slightly different. In this experiment, we did not consider the possible impact of this difference on results due to the inability to perform in situ spectral measurements.  
There are few valid data for analysis from summer because of the majority cloudy weather in summer. Our proposed method can effectively reconstruct outliers in summer on the basis of acquired winter data. However, the impact of increasing valid data from summer on the reconstruction results remains to be assessed.  
The spatial resolution of all optical images was resampled to 30 \(\times\) 30 m according to the spatial resolution of the majority of images being 30 \(\times\) 30 m in the dataset (Table 1). On the one hand, optical remote-sensing images with resolution higher than 30 m were difficult to obtain in the early stage. On the other hand, field measurements could not be carried out  
Figure 6: Execution time and the RMSE values with different rank sizes of core tensor.  
Figure 7: ROC curves of SWTSA for different rank sizes (AUC: Area Under Curve).  
due to the impact of COVID-19. Therefore, different resolutions and coregistration errors had an unclear effect on the experimental results.  
Lastly, this paper focused on exploring NDVI reconstruction methods for cloud-covered pixels on the basis of the data structure of NDVI from clear skies and their changing characteristics; therefore, reasons for long-term NDVI changes were not analyzed and discussed. In future work, we aim to quantitatively discuss the possible effects on NDVI changes that land use changes, deforestation, and riverbed evolution. Meanwhile, in order to enhance the data fusion capability of the algorithm, we aim to implement the proposed algorithm on the basis of Google Earth Engine (GEE) [52; 53], a cloud computing platform, in further work.
# Energy-Efficient Connected-Coverage Scheme in Wireless Sensor Networks
## 5 Algorithm for the Energy Saved Maximization Problem
### Improved DEEC Protocol
#### 5.2.1 Adjust the Probability of CHs  
For the DEEC protocol, sensor node \(s_{i}\) generates a random number between \([0,1]\) and compares it with the threshold \(T(n)\). If the random number is less than \(T(n)\), then this node will be selected as the CH [30]. The \(T(n)\) is represented as  
\[T(n)=\left\{\begin{array}{cc}\frac{p_{i}}{1-p_{i}\left(r\bmod \frac{1}{p_{i}}\right)}&if\ n\in G\\ 0&otherwise,\end{array}\right. \tag{15}\]  
where \(p(i)=\frac{E_{i}(r)}{E(r)}\). \(E_{i}(r)\) is the residual energy of the sensor node \(s_{i}\) and \(E(r)\) is the average remaining energy of all the sensor nodes in the \(r\) round. \(r\) is the current number of rounds, \(G\) is the set of nodes that have not been selected as CHs. From (14), it can be found that only the residual energy of sensor nodes is considered.  
The CH is responsible to fuse and transfer data that are sensed from other nodes; therefore, it should have enough residual energy. At the same time, because the distance between the CH and the sink is also an important factor influencing the energy consumption, to save more energy, the distance to the sink should be also considered. As described above, the probability of each CH \(CH_{i}\) selection can be modified as  
\[T(CH_{i})=T(n)*\left(\alpha*\frac{E_{i}(r)*E(i)}{\mathbb{E}(r)*E(a)}+\beta*D\right) \tag{16}\]  
where \(E(i)\) is the initial energy of the node, \(E(a)\) is the average energy of all sensor nodes in the WSN. \(D=(1-\frac{d(i,\sin k)-d_{\text{max}}}{d_{\text{max}}-d_{\text{min}}})\). \(d(i,\sin k)\) represents the distance from node \(s_{i}\) to sink, \(d_{\text{max}}\) and \(d_{\text{min}}\) are the maximum and minimum distance from node \(s_{i}\) to sink, respectively. \(\alpha\) and \(\beta\) are the weight coefficients and \(\alpha+\beta=1\). When \(\alpha\) is a larger value, the residual energy of sensor nodes is a more important factor influencing the CH selection. When \(\beta\) is greater than \(\alpha\), the sensor node that nears the sink has more opportunity to be chosen as the CH.
# The Impact of Additive Manufacturing on the Flexibility of a Manufacturing Supply Chain
###### Abstract  
There is an increasing need for supply chains that can rapidly respond to fluctuating demands and can provide customised products. This supply chain design requires the development of flexibility as a critical capability. To this end, firms are considering Additive Manufacturing (AM) as one strategic option that could enable such a capability. This paper develops a conceptual model that maps AM characteristics relevant to flexibility against key market disruption scenarios. Following the development of this model, a case study is undertaken to indicate the impact of adopting AM on supply chain flexibility from four major flexibility-related aspects: volume, mix, delivery, and new product introduction. An inter-process comparison is implemented in this case study using data collected from a manufacturing company that produces pipe fittings using Injection Moulding (IM). The supply chain employing IM in this case study shows greater volume and delivery flexibility levels (i.e., 65.68% and 92.8% for IM compared to 58.70% and 75.35% for AM, respectively) while the AM supply chain shows greater mix and new product introduction flexibility, indicated by the lower changeover time and cost of new product introduction to the system (i.e., 0.33 h and \(\copyright\) for AM compared to 4.91 h and \(\copyright\)0,000 for IM, respectively). This work will allow decision-makers to take timely decisions by providing useful information on the effect of AM adoption on supply chain flexibility in different sudden disruption scenarios such as demand uncertainty, demand variability, lead-time compression and product variety.  
additive manufacturing; 3D printing; manufacturing flexibility; supply chain management; injection moulding; smart factory +
Footnote †: journal: Applied Sciences
# Application of Machine Learning for Fault Classification and Location in a Radial Distribution Grid
## 2 Machine Learning Tools and Signal Processing
### Signal Processing Techniques  
Applying any of the machine learning techniques on a given measured data, such as current, the signals should first be transformed into useful features.  
Wavelet Transform (WT)  
Traveling waveforms are non-periodic by nature, consisting of localized high-frequency oscillations. These oscillations are superimposed on the power frequency and its harmonics. WT is a suitable tool to analyze such a signal having time-varying frequency. There are two concepts in WT: scaling and shifting. Scaling is the process of stretching or shrinking the signal in time while shifting is delaying or advancing the onset of the wavelet along the length of the signals. Unlike short-time Fourier Transform, which uses fixed window size in the frequency-time plane, WT uses variable widow sizes. It means that the use of longer time intervals is possible when it is required to get more precise low-frequency information from the signal [1]. This makes WT more accurate in localizing the signal both in time and in frequency than short-time Fourier Transform. WT is more popular in power system applications.  
Based on how the wavelets are scaled and shifted, WT is divided into two: continuous WT (CWT) and discrete WT (DWT) [23]. CWT of a time domain signal, \(x(t)\), is defined by Equation (7):  
\[CWT\ (a,b)\ =\ \frac{1}{\sqrt{a}}\int_{-\infty}^{+\infty}x(t).\psi^{*}\ ( \frac{t-b}{a})\ dt \tag{7}\]To implement wavelet transform digitally, discrete wavelet transforms (DWT) is used, which is computed using Equation (8). In order to extract sub-band information from a transient signal, multi-resolution wavelet can be applied. _Daubechies_ wavelets is widely used in analyzing traveling waves [24]. These wavelets are found to have a closer match with the processed signal. Being more localized and compact in time makes them more suitable to analyze both short and fast transients. Complete reconstruction of the signal is also possible.  
\[DWT(k,n,m)=\frac{1}{\sqrt{a_{0}^{m}}}\sum_{n}x[n]\ \psi\ (\frac{k-nb_{0}a_{0}^{m}}{a_{0} ^{m}}) \tag{8}\]  
where \(\psi\) (\(t\)) is the mother wavelet, \(a_{0}^{m}\) and \(nb_{0}a_{0}^{m}\) are scaling factors (a and b in Equation (7)), and \(n\) and m are integers. The coefficients in a standard DWT are determined by sampling the corresponding CWT on a dyadic grid.  
DWT have simple implementation and less computation requirements compared to CWT, and as a result, it becomes the choice of many researchers to use DWT in analyzing power system signals. Thus, many researchers use DWT to analyze the behavior of voltage/current transients produced by fault events in distribution grids [25]. Additionally, DWT-based multi-resolution analysis offers an effective approach to inspect features of a signal at different frequency bands and is widely used for locating faults in a distribution network.
## 2 Theoretical Background
### Security in Wireless Sensor Networks  
In most cases, WSNs are not managed centrally, are deployed in unsuitable environments, and use unreliable channels of communication. Therefore, WSN's security mitigation cannot be relied on standard IT network solutions due to the complexity and interconnection of a wide range of devices and protocols, as well as the variety of routing protocols and services available. Hence, current security mechanisms in use are insufficient. Many techniques and trends have been used to achieve a certain range of security levels, such as trust management and encryption techniques using lightweight methods to have a low-cost encryption for low-power and constrained devices. Routing protocols might also be susceptible to serious attacks such as injecting false or malicious routing information into a network, resulting in delays or packet losses due to routing conflicts. Many solutions have been proposed to avoid routing attacks such as encryption and information correlation between multiple nodes [6].  
Any proposed protocol should compromise between two competing priorities: security and performance. Low energy consumption, processing, and storage usage are all sacrificed for better performance against security level and vice versa. Key management is one method that may be used to ensure the security of WSNs [21]. On the basis of security policy, it is described as a set of activities and mechanisms that facilitate key distribution and adhere to the criteria of the keying process among nodes. It is necessary to generate, maintain, distribute, protect, and control the use of cryptographic keys. Since they have the capacity to update the keys in the sensor nodes, the techniques of key management are classified as either static or dynamic [22]. In static key management, the encryption key will be stored in a secret manager, the principles of the encrypted keys are identified before they are distributed, and the keys themselves remain unaltered for the duration that the network is operational, which makes it more vulnerable to attacks. Conversely, in dynamic key management, keys are updated frequently depending on the network policies.  
Group key management is broadly used in various modern collaborative and distributed applications to provide security [23]. Based on [22], group key management provides good performance in terms of energy and memory consumption and security level, in addition to low communication overhead with high scalability while it is used in WSNs.
# Exploring the Data Efficiency of Cross-Lingual Post-Training in Pretrained Language Models
## 3 Proposed Method
### Selecting Parameters to Transfer  
A language model consumes a word sequence and emits a contextualized vector representation of it. Then these vectors can be used to assign some probability to a word or to perform some tasks such as sequence classification.  
More formally, assume we have an input as a sequence of tokens \(\mathcal{T}=[t_{1},t_{2},\cdots,t_{n}]\), \(1\leq t_{i}\leq V\ \in\mathbb{N}\), where \(V\) is the vocabulary size. Then, the output of a language model \(LM\) is given by  
\[\mathbf{E}_{L_{S}}=[\mathbf{e}_{1}^{S};\mathbf{e}_{2}^{S};\cdots;\mathbf{e}_{ V}^{S}],\mathbf{e}_{i}^{S}\in\mathbb{R}^{d} \tag{1}\]  
\[\mathcal{H}_{0}=Embedding(\mathcal{T},\mathbf{E})=[\mathbf{e}_{t_{1}}^{S}, \mathbf{e}_{t_{2}}^{S},\cdots,\mathbf{e}_{n}^{S}] \tag{2}\]  
\[\mathcal{H}_{l}=Encoder_{l}(\mathcal{H}_{l-1},\theta_{l})=[\mathbf{h}_{1}^{L}, \mathbf{h}_{2}^{L},\cdots,\mathbf{h}_{n}^{L}] \tag{3}\]  
\[LM(\mathcal{T},\Theta)=\mathcal{H}_{C}=[\mathbf{h}_{1}^{C},\mathbf{h}_{2}^{C},\cdots,\mathbf{h}_{n}^{C}], \tag{4}\]  
where \(\mathbf{E}\) is the embedding matrix, \(Embedding(\cdot)\) is a lookup function, \(\theta_{l}\) is the parameters of the \(l\)th encoder layer, \(\Theta=[\mathbf{E}_{L_{S}},\theta_{1},\theta_{2},\cdots,\theta_{C}]\), and \(C\) is the number of encoder layers in \(LM\). Then, the probability of a token is computed as  
\[P(t_{i}=t_{j}|\mathcal{T})=\frac{\exp{(\mathbf{h}_{t_{i}}^{C}\cdot\mathbf{e}_ {t_{j}}^{S})}}{\sum_{k=1}^{n}\exp{(\mathbf{h}_{t_{k}}^{C}\cdot\mathbf{e}_{t_{j }}^{S})}},t_{i}=[MASK] \tag{5}\]  
for the masked language modeling task, and  
\[P(t_{n+1}=t_{j}|\mathcal{T})=\frac{\exp{(\mathbf{h}_{t_{n}}^{C}\cdot\mathbf{e }_{t_{j}}^{S})}}{\sum_{k=1}^{n}\exp{(\mathbf{h}_{t_{k}}^{C}\cdot\mathbf{e}_{t_ {j}}^{S})}} \tag{6}\]  
for the next word prediction task.  
Among the parameters in \(\Theta\), some could be helpful in modeling \(L_{T}\) while some could be harmful. The most important part of the modeling process is the contextualization of embedding vectors, performed by the encoder layers. We reuse them in post-training as these layers are known to acquire mostly language-independent knowledge [34; 36; 37; 38]. On the other hand, embedding vectors project the tokens in a language into the semantic space and thus cannot be directly transferred to another language. It is possible to indirectly use them using bilingual word embedding techniques [26; 45], but we randomly initialize the word vectors of \(L_{T}\) for simplicity.  
Modern transformer-based architectures have some additional parameters such as positional embeddings and language modeling head [1; 3]. We also reuse them in \(L_{T}\) as they are not language-dependent and have shown to improve performance in the preliminary experiments.
## 3 Methodology
### Point of Interest of Target Objects  
Since NFD detects objects by YOLO, the result shows the positions of bounding boxes, classes of objects, and their confidence only. The posture of the target objects would not be clearly shown and they may be almost or partially blocked. It was difficult to determine the closest point of the detected object based on the limited information (in Figure 4). There are methods for identifying the posture of objects such as adding another neural network for detecting the pose or labeling the orientation information in the dataset at the very beginning. However, it would increase the complexity of NFD so it is proposed to be handled in future work. To simplify the model, the concept of Point of Interest (POI) was introduced. Instead of determining the exact closest point of the object, the middle bottom of the bound box was selected as the POI. The distance estimated in the model was the distance between the camera and the POI.  
The simplification would introduce errors into the solution. The error varied depending on the pose of the object and the maximum error was half of the width of the detected object, which is shown in Section 4.1. In this work, the largest target object is a long fence that has a width of 1.42 m. Assume that a user is wearing a camera at 1.2 m height and standing at 2.5 m from the object (the blind area is around 2.4 m in that case). The maximum error introduced by the simplification is calculated by Equation (12), which introduces the maximum of 4% error on the actual distance.  
\[\epsilon_{max}=\frac{\sqrt{\left(h/2\right)^{2}+d^{2}}-d}{d}=\frac{\sqrt{\left( 1.42/2\right)^{2}+2.5^{2}}-2.5}{2.5}=4\% \tag{12}\]