esa-sceva/satcom-chunk-collection huggingface dataset:
# Applications of Multi-Agent Deep Reinforcement Learning: Models and Algorithms
## 1 Introduction
### Multi-Agent Reinforcement Learning
Knowledge exchange enables agents to consider their own and others' performances in order to address the moving target problem. The knowledge sharing feature of MARL has the advantage of achieving higher scalability, because a complex system-wide optimization problem can be decomposed into a set of distributed problems solved by individual agents in a distributed manner with faster learning [26,27]. Algorithm 2 shows the MARL algorithm. MARL enables an agent \(i\in I\) to observe its state \(s_{t}^{i}\in S^{i}\) (step 2), exchange knowledge (steps 3 and 4), and select the best possible action \(a_{t}^{i}\in A^{i}\) at time \(t\) in an independent manner (step 5), and then receive an immediate reward \(r_{t+1}^{i}(s_{t+1}^{i})\) at time \(t+1\) (step 6). Subsequently, agent \(i\) updates its Q-value \(Q_{t+1}^{i}(s_{t}^{i},a_{t}^{i})\) using the Q-function (step 7) as follows:  
\[Q_{t+1}^{i}(s_{t}^{i},a_{t}^{i})\leftarrow(1-\alpha)Q_{t}^{i}(s_{t}^{i},a_{t} ^{i})+\alpha[r_{t+1}^{i}(s_{t+1}^{i})+\gamma\sum_{j\in N}\eta^{ij}\text{max}_{ a^{j}\in A}Q_{t}^{j}(s_{t+1}^{j},a^{j})] \tag{2}\]  
where \(\sum_{j}\eta^{ij,j}=1\), \(\eta^{ij}\) represents the importance of neighboring agent \(j\in J^{i}\) at agent \(i\), and \(J^{i}\) represents a set of agent \(i\)'s neighboring agents. The joint action \(a_{t}=(a_{1}^{1},\dots,a_{t}^{i},\dots,a_{t}^{N})\) generates immediate reward \(r_{t+1}(s_{t+1})=(r_{t+1}^{1}(s_{t+1}^{1}),\dots,r_{t+1}^{i}(s_{t+1}^{i}), \dots,r_{t+1}^{N}(s_{t+1}^{N}))\) and converges to the optimal joint action as time goes by. ```
1:Procedure
2: Observe current state \(s_{t}^{i}\in S\)
3: Send Q-value \(Q_{t}^{i}(s_{t}^{i},a_{t}^{i})\) to neighboring agents \(J^{i}\)
4: Receive \(\text{max}_{a^{j}\in A^{j}}\)\(Q_{t}^{j}(s_{t}^{j},a^{j})\) from agent \(j\in J^{i}\)
5: Select action \(a_{t}^{i}= \begin{cases}\text{random,}&\text{if exploration}\\ \text{argmax}_{a\in A}Q_{t}^{i}(s_{t}^{i},a),&\text{if exploitation}\end{cases} \)
6: Receive immediate reward \(r_{t+1}^{i}(s_{t+1}^{i})\)
7: Update Q-value \(Q_{t+1}^{i}(s_{t}^{i},a_{t}^{i})\) using Equation (2)
8:End Procedure
```  
**Algorithm 2** MARL algorithm. There are two types of MARL systems [12]: (a) the cooperative environment in which each agent maximizes \(r_{t+1}^{i}(s_{t+1}^{i})\in R\) as part of the global reward \(R\) in order to maximize the system performance; and (b) the competitive environment in which each agent maximizes its own local reward, which may minimize neighboring agents' rewards, rather than the global reward, and so the sum of rewards of all agents equal to zero \(R=\sum_{i=1}^{N}R^{i}\Big{(}s^{i},a^{i},s^{i}\Big{)}=0\)[28]. This is also known as the zero-sum Markov game [29].
## 3 Problem Statement and Formulations  
### Problem Statement  
We consider a trucking company that manages a finite fleet of trucks to provide inland container transportation services for customers located in close proximity to an import-oriented terminal. The company owns a depot that serves as both a storage space for empty containers and a parking area for trucks. Empty containers are regarded as scarce and precious resources, and therefore the depot holds only a restricted quantity of empty containers. In addition, the capacity to accommodate empty containers is inevitably limited to a certain number due to the spatial constraints imposed by the depot itself. Trucks are homogeneous and are capable of transporting a single container at a time. They depart from the depot and are obliged to return there upon completion of their assigned tasks within the planning horizion. Container pick-ups and deliveries at customer locations and terminals must be confined to their respective time windows. If a truck arrives ahead of the scheduled start time, it should wait on-site until operations can begin.  
Customers act as the initiators of container movements and they have diverse inland container transportation requests to be fulfilled within a specified time window. These requests can be categorized based on criteria such as direction (inbound or outbound) and container status (full or empty). In an import-dominant terminal, inland container transportation requests generally fall into three main categories, as illustrated schematically in Figure 1:  
1. Inbound Full Container Transportation Request (IF): This involves transporting a fully laden container from the terminal (Origin) to a receiver's site (Destination). An IF request will free up an empty container once the freight has been unpacked. The emptied container could be either relocated to the depot to replenish the stock or delivered to a shipper's location.
2. Outbound Full Container Transportation Request (OF): This refers to hauling a fully laden container from a shipper's site (Origin) to the terminal (Destination). For an OF request, the shipper demands an empty container to load their cargo, which can either be sourced from the depot or released by an IF request.
3. Outbound Empty Container Transportation Request (OE): This entails transporting surplus empty containers, resulting from trade imbalances, to the terminal (Destination) for onward transportation. An OE request allows for flexibility in its origin, permitting the empty container to be retrieved from either the depot or the consignee.  
Summing up, the goal of the problem under investigation is to efficiently schedule the available trucks in a way that fulfills all customers' requests and minimizes the total travel cost. The peculiarity of the problem lies in deciding the destinations for the empty containers released from IF requests and determining the sources of empty containers demanded by OF/OE requests, while respecting empty-container-related constraints.
## 4 Results and Discussion
### Compressive Strength  
The Design of Experiments and statistical analysis with Minitab 19 software at 95% confidence were used. Variance analysis of the compressive strength gave results as shown in Table 7 and Figure 8 of the residuals for the plots of the regression model.  
When performing a lack-of-fit test in regression analysis, the \(p\)-value helps to determine whether the model adequately fits the data. From Table 7, the lack-of-fit having a \(p\)-value of 0.126 suggests that there was no significant evidence to reject the null hypothesis, which states that the model does not lack fit. In other words, the \(p\)-value indicates that the model fits the data adequately at a common significance level (\(\alpha\) = 0.05). In addition, Figure 8 also provides a typical residual plot quite close to line zero and indicates that the model's predictions were generally accurate, as the residuals (differences between observed  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Source**} & **Degrees of** & **Adjusted** & **Adjusted** & & \\  & **Freedom** & **Sum of** & **Mean of** & **Mean of** & **F-Value** & \(p\)**-Value** \\  & & **Squares** & **Squares** & & \\ \hline Model & 33 & 29,293.8 & 887.7 & 7235.91 & 0.000 \\ Linear & 9 & 29,153.2 & 3239.2 & 26,404.29 & 0.000 \\ Days & 2 & 26,906.7 & 13,453.3 & 109,663.36 & 0.000 \\ GO (\%) & 5 & 810.2 & 162.0 & 1320.88 & 0.000 \\ FA (\%) & 2 & 1436.3 & 718.1 & 5853.73 & 0.000 \\
2-Way Interactions & 24 & 140.6 & 5.9 & 47.77 & 0.000 \\ Days \(\times\) GO (\%) & 10 & 35.7 & 3.6 & 29.13 & 0.000 \\ Days \(\times\) FA (\%) & 4 & 56.5 & 14.1 & 115.18 & 0.000 \\ GO (\%) \(\times\) FA (\%) & 10 & 48.4 & 4.8 & 39.44 & 0.000 \\ Error & 128 & 15.7 & 0.1 & & \\ Lack-of-Fit & 20 & 3.3 & 0.2 & 1.43 & 0.126 \\ Pure Error & 108 & 12.4 & 0.1 & & \\ Total & 161 & 29,309.5 & & & \\ \hline \hline \end{tabular}
\end{table}
Table 7: Analysis of variance of compressive strength.  
Figure 7: Interaction plot of factors on (**a**) compressive strength and (**b**) flexural strength.  
and predicted values) were small. Thus, the experimental results show that the input factors and their interaction terms significantly affected the compressive strength with high statistical confidence.
## 3 Results and Discussion  
### NH\({}_{3}\) Concentration at Biogas Production Facility by Waste Type  
The analysis results of the ammonia concentration in biogas facilities are shown in Table 3. The results showed that the ammonia concentration from biogas facilities was between 0.04-28.47 ppm.  
Figure 2: Process of the Monte Carlo Simulation for estimating the uncertainty of the NH\({}_{3}\) emission factor.  
The range of ammonia concentration from A biogas production facility was between 0.04-8.80 ppm, and the average concentration was 1.65 ppm. The standard deviation was 2.10 ppm. The range of ammonia concentration from B biogas production facility was between 0.05-29.31 ppm, and the average concentration was 7.99 ppm. The standard deviation was 8.95 ppm. The range of ammonia concentration from C biogas production facility was between 0.04 and 28.47 ppm, and the average concentration was 2.41 ppm. The standard deviation was 4.70 ppm. The reason for such differences was attributed to biogas production facility A using a wet scrubber to manage the air pollutants, biogas production facility C only intermittently using the wet scrubber, and biogas production facility B not using a wet scrubber using a different method of processing.
## 2 Materials and Methods
### Methodology  
#### 2.3.1 Statistical Algorithm and Explanatory Variables  
Among the wide range of statistical algorithms, the Random Forest (RF, [23]) approach is particularly appropriate in a multifactorial and non-linear context such as modeling backscattering coefficients. This algorithm offers the major advantage of better predictive capability, with a performance far superior to that of multiple linear regressions [24]. In addition, this algorithm has been used in the field of remote sensing, notably with microwave images, in contexts of land use and land cover classification, or for the estimation of variables of interest such as soil moisture [30; 31].  
A quantitative estimation of the targeted variable is obtained through a weighted mean of an ensemble of estimations. The bootstrap aggregating procedure allows constructing independent trees on a subset of samples derived from the original dataset, and presents the following advantages: a reduced over fitting, a low influence of noise on the data, and a high stability of the results.  
Backscattering coefficients were statistically modeled by considering the same explanatory variables as those used for semi-empirical or physical-based models [9; 10; 11; 12; 13]; the angle of incidence of SAR images, the soil moisture, the clay and sand fractions, and the soil roughness (hrms and lc collected in the parallel and perpendicular directions).
## 3 Results and Discussion
### Limitations and Further Work  
The first limitation of this work is that there was no experiment to validate the value accuracy of mass maldistribution and pressure drop. However, as the objective of this paper focused on the effect of channel configuration on mass maldistribution and the same numerical method was proved to be reliable by experiments with a maximum difference of 10% due to our previous work in Liao, Y. [24], its value accuracy is considered to be an acceptable difference with experiment. Additionally, it is certain that the study of this work can be an instructive optimization for further cooling plate design in EVs. Another problem is that orthogonal experiments were carried out only with four, but not all parameters that can affect flow characteristics. More work is needed to study whether there are some key parameters missing. Furthermore, boundary conditions were also simplified in order to focus on the optimization method and configuration design. However, a cooling plate will experience a wide range of conditions during electric vehicle operation and it should be studied whether the design is still optimal for the practical EVs.  
Further work can be carried out on the experimental test to prove value accuracy of the simulation and improve the numerical models from experimental feedback. Future work can include a full study on the effect of all the parameters in channel configuration on mass maldistribution using an optimized algorithm programmed in Matlab.
# Acquisition and Processing of UAV Fault Data Based on Time Line Modeling Method
## 4 Processing of UAV Fault Data  
### The Introduction of UAV Fault Data and the Content of This Section  
The UAV fault data refers to the on-board logs that the drone records when a component of the drone fails. As can be seen from the previous section, this paper simulates GPS failure, accelerometer failure, engine failure, and remote-control failure in the same flight. Additionally, it realizes the original collection of on-board logs when the drone fails. This section introduces the processing method of UAV fault data proposed in this paper includes the Failure Time Anchoring method and Time Window Stretching method.
###### Abstract  
The multi-constellation, multi-frequency Global Navigation Satellite System (GNSS) has the potential to empower precise real-time kinematics (RTK) with higher accuracy, availability, continuity, and integrity. However, to enhance the robustness of the nonlinear filter, both the measurement quality and efficiency of parameter estimation need consideration, especially for GNSS challenging or denied environments where outliers and non-Gaussian noise exist. This study proposes a nonlinear Kalman filter with adaptive kernel bandwidth (KBW) based on the maximum correntropy criterion (AMC-KF). The proposed method excavates data features of higher order moments to enhance the robustness against noise. With the wide-lane and ionosphere-free combination, a dual frequency (DF) data-aided ambiguity resolution (AR) method is also derived to improve the measurement quality. The filtering strategy based on the DF data-aided AR method and AMC-KF is applied for multi-GNSS and DF RTK. To evaluate the proposed method, the short baseline test, long baseline test, and triangle network closure test are conducted with DF data from GPS and Galileo. For the short baseline test, the proposed filter strategy could improve the positioning accuracy by more than 30% on E and N components, and 60% on U. The superiority of the proposed adaptive KBW is validated both in efficiency and accuracy. The triangle network closure test shows that the proposed DF data-aided AR method could achieve a success rate of more than 93%. For the long baseline test, the integration of the above methods gains more than 40% positioning accuracy improvement on ENU components. This study shows that the proposed nonlinear strategy could enhance both robustness and accuracy without the assistance of external sensors and is applicable for multi-GNSS and dual-frequency RTK.  
multi-GNSS; real-time kinematic; maximum correntropy criterion; Kalman filter; wide-lane; ionosphere-free +
Footnote †: journal: 13 September 2022
###### Abstract  
Recently, the low earth orbit (LEO) mega-constellation faces serious time-varying interferences due to spectrum sharing, dense deployment, and high mobility. Therefore, it is important to study the interference avoidance techniques for the dense LEO satellite system. In this paper, the interference situational aware beam pointing optimization technique is proposed. Firstly, the angle of departure (AoD) and angle of arrival (AoA) of the interfering links are obtained to represent the time-varying interference. Then, the interference avoidance problem for dense LEO satellite systems is modeled as a non-convex optimization problem, and a particle swarm optimization (PSO) based method is proposed to obtain the optimal beam pointing of the user terminal (UT). Simulations show that the relative error of the mean signal-to-interference plus noise ratio (SINR) obtained by the proposed method is 0.51%, so the co-channel interference can be effectively mitigated for the dense LEO satellite communication system.  
LEO mega-constellation; interference situational aware; beam pointing optimization; co-channel interference +
Footnote †: journal: Electronics
# Automatic Symbol Resolution on Embedded Platforms by the Example of Smart TV Device
## 1 Introduction
### Area of Research  
This paper focuses on testing whether removing symbols from the firmwares of embedded devices can increase their security level. The proposed methods (Sections 5.1 and 5.2) and developed framework (Section 5.3) for automatic symbol resolution will be presented and tested on two examples of Smart TV brands: LG and Samsung. Those two brands were chosen because of their popularity on the market. Right now they both have almost 50% of the Smart TV market share ([[http://english.etnews.com/20180220200004](http://english.etnews.com/20180220200004)]([http://english.etnews.com/20180220200004](http://english.etnews.com/20180220200004)), accessed on 13 April 2021). According to the Wakefield study [28], in 2019 Samsung and LG had a 45% share of the Smart TV market in the USA (Figure 1), which makes them perfect targets for hackers [15; 16; 17]. We did not consider researching Vizio as it is mostly popular only in the US and its market share in the EU or other parts of the world is marginal. Moreover, even in the US, its market share reduced by almost 60% in the last 3 years [29]. If we analyze the world Smart TV market share, Sony is the only other manufacturer worth mentioning. However, it is constantly losing its market share to cheap Chinese devices produced at the lowest possible price, which often do not have many security mechanisms. For example, recently, researchers have found multiple security vulnerabilities [30] in devices produced by TCL, a leading Chinese Smart TV manufacturer [29]. Some researchers even suggested that the lack of security mechanisms in those devices is so severe that users should unplug them from the Internet immediately [31]. Furthermore, recently there have been many security vulnerabilities found in Sony Smart TVs [32; 33]. These were the reasons we have decided to focus our research on the biggest players in the Smart TV market: LG and Samsung. To present the results of the proposed method, Smart TVs equipped with Netcast (LG) and Orsay (Samsung) operating systems were used.
## 4 Results and Discussion
### Geophysical Logging Response Characteristics of Coal Quality
#### 4.3.2 Logging Response Characteristics of Ash Yield  
The sedimentary environment exerts a controlling influence on coal ash yield, which reflects the degree of mineral development [61,62]. Ash yield exhibits an initial decrease followed by an increase with rising apparent resistivity (Figure 7a,c). Elevated ash yield and inorganic mineral content enhance coal conductivity, reducing seam resistivity [63,64]. In the study area, blocky and banded calcite, pyrite, and clay minerals infill pores, fractures, or organic matter, further lowering apparent resistivity [25,65,66]. Minerals such as pyrite within ash enhance coal rock conductivity [67], increasing brittleness and promoting cleavage, pores, and fractures that amplify conductivity variations [47,51,68].  
Figure 6: Relation between moisture content in coal and logging parameters. (**a**) The relationship between moisture content and apparent resistivity; (**b**) The relationship between moisture content and lateral resistivity; (**c**) The relationship between moisture content and bulk density; (**d**) The relationship between moisture content and natural gamma; (**e**) The relationship between moisture content and short-spaced gamma-gamma; (**f**) The relationship between moisture content and acoustic transit time.  
Ash yield similarly trends with density, decreasing initially before rising (Figure 7c). This relationship is governed by material composition, coal rank, clay mineral content, and drilling-induced factors such as borehole diameter changes and mud invasion [69]. Syngenetic minerals predominantly influence coal seam pore structure through density, composition, and deformation, with density being the most critical factor [70]. Suffide minerals, denser and more conductive than silicates, increase seam density and reduce resistivity when enriched [71].  
Ash yield correlates positively with lateral resistivity, natural gamma (GR), short-spaced gamma-gamma (CGS), and acoustic transit time (AC) (Figure 7b,d-f). This arises because organic matter in coal has higher resistivity than hydrated minerals, and GR intensity increases with ash yield [14]. Clay minerals, with their high surface area, strongly adsorb radioactive elements and magmatic minerals during diagenesis, leading to higher GR values in ash-rich coals [14]. Late-stage sedimentation fills coal reservoir pores and fractures with banded minerals, yet pervasive pore-fracture systems elevate ash yield and AC [47; 72; 73].
# Easy Fabrication of Ultrafiltration Membrane via Polyethersulfone-Fumed Silica
## 3 Results and Discussion
### Water Contact Angle Analysis  
Water contact angle measurement is commonly employed to assess the hydrophilicity and wetting properties of membrane surfaces. Hydrophilicity behavior plays a crucial role in water treatment using membrane separation systems. The presence of silanol groups in fumed silica-added polymer membranes is likely to enhance membrane hydrophilicity, subsequently improving the permeation performance [37]. In this study, the water contact angle did not exhibit significant changes upon the addition of fumed silica, as illustrated in Figure 5. The native PES membrane has a contact angle of 60\({}^{\circ}\), while the highest observed angle is 65\({}^{\circ}\), with an average value of 63\({}^{\circ}\). This lack of significant change may be attributed to the low concentration of fumed silica added, which is insufficient to significantly impact the surface water contact angle. Fumed silica is an amorphous material, and adding silica/fumed silica to a membrane solution normally increases its hydrophilicity because it contains a substantial amount of the hydrophilic silanol group [38]. According to [PERSON] et al., the enhanced membrane hydrophilicity can be attributed to the FS residues within the polymer matrix, particularly on the membrane surface [17]. In this study, however, since the concentration of FS added to the PES solution was small, the increased hydrophobicity was not significant even though the trend was increasing.  
Figure 4: Protein rejection of PES membranes with varied fumed silica concentration.
# Agriculture 5.0 in Colombia: Opportunities Through the Emerging 6G Network
## 4 Results and Discussion
### Current Colombian Agricultural Outlook
#### 4.3.3 Digital Traceability in the Agro-Food Chain  
A proof-of-concept developed by [PERSON], [PERSON], and [PERSON] [66] demonstrated the feasibility of using Hyperledger Fabric to trace the production and distribution of Colombian organic coffee, enhancing transparency and trust across the supply chain. Their blockchain-based architecture enabled secure and immutable records from farm to final consumer. With 6G, these systems could benefit from real-time data transmission and massive device connectivity, thus expanding traceability applications to other priority export crops such as avocado, oil palm, and cacao, while also facilitating compliance with emerging international regulations like the EU Deforestation Regulation (EUDR).