esa-sceva/satcom-chunk-collection huggingface dataset:
# Benchmarking of Load Forecasting Methods Using Residential Smart Meter Data
## 2 Materials and Methods
### Forecasting Approaches
#### 2.2.1 Multivariate Adaptive Regression Splines (MARS)  
This approach is characterized as a regression algorithm and its foundations rely on simple linear regressors, making it easy to use and to interpret [13; 20]. As the load demand often represents a non-linear dependence between output(s) and inputs, the MARS model can cope with it, as it involves an ensemble of aggregated linear functions based on one or two hinge functions. The perspective of hinge functions can be approximated to:  
\[\text{h(x-k)}=\text{max(0;x-k)}=\text{\{x-k, if x>k; and 0 if x\leq k\}} \tag{1}\]  
where k is a constant, known as a knot. The MARS model will result in aggregating the hinge functions associated with each knot, and even considering linear dependence on single features and a bias (also known as the intercept term).  
Each knot can generate one or a pair of \"knees\" at the model function. Figure 1 illustrates this by considering a non-linear real function in light blue and an approximation made possible by using a MARS model (in dark blue). As the two hinge functions show, two knots are here considered (x = -10 and x = 10) and each individual hinge function are enough to model this desired non-linearity.  
To fine-tune the model, the associated knots must be found based on the inputs-outputs pairs provided during the training phase, being a data-driven model. Many candidate basis functions are generated in the forward phase, being generated in pairs h(x - k) and h(k - x). Each pair is added to the model if it contributes to reducing the model's performance. A subsequent backward phase is used to prune the terms in the final equation whenever they do not contribute to the overall performance. This backward process can be viewed to avoid model overfitting, as a generalized cross-validation is applied to  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Data Description** & **Number of Consumers** & **Available Data** \\ \hline Original Data & 5567 & From 1 November 2011 to 28 February 2014 \\ Selected Sample & 71 & From 21 December 2011 to 27 February 2014  \\ \hline \hline \end{tabular}
\end{table}
Table 1: Summary of the available dataset and the selected sample.  
Figure 1: Example of a MARS model to approximate a non-linear real function.  
compare the performance of model subsets. It ensures a trade-off between accuracy and model complexity.  
MARS models in the scope of this research were created and tested using PyEarth package [24] to be used in the Scikit library of Python (being a third-party library). By default, the maximum number of terms was considered as a value dependent on number of features and number of samples, with a maximum number of 1 for the terms generated by the forward pass, a value of 3.0 as penalty factor during the pruning pass and a zero tolerance.
## 4 Problem Modeling and Solution
### Overall Problem Description
\begin{table}
\begin{tabular}{c l} \hline \hline
**Notation** & **Description** \\ \hline \(s\) & The source node (input) \\ \(T\) & The set of all destination nodes (input) \\ \(N\) & The set of all switch nodes (input) \\ \(t\) & A destination node \\ \(n_{T}\) & The number of destination nodes (input) \\ \(P_{t}\) & The set of all simple paths from node \(s\) to node \(t\) (input) \\ \(L\) & The set of all links in the physical topology (input) \\ \(n\) & A switch node \\ \(l\) & A link \\ \(c_{n}\) & The flow entry capacity constraint for switch node \(n\) (input) \\ \(b_{l}\) & The available bandwidth of link \(l\) (input) \\ \(p\) & A path \\ \(F_{t}\) & The total flow capacity that can be transmitted through multiple paths from node \(s\) to node \(t\) in a unit of time \\ \(f_{p}\) & The amount of traffic transmitted by path \(p\) in a unit of time \\ \(L_{p}\) & The set of links contained in path \(p\) (input) \\ \(x_{p}\) & A binary variable. \(x_{p}=1\) if path \(p\) is in use, and \(x_{p}=0\) otherwise. \\ \(PT_{n}\) & The set of all paths traversing switch node \(n\) \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Summary of notations. It should be noted that the multi-topology is generated based on a static physical topology. The construction of the multi-topology does not take into account the future demands and specific mobility of devices, as these are unknown at the time of its creation. However, this limitation can be addressed by dynamically allocating mobile data flows across different logical topologies. The goal of the multi-topology construction is to maximize the average transmission capacity from the source node to all possible destination nodes, thus achieving a certain level of scalability and universality. Additionally, it provides a solid foundation for the dynamic allocation of mobile data flows.
## 3 Outage Rate Analysis
### Optimal \(\Delta_{t}^{*}\) with Different Standard Deviation \(\sigma\) of Shadowing  
To better understand the trade-off between path loss and macro-diversity in correlated shadowing channels, we first investigate the impact of standard deviation \(\sigma\) of the log-normal shadowing. The higher \(\sigma\) implies more severe large-scale fading, thereby macro-diversity becomes more important. Figure 5 shows the outage probabilities with different \(\Delta_{t}\) and \(\sigma\), which correspond to the \(x\)- and \(y\)-axes, respectively. The outage rates with MRC and SC are indicated by the blue and red meshed surfaces, respectively. As in the previous simulation results, the outage rate with MRC is always smaller than that with SC for the same condition. Also, when \(\Delta_{t}\) is fixed, \(P_{out}\) increases as \(\sigma\) increases due to higher shadowing. On the other hand, for the same \(\sigma\), there exists the optimal value of \(\Delta_{t}\) that minimizes \(P_{out}\).  
Figure 4: Outage probability \(P_{out}\) versus inter-separation between the transmitters \(\Delta_{t}\), when \(\gamma_{th}=10^{-3}\), \(l=30\) m, \(P_{T}=50\) dB, \(n=4\), and \(\sigma=8\) dB.  
Figure 6 shows the optimal inter-transmitter separation \(\Delta_{t}^{*}\) as a function of \(\sigma\) in dB. The blue solid and red dotted lines represent the numerical results of MRC and SC based on our analysis, respectively. Furthermore, the blue 'x'- and red 'o'-markers indicate the simulation results of MRC and SC, respectively. As shown in the figure, the two combining schemes have the same optimal value, even though they give different \(P_{out}\). As \(\sigma\) increases, \(\Delta_{t}^{*}\) increases, because the impact of the large-scale fading becomes larger. Thus, to overcome such severe shadowing, macro-diversity gain with smaller \(\rho\), which means higher \(\Delta_{t}\), is required.
# Bidirectional Representations for Low-Resource Spoken Language Understanding
Despite considerable improvement in speech modeling in recent years, speech representation models do not show the same ability as text-based language models to efficiently store semantic information, and a considerable amount of fine tuning is necessary to achieve decent performance [10]. In this research, we explore representation models of speech and demonstrate their usefulness for SLU, with a particular focus on low-resource solutions, when only a handful of examples is available to train and validate the models. We propose an architecture for pretraining of bidirectional speech transformers on a surrogate ASR task with the goal of learning qualitative representations that prove useful for solving language understanding tasks. We evaluate the resulting embeddings in an SLU task, intent recognition, where the objective is to, given a spoken command, identify the intent and any argument necessary. For example, the sentence \"switch the lights in the kitchen to blue\" would result in an intent of \"lights\" and arguments of \"room = kitchen\" and \"color = blue\", while \"play [PERSON]\" has an intent of \"music\" and an argument of \"artist = [PERSON]\". For completeness and comparison purposes, we also include results obtained after partially fine tuning the model. We are particularly interested in low-resource scenarios, where we control the number of examples used for training and validation to study the limits of what can be learned from the representations. When information is stored efficiently, few examples are necessary to learn how to extract it. In contrast, noisy embeddings require many examples to extract relevant information. For the SLU module, we propose class attention as a drop-in replacement of LSTMs. Class attention assigns a score to each position that relates to the importance of the corresponding token to predict the intent and arguments. Aside from the advantages of its small footprint, it provides us with a method to understand why a model makes predictions. In the remainder of this article, we first detail the architecture of the different components of the model. Then, we lay out the experimental setup and methodology. Finally, we evaluate the models and analyze the representations in detail.
# Statistical Analysis of the Crosswalk Lighting Design Correctness
## 2 State of the Art
### Technical Background: Lighting Design and Performance
Similarly as for roadways, there are also lighting classes for crosswalks, ranging from PC1 (the most restrictive one) to PC5, respectively. Performance requirements for pedestrian crossings [13] rely on the following photometric quantities:  
* \(E_{v,avg}\)--average vertical illuminance: refers to a light beam, incident with a vertical plane,
* \(U_{o,v}\)--overall vertical illuminance uniformity: the ratio of minimum and average values of \(E_{v}\),
* \(E_{h,avg}\)--average horizontal illuminance: refers to a light beam, incident with a horizontal plane,
* \(U_{o,h}\)--overall vertical illuminance uniformity: the ratio of minimum and average values of \(E_{h}\),
* \(E_{v,min}^{(6)}\)--the minimum value of \(E_{v}\), calculated for six reference points. In our considerations, we focus on the PC4 class. Requirements for all PC classes are defined in Table 2:  
The feature which distinguishes pedestrian crossing calculations from the roadway ones is that they require three calculation grids (see Figure 3a,b). Similar to roadway lighting planning, there is a set of layout and installation parameters, which are input variables for photometric computations. It includes road and crossing widths, crosswalk lamp locations and height (not to be confused with signal lights! ), crosswalk lamp fixture model and photometric curve. It should be noted that a crosswalk fixture has asymmetric (left or right) optics. Additionally, roadway lighting installation parameters also have to be taken into account, as they affect optical conditions on a crosswalk (e.g., contrast). A number of possible configurations related to an installation (road and crosswalk characteristics are immutable) makes the optimal lighting design searching process infeasible using brute force methods. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Class** & \(E_{v,avg}\) [**lx**] & \(U_{o,v}\) & \(E_{h,avg}\) [**lx**] & \(U_{o,h}\) & \(E_{v,min}^{(6)}\) [**lx**] \\ \hline PC1 & 75 & 0.35 & 75 & 0.40 & 5 \\ PC2 & 50 & 0.35 & 50 & 0.40 & 4 \\ PC3 & 35 & 0.35 & 35 & 0.40 & 4 \\ PC4 & 25 & 0.35 & 25 & 0.40 & 3 \\ PC5 & 15 & 0.35 & 15 & 0.40 & 2 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: PC-lighting classes for pedestrian crossings, according to the Polish standard [13].
# Sensors and Communication Simulation for Unmanned Traffic Management
## 6 Validation  
Next, we focus on the validation of the simulation platform. The focus of our simulator is to enable the assessment of complete UTM platforms and more specifically, the discovery of potential negative emergent behaviors on UTM systems. Therefore, the validation of each of the individual agent models is not so relevant. It is much more important for us to check that the proposed platform allows simulating complex scenarios in which multiple agents interact, in presence of contingencies, while generating UTM input sources in a coherent way. Once the simulation architecture has been validated, refinement of individual agent simulation models is always possible, and in fact, the agent-based simulation approach nicely supports this iterative improvement of individual models. In any case, some of the most critical simulation models have been validated in previous publications with real flights, as those related to trajectory prediction and telemetry communication in [30; 32].  
This system-level validation was performed using the simulator implementation described in Section 4.2. Simulation tests with hundreds of drones and dozens of sensors and communication nodes were performed, involving all types of contingency combinations. These tests were carried out using a modest hardware platform (2 core, 2.4 GHz, Intel Core2 Duo E6600 CPU from year 2006 and 6 GB of RAM) and show that the proposed platform scales well with the number of simulated entities without reaching computational bottlenecks, as can be seen in Figure 11. In addition, no relevant simulation-induced delays were detected as the number of entities increases. Moreover, entities usually send information to the UTM system each second which dampens the final impact of possible simulation delays.  
Figure 10: Sensor track processing schema.  
This scalability is due to some design decisions such the distributed simulation among different microservices as well as drone trajectories precomputation (as explained in Section 5.2) which allows performing the CPU-intensive trajectory-prediction process only when the expected trajectory changes. Scenarios with a greater number of entities could be easily simulated either scaling vertically by using a more powerful hardware platform, or horizontally by distributing microservices between a set of locally-networked, time-synchronized computers. Multiple instances of a given microservice may also be deployed to increase performance.  
Next, we show the correct operation of the platform in three example scenarios, involving just a few agents for clarity purposes. In order to generate the following graphs, an evaluation service was implemented to retrieve agents' state and the information provided to the UTM system using platform-provided tools.
## 3 Results and Discussion
### Synthetic Musk Fragrances in STP Influents  
The SMF concentrations in the influents of the four STPs were between 3.532 and 11.080 \(\upmu\)g/L (median, 6.756 \(\upmu\)g/L) (Table 3). Because more than 90% of the inflow water to all STPs was domestic sewage, there was no significant difference in SMF concentration between each STP. Five SMFs (HHCB, AHTN, MK, DPMI, and ATII) were most frequently detected in the sewage influents. The highest concentration (3.200 to 10.110 \(\upmu\)g/L) was found for HHCB, with a median value of 6.130 \(\upmu\)g/L. On a median basis, HHCB accounted for 90.7% of the total SMFs and was present in significantly higher concentrations than AHTN, MK, and DPMI (Table 3). The AHTN concentration (median, 0.373 \(\upmu\)g/L) was similar to that of MK (0.352 \(\upmu\)g/L). The DPMI concentration was half that of AHTN and MK, with a median value of 0.166 \(\upmu\)g/L. While ADBI was detected intermittently and at low levels, MA and MX were not detected at all in STP influents.  
The HHCB concentration in the STP influents in this study was higher than that previously reported in Italy [34], Spain [35; 36], France [22], and other studies in Korea [25] (Table S4). This might be due to the characteristics of the STPs, which were located in the densely populated city of Seoul, and most of the STP influents were therefore composed of domestic sewage. However, the AHTN concentration in the STP influent was similar to or lower than that reported in previous studies.  
The HHCB/AHTN concentration ratio in the sewage influents was 16.4, which was higher than the previously reported values of 4.5 [25], 1.2 [28], and 3.2 [26], and similar to the value obtained by [PERSON] and [PERSON] [34]. This result implies that significantly higher amounts of HHCB than AHTN are used in PCPs in Korea. Since Table 2 shows that the HHCB concentration was significantly higher in hair care products (e.g., shampoo) and perfumes, these can be the major source of HHCB in sewage influents.
# Delving Deep into Reverse Engineering of UEFI Firmwares via Human Interface Infrastructure
## 3 Understanding HII Framework and Related Technologies
### HII Protocol  
HII Protocol is a set of protocols and services used for constructing and managing user interfaces. Its primary purpose is to provide a standardized way to describe and manage user interfaces in the UEFI management interface, including everything from BIOS setup menus to full-featured graphical user interfaces.  
Through the analysis of these protocols, we propose a method to analyze the UEFI password policy produced by AMI vendors. To ascertain the storage location of the password policy in UEFI, we first need to be aware of protocols related to keyboard input, such as the EFI SIMPLE TEXT INPUT PROTOCOL. Due to the customizability of UEFI, many manufacturers redefine the keyboard input protocol based on this protocol. For instance, the AMI firmware manufacturer defines a keyboard input protocol named AMI EFIKEYCODE PROTOCOL as follows:  
```
1 structAMI_EFFIKEYCODE_PROTOCOL{
2 AMI_RESET_EXReset;
3 AMI_READ_EFI_KEYReadEffkey;
4 EFI_EVENTWaitForKeyEx;
5 EFI_SET_STATESetState;
6 EFI_REGISTER_KEYSTRORE_NOTIFYRegisterKeyNotify;
7 EFI_UNREGISTER_KEYSTRORE_NOTIFYUnregisterKeyNotify;
8}
```  
Based on the GUID of AMI EFIKEYCODE PROTOCOL as follows:  
```
1#defineAMI_EFFIKEYCODE_PROTOCOL_GUID{
20x0 ADPFB62D,0 xFF74,0x484C,
3{0x89,0x44,0 xF8,0x5C,0x4B,0 xEA,0x87,0x8}
4}
```  
We can accurately pinpoint the location in AMI firmware where this protocol is invoked. Subsequently, by analyzing the context and considering the characteristics of the password input, we can identify the password input location. After that, based on the program's execution logic, we can extract the UEFI password policy as illustrated in Figure 5.  
In HII, there is a protocol known as the HII Configuration Processing and Browser Protocol, which encompasses two other protocols: EFI FORM BROWSER2 PROTOCOL and EFI HII CONFIG ROUTING PROTOCOL. EFI FORM BROWSER2 PROTOCOL is used for displaying pages, and HII Drivers can retrieve information from current HII Driver configuration options that users have modified but not yet stored via its BrowserCallback function. EFI HII CONFIG ROUTING PROTOCOL is a global protocol that handles interactions on the Setup interface. We can implement some UEFI interface elements, such as drop-down boxes, based on these two protocols. Using the GUIDs of these two protocols, we can also locate their invocation points within the firmware as shown in Figure 6.  
Figure 5: UEFI Password Input Box Model.
## 4 Results
### Parameter Estimation Results
\begin{table}
\begin{tabular}{c c c c c} \hline \hline  & \multicolumn{4}{c}{**Urban Roadway**} & **Distribution of** \\ \cline{2-5}
**Variable** & \multicolumn{2}{c}{**FPL**} & \multicolumn{2}{c}{**RPLMV**} & **Random** \\ \cline{2-5}  & **Coefficient** & **t-Stat** & **Coefficient** & **t-Stat** & **Parameter** \\ \hline Constant & \(-3.523\)** & \(-11.26\) & \(-2.945\)** & \(-11.27\) & \\ \hline GENDER & \(0.603\)** & \(6.49\) & \(0.518\)** & \(7.06\) & \\ \hline AT\_FAULT & \(-0.302\)** & \(-3.71\) & \(-0.268\)** & \(-4.09\) & \\ \hline HELMET & \(-0.138\)** & \(-1.80\) & \(-0.075\) & \(-1.24\) & \\ \hline DRUNK & \(-0.617\)** & \(-4.57\) & \(-0.578\)** & \(-5.34\) & \\ \hline UNLICENSE & \(1.621\)** & \(5.69\) & \(1.554\)** & \(6.55\) & \\ \hline VIOLATION & \(0.410\) & \(1.43\) & \(0.378\) & \(1.61\) & \\ \hline ILLEGAL\_OVERTAKING & \(0.063\) & \(0.17\) & \(0.064\) & \(0.21\) & \\ \hline MOBILE\_USE & \(-0.758\) & \(-0.72\) & \(-0.686\) & \(-0.83\) & \\ \hline ASLEEP & \(0.043\) & \(0.06\) & \(0.023\)** & \(0.04\) & \\ \hline CUTTING\_FRONT & \(-0.787\)** & \(-6.32\) & \(-0.737\)** & \(-7.18\) & \\ \hline CURVE & \(0.157\) & \(1.28\) & \(0.128\) & \(1.29\) & \\ \hline ROUGH & \(-0.309\) & \(-0.75\) & \(-0.325\) & \(-0.94\) & \\ \hline SMOKE\_DUST\_FOG & \(-1.160\)** & \(-3.06\) & \(-1.027\)** & \(-3.51\) & \\ \hline AFTERNOON & \(-0.614\)** & \(-5.12\) & \(-0.553\)** & \(-5.77\) & \\ \hline MORNING & \(-0.512\)** & \(-3.67\) & \(-0.441\)** & \(-4.05\) & \\ \hline NIGHT\_LIGHT & \(0.179\)** & \(1.82\) & \(0.156\)** & \(2.00\) & \\ \hline DARK & \(0.454\)** & \(3.83\) & \(0.393\)** & \(4.14\) & \\ \hline FOREIGN & \(0.421\)** & \(2.05\) & \(-0.356\) & \(-1.27\) & \\ \hline S.D. of FOREIGN & & & \(2.726\)** & \(6.31\) & \(55.2\%\) Below Zero \\ \hline LOCAL\_ADDRESS & \(0.015\) & \(0.19\) & \(-0.245\)** & \(-3.80\) & \\ \hline S.D. of LOCAL\_ADDRESS & & & \(1.215\)** & \(19.42\) & \(58.0\%\) Below zero \\ \hline EXCESS\_SPEED\_LIMIT & \(1.078\)** & \(13.94\) & \(0.690\)** & \(10.44\) & \\ \hline S.D. of EXCESS\_SPEED\_LIMIT & & \(1.493\)** & \(16.89\) & \(32.1\%\) Below zero \\ \hline \multicolumn{5}{c}{Unobserved Heterogeneity in means} \\ \hline FOREIGN: RAINING & & \(1.672\)** & \(1.99\) & \\ \hline EXCESS\_SPEED\_LIMIT: RAINING & \(-1.124\)** & \(-1.92\) & \\ \hline \multicolumn{5}{c}{Unobserved Heterogeneity in variances} \\ \hline LOCAL\_ADDRESS: WET & & \(-3.175\)** & \(-14.32\) & \\ \hline EXCESS\_SPEED\_LIMIT: WET & & \(-1.925\)** & \(-4.97\) & \\ \hline \multicolumn{5}{c}{Model statics} \\ \hline LL(\(\beta\)) & \(-2439.899\) & \(-2431.010\) & \\ \hline LL(\(0\)) & \(-2714.900\) & \(-2714.900\) & \\ \hline McFadden \(\rho^{2}\) & \(0.1013\) & \(0.1046\) & \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Model result of urban roadways.
## 4 M2M-Based Division of MAC Layer Protocols
### Distributed Point Coordination Function-M (DPCF-M)  
A distributed point coordination function-M is proposed in [29] that addresses the energy constraints of M2M communication. This protocol is specifically designed for IoT functionality where two scenarios are considered. Local M2M communicating devices and gateway-capable nodes are exclusively enabled for short- and long-distance communication. This protocol uses the non-beacon CSMA/CA mode of IEEE 802.15.4. The gateway nodes also give facility of communication between remote servers using a cellular interface. This protocol outperforms the traditional 802.11 CSMA/CA in energy efficiency and throughput, but the introduction of additional hardware adds extra money to the cost of system upgrades. The protocol exhibits local communication collisions, as experienced by CSMA/CA, on the other hand.