esa-sceva/satcom-chunk-collection huggingface dataset:
# Smart Grid Data Management in a Heterogeneous Environment with a Hybrid Load Forecasting Model
## 3 Related Works
### Challenges of Applications in Power Consumption
#### 3.3.2 Power Consumption and Big Data Analytics Processing  
Big data requires a very efficient and fast communication system to communicate at a high speed with minimum latency to gain optimum efficiency [39]. In developing countries, a fast communication system may not be available or feasible. Hence, integrating big data analytics in the energy sector poses a big challenge. Furthermore, big data analytics can be used to communicate with the energy sector.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Reference** & **Country** & **Forecasting** & **Dataset** & **Result and Finding** & **Limitations** & **Accuracy** \\ \hline \multirow{10}{*}{This work} & \multirow{4}{*}{Iraq} & \multirow{4}{*}{A hybrid load forecasting model using fuzzy c-means clustering, ARIMA, and gradient-boosted tree model, (FCM-ARIMA-GBTL)} & \multirow{4}{*}{Power consumption data, weather data} & The results showed high accuracy in load forecasting with the proposed method (FCM-ARIMA-GBTL) & The need for more historical data to improve accuracy. & Presented in results section \\  & & & & & MAPE, MAE, and RMSE achievement in comparison with other evaluated models such as ARIMA and Gradient Boosted Trees alone. & & \\ \hline \multirow{10}{*}{Ref. [17], 2020} & \multirow{4}{*}{Japan} & \multirow{4}{*}{A hybrid model comprising a clustering technique and the Auto} & \multirow{4}{*}{Power consumption data} & The results show that a combination of clustering and the ARIMA model has & Forecasting per hour per day from 6 to 9 \\  & & & & & MAPE, MAE is used, \\  & & & & & MAPE = 2.7 \\  & & & & & MAPE = 2.  
technologies must handle large data from various sources, which require very large and efficient databases to store and retrieve data during operations. A system's ability to transport and process the data may become a bottleneck if the rate is slow, making the smart grid work inefficiently [37]. Moreover, most of the devices used in big data analytics technologies, such as sensors, operate on batteries that consume energy. The more devices, the higher the energy consumption [40]. The availability and implementation of these devices is also a challenge as they require experts and professionals to deploy and manage the smart devices, which seems quite challenging in Iraq. Additionally, data retrieval is also challenging, especially in remote areas, where an extensive network would have to be laid, thus requiring extra cost [41; 42; 43; 44]. In short, these smart systems' actual power consumption is unknown, and there is a need for thorough research to determine the actual consumption of these devices so that the accurate implementation feasibility of big data analytics can be determined.
## 3 Methodology
### Data Pre-Processing  
The point cloud data obtained by the RGB-D mapping system usually contain noise and varying point densities in different regions caused by measurement errors and high-frequency data streams. This complicates the estimation of local point cloud characteristics such as surface normal or curvature changes. Figure 3a shows a raw point cloud generated from RGB-D system. A sparse outlier removal algorithm [37] is used to distinguish and remove the isolated points from the original point cloud. The sparse outlier removal module corrects these irregularities by computing the mean \(\mu\) and standard deviation \(\sigma\) of nearest neighbor distances. The neighbor points are defined by k, which represents the number of points to use for mean distance estimation. On the premise that the distances are random values with a Gaussian distribution, we trim the points that fall outside the \(\mu\pm\alpha\cdot\sigma\). The value of \(\alpha\) depends on the size of the analyzed neighborhood. In our implementation, we set \(\alpha=1\) and k = 50, because experiments with multiple RGB-D datasets have confirmed the applicability of the \(\mu\pm\alpha\) thresholds, with approximatively 1% of the points considered to be noise (Figure 3b). Data redundancy and density inconsistency problems can occur when all RGB-D sequences are merged directly.  
Figure 1: Hardware scheme of the RGB-D sensor and the acquired RGB-D sequences.  
As a result, a voxelized grid algorithm was used to down-sample the point cloud, which is able to unify the point densities of the whole scenes and speed up data processing. The voxel grid structure creates a 3D voxel grid over the input point cloud data with a specific parameter \(\beta\). The value of \(\beta\) depends on the size of the voxel grid. Each voxel has its own specific boundary according to the setup size. After they are placed in their corresponding voxels, all the points present in the same voxel are removed and a centroid point for the point group is created (Figure 3b,c). Thus, the larger the voxel, the more points are eliminated.  
Figure 3: A sample of data pre-processing, (**a**) raw point cloud, (**b**) point cloud after outlier removal operation, (**c**) point cloud after down-sampling with different voxel size.  
Figure 2: Framework of indoor reconstruction from low-quality RGB-D sequences.
## 3 A Modified Optimization Model to Induce the Discharge of an ESS in Off-Peak Periods
### Conditions for Determining REC Multiplier for Discharging Power in Off-Peak Periods  
The REC multiplier for the discharging power in off-peak periods must be at least more than a specified value to initiate discharging. Therefore, it is important to consider the conditions for determining this multiplier as follows:  
1. Condition 1: The REC multiplier value is more than 1; and
2. Condition 2: The REC multiplier value is more than the product reciprocal of the parameters related to ESS operations.  
Condition 1 is the most common for determining the REC multiplier. As shown in Figure 1, wind farm operators can utilize their power resource generation in two ways. First, the generation transmitted to the electricity grid via the PCC can be paid out at a one-REC credit in the off-peak period, based on the settlement rules. In contrast, power generation for the ESS results in no profit for the operators. Charging loss occurs in ESS operations since the charging efficiency, which is less than 100%, governs the amount of charging power. In this case, the operators would sell their power generation to the main grid, rather than charge the battery, in order to increase revenue. Therefore, the REC multiplier value should be greater than 1 to be fully recognized as a whole power generation, regardless of charging loss.  
Condition 2 is a supplemental condition for determining the REC multiplier, along with Condition 1. As shown in Equations (7), (9), and (10), the turnaround and discharging efficiency, as well as the charging efficiency, would result in a charging and discharging loss since these efficiencies are also less than 100%. First, the power loss would occur during the charging operation, with an efficiency of \(\eta^{ch}\). The second loss is incurred by the inner operations of the ESS, modeled with the turnaround efficiency of \(\eta^{ESS,\ \ eff}\). Discharging loss occurs when the ESS discharges its power, with an efficiency of \(\eta^{chd}\). Although the power from the ESS and wind turbines both have the same effect of supplying electricity to the grid, the discharging power of the ESS is from partial wind power production with losses. The REC multiplier for the discharging power should be higher than that for the wind power to compensate for power losses. Hence, the minimum value for the REC multiplier in off-peak periods would be the reciprocal of the charging, turnaround, and discharging efficiencies.
## 3 Nelder-Mead Algorithm for the Optimal Completion of Incomplete PCMs
### The Coordinate Transformation Method  
Here, we use a simple coordinate transformation technique [38] (pp. 23-24); [34; 36] evolved from a trigonometric function \(\sin(z)\) for dual bound constraints (lower and upper bounds).  
Let \(\mathbf{x}=(x_{1},x_{2},\ldots,x_{k})\in\mathbb{R}^{k}_{+}\) be the original \(k\)-dimensional variable vector (or, equivalently, \(k\) missing comparisons), and \(\mathbf{z}=(z_{1},z_{2},\ldots,z_{k})\in\mathbb{R}^{k}\) be the new search vector.  
Let \(g:\mathbb{R}\rightarrow[l_{i},u_{i}]\) be an invertible function such that \(g(z_{i})=x_{i}\iff z_{i}=g^{-1}(x_{i})\).  
Again, define a function \(\psi:\mathbb{R}\rightarrow[0,1]\) such that \(\psi(z_{i})=\frac{1}{2}(\sin(z_{i})+1)\). It is obvious that \(\psi(z_{i})\in[0,1]\) for all \(z_{i}\in\mathbb{R}\).  
From the fact that \(l_{i}\leq x_{i}\leq u_{i}\), \(\psi(z_{i})\in[0,1]\), and hence \(x_{i}\) is bounded by (\(\psi(z_{i})=0\implies x_{i}=l_{i}\)) and (\(\psi(z_{i})=1\implies x_{i}=u_{i}\)), the coordinate transformation function could be expressed by \(g:\mathbb{R}\rightarrow[l_{i},u_{i}]\) such that:  
\[g(z_{i})=l_{i}+\frac{1}{2}(u_{i}-l_{i})(\sin(z_{i})+1). \tag{5}\]  
In short:  
\[x_{i}=l_{i}+\frac{1}{2}(u_{i}-l_{i})(\sin(z_{i})+1),\ i=1,2,\ldots,k. \tag{6}\]  
From the initial values \(\mathbf{x}_{0,i}\), where \(l_{i}\leq\mathbf{x}_{0,i}\leq u_{i}\), the initial values \(\mathbf{z}_{0,i}\) are calculated as  
\[\mathbf{z}_{0,i}=\sin^{-1}\left(2(\mathbf{x}_{0,i}-l_{i})/(u_{i}-l_{i})-1 \right),\ i=1,2,\ldots,k. \tag{7}\]  
In addition, the diameter of the initial simplex region may be vanishingly small. As a result, in order to avoid this problem, it is recommended to shift the initial coordinate values by \(2\pi\)[34], meaning that:  
\[\mathbf{z}_{0,i}=2\pi+\sin^{-1}\left(2(\mathbf{x}_{0,i}-l_{i})/(u_{i}-l_{i})- 1\right),\ i=1,2,\ldots,k. \tag{8}\]  
Note that \(\mathbf{z}_{0,i}\) is the \(i\)th component of \(\mathbf{z}_{0}\), and \(\mathbf{x}_{0,i}\) is the \(i\)th component of \(\mathbf{x}_{0}\).
## 5 Model Verification  
### Program Correctness Verification  
Without considering the damage, as for the shape function \(g(\theta)=\frac{3-\sin\varphi}{2\left(\sqrt{3}\cos\theta-\sin\theta\sin\varphi \right)}\) of the Z-P constitutive model proposed in this paper, the model degenerated into the Mohr-Coulomb constitutive model. By rewriting the program to adjust the shape function, the simulation calculation of an ideal circular tunnel was carried out. The results of the established Mohr-Coulomb constitutive model in Flac\({}^{\text{3D}}\) were further compared to verify the correctness and accuracy of the model's secondary development program described in this paper. Figure 5 shows the selected calculation model, with the tunnel excavation radius of 1.0 m, the boundary of 10 m away from the center of the tunnel, and the applied stress of 30 MPa. Moreover, the simulation parameters were set accordingly, including the bulk modulus \(K\) of 3.9 GPa, the shear modulus \(G\) of 2.8 GPa, the cohesion \(c\) of 3.45 MPa, and the internal friction angle \(\varphi\) of 30\({}^{\circ}\). The correlation flow method was utilized for computation. Figure 6 summarizes the calculation results.  
From Figure 6, it is evident that the calculation results of the two were practically in agreement. The average relative error of the calculated ho  
Figure 5: Calculation model of ideal circular tunnel: (**a**) Constraints of computational models; (**b**) Calculation model.  
Figure 6: Comparison of stress distribution between the Z-P constitutive model and the M-C constitutive model.  
the average percentage error of the radial stress was 1.246%, which satisfied the accuracy requirements, further proving the correctness of the secondary development program used in this paper. Compared to the M-C constitutive model, except for the basically same elastic region, the Z-P constitutive model had a higher stress value at the elastic-plastic interface and a smaller radius at the plastic region. Both were confirmed to share the same elastic space. The Z-P constitutive model took into account the influence of the intermediate principal stress on the strengthening effect of rock strength [35; 36].  
In addition, when the shape function was \(g(\theta)=1\), the model degenerated into a Drucker-Prager constitutive model in the form of a circumscribed circle, and the two would not be compared in this paper.
## 4 Discussion
### Challenges in Combining Remote Sensing and Forest Models  
There can be several reasons for challenges and uncertainties when combining different types of satellite data (here lidar and radar) with different resolutions and then integrating them into an individual-based forest model with its own resolution. Geolocation uncertainty can play a role especially when combining several data sources. The geolocation error of GEDI is approximately 10 m [14]. Because the GEDI measurements are merged with a forest cover map (resolution 50 m \(\times\) 50 m) in this study, and the remaining analyses are performed at 1 km\({}^{2}\) or 100 km\({}^{2}\) resolution, we expect that a 10 m deviation has minor effects on the results.  
It would also be interesting to analyze the temporal development of biomass and productivity in the forest stands. However, the period of the GEDI measurements is currently too short for this. In the coming years, it is planned that GEDI will continue to measure, which will allow us to further integrate data and expand our analysis.  
For this study, the forest cover map from TanDEM-X was chosen. Please note, other forest cover maps are also available, such as the map from [PERSON] et al. [6]. Most of these forest cover maps are mainly based on passive optical satellite signals. These have their limitations for tropical forests, mainly due to frequent cloud cover. The advantage of the TanDEM-X map is that radar measurements are almost unaffected by cloud cover.  
[PERSON] et al. conducted a fragmentation study in the Amazon rainforest and found that reforestation with secondary forest could reduce the edge effects [45]. This study showed that the biomass in edge forest areas or highly fragmented forest areas is lower and reforestation of the affected areas would be useful to better protect the forest fragments from edge effects.  
In summary, the validated workflow from [PERSON] et al. [16] and [PERSON] et al. [3] used for this study produced reasonable results and has the capability to transfer this model-data fusion to other continents (Africa and Asia). This could be used to investigate whether our results for the Amazon apply to other tropical regions.
# OTSU Multi-Threshold Image Segmentation Based on Improved Particle Swarm Algorithm
## 4 Multi-Threshold Segmentation Algorithm Based on Improved PSO
### Linear Optimization Learning Factor  
For the problem of low segmentation accuracy of the classical particle swarm algorithm, a linear function is introduced in the process of parameter optimization of the particle swarm algorithm to update the learning factor of particles, and the changes in learning factors \(c_{1}\) and \(c_{2}\) affect individual and swarm particles to approach the optimal solution, respectively. Most of the search results from global search to local search are particles, and the centers of gravity of particle searches are also different at different stages. To mitigate this issue, this paper uses asynchronous linear decrement and asynchronous linear incrementing learning factors to balance global search and local search, so that the learning factor can change with the procedure of iterations. According to the characteristics of the PSO algorithm, in the early iterations, it is more likely to rely on the mobile experiences of particle individuals for global search, while in the later iteration, it mainly relies on the mobile experience of the group for local search. The PSO algorithm can enhance the global search ability by taking the larger value of the learning factor \(c_{1}\) the smaller value of \(c_{2}\) in the early iterations, and the smaller value of \(c_{1}\) the larger value of \(c_{2}\) in the later stages can enhance the local search ability. In order to better play the global search ability and local search ability at different stages and improve the accuracy of the segmentation, we use the asynchronous change learning factor. The formula is:  
\[c_{1}(k)=c_{min}+(c_{max}-c_{min})(1-k/k_{max}) \tag{10}\]  
\[c_{1}(k)=c_{min}+(c_{max}-c_{min})(1-k/k_{max}) \tag{11}\]  
where \(c_{max}\) and \(c_{min}\) are the maximum and minimum values of the learning factor, \(c_{max}=2.5\), \(c_{min}=0.5\); \(k\) and \(k_{max}\) are the current number of iterations and the maximum number of iterations, respectively. The improved speed update formula is:  
\[v_{id}^{k+1}=\omega v_{id}^{k}+c_{1}(k)(p_{id}-z_{id}^{k})+c_{2}(k)(p_{gd}-z_ {id}^{k}) \tag{12}\]
# A Comprehensive Review of Open Caisson Modeling Technology: Current Practices and Future Prospects
###### Abstract  
The rapid advancement of modern megapolises has led to a dearth of surface space, and, in response, engineers have begun to trial substitutes below ground level. Shafts are generally used to provide temporary access and permanent work to the subsurface for tunnelling, as well as for lifts or ventilation purposes. In urban areas, one important design issue is the prediction of the excavation-induced displacements by open caisson shaft construction. Settlements and ground movements associated with open caisson shafts are influenced by the choice of construction method, soil composition, and excavation geometry. Compared with other geotechnical construction events, for instance, tunnelling, the literature relating to the ground deformations induced from open caisson shafts are comparatively limited. This review offers an evaluation of several case studies that utilize experimental and computational modeling techniques to provide clearer insights into earth pressure distribution and induced surface and subsurface soil displacements, as well as the associated ground deformations during open caisson shaft construction. The modeling test results are compared to the state of the practice ground deformation prediction theories and measured results from field monitoring data. Findings indicate that the lateral earth pressure distribution aligns closely with the theoretical predictions based on [PERSON]'s and [PERSON]'s models, and lateral earth pressure diminishes gradually until the onset of active wall displacement. Current modeling techniques generally fail to properly represent in situ stress states and large-scale complexities, emphasizing the need for hybrid approaches that combine physical and numerical methodologies. In future studies, modern approaches, including artificial intelligence (AI) monitoring (e.g., PINNs, ACPP), multi-field coupling models (e.g., THMC), and transparent soil testing, hold profound potential for real-time prediction, optimization, and visualization of soil deformation. Numerical-physical coupling tests will integrate theory and practice. Improving prediction reliability in complicated soil conditions such as composite and heterogenous strata using different modeling techniques is still unclear, and further investigation is therefore needed.  
open caisson shafts; soil-structure interaction; earth pressure; ground displacement; modeling; coupling; (n-p)
# A Deep Learning Approach for Automatic Hate Speech Detection in the Saudi Twittersphere
## 5 Experiments
### Experimental Settings  
**Implementation:** We implemented all neural network models using Keras library with a TensorFlow as a backend [40], while we implemented BERT using huggingface Pytorch library [41]. We ran the experiments on Google Collaboratory [42], which provides a free Jupyter notebook environment with GRU accelerator.  
**Hyperparameter tuning:** In all experimented models, we employed a grid search using fivefold cross-validation on the training set to find the optimal training-specific hyperparameters, such as the batch size, class weights, and learning rate, with F1-score as a scoring metric. We also used a grid search using fivefold cross-validation to find the best model-specific hypermeters such as the number of hidden layers, the number of units, pooling layer size, and the dropout rate. However, due to the limitation of time and computational resources, some of our model-specific hypermeters were selected on the basis of empirical findings reported previously, determined empirically through cross-validation over the training dataset (without grid search) or set to default values. All proposed network architectures were experimented with binary cross entropy as a loss function and \"Adam\" as an optimizer. We modified the loss function to incorporate class weights, since our dataset is relatively imbalanced, and we were particularly interested in increasing the recall of the positive class (hate class). Then, we evaluated all network architectures with and without incorporating class weights.  
When fine-tuning the BERT model, all hyperparameters were kept the same as in pretraining except batch size, learning rate, and number of training epochs. To select the best performing values, we performed a grid-search through fivefold cross-validation on a training set with F1 as a scoring metric. For batch size and number of epochs, we searched according to the suggested ranges in [14].  
\begin{table}
\begin{tabular}{c c c} \hline \hline  & **GHSD** & **RHSD (Testing)** \\ \hline \# Tweets & 8964 & 600 \\ \hline \# Hate instances & 2539 & 250 \\ \hline \# Non-hate instances & 6425 & 350 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Statistics of datasets used in the experiments.  
For the learning rate, we found that the suggested range produced poor results, and thus we extended the searched range to be \(\{0.2\times 10^{-5},\,0.3\times 10^{-5}\ldots\,\,3\times 10^{-5},\,5\times 10^{-5}\}\).
# A Novel Fiber Optic Sensor for Microparticle Velocity Measurement Using Multicore Fiber
###### Abstract  
A novel fiber optic sensor is proposed to achieve the measurement of single microparticle velocity in a two-dimensional space by translating the cores of a multicore fiber in their respective directions into independent spatial filters. The measurement of particle velocity with the accuracy of better than 2% and direction angles in a range from 0\({}^{\circ}\) to 120\({}^{\circ}\) is achieved. Compared with the previous method by means of manual-assembled fiber array, the measurement accuracy can be increased by using only one multicore fiber probe with the features of compact size and low cost. The technique has potential particle velocity measurement applications in environmental monitoring, biological study and combustion research.  
velocity measurement; multicore fiber; spatial filter method +
Footnote †: journal: Applied Physics Letters
## 2 Preliminaries and Problem Formulation
### Preliminaries  
In this paper, a directed graph \(\tilde{\mathcal{G}}\) is used to represent the communication state between agents. \(\tilde{\mathcal{G}}\) consists of a set of nodes \(\mathcal{V}=\{\mathcal{V}_{1},\mathcal{V}_{2},\cdots,\mathcal{V}_{n}\}\) and a set of edges between nodes \(\mathcal{E}\in\mathcal{V}\times\mathcal{V}\). In the graph, \((i,j)\in\mathcal{E}\) means that agent \(i\) can receive the date from agent \(j\). A self-loop connection is not considered here, i.e., \((i,i)\
otin\mathcal{E}\). \(A=[a_{ij}]\in\Re^{n\times n}\) represents the connectivity matrix, and the elements of \(A\) are demonstrated as: (1) \(a_{ij}=1\) if \((i,j)\in\mathcal{E}\), (2) \(a_{ij}=0\) if \((i,j)\
ot\in\mathcal{E}\). As \((i,i)\
otin\mathcal{E}\), which means that \(a_{ii}=0\). \(\triangle=\text{diag}(\triangle_{i})\in\Re^{n\times n}\) represents the in-degree matrix and \(\triangle_{i}=\sum_{j\in N_{i}}a_{ij}\). \(L=\triangle-A\) is the Laplacian matrix of \(\tilde{\mathcal{G}}\). Use \(\mu_{i}\) to indicate whether the data of the leader can be transferred to other nodes. \(\mu_{i}=0\) indicates that follower \(i\) can not receive the date from the leader, while \(\mu_{i}=1\) means yes. Define \(U=diag\{\mu_{1},\mu_{2},\cdots,\mu_{n}\}\).