esa-sceva/satcom-chunk-collection huggingface dataset:
# Risk-Sensitive Policy with Distributional Reinforcement Learning
## 2 Literature Review  
The core objective of the classical RL approach is to learn optimal decision-making policies without any concerns about the risk or safety [1]. The resulting policies are said to be _risk-neutral_. Nevertheless, there are numerous real-world applications requiring to take the risk into consideration in order to ensure safer decision making [3]. Two main approaches can be identified to achieving a safe RL. Firstly, the optimality criterion can be modified so that a safety factor is included. Secondly, the exploration process can be altered based on a risk metric [9]. These techniques give rise to _risk-sensitive_ or _risk-averse_ policies.  
Scientific research on risk-sensitive RL has been particularly active for the past decade. Various relevant risk criteria have been studied for that purpose. The most popular ones are undoubtedly the _mean-variance_[10; 11; 12] and the (_Conditional_) _Value at Risk_ (CVaR) [13; 14; 15]. Innovative techniques have been introduced for both _policy gradient_[16; 17; 18] and _value iteration_[19; 20; 21; 22; 23] approaches, with the proposed solutions covering both discrete and continuous action spaces. Additionally, risk-sensitive methodologies have also been studied in certain niche sub-fields of RL, such as robust adversarial RL [24], but also multi-agent RL [25].  
Focusing on the _value iteration_ methodology, the novel distributional RL approach [8] has been a key breakthrough, by providing access to the full probability distribution of the random return. For instance, [20] suggests achieving risk-sensitive decision-making via a distortion risk measure. Applied on top of the IQN distributional RL algorithm, this is, in fact, equivalent to changing the sampling distribution of the quantiles. In [21], a novel actor-critic framework is presented, based on the distributional RL approach for the critic component. The latter work is extended to the offline setting in [22], since training RL agents online may be prohibitive because of the risk inevitably induced by exploration. Additionally, [23] introduces the _Worst-Case Soft Actor Critic_ (WCSAC) algorithm, which is based on the approximation of the probability distribution of accumulated safety-costs in order to achieve risk control. More precisely, a certain level of CVaR, estimated from the distribution, is regarded as a safety constraint.  
In light of this literature, the novel solution introduced in this research paper presents key advantages. Firstly, the methodology proposed is relatively simple and can be applied on top of any distributional RL algorithm with minimal modifications to the core algorithm. Secondly, the proposed approach can span the entire potential trade-off between risk minimisation and expected return maximisation. According to the user's needs, the policy learnt can be risk-averse, risk-neutral or in between the two (risk-sensitive). Lastly, the solution presented improves the interpretability of the decision-making policy that was learnt.
## 3 The Construction of Dataset
### Principle of Pulse Shape Formation  
The differences in pulse shapes among charged particles primarily result from energy deposition, the plasma erosion effect, and the charge transit effect [19]. Heavy ions, compared with lighter ions of the same energy, induce denser ionization due to their shorter range and higher ionization density. Thus, the pulses generated by heavy ions generally exhibit longer durations, lower amplitudes, and slower rise times. These distinctive pulse characteristics offer valuable information about the particle, forming the basis for the application of deep-learning techniques in particle identification.
# Releasing Differentially Private Trajectories with Optimized Data Utility
## 5 Experimental Evaluations
### Evaluation Setup  
**Datasets.** Three real-world datasets were used for the experimental evaluations in this section. (i) The _GeoLife_[30] dataset contains a large number of moving trajectories collected from 182 mobile individuals over three years. (ii) The _Govalla_ dataset contains an undirected social network and check-ins collected by the Stanford Network Analysis Project (SNAP) from Gowalla, a popular location-based social network (LBSN), throughout 2008-2010 [33]. The social network contains friendships among individuals. A check-in (also called a record of location visit) is composed of an identity number, a location represented by the GPS latitude and longitude coordinates, the date and time when the individual visits that location. (iii) Another dataset, _Brightkite_, with a similar structure, was collected from a popular LBSN (Brightkite) [34] between 2009 and 2011.  
**Data preprocessing.** Note that there is a broad spread of locations as well as a long period in all datasets. To avoid a dramatic increase of time and memory overhead caused by high sparsity, we took the following steps to obtain a subset from each dataset. First, we restricted our evaluations to check-ins taking place in Beijing for GeoLife, Stockholm for Gowalla, and San Francisco for Brightkite. Then, only the individuals with at least 20 check-ins, whom we considered as active individuals, were retained. We present the statistical characteristics of the datasets after preprocessing in Table 3.  
**Competitors.** We compare UDPT with two state-of-the-art location privacy protection algorithms. (i) _AdaTrace_ is a differentially private publishing mechanism for trajectories [6]. It leverages an exponential mechanism to probabilistically merge locations based on location distances and then releases synthesized trajectories in a differentially private manner. Compared with other algorithms based on trajectory synthesis, such as [19; 20], AdaTrace claimed better preservation of multiple kinds of geographical utility including query error and trajectory length, so we choose it as our competitor. (ii) _MLCE_ is a mechanism combined with loss and conditional entropy [12]. It prefers choosing locations nearby actual locations as the obfuscated locations. Based on \(\epsilon\)-differential privacy, MLCE adds a remapping to the mechanism's output. Since the remapping is guided by two metrics, worst-case quality loss and conditional entropy, the authors of MLCE argue that higher utility and stronger privacy guarantee can be achieved, thus we choose it as our competitor.
# Resource Allocation for GBR Services in D2D-Enabled Communication
## 1 Introduction
However, to the best of our knowledge, there is no previous research on resource allocation for GBR services in D2D-enabled communications which jointly solves the transmit power control, CU-DU-channel matching, sharing mode selection, and channel bandwidth. In D2D communications, multiple DUs can be allowed to be assigned to one channel [18; 19; 20]. Conversely, multiple channels were assigned to one DU in [21]. In this way the data rate of the DU can be satisfied with higher probability, even though the SINR of the DU is very low. However, the increase in throughput was not significant and the complexity rapidly increased with the number of users performing multiple assignments. In this paper, we do not consider multiple assignments, which makes it rather straightforward to extend the work with additional computational complexity. The contributions of this paper are as follows. * In our previous study [22], a CU unconditionally shared the channel resource with a DU in a D2D underlaid cellular network. In contrast, in this paper, we consider a D2D-enabled communication where the channel sharing mode is considered, i.e., two orthogonal resources are allocated for a channel in a dedicated mode, or one single resource is shared in a shared mode. * In this paper, the SNR-based transmit power control is adopted. However, since the channel bandwidth can be adaptively varied to satisfy the required target rate depending on the experienced channel capacity, the transmit power also needs to be adaptively controlled, based on the channel bandwidth, as well as channel quality. * The Lagrangian relaxation is used for constraint relaxation in the 3-D matching problem with a 3-tuple of CU, DU, and channel. And, two 2-D problems are introduced to solve the problem iteratively with a sub-gradient-based bounding procedure. This sub-optimal problem is less computationally complex than the optimal 3-D matching problem, and it has little performance degradation. When the number of CUs is different from that of DUs, the 3-D matching problem becomes an unbalanced assignment problem, which can be solved by introducing dummy users based on a modified Hungarian method.
# Characteristics and Causes of Coastal Water Chemistry in Qionghai City, China
###### Abstract  
The coastal zone area of Qionghai City is one of the important coastal zones in the South China Sea, and its water environment has been affected by human activities such as urbanization and industrialization. In order to protect the water resources and ecological environment of this area, the water chemistry characteristics of the main watersheds and their causes in the coastal zone area of eastern Hainan Island were investigated to provide a scientific basis for environmental protection and sustainable development. In this study, the characteristics and sources of water chemical ion components were analyzed using a Piper trilinear diagram, Gibbs diagram, and correlation analysis with the coastal zone area of Qionghai city as the research object. The results show the following: (1) the dominant cation of water chemistry in the coastal zone of Qionghai City is Na\({}^{+}\) with a mean value of 35.001 mg\(\cdot\)L\({}^{-1}\), and the dominant anion is Cl\({}^{-}\) with a mean value of 30.69 mg\(\cdot\)L\({}^{-1}\); (2) the dominant cation content in the coastal zone of Qionghai City is Na\({}^{+}\)\(>\) Ca\({}^{2+}\)\(>\) Mg\({}^{2+}\)\(>\) K\({}^{+}\), and the dominant anion content is Cl\({}^{-}\)\(>\)SO\({}_{4}^{2-}\)\(>\) HCO\({}_{3}^{-}\)\(>\) CO\({}_{3}^{2-}\); (3) at the five collection sites in the study area, the ion concentrations showed different trends, with the highest ion concentration in the water samples collected from aquaculture ponds, and the main water chemistry type was Na-Cl; the lowest ion concentration was in the water samples collected from the rivers, and the main type of water chemistry was Ca-Mg-HCO\({}_{3}\). The source of water chemistry ions in the study area mainly included seawater, rock weathering, atmospheric precipitation, and evaporation concentration. The results of this study can provide a scientific basis for the development, utilization, and management of local water resources and provide basic data for environmental protection and sustainable development.  
Qionghai coastal zone; water chemical characteristics; cause analysis; source of ions +
Footnote †: journal: _Applied Sciences_  
0023
# An Upgraded Version of the Binary Search Space-Structured VQ Search Algorithm for AMR-WB Codec
## 2 ISF Quantization in AMR-WB
### Quantization of ISF Coefficients  
Ahead of a quantization process, a mean-removed and a first order moving average (MA) filtering are performed on the ISF coefficients to obtain a residual ISF vector, expressed as  
\[\mathbf{r}(n)=\mathbf{z}(n)-\mathbf{p}(n), \tag{6}\]  
where \(\mathbf{z}(n)\) and \(\mathbf{p}(n)\) respectively represent the mean-removed ISF vector and the predicted ISF vector at frame \(n\) using a first order MA prediction. The latter expression is defined as  
\[\mathbf{p}(n)=\frac{1}{3}\mathbf{\hat{r}}(n-1), \tag{7}\]  
where \(\mathbf{\hat{r}}(n-1)\) denotes the quantized residual vector at the previous frame.  
S-MSVQ is then performed on \(\mathbf{r}(n)\). Table 1 gives the structure of S-MSVQ for AMR-WB in the 8.85-23.85 kbps coding modes. In stage 1, \(\mathbf{r}(n)\) is split into two subvectors, that is, a nine-dimensional subvector \(\mathbf{r}_{1}(n)\) and a seven-dimensional subvector \(\mathbf{r}_{2}(n)\), associated with codebooks CB1 and CB2 respectively, for VQ encoding. In the beginning of stage 2, the quantization error vectors are split into five subvectors, symbolized as \(\mathbf{r}_{i}^{(2)}=\mathbf{r}_{i}-\mathbf{\hat{r}}_{i},i=1,2\), respectively. For example, \(\mathbf{r}^{(2)}\),1-3 in Table 1 denotes the subvector split from the first to the third components of \(\mathbf{r}_{1}\), on which VQ encoding is then performed over codebook CB11. Similarly, \(\mathbf{r}^{(2)}\),2,4-7 represents the subvector split from the 4 th to the 7 th components of \(\mathbf{r}_{2}\), on which VQ encoding is then performed over codebook CB22. Lastly, the Euclidean distance is used as a measure of the squared error ISF distortion in all the quantization processes.
# End to End Delay and Energy Consumption in a Two Tier Cluster Hierarchical Wireless Sensor Networks
## 7 End-To-End Delay Analysis
### Local and Exogenous Traffic  
For each CH, the arrival local (exogenous) traffic is given by the \(A_{i}(z)\) factors located at the CONT (TDMA) sub-frame. Let us denote by\(r_{L}\) (\(r_{E}\)) be the fraction of local (exogenous) traffic managed by a given CH. \(r_{L}\) (\(r_{E}\)) is given by,  
\[r_{L}=\frac{L^{\prime}(1)}{L^{\prime}(1)+\sum_{i\in TDMA}A^{\prime}_{i}(1)}= \frac{L^{\prime}(1)}{F^{\prime}(1)}=1-r_{E}. \tag{22}\]  
Then, taking into account Equation (22), Equation (21) can be split into the following three terms,  
\[\overline{W}(\text{st, ms})=r_{L}CT+\sum_{i\in TDMA}\frac{A^{\prime}_{i}(1)}{ F^{\prime}(1)}R_{i}T+\overline{W}^{\prime}. \tag{23}\]  
In Equation (23), the first terms refers to the local traffic; \(CT\) is the distance in number of mini-slots between the end of the successful slot \(\mathcal{C}\) for our tagged data packet and the end of the first available transmission slot \(\mathcal{T}\) of our CH in the TDMA sub-frame, see the values in the fourth column of Table 4. The second term of Equation (23) deals with the exogenous traffic; \(R_{i}T\) is the distance in number of mini-slots between the end of the reception slot \(\mathcal{R}_{i}\) and the end of the transmission slot \(\mathcal{T}\) of our CH (notice that the reception slot \(\mathcal{R}_{i}\) is coincident with the transmission slot \(\mathcal{T}_{i}\) of some CH in the neighboring outer ring), see Tables 4 and 6. Finally \(\overline{W}^{\prime}\) is the residual mean sojourn time in a given CH which is common to the local and to the exogenous traffic streams (we have not taken into account the option of priorities).  
Therefore, to evaluate each term in Equation (23) we proceed as follows. First, following the model in Section 7.1 we obtain \(\overline{W}(\text{st, ms})\), Equation (21); second we follow the analysis of Section 7.2 and we get the fractions \(r_{L}\) and \(r_{E}\), Equation (22), i.e., the first two terms in Equation (23), and third, we derive \(\overline{W}^{\prime}\) by subtracting the aforementioned two first terms from the total, \(\overline{W}(\text{st, ms})\). Observe that for a very low traffic load, only the first two terms of Equation (23) will contribute to \(\overline{W}(\text{st, ms})\) since \(\overline{W}^{\prime}\) will be practically null. This fact has been validated through the use of a parallel queue model outlined in Section 7.5.
## Materials and Methods
### Measurements  
The Noraxon Desktop DTS 8-channel wireless electromyography (EMG) system (Noraxon Ltd., Scottsdale, AZ, USA) was used in combination with bi-polar Al/AgCl surface electrodes. EMG data were recorded through analysis software MR 3.8 (Noraxon Ltd., USA) at a sample rate of 2000 Hz. EMG data of seven muscles on the right leg was recorded. The seven muscles are rectus femoris (RF), vastus medialis (VM), vastus lateralis (VL), semitendinosus (ST), biceps femoris (BF), gastrocnemius lateralis (GL), and peroneus longus (PL). Maximal voluntary contraction values (MVC) were also recorded. The MVC was realized by Isometric contraction. The participants sat in a self-contained chair designed by the Isomed2000 testing system. Knee joint angles of 105\({}^{\circ}\) and 150\({}^{\circ}\) were selected. Subjects performed a 6 s maximal voluntary contraction. The bi-polar surface electrodes were located in the positions shown in Figure 5, which complied with SENIAM recommendations ([[http://www.seniam.org](http://www.seniam.org)]([http://www.seniam.org](http://www.seniam.org))). Raw EMG signals were first filtered using a notch filter (50 Hz) to reduce effects of alternating current. Then a band-pass (10-500 Hz) was used to reduce potential effects of transitional adjustments. The integration was expressed as a percentage of the peak voltage of the MVCs. In each of these trials, EMG root mean square (RMS) values were obtained. Similar processing was done for raw EMG values obtained during reference contractions. Mean RMS values obtained during the task were then normalized (nRMS) to the corresponding reference RMS values.  
Figure 4: The extended NASA-TLX rating scales [19].  
Figure 5: The body parts and points where the bi-polar surface electrodes were located.  
An extended NASA TLX rating scale is used to record the discomfort of the size indices [19]. After each subject complete the eight trials, the TLX rating scales were acquired.
## 2 Experimental Setup and Materials
### Milk Samples and Somatic Cells  
To calibrate the sensor, we carried out experiments with a series of commercially produced cow milk samples with nominal fat percentages of 0.05%, 0.5%, 1.5%, 3.2%, 6%, and 10% (milk cream) using UHT pasteurization and homogenization. The protein content was claimed to be 3.0 g per 100 g of product for all of these milk samples.  
Figure 3: Block diagram of the pumping system used for testing the sensor: (1) milk container; (2) centrifugal pump; (3) laser scatterometric sensor.  
Figure 2: Photograph of the prototype sensor shown schematically in Figure 1: (1) sensor enclosure; (2) laser diode holder; (3) herringbone fitting for 14 mm hose; (4) silicone tube connected to the pumping system; (5) controller unit.  
To study the sensor response to the presence of somatic cells in milk, we performed experiments with MCF-7 human breast carcinoma cells (ATCC no. HTB-22), of which the mean size is \(\sim\)20 \(\upmu\)m. These cells are a relevant model of somatic cells that appear in the milk of mammals, that of cows in particular, due to mastitis. For routine procedures with cells, the following analytical grade reagents were used: DMEM/F12 medium, fetal bovine serum (FBS), L-glutamine, penicillin-streptomycinycin, and 0.05% trypsin-EDTA solution (PanEco, Moscow, Russia). Cells were cultured in T-25 flasks (TPP, Trasadingen, Switzerland) at 37 \({}^{\circ}\)C and 5% CO\({}_{2}\) in an S-Bt Smart Biotherm incubator (Biosan, Riga, Latvia). The medium used was DMEM containing 10% FBS, 2 mM L-glutamine, 100 units/mL penicillin, and 100 \(\upmu\)g/mL streptomycin. Cells were cultured for 7 days to achieve maximum confluency (\(\sim\)98%). Before the experiment, the cells were detached from the surface of the culture flask using a 0.05% trypsin-EDTA solution. Trypsin was inactivated with 10% FBS solution, cells were centrifuged at 350 g for 5 min, and then resuspended in DMEM medium without additives and counted using a counting chamber and a laboratory inverted microscope. Before being added to milk, the cell suspension was diluted so as to obtain the resulting concentration of cells in milk at a level of 10\({}^{6}\) cm\({}^{-3}\).
# Handover-Driven Interference Management for Co-Channel Deployment of Femto- and Macro-Cells
## 6 Performance Evaluation
### Evaluation Model
#### 6.1.1 System and Network Model  
We follow the simulation scenario of 3 GPP [30] for macro- and femtocell deployment. A single-sector macrocell on which multiple femtocells are overlaid is considered. The radius of the macrocell is 500 m and macrocell users are randomly distributed in the macrocell; the angle and the distance of each to the macro BS are randomly chosen with a uniform probability distribution. Femto BSs are randomly distributed within a macrocell and each of them serves a single subscribed user who is also randomly distributed within 50 m of it.  
Both macro and femto BSs operate in the frequency band of 2.5 GHz with a channel bandwidth of 5 MHz. A slot is defined as one unit of time and each slot interval is 1 ms. Transmit power changes at a slot time scale. For RB allocation, a scheme proposed by [PERSON] et al. [29] is used; first, it sequentially allocates each user the best resource block in terms of the expected capacity. The remaining RBs are then allocated to users in descending order of the ratio of the expected capacity to the pre-determined user weight. In the simulation, all users within a cell are given an identical weight. The minimum and maximum transmit powers inherently given to user devices are assumed to be \(-50\) and 25 dBm, respectively. The handover controller, if activated, makes handover decisions (at most 10 handovers at once) every 100 slots. The performance of HO-FIM (denoted by _per-user control_) was compared with the range control (named ARCHoN) of [PERSON] and [PERSON] [22]. The system load is defined as the sum of the transmitting powers of all users.  
\begin{table}
\begin{tabular}{p{113.8 pt} p{113.8 pt}} \hline \hline
**Parameter** & **Value** \\ \hline Network deployment & Femtocell: random deployment within a macrocell \\  & User: random placement within each’s initial serving cell \\ \hline Carrier frequency & 2.5 GHz \\ \hline System bandwidth & 5 MHz (10 resource blocks) \\ \hline Number of macrocell users in a macrocell at initialization & [10; 20; 50] \\ \hline Number of femtocell users in a macrocell at initialization & [10; 30; 50; 100] \\ \hline Number of femtocells in a macrocell & [10; 30; 50; 100] \\ \hline Transmit power range of a user & \(-50\)\(\sim\)25 dBm \\ \hline Fading & Frequency-selective Rayleigh fading with a maximum doppler spread of 30 Hz \\ \hline Shadowing & Lognormal shadowing with a standard deviation of 8 dB \\ \hline Minimum distance & 30 m \\ between a macro BS and its user & \\ \hline Thermal noise density & \(-174\) dBm/Hz \\ \hline User antenna gain & 0 dBi \\ \hline Slot interval & 1 ms \\ \hline Scheduling algorithm & [PERSON] et al.’s [29] \\ \hline \hline \end{tabular}
\end{table}
Table 1: Simulation parameters.
## 4 Numerical Simulation of the Model
### Convergence Criterion and Computational Time  
In order to set an adequate level of convergence, the guidelines proposed by the European Research Community on Flow, Turbulence and Combustion [34] were followed. This way, when stating that a case was converged, the scaled residuals of all the variables were always less than \(1.1\times 10^{-5}\). Different flow magnitudes were monitored in a cross-section located 20 mm downstream from the screen in each case, and it was verified that there was no change in the 4 th significant digit when stopping the simulations.  
To perform the simulations, an HP Proliant DL 585 G7 server (DL 585 G7, HP, Leon, Spain, 2013) that had 32 processors at 2.8 GHz, 64 GB of shared RAM memory and ran the Linux operating system was used. The communication library used was Platform MPI, and the required CPU time for the simulation was below one minute. It must be highlighted that for the simulations by [PERSON] et al. [17], the computational time with the same server oscillated between 66 and 360 h. Moreover, even using a not very powerful computer, no more than 10 min (less than 0.25% of the detailed model computation time) is needed in order to reach a correct convergence.