esa-sceva/satcom-chunk-collection huggingface dataset:
# Comparative Analysis of Skew-Join Strategies for Large-Scale Datasets with MapReduce and Spark
## 5 Conclusions  
Implementations of MapReduce are used to perform many operations on very large datasets, including join operations. It has become a prominent parallel and distributed programming model for efficiently handling massive datasets. The major obstacle of the model for join processing is data skew. If the data are significantly skewed, with some common join keys appearing to have a greater frequency than the others, the reduction tasks of these join keys will finish later than those of the others. Thus, any benefits from parallelism become meaningless. There are some algorithms that have been proposed to solve the problem of skew joins. Several surveys on solutions for skew joins have been made, but an adequate and systematic comparison in the Spark environment has still not been presented. Thus, this work was designed to provide a comprehensive comparison of several skew-join algorithms with mathematical models and experiments. We fully evaluated the hash-based partition, range-based partition, and multi-dimensional range partition algorithms in MapReduce on the Spark framework. We provided an analysis of the advantages and disadvantages of each algorithm. The cost model built was an important theoretical basis for the evaluation and comparison of the skew-join algorithms. Lastly, the experiments were conducted in Spark, the new generation of Big Data processing. Through the cost models and experimental results, this research presented a comparison of the three algorithms. This is a highly scientific contribution, since join operations are commonly used in Big Data environments. In the scope of this work, we only provided an evaluation of the three algorithms. It is necessary to conduct an investigation and evaluation of more skew-join algorithms to have an overview of the problem of skewed data processing. In addition, Apache Spark 3 was introduced to dynamically handle skew in sort-merge join operations by splitting and replicating skewed partitions. It would be interesting to compare the data-skew handling provided by Spark and the user-defined data-skew handling with other algorithms.  
Conceptualization, T.-C.P. and A.-C.P.; methodology, A.-C.P., H.-P.C. and T.-C.P.; software, T.-C.P. and T.-N.T.; validation, A.-C.P. and H.-P.C.; formal analysis, A.-C.P. and T.-C.P.; investigation, A.-C.P. and T.-C.P.; resources, A.-C.P. and T.-C.P.; data curation, T.-N.T. and H.-P.C.; writing--original draft preparation, T.-C.P. and T.-N.T.; writing--review and editing, H.-P.C. and A.-C.P.; visualization, T.-N.T. and H.-P.C.; supervision, T.-C.P.; project administration, A.-C.P. All authors have read and agreed to the published version of the manuscript.  
This research received no external funding.  
Not applicable.  
Data Availability Statement: Data are available on request by contacting the corresponding author.  
The authors declare no conflict of interest.
# Catalytic Degradation of 4-Ethylpyridine in Water by Heterogeneous Photo-Fenton Process
###### Abstract  
In this work, the degradation of 4-ethylpyridine (4 EP) in water by a heterogeneous photo-Fenton process (H\({}_{2}\)O\({}_{2}\)/Fe\({}_{3}\)O\({}_{4}\)/ultraviolet irradiation (UV)) was investigated. More rapid and effective 4 EP degradation was obtained with H\({}_{2}\)O\({}_{2}\)/Fe\({}_{3}\)O\({}_{4}\)/UV than Fenton-like (H\({}_{2}\)O\({}_{2}\)/Fe\({}_{3}\)O\({}_{4}\)) and UV/H\({}_{2}\)O\({}_{2}\), which is due to the larger production of hydroxyl radicals from the chemical and photolytic decomposition of H\({}_{2}\)O\({}_{2}\). The operational conditions were varied during 4 EP degradation experiments to evaluate the effects of pH, catalyst, concentration, and temperature on the kinetics and efficiency of H\({}_{2}\)O\({}_{2}\)/Fe\({}_{3}\)O\({}_{4}\)/UV oxidation. Under optimal conditions (100 mg/L 4 EP, [H\({}_{2}\)O\({}_{2}\)] = 1000 mg/L, Fe\({}_{3}\)O\({}_{4}\) = 40 mg/L, pH = 3 and room temperature, 300 rpm), 4 EP was totally declined and more than 93% of the total organic carbon (TOC) was eliminated. Liquid chromatography analysis confirmed the formation of aromatic and aliphatic intermediates (4-hydroxypridine, 4-pyridone, malonic, oxalic, and formic acids) that resulted in being mineralized. Ion chromatography analysis demonstrated the stoichiometric release of NH4+ ions during 4 EP degradation by heterogeneous photo-Fenton oxidation. The reuse of the heterogeneous catalyst was evaluated after chemical and heat treatment at different temperatures. The heat-treated catalyst at 500 \({}^{\circ}\)C presented similar activity than the pristine Fe\({}_{3}\)O\({}_{4}\). Accordingly, heterogeneous photo-Fenton oxidation can be an alternative method to treat wastewaters and groundwater contaminated with pyridine derivatives and other organic micropollutants. The combination of heterogeneous photo-Fenton oxidation with classical biological methods can be proposed to reduce the overall cost of the treatment in large-scale water treatment plants.  
Keywords:pyridine derivatives; catalyst; Fenton oxidation; UV irradiation; hydroxyl radicals; degradation +
Footnote †: journal: Applied Soft. 2019, 9, 5073; doi:10.3390/appr9235073
## 4 Energy Efficiency
### Transmission Power  
Given the transmission scheme and channel statistics, the SNR required for achieving a given BER depends on the detector. Consider channels \(\mathbf{H}\) with both small-scale Rayleigh fading and large-scale fading arising from path loss. We assume that the large-scale fading between a UE and the BS antennas is the same. Following [3; 33] suppose the UEs are uniformly distributed in a circular cell with radius \(\text{d}_{\text{max}}\) and minimum distance \(\text{d}_{\text{min}}\)  
\[d_{\text{min}}\leq d_{k}\leq d_{\text{max}},1\leq k\leq K, \tag{31}\]  
where \(d_{k}\) is the distance between UE-\(k\) and the BS, and the path loss caused by UE-\(k\) due to large-scale fading is modeled as  
\[\varphi_{k}=\frac{\bar{d}}{d^{\kappa}} \tag{32}\]  
where \(\kappa\geq 2\) is the path loss exponent, and the constant \(\bar{d}\geq 0\) regulates the channel attenuation at distance \(d_{\text{min}}\). In the channel model, the \((n,k)\)-th element in \(\mathbf{H}\) follows \(H_{nk}=\sqrt{\varphi_{k}}\widetilde{H}_{nk}\), where \(\varphi_{k}\) represents the path loss defined by (32) and \(\widetilde{H}_{nk}\) characterizes the independent Rayleigh fading between UE-\(k\) and the \(n\)-th BS antenna. We can write  
\[\mathbf{H}=\text{diag}(\sqrt{\varphi_{1}},\sqrt{\varphi_{2}},\cdots\sqrt{ \varphi_{K}})\widetilde{\mathbf{H}} \tag{33}\]  
where \(\widetilde{\mathbf{H}}\) is the Rayleigh fading component of \(\mathbf{H}\). We assume that the target BERs for the different UEs are equal and so are the average received SNR defined in (3). Assume that \(\text{SNR}_{\text{RX}_{k}}=\text{SNR}_{0}\) isrequired for achieving a target BER, i.e., BER\({}_{0}\). Then the transmitted power is allocated according to the large-scale fading to meet the requirement. As such, the total transmission power is  
\[\mathrm{P}_{\mathrm{DT}}=\sum_{k=1}^{K}\frac{\mathrm{SNR}_{0}}{\varphi_{k}}B \sigma^{2} \tag{34}\]  
where \(\sigma^{2}\) denotes the noise variance (in Joule/symbol), \(B\) is the transmission bandwidth, and \(B\sigma^{2}\) denotes the total noise power.
# Optics Based Label-Free Techniques and Applications in Brain Monitoring
## 5 OCT in Brain Imaging
### Current Clinical Trials Using Integrated OCT in Human  
[PERSON] et al. [152] experimented the visualization of the human subarachnoid space and brain cortex explored using OCT as a suitable in vivo neuroimaging modality of subarachnoid  
Figure 4: Basic optical coherence tomography (OCT) setup based on Michelson interferometer.  
space. Patients (n = 26) with front lateral craniotomy were experimented on. The temporal and frontal archnoid mater and adjacent anatomical structures were scanned using microscope-integrated three-dimensional OCT (iOCT). Analysis revealed a detailed depiction of the subarachnoid space (SAS) (76.9%) with delineation of the internal microanatomical structures such as the aarchoid barrier cell membrane (ABCM; 96.2%), trabecular system (50.2%), internal blood vessels (96.2%), pia mater (26.9%) and the brain cortex (96.2%). Orthogonal distance measuring was possible. The SAS showed a mean depth of 570 \(\upmu\)m frontotemporal. The ABCM showed a mean depth of 74 \(\upmu\)m frontotemporal. These results indicate that OCT provides a dynamic, non-invasive tool for real-time imaging of the SAS and adjacent anatomical structures at micrometer spatial resolution.  
In addition, [PERSON] et al. [153] conducted an in vivo study to explore the value of a 3-dimensional microscope-integrated extravascular OCT as the first suitable intraoperative imaging modality of a cerebral aneurysm (CA) and parent vessel wall morphology. Incidental CAs (Patients number = 16) of the anterior circulation with an indication for microsurgical clipping were scanned. The analysis revealed that intraoperative OCT achieved to delineate the microstructural composition of the parent vessel in all cases and the CA wall in 68.8%. Clinical relevant characteristics such as thickness, calcification, residual tunica media and atherosclerotic plaque of CA wall could be demonstrated with high image quality approaching the spatial resolution of histopathology.
# A Deadline-Aware Scheduling and Forwarding Scheme in Wireless Sensor Networks
## 6 Performance Study
### Performance Comparison with Other Algorithms
#### 6.2.2 Effects of Event Rate  
In this subsection, we compare the network performance of the three algorithms for different values of event rate, which change from 0.1 to two packets per second. According to the network parameters, such as node density, delay bound and delay-constrained success ratio, the duty cycle interval is calculated as 18.51 s by using the proposed algorithm. As seen in Figure 6(a), the packet delivery ratio tends to decrease when the event rate increases. However, LPF shows the strongest decline, followed by the DASF and RAW algorithms, because the number of hops under RAW tends to be the smallest compared to LPF and DASF. For instance, when event rate is two packets per second, 99.58% of the packets can be delivered successfully to the sink under RAW followed by 98.34% and 96.85% under DASF and LPF algorithms, respectively. Even though the packet delivery ratio of DASF is slightly lower than that of the RAW algorithm, it is still acceptable for most scenarios in WSNs.  
As far as the latency is concerned, Figure 6(b) shows that the averages of E2E delay remain relatively consistent under the three considered algorithms. The reason is that all network parameters except event rate are set to default values. Compared to the results in Subsection 6.2.1, we can clearly see that average E2E delay depends more on node density than event rate, _i.e._, node density has a great deal of effect on the average E2E delay. In addition, our algorithm can reach the lowest average E2E delay among the three algorithms. For example, when event rate is set to 1.5 packets per second, packets are forwarded to the sink, on average, in 9.8 s under DASF compared to 12.2 s and 27.4 s in cases of LPF and RAW algorithms, respectively.  
Figure 6(c) illustrates the proportion of packets that can satisfy the constraint that E2E delay is under the delay bound (20 s) with the required DCSR (95%). While DASF and LPF can meet the required DCSR, in contrast, RAW does not satisfy. Note that the collected DCSR of our algorithm stays higher than that of LPF. For instance, when the event rate is 1.0 packets per second, 96.9% of the packets can reach the sink successfully within the delay bound compared to 95.5% and 27.3% with LPF and RAW algorithms, respectively.
# A Frame-to-Frame Scan Matching Algorithm for 2D Lidar Based on Attention
## 3 Attention Area Selection Module
### Current Frame Attention Area Selection  
In the current frame, attention area selection determines the data that should be used to match the reference frame to the current frame. In this case, where the attention area of the reference frame and the initial pose (\(x_{0}\), \(y_{0}\), \(\theta_{0}\)) of the current frame are known, Equation (6) is used to convert the data from the current frame into a world coordinate system through the initial pose:  
\[p_{w}=T(x_{0},y_{0},\theta_{0})\cdot p_{c}=T\cdot p_{c}\] \[T=\left[\begin{array}{ccc}\cos(\theta_{0})&-\sin(\theta_{0})&x _{0}\\ \sin(\theta_{0})&\cos(\theta_{0})&y_{0}\\ 0&0&1\end{array}\right], \tag{6}\]  
Figure 2: Selection of the attention area in the reference frame. (**a**) Flow diagram; (**b**) schematic diagram. The red triangle is the robot, the red dots are the lidar data, and the purple circle is the attention area.  
where \(p_{c}\) represents the point cloud in the current frame coordinate system, and \(p_{w}\) represents the point cloud in the world coordinate system.  
The attention area of the reference frame is expanded. If the point in the current frame is located in the expansion layer of the attention area of the reference frame, then the point needs to be matched with the landmark of the reference frame, as shown in Figure 3.  
In Figure 3, the red triangle and the red dots represent the reference frame and its laser data, the green triangle and the green dots represent the current frame and the corresponding laser data, and the purple-red area represents the expansion layer of the reference frame's attention area. The initial pose of the current frame is determined by wheel odometry, but due to the limited accuracy of wheel odometry, the lidar data of the current frame does not coincide with the reference frame. This error is compensated for through the expansion layer. All of the lidar points in the purple-red area in the current frame are used to match the landmarks of the reference frame, and the set of these points is the attention area of the current frame.
# Ball Tracking Based on Multiscale Feature Enhancement and Cooperative Trajectory Matching
## 3 Methods
### Heuristic Compound Sampling Strategy
#### 3.2.2 Heuristic Sampling  
Cropping leads to many useless background patches due to the undersized balls. Patches containing balls are positive samples, whereas patches that do not contain balls are negative samples. The positive and negative samples will be seriously imbalanced. The number of ball samples of different shapes also shows a long-tailed distribution. Imbalanced morphological distribution will eventually reduce the model's generalization ability in real detection scenarios. To tackle this dilemma, we proposed sampling positive and negative samples according to categories while mining complex samples. The Heuristic Compound Sampling Strategy covers different image granularities: patch-level and image-level.  
Considering the balance of positive and negative samples, we adopted a heuristic rule to sample patches containing balls and patches with only background information. For the positive sample, assuming the coordinates of the center point of the true ball labeling box and the size, use Equations (10) to (11) to obtain the horizontal and vertical clipping offsets and Equations (12) and (13) to obtain the corresponding clipping boundaries. A ball patch can be obtained according to Equations (10) to (13). Repeating the above process can obtain a set of multiple positive samples. Negative sample sampling is from any part of the original image that does not include the ball.  
\[off_{x}=max(0,random(w,psize-w)) \tag{10}\] \[off_{y}=max(0,random(h,psize-h))\] (11) \[border_{x}=[max(0,x_{0}-off_{x}),max(0,x_{0}-off_{x})+psize]\] (12) \[border_{y}=[max(0,y_{0}-off_{y}),max(0,y_{0}-off_{y})+psize] \tag{13}\]  
where \(w\) and \(h\) represent the width and height of the original image, _psize_ represents patch size, and \(x_{0}\) and \(y_{0}\) represent the center point of the object box.
# A Novel Dataset for Fabric Defect Detection: Bridging Gaps in Anomaly Detection
## 2 Related Work
### Fabric Defect Detection  
There are multiple ways of grouping different types of fabric defect detection approaches. One way consists of motif-based approaches and non-motif-based approaches. Motif-based methods compare recurring motifs to detect defects, and as such require a defect-free ground truth of the motifs in a fabric. This ground truth is hard to acquire in industry conditions, so these approaches are less used than non-motif-based approaches [25].  
Non-motif-based approaches, which have undergone far more research, can be further subdivided into other categories, generally classified as:  
* Statistical approaches;
* Spectral approaches;
* Model-based approaches;
* Structural-based approaches;
* Learning-based approaches.  
However, due to the recent interest in artificial intelligence (AI) and deep learning (DL), some authors [26] have started categorizing the former four approaches as traditional approaches, and the latter one as a separate approach, further subdivided into:* Classical machine learning methods;
* Deep learning methods.  
We adopted this categorization and cover each of these types in the following subsections.
###### Abstract  
Resistance to chemotherapy and recurrence are major hurdles to treating hormone receptor-negative breast cancer. The crude extract and natural products obtained from medicinal plants are believed to be multitargeted and possess less toxicity as compared to synthetic compounds. The aerial parts and roots of _Moricandia sinica (Boiss.) Boiss_ were used to prepare the crude extracts in solvents of different polarities. Human breast cancer cell lines (MCF7 and MDA-MB-231), liver carcinoma (HepG2), and nontumorigenic cells of human origin (human umbilical vein endothelial cells (HUVEC)) were treated with a serial dilution of crude extracts obtained from the aerial and roots of _Moricandia sinica (Boiss.) Boiss_. The methanol extract of the shoots exhibited a higher level of cytotoxicity against MDA-MB-231 cells than against any other cancer and nontumorigenic cells lines. Six new compounds were identified by gas chromatography-mass spectrophotometry analysis in the shoots extract of _Moricandia sinica (Boiss.) Boiss_, and 2-Tridecen-1-ol was one of the major compounds that represent more than 35% of the extract. M-phase inducer phabases 1 and 2 (CDC 25A and B) were identified as the specific protein target for 2-Tridecen-1-ol by the Swiss protein target prediction tool. In silico molecular docking showed the binding of 2-Tridecen-1-ol with CDC 25 B with a higher binding energy as compared to CDC 25A. The possible molecular mechanism of anticancer activity of _Moricandia sinica (Boiss.) Boiss_ in MDA-MB-231 breast cancer is through inhibition of M-phase inducer phosphatases 1 and 2 via 2-Tridecen-1-ol. Further investigations in breast cancer models are needed to explore the therapeutic potential of _Moricandia sinica (Boiss.) Boiss_ and 2-Tridecen-1-ol as an efficient remedy with a possibly less toxic approach to treat triple-negative breast cancer.  
Moricandia sinica (Boiss.) Boiss; triple negative breast cancer; MDA-MB-231; natural product width=1
## 1 Introduction
### Hardware Selection Considerations  
Concerning the hardware, most existing game-based therapies focus on two fundamental features: to induce as much immersion as possible and to accurately track the movements of the patient. The former improves the user's engagement in a task that is eminently physical (increases the adherence to treatment), whereas the latter is necessary for assessing if the movements have been properly performed (reduces the need for supervision by a therapist).  
To accurately capture the movements of the patient during the rehabilitation session, it is desirable to use sensors with six degrees of freedom (movement and rotation in the three axes). This is why the Kinect(tm) device is one of the most widely-used systems for motion capture in the field of physiological therapy [23, 24].  
Virtual reality devices, by their own nature, are capable of very accurate motion capture, since precise tracking of the user's head and hand movements is required for an immersive simulation, which is precisely the other criterion listed before. Therefore, VR provides reliable and precise data about the hands and head motions, and at the same time, produces an immersive simulation. On top of that, unlike other commercial closed solutions, such as video game consoles, it is easy to develop and distribute custom software for most VR devices.  
Although VR headsets have their drawbacks (for instance, they may be challenging to use for some stroke survivors and cerebral palsy patients [25]), rehabilitation solutions based on immersive environments are more effective than traditional [26] because they:  
* Improve the engagement of the patient: Exergames increase the energy expenditure of the patient and involve both cognitive and physiological rewarding tasks [27], improving the adherence to treatment.
* Provide physical fidelity to real movements: The patient performs motions similar to those needed for analogous daily life situations.
* Provide cognitive fidelity to a real situation: The patient completes activities inside an environment designed to be similar to the real world.  
Therefore, it is expected that an immersive VR solution for telerehabilitation will increase the patient's adherence to treatment and improve the fidelity of the movements performed. However, such a solution requires not only the use of a VR head-mounted display (HMD), but also a high-performance computer that renders the virtual experience and sends the movement and session information to the therapist. Such a computer could be a considerable expense--an economic barrier to access that limits this solution.
# Virtual Fire Evacuation Drills through a Web-Based Serious Game
## 2 Framework  
The framework of this study is shown in Figure 1 and includes three parts.  
1. Technical methods for a web-based serious game.  
A prototype of the drill system is necessary for the construction of a web-based serious game. In this study, an existing stand-alone fire drill system developed by the authors [28] was adopted as a prototype. Aside from the above prototype system, the proposed web-based serious game still needs system design, an optimization solution, and an evaluation model.  
In this study, the system framework of the serious game includes the database, front end, and back ends. The database is used to store drill data, the front end is used for the user interface, and the back end is used for function operation.  
Figure 1: Framework of this study.  
In the optimization solution, the fire scene and web codes of the serious game were optimized. Additionally, a test of the rendering performance of the web-based serious game was performed to validate the effectiveness of the proposed optimization solution.  
In the evaluation of the drills, a solution was designed to obtain and visualize the evacuation paths of numerous trainees, which can be used to reveal the evacuation rules. Additionally, a model based on the integrated hazard dose (IHD) [27] was proposed to evaluate the safety of evacuation drills. In this way, feedback evaluation results can be provided to improve the abilities of the trainees.  
A virtual fire evacuation drill scheme was designed. Drills were performed three times to examine the effectiveness of the proposed serious game with the same fire scene. In this study, more than 100 people participated in virtual fire evacuation drills according to the drill scheme.  
Result analysis and discussions  
The evacuation characteristics were analyzed based on the evacuation spatio-temporal paths of more than 100 trainees. Subsequently, the effectiveness of the proposed serious game was examined according to the evaluation results of the three drills.