Reviewerly/exHarmony huggingface dataset:
Modeling the User Experience of Watching 360° Videos with Head-Mounted Displays

Conducting user studies to quantify the Quality of Experience (QoE) of watching the increasingly more popular 360° videos in Head-Mounted Displays (HMDs) is time-consuming, tedious, and expensive. Deriving QoE models, however, is very challenging because of the diverse viewing behaviors and complex QoE features and factors. In this article, we compile a wide spectrum of QoE features and factors that may contribute to the overall QoE. We design and conduct a user study to build a dataset of the overall QoE, QoE features, and QoE factors. Using the dataset, we derive the QoE models for both the Mean Opinion Score (MOS) and Individual Score (IS), where MOS captures the aggregated QoE across all subjects, while IS captures the QoE of individual subjects. Our derived overall QoE models achieve 0.98 and 0.91 in Pearson’s Linear Correlation Coefficient (PLCC) for MOS and IS, respectively. Besides, we make several new observations on our user study results, such as (1) content factors dominate the overall QoE across all factor categories, (2) Video Multi-Method Assessment Fusion (VMAF) is the dominating factor among content factors, and (3) the perceived cybersickness is affected by human factors more among others. Our proposed user study design is useful for QoE modeling (specifically) and subjective evaluations (in general) of emerging 360° tiled video streaming to HMDs.
Point-Set Kernel Clustering

Measuring similarity between two objects is the core operation in existing clustering algorithms in grouping similar objects into clusters. This paper introduces a new similarity measure called point-set kernel which computes the similarity between an object and a set of objects. The proposed clustering procedure utilizes this new measure to characterize every cluster grown from a seed object. We show that the new clustering procedure is both effective and efficient that enables it to deal with large scale datasets. In contrast, existing clustering algorithms are either efficient or effective. In comparison with the state-of-the-art density-peak clustering and scalable kernel k-means clustering, we show that the proposed algorithm is more effective and runs orders of magnitude faster when applying to datasets of millions of data points, on a commonly used computing machine.
Feasibility Study on Using AI and VR for Decision-Making Training of Basketball Players

Decision-making plays an essential role in basketball offense. Offensive players must make effective decisions to score a basket in various defensive situations. Virtual reality (VR) has been widely used in the training of athletes to strengthen their ability to make optimal decisions by creating controllable repeatable training scenarios. In this article, an action-aware offensive decision-making training system for basketball using VR and artificial intelligence is proposed. The proposed system is composed of different virtual defensive scenarios and an offensive action recognition framework. Trainees wearing head-mounted display and a motion capture suit are trained by intuitively interacting with the VR system and receive decision suggestions when a bad one is made. This study changes the training media and methods to create an immersive training environment during the training phase and evaluates the training effectiveness. These training scenarios are a conventional tactics board, the proposed VR system with a prerecorded <inline-formula><tex-math notation="LaTeX">$360^\circ$</tex-math></inline-formula> panorama video, and the VR system with computer-simulated virtual content. Results indicate that the training scenario affects the training in terms of decision time.
Dense sampling and detail enhancement network: Improved small object detection based on dense sampling and detail enhancement

Small objects only occupy a few pixels in an image, which results in low performance of small object detection for existing object detection algorithms. Therefore, the authors propose a dense sampling and detail enhancement network (DSDE-Net) to address this issue. The network contains a dense sampling module used to increase the resolution of feature maps and expand the receptive field, which includes an atrous spatial pyramid pooling network and a coordinate attention mechanism to systematically process feature maps. Simultaneously, the authors introduce a detail enhancement branch that contains edge and detailed information to generate detailed enhancement feature maps through Gaussian filtering to compensate for the loss of small object information that occurs in the feature extraction process. The experimental results demonstrate that the proposed network outperformed related methods. Compared with the state-of-the-art algorithm DetectoRS, it effectively achieves approximately 4.6% improvement on the minicoco2021 dataset and 4.2% improvement on the remotely sensed dataset VisDrone.
Use of electric network frequency presence in video material for time estimation

In this research, the possibility of estimating the time a video was recorded at through electric network frequency is explored by examining various light sources in differentiating circumstances. This research focuses on videos made with smartphones. The smartphone cameras make use of an integrated complementary metal oxide semiconductor sensor. The filmed videos are analyzed using software, which employs a small electric network frequency (ENF) database to determine the time of recording of a video made in experimental circumstances. This research shows that in ideal circumstances, it is possible to determine the time stamp of a video recording made with a smartphone. However, it becomes clear that different light sources greatly influence the outcome. The best results are achieved with Halogen and Incandescent light sources, both of which also seem promising in less ideal circumstances. LED sources do work in ideal circumstances and, however, do not show much success in lesser circumstances. This research further demonstrates that there is potential in using ENF to determine a time stamp of recorded videos and provides validation on prior research on this topic. It proves usable in ideal circumstances with the presence of a clear light source on a white wall. With additional research, it has potential to become a feasible method to use for forensic settings in circumstances that are less ideal.
Innovative Interaction Approach in IMM Filtering for Vehicle Motion Models With Unequal States Dimension

Robust and adaptive vehicle state estimation and tracking algorithms have become a very important part within the autonomous driving field. The family of interacting multiple model (IMM) filters has shown to provide very effective and accurate state estimation in systems whose behavior patterns change significantly over time. The main reason for the improved performance of IMM filters compared to single model approaches is the mode mixing, which constantly aligns the different models. This paper proposes an innovative way for the mode mixing, when the state-vectors of the models are of different size. The proposed mixing approach consists of two parts: firstly mixing the common states and secondly weighting between original and mixed states based on the model probabilities. Results from artificial simulations and real world measurement setups are shown to demonstrate the validity of the approach. Compared to previously suggested solutions, the proposed approach is more general and the overall complexity of the mode mixing step is reduced, which is the main contribution of the presented paper.
Information-Theoretic Online Multi-Camera Extrinsic Calibration

Calibration of multi-camera systems is essential for lifelong use of vision-based headsets and autonomous robots. In this work, we present an information-based framework for online extrinsic calibration of multi-camera systems. While previous work largely focuses on monocular, stereo, or strictly non-overlapping field-of-view (FoV) setups, we allow arbitrary configurations while also exploiting overlapping pairwise FoV when possible. In order to efficiently solve for the extrinsic calibration parameters, which increase linearly with the number of cameras, we propose a novel entropy-based keyframe measure and bound the backend optimization complexity by selecting informative motion segments that minimize the maximum entropy across all extrinsic parameter partitions. We validate the pipeline on three distinct platforms to demonstrate the generality of the method for resolving the extrinsics and performing downstream tasks. Our code is available at <uri>https://github.com/edexheim/info_ext_calib</uri>.
GANBA: Generative Adversarial Network for Biometric Anti-Spoofing

Automatic speaker verification (ASV) is a voice biometric technology whose security might be compromised by spoofing attacks. To increase the robustness against spoofing attacks, presentation attack detection (PAD) or anti-spoofing systems for detecting replay, text-to-speech and voice conversion-based spoofing attacks are being developed. However, it was recently shown that adversarial spoofing attacks may seriously fool anti-spoofing systems. Moreover, the robustness of the whole biometric system (ASV + PAD) against this new type of attack is completely unexplored. In this work, a new generative adversarial network for biometric anti-spoofing (GANBA) is proposed. GANBA has a twofold basis: (1) it jointly employs the anti-spoofing and ASV losses to yield very damaging adversarial spoofing attacks, and (2) it trains the PAD as a discriminator in order to make them more robust against these types of adversarial attacks. The proposed system is able to generate adversarial spoofing attacks which can fool the complete voice biometric system. Then, the resulting PAD discriminators of the proposed GANBA can be used as a defense technique for detecting both original and adversarial spoofing attacks. The physical access (PA) and logical access (LA) scenarios of the ASVspoof 2019 database were employed to carry out the experiments. The experimental results show that the GANBA attacks are quite effective, outperforming other adversarial techniques when applied in white-box and black-box attack setups. In addition, the resulting PAD discriminators are more robust against both original and adversarial spoofing attacks.
High Dynamic Range Imaging for Dynamic Scenes With Large-Scale Motions and Severe Saturation

Two key challenges exist in high dynamic range (HDR) imaging from multiexposure low dynamic range (LDR) images for dynamic scenes: 1) aligning the input images with large-scale foreground motions and 2) recovering large saturated regions from a limited number of input LDR images. To tackle these challenges, several deep convolutional neural networks have been proposed that have made significant progress. However, these methods tend to suffer from ghosting and saturation artifacts when applied to some challenging scenes. In this article, we propose an end-to-end deformable HDR imaging network, called DHDRNet, which attempts to alleviate these problems by building an effective aligning module and adopting self-guided attention. First, we analyze the alignment process in the HDR imaging task and correspondingly design a pyramidal deformable module (PDM) that aligns LDR images at multiple scales and reconstructs aligned features in a coarse-to-fine manner. In this way, the proposed DHDRNet can handle large-scale complex motions and suppress ghosting artifacts caused by misalignments. Moreover, we adopt self-guided attention to reduce the influence of saturated regions during the aligning and merging processes, which helps suppress artifacts and retain fine details in the final HDR image. Extensive qualitative and quantitative comparisons demonstrate that the proposed model outperforms the existing start-of-the-art methods and that it is robust to challenging scenes with large-scale motions and severe saturation. The source code is available at: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/Tx000/DHDRNet</uri> .
Defending against SQL Injection Attacks in Web Applications using Machine Learning and Natural Language Processing

Today, most organizations use web applications for the delivery of services over the Internet. The risks to web applications have increased as their use has risen. SQL Injection Attack is a commonly exploited vulnerability used for stealing credentials, destroying and compromising data, and bypassing authentication and authorization controls of a web application. Traditional methods of detecting SQL injection attacks include software and hardware-based Web Application Firewalls, programmatic defense techniques like input filtering, input validation, using parameterized queries etc. and static and dynamic analysis are not sufficient for detection and prevention of SQLIA in web applications. In this paper, we present an approach to detecting SQLIA using NLP and Machine Learning. Experimental results show that the approach can detect SQLIA with precision, recall and an f1-score of 99.9.
KerasBERT: Modeling the Keras Language

We introduce a new application domain to evaluate the knowledge retention of language models. Our model, Keras-BERT, is trained on the Keras code documentation. This is a unique challenge of language modeling small datasets, as well as combining natural language and code data. We evaluate how well KerasBERT learns the Keras Deep Learning framework through cloze test evaluation. We present miscellaneous properties of these cloze tests such as mask positioning and prompt paraphrasing. KerasBERT is an 80 million parameter RoBERTa model, which we compare to the Zero-Shot learning capability of the 6 billion parameter GPT-Neo model. We present a suite of cloze tests crafted from the Keras documentation to evaluate these models. We find some exciting completions that show KerasBERT is a promising direction for question answering and schema-free database querying. We conclude our work by discussing some future directions for KerasBERT and the development of language models for code documentation support.
Can Neural Clone Detection Generalize to Unseen Functionalitiesƒ

Many recently proposed code clone detectors exploit neural networks to capture latent semantics of source code, thus achieving impressive results for detecting semantic clones. These neural clone detectors rely on the availability of large amounts of labeled training data. We identify a key oversight in the current evaluation methodology for neural clone detection: cross-functionality generalization (i.e., detecting semantic clones of which the functionalities are unseen in training). Specifically, we focus on this question: do neural clone detectors truly learn the ability to detect semantic clones, or they just learn how to model specific functionalities in training data while cannot generalize to realistic unseen functionalitiesƒ This paper investigates how the generalizability can be evaluated and improved.Our contributions are 3-folds: (1) We propose an evaluation methodology that can systematically measure the cross-functionality generalizability of neural clone detection. Based on this evaluation methodology, an empirical study is conducted and the results indicate that current neural clone detectors cannot generalize well as expected. (2) We conduct empirical analysis to understand key factors that can impact the generalizability. We investigate 3 factors: training data diversity, vocabulary, and locality. Results show that the performance loss on unseen functionalities can be reduced through addressing the out-of-vocabulary problem and increasing training data diversity. (3) We propose a human-in-the-loop mechanism that help adapt neural clone detectors to new code repositories containing lots of unseen functionalities. It improves annotation efficiency with the combination of transfer learning and active learning. Experimental results show that it reduces the amount of annotations by about 88%. Our code and data are publicly available <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> .
Emotion Detection using Deep Learning

Emotion detection is one of the challenging tasks in Natural Language Processing. The applications of emotion detection can be utilized in different fields like data mining, psychology, and human-computer interaction. The paper aims to train different learning- based models in a supervised framework. In this paper, we have investigated different learning-based methods used in emotion detection. The methods are artificial neural networks and deep learning methods to investigate the best solution for emotion detection problem. The algorithms tested are neural networks approach as Perceptron, Multilayer Perceptron, and deep learning approach as CNN-LSTM, CNN-BiLSTM, CNN-GRU, CNN-BiGRU, BiLSTM and CNN using different features representation methods like TF-IDF, N-Grams, word-based embeddings, and contextualized embeddings. We have tested the algorithms on ISEAR dataset. The results show that the features representation method, applying preprocessing on dataset or not, and the number of emotion classes classified have impact on the models' performance. The results also show that BERT and DNN dense layer as a classifier outperform all other approaches by achieving 0.71 macro-average f1-measure for 7 emotions, 0.76 macro-average f1-measure for 5 emotions and 0.8 macro- average f1-measure for 4 emotions.
Translation as a source of pragmatic interference?

In this paper, we pursue two main (interconnected) goals, relying on a comparable and parallel corpus of French and Italian cleft sentences. First, we assess the most typical formal and functional properties of clefts in these languages, while also considering variation due to the different types of data. Second, we verify the presence of interference in the translation of clefts from French into Italian. Translators’ choices mostly align with information-structural patterns found in original Italian texts, but word-for-word translation styles can increase the frequency of less-common cleft types in Italian, such as Broad Focus and Information Focus clefts. However, odd or infelicitous pragmatic effects arise only in very few cases.
Individual Animal and Herd Identification Using Custom YOLO v3 and v4 with Images Taken from a UAV Camera at Different Altitudes

In this study, an unmanned aerial vehicle (UAV) captures images of wild animals at different altitudes in order to compare the individual and herd identification capabilities of custom YOLO v3 and v4 models. Previous studies showed that UAVs can disturb animals in the wild at certain altitude, such that it is necessary to maintain altitude that does not disturb them. However, as the UAV altitude increases, the captured images lose features critical to YOLO in classification. We investigate and compare the accuracy of custom YOLO v3 and v4, especially from the acceptable minimum altitude and higher. We studied eight classes of wild African animals, namely, individual and herd of giraffes (Giraffa camelopardalis), individual and herd of white rhinos (Ceratotherium simum), individual and herd of wildebeests (Connochaetes taurinus), and individual and herd of zebras (Equus quagga). As UAV altitude increased, some image features are lost resulting to a model detection accuracy as low as 68.86%. The customised YOLO v4 model has 51.70 FPS outperforming customised YOLO v3 by an increased model speed of 13.7%.
Multi-frame Motion Segmentation by Combining Two-Frame Results

Abstract In this paper we consider the motion segmentation problem on sparse and unstructured datasets involving rigid motions, motivated by multibody structure from motion. In particular, we assume only two-frame correspondences as input without prior knowledge about trajectories. Inspired by the success of synchronization methods, we address this problem by introducing a two-stage approach: first, motion segmentation is addressed on image pairs independently; then, two-frame results are combined in a robust way to compute the final multi-frame segmentation. Our synthetic and real experiments demonstrate that the proposed approach is very effective in reducing the errors among two-frame results and it can cope with a large amount of mismatches. Moreover, our method can be profitably used to build a multibody structure from motion pipeline.
Dataset on enrichment of selected trace metals in the soil from designated abandoned historical gold mine solid waste dump sites near residential areas, Witwatersrand Basin, South Africa

Soil is a naturally occurring non-renewable resource, and good soil quality is a prerequisite for the survival of plants, animals and humans. Soil quality depends on the content and distribution of trace elements. Gold mining in the Witwatersrand Basin in South Africa left behind enormous unrehabilitated mining waste tailings near the City of Johannesburg that are contaminated by trace elements. Heavy metals change the physical and chemical properties of the soil derived from the mining waste material, consequently disturbing the normal functions of the soil and posing a potential health risk to plants, animals, and humans. To assess soil quality from abandoned historical gold mine residues, three conglomerate samples were petrologically examined using hand specimen, polished thin sections viewed under a reflected light microscope, and X-ray diffraction, to determine the mineralogical composition. The minerals include quartz, mica, chlorite, calcite and sulphides. Geochemical data of major elements measured by weight percent (wt%) were generated using X-ray fluorescence (XRF) technique and are NiO < Cr2O3 < V2O5 < ZrO2 < MnO < P2O5 < TiO2 < Al2O3 < CaO < MgO < Na2O < K2O < Fe2O3 < SiO2. The geochemistry of trace elements, pH, and electrical conductivity (EC), were determined from 21 soil samples. The samples were collected from 30 to 150 cm depths from nine sites and investigated using inductively coupled plasma optical emission spectroscopy (ICP-EOS) and XRF. ICP-EOS data (mg/kg): Fe > S > Cu > Mn > Cr > Zn > Ni > Co > Mo > P, and XRF data (mg/kg): S > F > Cl > V > Cr > Zn > Co > Cu > Ni > Mo. The trace elements data are variable in the tailing soil than in the crustal background soil. According to ICP-EOS data, the surveyed sites show increased Cu content, pH values of 1.9-5.3 and EC values of 43-679 mS/m, indicating soil contamination.
Data analytics for cybersecurity enhancement of transformer protection

Electric power substations are experiencing an accelerated pace of digital transformation including the deployment of LAN-based IEC 61850 communication protocols that facilitate accessibility to substation data while also increasing remote access points and exposure to complex cyberattacks. In this environment, machine learning algorithms will play a vital role in cyberattack detection and mitigation and natural questions arise as to the most effective models in the context of smart grid substations. This paper compares the performance of three autoencoder-based anomaly detection systems including linear, fully connected, and convolutional autoencoders, as well as long short-term memory (LSTM) neural network for cybersecurity enhancement of transformer protection. The simulation results indicated that the LSTM model outperforms the other models for detecting cyberattacks targeting asymmetrical fault data. The linear autoencoder, fully connected autoencoder and 1D CNN further outperform the LSTM model for detecting cyberattacks targeting the symmetrical fault data.