Reviewerly/exHarmony huggingface dataset:
Desktop VR Is Better Than Non-ambulatory HMD VR for Spatial Learning

Use of virtual reality (VR) technology is proliferating for designing and upgrading entertainment devices, and creating virtual environments that could be used for research and training. VR is becoming a strong research tool by providing a tighter control on the experimental environment and by allowing almost limitless possibilities of creating ecologically valid stimuli. However, the enhanced fidelity between the real and virtual worlds that VR provides does not always benefit human performance. For a better understanding, and increasing VR’s usability, we need to identify the relevant constituent components of immersive technologies, and differentiate their roles, for example, how visual and interaction fidelity differentially improves human performance. We conducted an experiment to examine how two common VR display modes, head mounted display (HMD) and desktop (DT), would affect spatial learning when we restrict ambulatory locomotion in HMD. This manipulation allowed examining the role of varying visual fidelity with low interaction fidelity. We used a between-group design with 40 naïve participants. They explored a virtual environment and later drew its sketch-map. Our results showed participants spent more time and perceived less motion-sickness and task effort using desktop than HMD VR. With reduced interaction fidelity, the high visual fidelity of HMD as compared to desktop resulted in similar or poorer performance on different spatial learning tasks after accounting for motion-sickness and workload effort. Participants were better in recalling spatial components related to junction and cyclic order of the navigated virtual space in desktop versus HMD VR, and performed equally well on components related to street segments and object associations. We explain these results in terms of deficient idiothetic information in nonambulatory HMD and lesser sensory conflicts in desktop mode. Overall, our results highlight the differential effect of visual versus interaction fidelity on human performance based on using immersive technologies, how such an effect depends on the nature of cognitive and functional behavior users employ, and the higher usability of traditional desktop VR. These results are relevant for developing customized and sustainable virtual reality human-computer interactions.
Development of Virtual Reality Serious Game for Underground Rock-Related Hazards Safety Training

Traditional safety training media to transfer safety knowledge specific to the rock-related hazards in underground mines are mainly video or manuals, which are inefficient and bring a poor training experience.In this paper, we designed and developed a serious game based on virtual reality (VR) technology in order to efficiently transfer safety knowledge and enable enhanced interactive safety training.For different training purposes and users, we designed two modes, one for professional scaling training suitable for novice scalers, the other for rock-related hazards perception training suitable for other miners.Our game is built based on game engine-Unity3D and equipped with HTC VIVE to improve immersion.The game pipeline is to have trainees basically understand safety knowledge through guided interaction and then make a self-adaptive practice to fully master it.We evaluated the effectiveness of our game, and the results of the comparative experiment show that our game is more efficient than the instructional video in both training modes.The application of our game is proven to have the potential to change the safety situation of underground mines and evaluate the level of safety awareness and risk aversion of the miners in the future.
Self Multi-Head Attention for Speaker Recognition

Most state-of-the-art Deep Learning (DL) approaches for speaker recognition work on a short utterance level.Given the speech signal, these algorithms extract a sequence of speaker embeddings from short segments and those are averaged to obtain an utterance level speaker representation.In this work we propose the use of an attention mechanism to obtain a discriminative speaker embedding given non fixed length speech utterances.Our system is based on a Convolutional Neural Network (CNN) that encodes short-term speaker features from the spectrogram and a self multi-head attention model that maps these representations into a long-term speaker embedding.The attention model that we propose produces multiple alignments from different subsegments of the CNN encoded states over the sequence.Hence this mechanism works as a pooling layer which decides the most discriminative features over the sequence to obtain an utterance level representation.We have tested this approach for the verification task for the VoxCeleb1 dataset.The results show that self multi-head attention outperforms both temporal and statistical pooling methods with a 18% of relative EER.Obtained results show a 58% relative improvement in EER compared to i-vector+PLDA.
Optimized Verifiable Fine-Grained Keyword Search in Dynamic Multi-owner Settings

Ciphertext-Policy Attribute-Based Keyword Search (CP-ABKS) schemes support both fine-grained access control and keyword-based ciphertext retrieval, which make these schemes attractive for resource-constrained users (i.e., mobile or wearable devices, sensor nodes, etc.) to store, share and search encrypted data in the public cloud. However, ciphertext length and decryption overhead in the existing CP-ABKS schemes grow with the complexity of access policies or the number of data users' attributes. Moreover, such schemes generally do not consider the practical multi-owner setting (e.g., each file needs to be signed by multiple data owners before being uploaded to the cloud server) or prevent malicious cloud servers from returning incorrect search results. To overcome these limitations, in this paper we first design an optimized Verifiable Fine-grained Keyword Search scheme in the static Multi-owner setting (termed as basic VFKSM), which achieves short ciphertext length, fast ciphertext transformation, accelerated search process, and authentic search result verification. Then, we extend the basic VFKSM to support multi-keyword search and multi-owner update (also called as extended VFKSM). Finally, we prove that the basic (or extended) VFKSM resists the Chosen-Keyword Attack (CKA) and external Keyword-Guessing Attack (KGA). We also evaluate the performance of these schemes using various public datasets.
Geochemistry of surface sediments from the northwestern Gulf of Mexico: implications for provenance and heavy metal contamination

Thirty-five near-surface sediment samples were recovered from the continental shelf and upper slope regions of the northwestern (NW) Gulf of Mexico. The geochemical data of the sediments recovered were examined to investigate the weathering intensity, provenance, palaeo-oxygenation condition, and level of heavy metal contamination. The sediments analysed showed a moderate to high intensity of chemical weathering. Major and trace element concentrations indicated a terrigenous origin, closely related to the weathering of rocks rich in aluminosilicates. The results of this study further revealed that major rivers, the Bravo and Soto La Marina, played an important role in delivering sediments to the study area. The concentration of transition trace elements such as Cr, Cu, Ni, and V revealed that the sediments were derived from intermediate rocks such as andesite. The V/Cr, Ni/Co, and Cu/Zn ratios in the sediments were &lt;2, &lt;5, and &lt;1, respectively, suggesting a depositional process occurred under well-oxygenated conditions. Principle Component Analysis (PCA) did not show a significant difference in sediment texture between the continental shelf and slope areas. The enrichment factor (EF) and Geo-accumulation index (Igeo) values were &lt;2 and &lt;1, respectively, suggesting the absence of an anthropogenic input.
Dual-alignment Feature Embedding for Cross-modality Person Re-identification

Person re-identification aims at searching pedestrians across different cameras, which is a key problem in video surveillance. With requirements in night environment, RGB-infrared person re-identification which could be regarded as a cross-modality matching problem, has gained increasing attention in recent years. Aside from cross-modality discrepancy, RGB-infrared person re-identification also suffers from human pose and view point differences. We design a dual-alignment feature embedding method to extract discriminative modality-invariant features. The concept of dual-alignment is two folds: spatial and modality alignments. We adopt the part-level features to extract fine-grained camera-invariant information. We introduce distribution loss function and correlation loss function to align the embedding features across visible and infrared modalities. Finally, we can extract modality-invariant features with robust and rich identity embeddings for cross-modality person re-identification. Experiment confirms that the proposed baseline and improvement achieves competitive results with the state-of-the-art methods on two datasets. For instance, We achieve (57.5+12.6)% rank-1 accuracy and (57.3+11.8)% mAP on the RegDB dataset.
Alleviating Feature Confusion for Generative Zero-shot Learning

Lately, generative adversarial networks (GANs) have been successfully applied to zero-shot learning (ZSL) and achieved state-of-the-art performance. By synthesizing virtual unseen visual features, GAN-based methods convert the challenging ZSL task into a supervised learning problem. However, since real unseen visual features are not available at the training stage, GAN-based ZSL methods have to train the GAN generator on the seen categories and further apply it to unseen instances. An inevitable issue of such a paradigm is that the synthesized unseen features are prone to seen references and incapable to reflect the novelty and diversity of real unseen instances. In a nutshell, the synthesized features are confusing. One cannot tell unseen categories from seen ones using the synthesized features. As a result, the synthesized features are too subtle to be classified in generalized zero-shot learning (GZSL) which involves both seen and unseen categories at the test stage. In this paper, we first introduce the feature confusion issue. Then, we propose a new feature generating network, named alleviating feature confusion GAN (AFC-GAN), to challenge the issue. Specifically, we present a boundary loss which maximizes the decision boundary of seen categories and unseen ones. Furthermore, a novel metric named feature confusion score (FCS) is proposed to quantify the feature confusion. Extensive experiments on five widely used datasets verify that our method is able to outperform previous state-of-the-arts under both ZSL and GZSL protocols.
A Comprehensive Study of Deep Learning for Side-Channel Analysis

Recently, several studies have been published on the application of deep learning to enhance Side-Channel Attacks (SCA). These seminal works have practically validated the soundness of the approach, especially against implementations protected by masking or by jittering. Concurrently, important open issues have emerged. Among them, the relevance of machine (and thereby deep) learning based SCA has been questioned in several papers based on the lack of relation between the accuracy, a typical performance metric used in machine learning, and common SCA metrics like the Guessing entropy or the key-discrimination success rate. Also, the impact of the classical side-channel counter-measures on the efficiency of deep learning has been questioned, in particular by the semi-conductor industry. Both questions enlighten the importance of studying the theoretical soundness of deep learning in the context of side-channel and of developing means to quantify its efficiency, especially with respect to the optimality bounds published so far in the literature for side-channel leakage exploitation. The first main contribution of this paper directly concerns the latter point. It is indeed proved that minimizing the Negative Log Likelihood (NLL for short) loss function during the training of deep neural networks is actually asymptotically equivalent to maximizing the Perceived Information introduced by Renauld et al. at EUROCRYPT 2011 as a lower bound of the Mutual Information between the leakage and the target secret. Hence, such a training can be considered as an efficient and effective estimation of the PI, and thereby of the MI (known to be complex to accurately estimate in the context of secure implementations). As a second direct consequence of our main contribution, it is argued that, in a side-channel exploitation context, choosing the NLL loss function to drive the training is sound from an information theory point of view. As a third contribution, classical counter-measures like Boolean masking or execution flow shuffling, initially dedicated to classical SCA, are proved to stay sound against deep Learning based attacks.
An Adaptive Image Steganography Method Based on Histogram of Oriented Gradient and PVD-LSB Techniques

Pixel value differencing (PVD) and least significant bit substitution (LSB) are two widely used schemes in image steganography. These two methods do not consider different content in a cover image for hiding the secret data. The content of most digital images has different edge directions in each pixel, and the local object shape or appearance is mostly characterized by the distribution of its intensity gradients or edge directions. Exploiting these characteristics for embedding various secret information in different edge directions will eliminate sequential embedding and improve robustness. Thus, a histogram of oriented gradient (HOG) algorithm is proposed to find the dominant edge direction for each 2x2 block of cover images. Blocks of interest (BOIs) are determined adaptively based on the gradient magnitude and angle of the cover image. Then, the PVD algorithm is used to hide secret data in the dominant edge direction, while the LSB substitution is utilized in the other two remaining pixels. Extensive experiments using various standard images reveal that the proposed scheme provides high embedding capacity and better visual quality compared with several other PVD- and LSB-based methods. Moreover, it resists various steganalysis techniques, such as pixel difference histogram and RS analysis.
Target-Aspect-Sentiment Joint Detection for Aspect-Based Sentiment Analysis

Aspect-based sentiment analysis (ABSA) aims to detect the targets (which are composed by continuous words), aspects and sentiment polarities in text. Published datasets from SemEval-2015 and SemEval-2016 reveal that a sentiment polarity depends on both the target and the aspect. However, most of the existing methods consider predicting sentiment polarities from either targets or aspects but not from both, thus they easily make wrong predictions on sentiment polarities. In particular, where the target is implicit, i.e., it does not appear in the given text, the methods predicting sentiment polarities from targets do not work. To tackle these limitations in ABSA, this paper proposes a novel method for target-aspect-sentiment joint detection. It relies on a pre-trained language model and can capture the dependence on both targets and aspects for sentiment prediction. Experimental results on the SemEval-2015 and SemEval-2016 restaurant datasets show that the proposed method achieves a high performance in detecting target-aspect-sentiment triples even for the implicit target cases; moreover, it even outperforms the state-of-the-art methods for those subtasks of target-aspect-sentiment detection that they are competent to.
PDDLStream: Integrating Symbolic Planners and Blackbox Samplers via Optimistic Adaptive Planning

Many planning applications involve complex relationships defined on high-dimensional, continuous variables. For example, robotic manipulation requires planning with kinematic, collision, visibility, and motion constraints involving robot configurations, object poses, and robot trajectories. These constraints typically require specialized procedures to sample satisfying values. We extend PDDL to support a generic, declarative specification for these procedures that treats their implementation as black boxes. We provide domain-independent algorithms that reduce PDDLStream problems to a sequence of finite PDDL problems. We also introduce an algorithm that dynamically balances exploring new candidate plans and exploiting existing ones. This enables the algorithm to greedily search the space of parameter bindings to more quickly solve tightly-constrained problems as well as locally optimize to produce low-cost solutions. We evaluate our algorithms on three simulated robotic planning domains as well as several real-world robotic tasks.
Face Detection and Segmentation Based on Improved Mask R-CNN

Deep convolutional neural networks have been successfully applied to face detection recently. Despite making remarkable progress, most of the existing detection methods only localize each face using a bounding box, which cannot segment each face from the background image simultaneously. To overcome this drawback, we present a face detection and segmentation method based on improved Mask R-CNN, named G-Mask, which incorporates face detection and segmentation into one framework aiming to obtain more fine-grained information of face. Specifically, in this proposed method, ResNet-101 is utilized to extract features, RPN is used to generate RoIs, and RoIAlign faithfully preserves the exact spatial locations to generate binary mask through Fully Convolution Network (FCN). Furthermore, Generalized Intersection over Union (GIoU) is used as the bounding box loss function to improve the detection accuracy. Compared with Faster R-CNN, Mask R-CNN, and Multitask Cascade CNN, the proposed G-Mask method has achieved promising results on FDDB, AFW, and WIDER FACE benchmarks.
Enabling Privacy-Assured Fog-Based Data Aggregation in E-Healthcare Systems

Wearable body area network is a key component of the modern-day e-healthcare system (e.g., telemedicine), particularly as the number and types of wearable medical monitoring systems increase. The importance of such systems is reinforced in the current COVID-19 pandemic. In addition to the need for a secure collection of medical data, there is also a need to process data in real-time. In this article, we design an improved symmetric homomorphic cryptosystem and a fog-based communication architecture to support delay- or time-sensitive monitoring and other-related applications. Specifically, medical data can be analyzed at the fog servers in a secure manner. This will facilitate decision making, for example, allowing relevant stakeholders to detect and respond to emergency situations, based on real-time data analysis. We present two attack games to demonstrate that our approach is secure (i.e., chosen-plaintext attack resilience under the computational Diffie-Hellman assumption), and evaluate the complexity of its computations. A comparative summary of its performance and three other related approaches suggests that our approach enables privacy-assured medical data aggregation, and the simulation experiments using Microsoft Azure further demonstrate the utility of our scheme.
Deep-Learning–Based Characterization of Tumor-Infiltrating Lymphocytes in Breast Cancers From Histopathology Images and Multiomics Data

PURPOSE Tumor-infiltrating lymphocytes (TILs) and their spatial characterizations on whole-slide images (WSIs) of histopathology sections have become crucial in diagnosis, prognosis, and treatment response prediction for different cancers. However, fully automatic assessment of TILs on WSIs currently remains a great challenge because of the heterogeneity and large size of WSIs. We present an automatic pipeline based on a cascade-training U-net to generate high-resolution TIL maps on WSIs. METHODS We present global cell-level TIL maps and 43 quantitative TIL spatial image features for 1,000 WSIs of The Cancer Genome Atlas patients with breast cancer. For more specific analysis, all the patients were divided into three subtypes, namely, estrogen receptor (ER)–positive, ER-negative, and triple-negative groups. The associations between TIL scores and gene expression and somatic mutation were examined separately in three breast cancer subtypes. Both univariate and multivariate survival analyses were performed on 43 TIL image features to examine the prognostic value of TIL spatial patterns in different breast cancer subtypes. RESULTS The TIL score was in strong association with immune response pathway and genes (eg, programmed death-1 and CLTA4). Different breast cancer subtypes showed TIL score in association with mutations from different genes suggesting that different genetic alterations may lead to similar phenotypes. Spatial TIL features that represent density and distribution of TIL clusters were important indicators of the patient outcomes. CONCLUSION Our pipeline can facilitate computational pathology-based discovery in cancer immunology and research on immunotherapy. Our analysis results are available for the research community to generate new hypotheses and insights on breast cancer immunology and development.
Deep learning for population size history inference: Design, comparison and combination with approximate Bayesian computation

Abstract For the past decades, simulation‐based likelihood‐free inference methods have enabled researchers to address numerous population genetics problems. As the richness and amount of simulated and real genetic data keep increasing, the field has a strong opportunity to tackle tasks that current methods hardly solve. However, high data dimensionality forces most methods to summarize large genomic data sets into a relatively small number of handcrafted features (summary statistics). Here, we propose an alternative to summary statistics, based on the automatic extraction of relevant information using deep learning techniques. Specifically, we design artificial neural networks (ANNs) that take as input single nucleotide polymorphic sites (SNPs) found in individuals sampled from a single population and infer the past effective population size history. First, we provide guidelines to construct artificial neural networks that comply with the intrinsic properties of SNP data such as invariance to permutation of haplotypes, long scale interactions between SNPs and variable genomic length. Thanks to a Bayesian hyperparameter optimization procedure, we evaluate the performance of multiple networks and compare them to well‐established methods like Approximate Bayesian Computation (ABC). Even without the expert knowledge of summary statistics, our approach compares fairly well to an ABC approach based on handcrafted features. Furthermore, we show that combining deep learning and ABC can improve performance while taking advantage of both frameworks. Finally, we apply our approach to reconstruct the effective population size history of cattle breed populations.