Reviewerly/exHarmony huggingface dataset:
Face Reconstruction from Voice using Generative Adversarial Networks

Voice profiling aims at inferring various human parameters from their speech, e.g. gender, age, etc. In this paper, we address the challenge posed by a subtask of voice profiling - reconstructing someone's face from their voice. The task is designed to answer the question: given an audio clip spoken by an unseen person, can we picture a face that has as many common elements, or associations as possible with the speaker, in terms of identity? To address this problem, we propose a simple but effective computational framework based on generative adversarial networks (GANs). The network learns to generate faces from voices by matching the identities of generated faces to those of the speakers, on a training set. We evaluate the performance of the network by leveraging a closely related task - cross-modal matching. The results show that our model is able to generate faces that match several biometric characteristics of the speaker, and results in matching accuracies that are much better than chance. The code is publicly available in https://github.com/cmu-mlsp/reconstructing_faces_from_voices
ReMASC: Realistic Replay Attack Corpus for Voice Controlled Systems

This paper introduces a new database of voice recordings with the goal of supporting research on vulnerabilities and protection of voice-controlled systems (VCSs). In contrast to prior efforts, the proposed database contains both genuine voice commands and replayed recordings of such commands, collected in realistic VCSs usage scenarios and using modern voice assistant development kits. Specifically, the database contains recordings from four systems (each with a different microphone array) in a variety of environmental conditions with different forms of background noise and relative positions between speaker and device. To the best of our knowledge, this is the first publicly available database that has been specifically designed for the protection of state-of-the-art voice-controlled systems against various replay attacks in various conditions and environments.
Feature Exploration for Almost Zero-Resource ASR-Free Keyword Spotting Using a Multilingual Bottleneck Extractor and Correspondence Autoencoders

We compare features for dynamic time warping (DTW) when used to bootstrap keyword spotting (KWS) in an almost zero-resource setting. Such quickly-deployable systems aim to support United Nations (UN) humanitarian relief efforts in parts of Africa with severely under-resourced languages. Our objective is to identify acoustic features that provide acceptable KWS performance in such environments. As supervised resource, we restrict ourselves to a small, easily acquired and independently compiled set of isolated keywords. For feature extraction, a multilingual bottleneck feature (BNF) extractor, trained on well-resourced out-of-domain languages, is integrated with a correspondence autoencoder (CAE) trained on extremely sparse in-domain data. On their own, BNFs and CAE features are shown to achieve a more than 2% absolute performance improvement over baseline MFCCs. However, by using BNFs as input to the CAE, even better performance is achieved, with a more than 11% absolute improvement in ROC AUC over MFCCs and more than twice as many top-10 retrievals for two evaluated languages, English and Luganda. We conclude that integrating BNFs with the CAE allows both large out-of-domain and sparse in-domain resources to be exploited for improved ASR-free keyword spotting.
Combining Black-Box and White-Box Techniques for Learning Register Automata

Model learning is a black-box technique for constructing state machine models of software and hardware components, which has been successfully used in areas such as telecommunication, banking cards, network protocols, and control software. The underlying theoretic framework (active automata learning) was first introduced in a landmark paper by Dana Angluin in 1987 for finite state machines. In order to make model learning more widely applicable, it must be further developed to scale better to large models and to generate richer classes of models. Recently, various techniques have been employed to extend automata learning to extended automata models, which combine control flow with guards and assignments to data variables. Such techniques infer guards over data parameters and assignments from observations of test output. In the black-box model of active automata learning this can be costly and require many tests, while in many application scenarios source code is available for analysis. In this paper, we explore some directions for future research on how black-box model learning can be enhanced using white-box information extraction methods, with the aim to maintain the benefits of dynamic black-box methods while making effective use of information that can be obtained through white-box techniques.
PHAROS 2.0—A PHysical Assistant RObot System Improved

There are great physical and cognitive benefits for older adults who are engaged in active aging, a process that should involve daily exercise. In our previous work on the PHysical Assistant RObot System (PHAROS), we developed a system that proposed and monitored physical activities. The system used a social robot to analyse, by means of computer vision, the exercise a person was doing. Then, a recommender system analysed the exercise performed and indicated what exercise to perform next. However, the system needed certain improvements. On the one hand, the vision system captured the movement of the person and indicated whether the exercise had been done correctly or not. On the other hand, the recommender system was based purely on a ranking system that did not take into account temporal evolution and preferences. In this work, we propose an evolution of PHAROS, PHAROS 2.0, incorporating improvements in both of the previously mentioned aspects. In the motion capture aspect, we are now able to indicate the degree of completeness of each exercise, identifying the part that has not been done correctly, and a real-time performance correction. In this way, the recommender system receives a greater amount of information and so can more accurately indicate the exercise to be performed. In terms of the recommender system, an algorithm was developed to weigh the performance, temporal evolution and preferences, providing a more accurate recommendation, as well as expanding the recommendation to a batch of exercises, instead of just one.
Awareness of Research Data Management Services at Academic Libraries in Jordan: Roles, Responsibilities and Challenges

Research data management services require new skills and collaboration among library staff to work with both researchers and end-users as they manage data going forward. However, there is an urgent need to increase awareness of the new trends about the partnership between research community and academic libraries, there is also a need for new research data management (RDM)-related skills among librarians. And therefore, this research aims at investigating requirements, roles and responsibilities as well as challenges of RDM services in academic libraries in Jordan. A questionnaire was developed and used to collect the required data from 21 academic libraries in Jordan. The results indicate high perception and awareness of libraries roles and responsibilities of RDM as well as requirements and challenges in academic libraries in Jordan to provide RDM services. It was noted that all factors such as job title, experience and library type had no effect on the results.
A novel classification-selection approach for the self updating of template-based face recognition systems

The boosting on the need of security notably increased the amount of possible facial recognition applications, especially due to the success of the Internet of Things (IoT) paradigm. However, although handcrafted and deep learning-inspired facial features reached a significant level of compactness and expressive power, the facial recognition performance still suffers from intra-class variations such as ageing, facial expressions, lighting changes, and pose. These variations cannot be captured in a single acquisition and require multiple acquisitions of long duration, which are expensive and need a high level of collaboration from the users. Among others, self-update algorithms have been proposed in order to mitigate these problems. Self-updating aims to add novel templates to the users' gallery among the inputs submitted during system operations. Consequently, computational complexity and storage space tend to be among the critical requirements of these algorithms. The present paper deals with the above problems by a novel template-based self-update algorithm, able to keep over time the expressive power of a limited set of templates stored in the system database. The rationale behind the proposed approach is in the working hypothesis that a dominating mode characterises the features' distribution given the client. Therefore, the key point is to select the best templates around that mode. We propose two methods, which are tested on systems based on handcrafted features and deep-learning-inspired autoencoders at the state-of-the-art. Three benchmark data sets are used. Experimental results confirm that, by effective and compact feature sets which can support our working hypothesis, the proposed classification-selection approaches overcome the problem of manual updating and, in case, stringent computational requirements.
Super-Resolution for Improving EEG Spatial Resolution using Deep Convolutional Neural Network—Feasibility Study

Electroencephalography (EEG) has relatively poor spatial resolution and may yield incorrect brain dynamics and distort topography; thus, high-density EEG systems are necessary for better analysis. Conventional methods have been proposed to solve these problems, however, they depend on parameters or brain models that are not simple to address. Therefore, new approaches are necessary to enhance EEG spatial resolution while maintaining its data properties. In this work, we investigated the super-resolution (SR) technique using deep convolutional neural networks (CNN) with simulated EEG data with white Gaussian and real brain noises, and experimental EEG data obtained during an auditory evoked potential task. SR EEG simulated data with white Gaussian noise or brain noise demonstrated a lower mean squared error and higher correlations with sensor information, and detected sources even more clearly than did low resolution (LR) EEG. In addition, experimental SR data also demonstrated far smaller errors for N1 and P2 components, and yielded reasonable localized sources, while LR data did not. We verified our proposed approach’s feasibility and efficacy, and conclude that it may be possible to explore various brain dynamics even with a small number of sensors.
FSPool: Learning Set Representations with Featurewise Sort Pooling

Traditional set prediction models can struggle with simple datasets due to an issue we call the responsibility problem. We introduce a pooling method for sets of feature vectors based on sorting features across elements of the set. This can be used to construct a permutation-equivariant auto-encoder that avoids this responsibility problem. On a toy dataset of polygons and a set version of MNIST, we show that such an auto-encoder produces considerably better reconstructions and representations. Replacing the pooling function in existing set encoders with FSPool improves accuracy and convergence speed on a variety of datasets.
Running virtual: The effect of virtual reality on exercise

Research has shown that exercise among college aged persons has dropped over recent years (Lindahl, 2015;Sheppard, 2016).Many factors could be contributing to this reduction in exercise including: large workloads, the need to work during school, or perhaps technology use.A number of recent studies are showing the benefits of using virtual reality systems in exercise and are demonstrating that the use of such technology can lead to an increase in the number of young adults engaging in exercise.This study focuses on the effects that virtual reality has on heart rate and other bodily sensations during a typical work out.This study also analyses the participants ability to pay less attention to their bodily sensations during exercise when using a virtual reality system.During this experiment, participants were exposed to two different conditions.Condition one being a traditional work out, riding an exercise bike at a middle tension level.Condition two was the same but the participant was wearing a virtual reality headset.The data collected led to the conclusion that working out while wearing a virtual reality headset will lead to a higher heart rate, and in turn can lead to burning more calories during a workout.The study also found participants who wore the virtual reality headset were able to remove themselves from their bodily sensations allowing them to workout longer.
Quantum Cryptography for Internet of Things Security

Internet of things (IoT) is a developing technology with a lot of scope in the future. It can ease various different tasks for us. On one hand, IoT is useful for us, on the other hand, it has many serious security threats, like data breaches, side-channel attacks, and virus and data authentication. Classical cryptographic algorithms, like the Rivest-Shamir-Adleman (RSA) algorithm, work well under the classical computers. But the technology is slowly shifting towards quantum computing, which has immense processing power and is more than enough to break the current cryptographic algorithms easily. So it is required that we have to design quantum cryptographic algorithms to prevent our systems from security breaches even before quantum computers come in the market for commercial uses. IoT will also be one of the disciplines, which needs to be secured to prevent any malicious activities. In this paper, we review the common security threats in IoT and the presently available solutions with their drawbacks. Then quantum cryptography is introduced with some of its variations. And finally, the analysis has been carried out in terms of the pros and cons of implementing quantum cryptography for IoT security.
Classification of engraved pottery sherds mixing deep-learning features by compact bilinear pooling

The ARCADIA project aims at using pattern recognition and machine learning to promote a systematic analysis of the large corpus of archaeological pottery fragments excavated in Saran (France). Dating from the High Middle Ages, these sherds have been engraved with repeated patterns using a carved wooden wheel. The study of these engraved patterns allows archaeologists to better understand the diffusion of ceramic productions. In this paper, we present a method that classifies patterns of ceramic sherds by combining deep learning-based features extracted from some pre-trained Convolutional Neural Network (CNN) models. A dataset composed of 888 digital patterns extracted from 3D scans of pottery sherds was used to evaluate our approach. The classification capacity of each CNN model was first assessed individually. Then, several combinations of common pooling methods using different classifiers were tested. The best result was obtained when features of the VGG19 and ResNet50 models were combined using Compact Bilinear Pooling (CBP) with a high classification rate of 95.23%.
Variational Metric Scaling for Metric-Based Meta-Learning

Metric-based meta-learning has attracted a lot of attention due to its effectiveness and efficiency in few-shot learning. Recent studies show that metric scaling plays a crucial role in the performance of metric-based meta-learning algorithms. However, there still lacks a principled method for learning the metric scaling parameter automatically. In this paper, we recast metric-based meta-learning from a Bayesian perspective and develop a variational metric scaling framework for learning a proper metric scaling parameter. Firstly, we propose a stochastic variational method to learn a single global scaling parameter. To better fit the embedding space to a given data distribution, we extend our method to learn a dimensional scaling vector to transform the embedding space. Furthermore, to learn task-specific embeddings, we generate task-dependent dimensional scaling vectors with amortized variational inference. Our method is end-to-end without any pre-training and can be used as a simple plug-and-play module for existing metric-based meta-algorithms. Experiments on miniImageNet show that our methods can be used to consistently improve the performance of existing metric-based meta-algorithms including prototypical networks and TADAM.
Explanation vs Attention: A Two-Player Game to Obtain Attention for VQA

In this paper, we aim to obtain improved attention for a visual question answering (VQA) task. It is challenging to provide supervision for attention. An observation we make is that visual explanations as obtained through class activation mappings (specifically Grad-CAM) that are meant to explain the performance of various networks could form a means of supervision. However, as the distributions of attention maps and that of Grad-CAMs differ, it would not be suitable to directly use these as a form of supervision. Rather, we propose the use of a discriminator that aims to distinguish samples of visual explanation and attention maps. The use of adversarial training of the attention regions as a two-player game between attention and explanation serves to bring the distributions of attention maps and visual explanations closer. Significantly, we observe that providing such a means of supervision also results in attention maps that are more closely related to human attention resulting in a substantial improvement over baseline stacked attention network (SAN) models. It also results in a good improvement in rank correlation metric on the VQA task. This method can also be combined with recent MCB based methods and results in consistent improvement. We also provide comparisons with other means for learning distributions such as based on Correlation Alignment (Coral), Maximum Mean Discrepancy (MMD) and Mean Square Error (MSE) losses and observe that the adversarial loss outperforms the other forms of learning the attention maps. Visualization of the results also confirms our hypothesis that attention maps improve using this form of supervision.
Effective Decoding in Graph Auto-Encoder Using Triadic Closure

The (variational) graph auto-encoder and its variants have been popularly used for representation learning on graph-structured data. While the encoder is often a powerful graph convolutional network, the decoder reconstructs the graph structure by only considering two nodes at a time, thus ignoring possible interactions among edges. On the other hand, structured prediction, which considers the whole graph simultaneously, is computationally expensive. In this paper, we utilize the well-known triadic closure property which is exhibited in many real-world networks. We propose the triad decoder, which considers and predicts the three edges involved in a local triad together. The triad decoder can be readily used in any graph-based auto-encoder. In particular, we incorporate this to the (variational) graph auto-encoder. Experiments on link prediction, node clustering and graph generation show that the use of triads leads to more accurate prediction, clustering and better preservation of the graph characteristics.
Fusion of augmented reality imaging with the endoscopic view for endonasal skull base surgery; a novel application for surgical navigation based on intraoperative cone beam computed tomography and optical tracking

Objective Surgical navigation is a well-established tool in endoscopic skull base surgery. However, navigational and endoscopic views are usually displayed on separate monitors, forcing the surgeon to focus on one or the other. Aiming to provide real-time integration of endoscopic and diagnostic imaging information, we present a new navigation technique based on augmented reality with fusion of intraoperative cone beam computed tomography (CBCT) on the endoscopic view. The aim of this study was to evaluate the accuracy of the method. Material and methods An augmented reality surgical navigation system (ARSN) with 3D CBCT capability was used. The navigation system incorporates an optical tracking system (OTS) with four video cameras embedded in the flat detector of the motorized C-arm. Intra-operative CBCT images were fused with the view of the surgical field obtained by the endoscope's camera. Accuracy of CBCT image co-registration was tested using a custom-made grid with incorporated 3D spheres. Results Co-registration of the CBCT image on the endoscopic view was performed. Accuracy of the overlay, measured as mean target registration error (TRE), was 0.55 mm with a standard deviation of 0.24 mm and with a median value of 0.51mm and interquartile range of 0.39˗˗0.68 mm. Conclusion We present a novel augmented reality surgical navigation system, with fusion of intraoperative CBCT on the endoscopic view. The system shows sub-millimeter accuracy.
Solar Irradiance Capturing in Cloudy Sky Days–A Convolutional Neural Network Based Image Regression Approach

Global horizontal irradiance (GHI) is a critical index to indicate the output power of the photovaltaic (PV).In traditional approaches, the local GHI can be measured with very expensive instruments, and the large-area GHI collection depends on complex satellite-based models, solargis algorithms, and the high-performance computers (HPC).In this paper, a novel approach is proposed to capture the GHI conveniently and accurately.Considering the nonstationary property of the GHI on cloudy days, the GHI capturing is cast as an image regression problem.In traditional approaches, the image regression problem is treated as two parts, feature extraction (for the images) and regression model (for the regression targets), which are optimized separately and blocked the interconnections.Considering the nonlinear regression capability, a convolutional neural network (CNN) based image regression approach is proposed to provide an End-to-End solution for the cloudy day GHI capturing problem in this paper.The multilayer CNN is based on the AlexNet and VGG.The L2 (least square errors) with regularization is used as the loss function in the regression layer.For data cleaning, the Gaussian mixture model with Bayesian inference is employed to detect and eliminate the anomaly data in a nonparametric manner.The purified data are used as input data for the proposed image regression approach.In the experiments, three-month sky images and GHI data (with 1-min resolution) are provided by the National Renewable Energy Laboratory (NREL) with the HPC system.The numerical results demonstrate the feasibility and effectiveness of the proposed approach.