Reviewerly/exHarmony huggingface dataset:
A Study on Combating Emerging Threat of Deepfake Weaponization

A breakthrough in the emerging use of machine learning and deep learning is the concept of autoencoders and GAN (Generative Adversarial Networks), architectures that can generate believable synthetic content called deepfakes. The threat lies when these low-tech doctored images, videos, and audios blur the line between fake and genuine content and are used as weapons to cause damage to an unprecedented degree. This paper presents a survey of the underlying technology of deepfakes and methods proposed for their detection. Based on a detailed study of all the proposed models of detection, this paper presents SSTNet as the best model to date, that uses spatial, temporal, and steganalysis for detection. The threat posed by document and signature forgery, which is yet to be explored by researchers, has also been highlighted in this paper. This paper concludes with the discussion of research directions in this field and the development of more robust techniques to deal with the increasing threats surrounding deepfake technology.
PRODeep: a platform for robustness verification of deep neural networks

Deep neural networks (DNNs) have been applied in safety-critical domains such as self driving cars, aircraft collision avoidance systems, malware detection, etc. In such scenarios, it is important to give a safety guarantee to the robustness property, namely that outputs are invariant under small perturbations on the inputs. For this purpose, several algorithms and tools have been developed recently. In this paper, we present PRODeep, a platform for robustness verification of DNNs. PRODeep incorporates constraint-based, abstraction-based, and optimisation-based robustness checking algorithms. It has a modular architecture, enabling easy comparison of different algorithms. With experimental results, we illustrate the use of the tool, and easy combination of those techniques.
Bias-free source-independent quantum random number generator

A bias-free source-independent quantum random number generator scheme based on the measurement of vacuum fluctuation is proposed to realize the effective elimination of system bias and common mode noise introduced by the local oscillator. Optimal parameter settings are derived to avoid the system recording two canonically conjugate quadratures simultaneously in each measurement. In particular, it provides a new approach to investigate the performance difference between measuring two quadratures of equal and unequal intensity. It is experimentally demonstrated that the system supports 4.2 Gbps bias-free source-independent random number generation, where its common mode rejection ratio reaches 61.17 dB. Furthermore, the scheme offers an all-optical method facilitating the integration of source-independent quantum random number generators into compact chips.
Dual Indicators to Analyze AI Benchmarks: Difficulty, Discrimination, Ability, and Generality

Item response theory (IRT) can be applied to the analysis of the evaluation of results from AI benchmarks. The two-parameter IRT model provides two indicators (difficulty and discrimination) on the side of the item (or AI problem) while only one indicator (ability) on the side of the respondent (or AI agent). In this paper we analyse how to make this set of indicators dual, by adding a fourth indicator, generality, on the side of the respondent. Generality is meant to be dual to discrimination, and it is based on difficulty. Namely, generality is defined as a new metric that evaluates whether an agent is consistently good at easy problems and bad at difficult ones. With the addition of generality, we see that this set of four key indicators can give us more insight on the results of AI benchmarks. In particular, we explore two popular benchmarks in AI, the Arcade Learning Environment (Atari 2600 games) and the General Video Game AI competition. We provide some guidelines to estimate and interpret these indicators for other AI benchmarks and competitions.
Entropic uncertainty relations from quantum designs

In the course of the last decades entropic uncertainty relations have attracted much attention not only due to their fundamental role as manifestation of non-classicality of quantum mechanics, but also as major tools for applications of quantum information theory. Amongst the latter are protocols for the detection of quantum correlations or for the secure distribution of secret keys. In this work we show how to derive entropic uncertainty relations for sets of measurements whose effects form quantum designs. The key property of quantum designs is their indistinguishability from truly random quantum processes as long as one is concerned with moments up to some finite order. Exploiting this characteristic enables us to evaluate polynomial functions of measurement probabilities which leads to lower bounds on sums of generalized entropies. As an application we use the derived uncertainty relations to investigate the incompatibility of sets of binary observables.
Integrating Out Nuisance Parameters for Computationally More Efficient Bayesian Estimation – An Illustration and Tutorial

Formulae display:?Mathematical formulae have been encoded as MathML and are displayed in this HTML version using MathJax in order to improve their display. Uncheck the box to turn MathJax off. This feature requires Javascript. Click on a formula to zoom.
Multimodal Sentiment Analysis with Multi-perspective Fusion Network Focusing on Sense Attentive Language

Multimodal sentiment analysis aims to learn a joint representation of multiple features. As demonstrated by previous studies, it is shown that the language modality may contain more semantic information than that of other modalities. Based on this observation, we propose a Multi-perspective Fusion Network(MPFN) focusing on Sense Attentive Language for multimodal sentiment analysis. Different from previous studies, we use the language modality as the main part of the final joint representation, and propose a multi-stage and uni-stage fusion strategy to get the fusion representation of the multiple modalities to assist the final language-dominated multimodal representation. In our model, a Sense-Level Attention Network is proposed to dynamically learn the word representation which is guided by the fusion of the multiple modalities. As in turn, the learned language representation can also help the multi-stage and uni-stage fusion of the different modalities. In this way, the model can jointly learn a well integrated final representation focusing on the language and the interactions between the multiple modalities both on multi-stage and uni-stage. Several experiments are carried on the CMU-MOSI, the CMU-MOSEI and the YouTube public datasets. The experiments show that our model performs better or competitive results compared with the baseline models.
Entanglement-breaking superchannels

In this paper we initiate the study of entanglement-breaking (EB) superchannels. These are processes that always yield separable maps when acting on one side of a bipartite completely positive (CP) map. EB superchannels are a generalization of the well-known EB channels. We give several equivalent characterizations of EB supermaps and superchannels. Unlike its channel counterpart, we find that not every EB superchannel can be implemented as a measure-and-prepare superchannel. We also demonstrate that many EB superchannels can be superactivated, in the sense that they can output non-separable channels when wired in series. We then introduce the notions of CPTP- and CP-complete images of a superchannel, which capture deterministic and probabilistic channel convertibility, respectively. This allows us to characterize the power of EB superchannels for generating CP maps in different scenarios, and it reveals some fundamental differences between channels and superchannels. Finally, we relax the definition of separable channels to include (p,q)-non-entangling channels, which are bipartite channels that cannot generate entanglement using p- and q-dimensional ancillary systems. By introducing and investigating k-EB maps, we construct examples of (p,q)-EB superchannels that are not fully entanglement breaking. Partial results on the characterization of (p,q)-EB superchannels are also provided.
Does stress impact technical interview performance?

Software engineering candidates commonly participate in whiteboard technical interviews as part of a hiring assessment. During these sessions, candidates write code while thinking aloud as they work towards a solution, under the watchful eye of an interviewer. While technical interviews should allow for an unbiased and inclusive assessment of problem-solving ability, surprisingly, technical interviews may be instead a procedure for identifying candidates who best handle and migrate stress solely caused by being examined by an interviewer (performance anxiety).
Beyond Local Nash Equilibria for Adversarial Networks

Save for some special cases, current training methods for Generative Adversarial Networks (GANs) are at best guaranteed to converge to a 'local Nash equilibrium' (LNE). Such LNEs, however, can be arbitrarily far from an actual Nash equilibrium (NE), which implies that there are no guarantees on the quality of the found generator or classifier. This paper proposes to model GANs explicitly as finite games in mixed strategies, thereby ensuring that every LNE is an NE. We use the Parallel Nash Memory as a solution method, which is proven to monotonically converge to a resource-bounded Nash equilibrium. We empirically demonstrate that our method is less prone to typical GAN problems such as mode collapse and produces solutions that are less exploitable than those produced by GANs and MGANs.
Practical Poisoning Attacks on Neural Networks

Data poisoning attacks on machine learning models have attracted much recent attention, wherein poisoning samples are injected at the training phase to achieve adversarial goals at test time. Although existing poisoning techniques prove to be effective in various scenarios, they rely on certain assumptions on the adversary knowledge and capability to ensure efficacy, which may be unrealistic in practice. This paper presents a new, practical targeted poisoning attack method on neural networks in vision domain, namely BlackCard. BlackCard possesses a set of critical properties for ensuring attacking efficacy in practice, which has never been simultaneously achieved by any existing work, including knowledge-oblivious, clean-label, and clean-test. Importantly, we show that the effectiveness of BlackCard can be intuitively guaranteed by a set of analytical reasoning and observations, through exploiting an essential characteristic of gradient-descent optimization which is pervasively adopted in DNN models. We evaluate the efficacy of BlackCard for generating targeted poisoning attacks via extensive experiments using various datasets and DNN models. Results show that BlackCard is effective with a rather high success rate while preserving all the claimed properties.
LSMAT Least Squares Medial Axis Transform

The medial axis transform has applications in numerous fields including visualization, computer graphics, and computer vision. Unfortunately, traditional medial axis transformations are usually brittle in the presence of outliers, perturbations and/or noise along the boundary of objects. To overcome this limitation, we introduce a new formulation of the medial axis transform which is naturally robust in the presence of these artifacts. Unlike previous work which has approached the medial axis from a computational geometry angle, we consider it from a numerical optimization perspective. In this work, we follow the definition of the medial axis transform as "the set of maximally inscribed spheres". We show how this definition can be formulated as a least squares relaxation where the transform is obtained by minimizing a continuous optimization problem. The proposed approach is inherently parallelizable by performing independant optimization of each sphere using Gauss-Newton, and its least-squares form allows it to be significantly more robust compared to traditional computational geometry approaches. Extensive experiments on 2D and 3D objects demonstrate that our method provides superior results to the state of the art on both synthetic and real-data.
Face Recognition for Smart Door Lock System using Hierarchical Network

Face recognition system is broadly used for human identification because of its capacity to measure the facial points and recognize the identity in an unobtrusive way. The application of face recognition systems can be applied to surveillance at home, workplaces, and campuses, accordingly. The problem with existing face recognition systems is that they either rely on the facial key points and landmarks or the face embeddings from FaceNet for the recognition process. In this paper, we propose a hierarchical network (HN) framework which uses pre-trained architecture for recognizing faces followed by the validation from face embeddings using FaceNet. We also designed a real-time face recognition security door lock system connected with raspberry pi as an implication of the proposed method. The evaluation of the proposed work has been conducted on the dataset collected from 12 students from Faculty of Engineering and Technology, University of Sindh. The experimental results show that the proposed method achieves better results over existing works. We also carried out a comparison on random faces acquired from the Internet to perform face recognition and results shows that the proposed HN framework is resilient to the randomly acquired faces.
Coarse Temporal Attention Network (CTA-Net) for Driver’s Activity Recognition

There is significant progress in recognizing traditional human activities from videos focusing on highly distinctive actions involving discriminative body movements, bodyobject and/or human-human interactions.Driver's activities are different since they are executed by the same subject with similar body parts movements, resulting in subtle changes.To address this, we propose a novel framework by exploiting the spatiotemporal attention to model the subtle changes.Our model is named Coarse Temporal Attention Network (CTA-Net), in which coarse temporal branches are introduced in a trainable glimpse network.The goal is to allow the glimpse to capture high-level temporal relationships, such as 'during', 'before' and 'after' by focusing on a specific part of a video.These branches also respect the topology of the temporal dynamics in the video, ensuring that different branches learn meaningful spatial and temporal changes.The model then uses an innovative attention mechanism to generate high-level action specific contextual information for activity recognition by exploring the hidden states of an LSTM.The attention mechanism helps in learning to decide the importance of each hidden state for the recognition task by weighing them when constructing the representation of the video.Our approach is evaluated on four publicly accessible datasets and significantly outperforms the state-of-the-art by a considerable margin with only RGB video as input.
A Novel Joint Framework for Multiple Chinese Events Extraction

Event extraction is an essential yet challenging task in information extraction. Previous approaches have paid little attention to the problem of roles overlap which is a common phenomenon in practice. To solve this problem, this paper defines event relation triple to explicitly represent relations among triggers, arguments and roles which are incorporated into the model to learn their inter-dependencies. A novel joint framework for multiple Chinese events extraction is proposed which jointly performs predictions for event triggers and arguments based on shared feature representations from pre-trained language model. Experimental comparison with state-of-the-art baselines on ACE 2005 dataset shows the superiority of the proposed method in both trigger classification and argument classification.
An enhanced reversible patient data hiding algorithm for E-healthcare

The feasibility of future developments in computing technologies strongly relies on the existence of key technologies and their deployments. One of the areas where they are being deployed is telemedicine, it requires communication of patient report to a specialized doctor in a real time scenario. Therefore, any harm to patient medical data can lead to a faulty diagnosis that can be lethal for the patient. To ensure safe and secure communication in E-healthcare framework, a high capacity reversible data hiding algorithm in the encrypted domain has been proposed in this paper. The proposed algorithm gives a higher embedding rate by embedding n binary bits of patient data in basek numeral framework at every pixel of the cover image without any occurrence of underflow and overflow problem. In addition to electronic patient information (EPI), a fragile watermark has been embedded for observing any tamper to the patient data during transmission phase. The experimental study reveals that for all types of test images, proposed method has an embedding rate nearly twice larger than the compared methods, precisely recover EPI with a PSNR value of ∞ dB between the cover image and reconstructed image successfully.
Deep Learning-Based Pupil Center Detection for Fast and Accurate Eye Tracking System

In augmented reality (AR) or virtual reality (VR) systems, eye tracking is a key technology and requires significant accuracy as well as real-time operation. Many techniques for detecting pupil centers with error range of iris radius have been developed, but few techniques have precise performance with error range of pupil radius. In addition, the conventional methods rarely guarantee real-time pupil center detection in a general-purpose computer environment due to high complexity. Thus, we propose more accurate pupil center detection by improving the representation quality of the network in charge of pupil center detection. This is realized by representation learning based on mutual information. Also, the latency of the entire system is greatly reduced by using non-local block and self-attention block with large receptive field, which makes it accomplish real-time operation. The proposed system not only shows real-time performance of 52 FPS in a general-purpose computer environment but also provides state-of-the-art accuracy in terms of fine level index of 96.71%, 99.84% and 96.38% for BioID, GI4E and Talking Face Video datasets, respectively.
Limit theorems for sequential MCMC methods

Sequential Monte Carlo (SMC) methods, also known as particle filters, constitute a class of algorithms used to approximate expectations with respect to a sequence of probability distributions as well as the normalising constants of those distributions. Sequential MCMC methods are an alternative class of techniques addressing similar problems in which particles are sampled according to an MCMC kernel rather than conditionally independently at each time step. These methods were introduced over twenty years ago by Berzuini et al. (1997). Recently, there has been a renewed interest in such algorithms as they demonstrate an empirical performance superior to that of SMC methods in some applications. We establish a strong law of large numbers and a central limit theorem for sequential MCMC methods and provide conditions under which errors can be controlled uniformly in time. In the context of state-space models, we provide conditions under which sequential MCMC methods can indeed outperform standard SMC methods in terms of asymptotic variance of the corresponding Monte Carlo estimators.
Passive-state preparation in continuous-variable measurement-device-independent quantum key distribution

Continuous-variable (CV) measurement-device-independent (MDI) quantum key distribution (QKD) is immune to imperfect detection devices, which can eliminate all kinds of attacks on practical detectors. Here we first propose a CV-MDI QKD scheme using unidimensional modulation (UD) in general phase-sensitive channels. The UD CV-MDI QKD protocol is implemented with the Gaussian modulation of a single quadrature of the coherent states prepared by two legitimate senders, aiming to simplify the implementation compared with the standard, symmetrically Gaussian-modulated CVMDI QKD protocol. Our scheme reduces the complexity of the system since it ignores the requirement in one of the quadrature modulations as well as the corresponding parameter estimations. The security of our proposed scheme is analyzed against collective attacks, and the finite-size analysis under realistic conditions is taken into account. UD CV-MDI QKD shows a comparable performance to that of its symmetrical counterpart, which will facilitate the simplification and practical implementation of the CV-MDI QKD protocols.
Consistent Transcription and Translation of Speech

The conventional paradigm in speech translation starts with a speech recognition step to generate transcripts, followed by a translation step with the automatic transcripts as input. To address various shortcomings of this paradigm, recent work explores end-to-end trainable direct models that translate without transcribing. However, transcripts can be an indispensable output in practical applications, which often display transcripts alongside the translations to users. We make this common requirement explicit and explore the task of jointly transcribing and translating speech. Although high accuracy of transcript and translation are crucial, even highly accurate systems can suffer from inconsistencies between both outputs that degrade the user experience. We introduce a methodology to evaluate consistency and compare several modeling approaches, including the traditional cascaded approach and end-to-end models. We find that direct models are poorly suited to the joint transcription/translation task, but that end-to-end models that feature a coupled inference procedure are able to achieve strong consistency. We further introduce simple techniques for directly optimizing for consistency, and analyze the resulting trade-offs between consistency, transcription accuracy, and translation accuracy. 1
OCR Post Correction for Endangered Language Texts

There is little to no data available to build natural language processing models for most endangered languages.However, textual data in these languages often exists in formats that are not machine-readable, such as paper books and scanned images.In this work, we address the task of extracting text from these resources.We create a benchmark dataset of transcriptions for scanned books in three critically endangered languages and present a systematic analysis of how general-purpose OCR tools are not robust to the data-scarce setting of endangered languages.We develop an OCR postcorrection method tailored to ease training in this data-scarce setting, reducing the recognition error rate by 34% on average across the three languages.1