Reviewerly/exHarmony huggingface dataset:
"Seeing is not Always Believing": Detecting Perception Error Attacks Against Autonomous Vehicles

Due to the great achievements in artificial intelligence, it is predicted that autonomous vehicles with little or even no human involvement will come to market in the near future. Autonomous vehicles are equipped with multiple types of sensors. An autonomous vehicle relies on its sensors to perceive its environment, and this sensory information plays a key role in the vehicle's driving decisions. Hence, ensuring the trustworthiness of the sensor data is crucial for drivers’ safety. In this article, we discuss the impact of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">perception error attacks (PEAs)</i> on autonomous vehicles, and propose a countermeasure called LIFE ( <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">L</b> IDAR and <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">I</b> mage data <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">F</b> usion for detecting perception <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">E</b> rrors). LIFE detects PEAs by analyzing the consistency between camera image data and LIDAR data using novel machine learning and computer vision algorithms. The performance of LIFE has been evaluated extensively using the KITTI dataset.
Subjective and Objective Evaluation of Deepfake Videos

Practically anyone can now generate a realistic looking deepfake video. It is clear that the online prevalence of such fake videos will erode the societal trust in video evidence even further. To counter the looming threat, many methods to detect deepfakes were recently proposed by the research community. However, it is still unclear how realistic deep-fake videos are for an average person and whether the algorithms are significantly better than humans at detecting them. Therefore, this paper, presents a subjective study, which, using 60 naïve subjects, evaluates how hard it is for humans to see if a video is a deepfake or not. For the study, 120 videos (60 deepfakes and 60 originals) were manually selected from the Facebook database used in Kaggle’s Deepfake Detection Challenge 2020. The results of the subjective evaluation were compared with two state of the art deepfake detection methods, based on Xception and EfficientNet (B4 variant) neural network models pre-trained on two other public databases: Google and Jiqsaw subset from FaceForensics++ and Celeb-DF v2 dataset. The experiments demonstrate that while the human perception is very different from the perception of a machine, both successfully but in different ways are fooled by deepfakes. Specifically, algorithms struggle to detect the deepfake videos that humans find to be very easy to spot.
Auto-sharing parameters for transfer learning based on multi-objective optimization

Transfer learning methods exploit similarities between different datasets to improve the performance of the target task by transferring knowledge from source tasks to the target task. “What to transfer” is a main research issue in transfer learning. The existing transfer learning method generally needs to acquire the shared parameters by integrating human knowledge. However, in many real applications, an understanding of which parameters can be shared is unknown beforehand. Transfer learning model is essentially a special multi-objective optimization problem. Consequently, this paper proposes a novel auto-sharing parameter technique for transfer learning based on multi-objective optimization and solves the optimization problem by using a multi-swarm particle swarm optimizer. Each task objective is simultaneously optimized by a sub-swarm. The current best particle from the sub-swarm of the target task is used to guide the search of particles of the source tasks and vice versa. The target task and source task are jointly solved by sharing the information of the best particle, which works as an inductive bias. Experiments are carried out to evaluate the proposed algorithm on several synthetic data sets and two real-world data sets of a school data set and a landmine data set, which show that the proposed algorithm is effective.
Subject 001 - A Detailed Self-Report of Virtual Reality Induced Sickness

Cybersickness is a show stopper and fortunately, the awareness of this has massively raised within the last couple of years. Still, a lot of people do not know how it feels and draw wrong or unrealistic claims from existing results. With this detailed self-report from an expert subject who is hypersensitive to cybersickness, we want to give a link to how it feels for the individual. Finally, the self-report is also set in a relation to general observations we made within experiments in the last years.
A Neural Question Answering System for Basic Questions about Subroutines

A question answering (QA) system is a type of conversational AI that generates natural language answers to questions posed by human users. QA systems often form the backbone of interactive dialogue systems, and have been studied extensively for a wide variety of tasks ranging from restaurant recommendations to medical diagnostics. Dramatic progress has been made in recent years, especially from the use of encoder-decoder neural architectures trained with big data input. In this paper, we take initial steps to bringing state-of-the-art neural QA technologies to Software Engineering applications by designing a context-based QA system for basic questions about subroutines. We curate a training dataset of 10.9 million question/context/answer tuples based on rules we extract from recent empirical studies. Then, we train a custom neural QA model with this dataset and evaluate the model in a study with professional programmers. We demonstrate the strengths and weaknesses of the system, and lay the groundwork for its use in eventual dialogue systems for software engineering.
A Parallelizable Lattice Rescoring Strategy with Neural Language Models

This paper proposes a parallel computation strategy and a posterior-based lattice expansion algorithm for efficient lattice rescoring with neural language models (LMs) for automatic speech recognition. First, lattices from first-pass decoding are expanded by the proposed posterior-based lattice expansion algorithm. Second, each expanded lattice is converted into a minimal list of hypotheses that covers every arc. Each hypothesis is constrained to be the best path for at least one arc it includes. For each lattice, the neural LM scores of the minimal list are computed in parallel and are then integrated back to the lattice in the rescoring stage. Experiments on the Switchboard dataset show that the proposed rescoring strategy obtains comparable recognition performance and generates more compact lattices than a competitive baseline method. Furthermore, the parallel rescoring method offers more flexibility by simplifying the integration of PyTorch-trained neural LMs for lattice rescoring with Kaldi.
Collusion resistant secret sharing scheme for secure data storage and processing over cloud

Shamir secret sharing (SSS) is considered as a promising method for outsourcing the data securely due to its ability to support privacy preserving data processing while ensuring data availability. Major drawbacks of original SSS scheme are its susceptibility to collusion attack and high storage overhead. Hence in this paper, we first propose a modified SSS scheme (MSSS) which can resist collusion attack and provide adequate security even with two shares. However, the storage overhead of this scheme is high when it is extended to ensure data availability and integrity in cloud storage systems. Therefore, a modified ramp secret sharing (MRSS) with reduced storage overhead compared to MSSS scheme is also proposed in this paper. The proposed schemes can be employed for any privacy preserving data processing application which involve linear operations on the data. In this paper, in order to demonstrate the capability of proposed schemes to support privacy preserving data processing, Haar discrete wavelet transform (DWT) computation on medical images is considered as an example as DWT is widely used in feature extraction for disease diagnosis from pathological images. We present an algorithm for computing Haar DWT from medical image shares. The security of the proposed scheme is evaluated through mathematical cryptanalysis and resistance against various statistical attacks. The performance analysis shows that shared domain DWT offers same accuracy levels as that of plaintext domain.
Seeing in the Dark by Component-GAN

Recently, Retinex theory based low-light image enhancement (LLIE) algorithms have achieved impressive results in controlled environment. However, the majority of deep learning based LLIE algorithms leverage relighting by enhancing the illumination components that directly determines the image brightness, regretfully, they ignore the information of reflectance components, which may cause problems such as image noise and color distortion in reconstructed images. To tackle this problem, in this letter, we propose a component enhancement network based on Generative Adversarial Network (Component-GAN) for recovering clear images from low-light ones. Specifically, the network is composed of the decomposition part for dividing the paired low/normal-light images into illumination components and reflectance components, and the enhancement part for generating high-quality images. It is worth to note that we provide two branches of component enhancement network, which are parallel to improve the two components simultaneously. Hereby, we treat the reconstruction part as the generative network and adopt discriminative network to boost image reconstruction performance. Through extensive experiments, the proposed approach outperforms some state-of-the-art LLIE methods in terms of visual and subjective qualities.
Adaptive Adapters: An Efficient Way to Incorporate BERT Into Neural Machine Translation

Large-scale pre-trained language models (e.g., BERT) have attracted great attention in recent years. It is straightforward to fine-tune them on natural language understanding tasks such as text classification, however, effectively and efficiently incorporating them into natural language generation tasks such as neural machine translation remains a challenging problem. In this paper, we integrate two pre-trained BERT models from the source and target language domains into a sequence-to-sequence model by introducing light-weight adapter modules. The adapters are inserted between BERT layers and tuned on downstream tasks, while the parameters of BERT models are fixed during fine-tuning. As pre-trained language models are usually very deep, inserting adapters into all layers will result in a considerable scale of new parameters. To deal with this problem, we introduce latent variables to decide whether using adapters or not in each layer, which are learned during fine-tuning. In this way, the model is able to automatically determine which adapters to use, therefore hugely promoting the parameter efficiency and decoding speed. We evaluate the proposed framework on various neural machine translation tasks. Equipped with parallel sequence decoding, our model consistently outperforms autoregressive baselines while reducing the inference latency by half. With automatic adapter selection, the proposed model further achieves 20% speedup while still outperforming autoregressive baselines. When applied to autoregressive decoding, the proposed model can also achieve comparable performance with the state-of-the-art baseline models.
Understanding Language Selection in Multi-language Software Projects on GitHub

There are hundreds of programming languages available for software development today. As a result, modern software is increasingly developed in multiple languages. In this context, there is an urgent need for automated tools for multi-language software quality assurance. To that end, it is useful to first understand how languages are chosen by developers in multi-language software projects. One intuitive perspective towards the understanding would be to explore the potential functionality relevance of those choices. With a plethora of publicly hosted multi-language software projects available on GitHub, we were able to obtain thousands of popular, relevant repositories across 10 years from 2010 to 2019 to enable the exploration. We start by estimating the functionality domain of each project through topic modeling, followed by studying the statistical correlation between these domains and language selection over all the sample projects through association mining. We proceed with an evolutionary characterization of these projects to provide a longitudinal view of how the association has changed over the years. Our findings offer useful insights into the rationale behind developers' choices of language combinations in multi-language software construction.
Group Target Tracking Based on MS-MeMBer Filters

This paper presents a new group target tracking method based on the standard multi-sensor multi-target multi-Bernoulli (MS-MeMBer) filter. In the prediction step, the group structure is used to constrain the movement of the constituent members within the respective groups. Specifically, the group of members is considered as an undirected random graph. Combined with the virtual leader-follower model, the motion equation of the members within groups is formulated. In the update step, the partitioning problem of multiple sensors is transformed into a multi-dimensional assignment (MDA) problem. Compared with the original two-step greedy partitioning mechanism, the MDA algorithm achieves better measurement partitions in group target tracking scenarios. To evaluate the performance of the proposed method, a simulation scenario including group splitting and merging is established. Results show that, compared with the standard MS-MeMBer filter, our method can effectively estimate the cardinality of members and groups at the cost of increasing computational load. The filtering accuracy of the proposed method outperforms that of the MS-MeMBer filter.
Contrastive Predictive Coding Supported Factorized Variational Autoencoder For Unsupervised Learning Of Disentangled Speech Representations

In this work we address disentanglement of style and content in speech signals. We propose a fully convolutional variational autoencoder employing two encoders: a content encoder and a style encoder. To foster disentanglement, we propose adversarial contrastive predictive coding. This new disentanglement method does neither need parallel data nor any supervision. We show that the proposed technique is capable of separating speaker and content traits into the two different representations and show competitive speaker-content disentanglement performance compared to other unsupervised approaches. We further demonstrate an increased robustness of the content representation against a train-test mismatch compared to spectral features, when used for phone recognition.
Adversarial domain adaptation network for tumor image diagnosis

In medical fields it is very difficult and time-consuming to label samples. Only a small number of labeled samples or unlabeled samples are often encountered in medical fields. How to deal with this problem in medical diagnosis? Domain adaptation is an effective machine learning method to solve the scarce or no labeled samples problem. In this paper, an improved adversarial domain adaptation network is presented to solve this problem in tumor image diagnosis. We construct the tumor image diagnosis adversarial domain adaptation network framework. The framework consists of three parts: a feature extractor, a domain discriminator and a label predictor. The loss function of framework is designed in the paper. We design a feature extractor to acquire the domain invariant feature between the source domain and target domain. The feature extractor is an improved neural network consisting of several convolutional layers and sampling layers. The feature extractor pre-processes the input samples and gets the initial feature of the input space. The domain discriminator predicts the domain label and works adversarial with the feature extractor. The adversarial learning between the feature extractor and the domain discriminator acts as a regularization in the framework. The adversarial game can minimize the distance between the two domains and make sure the feature extractor learns the domain invariant feature. A label predictor is designed to classify the tumor image of the target domain based on the domain-invariant feature. The experiments on tumor image dataset are presented to validate the performance of proposed method. The experimental results show that the proposed method is superior to the other domain adaptation methods.
Mutually unbiased balanced functions and generalized random access codes

Quantum resources and protocols are known to outperform their classical counterparts in variety of communication and information processing tasks. Random Access Codes (RACs) are one such cryptographically significant family of bipartite communication tasks, wherein, the sender encodes a data set (typically a string of input bits) onto a physical system of bounded dimension and transmits it to the receiver, who then attempts to guess a randomly chosen part of the sender's data set (typically one of the sender's input bits). In this work, we introduce a generalization of this task wherein the receiver, in addition to the individual input bits, aims to retrieve randomly chosen functions of sender's input string. Specifically, we employ sets of mutually unbiased balanced functions (MUBS), such that perfect knowledge of any one of the constituent functions yields no knowledge about the others. We investigate and bound the performance of (i.) classical, (ii.) quantum prepare and measure, and (iii.) entanglement assisted classical communication (EACC) protocols for the resultant generalized RACs (GRACs). Finally, we detail the case of GRACs with three input bits, find maximal success probabilities for classical, quantum and EACC protocols, along with the effect of noisy quantum channels on the performance of quantum protocols. Moreover, with the help of this case study, we reveal several characteristic properties of the GRACs which deviate from the standard RACs.
Semi-Supervised Contrastive Learning With Similarity Co-Calibration

Semi-supervised learning acts as an effective way to leverage massive unlabeled data. In this paper, we propose a novel training strategy, termed as <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Semi-supervised Contrastive Learning (SsCL)</b> , which combines the well-known contrastive loss in self-supervised learning with the cross entropy loss in semi-supervised learning, and jointly optimizes the two objectives in an end-to-end way. The highlight is that different from self-training based semi-supervised learning that conducts prediction and retraining over the same model weights, SsCL interchanges the predictions over the unlabeled data between the two branches, and thus formulates a co-calibration procedure, which we find is beneficial for better prediction and avoids being trapped in local minimum. Towards this goal, the contrastive loss branch models pairwise similarities among samples, using the pseudo labels generated from the cross entropy branch, and in turn calibrates the prediction distribution of the cross entropy branch with the contrastive similarity. We show that SsCL produces more discriminative representation and is beneficial to semi-supervised learning. Notably, on ImageNet with ResNet50 as the backbone, SsCL achieves <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\bm {60.2\%}$</tex-math></inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\bm {72.1\%}$</tex-math></inline-formula> top-1 accuracy with 1% and 10% labeled samples respectively, which significantly outperforms the baseline, and is better than previous semi-supervised and self-supervised methods.
Multi-Factor Authentication to Systems Login

Multi-Factor Authentication is an electronic authentication method in which a computer user is granted access to an application or a website only after successfully presenting two or more factors, or pieces of evidence. It is the first step to protect systems against intruders since the traditional log-in methods (username and password) are not completely protected from hackers, since they can guess them easily using tools. Current Systems use additional methods to increase security, such as using two-factor authentication based on a one-time password via mobile or email, or authentication based on biometrics (fingerprint, eye iris or retina, and face recognition) or via token devices. However, these methods require additional hardware equipment with high cost at the level of small and medium companies. This paper proposes a multi-factor authentication system that combines ease of use and low-cost factors. The system does not need any special settings or infrastructure. It relies on graphical passwords, so the user, in registration phase, chooses three images and memorizes them. In the login phase, the user needs only to choose the correct images that he considered during the registration process in a specific order. The proposed system overcomes many different security threats, such as key-loggers, screen capture attack or shoulder surfing. The proposed method was applied to 170 participants, 75% of them are males and 25% are females, classified according to their age, education level, web experience. One-third of them did not have sufficient knowledge about various security threats.
Human-robot Interaction Method Combining Human Pose Estimation and Motion Intention Recognition

Although human pose estimation technology based on RGB images is becoming more and more mature, most of the current mainstream methods rely on depth camera to obtain human joints information. These interaction frameworks are affected by the infrared detection distance so that they cannot well adapt to the interaction scene of different distance. Therefore, the purpose of this paper is to build a modular interactive framework based on RGB images, which aims to alleviate the problem of high dependence on depth camera and low adaptability to distance in the current human-robot interaction (HRI) framework based on human body by using advanced human pose estimation technology. To enhance the adaptability of the HRI framework to different distances, we adopt optical cameras instead of depth cameras as acquisition equipment. Firstly, the human joints information is extracted by a human pose estimation network. Then, a joints sequence filter is designed in the intermediate stage to reduce the influence of unreasonable skeletons on the interaction results. Finally, a human intention recognition model is built to recognize the human intention from reasonable joints information, and drive the robot to respond according to the predicted intention. The experimental results show that our interactive framework is more robust in the distance than the framework based on depth camera and is able to achieve effective interaction under different distances, illuminations, costumes, customers, and scenes.