Reviewerly/exHarmony huggingface dataset:
DAKOTA: Sensor and Touch Screen-Based Continuous Authentication on a Mobile Banking Application

Authenticating a user in the right way is essential to IT systems, where the risks are becoming more and more complex. Especially in the mobile world, banking applications are among the most delicate systems requiring strict rules and regulations. Existing approaches often require point-of-entry authentication accompanied by a one-time password as a second-factor authentication. However, this requires active participation of the user and there is continuous authentication during a session. In this paper, we investigate whether it is possible to continuously authenticate users via behavioral biometrics with a certain performance on a mobile banking application. A currently used mobile banking application in Turkey is chosen as the case, and we developed a continuous authentication scheme, named DAKOTA, on top of this application. The DAKOTA system records data from the touch screen and the motion sensors on the phone to monitor and model the user's behavioral patterns. Forty-five participants completed the predefined banking transactions. This data is used to train seven different classification algorithms. The results reveal that binary-SVM with RBF kernel reaches the lowest error scores, 3.5% equal error rate (EER). Using the end-to-end DAKOTA system, we investigate the performance in real-time, both in terms of authentication accuracy and resource usage. We show that it does not bring extra overhead in terms of power and memory usage compared to the original banking application and we can achieve a 90% true positive recognition rate, on average.
Video Frame Interpolation via Generalized Deformable Convolution

Video frame interpolation aims at synthesizing intermediate frames from nearby source frames while maintaining spatial and temporal consistencies. The existing deep-learning-based video frame interpolation methods can be roughly divided into two categories: flow-based methods and kernel-based methods. The performance of flow-based methods is often jeopardized by the inaccuracy of flow map estimation due to oversimplified motion models, while that of kernel-based methods tends to be constrained by the rigidity of kernel shape. To address these performance-limiting issues, a novel mechanism named generalized deformable convolution is proposed, which can effectively learn motion information in a data-driven manner and freely select sampling points in space-time. We further develop a new video frame interpolation method based on this mechanism. Our extensive experiments demonstrate that the new method performs favorably against the state-of-the-art, especially when dealing with complex motions. Code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/zhshi0816/GDConvNet</uri> .
A Novel Visual Representation on Text Using Diverse Conditional GAN for Visual Recognition

Automatic image visual recognition can make full use of largely available images with text descriptions on social media platforms to build large-scale image labeled datasets. In this paper, we propose a novel visual text representation, named DG-VRT (Diverse GAN-Visual Representation on Text), which extracts visual features from synthetic images generated by a diverse conditional Generative Adversarial Network (DCGAN) on the text, for visual recognition. The DCGAN incorporates the current state-of-the-art text-to-image GANs and generates multiple synthetic images with various prior noises conditioned on a text. Then we extract deep visual features from the generated synthetic images to explore the underlying visual concepts and provide a visual transformation on text in feature space. Finally, we combine image-level visual features, text-level features and visual features based on synthetic images together to recognize the images, and we also extend the proposed work to semantic segmentation. We conduct extensive experiments on two benchmark datasets and the experimental results demonstrate the efficacy of our proposed representation on text for visual recognition.
Human action recognition in drone videos using a few aerial training examples

Drones are enabling new forms of human actions surveillance due to their low cost and fast mobility. However, using deep neural networks for automatic aerial action recognition is difficult due to the need for a large number of training aerial human action videos. Collecting a large number of human action aerial videos is costly, time-consuming, and difficult. In this paper, we explore two alternative data sources to improve aerial action classification when only a few training aerial examples are available. As a first data source, we resort to video games. We collect plenty of aerial game action videos using two gaming engines. For the second data source, we leverage conditional Wasserstein Generative Adversarial Networks to generate aerial features from ground videos. Given that both data sources have some limitations, e.g. game videos are biased towards specific actions categories (fighting, shooting, etc.,), and it is not easy to generate good discriminative GAN-generated features for all types of actions, we need to efficiently integrate two dataset sources with few available real aerial training videos. To address this challenge of the heterogeneous nature of the data, we propose to use a disjoint multitask learning framework. We feed the network with real and game, or real and GAN-generated data in an alternating fashion to obtain an improved action classifier. We validate the proposed approach on two aerial action datasets and demonstrate that features from aerial game videos and those generated from GAN can be extremely useful for an improved action recognition in real aerial videos when only a few real aerial training examples are available.
Differential phase shift quantum secret sharing using a twin field

Quantum secret sharing (QSS) is essential for multiparty quantum communication, which is one of cornerstones in the future quantum internet. However, a linear rate-distance limitation severely constrains the secure key rate and transmission distance of QSS. Here, we present a practical QSS protocol among three participants based on the differential phase shift scheme and twin field ideas for the solution of high-efficiency multiparty communication task. In contrast to a formerly proposed differential phase shift QSS protocol, our protocol can break the linear rate-distance bound, theoretically improving the secret key rate by three orders of magnitude in a 300-km-long fiber. Furthermore, the new protocol is secure against Trojan horse attacks that cannot be resisted by previous differential phase shift QSS.
Evaluating the utility of synthetic COVID-19 case data

Concerns about patient privacy have limited access to COVID-19 datasets. Data synthesis is one approach for making such data broadly available to the research community in a privacy protective manner.Evaluate the utility of synthetic data by comparing analysis results between real and synthetic data.A gradient boosted classification tree was built to predict death using Ontario's 90 514 COVID-19 case records linked with community comorbidity, demographic, and socioeconomic characteristics. Model accuracy and relationships were evaluated, as well as privacy risks. The same model was developed on a synthesized dataset and compared to one from the original data.The AUROC and AUPRC for the real data model were 0.945 [95% confidence interval (CI), 0.941-0.948] and 0.34 (95% CI, 0.313-0.368), respectively. The synthetic data model had AUROC and AUPRC of 0.94 (95% CI, 0.936-0.944) and 0.313 (95% CI, 0.286-0.342) with confidence interval overlap of 45.05% and 52.02% when compared with the real data. The most important predictors of death for the real and synthetic models were in descending order: age, days since January 1, 2020, type of exposure, and gender. The functional relationships were similar between the two data sets. Attribute disclosure risks were 0.0585, and membership disclosure risk was low.This synthetic dataset could be used as a proxy for the real dataset.
An anonymity communication protocol for security and privacy of clients in IoT-based mobile health transactions

Mobile Health (mHealth) technology facilitates patients’ care and decreases healthcare costs by remotely monitoring healthcare processes and medical data management. Nonetheless, there are many concerns regarding the security, privacy, and anonymity of the mHealth transactions. Conventionally, privacy includes anonymity, untraceability, unlinkability, unforgeability, and confidentiality. Transactions’ confidentiality can be easily protected using confidential transaction techniques. However, anonymity, untraceability, unlinkability, and unforgeability properties are quite challenging issues in healthcare transactions. This paper proposes a UDP-based anonymous protocol for mHealth transactions to protect the security of data and privacy of clients’ identities. In this work, we aim to investigate utilizing one of the best anonymity solutions, onion structure (onion encryption, onion routing, and onion network), with blockchain smart contracts upon the UDP platform to make a protocol for transferring data in IoT-based mHealth applications. In the proposed protocol, every malicious client who sends false data can be easily detected using an identity disclosure process. In this way, mHealth clients, without exchanging sensitive identities, can establish an authorized connection and be protected against different kinds of attacks. According to our experimental results and security/privacy proofs, the proposed protocol has an acceptable computational cost and security protections for IoT-based mHealth transactions.
These <i>do not</i> Look Like Those: An Interpretable Deep Learning Model for Image Recognition

Interpretation of the reasoning process of a prediction made by a deep learning model is always desired. However, when it comes to the predictions of a deep learning model that directly impacts on the lives of people then the interpretation becomes a necessity. In this paper, we introduce a deep learning model: negative-positive prototypical part network (NP-ProtoPNet). This model attempts to imitate human reasoning for image recognition while comparing the parts of a test image with the corresponding parts of the images from known classes. We demonstrate our model on the dataset of chest X-ray images of Covid-19 patients, pneumonia patients and normal people. The accuracy and precision that our model receives is on par with the best performing non-interpretable deep learning models.
Break the Loop: Gender Imbalance in Music Recommenders

As recommender systems play an important role in everyday life, there is an increasing pressure that such systems are fair. Besides serving diverse groups of users, recommenders need to represent and serve item providers fairly as well. In interviews with music artists, we identified that gender fairness is one of the artists' main concerns. They emphasized that female artists should be given more exposure in music recommendations. We analyze a widely-used collaborative filtering approach with two public datasets-enriched with gender information-to understand how this approach performs with respect to the artists' gender. To achieve gender balance, we propose a progressive re-ranking method that is based on the insights from the interviews. For the evaluation, we rely on a simulation of feedback loops and provide an in-depth analysis using state-of-the-art performance measures and metrics concerning gen-der fairness.
Resilience of Quantum Random Access Memory to Generic Noise

Quantum random access memory (QRAM)--memory which stores classical data but allows queries to be performed in superposition--is required for the implementation of numerous quantum algorithms. While naive implementations of QRAM are highly susceptible to decoherence and hence not scalable, it has been argued that the bucket brigade QRAM architecture [Giovannetti et al., Phys. Rev. Lett. 100 160501 (2008)] is highly resilient to noise, with the infidelity of a query scaling only logarithmically with the memory size. In prior analyses, however, this favorable scaling followed directly from the use of contrived noise models, thus leaving open the question of whether experimental implementations would actually enjoy the purported scaling advantage. In this work, we study the effects of decoherence on QRAM in full generality. Our main result is a proof that this favorable infidelity scaling holds for arbitrary error channels (including, e.g., depolarizing noise and coherent errors). Our proof identifies the origin of this noise resilience as the limited entanglement among the memory's components, and it also reveals that significant architectural simplifications can be made while preserving the noise resilience. We verify these results numerically using a novel classical algorithm for the efficient simulation of noisy QRAM circuits. Our findings indicate that QRAM can be implemented with existing hardware in realistically noisy devices, and that high-fidelity queries are possible without quantum error correction. Furthermore, we also prove that the benefits of the bucket-brigade architecture persist when quantum error correction is used, in which case the scheme offers improved hardware efficiency and resilience to logical errors.
A hybrid medical text classification framework: Integrating attentive rule construction and neural network

The main objective of this work is to improve the quality and transparency of the medical text classification solutions. Conventional text classification methods provide users with only a restricted mechanism (based on frequency) for selecting features. In this paper, a three-stage hybrid method combining the gated attention-based bi-directional Long Short-Term Memory (ABLSTM) and the regular expression based classifier is proposed for medical text classification tasks. The bi-directional Long Short-Term Memory (LSTM) architecture with an attention layer allows the network to weigh words according to their perceived importance and focus on crucial parts of a sentence. Feature words (or keywords) extracted by ABLSTM model are utilized to guide the regular expression rule construction. Our proposed approach leverages the advantages of both the interpretability of rule-based algorithms and the computational power of deep learning approaches for a production-ready scenario. Experimental results on real-world medical online query data clearly validate the superiority of our system in selecting domain-specific and topic-related features. Results show that the proposed approach achieves an accuracy of 0.89 and an F1-score of 0.92 respectively. Furthermore, our experimentation also illustrates the versatility of regular expressions as a user-level tool for focusing on desired patterns and providing interpretable solutions for human modification.
GraphIQA: Learning Distortion Graph Representations for Blind Image Quality Assessment

A good distortion representation is crucial for the success of deep blind image quality assessment (BIQA).However, most previous methods do not effectively model the relationship between distortions or the distribution of samples with same distortion type but different distortion levels.In this work, we start from the analysis of the relationship between perceptual image quality and distortion-related factors, such as distortion types and levels.Then, we propose a Distortion Graph Representation (DGR) learning framework for IQA, named GraphIQA, in which each distortion is represented as a graph, i.e., DGR.One can distinguish distortion types by learning the contrast relationship between these different DGRs, and infer the ranking distribution of samples from different levels in a DGR.Specifically, we develop two sub-networks to learn the DGRs: a) Type Discrimination Network (TDN) that aims to embed DGR into a compact code for better discriminating distortion types and learning the relationship between types; b) Fuzzy Prediction Network (FPN) that aims to extract the distributional characteristics of the samples in a DGR and predicts fuzzy degrees based on a Gaussian prior.Experiments show that our GraphIQA achieves the state-of-the-art performance on many benchmark datasets of both synthetic and authentic distortions.
Hand Gesture Recognition Based on Auto-Landmark Localization and Reweighted Genetic Algorithm for Healthcare Muscle Activities

Due to the constantly increasing demand for the automatic localization of landmarks in hand gesture recognition, there is a need for a more sustainable, intelligent, and reliable system for hand gesture recognition. The main purpose of this study was to develop an accurate hand gesture recognition system that is capable of error-free auto-landmark localization of any gesture dateable in an RGB image. In this paper, we propose a system based on landmark extraction from RGB images regardless of the environment. The extraction of gestures is performed via two methods, namely, fused and directional image methods. The fused method produced greater extracted gesture recognition accuracy. In the proposed system, hand gesture recognition (HGR) is done via several different methods, namely, (1) HGR via point-based features, which consist of (i) distance features, (ii) angular features, and (iii) geometric features; (2) HGR via full hand features, which are composed of (i) SONG mesh geometry and (ii) active model. To optimize these features, we applied gray wolf optimization. After optimization, a reweighted genetic algorithm was used for classification and gesture recognition. Experimentation was performed on five challenging datasets: Sign Word, Dexter1, Dexter + Object, STB, and NYU. Experimental results proved that auto landmark localization with the proposed feature extraction technique is an efficient approach towards developing a robust HGR system. The classification results of the reweighted genetic algorithm were compared with Artificial Neural Network (ANN) and decision tree. The developed system plays a significant role in healthcare muscle exercise.
Dynamic 3D scrambled image based RGB image encryption scheme using hyperchaotic system and DNA encoding

Hundreds of image encryption schemes based on the conventional 2D image space exist in the literature. Very few were built using the 3D image space. In this study, for better security, we present a new RGB (color image) encryption scheme based on the Dynamic 3D scrambled image (D3DSI), 5D multi-wing-hyperchaotic-system and DNA computing. After providing a color image as input, its three components are reshaped in a 1D array. These three components are concatenated to form a single 1D array. A 3D scrambled image with arbitrary dimensions is created. Pixels from the 1D array are shifted randomly to the different cells of the 3D scrambled image. Afterwards, for the realization of the diffusion effects, an XOR operation is performed between the key image and the 3D scrambled image. Next, the pixels data and the DNA key image are DNA-encoded. The pixels data in the form of DNA strands is once again scrambled using the same 3D scrambled image. Once again, the pixels data in DNA strands is XORed with the DNA-encoded key image. Lastly these DNA strands are transformed back to the decimal form. Out of this decimal form, three constituent color components, i.e., red, green and blue are separated and merged together to form the color cipher image. To obtain plaintext sensitivity, an image characteristic has been used to temper the one initial value of the chaotic map being used. The simulation results and the security analysis both portray better security, defiance to the varied threats and a real world applicability of the proposed image cipher.
FCMF: Federated collective matrix factorization for heterogeneous collaborative filtering

Protecting users’ privacy has drawn tremendous attention from the community of recommender systems, i.e., both the original data and the learned model parameters should not be exposed. Federated learning is an emerging and promising paradigm, where a server collects gradients from multiple distributed parts and then updates the model parameters with the aggregated gradients. However, there are some security issues neglected in existing works. For example, the server may infer the users’ rating behaviors on the items via the received gradients. In this paper, we focus on heterogeneous collaborative filtering (HCF) by exploiting users’ different types of feedback such as 5-star numerical ratings and like/dislike binary ratings in a privacy-aware manner. Specifically, we design a novel and generic federated matrix factorization algorithm for HCF, i.e., federated collective matrix factorization (FCMF). The main goal of our FCMF is to leverage the heterogeneous feedback data to accurately estimate users’ preferences on the premise of protecting users’ private information. Therefore, we keep the original rating data and the users’ latent feature vectors locally, and choose the low sensitive items’ latent vectors as a bridge for joint training. Furthermore, we use homomorphic encryption and differential privacy to ensure the security of both participants in collective training. To study the effectiveness of our FCMF, we conduct extensive empirical studies on four real-world datasets and find that our FCMF is equivalent to the centralized method that aggregates the heterogeneous data in one single place. Moreover, the introduction of homomorphic encryption and differential privacy do not affect the recommendation accuracy much.
Discriminative feature alignment: Improving transferability of unsupervised domain adaptation by Gaussian-guided latent alignment

In this paper, we focus on the unsupervised domain adaptation problem where an approximate inference model is to be learned from a labeled data domain and expected to generalize well to an unlabeled domain. The success of unsupervised domain adaptation largely relies on the cross-domain feature alignment. Previous work has attempted to directly align features by classifier-induced discrepancies. Nevertheless, a common feature space cannot always be learned via this direct feature alignment especially when large domain gaps exist. To solve this problem, we introduce a Gaussian-guided latent alignment approach to align the latent feature distributions of the two domains under the guidance of a prior. In such an indirect way, the distributions over the samples from the two domains will be constructed on a common feature space, i.e., the space of the prior, which promotes better feature alignment. To effectively align the target latent distribution with this prior distribution, we also propose a novel unpaired L1-distance by taking advantage of the formulation of the encoder-decoder. The extensive evaluations on nine benchmark datasets validate the superior knowledge transferability through outperforming state-of-the-art methods and the versatility of the proposed method by improving the existing work significantly.