OpenRaiser/Intern-Atlas huggingface dataset:
Deep Q-Network-based Adaptive Alert Threshold Selection Policy for Payment Fraud Systems in Retail Banking
Venue: arXiv cs.LG 2020
Year: 2020

Machine learning models have widely been used in fraud detection systems. Most of the research and development efforts have been concentrated on improving the performance of the fraud scoring models. Yet, the downstream fraud alert systems still have limited to no model adoption and rely on manual steps. Alert systems are pervasively used across all payment channels in retail banking and play an important role in the overall fraud detection process. Current fraud detection systems end up with large numbers of dropped alerts due to their inability to account for the alert processing capacity. Ideally, alert threshold selection enables the system to maximize the fraud detection while balancing the upstream fraud scores and the available bandwidth of the alert processing teams. However, in practice, fixed thresholds that are used for their simplicity do not have this ability. In this paper, we propose an enhanced threshold selection policy for fraud alert systems. The proposed approach formulates the threshold selection as a sequential decision making problem and uses Deep Q-Network based reinforcement learning. Experimental results show that this adaptive approach outperforms the current static solutions by reducing the fraud losses as well as improving the operational efficiency of the alert system.
Runtime vs Scheduler: Analyzing Dask's Overheads
Venue: arXiv cs.DC 2020
Year: 2020

Dask is a distributed task framework which is commonly used by data scientists to parallelize Python code on computing clusters with little programming effort. It uses a sophisticated work-stealing scheduler which has been hand-tuned to execute task graphs as efficiently as possible. But is scheduler optimization a worthwhile effort for Dask? Our paper shows on many real world task graphs that even a completely random scheduler is surprisingly competitive with its built-in scheduler and that the main bottleneck of Dask lies in its runtime overhead. We develop a drop-in replacement for the Dask central server written in Rust which is backwards compatible with existing Dask programs. Thanks to its efficient runtime, our server implementation is able to scale up to larger clusters than Dask and consistently outperforms it on a variety of task graphs, despite the fact that it uses a simpler scheduling algorithm.
Progressive Defense Against Adversarial Attacks for Deep Learning as a Service in Internet of Things
Venue: arXiv cs.CR 2020
Year: 2020

Nowadays, Deep Learning as a service can be deployed in Internet of Things (IoT) to provide smart services and sensor data processing. However, recent research has revealed that some Deep Neural Networks (DNN) can be easily misled by adding relatively small but adversarial perturbations to the input (e.g., pixel mutation in input images). One challenge in defending DNN against these attacks is to efficiently identifying and filtering out the adversarial pixels. The state-of-the-art defense strategies with good robustness often require additional model training for specific attacks. To reduce the computational cost without loss of generality, we present a defense strategy called a progressive defense against adversarial attacks (PDAAA) for efficiently and effectively filtering out the adversarial pixel mutations, which could mislead the neural network towards erroneous outputs, without a-priori knowledge about the attack type. We evaluated our progressive defense strategy against various attack methods on two well-known datasets. The result shows it outperforms the state-of-the-art while reducing the cost of model training by 50% on average.
Improving the Iterative Closest Point Algorithm using Lie Algebra
Venue: arXiv cs.RO 2020
Year: 2020

Mapping algorithms that rely on registering point clouds inevitably suffer from local drift, both in localization and in the built map. Applications that require accurate maps, such as environmental monitoring, benefit from additional sensor modalities that reduce such drift. In our work, we target the family of mappers based on the Iterative Closest Point (ICP) algorithm which use additional orientation sources such as the Inertial Measurement Unit (IMU). We introduce a new angular penalty term derived from Lie algebra. Our formulation avoids the need for tuning arbitrary parameters. Orientation covariance is used instead, and the resulting error term fits into the ICP cost function minimization problem. Experiments performed on our own real-world data and on the KITTI dataset show consistent behavior while suppressing the effect of outlying IMU measurements. We further discuss promising experiments, which should lead to optimal combination of all error terms in the ICP cost function minimization problem, allowing us to smoothly combine the geometric and inertial information provided by robot sensors.
Decentralized Deep Learning using Momentum-Accelerated Consensus
Venue: arXiv cs.LG 2020
Year: 2020

We consider the problem of decentralized deep learning where multiple agents collaborate to learn from a distributed dataset. While there exist several decentralized deep learning approaches, the majority consider a central parameter-server topology for aggregating the model parameters from the agents. However, such a topology may be inapplicable in networked systems such as ad-hoc mobile networks, field robotics, and power network systems where direct communication with the central parameter server may be inefficient. In this context, we propose and analyze a novel decentralized deep learning algorithm where the agents interact over a fixed communication topology (without a central server). Our algorithm is based on the heavy-ball acceleration method used in gradient-based optimization. We propose a novel consensus protocol where each agent shares with its neighbors its model parameters as well as gradient-momentum values during the optimization process. We consider both strongly convex and non-convex objective functions and theoretically analyze our algorithm's performance. We present several empirical comparisons with competing decentralized learning methods to demonstrate the efficacy of our approach under different communication topologies.
Robustness-aware 2-bit quantization with real-time performance for neural network
Venue: arXiv cs.LG 2020
Year: 2020

Quantized neural network (NN) with a reduced bit precision is an effective solution to reduces the computational and memory resource requirements and plays a vital role in machine learning. However, it is still challenging to avoid the significant accuracy degradation due to its numerical approximation and lower redundancy. In this paper, a novel robustness-aware 2-bit quantization scheme is proposed for NN base on binary NN and generative adversarial network(GAN), witch improves the performance by enriching the information of binary NN, efficiently extract the structural information and considering the robustness of the quantized NN. Specifically, using shift addition operation to replace the multiply-accumulate in the quantization process witch can effectively speed the NN. Meanwhile, a structural loss between the original NN and quantized NN is proposed to such that the structural information of data is preserved after quantization. The structural information learned from NN not only plays an important role in improving the performance but also allows for further fine tuning of the quantization network by applying the Lipschitz constraint to the structural loss. In addition, we also for the first time take the robustness of the quantized NN into consideration and propose a non-sensitive perturbation loss function by introducing an extraneous term of spectral norm. The experiments are conducted on CIFAR-10 and ImageNet datasets with popular NN( such as MoblieNetV2, SqueezeNet, ResNet20, etc). The experimental results show that the proposed algorithm is more competitive under 2-bit-precision than the state-of-the-art quantization methods. Meanwhile, the experimental results also demonstrate that the proposed method is robust under the FGSM adversarial samples attack.
Document-Level Relation Extraction with Adaptive Thresholding and Localized Context Pooling
Venue: arXiv cs.CL 2020
Year: 2020

Document-level relation extraction (RE) poses new challenges compared to its sentence-level counterpart. One document commonly contains multiple entity pairs, and one entity pair occurs multiple times in the document associated with multiple possible relations. In this paper, we propose two novel techniques, adaptive thresholding and localized context pooling, to solve the multi-label and multi-entity problems. The adaptive thresholding replaces the global threshold for multi-label classification in the prior work with a learnable entities-dependent threshold. The localized context pooling directly transfers attention from pre-trained language models to locate relevant context that is useful to decide the relation. We experiment on three document-level RE benchmark datasets: DocRED, a recently released large-scale RE dataset, and two datasets CDRand GDA in the biomedical domain. Our ATLOP (Adaptive Thresholding and Localized cOntext Pooling) model achieves an F1 score of 63.4, and also significantly outperforms existing models on both CDR and GDA.
Mixed-Precision Embedding Using a Cache
Venue: arXiv cs.LG 2020
Year: 2020

In recommendation systems, practitioners observed that increase in the number of embedding tables and their sizes often leads to significant improvement in model performances. Given this and the business importance of these models to major internet companies, embedding tables for personalization tasks have grown to terabyte scale and continue to grow at a significant rate. Meanwhile, these large-scale models are often trained with GPUs where high-performance memory is a scarce resource, thus motivating numerous work on embedding table compression during training. We propose a novel change to embedding tables using a cache memory architecture, where the majority of rows in an embedding is trained in low precision, and the most frequently or recently accessed rows cached and trained in full precision. The proposed architectural change works in conjunction with standard precision reduction and computer arithmetic techniques such as quantization and stochastic rounding. For an open source deep learning recommendation model (DLRM) running with Criteo-Kaggle dataset, we achieve 3x memory reduction with INT8 precision embedding tables and full-precision cache whose size are 5% of the embedding tables, while maintaining accuracy. For an industrial scale model and dataset, we achieve even higher >7x memory reduction with INT4 precision and cache size 1% of embedding tables, while maintaining accuracy, and 16% end-to-end training speedup by reducing GPU-to-host data transfers.
Full-Duplex and Dynamic-TDD: Pushing the Limits of Spectrum Reuse in Multi-Cell Communications
Venue: arXiv cs.IT 2020
Year: 2020

Although in cellular networks full-duplex and dynamic time-division duplexing promise increased spectrum efficiency, their potential is so far challenged by increased interference. While previous studies have shown that self-interference can be suppressed to a sufficient level, we show that the cross-link interference for both duplexing modes, especially from base station to base station, is the remaining challenge in multi-cell networks, restricting the uplink performance. Using beamforming techniques of low-complexity, we show that this interference can be mitigated, and that full-duplex and dynamic time-division duplexing can substantially increase the capacity of multi-cell networks. Our results suggest that if we can control the cross link interference in full-duplex, then we can almost double the multi cell network capacity as well as user throughput. Therefore, the techniques in this paper have the potentiality to enable a smooth introduction of full-duplex into cellular systems.
Serverless Containers -- rising viable approach to Scientific Workflows
Venue: arXiv cs.DC 2020
Year: 2020

Increasing popularity of the serverless computing approach has led to the emergence of new cloud infrastructures working in Container-as-a-Service (CaaS) model like AWS Fargate, Google Cloud Run, or Azure Container Instances. They introduce an innovative approach to running cloud containers where developers are freed from managing underlying resources. In this paper, we focus on evaluating capabilities of elastic containers and their usefulness for scientific computing in the scientific workflow paradigm using AWS Fargate and Google Cloud Run infrastructures. For experimental evaluation of our approach, we extended HyperFlow engine to support these CaaS platform, together with adapting four real-world scientific workflows composed of several dozen to over a hundred of tasks organized into a dependency graph. We used these workflows to create cost-performance benchmarks and flow execution plots, measuring delays, elasticity, and scalability. The experiments proved that serverless containers can be successfully applied for scientific workflows. Also, the results allow us to gain insights on specific advantages and limits of such platforms.
Probing and Fine-tuning Reading Comprehension Models for Few-shot Event Extraction
Venue: arXiv cs.CL 2020
Year: 2020

We study the problem of event extraction from text data, which requires both detecting target event types and their arguments. Typically, both the event detection and argument detection subtasks are formulated as supervised sequence labeling problems. We argue that the event extraction models so trained are inherently label-hungry, and can generalize poorly across domains and text genres.We propose a reading comprehension framework for event extraction.Specifically, we formulate event detection as a textual entailment prediction problem, and argument detection as a question answer-ing problem. By constructing proper query templates, our approach can effectively distill rich knowledge about tasks and label semantics from pretrained reading comprehension models. Moreover, our model can be fine-tuned with a small amount of data to boost its performance. Our experiment results show that our method performs strongly for zero-shot and few-shot event extraction, and it achieves state-of-the-art performance on the ACE 2005 benchmark when trained with full supervision.
NU-GAN: High resolution neural upsampling with GAN
Venue: arXiv cs.SD 2020
Year: 2020

In this paper, we propose NU-GAN, a new method for resampling audio from lower to higher sampling rates (upsampling). Audio upsampling is an important problem since productionizing generative speech technology requires operating at high sampling rates. Such applications use audio at a resolution of 44.1 kHz or 48 kHz, whereas current speech synthesis methods are equipped to handle a maximum of 24 kHz resolution. NU-GAN takes a leap towards solving audio upsampling as a separate component in the text-to-speech (TTS) pipeline by leveraging techniques for audio generation using GANs. ABX preference tests indicate that our NU-GAN resampler is capable of resampling 22 kHz to 44.1 kHz audio that is distinguishable from original audio only 7.4% higher than random chance for single speaker dataset, and 10.8% higher than chance for multi-speaker dataset.
Sample Efficient Reinforcement Learning with REINFORCE
Venue: arXiv cs.LG 2020
Year: 2020

Policy gradient methods are among the most effective methods for large-scale reinforcement learning, and their empirical success has prompted several works that develop the foundation of their global convergence theory. However, prior works have either required exact gradients or state-action visitation measure based mini-batch stochastic gradients with a diverging batch size, which limit their applicability in practical scenarios. In this paper, we consider classical policy gradient methods that compute an approximate gradient with a single trajectory or a fixed size mini-batch of trajectories under soft-max parametrization and log-barrier regularization, along with the widely-used REINFORCE gradient estimation procedure. By controlling the number of "bad" episodes and resorting to the classical doubling trick, we establish an anytime sub-linear high probability regret bound as well as almost sure global convergence of the average regret with an asymptotically sub-linear rate. These provide the first set of global convergence and sample efficiency results for the well-known REINFORCE algorithm and contribute to a better understanding of its performance in practice.
Learning Occupancy Function from Point Clouds for Surface Reconstruction
Venue: arXiv cs.CV 2020
Year: 2020

Implicit function based surface reconstruction has been studied for a long time to recover 3D shapes from point clouds sampled from surfaces. Recently, Signed Distance Functions (SDFs) and Occupany Functions are adopted in learning-based shape reconstruction methods as implicit 3D shape representation. This paper proposes a novel method for learning occupancy functions from sparse point clouds and achieves better performance on challenging surface reconstruction tasks. Unlike the previous methods, which predict point occupancy with fully-connected multi-layer networks, we adapt the point cloud deep learning architecture, Point Convolution Neural Network (PCNN), to build our learning model. Specifically, we create a sampling operator and insert it into PCNN to continuously sample the feature space at the points where occupancy states need to be predicted. This method natively obtains point cloud data's geometric nature, and it's invariant to point permutation. Our occupancy function learning can be easily fit into procedures of point cloud up-sampling and surface reconstruction. Our experiments show state-of-the-art performance for reconstructing With ShapeNet dataset and demonstrate this method's well-generalization by testing it with McGill 3D dataset \cite{siddiqi2008retrieving}. Moreover, we find the learned occupancy function is relatively more rotation invariant than previous shape learning methods.
A Hybrid Approach to Coded Compressed Sensing where Coupling Takes Place via the Outer Code
Venue: arXiv cs.IT 2020
Year: 2020

This article seeks to advance coded compressed sensing (CCS) as a practical scheme for unsourced random access. The original CCS algorithm features a concatenated structure where an inner code is tasked with support recovery, and an outer tree code conducts message disambiguation. Recently, a link between CCS and sparse regression codes was established, leading to the application of approximate message passing (AMP) to CCS. This connection was subsequently strengthened by integrating AMP and belief propagation on the outer code through a dynamic denoiser. Along these lines, this work shows how block diagonal sensing matrices akin to those used in traditional CCS, together with the aforementioned dynamic denoiser, form an effective means to get good performance at low-complexity. This novel architecture can be used to scale this scheme to dimensions that were previously impractical. Findings are supported by numerical simulations.
Fusing Keys for Secret Communications: Towards Information-Theoretic Security
Venue: arXiv cs.CR 2020
Year: 2020

Modern cryptography is essential to communication and information security for performing all kinds of security actions, such as encryption, authentication, and signature. However, the exposure possibility of keys poses a great threat to almost all modern cryptography. This article proposes a key-fusing framework, which enables a high resilience to key exposure by fusing multiple imperfect keys. The correctness of the scheme is strictly verified through a toy model that is general enough to abstract the physical-layer key generation (PLKG) mechanisms. Analysis and results demonstrate that the proposed scheme can dramatically reduce secret outage probability, so that key sources with even high exposure probability can be practically beneficial for actual secret communication. Our framework paves the way for achieving information-theoretic security by integrating various key sources, such as physical layer key generation, lattice-based cryptography, and quantum cryptography.
Selection of the optimal embedding positions of digital audio watermarking in wavelet domain
Venue: arXiv cs.CR 2020
Year: 2020

This work studied embedding positions of digital audio watermarking in wavelet domain, to make beginners understand the nature of watermarking in a short time. Based on the theory of wavelet transform, this paper analyzed statistical distributions of each level after transformation and the features of watermark embedded in different transform levels. Through comparison and analysis, we found that watermark was suitable for embedding into the coefficients of the first four levels of wavelet transform. In current state-of-art approaches, the embedding algorithms were always to replace the coefficient values of the embedded positions. In contrast this paper proposed an embedding algorithm of selfadaptive interpolation to achieve a better imperceptibility. In order to reduce the computational complexity, we took a pseudo random sequence with a length of 31 bits as the watermark. In the experiments, watermark was embedded in different locations, including different transform levels, high-frequency coefficients and low-frequency coefficients, high-energy regions and low-frequency regions. Results showed that the imperceptibility was better than traditional embedding algorithms. The bit error rates of the extracted watermark were calculated and we analyzed the robustness and fragility of each embedded signal. At last we concluded the best embedding positions of watermark for different applications and our future work.
A Novel Variable Stiffness Soft Robotic Gripper
Venue: arXiv cs.RO 2020
Year: 2020

We propose a novel tri-fingered soft robotic gripper with decoupled stiffness and shape control capability for performing adaptive grasping with minimum system complexity. The proposed soft fingers adaptively conform to object shapes facilitating the handling of objects of different types, shapes, and sizes. Each soft gripper finger has an inextensible articulable backbone and is actuated by pneumatic muscles. We derive a kinematic model of the gripper and use an empirical approach to map input pressures to stiffness and bending deformation of fingers. We use these mappings to achieve decoupled stiffness and shape control. We conduct tests to quantify the ability to hold objects as the gripper changes orientation, the ability to maintain the grasping status as the gripper moves, and the amount of force required to release the object from the gripped fingers, respectively. The results validate the proposed gripper's performance and show how stiffness control can improve the grasping quality.
On Finite and Unrestricted Query Entailment beyond SQ with Number Restrictions on Transitive Roles
Venue: arXiv cs.LO 2020
Year: 2020

We study the description logic SQ with number restrictions applicable to transitive roles, extended with either nominals or inverse roles. We show tight 2EXPTIME upper bounds for unrestricted entailment of regular path queries for both extensions and finite entailment of positive existential queries for nominals. For inverses, we establish 2EXPTIME-completeness for unrestricted and finite entailment of instance queries (the latter under restriction to a single, transitive role).