OpenRaiser/Intern-Atlas huggingface dataset:
Reinforcement Learning via Fenchel-Rockafellar Duality
Venue: arXiv cs.LG 2020
Year: 2020

We review basic concepts of convex duality, focusing on the very general and supremely useful Fenchel-Rockafellar duality. We summarize how this duality may be applied to a variety of reinforcement learning (RL) settings, including policy evaluation or optimization, online or offline learning, and discounted or undiscounted rewards. The derivations yield a number of intriguing results, including the ability to perform policy evaluation and on-policy policy gradient with behavior-agnostic offline data and methods to learn a policy via max-likelihood optimization. Although many of these results have appeared previously in various forms, we provide a unified treatment and perspective on these results, which we hope will enable researchers to better use and apply the tools of convex duality to make further progress in RL.
Attention over Parameters for Dialogue Systems
Venue: arXiv cs.CL 2020
Year: 2020

Dialogue systems require a great deal of different but complementary expertise to assist, inform, and entertain humans. For example, different domains (e.g., restaurant reservation, train ticket booking) of goal-oriented dialogue systems can be viewed as different skills, and so does ordinary chatting abilities of chit-chat dialogue systems. In this paper, we propose to learn a dialogue system that independently parameterizes different dialogue skills, and learns to select and combine each of them through Attention over Parameters (AoP). The experimental results show that this approach achieves competitive performance on a combined dataset of MultiWOZ, In-Car Assistant, and Persona-Chat. Finally, we demonstrate that each dialogue skill is effectively learned and can be combined with other skills to produce selective responses.
Behavioural Theory of Reflective Algorithms I: Reflective Sequential Algorithms
Venue: arXiv cs.LO 2020
Year: 2020

We develop a behavioural theory of reflective sequential algorithms (RSAs), i.e. sequential algorithms that can modify their own behaviour. The theory comprises a set of language-independent postulates defining the class of RSAs, an abstract machine model, and the proof that all RSAs are captured by this machine model. As in Gurevich's behavioural theory for sequential algorithms RSAs are sequential-time, bounded parallel algorithms, where the bound depends on the algorithm only and not on the input. Different from the class of sequential algorithms every state of an RSA includes a representation of the algorithm in that state, thus enabling linguistic reflection. Bounded exploration is preserved using terms as values. The model of reflective sequential abstract state machines (rsASMs) extends sequential ASMs using extended states that include an updatable representation of the main ASM rule to be executed by the machine in that state. Updates to the representation of ASM signatures and rules are realised by means of a sophisticated tree algebra.
Discovering Nonlinear Relations with Minimum Predictive Information Regularization
Venue: arXiv cs.LG 2020
Year: 2020

Identifying the underlying directional relations from observational time series with nonlinear interactions and complex relational structures is key to a wide range of applications, yet remains a hard problem. In this work, we introduce a novel minimum predictive information regularization method to infer directional relations from time series, allowing deep learning models to discover nonlinear relations. Our method substantially outperforms other methods for learning nonlinear relations in synthetic datasets, and discovers the directional relations in a video game environment and a heart-rate vs. breath-rate dataset.
IRS-based Wireless Jamming Attacks: When Jammers can Attack without Power
Venue: arXiv cs.IT 2020
Year: 2020

This paper proposes to use Intelligent Reflecting Surface (IRS) as a green jammer to attack a legitimate communication without using any internal energy to generate jamming signals. In particular, the IRS is used to intelligently reflect the signals from the legitimate transmitter to the legitimate receiver (LR) to guarantee that the received signals from direct and reflecting links can be added destructively, which thus diminishes the Signal-to-Interference-plus-Noise Ratio (SINR) at the LR. To minimize the received signal power at the LR, we consider the joint optimization of magnitudes of reflection coefficients and discrete phase shifts at the IRS. Based on the block coordinate descent, semidefinite relaxation, and Gaussian randomization techniques, the solution can be obtained efficiently. Through simulation results, we show that by using the IRS-based jammer, we can reduce the signal power received at the LR by up to 99\%. Interestingly, the performance of the proposed IRS-based jammer is even better than that of the conventional active jamming attacks in some scenarios.
Fast and robust multiplane single molecule localization microscopy using deep neural network
Venue: arXiv cs.CV 2020
Year: 2020

Single molecule localization microscopy is widely used in biological research for measuring the nanostructures of samples smaller than the diffraction limit. This study uses multifocal plane microscopy and addresses the 3D single molecule localization problem, where lateral and axial locations of molecules are estimated. However, when we multifocal plane microscopy is used, the estimation accuracy of 3D localization is easily deteriorated by the small lateral drifts of camera positions. We formulate a 3D molecule localization problem along with the estimation of the lateral drifts as a compressed sensing problem, A deep neural network was applied to accurately and efficiently solve this problem. The proposed method is robust to the lateral drifts and achieves an accuracy of 20 nm laterally and 50 nm axially without an explicit drift correction.
Scalable Hybrid HMM with Gaussian Process Emission for Sequential Time-series Data Clustering
Venue: arXiv cs.LG 2020
Year: 2020

Hidden Markov Model (HMM) combined with Gaussian Process (GP) emission can be effectively used to estimate the hidden state with a sequence of complex input-output relational observations. Especially when the spectral mixture (SM) kernel is used for GP emission, we call this model as a hybrid HMM-GPSM. This model can effectively model the sequence of time-series data. However, because of a large number of parameters for the SM kernel, this model can not effectively be trained with a large volume of data having (1) long sequence for state transition and 2) a large number of time-series dataset in each sequence. This paper proposes a scalable learning method for HMM-GPSM. To effectively train the model with a long sequence, the proposed method employs a Stochastic Variational Inference (SVI) approach. Also, to effectively process a large number of data point each time-series data, we approximate the SM kernel using Reparametrized Random Fourier Feature (R-RFF). The combination of these two techniques significantly reduces the training time. We validate the proposed learning method in terms of its hidden-sate estimation accuracy and computation time using large-scale synthetic and real data sets with missing values.
Efficient ML Direction of Arrival Estimation assuming Unknown Sensor Noise Powers
Venue: arXiv cs.IT 2020
Year: 2020

This paper presents an efficient method for computing maximum likelihood (ML) direction of arrival (DOA) estimates assuming unknown sensor noise powers. The method combines efficient Alternate Projection (AP) procedures with Newton iterations. The efficiency of the method lies in the fact that all its intermediate steps have low complexity. The main contribution of this paper is the method's last step, in which a concentrated cost function is maximized in both the DOAs and noise powers in a few iterations through a Newton procedure. This step has low complexity because it employs closed-form expressions of the cost function's gradients and Hessians, which are presented in the paper. The method's total computational burden is of just a few mega-flops in typical cases. We present the method for the deterministic and stochastic ML estimators. An analysis of the deterministic ML cost function's gradient reveals an unexpected drawback of its associated estimator: if the noise powers are unknown, then it is either degenerate or inconsistent. The root-mean-square (RMS) error performance and computational burden of the method are assessed numerically.
Softmax-based Classification is k-means Clustering: Formal Proof, Consequences for Adversarial Attacks, and Improvement through Centroid Based Tailoring
Venue: arXiv cs.LG 2020
Year: 2020

We formally prove the connection between k-means clustering and the predictions of neural networks based on the softmax activation layer. In existing work, this connection has been analyzed empirically, but it has never before been mathematically derived. The softmax function partitions the transformed input space into cones, each of which encompasses a class. This is equivalent to putting a number of centroids in this transformed space at equal distance from the origin, and k-means clustering the data points by proximity to these centroids. Softmax only cares in which cone a data point falls, and not how far from the centroid it is within that cone. We formally prove that networks with a small Lipschitz modulus (which corresponds to a low susceptibility to adversarial attacks) map data points closer to the cluster centroids, which results in a mapping to a k-means-friendly space. To leverage this knowledge, we propose Centroid Based Tailoring as an alternative to the softmax function in the last layer of a neural network. The resulting Gauss network has similar predictive accuracy as traditional networks, but is less susceptible to one-pixel attacks; while the main contribution of this paper is theoretical in nature, the Gauss network contributes empirical auxiliary benefits.
Universal Wait-Free Memory Reclamation
Venue: arXiv cs.DC 2020
Year: 2020

In this paper, we present a universal memory reclamation scheme, Wait-Free Eras (WFE), for deleted memory blocks in wait-free concurrent data structures. WFE's key innovation is that it is completely wait-free. Although some prior techniques provide similar guarantees for certain data structures, they lack support for arbitrary wait-free data structures. Consequently, developers are typically forced to marry their wait-free data structures with lock-free Hazard Pointers or (potentially blocking) epoch-based memory reclamation. Since both these schemes provide weaker progress guarantees, they essentially forfeit the strong progress guarantee of wait-free data structures. Though making the original Hazard Pointers scheme or epoch-based reclamation completely wait-free seems infeasible, we achieved this goal with a more recent, (lock-free) Hazard Eras scheme, which we extend to guarantee wait-freedom. As this extension is non-trivial, we discuss all challenges pertaining to the construction of universal wait-free memory reclamation. WFE is implementable on ubiquitous x86_64 and AArch64 (ARM) architectures. Its API is mostly compatible with Hazard Pointers, which allows easy transitioning of existing data structures into WFE. Our experimental evaluations show that WFE's performance is close to epoch-based reclamation and almost matches the original Hazard Eras scheme, while providing the stronger wait-free progress guarantee.
SUR-FeatNet: Predicting the Satisfied User Ratio Curvefor Image Compression with Deep Feature Learning
Venue: arXiv cs.MM 2020
Year: 2020

The satisfied user ratio (SUR) curve for a lossy image compression scheme, e.g., JPEG, characterizes the complementary cumulative distribution function of the just noticeable difference (JND), the smallest distortion level that can be perceived by a subject when a reference image is compared to a distorted one. A sequence of JNDs can be defined with a suitable successive choice of reference images. We propose the first deep learning approach to predict SUR curves. We show how to apply maximum likelihood estimation and the Anderson-Darling test to select a suitable parametric model for the distribution function. We then use deep feature learning to predict samples of the SUR curve and apply the method of least squares to fit the parametric model to the predicted samples. Our deep learning approach relies on a siamese convolutional neural network, transfer learning, and deep feature learning, using pairs consisting of a reference image and a compressed image for training. Experiments on the MCL-JCI dataset showed state-of-the-art performance. For example, the mean Bhattacharyya distances between the predicted and ground truth first, second, and third JND distributions were 0.0810, 0.0702, and 0.0522, respectively, and the corresponding average absolute differences of the peak signal-to-noise ratio at a median of the first JND distribution were 0.58, 0.69, and 0.58 dB. Further experiments on the JND-Pano dataset showed that the method transfers well to high resolution panoramic images viewed on head-mounted displays.
State Transition Modeling of the Smoking Behavior using LSTM Recurrent Neural Networks
Venue: arXiv cs.CV 2020
Year: 2020

The use of sensors has pervaded everyday life in several applications including human activity monitoring, healthcare, and social networks. In this study, we focus on the use of smartwatch sensors to recognize smoking activity. More specifically, we have reformulated the previous work in detection of smoking to include in-context recognition of smoking. Our presented reformulation of the smoking gesture as a state-transition model that consists of the mini-gestures hand-to-lip, hand-on-lip, and hand-off-lip, has demonstrated improvement in detection rates nearing 100% using conventional neural networks. In addition, we have begun the utilization of Long-Short-Term Memory (LSTM) neural networks to allow for in-context detection of gestures with accuracy nearing 97%.
Self learning robot using real-time neural networks
Venue: arXiv cs.RO 2020
Year: 2020

With the advancements in high volume, low precision computational technology and applied research on cognitive artificially intelligent heuristic systems, machine learning solutions through neural networks with real-time learning has seen an immense interest in the research community as well the industry. This paper involves research, development and experimental analysis of a neural network implemented on a robot with an arm through which evolves to learn to walk in a straight line or as required. The neural network learns using the algorithms of Gradient Descent and Backpropagation. Both the implementation and training of the neural network is done locally on the robot on a raspberry pi 3 so that its learning process is completely independent. The neural network is first tested on a custom simulator developed on MATLAB and then implemented on the raspberry computer. Data at each generation of the evolving network is stored, and analysis both mathematical and graphical is done on the data. Impact of factors like the learning rate and error tolerance on the learning process and final output is analyzed.
PaRoT: A Practical Framework for Robust Deep Neural Network Training
Venue: arXiv cs.LG 2020
Year: 2020

Deep Neural Networks (DNNs) are finding important applications in safety-critical systems such as Autonomous Vehicles (AVs), where perceiving the environment correctly and robustly is necessary for safe operation. Raising unique challenges for assurance due to their black-box nature, DNNs pose a fundamental problem for regulatory acceptance of these types of systems. Robust training --- training to minimize excessive sensitivity to small changes in input --- has emerged as one promising technique to address this challenge. However, existing robust training tools are inconvenient to use or apply to existing codebases and models: they typically only support a small subset of model elements and require users to extensively rewrite the training code. In this paper we introduce a novel framework, PaRoT, developed on the popular TensorFlow platform, that greatly reduces the barrier to entry. Our framework enables robust training to be performed on arbitrary DNNs without any rewrites to the model. We demonstrate that our framework's performance is comparable to prior art, and exemplify its ease of use on off-the-shelf, trained models and its testing capabilities on a real-world industrial application: a traffic light detection network.
Data Structure Primitives on Persistent Memory: An Evaluation
Venue: arXiv cs.DB 2020
Year: 2020

Persistent Memory (PMem), as already available, e.g., with Intel Optane DC Persistent Memory, represents a very promising, next-generation memory solution with a significant impact on database architectures. Several data structures for this new technology and its properties have already been proposed. However, primarily only complete structures are presented and evaluated. Thus, the implications of the individual ideas and PMem features are concealed. Therefore, in this paper, we disassemble the structures presented so far, identify their underlying design primitives, and assign them to appropriate design goals regarding PMem. As a result of our comprehensive experiments on real PM hardware, we can reveal the trade-offs of the primitives for various access patterns. This allowed us to pinpoint their best use cases as well as vulnerabilities. Besides our general insights regarding PMem-based data structure design, we also discovered new combinations not examined in the literature so far.
Plunge into the Underworld: A Survey on Emergence of Darknet
Venue: arXiv cs.CR 2020
Year: 2020

The availability of sophisticated technologies and methods of perpetrating criminogenic activities in the cyberspace is a pertinent societal problem. Darknet is an encrypted network technology that uses the internet infrastructure and can only be accessed using special network configuration and software tools to access its contents which are not indexed by search engines. Over the years darknets traditionally are used for criminogenic activities and famously acclaimed to promote cybercrime, procurements of illegal drugs, arms deals, and cryptocurrency markets. In countries with oppressive regimes, censorship of digital communications, and strict policies prompted journalists and freedom fighters to seek freedom using darknet technologies anonymously while others simply exploit it for illegal activities. Recently, MIT's Lincoln Laboratory of Artificial Intelligence augmented a tool that can be used to expose illegal activities behind the darknet. We studied relevant literature reviews to help researchers to better understand the darknet technologies, identify future areas of research on the darknet and ultimately to optimize how data-driven insights can be utilized to support governmental agencies in unraveling the depths of darknet technologies. This paper focuses on the use of the internet for crimes, deanonymization of TOR-services, darknet a new digital street for illicit drugs, research questions and hypothesis to guide researchers in further studies. Finally, in this study, we propose a model to examine and investigate anonymous online illicit markets.
Examining Potential Usability and Health Beliefs Among Young Adults Using a Conversational Agent for HPV Vaccine Counseling
Venue: arXiv cs.HC 2020
Year: 2020

The human papillomavirus (HPV) vaccine is the most effective way to prevent HPV-related cancers. Integrating provider vaccine counseling is crucial to improving HPV vaccine completion rates. Automating the counseling experience through a conversational agent could help improve HPV vaccine coverage and reduce the burden of vaccine counseling for providers. In a previous study, we tested a simulated conversational agent that provided HPV vaccine counseling for parents using the Wizard of OZ protocol. In the current study, we assessed the conversational agent among young college adults (n=24), a population that may have missed the HPV vaccine during their adolescence when vaccination is recommended. We also administered surveys for system and voice usability, and for health beliefs concerning the HPV vaccine. Participants perceived the agent to have high usability that is slightly better or equivalent to other voice interactive interfaces, and there is some evidence that the agent impacted their beliefs concerning the harms, uncertainty, and risk denials for the HPV vaccine. Overall, this study demonstrates the potential for conversational agents to be an impactful tool for health promotion endeavors.
On Thompson Sampling for Smoother-than-Lipschitz Bandits
Venue: arXiv cs.LG 2020
Year: 2020

Thompson Sampling is a well established approach to bandit and reinforcement learning problems. However its use in continuum armed bandit problems has received relatively little attention. We provide the first bounds on the regret of Thompson Sampling for continuum armed bandits under weak conditions on the function class containing the true function and sub-exponential observation noise. Our bounds are realised by analysis of the eluder dimension, a recently proposed measure of the complexity of a function class, which has been demonstrated to be useful in bounding the Bayesian regret of Thompson Sampling for simpler bandit problems under sub-Gaussian observation noise. We derive a new bound on the eluder dimension for classes of functions with Lipschitz derivatives, and generalise previous analyses in multiple regards.
$μ$VulDeePecker: A Deep Learning-Based System for Multiclass Vulnerability Detection
Venue: arXiv cs.CR 2020
Year: 2020

Fine-grained software vulnerability detection is an important and challenging problem. Ideally, a detection system (or detector) not only should be able to detect whether or not a program contains vulnerabilities, but also should be able to pinpoint the type of a vulnerability in question. Existing vulnerability detection methods based on deep learning can detect the presence of vulnerabilities (i.e., addressing the binary classification or detection problem), but cannot pinpoint types of vulnerabilities (i.e., incapable of addressing multiclass classification). In this paper, we propose the first deep learning-based system for multiclass vulnerability detection, dubbed $μ$VulDeePecker. The key insight underlying $μ$VulDeePecker is the concept of code attention, which can capture information that can help pinpoint types of vulnerabilities, even when the samples are small. For this purpose, we create a dataset from scratch and use it to evaluate the effectiveness of $μ$VulDeePecker. Experimental results show that $μ$VulDeePecker is effective for multiclass vulnerability detection and that accommodating control-dependence (other than data-dependence) can lead to higher detection capabilities.