OpenRaiser/Intern-Atlas huggingface dataset:
Composition Properties of Bayesian Differential Privacy
Venue: arXiv cs.CR 2019
Year: 2019

Differential privacy is a rigorous privacy standard that has been applied to a range of data analysis tasks. To broaden the application scenarios of differential privacy when data records have dependencies, the notion of Bayesian differential privacy has been recently proposed. However, it is unknown whether Bayesian differential privacy preserves three nice properties of differential privacy: sequential composability, parallel composability, and post-processing. In this paper, we provide an affirmative answer to this question; i.e., Bayesian differential privacy still have these properties. The idea behind sequential composability is that if we have $m$ algorithms $Y_1, Y_2, \ldots, Y_m$, where $Y_{\ell}$ is independently $ε_{\ell}$-Bayesian differential private for ${\ell}=1,2,\ldots,m$, then by feeding the result of $Y_1$ into $Y_2$, the result of $Y_2$ into $Y_3$, and so on, we will finally have an $\sum_{\ell=1}^m ε_{\ell}$-Bayesian differential private algorithm. For parallel composability, we consider the situation where a database is partitioned into $m$ disjoint subsets. The $\ell$-th subset is input to a Bayesian differential private algorithm $Y_{\ell}$, for ${\ell}=1,2,\ldots,m$. Then the parallel composition of $Y_1$, $Y_2$, $\ldots$, $Y_m$ will be $\max_{\ell=1}^m ε_{\ell}$-Bayesian differential private. The post-processing property means that a data analyst, without additional knowledge about the private database, cannot compute a function of the output of a Bayesian differential private algorithm and reduce its privacy guarantee.
Laplacian Smoothing Stochastic Gradient Markov Chain Monte Carlo
Venue: arXiv cs.LG 2019
Year: 2019

As an important Markov Chain Monte Carlo (MCMC) method, stochastic gradient Langevin dynamics (SGLD) algorithm has achieved great success in Bayesian learning and posterior sampling. However, SGLD typically suffers from slow convergence rate due to its large variance caused by the stochastic gradient. In order to alleviate these drawbacks, we leverage the recently developed Laplacian Smoothing (LS) technique and propose a Laplacian smoothing stochastic gradient Langevin dynamics (LS-SGLD) algorithm. We prove that for sampling from both log-concave and non-log-concave densities, LS-SGLD achieves strictly smaller discretization error in $2$-Wasserstein distance, although its mixing rate can be slightly slower. Experiments on both synthetic and real datasets verify our theoretical results, and demonstrate the superior performance of LS-SGLD on different machine learning tasks including posterior sampling, Bayesian logistic regression and training Bayesian convolutional neural networks. The code is available at \url{https://github.com/BaoWangMath/LS-MCMC}.
Scene Graph based Image Retrieval -- A case study on the CLEVR Dataset
Venue: arXiv cs.AI 2019
Year: 2019

With the prolification of multimodal interaction in various domains, recently there has been much interest in text based image retrieval in the computer vision community. However most of the state of the art techniques model this problem in a purely neural way, which makes it difficult to incorporate pragmatic strategies in searching a large scale catalog especially when the search requirements are insufficient and the model needs to resort to an interactive retrieval process through multiple iterations of question-answering. Motivated by this, we propose a neural-symbolic approach for a one-shot retrieval of images from a large scale catalog, given the caption description. To facilitate this, we represent the catalog and caption as scene-graphs and model the retrieval task as a learnable graph matching problem, trained end-to-end with a REINFORCE algorithm. Further, we briefly describe an extension of this pipeline to an iterative retrieval framework, based on interactive questioning and answering.
The Relationship between the Consistency of Users' Ratings and Recommendation Calibration
Venue: arXiv cs.IR 2019
Year: 2019

Fairness in recommender systems has recently received attention from researchers. Unfair recommendations have negative impact on the effectiveness of recommender systems as it may degrade users' satisfaction, loyalty, and at worst, it can lead to or perpetuate undesirable social dynamics. One of the factors that may impact fairness is calibration, the degree to which users' preferences on various item categories are reflected in the recommendations they receive. The ability of a recommendation algorithm for generating effective recommendations may depend on the meaningfulness of the input data and the amount of information available in users' profile. In this paper, we aim to explore the relationship between the consistency of users' ratings behavior and the degree of calibrated recommendations they receive. We conduct our analysis on different groups of users based on the consistency of their ratings. Our experimental results on a movie dataset and several recommendation algorithms show that there is a positive correlation between the consistency of users' ratings behavior and the degree of calibration in their recommendations, meaning that user groups with higher inconsistency in their ratings receive less calibrated recommendations.
Online Robustness Training for Deep Reinforcement Learning
Venue: arXiv cs.LG 2019
Year: 2019

In deep reinforcement learning (RL), adversarial attacks can trick an agent into unwanted states and disrupt training. We propose a system called Robust Student-DQN (RS-DQN), which permits online robustness training alongside Q networks, while preserving competitive performance. We show that RS-DQN can be combined with (i) state-of-the-art adversarial training and (ii) provably robust training to obtain an agent that is resilient to strong attacks during training and evaluation.
MRNN: A Multi-Resolution Neural Network with Duplex Attention for Document Retrieval in the Context of Question Answering
Venue: arXiv cs.IR 2019
Year: 2019

The primary goal of ad-hoc retrieval (document retrieval in the context of question answering) is to find relevant documents satisfied the information need posted in a natural language query. It requires a good understanding of the query and all the documents in a corpus, which is difficult because the meaning of natural language texts depends on the context, syntax,and semantics. Recently deep neural networks have been used to rank search results in response to a query. In this paper, we devise a multi-resolution neural network(MRNN) to leverage the whole hierarchy of representations for document retrieval. The proposed MRNN model is capable of deriving a representation that integrates representations of different levels of abstraction from all the layers of the learned hierarchical representation.Moreover, a duplex attention component is designed to refinethe multi-resolution representation so that an optimal contextfor matching the query and document can be determined. More specifically, the first attention mechanism determines optimal context from the learned multi-resolution representation for the query and document. The latter attention mechanism aims to fine-tune the representation so that the query and the relevant document are closer in proximity. The empirical study shows that MRNN with the duplex attention is significantly superior to existing models used for ad-hoc retrieval on benchmark datasets including SQuAD, WikiQA, QUASAR, and TrecQA.
Sentiment analysis model for Twitter data in Polish language
Venue: arXiv cs.CL 2019
Year: 2019

Text mining analysis of tweets gathered during Polish presidential election on May 10th, 2015. The project included implementation of engine to retrieve information from Twitter, building document corpora, corpora cleaning, and creating Term-Document Matrix. Each tweet from the text corpora was assigned a category based on its sentiment score. The score was calculated using the number of positive and/or negative emoticons and Polish words in each document. The result data set was used to train and test four machine learning classifiers, to select these providing most accurate automatic tweet classification results. The Naive Bayes and Maximum Entropy algorithms achieved the best accuracy of respectively 71.76% and 77.32%. All implementation tasks were completed using R programming language.
Geono-Cluster: Interactive Visual Cluster Analysis for Biologists
Venue: arXiv cs.HC 2019
Year: 2019

Biologists often perform clustering analysis to derive meaningful patterns, relationships, and structures from data instances and attributes. Though clustering plays a pivotal role in biologists' data exploration, it takes non-trivial efforts for biologists to find the best grouping in their data using existing tools. Visual cluster analysis is currently performed either programmatically or through menus and dialogues in many tools, which require parameter adjustments over several steps of trial-and-error. In this paper, we introduce Geono-Cluster, a novel visual analysis tool designed to support cluster analysis for biologists who do not have formal data science training. Geono-Cluster enables biologists to apply their domain expertise into clustering results by visually demonstrating how their expected clustering outputs should look like with a small sample of data instances. The system then predicts users' intentions and generates potential clustering results. Our study follows the design study protocol to derive biologists' tasks and requirements, design the system, and evaluate the system with experts on their own dataset. Results of our study with six biologists provide initial evidence that Geono-Cluster enables biologists to create, refine, and evaluate clustering results to effectively analyze their data and gain data-driven insights. At the end, we discuss lessons learned and the implications of our study.
Multiple Futures Prediction
Venue: arXiv cs.LG 2019
Year: 2019

Temporal prediction is critical for making intelligent and robust decisions in complex dynamic environments. Motion prediction needs to model the inherently uncertain future which often contains multiple potential outcomes, due to multi-agent interactions and the latent goals of others. Towards these goals, we introduce a probabilistic framework that efficiently learns latent variables to jointly model the multi-step future motions of agents in a scene. Our framework is data-driven and learns semantically meaningful latent variables to represent the multimodal future, without requiring explicit labels. Using a dynamic attention-based state encoder, we learn to encode the past as well as the future interactions among agents, efficiently scaling to any number of agents. Finally, our model can be used for planning via computing a conditional probability density over the trajectories of other agents given a hypothetical rollout of the 'self' agent. We demonstrate our algorithms by predicting vehicle trajectories of both simulated and real data, demonstrating the state-of-the-art results on several vehicle trajectory datasets.
Polar Coding and Random Spreading for Unsourced Multiple Access
Venue: arXiv cs.IT 2019
Year: 2019

This article presents a novel transmission scheme for the unsourced, uncoordinated Gaussian multiple access problem. The proposed scheme leverages notions from single-user coding, random spreading, minimum-mean squared error (MMSE) estimation, and successive interference cancellation. Specifically, every message is split into two parts: the first fragment serves as the argument to an injective function that determines which spreading sequence should be employed, whereas the second component of the message is encoded using a polar code. The latter coded bits are then spread using the sequence determined during the first step. The ensuing signal is transmitted through a Gaussian multiple-access channel (GMAC). On the receiver side, active sequences are detected using a correlation-based energy detector, thereby simultaneously recovering individual signature sequences and their generating information bits in the form of preimages of the sequence selection function. Using the set of detected active spreading sequences, an MMSE estimator is employed to produce log-likelihood ratios (LLRs) for the second part of the messages corresponding to these detected users. The LLRs associated with each detected user are then passed to a list decoder of the polar code, which performs single-user decoding to decode the second portion of the message. This decoding operation proceeds iteratively by subtracting the interference due to the successfully decoded messages from the received signal, and repeating the above steps on the residual signal. At this stage, the proposed algorithm outperforms alternate existing low-complexity schemes when the number of active uses is below 225.
Integrated guidance and control framework for the waypoint navigation of a miniature aircraft with highly coupled longitudinal and lateral dynamics
Venue: arXiv cs.RO 2019
Year: 2019

A solution to the waypoint navigation problem for fixed wing micro air vehicles (MAV) is addressed in this paper, in the framework of integrated guidance and control (IGC). IGC yields a single step solution to the waypoint navigation problem, unlike conventional multiple loop design. The pure proportional navigation (PPN) guidance law is integrated with the MAV dynamics. A multivariable static output feedback (SOF) controller is designed for the linear state space model formulated in the IGC framework. The waypoint navigation algorithm handles the minimum turn radius constraint of the MAV. The algorithm also evaluates the feasibility of reaching a waypoint. Extensive non-linear simulations are performed on high fidelity 150 mm wingspan MAV model to demonstrate the potential advantages of the proposed waypoint navigation algorithm.
Technical Report: Co-learning of geometry and semantics for online 3D mapping
Venue: arXiv cs.CV 2019
Year: 2019

This paper is a technical report about our submission for the ECCV 2018 3DRMS Workshop Challenge on Semantic 3D Reconstruction \cite{Tylecek2018rms}. In this paper, we address 3D semantic reconstruction for autonomous navigation using co-learning of depth map and semantic segmentation. The core of our pipeline is a deep multi-task neural network which tightly refines depth and also produces accurate semantic segmentation maps. Its inputs are an image and a raw depth map produced from a pair of images by standard stereo vision. The resulting semantic 3D point clouds are then merged in order to create a consistent 3D mesh, in turn used to produce dense semantic 3D reconstruction maps. The performances of each step of the proposed method are evaluated on the dataset and multiple tasks of the 3DRMS Challenge, and repeatedly surpass state-of-the-art approaches.
Generalising KAT to verify weighted computations
Venue: arXiv cs.LO 2019
Year: 2019

Kleene algebra with tests (KAT) was introduced as an algebraic structure to model and reason about classic imperative programs, i.e. sequences of discrete transitions guarded by Boolean tests. This paper introduces two generalisations of this structure able to express programs as weighted transitions and tests with outcomes in non necessarily bivalent truth spaces: graded Kleene algebra with tests (GKAT) and a variant where tests are also idempotent (I-GKAT). On this context, and in analogy to Kozen's encoding of Propositional Hoare Logic (PHL) in KAT [22], we discuss the encoding of a graded PHL in I-GKAT and of its while-free fragment in GKAT. Moreover, to establish semantics for these structures four new algebras are defined: FSET(T), FREL(K,T) and FLANG(K,T) over complete residuated lattices K and T, and M(n,A) over a GKAT or I-GKAT A. As a final exercise, the paper discusses some program equivalence proofs in a graded context.
Molecular Communication in H-Diffusion
Venue: arXiv cs.IT 2019
Year: 2019

The random propagation of molecules in a fluid medium is characterized by the spontaneous diffusion law as well as the interaction between the environment and molecules. In this paper, we embody the anomalous diffusion theory for modeling and analysis in molecular communication. We employ H-diffusion to model a non-Fickian behavior of molecules in diffusive channels. H-diffusion enables us to model anomalous diffusion as the subordinate relationship between self-similar parent and directing processes and their corresponding probability density functions with two H-variates in a unified fashion. In addition, we introduce standard H-diffusion to make a bridge of normal diffusion across well-known anomalous diffusions such as space-time fractional diffusion, Erdelyi-Kober fractional diffusion, grey Brownian motion, fractional Brownian motion, and Brownian motion. We then characterize the statistical properties of uncertainty of the random propagation time of a molecule governed by H-diffusion laws by introducing a general class of molecular noise---called H-noise. Since H-noise can be an algebraic tailed process, we provide a concept of H-noise power using finite logarithm moments based on zero-order statistics. Finally, we develop a unifying framework for error probability analysis in a timing-based molecular communication system with a concept of signal-to-noise power ratio.
Semi-Supervised Medical Image Segmentation via Learning Consistency under Transformations
Venue: arXiv cs.CV 2019
Year: 2019

The scarcity of labeled data often limits the application of supervised deep learning techniques for medical image segmentation. This has motivated the development of semi-supervised techniques that learn from a mixture of labeled and unlabeled images. In this paper, we propose a novel semi-supervised method that, in addition to supervised learning on labeled training images, learns to predict segmentations consistent under a given class of transformations on both labeled and unlabeled images. More specifically, in this work we explore learning equivariance to elastic deformations. We implement this through: 1) a Siamese architecture with two identical branches, each of which receives a differently transformed image, and 2) a composite loss function with a supervised segmentation loss term and an unsupervised term that encourages segmentation consistency between the predictions of the two branches. We evaluate the method on a public dataset of chest radiographs with segmentations of anatomical structures using 5-fold cross-validation. The proposed method reaches significantly higher segmentation accuracy compared to supervised learning. This is due to learning transformation consistency on both labeled and unlabeled images, with the latter contributing the most. We achieve the performance comparable to state-of-the-art chest X-ray segmentation methods while using substantially fewer labeled images.
Tustin neural networks: a class of recurrent nets for adaptive MPC of mechanical systems
Venue: arXiv cs.NE 2019
Year: 2019

The use of recurrent neural networks to represent the dynamics of unstable systems is difficult due to the need to properly initialize their internal states, which in most of the cases do not have any physical meaning, consequent to the non-smoothness of the optimization problem. For this reason, in this paper focus is placed on mechanical systems characterized by a number of degrees of freedom, each one represented by two states, namely position and velocity. For these systems, a new recurrent neural network is proposed: Tustin-Net. Inspired by second-order dynamics, the network hidden states can be straightforwardly estimated, as their differential relationships with the measured states are hardcoded in the forward pass. The proposed structure is used to model a double inverted pendulum and for model-based Reinforcement Learning, where an adaptive Model Predictive Controller scheme using the Unscented Kalman Filter is proposed to deal with parameter changes in the system.
Lifting Sum-of-Squares Lower Bounds: Degree-$2$ to Degree-$4$
Venue: arXiv cs.CC 2019
Year: 2019

The degree-$4$ Sum-of-Squares (SoS) SDP relaxation is a powerful algorithm that captures the best known polynomial time algorithms for a broad range of problems including MaxCut, Sparsest Cut, all MaxCSPs and tensor PCA. Despite being an explicit algorithm with relatively low computational complexity, the limits of degree-$4$ SoS SDP are not well understood. For example, existing integrality gaps do not rule out a $(2-\varepsilon)$-algorithm for Vertex Cover or a $(0.878+\varepsilon)$-algorithm for MaxCut via degree-$4$ SoS SDPs, each of which would refute the notorious Unique Games Conjecture. We exhibit an explicit mapping from solutions for degree-$2$ Sum-of-Squares SDP (Goemans-Williamson SDP) to solutions for the degree-$4$ Sum-of-Squares SDP relaxation on boolean variables. By virtue of this mapping, one can lift lower bounds for degree-$2$ SoS SDP relaxation to corresponding lower bounds for degree-$4$ SoS SDPs. We use this approach to obtain degree-$4$ SoS SDP lower bounds for MaxCut on random $d$-regular graphs, Sherington-Kirkpatrick model from statistical physics and PSD Grothendieck problem. Our constructions use the idea of pseudocalibration towards candidate SDP vectors, while it was previously only used to produce the candidate matrix which one would show is PSD using much technical work. In addition, we develop a different technique to bound the spectral norms of _graphical matrices_ that arise in the context of SoS SDPs. The technique is much simpler and yields better bounds in many cases than the _trace method_ -- which was the sole technique for this purpose.
Dynamic Graph Embedding via LSTM History Tracking
Venue: arXiv cs.LG 2019
Year: 2019

Many real world networks are very large and constantly change over time. These dynamic networks exist in various domains such as social networks, traffic networks and biological interactions. To handle large dynamic networks in downstream applications such as link prediction and anomaly detection, it is essential for such networks to be transferred into a low dimensional space. Recently, network embedding, a technique that converts a large graph into a low-dimensional representation, has become increasingly popular due to its strength in preserving the structure of a network. Efficient dynamic network embedding, however, has not yet been fully explored. In this paper, we present a dynamic network embedding method that integrates the history of nodes over time into the current state of nodes. The key contribution of our work is 1) generating dynamic network embedding by combining both dynamic and static node information 2) tracking history of neighbors of nodes using LSTM 3) significantly decreasing the time and memory by training an autoencoder LSTM model using temporal walks rather than adjacency matrices of graphs which are the common practice. We evaluate our method in multiple applications such as anomaly detection, link prediction and node classification in datasets from various domains.
Integrating Dictionary Feature into A Deep Learning Model for Disease Named Entity Recognition
Venue: arXiv cs.CL 2019
Year: 2019

In recent years, Deep Learning (DL) models are becoming important due to their demonstrated success at overcoming complex learning problems. DL models have been applied effectively for different Natural Language Processing (NLP) tasks such as part-of-Speech (PoS) tagging and Machine Translation (MT). Disease Named Entity Recognition (Disease-NER) is a crucial task which aims at extracting disease Named Entities (NEs) from text. In this paper, a DL model for Disease-NER using dictionary information is proposed and evaluated on National Center for Biotechnology Information (NCBI) disease corpus and BC5CDR dataset. Word embeddings trained over general domain texts as well as biomedical texts have been used to represent input to the proposed model. This study also compares two different Segment Representation (SR) schemes, namely IOB2 and IOBES for Disease-NER. The results illustrate that using dictionary information, pre-trained word embeddings, character embeddings and CRF with global score improves the performance of Disease-NER system.