OpenRaiser/Intern-Atlas huggingface dataset:
Community Detection and Improved Detectability in Multiplex Networks
Venue: arXiv cs.SI 2019
Year: 2019

We investigate the widely encountered problem of detecting communities in multiplex networks, such as social networks, with an unknown arbitrary heterogeneous structure. To improve detectability, we propose a generative model that leverages the multiplicity of a single community in multiple layers, with no prior assumption on the relation of communities among different layers. Our model relies on a novel idea of incorporating a large set of generic localized community label constraints across the layers, in conjunction with the celebrated Stochastic Block Model (SBM) in each layer. Accordingly, we build a probabilistic graphical model over the entire multiplex network by treating the constraints as Bayesian priors. We mathematically prove that these constraints/priors promote existence of identical communities across layers without introducing further correlation between individual communities. The constraints are further tailored to render a sparse graphical model and the numerically efficient Belief Propagation algorithm is subsequently employed. We further demonstrate by numerical experiments that in the presence of consistent communities between different layers, consistent communities are matched, and the detectability is improved over a single layer. We compare our model with a "correlated model" which exploits the prior knowledge of community correlation between layers. Similar detectability improvement is obtained under such a correlation, even though our model relies on much milder assumptions than the correlated model. Our model even shows a better detection performance over a certain correlation and signal to noise ratio (SNR) range. In the absence of community correlation, the correlation model naturally fails, while ours maintains its performance.
Modeling Peoples Voting Behavior with Poll Information
Venue: arXiv cs.GT 2019
Year: 2019

Despite the prevalence of voting systems in the real world there is no consensus among researchers of how people vote strategically, even in simple voting settings. This paper addresses this gap by comparing different approaches that have been used to model strategic voting, including expected utility maximization, heuristic decisionmaking, and bounded rationality models. The models are applied to data collected from hundreds of people in controlled voting experiments, where people vote after observing non-binding poll information. We introduce a new voting model, the Attainability- Utility (AU) heuristic, which weighs the popularity of a candidate according to the poll, with the utility of the candidate to the voter. We argue that the AU model is cognitively plausible, and show that it is able to predict peoples voting behavior significantly better than other models from the literature. It was almost at par with (and sometimes better than) a machine learning algorithm that uses substantially more information. Our results provide new insights into the strategic considerations of voters, that undermine the prevalent assumptions of much theoretical work in social choice.
Formalism for Supporting the Development of Verifiably Safe Medical Guidelines with Statecharts
Venue: arXiv cs.SE 2019
Year: 2019

Improving the effectiveness and safety of patient care is the ultimate objective for medical cyber-physical systems. Many medical best practice guidelines exist, but most of the existing guidelines in handbooks are difficult for medical staff to remember and apply clinically. Furthermore, although the guidelines have gone through clinical validations, validations by medical professionals alone do not provide guarantees for the safety of medical cyber-physical systems. Hence, formal verification is also needed. The paper presents the formal semantics for a framework that we developed to support the development of verifiably safe medical guidelines. The framework allows computer scientists to work together with medical professionals to transform medical best practice guidelines into executable statechart models, Yakindu in particular, so that medical functionalities and properties can be quickly prototyped and validated. Existing formal verification technologies, UPPAAL timed automata in particular, is integrated into the framework to provide formal verification capabilities to verify safety properties. However, some components used/built into the framework, such as the open-source Yakindu statecharts as well as the transformation rules from statecharts to timed automata, do not have built-in semantics. The ambiguity becomes unavoidable unless formal semantics is defined for the framework, which is what the paper is to present.
Hypernym Detection Using Strict Partial Order Networks
Venue: arXiv cs.AI 2019
Year: 2019

This paper introduces Strict Partial Order Networks (SPON), a novel neural network architecture designed to enforce asymmetry and transitive properties as soft constraints. We apply it to induce hypernymy relations by training with is-a pairs. We also present an augmented variant of SPON that can generalize type information learned for in-vocabulary terms to previously unseen ones. An extensive evaluation over eleven benchmarks across different tasks shows that SPON consistently either outperforms or attains the state of the art on all but one of these benchmarks.
MemGuard: Defending against Black-Box Membership Inference Attacks via Adversarial Examples
Venue: arXiv cs.CR 2019
Year: 2019

In a membership inference attack, an attacker aims to infer whether a data sample is in a target classifier's training dataset or not. Specifically, given a black-box access to the target classifier, the attacker trains a binary classifier, which takes a data sample's confidence score vector predicted by the target classifier as an input and predicts the data sample to be a member or non-member of the target classifier's training dataset. Membership inference attacks pose severe privacy and security threats to the training dataset. Most existing defenses leverage differential privacy when training the target classifier or regularize the training process of the target classifier. These defenses suffer from two key limitations: 1) they do not have formal utility-loss guarantees of the confidence score vectors, and 2) they achieve suboptimal privacy-utility tradeoffs. In this work, we propose MemGuard, the first defense with formal utility-loss guarantees against black-box membership inference attacks. Instead of tampering the training process of the target classifier, MemGuard adds noise to each confidence score vector predicted by the target classifier. Our key observation is that attacker uses a classifier to predict member or non-member and classifier is vulnerable to adversarial examples. Based on the observation, we propose to add a carefully crafted noise vector to a confidence score vector to turn it into an adversarial example that misleads the attacker's classifier. Our experimental results on three datasets show that MemGuard can effectively defend against membership inference attacks and achieve better privacy-utility tradeoffs than existing defenses. Our work is the first one to show that adversarial examples can be used as defensive mechanisms to defend against membership inference attacks.
Generating Geological Facies Models with Fidelity to Diversity and Statistics of Training Images using Improved Generative Adversarial Networks
Venue: arXiv cs.LG 2019
Year: 2019

This paper presents a methodology and workflow that overcome the limitations of the conventional Generative Adversarial Networks (GANs) for geological facies modeling. It attempts to improve the training stability and guarantee the diversity of the generated geology through interpretable latent vectors. The resulting samples are ensured to have the equal probability (or an unbiased distribution) as from the training dataset. This is critical when applying GANs to generate unbiased and representative geological models that can be further used to facilitate objective uncertainty evaluation and optimal decision-making in oil field exploration and development. We proposed and implemented a new variant of GANs called Info-WGAN for the geological facies modeling that combines Information Maximizing Generative Adversarial Network (InfoGAN) with Wasserstein distance and Gradient Penalty (GP) for learning interpretable latent codes as well as generating stable and unbiased distribution from the training data. Different from the original GAN design, InfoGAN can use the training images with full, partial, or no labels to perform disentanglement of the complex sedimentary types exhibited in the training dataset to achieve the variety and diversity of the generated samples. This is accomplished by adding additional categorical variables that provide disentangled semantic representations besides the mere randomized latent vector used in the original GANs. By such means, a regularization term is used to maximize the mutual information between such latent categorical codes and the generated geological facies in the loss function. Furthermore, the resulting unbiased sampling by Info-WGAN makes the data conditioning much easier than the conventional GANs in geological modeling because of the variety and diversity as well as the equal probability of the unconditional sampling by the generator.
The Query Complexity of Mastermind with $\ell_p$ Distances
Venue: arXiv cs.DS 2019
Year: 2019

Consider a variant of the Mastermind game in which queries are $\ell_p$ distances, rather than the usual Hamming distance. That is, a codemaker chooses a hidden vector $\mathbf{y}\in\{-k,-k+1,\dots,k-1,k\}^n$ and answers to queries of the form $\Vert\mathbf{y}-\mathbf{x}\Vert_p$ where $\mathbf{x}\in\{-k,-k+1,\dots,k-1,k\}^n$. The goal is to minimize the number of queries made in order to correctly guess $\mathbf{y}$. Motivated by this question, in this work, we develop a nonadaptive polynomial time algorithm that works for a natural class of separable distance measures, i.e.\ coordinate-wise sums of functions of the absolute value. This in particular includes distances such as the smooth max (LogSumExp) as well as many widely-studied $M$-estimator losses, such as $\ell_p$ norms, the $\ell_1$-$\ell_2$ loss, the Huber loss, and the Fair estimator loss. When we apply this result to $\ell_p$ queries, we obtain an upper bound of $O\left(\min\left\{n,\frac{n\log k}{\log n}\right\}\right)$ queries for any real $1\leq p<\infty$. We also show matching lower bounds up to constant factors for the $\ell_p$ problem, even for adaptive algorithms for the approximation version of the problem, in which the problem is to output $\mathbf{y}'$ such that $\Vert\mathbf{y}'-\mathbf{y}\Vert_p\leq R$ for any $R\leq k^{1-\varepsilon}n^{1/p}$ for constant $\varepsilon>0$. Thus, essentially any approximation of this problem is as hard as finding the hidden vector exactly, up to constant factors. Finally, we show that for the noisy version of the problem, i.e. the setting when the codemaker answers queries with any $q = (1\pm\varepsilon)\Vert\mathbf{y}-\mathbf{x}\Vert_p$, there is no query efficient algorithm.
Learning deep representations for video-based intake gesture detection
Venue: arXiv cs.CV 2019
Year: 2019

Automatic detection of individual intake gestures during eating occasions has the potential to improve dietary monitoring and support dietary recommendations. Existing studies typically make use of on-body solutions such as inertial and audio sensors, while video is used as ground truth. Intake gesture detection directly based on video has rarely been attempted. In this study, we address this gap and show that deep learning architectures can successfully be applied to the problem of video-based detection of intake gestures. For this purpose, we collect and label video data of eating occasions using 360-degree video of 102 participants. Applying state-of-the-art approaches from video action recognition, our results show that (1) the best model achieves an $F_1$ score of 0.858, (2) appearance features contribute more than motion features, and (3) temporal context in form of multiple video frames is essential for top model performance.
Dimension Estimation Using Autoencoders
Venue: arXiv cs.LG 2019
Year: 2019

Dimension Estimation (DE) and Dimension Reduction (DR) are two closely related topics, but with quite different goals. In DE, one attempts to estimate the intrinsic dimensionality or number of latent variables in a set of measurements of a random vector. However, in DR, one attempts to project a random vector, either linearly or non-linearly, to a lower dimensional space that preserves the information contained in the original higher dimensional space. Of course, these two ideas are quite closely linked since, for example, doing DR to a dimension smaller than suggested by DE will likely lead to information loss. Accordingly, in this paper we will focus on a particular class of deep neural networks called autoencoders which are used extensively for DR but are less well studied for DE. We show that several important questions arise when using autoencoders for DE, above and beyond those that arise for more classic DR/DE techniques such as Principal Component Analysis. We address autoencoder architectural choices and regularization techniques that allow one to transform autoencoder latent layer representations into estimates of intrinsic dimension.
Graph Policy Gradients for Large Scale Unlabeled Motion Planning with Constraints
Venue: arXiv cs.RO 2019
Year: 2019

In this paper, we present a learning method to solve the unlabelled motion problem with motion constraints and space constraints in 2D space for a large number of robots. To solve the problem of arbitrary dynamics and constraints we propose formulating the problem as a multi-agent problem. In contrast to previous works that propose using learning solutions for unlabelled motion planning with constraints, we are able to demonstrate the scalability of our methods for a large number of robots. The curse of dimensionality one encounters when working with a large number of robots is mitigated by employing a graph convolutional neural (GCN) network to parametrize policies for the robots. The GCN reduces the dimensionality of the problem by learning filters that aggregate information among robots locally, similar to how a convolutional neural network is able to learn local features in an image. Additionally, by employing a GCN we are also able to overcome the computational overhead of training policies for a large number of robots by first training graph filters for a small number of robots followed by zero-shot policy transfer to a larger number of robots. We demonstrate the effectiveness of our framework through various simulations.
Do Massively Pretrained Language Models Make Better Storytellers?
Venue: arXiv cs.CL 2019
Year: 2019

Large neural language models trained on massive amounts of text have emerged as a formidable strategy for Natural Language Understanding tasks. However, the strength of these models as Natural Language Generators is less clear. Though anecdotal evidence suggests that these models generate better quality text, there has been no detailed study characterizing their generation abilities. In this work, we compare the performance of an extensively pretrained model, OpenAI GPT2-117 (Radford et al., 2019), to a state-of-the-art neural story generation model (Fan et al., 2018). By evaluating the generated text across a wide variety of automatic metrics, we characterize the ways in which pretrained models do, and do not, make better storytellers. We find that although GPT2-117 conditions more strongly on context, is more sensitive to ordering of events, and uses more unusual words, it is just as likely to produce repetitive and under-diverse text when using likelihood-maximizing decoding algorithms.
Bit-level Optimized Neural Network for Multi-antenna Channel Quantization
Venue: arXiv cs.IT 2019
Year: 2019

Quantized channel state information (CSI) plays a critical role in precoding design which helps reap the merits of multiple-input multiple-output (MIMO) technology. In order to reduce the overhead of CSI feedback, we propose a deep learning based CSI quantization method by developing a joint convolutional residual network (JCResNet) which benefits MIMO channel feature extraction and recovery from the perspective of bit-level quantization performance. Experiments show that our proposed method substantially improves the performance.
Technical report on Conversational Question Answering
Venue: arXiv cs.CL 2019
Year: 2019

Conversational Question Answering is a challenging task since it requires understanding of conversational history. In this project, we propose a new system RoBERTa + AT +KD, which involves rationale tagging multi-task, adversarial training, knowledge distillation and a linguistic post-process strategy. Our single model achieves 90.4(F1) on the CoQA test set without data augmentation, outperforming the current state-of-the-art single model by 2.6% F1.
Dynamic Approximate Shortest Paths and Beyond: Subquadratic and Worst-Case Update Time
Venue: arXiv cs.DS 2019
Year: 2019

Consider the following distance query for an $n$-node graph $G$ undergoing edge insertions and deletions: given two sets of nodes $I$ and $J$, return the distances between every pair of nodes in $I\times J$. This query is rather general and captures several versions of the dynamic shortest paths problem. In this paper, we develop an efficient $(1+ε)$-approximation algorithm for this query using fast matrix multiplication. Our algorithm leads to answers for some open problems for Single-Source and All-Pairs Shortest Paths (SSSP and APSP), as well as for Diameter, Radius, and Eccentricities. Below are some highlights. Note that all our algorithms guarantee worst-case update time and are randomized (Monte Carlo), but do not need the oblivious adversary assumption. Subquadratic update time for SSSP, Diameter, Centralities, ect.: When we want to maintain distances from a single node explicitly (without queries), a fundamental question is to beat trivially calling Dijkstra's static algorithm after each update, taking $Θ(n^2)$ update time on dense graphs. It was known to be improbable for exact algorithms and for combinatorial any-approximation algorithms to polynomially beat the $Ω(n^2)$ bound (under some conjectures) [Roditty, Zwick, ESA'04; Abboud, V. Williams, FOCS'14]. Our algorithm with $I=\{s\}$ and $J=V(G)$ implies a $(1+ε)$-approximation algorithm for this, guaranteeing $\tilde O(n^{1.823}/ε^2)$ worst-case update time for directed graphs with positive real weights in $[1, W]$. With ideas from [Roditty, V. Williams, STOC'13], we also obtain the first subquadratic worst-case update time for $(5/3+ε)$-approximating the eccentricities and $(1.5+ε)$-approximating the diameter and radius for unweighted graphs (with small additive errors). [...]
IFR-Net: Iterative Feature Refinement Network for Compressed Sensing MRI
Venue: arXiv cs.LG 2019
Year: 2019

To improve the compressive sensing MRI (CS-MRI) approaches in terms of fine structure loss under high acceleration factors, we have proposed an iterative feature refinement model (IFR-CS), equipped with fixed transforms, to restore the meaningful structures and details. Nevertheless, the proposed IFR-CS still has some limitations, such as the selection of hyper-parameters, a lengthy reconstruction time, and the fixed sparsifying transform. To alleviate these issues, we unroll the iterative feature refinement procedures in IFR-CS to a supervised model-driven network, dubbed IFR-Net. Equipped with training data pairs, both regularization parameter and the utmost feature refinement operator in IFR-CS become trainable. Additionally, inspired by the powerful representation capability of convolutional neural network (CNN), CNN-based inversion blocks are explored in the sparsity-promoting denoising module to generalize the sparsity-enforcing operator. Extensive experiments on both simulated and in vivo MR datasets have shown that the proposed network possesses a strong capability to capture image details and preserve well the structural information with fast reconstruction speed.
Application of Fuzzy Clustering for Text Data Dimensionality Reduction
Venue: arXiv cs.CL 2019
Year: 2019

Large textual corpora are often represented by the document-term frequency matrix whose elements are the frequency of terms; however, this matrix has two problems: sparsity and high dimensionality. Four dimension reduction strategies are used to address these problems. Of the four strategies, unsupervised feature transformation (UFT) is a popular and efficient strategy to map the terms to a new basis in the document-term frequency matrix. Although several UFT-based methods have been developed, fuzzy clustering has not been considered for dimensionality reduction. This research explores fuzzy clustering as a new UFT-based approach to create a lower-dimensional representation of documents. Performance of fuzzy clustering with and without using global term weighting methods is shown to exceed principal component analysis and singular value decomposition. This study also explores the effect of applying different fuzzifier values on fuzzy clustering for dimensionality reduction purpose.
Deep Mangoes: from fruit detection to cultivar identification in colour images of mango trees
Venue: arXiv cs.CV 2019
Year: 2019

This paper presents results on the detection and identification mango fruits from colour images of trees. We evaluate the behaviour and the performances of the Faster R-CNN network to determine whether it is robust enough to "detect and classify" fruits under particularly heterogeneous conditions in terms of plant cultivars, plantation scheme, and visual information acquisition contexts. The network is trained to distinguish the 'Kent', 'Keitt', and "Boucodiekhal" mango cultivars from 3,000 representative labelled fruit annotations. The validation set composed of about 7,000 annotations was then tested with a confidence threshold of 0.7 and a Non-Maximal-Suppression threshold of 0.25. With a F1-score of 0.90, the Faster R-CNN is well suitable to the simple fruit detection in tiles of 500x500 pixels. We then combine a multi-tiling approach with a Jaccard matrix to merge the different parts of objects detected several times, and thus report the detections made at the tile scale to the native 6,000x4,000 pixel size images. Nonetheless with a F1-score of 0.56, the cultivar identification Faster R-CNN network presents some limitations for simultaneously detecting the mango fruits and identifying their respective cultivars. Despite the proven errors in fruit detection, the cultivar identification rates of the detected mango fruits are in the order of 80%. The ideal solution could combine a Mask R-CNN for the image pre-segmentation of trees and a double-stream Faster R-CNN for detecting the mango fruits and identifying their respective cultivar to provide predictions more relevant to users' expectations.