OpenRaiser/Intern-Atlas huggingface dataset:
On the Optimality of Treating Inter-Cell Interference as Noise: Downlink Cellular Networks and Uplink-Downlink Duality
Venue: arXiv cs.IT 2019
Year: 2019

We consider the information-theoretic optimality of treating inter-cell interference as noise (multi-cell TIN) in downlink cellular networks. We focus on scenarios modeled by the Gaussian interfering broadcast channel (IBC), comprising $K$ mutually interfering Gaussian broadcast channels (BCs), each formed by a base station communicating independent messages to an arbitrary number of users. We establish a new power allocation duality between the IBC and its dual interfering multiple access channel (IMAC), which entails that the corresponding generalized degrees-of-freedom regions achieved through multi-cell TIN and power control (TINA regions) for both networks are identical. As by-products of this duality, we obtain an explicit characterization of the IBC TINA region from a previously established characterization of the IMAC TINA region; and identify a multi-cell convex-TIN regime in which the IBC TINA region is a polyhedron (hence convex) without the need for time-sharing. We then identify a smaller multi-cell TIN regime in which the IBC TINA region is optimal and multi-cell TIN achieves the entire capacity region of the IBC, up to a constant gap. This is accomplished by deriving a new genie-aided outer bound for the IBC, that reveals a novel BC-type order that holds amongst users in each constituent BC (or cell) under inter-cell interference, which in turn is not implied by previously known BC-type orders (i.e. degraded, less noisy and more capable orders). The multi-cell TIN regime that we identify for the IBC coincides with a corresponding multi-cell TIN regime previously identified for the IMAC, hence establishing a comprehensive uplink-downlink duality of multi-cell TIN in the GDoF (and approximate capacity) sense.
New Attacks and Defenses for Randomized Caches
Venue: arXiv cs.CR 2019
Year: 2019

The last level cache is vulnerable to timing based side channel attacks because it is shared by the attacker and the victim processes even if they are located on different cores. These timing attacks evict the victim cache lines using small conflict groups(SCG), and monitor the cache to observe when the victim uses these cache lines again. A conflict group is a collection of cache lines which will evict the target cache line. Randomization is often used by defenses to prevent creation of SCGs. We introduce new attacks to demonstrate that the current randomization schemes require an extremely high refresh rate to be secure, on average a 15\% performance overhead, and upto 50\% in the worst case. Next, we propose a new randomization strategy using an indirection table, which mitigates this issue. Addresses of cache lines are encrypted and used to lookup the indirection table entry. Each indirection table entry stores a mapping to a randomly chosen cache set. The cache line is placed into this randomly chosen set. The encryption key changes upto 50x faster than CEASER's default rate, by using evictions to trigger the re-randomization. Instead of moving cache lines, this mechanism re-randomizes one iTable entry at a time, whenever the cache lines corresponding to the iTable entry are naturally evicted. Thus, the miss rate is not much worse than the baseline. We quantitatively show that our scheme does almost as well as a fully associative cache to defend against these attacks. We also demonstrate new attacks that target the iTable by oversubscribing its entries, and quantitatively show that our scheme is resilient against new attacks for trillions of years. We estimate low area ( < 7\%) and power overhead compared to a baseline inclusive last-level cache. Lastly, we evaluate a low performance overhead (<4%) using the SPECrate 2017 and PARSEC 3.0 benchmarks.
Coin_flipper at eHealth-KD Challenge 2019: Voting LSTMs for Key Phrases and Semantic Relation Identification Applied to Spanish eHealth Texts
Venue: arXiv cs.CL 2019
Year: 2019

This paper describes our approach presented for the eHealth-KD 2019 challenge. Our participation was aimed at testing how far we could go using generic tools for Text-Processing but, at the same time, using common optimization techniques in the field of Data Mining. The architecture proposed for both tasks of the challenge is a standard stacked 2-layer bi-LSTM. The main particularities of our approach are: (a) The use of a surrogate function of F1 as loss function to close the gap between the minimization function and the evaluation metric, and (b) The generation of an ensemble of models for generating predictions by majority vote. Our system ranked second with an F1 score of 62.18% in the main task by a narrow margin with the winner that scored 63.94%.
On the Importance of Subword Information for Morphological Tasks in Truly Low-Resource Languages
Venue: arXiv cs.CL 2019
Year: 2019

Recent work has validated the importance of subword information for word representation learning. Since subwords increase parameter sharing ability in neural models, their value should be even more pronounced in low-data regimes. In this work, we therefore provide a comprehensive analysis focused on the usefulness of subwords for word representation learning in truly low-resource scenarios and for three representative morphological tasks: fine-grained entity typing, morphological tagging, and named entity recognition. We conduct a systematic study that spans several dimensions of comparison: 1) type of data scarcity which can stem from the lack of task-specific training data, or even from the lack of unannotated data required to train word embeddings, or both; 2) language type by working with a sample of 16 typologically diverse languages including some truly low-resource ones (e.g. Rusyn, Buryat, and Zulu); 3) the choice of the subword-informed word representation method. Our main results show that subword-informed models are universally useful across all language types, with large gains over subword-agnostic embeddings. They also suggest that the effective use of subwords largely depends on the language (type) and the task at hand, as well as on the amount of available data for training the embeddings and task-based models, where having sufficient in-task data is a more critical requirement.
Optimizing Nondecomposable Data Dependent Regularizers via Lagrangian Reparameterization offers Significant Performance and Efficiency Gains
Venue: arXiv cs.CV 2019
Year: 2019

Data dependent regularization is known to benefit a wide variety of problems in machine learning. Often, these regularizers cannot be easily decomposed into a sum over a finite number of terms, e.g., a sum over individual example-wise terms. The $F_β$ measure, Area under the ROC curve (AUCROC) and Precision at a fixed recall (P@R) are some prominent examples that are used in many applications. We find that for most medium to large sized datasets, scalability issues severely limit our ability in leveraging the benefits of such regularizers. Importantly, the key technical impediment despite some recent progress is that, such objectives remain difficult to optimize via backpropapagation procedures. While an efficient general-purpose strategy for this problem still remains elusive, in this paper, we show that for many data-dependent nondecomposable regularizers that are relevant in applications, sizable gains in efficiency are possible with minimal code-level changes; in other words, no specialized tools or numerical schemes are needed. Our procedure involves a reparameterization followed by a partial dualization -- this leads to a formulation that has provably cheap projection operators. We present a detailed analysis of runtime and convergence properties of our algorithm. On the experimental side, we show that a direct use of our scheme significantly improves the state of the art IOU measures reported for MSCOCO Stuff segmentation dataset.
Hidden Stratification Causes Clinically Meaningful Failures in Machine Learning for Medical Imaging
Venue: arXiv cs.LG 2019
Year: 2019

Machine learning models for medical image analysis often suffer from poor performance on important subsets of a population that are not identified during training or testing. For example, overall performance of a cancer detection model may be high, but the model still consistently misses a rare but aggressive cancer subtype. We refer to this problem as hidden stratification, and observe that it results from incompletely describing the meaningful variation in a dataset. While hidden stratification can substantially reduce the clinical efficacy of machine learning models, its effects remain difficult to measure. In this work, we assess the utility of several possible techniques for measuring and describing hidden stratification effects, and characterize these effects on multiple medical imaging datasets. We find evidence that hidden stratification can occur in unidentified imaging subsets with low prevalence, low label quality, subtle distinguishing features, or spurious correlates, and that it can result in relative performance differences of over 20% on clinically important subsets. Finally, we explore the clinical implications of our findings, and suggest that evaluation of hidden stratification should be a critical component of any machine learning deployment in medical imaging.
Multi-Agent Actor-Critic with Hierarchical Graph Attention Network
Venue: arXiv cs.LG 2019
Year: 2019

Most previous studies on multi-agent reinforcement learning focus on deriving decentralized and cooperative policies to maximize a common reward and rarely consider the transferability of trained policies to new tasks. This prevents such policies from being applied to more complex multi-agent tasks. To resolve these limitations, we propose a model that conducts both representation learning for multiple agents using hierarchical graph attention network and policy learning using multi-agent actor-critic. The hierarchical graph attention network is specially designed to model the hierarchical relationships among multiple agents that either cooperate or compete with each other to derive more advanced strategic policies. Two attention networks, the inter-agent and inter-group attention layers, are used to effectively model individual and group level interactions, respectively. The two attention networks have been proven to facilitate the transfer of learned policies to new tasks with different agent compositions and allow one to interpret the learned strategies. Empirically, we demonstrate that the proposed model outperforms existing methods in several mixed cooperative and competitive tasks.
Fast Fixed Dimension L2-Subspace Embeddings of Arbitrary Accuracy, With Application to L1 and L2 Tasks
Venue: arXiv cs.LG 2019
Year: 2019

We give a fast oblivious L2-embedding of $A\in \mathbb{R}^{n x d}$ to $B\in \mathbb{R}^{r x d}$ satisfying $(1-\varepsilon)\|A x\|_2^2 \le \|B x\|_2^2 <= (1+\varepsilon) \|Ax\|_2^2.$ Our embedding dimension $r$ equals $d$, a constant independent of the distortion $\varepsilon$. We use as a black-box any L2-embedding $Π^T A$ and inherit its runtime and accuracy, effectively decoupling the dimension $r$ from runtime and accuracy, allowing downstream machine learning applications to benefit from both a low dimension and high accuracy (in prior embeddings higher accuracy means higher dimension). We give applications of our L2-embedding to regression, PCA and statistical leverage scores. We also give applications to L1: 1.) An oblivious L1-embedding with dimension $d+O(d\ln^{1+η} d)$ and distortion $O((d\ln d)/\ln\ln d)$, with application to constructing well-conditioned bases; 2.) Fast approximation of L1-Lewis weights using our L2 embedding to quickly approximate L2-leverage scores.
Improving Resource Allocation in Software-Defined Networks using Clustering
Venue: arXiv cs.NI 2019
Year: 2019

Software-defined networks (SDNs) are a huge evolution in simplifying implementation and network operation which have reduced costs and made the network programmable. Although SDNs are a suitable option for solving some of the previous problems, but they have some challenges as any other new technology. Resource allocation and balance control in network is one of the main challenges of this technology which is studied in this paper. In this study, a new approach is proposed for improving memory resource allocation in network using load distribution clusters. Since in the proposed method, K-mean++ algorithm is used for clustering, load balancing of clusters can be used to preserve load balance of the network. In the proposed method, data with higher recall is transmitted to high-quality clusters in terms of average number of hubs and lower average delay between server and user. In the proposed method, by increasing number of clusters, higher memory is created in the network.
Multi-Modal Citizen Science: From Disambiguation to Transcription of Classical Literature
Venue: arXiv cs.CL 2019
Year: 2019

The engagement of citizens in the research projects, including Digital Humanities projects, has risen in prominence in recent years. This type of engagement not only leads to incidental learning of participants but also indicates the added value of corpus enrichment via different types of annotations undertaken by users generating so-called smart texts. Our work focuses on the continuous task of adding new layers of annotation to Classical Literature. We aim to provide more extensive tools for readers of smart texts, enhancing their reading comprehension and at the same time empowering the language learning by introducing intellectual tasks, i.e., linking, tagging, and disambiguation. The current study adds a new mode of annotation-audio annotations-to the extensively annotated corpus of poetry by the Persian poet Hafiz. By proposing tasks with three different difficulty levels, we estimate the users' ability of providing correct annotations in order to rate their answers in further stages of the project, where no ground truth data is available. While proficiency in Persian is beneficial, annotators with no knowledge of Persian are also able to add annotations to the corpus.
Point Attention Network for Semantic Segmentation of 3D Point Clouds
Venue: arXiv cs.CV 2019
Year: 2019

Convolutional Neural Networks (CNNs) have performed extremely well on data represented by regularly arranged grids such as images. However, directly leveraging the classic convolution kernels or parameter sharing mechanisms on sparse 3D point clouds is inefficient due to their irregular and unordered nature. We propose a point attention network that learns rich local shape features and their contextual correlations for 3D point cloud semantic segmentation. Since the geometric distribution of the neighboring points is invariant to the point ordering, we propose a Local Attention-Edge Convolution (LAE Conv) to construct a local graph based on the neighborhood points searched in multi-directions. We assign attention coefficients to each edge and then aggregate the point features as a weighted sum of its neighbors. The learned LAE-Conv layer features are then given to a point-wise spatial attention module to generate an interdependency matrix of all points regardless of their distances, which captures long-range spatial contextual features contributing to more precise semantic information. The proposed point attention network consists of an encoder and decoder which, together with the LAE-Conv layers and the point-wise spatial attention modules, make it an end-to-end trainable network for predicting dense labels for 3D point cloud segmentation. Experiments on challenging benchmarks of 3D point clouds show that our algorithm can perform at par or better than the existing state of the art methods.
Information Design in Spatial Resource Competition
Venue: arXiv cs.GT 2019
Year: 2019

We consider the information design problem in spatial resource competition settings. Agents gather at a location deciding whether to move to another location for possibly higher level of resources, and the utility each agent gets by moving to the other location decreases as more agents move there. The agents do not observe the resource level at the other location while a principal does and the principal would like to carefully release this information to attract a proper number of agents to move. We adopt the Bayesian persuasion framework and analyze the principal's optimal signaling mechanism design problem. We study both private and public signaling mechanisms. For private signaling, we show the optimal mechanism can be computed in polynomial time with respect to the number of agents. Obtaining the optimal private mechanism involves two steps: first, solve a linear program to get the marginal probability each agent should be recommended to move; second, sample the moving agents satisfying the marginal probabilities with a sequential sampling procedure. For public signaling, we show the sender preferred equilibrium has a simple threshold structure and the optimal public mechanism with respect to the sender preferred equilibrium can be computed in polynomial time. We support our analytical results with numerical computations that show the optimal private and public signaling mechanisms achieve substantially higher social welfare compared with no information or full information benchmarks in many settings.
Long-Term Robot Navigation in Indoor Environments Estimating Patterns in Traversability Changes
Venue: arXiv cs.RO 2019
Year: 2019

Nowadays, mobile robots are deployed in many indoor environments, such as offices or hospitals. These environments are subject to changes in the traversability that often happen by following repeating patterns. In this paper, we investigate the problem of navigating in such environments over extended periods of time by capturing these patterns and exploiting this knowledge to make informed decisions. Our approach incrementally estimates a model of the traversability changes from robot's observations and uses a probabilistic graphical model to make predictions at currently unobserved locations. In the belief space defined by the predictions, we plan paths that trade off the risk to encounter obstacles and the information gain of visiting unknown locations. We implemented our approach and tested it in different indoor environments. The experiments suggest that in the long run, our approach leads to navigation along shorter paths compared to following a greedy shortest path policy.
Maximal adversarial perturbations for obfuscation: Hiding certain attributes while preserving rest
Venue: arXiv cs.LG 2019
Year: 2019

In this paper we investigate the usage of adversarial perturbations for the purpose of privacy from human perception and model (machine) based detection. We employ adversarial perturbations for obfuscating certain variables in raw data while preserving the rest. Current adversarial perturbation methods are used for data poisoning with minimal perturbations of the raw data such that the machine learning model's performance is adversely impacted while the human vision cannot perceive the difference in the poisoned dataset due to minimal nature of perturbations. We instead apply relatively maximal perturbations of raw data to conditionally damage model's classification of one attribute while preserving the model performance over another attribute. In addition, the maximal nature of perturbation helps adversely impact human perception in classifying hidden attribute apart from impacting model performance. We validate our result qualitatively by showing the obfuscated dataset and quantitatively by showing the inability of models trained on clean data to predict the hidden attribute from the perturbed dataset while being able to predict the rest of attributes.
On the use of BERT for Neural Machine Translation
Venue: arXiv cs.CL 2019
Year: 2019

Exploiting large pretrained models for various NMT tasks have gained a lot of visibility recently. In this work we study how BERT pretrained models could be exploited for supervised Neural Machine Translation. We compare various ways to integrate pretrained BERT model with NMT model and study the impact of the monolingual data used for BERT training on the final translation quality. We use WMT-14 English-German, IWSLT15 English-German and IWSLT14 English-Russian datasets for these experiments. In addition to standard task test set evaluation, we perform evaluation on out-of-domain test sets and noise injected test sets, in order to assess how BERT pretrained representations affect model robustness.
Movie Recommender Systems: Implementation and Performance Evaluation
Venue: arXiv cs.IR 2019
Year: 2019

Over the years, explosive growth in the number of items in the catalog of e-commerce businesses, such as Amazon, Netflix, Pandora, etc., have warranted the development of recommender systems to guide consumers towards their desired products based on their preferences and tastes. Some of the popular approaches for building recommender systems, for mining user, derived input datasets, are: content-based systems, collaborative filtering, latent-factor systems using Singular Value Decomposition (SVD), and Restricted Boltzmann Machines (RBM). In this project, user-user collaborative filtering, item-item collaborative filtering, content-based recommendation, SVD, and neural networks were chosen for implementation in Python to predict the user ratings of unwatched movies for each user, and their performances were evaluated and compared.
Hamiltonian Graph Networks with ODE Integrators
Venue: arXiv cs.LG 2019
Year: 2019

We introduce an approach for imposing physically informed inductive biases in learned simulation models. We combine graph networks with a differentiable ordinary differential equation integrator as a mechanism for predicting future states, and a Hamiltonian as an internal representation. We find that our approach outperforms baselines without these biases in terms of predictive accuracy, energy accuracy, and zero-shot generalization to time-step sizes and integrator orders not experienced during training. This advances the state-of-the-art of learned simulation, and in principle is applicable beyond physical domains.
Research Commentary on Recommendations with Side Information: A Survey and Research Directions
Venue: arXiv cs.IR 2019
Year: 2019

Recommender systems have become an essential tool to help resolve the information overload problem in recent decades. Traditional recommender systems, however, suffer from data sparsity and cold start problems. To address these issues, a great number of recommendation algorithms have been proposed to leverage side information of users or items (e.g., social network and item category), demonstrating a high degree of effectiveness in improving recommendation performance. This Research Commentary aims to provide a comprehensive and systematic survey of the recent research on recommender systems with side information. Specifically, we provide an overview of state-of-the-art recommendation algorithms with side information from two orthogonal perspectives. One involves the different methodologies of recommendation: the memory-based methods, latent factor, representation learning, and deep learning models. The others cover different representations of side information, including structural data (flat, network, and hierarchical features, and knowledge graphs); and non-structural data (text, image and video features). Finally, we discuss challenges and provide new potential directions in recommendation, along with the conclusion of this survey.