OpenRaiser/Intern-Atlas huggingface dataset:
It's not Greek to mBERT: Inducing Word-Level Translations from Multilingual BERT
Venue: arXiv cs.CL 2020
Year: 2020

Recent works have demonstrated that multilingual BERT (mBERT) learns rich cross-lingual representations, that allow for transfer across languages. We study the word-level translation information embedded in mBERT and present two simple methods that expose remarkable translation capabilities with no fine-tuning. The results suggest that most of this information is encoded in a non-linear way, while some of it can also be recovered with purely linear tools. As part of our analysis, we test the hypothesis that mBERT learns representations which contain both a language-encoding component and an abstract, cross-lingual component, and explicitly identify an empirical language-identity subspace within mBERT representations.
On the Guaranteed Almost Equivalence between Imitation Learning from Observation and Demonstration
Venue: arXiv cs.RO 2020
Year: 2020

Imitation learning from observation (LfO) is more preferable than imitation learning from demonstration (LfD) due to the nonnecessity of expert actions when reconstructing the expert policy from the expert data. However, previous studies imply that the performance of LfO is inferior to LfD by a tremendous gap, which makes it challenging to employ LfO in practice. By contrast, this paper proves that LfO is almost equivalent to LfD in the deterministic robot environment, and more generally even in the robot environment with bounded randomness. In the deterministic robot environment, from the perspective of the control theory, we show that the inverse dynamics disagreement between LfO and LfD approaches zero, meaning that LfO is almost equivalent to LfD. To further relax the deterministic constraint and better adapt to the practical environment, we consider bounded randomness in the robot environment and prove that the optimizing targets for both LfD and LfO remain almost same in the more generalized setting. Extensive experiments for multiple robot tasks are conducted to empirically demonstrate that LfO achieves comparable performance to LfD. In fact, most common robot systems in reality are the robot environment with bounded randomness (i.e., the environment this paper considered). Hence, our findings greatly extend the potential of LfO and suggest that we can safely apply LfO without sacrificing the performance compared to LfD in practice.
Volumetric Calculation of Quantization Error in 3-D Vision Systems
Venue: arXiv cs.CV 2020
Year: 2020

This paper investigates how the inherent quantization of camera sensors introduces uncertainty in the calculated position of an observed feature during 3-D mapping. It is typically assumed that pixels and scene features are points, however, a pixel is a two-dimensional area that maps onto multiple points in the scene. This uncertainty region is a bound for quantization error in the calculated point positions. Earlier studies calculated the volume of two intersecting pixel views, approximated as a cuboid, by projecting pyramids and cones from the pixels into the scene. In this paper, we reverse this approach by generating an array of scene points and calculating which scene points are detected by which pixel in each camera. This enables us to map the uncertainty regions for every pixel correspondence for a given camera system in one calculation, without approximating the complex shapes. The dependence of the volumes of the uncertainty regions on camera baseline length, focal length, pixel size, and distance to object, shows that earlier studies overestimated the quantization error by at least a factor of two. For static camera systems the method can also be used to determine volumetric scene geometry without the need to calculate disparity maps.
Few-shot model-based adaptation in noisy conditions
Venue: arXiv cs.LG 2020
Year: 2020

Few-shot adaptation is a challenging problem in the context of simulation-to-real transfer in robotics, requiring safe and informative data collection. In physical systems, additional challenge may be posed by domain noise, which is present in virtually all real-world applications. In this paper, we propose to perform few-shot adaptation of dynamics models in noisy conditions using an uncertainty-aware Kalman filter-based neural network architecture. We show that the proposed method, which explicitly addresses domain noise, improves few-shot adaptation error over a blackbox adaptation LSTM baseline, and over a model-free on-policy reinforcement learning approach, which tries to learn an adaptable and informative policy at the same time. The proposed method also allows for system analysis by analyzing hidden states of the model during and after adaptation.
Really Embedding Domain-Specific Languages into C++
Venue: arXiv cs.PL 2020
Year: 2020

Domain-specific languages (DSLs) are both pervasive and powerful, but remain difficult to integrate into large projects. As a result, while DSLs can bring distinct advantages in performance, reliability, and maintainability, their use often involves trading off other good software-engineering practices. In this paper, we describe an extension to the Clang C++ compiler to support syntax plugins, and we demonstrate how this mechanism allows making use of DSLs inside of a C++ code base without needing to separate the DSL source code from the surrounding C++ code.
Barrington Plays Cards: The Complexity of Card-based Protocols
Venue: arXiv cs.CC 2020
Year: 2020

In this paper we study the computational complexity of functions that have efficient card-based protocols. Card-based protocols were proposed by den Boer [EUROCRYPT '89] as a means for secure two-party computation. Our contribution is two-fold: We classify a large class of protocols with respect to the computational complexity of functions they compute, and we propose other encodings of inputs which require fewer cards than the usual 2-card representation.
Pitfalls of Provably Secure Systems in Internet The Case of Chronos-NTP
Venue: arXiv cs.CR 2020
Year: 2020

The critical role that Network Time Protocol (NTP) plays in the Internet led to multiple efforts to secure it against time-shifting attacks. A recent proposal for enhancing the security of NTP with Chronos against on-path attackers seems the most promising one and is on a standardisation track of the IETF. In this work we demonstrate off-path attacks against Chronos enhanced NTP clients. The weak link is a central security feature of Chronos: The server pool generation mechanism using DNS. We show that the insecurity of DNS allows to subvert the security of Chronos making the time-shifting attacks against Chronos-NTP even easier than attacks against plain NTP.
Position paper: A systematic framework for categorising IoT device fingerprinting mechanisms
Venue: arXiv cs.NI 2020
Year: 2020

The popularity of the Internet of Things (IoT) devices makes it increasingly important to be able to fingerprint them, for example in order to detect if there are misbehaving or even malicious IoT devices in one's network. The aim of this paper is to provide a systematic categorisation of machine learning augmented techniques that can be used for fingerprinting IoT devices. This can serve as a baseline for comparing various IoT fingerprinting mechanisms, so that network administrators can choose one or more mechanisms that are appropriate for monitoring and maintaining their network. We carried out an extensive literature review of existing papers on fingerprinting IoT devices -- paying close attention to those with machine learning features. This is followed by an extraction of important and comparable features among the mechanisms outlined in those papers. As a result, we came up with a key set of terminologies that are relevant both in the fingerprinting context and in the IoT domain. This enabled us to construct a framework called IDWork, which can be used for categorising existing IoT fingerprinting mechanisms in a way that will facilitate a coherent and fair comparison of these mechanisms. We found that the majority of the IoT fingerprinting mechanisms take a passive approach -- mainly through network sniffing -- instead of being intrusive and interactive with the device of interest. Additionally, a significant number of the surveyed mechanisms employ both static and dynamic approaches, in order to benefit from complementary features that can be more robust against certain attacks such as spoofing and replay attacks.
For self-supervised learning, Rationality implies generalization, provably
Venue: arXiv cs.LG 2020
Year: 2020

We prove a new upper bound on the generalization gap of classifiers that are obtained by first using self-supervision to learn a representation $r$ of the training data, and then fitting a simple (e.g., linear) classifier $g$ to the labels. Specifically, we show that (under the assumptions described below) the generalization gap of such classifiers tends to zero if $\mathsf{C}(g) \ll n$, where $\mathsf{C}(g)$ is an appropriately-defined measure of the simple classifier $g$'s complexity, and $n$ is the number of training samples. We stress that our bound is independent of the complexity of the representation $r$. We do not make any structural or conditional-independence assumptions on the representation-learning task, which can use the same training dataset that is later used for classification. Rather, we assume that the training procedure satisfies certain natural noise-robustness (adding small amount of label noise causes small degradation in performance) and rationality (getting the wrong label is not better than getting no label at all) conditions that widely hold across many standard architectures. We show that our bound is non-vacuous for many popular representation-learning based classifiers on CIFAR-10 and ImageNet, including SimCLR, AMDIM and MoCo.
Detecting Objectifying Language in Online Professor Reviews
Venue: arXiv cs.CL 2020
Year: 2020

Student reviews often make reference to professors' physical appearances. Until recently RateMyProfessors.com, the website of this study's focus, used a design feature to encourage a "hot or not" rating of college professors. In the wake of recent #MeToo and #TimesUp movements, social awareness of the inappropriateness of these reviews has grown; however, objectifying comments remain and continue to be posted in this online context. We describe two supervised text classifiers for detecting objectifying commentary in professor reviews. We then ensemble these classifiers and use the resulting model to track objectifying commentary at scale. We measure correlations between objectifying commentary, changes to the review website interface, and teacher gender across a ten-year period.
To Lane or Not to Lane? Comparing On-Road Experiences in Developing and Developed Countries using a New Simulator "RoadBird"
Venue: arXiv cs.OH 2020
Year: 2020

Even though the traffic systems in developed countries have been analyzed with rigor and operated efficiently, the same does not generally hold for developing countries due to inadequate planning, design, and operations of their transportation systems. Because of inherent differences between internal infrastructures, the systems deployed in developed countries may not be amenable to developing ones. Besides, the traffic systems of developing countries are not well-studied in the literature to the best of our knowledge. For example, it is yet to explore how a developed country's lane-based traffic flow would perform in the context of a developing country, which generally experiences non-lane-based traffic. As such, by using our newly developed traffic simulator 'RoadBird', we investigate outcomes of both lane-based and non-lane-based traffic from the contexts of both developing and developed countries. To do so, we run simulations over real road topologies (extracted from the GIS maps of major cities such as Dhaka, Miami, and Riyadh) considering different scenarios such as lane-based or non-lane-based flows, homogeneous or heterogeneous traffic, with or without pedestrians, etc. We also incorporate different car-following and lane-changing models to mimic traffic behaviors and investigate their performances. While the lane changing dilemma remains an open research question, our experimental evidences indicate: (i) lane-based approaches will not necessarily perform better in the case of currently-adopted non-lane-based scenarios; and (ii) non-lane-based strategies may benefit system performance in lane-based scenarios while having heavy mixed traffic. Nonetheless, we reveal several new insights for on-road experiences both in developing and developed countries.
Flow-FL: Data-Driven Federated Learning for Spatio-Temporal Predictions in Multi-Robot Systems
Venue: arXiv cs.RO 2020
Year: 2020

In this paper, we show how the Federated Learning (FL) framework enables learning collectively from distributed data in connected robot teams. This framework typically works with clients collecting data locally, updating neural network weights of their model, and sending updates to a server for aggregation into a global model. We explore the design space of FL by comparing two variants of this concept. The first variant follows the traditional FL approach in which a server aggregates the local models. In the second variant, that we call Flow-FL, the aggregation process is serverless thanks to the use of a gossip-based shared data structure. In both variants, we use a data-driven mechanism to synchronize the learning process in which robots contribute model updates when they collect sufficient data. We validate our approach with an agent trajectory forecasting problem in a multi-agent setting. Using a centralized implementation as a baseline, we study the effects of staggered online data collection, and variations in data flow, number of participating robots, and time delays introduced by the decentralization of the framework in a multi-robot setting.
A case where a spindly two-layer linear network whips any neural network with a fully connected input layer
Venue: arXiv cs.LG 2020
Year: 2020

It was conjectured that any neural network of any structure and arbitrary differentiable transfer functions at the nodes cannot learn the following problem sample efficiently when trained with gradient descent: The instances are the rows of a $d$-dimensional Hadamard matrix and the target is one of the features, i.e. very sparse. We essentially prove this conjecture: We show that after receiving a random training set of size $k < d$, the expected square loss is still $1-\frac{k}{(d-1)}$. The only requirement needed is that the input layer is fully connected and the initial weight vectors of the input nodes are chosen from a rotation invariant distribution. Surprisingly the same type of problem can be solved drastically more efficient by a simple 2-layer linear neural network in which the $d$ inputs are connected to the output node by chains of length 2 (Now the input layer has only one edge per input). When such a network is trained by gradient descent, then it has been shown that its expected square loss is $\frac{\log d}{k}$. Our lower bounds essentially show that a sparse input layer is needed to sample efficiently learn sparse targets with gradient descent when the number of examples is less than the number of input features.
Proportional Representation under Single-Crossing Preferences Revisited
Venue: arXiv cs.GT 2020
Year: 2020

We study the complexity of determining a winning committee under the Chamberlin--Courant voting rule when voters' preferences are single-crossing on a line, or, more generally, on a median graph (this class of graphs includes, e.g., trees and grids). For the line, Skowron et al. (2015) describe an $O(n^2mk)$ algorithm (where $n$, $m$, $k$ are the number of voters, the number of candidates and the committee size, respectively); we show that a simple tweak improves the time complexity to $O(nmk)$. We then improve this bound for $k=Ω(\log n)$ by reducing our problem to the $k$-link path problem for DAGs with concave Monge weights, obtaining a $nm2^{O\left(\sqrt{\log k\log\log n}\right)}$ algorithm for the general case and a nearly linear algorithm for the Borda misrepresentation function. For trees, we point out an issue with the algorithm proposed by Clearwater, Puppe and Slinko (2015), and develop a $O(nmk)$ algorithm for this case as well. For grids, we formulate a conjecture about the structure of optimal solutions, and describe a polynomial-time algorithm that finds a winning committee if this conjecture is true; we also explain how to convert this algorithm into a bicriterial approximation algorithm whose correctness does not depend on the conjecture.
Zoom-CAM: Generating Fine-grained Pixel Annotations from Image Labels
Venue: arXiv cs.CV 2020
Year: 2020

Current weakly supervised object localization and segmentation rely on class-discriminative visualization techniques to generate pseudo-labels for pixel-level training. Such visualization methods, including class activation mapping (CAM) and Grad-CAM, use only the deepest, lowest resolution convolutional layer, missing all information in intermediate layers. We propose Zoom-CAM: going beyond the last lowest resolution layer by integrating the importance maps over all activations in intermediate layers. Zoom-CAM captures fine-grained small-scale objects for various discriminative class instances, which are commonly missed by the baseline visualization methods. We focus on generating pixel-level pseudo-labels from class labels. The quality of our pseudo-labels evaluated on the ImageNet localization task exhibits more than 2.8% improvement on top-1 error. For weakly supervised semantic segmentation our generated pseudo-labels improve a state of the art model by 1.1%.
Cross-Lingual Relation Extraction with Transformers
Venue: arXiv cs.CL 2020
Year: 2020

Relation extraction (RE) is one of the most important tasks in information extraction, as it provides essential information for many NLP applications. In this paper, we propose a cross-lingual RE approach that does not require any human annotation in a target language or any cross-lingual resources. Building upon unsupervised cross-lingual representation learning frameworks, we develop several deep Transformer based RE models with a novel encoding scheme that can effectively encode both entity location and entity type information. Our RE models, when trained with English data, outperform several deep neural network based English RE models. More importantly, our models can be applied to perform zero-shot cross-lingual RE, achieving the state-of-the-art cross-lingual RE performance on two datasets (68-89% of the accuracy of the supervised target-language RE model). The high cross-lingual transfer efficiency without requiring additional training data or cross-lingual resources shows that our RE models are especially useful for low-resource languages.
Long-Term Face Tracking for Crowded Video-Surveillance Scenarios
Venue: arXiv cs.CV 2020
Year: 2020

Most current multi-object trackers focus on short-term tracking, and are based on deep and complex systems that do not operate in real-time, often making them impractical for video-surveillance. In this paper, we present a long-term multi-face tracking architecture conceived for working in crowded contexts, particularly unconstrained in terms of movement and occlusions, and where the face is often the only visible part of the person. Our system benefits from advances in the fields of face detection and face recognition to achieve long-term tracking. It follows a tracking-by-detection approach, combining a fast short-term visual tracker with a novel online tracklet reconnection strategy grounded on face verification. Additionally, a correction module is included to correct past track assignments with no extra computational cost. We present a series of experiments introducing novel, specialized metrics for the evaluation of long-term tracking capabilities and a video dataset that we publicly release. Findings demonstrate that, in this context, our approach allows to obtain up to 50% longer tracks than state-of-the-art deep learning trackers.
Picture-to-Amount (PITA): Predicting Relative Ingredient Amounts from Food Images
Venue: arXiv cs.CV 2020
Year: 2020

Increased awareness of the impact of food consumption on health and lifestyle today has given rise to novel data-driven food analysis systems. Although these systems may recognize the ingredients, a detailed analysis of their amounts in the meal, which is paramount for estimating the correct nutrition, is usually ignored. In this paper, we study the novel and challenging problem of predicting the relative amount of each ingredient from a food image. We propose PITA, the Picture-to-Amount deep learning architecture to solve the problem. More specifically, we predict the ingredient amounts using a domain-driven Wasserstein loss from image-to-recipe cross-modal embeddings learned to align the two views of food data. Experiments on a dataset of recipes collected from the Internet show the model generates promising results and improves the baselines on this challenging task. A demo of our system and our data is availableat: foodai.cs.rutgers.edu.
Printmaking, Puzzles, and Studio Closets: Using Artistic Metaphors to Reimagine the User Interface for Designing Immersive Visualizations
Venue: arXiv cs.HC 2020
Year: 2020

We, as a society, need artists to help us interpret and explain science, but what does an artist's studio look like when today's science is built upon the language of large, increasingly complex data? This paper presents a data visualization design interface that lifts the barriers for artists to engage with actively studied, 3D multivariate datasets. To accomplish this, the interface must weave together the need for creative artistic processes and the challenging constraints of real-time, data-driven 3D computer graphics. The result is an interface for a technical process, but technical in the way artistic printmaking is technical, not in the sense of computer scripting and programming. Using metaphor, computer graphics algorithms and shader program parameters are reimagined as tools in an artist's printmaking studio. These artistic metaphors and language are merged with a puzzle-piece approach to visual programming and matching iconography. Finally, artists access the interface using a web browser, making it possible to design immersive multivariate data visualizations that can be displayed in VR and AR environments using familiar drawing tablets and touch screens. We report on insights from the interdisciplinary design of the interface and early feedback from artists.
Deep Learning in the Era of Edge Computing: Challenges and Opportunities
Venue: arXiv cs.LG 2020
Year: 2020

The era of edge computing has arrived. Although the Internet is the backbone of edge computing, its true value lies at the intersection of gathering data from sensors and extracting meaningful information from the sensor data. We envision that in the near future, majority of edge devices will be equipped with machine intelligence powered by deep learning. However, deep learning-based approaches require a large volume of high-quality data to train and are very expensive in terms of computation, memory, and power consumption. In this chapter, we describe eight research challenges and promising opportunities at the intersection of computer systems, networking, and machine learning. Solving those challenges will enable resource-limited edge devices to leverage the amazing capability of deep learning. We hope this chapter could inspire new research that will eventually lead to the realization of the vision of intelligent edge.