OpenRaiser/Intern-Atlas huggingface dataset:
A Little Charity Guarantees Almost Envy-Freeness
Venue: arXiv cs.GT 2020
Year: 2020

Fair division of indivisible goods is a very well-studied problem. The goal of this problem is to distribute $m$ goods to $n$ agents in a "fair" manner, where every agent has a valuation for each subset of goods. We assume general valuations. Envy-freeness is the most extensively studied notion of fairness. However, envy-free allocations do not always exist when goods are indivisible. The notion of fairness we consider here is "envy-freeness up to any good" (EFX) where no agent envies another agent after the removal of any single good from the other agent's bundle. It is not known if such an allocation always exists even when $n=3$. We show there is always a partition of the set of goods into $n+1$ subsets $(X_1,\ldots,X_n,P)$ where for $i \in [n]$, $X_i$ is the bundle allocated to agent $i$ and the set $P$ is unallocated (or donated to charity) such that we have$\colon$ 1) envy-freeness up to any good, 2) no agent values $P$ higher than her own bundle, and 3) fewer than $n$ goods go to charity, i.e., $|P| < n$ (typically $m \gg n$). Our proof is constructive. When agents have additive valuations and $\lvert P \rvert$ is large (i.e., when $|P|$ is close to $n$), our allocation also has a good maximin share (MMS) guarantee. Moreover, a minor variant of our algorithm also shows the existence of an allocation which is $4/7$ groupwise maximin share (GMMS): this is a notion of fairness stronger than MMS. This improves upon the current best bound of $1/2$ known for an approximate GMMS allocation.
Entropy and Compression: A simple proof of an inequality of Khinchin-Ornstein-Shields
Venue: arXiv cs.IT 2020
Year: 2020

This paper concerns the folklore statement that ``entropy is a lower bound for compression''. More precisely we derive from the entropy theorem a simple proof of a pointwise inequality firstly stated by Ornstein and Shields and which is the almost-sure version of an average inequality firstly stated by Khinchin in 1953. We further give an elementary proof of original Khinchin inequality that can be used as an exercise for Information Theory students and we conclude by giving historical and technical notes of such inequality.
Fast Modeling L2 Cache Reuse Distance Histograms Using Combined Locality Information from Software Traces
Venue: arXiv cs.AR 2020
Year: 2020

To mitigate the performance gap between CPU and the main memory, multi-level cache architectures are widely used in modern processors. Therefore, modeling the behaviors of the downstream caches becomes a critical part of the processor performance evaluation in the early stage of Design Space Exploration (DSE). In this paper, we propose a fast and accurate L2 cache reuse distance histogram model, which can be used to predict the behaviors of the multi-level cache architectures where the L1 cache uses the LRU replacement policy and the L2 cache uses LRU/Random replacement policies. We use the profiled L1 reuse distance histogram and two newly proposed metrics, namely the RST table and the Hit-RDH, that describing more detailed information of the software traces as the inputs. For a given L1 cache configuration, the profiling results can be reused for different configurations of the L2 cache. The output of our model is the L2 cache reuse distance histogram, based on which the L2 cache miss rates can be evaluated. We compare the L2 cache miss rates with the results from gem5 cycle-accurate simulations of 15 benchmarks chosen from SPEC CPU 2006 and 9 benchmarks from SPEC CPU 2017. The average absolute error is less than 5%, while the evaluation time for each L2 configuration can be sped up almost 30X for four L2 cache candidates.
Error Exponents for Asynchronous Multiple Access Channels. Controlled Asynchronism may Outperform Synchronism
Venue: arXiv cs.IT 2020
Year: 2020

Exponential error bounds achievable by universal coding and decoding are derived for frame-asynchronous discrete memoryless %asynchronous multiple access channels with two senders, via the method of subtypes, a refinement of the method of types. Maximum empirical multi-information decoding is employed. A key tool is an improved packing lemma, that overcomes the technical difficulty caused by codeword repetitions, via an induction based new argument. The asymptotic form of the bounds admits numerical evaluation. This demostrates that error exponents achievable by synchronous transmission (if possible) can be superseeded via controlled asynchronism, i.e. a deliberate shift of the codewords.
Information-Centric Grant-Free Access for IoT Fog Networks: Edge vs Cloud Detection and Learning
Venue: arXiv cs.IT 2020
Year: 2020

A multi-cell Fog-Radio Access Network (F-RAN) architecture is considered in which Internet of Things (IoT) devices periodically make noisy observations of a Quantity of Interest (QoI) and transmit using grant-free access in the uplink. The devices in each cell are connected to an Edge Node (EN), which may also have a finite-capacity fronthaul link to a central processor. In contrast to conventional information-agnostic protocols, the devices transmit using a Type-Based Multiple Access (TBMA) protocol that is tailored to enable the estimate of the field of correlated QoIs in each cell based on the measurements received from IoT devices. In this paper, this form of information-centric radio access is studied for the first time in a multi-cell F-RAN model with edge or cloud detection. Edge and cloud detection are designed and compared for a multi-cell system. Optimal model-based detectors are introduced and the resulting asymptotic behavior of the probability of error at cloud and edge is derived. Then, for the scenario in which a statistical model is not available, data-driven edge and cloud detectors are discussed and evaluated in numerical results.
Recognizing Stick Graphs with and without Length Constraints
Venue: arXiv cs.CG 2020
Year: 2020

Stick graphs are intersection graphs of horizontal and vertical line segments that all touch a line of slope -1 and lie above this line. De Luca et al. [GD'18] considered the recognition problem of stick graphs when no order is given (STICK), when the order of either one of the two sets is given (STICK_A), and when the order of both sets is given (STICK_AB). They showed how to solve STICK_AB efficiently. In this paper, we improve the running time of their algorithm, and we solve STICK_A efficiently. Further, we consider variants of these problems where the lengths of the sticks are given as input. We show that these variants of STICK, STICK_A, and STICK_AB are all NP-complete. On the positive side, we give an efficient solution for STICK_AB with fixed stick lengths if there are no isolated vertices.
Minimizers of the Empirical Risk and Risk Monotonicity
Venue: arXiv cs.LG 2020
Year: 2020

Plotting a learner's average performance against the number of training samples results in a learning curve. Studying such curves on one or more data sets is a way to get to a better understanding of the generalization properties of this learner. The behavior of learning curves is, however, not very well understood and can display (for most researchers) quite unexpected behavior. Our work introduces the formal notion of \emph{risk monotonicity}, which asks the risk to not deteriorate with increasing training set sizes in expectation over the training samples. We then present the surprising result that various standard learners, specifically those that minimize the empirical risk, can act \emph{non}monotonically irrespective of the training sample size. We provide a theoretical underpinning for specific instantiations from classification, regression, and density estimation. Altogether, the proposed monotonicity notion opens up a whole new direction of research.
Graph-Structured Visual Imitation
Venue: arXiv cs.RO 2020
Year: 2020

We cast visual imitation as a visual correspondence problem. Our robotic agent is rewarded when its actions result in better matching of relative spatial configurations for corresponding visual entities detected in its workspace and teacher's demonstration. We build upon recent advances in Computer Vision,such as human finger keypoint detectors, object detectors trained on-the-fly with synthetic augmentations, and point detectors supervised by viewpoint changes and learn multiple visual entity detectors for each demonstration without human annotations or robot interactions. We empirically show the proposed factorized visual representations of entities and their spatial arrangements drive successful imitation of a variety of manipulation skills within minutes, using a single demonstration and without any environment instrumentation. It is robust to background clutter and can effectively generalize across environment variations between demonstrator and imitator, greatly outperforming unstructured non-factorized full-frame CNN encodings of previous works.
Laplacian-regularized graph bandits: Algorithms and theoretical analysis
Venue: arXiv cs.LG 2020
Year: 2020

We consider a stochastic linear bandit problem with multiple users, where the relationship between users is captured by an underlying graph and user preferences are represented as smooth signals on the graph. We introduce a novel bandit algorithm where the smoothness prior is imposed via the random-walk graph Laplacian, which leads to a single-user cumulative regret scaling as $\tilde{\mathcal{O}}(Ψd \sqrt{T})$ with time horizon $T$, feature dimensionality $d$, and the scalar parameter $Ψ\in (0,1)$ that depends on the graph connectivity. This is an improvement over $\tilde{\mathcal{O}}(d \sqrt{T})$ in \algo{LinUCB}~\Ccite{li2010contextual}, where user relationship is not taken into account. In terms of network regret (sum of cumulative regret over $n$ users), the proposed algorithm leads to a scaling as $\tilde{\mathcal{O}}(Ψd\sqrt{nT})$, which is a significant improvement over $\tilde{\mathcal{O}}(nd\sqrt{T})$ in the state-of-the-art algorithm \algo{Gob.Lin} \Ccite{cesa2013gang}. To improve scalability, we further propose a simplified algorithm with a linear computational complexity with respect to the number of users, while maintaining the same regret. Finally, we present a finite-time analysis on the proposed algorithms, and demonstrate their advantage in comparison with state-of-the-art graph-based bandit algorithms on both synthetic and real-world data.
And the Bit Goes Down: Revisiting the Quantization of Neural Networks
Venue: arXiv cs.CV 2020
Year: 2020

In this paper, we address the problem of reducing the memory footprint of convolutional network architectures. We introduce a vector quantization method that aims at preserving the quality of the reconstruction of the network outputs rather than its weights. The principle of our approach is that it minimizes the loss reconstruction error for in-domain inputs. Our method only requires a set of unlabelled data at quantization time and allows for efficient inference on CPU by using byte-aligned codebooks to store the compressed weights. We validate our approach by quantizing a high performing ResNet-50 model to a memory size of 5MB (20x compression factor) while preserving a top-1 accuracy of 76.1% on ImageNet object classification and by compressing a Mask R-CNN with a 26x factor.
Order and Chaos: NTK views on DNN Normalization, Checkerboard and Boundary Artifacts
Venue: arXiv cs.LG 2020
Year: 2020

We analyze architectural features of Deep Neural Networks (DNNs) using the so-called Neural Tangent Kernel (NTK), which describes the training and generalization of DNNs in the infinite-width setting. In this setting, we show that for fully-connected DNNs, as the depth grows, two regimes appear: "order", where the (scaled) NTK converges to a constant, and "chaos", where it converges to a Kronecker delta. Extreme order slows down training while extreme chaos hinders generalization. Using the scaled ReLU as a nonlinearity, we end up in the ordered regime. In contrast, Layer Normalization brings the network into the chaotic regime. We observe a similar effect for Batch Normalization (BN) applied after the last nonlinearity. We uncover the same order and chaos modes in Deep Deconvolutional Networks (DC-NNs). Our analysis explains the appearance of so-called checkerboard patterns and border artifacts. Moving the network into the chaotic regime prevents checkerboard patterns; we propose a graph-based parametrization which eliminates border artifacts; finally, we introduce a new layer-dependent learning rate to improve the convergence of DC-NNs. We illustrate our findings on DCGANs: the ordered regime leads to a collapse of the generator to a checkerboard mode, which can be avoided by tuning the nonlinearity to reach the chaotic regime. As a result, we are able to obtain good quality samples for DCGANs without BN.
Online learning for min-max discrete problems
Venue: arXiv cs.DS 2020
Year: 2020

We study various discrete nonlinear combinatorial optimization problems in an online learning framework. In the first part, we address the question of whether there are negative results showing that getting a vanishing (or even vanishing approximate) regret is computational hard. We provide a general reduction showing that many (min-max) polynomial time solvable problems not only do not have a vanishing regret, but also no vanishing approximation $α$-regret, for some $α$ (unless $NP=BPP$). Then, we focus on a particular min-max problem, the min-max version of the vertex cover problem which is solvable in polynomial time in the offline case. The previous reduction proves that there is no $(2-ε)$-regret online algorithm, unless Unique Game is in $BPP$; we prove a matching upper bound providing an online algorithm based on the online gradient descent method. Then, we turn our attention to online learning algorithms that are based on an offline optimization oracle that, given a set of instances of the problem, is able to compute the optimum static solution. We show that for different nonlinear discrete optimization problems, it is strongly $NP$-hard to solve the offline optimization oracle, even for problems that can be solved in polynomial time in the static case (e.g. min-max vertex cover, min-max perfect matching, etc.). On the positive side, we present an online algorithm with vanishing regret that is based on the follow the perturbed leader algorithm for a generalized knapsack problem.
Motion Sensor-based Privacy Attack on Smartphones
Venue: arXiv cs.CR 2020
Year: 2020

In this paper, we build a speech privacy attack that exploits speech reverberations generated from a smartphone's in-built loudspeaker captured via a zero-permission motion sensor (accelerometer). We design our attack Spearphone2, and demonstrate that speech reverberations from inbuilt loudspeakers, at an appropriate loudness, can impact the accelerometer, leaking sensitive information about the speech. In particular, we show that by exploiting the affected accelerometer readings and carefully selecting feature sets along with off-the-shelf machine learning techniques, Spearphone can successfully perform gender classification (accuracy over 90%) and speaker identification (accuracy over 80%) for any audio/video playback on the smartphone. Our results with testing the attack on a voice call and voice assistant response were also encouraging, showcasing the impact of the proposed attack. In addition, we perform speech recognition and speech reconstruction to extract more information about the eavesdropped speech to an extent. Our work brings to light a fundamental design vulnerability in many currently-deployed smartphones, which may put people's speech privacy at risk while using the smartphone in the loudspeaker mode during phone calls, media playback or voice assistant interactions.
Motion Planning Networks: Bridging the Gap Between Learning-based and Classical Motion Planners
Venue: arXiv cs.RO 2020
Year: 2020

This paper describes Motion Planning Networks (MPNet), a computationally efficient, learning-based neural planner for solving motion planning problems. MPNet uses neural networks to learn general near-optimal heuristics for path planning in seen and unseen environments. It takes environment information such as raw point-cloud from depth sensors, as well as a robot's initial and desired goal configurations and recursively calls itself to bidirectionally generate connectable paths. In addition to finding directly connectable and near-optimal paths in a single pass, we show that worst-case theoretical guarantees can be proven if we merge this neural network strategy with classical sample-based planners in a hybrid approach while still retaining significant computational and optimality improvements. To train the MPNet models, we present an active continual learning approach that enables MPNet to learn from streaming data and actively ask for expert demonstrations when needed, drastically reducing data for training. We validate MPNet against gold-standard and state-of-the-art planning methods in a variety of problems from 2D to 7D robot configuration spaces in challenging and cluttered environments, with results showing significant and consistently stronger performance metrics, and motivating neural planning in general as a modern strategy for solving motion planning problems efficiently.
Hybrid Offline-Online Design for UAV-Enabled Data Harvesting in Probabilistic LoS Channel
Venue: arXiv cs.IT 2020
Year: 2020

This paper considers an unmanned aerial vehicle (UAV)-enabled wireless sensor network (WSN) in urban areas, where a UAV is deployed to collect data from distributed sensor nodes (SNs) within a given duration. To characterize the occasional building blockage between the UAV and SNs, we construct the probabilistic line-of-sight (LoS) channel model for a Manhattan-type city by using the combined simulation and data regression method, which is shown in the form of a generalized logistic function of the UAV-SN elevation angle. We assume that only the knowledge of SNs' locations and the probabilistic LoS channel model is known a priori, while the UAV can obtain the instantaneous LoS/Non-LoS channel state information (CSI) with the SNs in real time along its flight. Our objective is to maximize the minimum (average) data collection rate from all the SNs for the UAV. To this end, we formulate a new rate maximization problem by jointly optimizing the UAV three-dimensional (3D) trajectory and transmission scheduling of SNs. Although the optimal solution is intractable due to the lack of the complete UAV-SNs CSI, we propose in this paper a novel and general design method, called hybrid offline-online optimization, to obtain a suboptimal solution to it, by leveraging both the statistical and real-time CSI. Essentially, our proposed method decouples the joint design of UAV trajectory and communication scheduling into two phases: namely, an offline phase that determines the UAV path prior to its flight based on the probabilistic LoS channel model, followed by an online phase that adaptively adjusts the UAV flying speeds along the offline optimized path as well as communication scheduling based on the instantaneous UAV-SNs CSI and SNs' individual amounts of data received accumulatively.
Representing Attitudes Towards Ambiguity in Hilbert Space: Foundations and Applications
Venue: arXiv cs.AI 2020
Year: 2020

We provide here a general mathematical framework to model attitudes towards ambiguity which uses the formalism of quantum theory as a ``purely mathematical formalism, detached from any physical interpretation''. We show that the quantum-theoretic framework enables modelling of the "Ellsberg paradox", but it also successfully applies to more concrete human decision-making (DM) tests involving financial, managerial and medical decisions. In particular, we elaborate a mathematical representation of various empirical studies which reveal that attitudes of managers towards uncertainty shift from "ambiguity seeking" to "ambiguity aversion", and viceversa, thus exhibiting "hope effects" and "fear effects". The present framework provides a promising direction towards the development of a unified theory of decisions in the presence of uncertainty.
Random Projections and Sampling Algorithms for Clustering of High-Dimensional Polygonal Curves
Venue: arXiv cs.LG 2020
Year: 2020

We study the $k$-median clustering problem for high-dimensional polygonal curves with finite but unbounded number of vertices. We tackle the computational issue that arises from the high number of dimensions by defining a Johnson-Lindenstrauss projection for polygonal curves. We analyze the resulting error in terms of the Fréchet distance, which is a tractable and natural dissimilarity measure for curves. Our clustering algorithms achieve sublinear dependency on the number of input curves via subsampling. Also, we show that the Fréchet distance can not be approximated within any factor of less than $\sqrt{2}$ by probabilistically reducing the dependency on the number of vertices of the curves. As a consequence we provide a fast, CUDA-parallelized version of the Alt and Godau algorithm for computing the Fréchet distance and use it to evaluate our results empirically.
Adaptive Prior Selection for Repertoire-based Online Adaptation in Robotics
Venue: arXiv cs.RO 2020
Year: 2020

Repertoire-based learning is a data-efficient adaptation approach based on a two-step process in which (1) a large and diverse set of policies is learned in simulation, and (2) a planning or learning algorithm chooses the most appropriate policies according to the current situation (e.g., a damaged robot, a new object, etc.). In this paper, we relax the assumption of previous works that a single repertoire is enough for adaptation. Instead, we generate repertoires for many different situations (e.g., with a missing leg, on different floors, etc.) and let our algorithm selects the most useful prior. Our main contribution is an algorithm, APROL (Adaptive Prior selection for Repertoire-based Online Learning) to plan the next action by incorporating these priors when the robot has no information about the current situation. We evaluate APROL on two simulated tasks: (1) pushing unknown objects of various shapes and sizes with a robotic arm and (2) a goal reaching task with a damaged hexapod robot. We compare with "Reset-free Trial and Error" (RTE) and various single repertoire-based baselines. The results show that APROL solves both the tasks in less interaction time than the baselines. Additionally, we demonstrate APROL on a real, damaged hexapod that quickly learns to pick compensatory policies to reach a goal by avoiding obstacles in the path.
OmniNet: A unified architecture for multi-modal multi-task learning
Venue: arXiv cs.LG 2020
Year: 2020

Transformer is a popularly used neural network architecture, especially for language understanding. We introduce an extended and unified architecture that can be used for tasks involving a variety of modalities like image, text, videos, etc. We propose a spatio-temporal cache mechanism that enables learning spatial dimension of the input in addition to the hidden states corresponding to the temporal input sequence. The proposed architecture further enables a single model to support tasks with multiple input modalities as well as asynchronous multi-task learning, thus we refer to it as OmniNet. For example, a single instance of OmniNet can concurrently learn to perform the tasks of part-of-speech tagging, image captioning, visual question answering and video activity recognition. We demonstrate that training these four tasks together results in about three times compressed model while retaining the performance in comparison to training them individually. We also show that using this neural network pre-trained on some modalities assists in learning unseen tasks such as video captioning and video question answering. This illustrates the generalization capacity of the self-attention mechanism on the spatio-temporal cache present in OmniNet.