OpenRaiser/Intern-Atlas huggingface dataset:
Fine-grained Interpretation and Causation Analysis in Deep NLP Models
Venue: arXiv cs.CL 2021
Year: 2021

This paper is a write-up for the tutorial on "Fine-grained Interpretation and Causation Analysis in Deep NLP Models" that we are presenting at NAACL 2021. We present and discuss the research work on interpreting fine-grained components of a model from two perspectives, i) fine-grained interpretation, ii) causation analysis. The former introduces methods to analyze individual neurons and a group of neurons with respect to a language property or a task. The latter studies the role of neurons and input features in explaining decisions made by the model. We also discuss application of neuron analysis such as network manipulation and domain adaptation. Moreover, we present two toolkits namely NeuroX and Captum, that support functionalities discussed in this tutorial.
Dynamic Proportional Rankings
Venue: arXiv cs.GT 2021
Year: 2021

Proportional ranking rules aggregate approval-style preferences of agents into a collective ranking such that groups of agents with similar preferences are adequately represented. Motivated by the application of live Q&A platforms, where submitted questions need to be ranked based on the interests of the audience, we study a dynamic extension of the proportional rankings setting. In our setting, the goal is to maintain the proportionality of a ranking when alternatives (i.e., questions) -- not necessarily from the top of the ranking -- get selected sequentially. We propose generalizations of well-known aggregation rules to this setting and study their monotonicity and proportionality properties. We also evaluate the performance of these rules experimentally, using realistic probabilistic assumptions on the selection procedure.
A Review of Intrusion Detection Systems and Their Evaluation in the IoT
Venue: arXiv cs.CR 2021
Year: 2021

Intrusion Detection Systems (IDS) are key components for securing critical infrastructures, capable of detecting malicious activities on networks or hosts. The procedure of implementing a IDS for Internet of Things (IoT) networks is not without challenges due to the variability of these systems and specifically the difficulty in accessing data. The specifics of these very constrained devices render the design of an IDS capable of dealing with the varied attacks a very challenging problem and a very active research subject. In the current state of literature, a number of approaches have been proposed to improve the efficiency of intrusion detection, catering to some of these limitations, such as resource constraints and mobility. In this article, we review works on IDS specifically for these kinds of devices from 2008 to 2018, collecting a total of 51 different IDS papers. We summarise the current themes of the field, summarise the techniques employed to train and deploy the IDSs and provide a qualitative evaluations of these approaches. While these works provide valuable insights and solutions for sub-parts of these constraints, we discuss the limitations of these solutions as a whole, in particular what kinds of attacks these approaches struggle to detect and the setup limitations that are unique to this kind of system. We find that although several paper claim novelty of their approach little inter paper comparisons have been made, that there is a dire need for sharing of datasets and almost no shared code repositories, consequently raising the need for a thorough comparative evaluation.
Multi-Modal Image Captioning for the Visually Impaired
Venue: arXiv cs.CL 2021
Year: 2021

One of the ways blind people understand their surroundings is by clicking images and relying on descriptions generated by image captioning systems. Current work on captioning images for the visually impaired do not use the textual data present in the image when generating captions. This problem is critical as many visual scenes contain text. Moreover, up to 21% of the questions asked by blind people about the images they click pertain to the text present in them. In this work, we propose altering AoANet, a state-of-the-art image captioning model, to leverage the text detected in the image as an input feature. In addition, we use a pointer-generator mechanism to copy the detected text to the caption when tokens need to be reproduced accurately. Our model outperforms AoANet on the benchmark dataset VizWiz, giving a 35% and 16.2% performance improvement on CIDEr and SPICE scores, respectively.
A Coupled Alpha Complex
Venue: arXiv cs.CG 2021
Year: 2021

The alpha complex is a subset of the Delaunay triangulation and is often used in computational geometry and topology. One of the main drawbacks of using the alpha complex is that it is non-monotone, in the sense that if ${\cal X}\subset{\cal X}'$ it is not necessarily (and generically not) the case that the corresponding alpha complexes satisfy ${\cal A}_r({\cal X})\subset{\cal A}_r({\cal X}')$. The lack of monotonicity may introduce significant computational costs when using the alpha complex, and in some cases even render it unusable. In this work we present a new construction based on the alpha complex, that is homotopy equivalent to the alpha complex while maintaining monotonicity. We provide the formal definitions and algorithms required to construct this complex, and to compute its homology. In addition, we analyze the size of this complex in order to argue that it is not significantly more costly to use than the standard alpha complex.
Weakly Private Information Retrieval Under Rényi Divergence
Venue: arXiv cs.IT 2021
Year: 2021

Private information retrieval (PIR) is a protocol that guarantees the privacy of a user who is in communication with databases. The user wants to download one of the messages stored in the databases while hiding the identity of the desired message. Recently, the benefits that can be obtained by weakening the privacy requirement have been studied, but the definition of weak privacy needs to be elaborated upon. In this paper, we attempt to quantify the weak privacy (i.e., information leakage) in PIR problems by using the Rényi divergence that generalizes the Kullback-Leibler divergence. By introducing Rényi divergence into the existing PIR problem, the tradeoff relationship between privacy (information leakage) and PIR performance (download cost) is characterized via convex optimization. Furthermore, we propose an alternative PIR scheme with smaller message sizes than the Tian-Sun-Chen (TSC) scheme. The proposed scheme cannot achieve the PIR capacity of perfect privacy since the message size of the TSC scheme is the minimum to achieve the PIR capacity. However, we show that the proposed scheme can be better than the TSC scheme in the weakly PIR setting, especially under a low download cost regime.
Topology Optimization for Large-Scale Additive Manufacturing: Generating designs tailored to the deposition nozzle size
Venue: arXiv cs.CG 2021
Year: 2021

Additive Manufacturing (AM) processes intended for large scale components deposit large volumes of material to shorten process duration. This reduces the resolution of the AM process, which is typically defined by the size of the deposition nozzle. If the resolution limitation is not considered when designing for Large-Scale Additive Manufacturing (LSAM), difficulties can arise in the manufacturing process, which may require the adaptation of the deposition parameters. This work incorporates the nozzle size constraint into Topology Optimization (TO) in order to generate optimized designs suitable to the process resolution. This article proposes and compares two methods, which are based on existing TO techniques that enable control of minimum and maximum member size, and of minimum cavity size. The first method requires the minimum and maximum member size to be equal to the deposition nozzle size, thus design features of uniform width are obtained in the optimized design. The second method defines the size of the solid members sufficiently small for the resulting structure to resemble a structural skeleton, which can be interpreted as the deposition path. Through filtering and projection techniques, the thin structures are thickened according to the chosen nozzle size. Thus, a topology tailored to the size of the deposition nozzle is obtained along with a deposition proposal. The methods are demonstrated and assessed using 2D and 3D benchmark problems.
Does publicity in the science press drive citations? A vindication of peer review
Venue: arXiv cs.DL 2021
Year: 2021

We study how publicity, in the form of highlighting in the science press, affects the citations of research papers. After a brief review of prior work, we analyze papers published in Physical Review Letters (PRL) that are highlighted across eight different platforms. Using multiple linear regression we identify how each platform contributes to citations. We also analyze how frequently the highlighted papers end up in the top 1% cited papers in their field. We find that the strongest predictors of medium-term citation impact -- up to 7 years post-publication -- are Viewpoints in Physics, followed by Research Highlights in Nature, Editors' Suggestions in PRL, and Research Highlights in Nature Physics. Our key conclusions are that (a) highlighting for importance identifies a citation advantage, which (b) is stratified according to the degree of vetting during peer review (internal and external to the journal). This implies that we can view highlighting platforms as predictors of citation accrual, with varying degrees of strength that mirror each platform's vetting level.
On Using P2P Technology for Decentralized Detection of Service Level Agreement Violations
Venue: arXiv cs.NI 2021
Year: 2021

Critical networked services enable significant revenue for network operators and, in turn, are regulated by Service Level Agreements (SLAs). In order to ensure SLAs are being met, service levels need to be monitored. One technique for this involves active measurement mechanisms which employ measurement probes along the network to inject synthetic traffic and compute the network performance. However, these mechanisms are expensive in terms of resources consumption. Thus, these mechanisms usually can cover only a fraction of what could be measured, which can lead to SLA violations being missed. Besides that, the definition of this fraction is a practice done by human administrators, which does not scale well and does not adapt to highly dynamic networking patterns. In this article, we examine the potential benefits of using P2P technology to improve the detection of SLA Violations. We first describe the principles of a P2P-based steering of active measurement mechanisms. These principles are characterized by a high degree of decentralized decision making across a network using a self-organizing overlay. In a second step, we present measurement session activation strategies based on these principles. These strategies do not require human intervention, are adaptive to changes in network conditions, and independent of the underlying active measurement technology.
Learning to Win, Lose and Cooperate through Reward Signal Evolution
Venue: arXiv cs.MA 2021
Year: 2021

Solving a reinforcement learning problem typically involves correctly prespecifying the reward signal from which the algorithm learns. Here, we approach the problem of reward signal design by using an evolutionary approach to perform a search on the space of all possible reward signals. We introduce a general framework for optimizing $N$ goals given $n$ reward signals. Through experiments we demonstrate that such an approach allows agents to learn high-level goals - such as winning, losing and cooperating - from scratch without prespecified reward signals in the game of Pong. Some of the solutions found by the algorithm are surprising, in the sense that they would probably not have been chosen by a person trying to hand-code a given behaviour through a specific reward signal. Furthermore, it seems that the proposed approach may also benefit from higher stability of the training performance when compared with the typical score-based reward signals.
An Annotated Commodity News Corpus for Event Extraction
Venue: arXiv cs.CL 2021
Year: 2021

Commodity News contains a wealth of information such as sum-mary of the recent commodity price movement and notable events that led tothe movement. Through event extraction, useful information extracted fromcommodity news is extremely useful in mining for causal relation betweenevents and commodity price movement, which can be used for commodity priceprediction. To facilitate the future research, we introduce a new dataset withthe following information identified and annotated: (i) entities (both nomi-nal and named), (ii) events (trigger words and argument roles), (iii) eventmetadata: modality, polarity and intensity and (iv) event-event relations.
IMPULSE: A 65nm Digital Compute-in-Memory Macro with Fused Weights and Membrane Potential for Spike-based Sequential Learning Tasks
Venue: arXiv cs.AR 2021
Year: 2021

The inherent dynamics of the neuron membrane potential in Spiking Neural Networks (SNNs) allows processing of sequential learning tasks, avoiding the complexity of recurrent neural networks. The highly-sparse spike-based computations in such spatio-temporal data can be leveraged for energy-efficiency. However, the membrane potential incurs additional memory access bottlenecks in current SNN hardware. To that effect, we propose a 10T-SRAM compute-in-memory (CIM) macro, specifically designed for state-of-the-art SNN inference. It consists of a fused weight (WMEM) and membrane potential (VMEM) memory and inherently exploits sparsity in input spikes leading to 97.4% reduction in energy-delay-product (EDP) at 85% sparsity (typical of SNNs considered in this work) compared to the case of no sparsity. We propose staggered data mapping and reconfigurable peripherals for handling different bit-precision requirements of WMEM and VMEM, while supporting multiple neuron functionalities. The proposed macro was fabricated in 65nm CMOS technology, achieving an energy-efficiency of 0.99TOPS/W at 0.85V supply and 200MHz frequency for signed 11-bit operations. We evaluate the SNN for sentiment classification from the IMDB dataset of movie reviews and achieve within 1% accuracy of an LSTM network with 8.5x lower parameters.
PoBRL: Optimizing Multi-Document Summarization by Blending Reinforcement Learning Policies
Venue: arXiv cs.AI 2021
Year: 2021

We propose a novel reinforcement learning based framework PoBRL for solving multi-document summarization. PoBRL jointly optimizes over the following three objectives necessary for a high-quality summary: importance, relevance, and length. Our strategy decouples this multi-objective optimization into different subproblems that can be solved individually by reinforcement learning. Utilizing PoBRL, we then blend each learned policies together to produce a summary that is a concise and complete representation of the original input. Our empirical analysis shows state-of-the-art performance on several multi-document datasets. Human evaluation also shows that our method produces high-quality output.
Learning to Route via Theory-Guided Residual Network
Venue: arXiv cs.LG 2021
Year: 2021

The heavy traffic and related issues have always been concerns for modern cities. With the help of deep learning and reinforcement learning, people have proposed various policies to solve these traffic-related problems, such as smart traffic signal control systems and taxi dispatching systems. People usually validate these policies in a city simulator, since directly applying them in the real city introduces real cost. However, these policies validated in the city simulator may fail in the real city if the simulator is significantly different from the real world. To tackle this problem, we need to build a real-like traffic simulation system. Therefore, in this paper, we propose to learn the human routing model, which is one of the most essential part in the traffic simulator. This problem has two major challenges. First, human routing decisions are determined by multiple factors, besides the common time and distance factor. Second, current historical routes data usually covers just a small portion of vehicles, due to privacy and device availability issues. To address these problems, we propose a theory-guided residual network model, where the theoretical part can emphasize the general principles for human routing decisions (e.g., fastest route), and the residual part can capture drivable condition preferences (e.g., local road or highway). Since the theoretical part is composed of traditional shortest path algorithms that do not need data to train, our residual network can learn human routing models from limited data. We have conducted extensive experiments on multiple real-world datasets to show the superior performance of our model, especially with small data. Besides, we have also illustrated why our model is better at recovering real routes through case studies.
Learning-Based Robust Resource allocation for D2D Underlaying Cellular Network
Venue: arXiv cs.IT 2021
Year: 2021

In this paper, we study the resource allocation in D2D underlaying cellular network with uncertain channel state information (CSI). For satisfying the diversity requirements of different users, i.e. the minimum rate requirement for cellular user and the reliability requirement for D2D user, we attempt to maximize the cellular user's throughput whilst ensuring a chance constraint for D2D user. Then, a robust resource allocation framework is proposed for solving the highly intractable chance constraint about D2D reliability requirement, where the CSI uncertainties are represented as a deterministic set and the reliability requirement is enforced to hold for any uncertain CSI within it. Then, a symmetrical-geometry-based learning approach is developed to model the uncertain CSI into polytope, ellipsoidal and box. After that, we derive the robust counterpart of the chance constraint under these uncertainty sets as the computation convenient convex sets. To overcome the conservatism of the symmetrical-geometry-based uncertainty sets, we develop a support vector clustering (SVC)-based approach to model uncertain CSI as a compact convex uncertainty set. Based on that, the chance constraint of D2D is converted into a linear convex set. Then, we develop a bisection search-based power allocation algorithm for solving the resource allocation in D2D underlaying cellular network with different robust counterparts. Finally, we conduct the simulation to compare the proposed robust optimization approaches with the non-robust one.
Divided We Rule: Influencer Polarization on Twitter During Political Crises in India
Venue: arXiv cs.SI 2021
Year: 2021

Influencers are key to the nature and networks of information propagation on social media. Influencers are particularly important in political discourse through their engagement with issues, and may derive their legitimacy either solely or in large part through online operation, or have an offline sphere of expertise such as entertainers, journalists etc. To quantify influencers' political engagement and polarity, we use Google's Universal Sentence Encoder (USE) to encode the tweets of 6k influencers and 26k Indian politicians during political crises in India. We then obtain aggregate vector representations of the influencers based on their tweet embeddings, which alongside retweet graphs help compute their stance and polarity with respect to these political issues. We find that influencers engage with the topics in a partisan manner, with polarized influencers being rewarded with increased retweeting and following. Moreover, we observe that specific groups of influencers are consistently polarized across all events. We conclude by discussing how our study provides insights into the political schisms of present-day India, but also offers a means to study the role of influencers in exacerbating political polarization in other contexts.
Fine-Grained View on Bribery for Group Identification
Venue: arXiv cs.GT 2021
Year: 2021

Given a set of agents qualifying or disqualifying each other, group identification is the task of identifying a socially qualified subgroup of agents. Social qualification depends on the specific rule used to aggregate individual qualifications. The classical bribery problem in this context asks how many agents need to change their qualifications in order to change the outcome in a certain way. Complementing previous results showing polynomial-time solvability or NP-hardness of bribery for various social rules in the constructive (aiming at making specific agents socially qualified) or destructive (aiming at making specific agents socially disqualified) setting, we provide a comprehensive picture of the parameterized computational complexity landscape. Conceptually, we also consider a more fine-grained concept of bribery cost, where we ask how many single qualifications need to be changed, nonunit prices for different bribery actions, and a more general bribery goal that combines the constructive and destructive setting.
EMISSOR: A platform for capturing multimodal interactions as Episodic Memories and Interpretations with Situated Scenario-based Ontological References
Venue: arXiv cs.HC 2021
Year: 2021

We present EMISSOR: a platform to capture multimodal interactions as recordings of episodic experiences with explicit referential interpretations that also yield an episodic Knowledge Graph (eKG). The platform stores streams of multiple modalities as parallel signals. Each signal is segmented and annotated independently with interpretation. Annotations are eventually mapped to explicit identities and relations in the eKG. As we ground signal segments from different modalities to the same instance representations, we also ground different modalities across each other. Unique to our eKG is that it accepts different interpretations across modalities, sources and experiences and supports reasoning over conflicting information and uncertainties that may result from multimodal experiences. EMISSOR can record and annotate experiments in virtual and real-world, combine data, evaluate system behavior and their performance for preset goals but also model the accumulation of knowledge and interpretations in the Knowledge Graph as a result of these episodic experiences.
DRILL: Dynamic Representations for Imbalanced Lifelong Learning
Venue: arXiv cs.CL 2021
Year: 2021

Continual or lifelong learning has been a long-standing challenge in machine learning to date, especially in natural language processing (NLP). Although state-of-the-art language models such as BERT have ushered in a new era in this field due to their outstanding performance in multitask learning scenarios, they suffer from forgetting when being exposed to a continuous stream of data with shifting data distributions. In this paper, we introduce DRILL, a novel continual learning architecture for open-domain text classification. DRILL leverages a biologically inspired self-organizing neural architecture to selectively gate latent language representations from BERT in a task-incremental manner. We demonstrate in our experiments that DRILL outperforms current methods in a realistic scenario of imbalanced, non-stationary data without prior knowledge about task boundaries. To the best of our knowledge, DRILL is the first of its kind to use a self-organizing neural architecture for open-domain lifelong learning in NLP.