OpenRaiser/Intern-Atlas huggingface dataset:
Applying Machine Learning and AI Explanations to Analyze Vaccine Hesitancy
Venue: arXiv cs.LG 2022
Year: 2022

The paper quantifies the impact of race, poverty, politics, and age on COVID-19 vaccination rates in counties in the continental US. Both, OLS regression analysis and Random Forest machine learning algorithms are applied to quantify factors for county-level vaccination hesitancy. The machine learning model considers joint effects of variables (race/ethnicity, partisanship, age, etc.) simultaneously to capture the unique combination of these factors on the vaccination rate. By implementing a state-of-the-art Artificial Intelligence Explanations (AIX) algorithm, it is possible to solve the black box problem with machine learning models and provide answers to the "how much" question for each measured impact factor in every county. For most counties, a higher percentage vote for Republicans, a greater African American population share, and a higher poverty rate lower the vaccination rate. While a higher Asian population share increases the predicted vaccination rate. The impact on the vaccination rate from the Hispanic population proportion is positive in the OLS model, but only positive for counties with a high Hispanic population (>65%) in the Random Forest model. Both the proportion of seniors and the one for young people in a county have a significant impact in the OLS model - positive and negative, respectively. In contrast, the impacts are ambiguous in the Random Forest model. Because results vary between geographies and since the AIX algorithm is able to quantify vaccine impacts individually for each county, this research can be tailored to local communities. An interactive online mapping dashboard that identifies impact factors for individual U.S. counties is available at https://www.cpp.edu/~clange/vacmap.html. It is apparent that the influence of impact factors is not universally the same across different geographies.
Evaluation of Neural Networks Defenses and Attacks using NDCG and Reciprocal Rank Metrics
Venue: arXiv cs.CR 2022
Year: 2022

The problem of attacks on neural networks through input modification (i.e., adversarial examples) has attracted much attention recently. Being relatively easy to generate and hard to detect, these attacks pose a security breach that many suggested defenses try to mitigate. However, the evaluation of the effect of attacks and defenses commonly relies on traditional classification metrics, without adequate adaptation to adversarial scenarios. Most of these metrics are accuracy-based, and therefore may have a limited scope and low distinctive power. Other metrics do not consider the unique characteristics of neural networks functionality, or measure the effect of the attacks indirectly (e.g., through the complexity of their generation). In this paper, we present two metrics which are specifically designed to measure the effect of attacks, or the recovery effect of defenses, on the output of neural networks in multiclass classification tasks. Inspired by the normalized discounted cumulative gain and the reciprocal rank metrics used in information retrieval literature, we treat the neural network predictions as ranked lists of results. Using additional information about the probability of the rank enabled us to define novel metrics that are suited to the task at hand. We evaluate our metrics using various attacks and defenses on a pretrained VGG19 model and the ImageNet dataset. Compared to the common classification metrics, our proposed metrics demonstrate superior informativeness and distinctiveness.
Low-latency, Scalable, DeFi with Zef
Venue: arXiv cs.CR 2022
Year: 2022

Zef was recently proposed to extend the low-latency, Byzantine-Fault Tolerant (BFT) payment protocol FastPay with anonymous coins. This report explores further extensions of FastPay and Lef beyond payments. We start by off-chain assets (e.g. NFTs). We introduce the idea of on-demand BFT consensus instances throught the example of atomic swaps between account owners.
Improved Multi-objective Data Stream Clustering with Time and Memory Optimization
Venue: arXiv cs.LG 2022
Year: 2022

The analysis of data streams has received considerable attention over the past few decades due to sensors, social media, etc. It aims to recognize patterns in an unordered, infinite, and evolving stream of observations. Clustering this type of data requires some restrictions in time and memory. This paper introduces a new data stream clustering method (IMOC-Stream). This method, unlike the other clustering algorithms, uses two different objective functions to capture different aspects of the data. The goal of IMOC-Stream is to: 1) reduce computation time by using idle times to apply genetic operations and enhance the solution. 2) reduce memory allocation by introducing a new tree synopsis. 3) find arbitrarily shaped clusters by using a multi-objective framework. We conducted an experimental study with high dimensional stream datasets and compared them to well-known stream clustering techniques. The experiments show the ability of our method to partition the data stream in arbitrarily shaped, compact, and well-separated clusters while optimizing the time and memory. Our method also outperformed most of the stream algorithms in terms of NMI and ARAND measures.
NorDiaChange: Diachronic Semantic Change Dataset for Norwegian
Venue: arXiv cs.CL 2022
Year: 2022

We describe NorDiaChange: the first diachronic semantic change dataset for Norwegian. NorDiaChange comprises two novel subsets, covering about 80 Norwegian nouns manually annotated with graded semantic change over time. Both datasets follow the same annotation procedure and can be used interchangeably as train and test splits for each other. NorDiaChange covers the time periods related to pre- and post-war events, oil and gas discovery in Norway, and technological developments. The annotation was done using the DURel framework and two large historical Norwegian corpora. NorDiaChange is published in full under a permissive licence, complete with raw annotation data and inferred diachronic word usage graphs (DWUGs).
Jamming Attacks on Federated Learning in Wireless Networks
Venue: arXiv cs.LG 2022
Year: 2022

Federated learning (FL) offers a decentralized learning environment so that a group of clients can collaborate to train a global model at the server, while keeping their training data confidential. This paper studies how to launch over-the-air jamming attacks to disrupt the FL process when it is executed over a wireless network. As a wireless example, FL is applied to learn how to classify wireless signals collected by clients (spectrum sensors) at different locations (such as in cooperative sensing). An adversary can jam the transmissions for the local model updates from clients to the server (uplink attack), or the transmissions for the global model updates the server to clients (downlink attack), or both. Given a budget imposed on the number of clients that can be attacked per FL round, clients for the (uplink/downlink) attack are selected according to their local model accuracies that would be expected without an attack or ranked via spectrum observations. This novel attack is extended to general settings by accounting different processing speeds and attack success probabilities for clients. Compared to benchmark attack schemes, this attack approach degrades the FL performance significantly, thereby revealing new vulnerabilities of FL to jamming attacks in wireless networks.
The Combinatorics of \textit{Salva Veritate} Principles
Venue: arXiv cs.CL 2022
Year: 2022

Various concepts of grammatical compositionality arise in many theories of both natural and artificial languages, and often play a key role in accounts of the syntax-semantics interface. We propose that many instances of compositionality should entail non-trivial combinatorial claims about the expressive power of languages which satisfy these compositional properties. As an example, we present a formal analysis demonstrating that a particular class of languages which admit salva vertitate substitutions - a property which we claim to be a particularly strong example of compositional principle - must also satisfy a very natural combinatorial constraint identified in this paper.
D-Box: DMA-enabled Compartmentalization for Embedded Applications
Venue: arXiv cs.CR 2022
Year: 2022

Embedded and Internet-of-Things (IoT) devices have seen an increase in adoption in many domains. The security of these devices is of great importance as they are often used to control critical infrastructure, medical devices, and vehicles. Existing solutions to isolate microcontroller (MCU) resources in order to increase their security face significant challenges such as specific hardware unavailability, Memory Protection Unit (MPU) limitations and a significant lack of Direct Memory Access (DMA) support. Nevertheless, DMA is fundamental for the power and performance requirements of embedded applications. In this paper, we present D-Box, a systematic approach to enable secure DMA operations for compartmentalization solutions of embedded applications using real-time operating systems (RTOS). D-Box defines a reference architecture and a workflow to protect DMA operations holistically. It provides practical methods to harden the kernel and define capability-based security policies for easy definition of DMA operations with strong security properties. We implemented a D-Box prototype for the Cortex-M3/M4 on top of the popular FreeRTOS-MPU (F-MPU). The D-Box procedures and a stricter security model enabled DMA operations, yet it exposed 41 times less ROP (return-orienting-programming) gadgets when compared with the standard F-MPU. D-Box adds only a 2% processor overhead while reducing the power consumption of peripheral operation benchmarks by 18.2%. The security properties and performance of D-Box were tested and confirmed on a real-world case study of a Programmable Logic Controller (PLC) application.
Examining and Mitigating the Impact of Crossbar Non-idealities for Accurate Implementation of Sparse Deep Neural Networks
Venue: arXiv cs.LG 2022
Year: 2022

Recently several structured pruning techniques have been introduced for energy-efficient implementation of Deep Neural Networks (DNNs) with lesser number of crossbars. Although, these techniques have claimed to preserve the accuracy of the sparse DNNs on crossbars, none have studied the impact of the inexorable crossbar non-idealities on the actual performance of the pruned networks. To this end, we perform a comprehensive study to show how highly sparse DNNs, that result in significant crossbar-compression-rate, can lead to severe accuracy losses compared to unpruned DNNs mapped onto non-ideal crossbars. We perform experiments with multiple structured-pruning approaches (such as, C/F pruning, XCS and XRS) on VGG11 and VGG16 DNNs with benchmark datasets (CIFAR10 and CIFAR100). We propose two mitigation approaches - Crossbar column rearrangement and Weight-Constrained-Training (WCT) - that can be integrated with the crossbar-mapping of the sparse DNNs to minimize accuracy losses incurred by the pruned models. These help in mitigating non-idealities by increasing the proportion of low conductance synapses on crossbars, thereby improving their computational accuracies.
Optimal alphabet for single text compression
Venue: arXiv cs.IT 2022
Year: 2022

A text written using symbols from a given alphabet can be compressed using the Huffman code, which minimizes the length of the encoded text. It is necessary, however, to employ a text-specific codebook, i.e. the symbol-codeword dictionary, to decode the original text. Thus, the compression performance should be evaluated by the full code length, i.e. the length of the encoded text plus the length of the codebook. We studied several alphabets for compressing texts -- letters, n-grams of letters, syllables, words, and phrases. If only sufficiently short texts are retained, an alphabet of letters or two-grams of letters is optimal. For the majority of Project Gutenberg texts, the best alphabet (the one that minimizes the full code length) is given by syllables or words, depending on the representation of the codebook. Letter 3 and 4-grams, having on average comparable length to syllables/words, perform noticeably worse than syllables or words. Word 2-grams also are never the best alphabet, on the account of having a very large codebook. We also show that the codebook representation is important -- switching from a naive representation to a compact one significantly improves the matters for alphabets with large number of symbols, most notably the words. Thus, meaning-expressing elements of the language (syllables or words) provide the best compression alphabet.
Integrated Sensing and Communication with Millimeter Wave Full Duplex Hybrid Beamforming
Venue: arXiv cs.IT 2022
Year: 2022

Integrated Sensing and Communication (ISAC) has attracted substantial attraction in recent years for spectral efficiency improvement, enabling hardware and spectrum sharing for simultaneous sensing and signaling operations. In-band Full Duplex (FD) is being considered as a key enabling technology for ISAC applications due to its simultaneous transmission and reception capability. In this paper, we present an FD-based ISAC system operating at millimeter Wave (mmWave) frequencies, where a massive Multiple-Input Multiple-Output (MIMO) Base Station (BS) node employing hybrid Analog and Digital (A/D) beamforming is communicating with a DownLink (DL) multi-antenna user and the same waveform is utilized at the BS receiver for sensing the radar targets in its coverage environment. We develop a sensing algorithm that is capable of estimating Direction of Arrival (DoA), range, and relative velocity of the radar targets. A joint optimization framework for designing the A/D transmit and receive beamformers as well as the Self-Interference (SI) cancellation is presented with the objective to maximize the achievable DL rate and the accuracy of the radar target sensing performance. Our simulation results, considering fifth Generation (5G) Orthogonal Frequency Division Multiplexing (OFDM) waveforms, verify our approach's high precision in estimating DoA, range, and velocity of multiple radar targets, while maximizing the DL communication rate.
Multi-agent Motion Planning from Signal Temporal Logic Specifications
Venue: arXiv cs.RO 2022
Year: 2022

We tackle the challenging problem of multi-agent cooperative motion planning for complex tasks described using signal temporal logic (STL), where robots can have nonlinear and nonholonomic dynamics. Existing methods in multi-agent motion planning, especially those based on discrete abstractions and model predictive control (MPC), suffer from limited scalability with respect to the complexity of the task, the size of the workspace, and the planning horizon. We present a method based on {\em timed waypoints\/} to address this issue. We show that timed waypoints can help abstract nonlinear behaviors of the system as safety envelopes around the reference path defined by those waypoints. Then the search for waypoints satisfying the STL specifications can be inductively encoded as a mixed-integer linear program. The agents following the synthesized timed waypoints have their tasks automatically allocated, and are guaranteed to satisfy the STL specifications while avoiding collisions. We evaluate the algorithm on a wide variety of benchmarks. Results show that it supports multi-agent planning from complex specification over long planning horizons, and significantly outperforms state-of-the-art abstraction-based and MPC-based motion planning methods. The implementation is available at https://github.com/sundw2014/STLPlanning.
DapStep: Deep Assignee Prediction for Stack Trace Error rePresentation
Venue: arXiv cs.SE 2022
Year: 2022

The task of finding the best developer to fix a bug is called bug triage. Most of the existing approaches consider the bug triage task as a classification problem, however, classification is not appropriate when the sets of classes change over time (as developers often do in a project). Furthermore, to the best of our knowledge, all the existing models use textual sources of information, i.e., bug descriptions, which are not always available. In this work, we explore the applicability of existing solutions for the bug triage problem when stack traces are used as the main data source of bug reports. Additionally, we reformulate this task as a ranking problem and propose new deep learning models to solve it. The models are based on a bidirectional recurrent neural network with attention and on a convolutional neural network, with the weights of the models optimized using a ranking loss function. To improve the quality of ranking, we propose using additional information from version control system annotations. Two approaches are proposed for extracting features from annotations: manual and using an additional neural network. To evaluate our models, we collected two datasets of real-world stack traces. Our experiments show that the proposed models outperform existing models adapted to handle stack traces. To facilitate further research in this area, we publish the source code of our models and one of the collected datasets.
Deep Leaning-Based Ultra-Fast Stair Detection
Venue: arXiv cs.CV 2022
Year: 2022

Staircases are some of the most common building structures in urban environments. Stair detection is an important task for various applications, including the environmental perception of exoskeleton robots, humanoid robots, and rescue robots and the navigation of visually impaired people. Most existing stair detection algorithms have difficulty dealing with the diversity of stair structure materials, extreme light and serious occlusion. Inspired by human perception, we propose an end-to-end method based on deep learning. Specifically, we treat the process of stair line detection as a multitask involving coarse-grained semantic segmentation and object detection. The input images are divided into cells, and a simple neural network is used to judge whether each cell contains stair lines. For cells containing stair lines, the locations of the stair lines relative to each cell are regressed. Extensive experiments on our dataset show that our method can achieve high performance in terms of both speed and accuracy. A lightweight version can even achieve 300+ frames per second with the same resolution. Our code and dataset will be soon available at GitHub.
Argus++: Robust Real-time Activity Detection for Unconstrained Video Streams with Overlapping Cube Proposals
Venue: arXiv cs.CV 2022
Year: 2022

Activity detection is one of the attractive computer vision tasks to exploit the video streams captured by widely installed cameras. Although achieving impressive performance, conventional activity detection algorithms are usually designed under certain constraints, such as using trimmed and/or object-centered video clips as inputs. Therefore, they failed to deal with the multi-scale multi-instance cases in real-world unconstrained video streams, which are untrimmed and have large field-of-views. Real-time requirements for streaming analysis also mark brute force expansion of them unfeasible. To overcome these issues, we propose Argus++, a robust real-time activity detection system for analyzing unconstrained video streams. The design of Argus++ introduces overlapping spatio-temporal cubes as an intermediate concept of activity proposals to ensure coverage and completeness of activity detection through over-sampling. The overall system is optimized for real-time processing on standalone consumer-level hardware. Extensive experiments on different surveillance and driving scenarios demonstrated its superior performance in a series of activity detection benchmarks, including CVPR ActivityNet ActEV 2021, NIST ActEV SDL UF/KF, TRECVID ActEV 2020/2021, and ICCV ROAD 2021.
Multi-Narrative Semantic Overlap Task: Evaluation and Benchmark
Venue: arXiv cs.CL 2022
Year: 2022

In this paper, we introduce an important yet relatively unexplored NLP task called Multi-Narrative Semantic Overlap (MNSO), which entails generating a Semantic Overlap of multiple alternate narratives. As no benchmark dataset is readily available for this task, we created one by crawling 2,925 narrative pairs from the web and then, went through the tedious process of manually creating 411 different ground-truth semantic overlaps by engaging human annotators. As a way to evaluate this novel task, we first conducted a systematic study by borrowing the popular ROUGE metric from text-summarization literature and discovered that ROUGE is not suitable for our task. Subsequently, we conducted further human annotations/validations to create 200 document-level and 1,518 sentence-level ground-truth labels which helped us formulate a new precision-recall style evaluation metric, called SEM-F1 (semantic F1). Experimental results show that the proposed SEM-F1 metric yields higher correlation with human judgement as well as higher inter-rater-agreement compared to ROUGE metric.
A Thousand Words Are Worth More Than a Picture: Natural Language-Centric Outside-Knowledge Visual Question Answering
Venue: arXiv cs.CV 2022
Year: 2022

Outside-knowledge visual question answering (OK-VQA) requires the agent to comprehend the image, make use of relevant knowledge from the entire web, and digest all the information to answer the question. Most previous works address the problem by first fusing the image and question in the multi-modal space, which is inflexible for further fusion with a vast amount of external knowledge. In this paper, we call for a paradigm shift for the OK-VQA task, which transforms the image into plain text, so that we can enable knowledge passage retrieval, and generative question-answering in the natural language space. This paradigm takes advantage of the sheer volume of gigantic knowledge bases and the richness of pre-trained language models. A Transform-Retrieve-Generate framework (TRiG) framework is proposed, which can be plug-and-played with alternative image-to-text models and textual knowledge bases. Experimental results show that our TRiG framework outperforms all state-of-the-art supervised methods by at least 11.1% absolute margin.
CommonsenseQA 2.0: Exposing the Limits of AI through Gamification
Venue: arXiv cs.CL 2022
Year: 2022

Constructing benchmarks that test the abilities of modern natural language understanding models is difficult - pre-trained language models exploit artifacts in benchmarks to achieve human parity, but still fail on adversarial examples and make errors that demonstrate a lack of common sense. In this work, we propose gamification as a framework for data construction. The goal of players in the game is to compose questions that mislead a rival AI while using specific phrases for extra points. The game environment leads to enhanced user engagement and simultaneously gives the game designer control over the collected data, allowing us to collect high-quality data at scale. Using our method we create CommonsenseQA 2.0, which includes 14,343 yes/no questions, and demonstrate its difficulty for models that are orders-of-magnitude larger than the AI used in the game itself. Our best baseline, the T5-based Unicorn with 11B parameters achieves an accuracy of 70.2%, substantially higher than GPT-3 (52.9%) in a few-shot inference setup. Both score well below human performance which is at 94.1%.