OpenRaiser/Intern-Atlas huggingface dataset:
Anomaly Detection in Audio with Concept Drift using Adaptive Huffman Coding
Venue: arXiv cs.SD 2021
Year: 2021

When detecting anomalies in audio, it can often be necessary to consider concept drift: the distribution of the data may drift over time because of dynamically changing environments, and anomalies may become normal as time elapses. We propose to use adaptive Huffman coding for anomaly detection in audio with concept drift. Compared with the existing method of adaptive Gaussian mixture modeling (AGMM), adaptive Huffman coding does not require a priori information about the clusters and can adjust the number of clusters dynamically depending on the amount of variation in the audio. To control the size of the Huffman tree, we propose to merge clusters that are close to each other instead of replacing rare clusters with new data. This reduces redundancy in the Huffman tree while ensuring that it never forgets past information. On a dataset of audio with concept drift which we have curated ourselves, our proposed method achieves higher area under the curve (AUC) compared with AGMM and fixed-length Huffman trees. The proposed approach is also time-efficient and can be easily extended to other types of time series data (e.g., video).
Exploring the dynamics of protest against National Register of Citizens & Citizenship Amendment Act through online social media: the Indian experience
Venue: arXiv cs.CY 2021
Year: 2021

The generic fluidity observed in the nature of political protest movements across the world during the last decade weigh heavily with the presence of social media. As such, there is a possibility to study the contemporary movements with an interdisciplinary approach combining computational analytics with social science perspectives. The present study has put efforts to understand such dynamics in the context of the ongoing nationwide movement in India opposing the NRC-CAA enactment. The transformative nature of individual discontent into collective mobilization, especially with a reflective intervention in social media across a sensitive region of the nation state, is presented here with a combination of qualitative (fieldwork) and quantitative (computing) techniques. The study is augmented further by the primary data generation coupled with real-time application of analytical approaches.
Automatic Code Generation using Pre-Trained Language Models
Venue: arXiv cs.CL 2021
Year: 2021

Recent advancements in natural language processing \cite{gpt2} \cite{BERT} have led to near-human performance in multiple natural language tasks. In this paper, we seek to understand whether similar techniques can be applied to a highly structured environment with strict syntax rules. Specifically, we propose an end-to-end machine learning model for code generation in the Python language built on-top of pre-trained language models. We demonstrate that a fine-tuned model can perform well in code generation tasks, achieving a BLEU score of 0.22, an improvement of 46\% over a reasonable sequence-to-sequence baseline. All results and related code used for training and data processing are available on GitHub.
Dynamical Analysis of the EIP-1559 Ethereum Fee Market
Venue: arXiv cs.GT 2021
Year: 2021

Participation in permissionless blockchains results in competition over system resources, which needs to be controlled with fees. Ethereum's current fee mechanism is implemented via a first-price auction that results in unpredictable fees as well as other inefficiencies. EIP-1559 is a recent, improved proposal that introduces a number of innovative features such as a dynamically adaptive base fee that is burned, instead of being paid to the miners. Despite intense interest in understanding its properties, several basic questions such as whether and under what conditions does this protocol self-stabilize have remained elusive thus far. We perform a thorough analysis of the resulting fee market dynamic mechanism via a combination of tools from game theory and dynamical systems. We start by providing bounds on the step-size of the base fee update rule that suffice for global convergence to equilibrium via Lyapunov arguments. In the negative direction, we show that for larger step-sizes instability and even formally chaotic behavior are possible under a wide range of settings. We complement these qualitative results with quantitative bounds on the resulting range of base fees. We conclude our analysis with a thorough experimental case study that corroborates our theoretical findings.
Symbolic regression for scientific discovery: an application to wind speed forecasting
Venue: arXiv cs.LG 2021
Year: 2021

Symbolic regression corresponds to an ensemble of techniques that allow to uncover an analytical equation from data. Through a closed form formula, these techniques provide great advantages such as potential scientific discovery of new laws, as well as explainability, feature engineering as well as fast inference. Similarly, deep learning based techniques has shown an extraordinary ability of modeling complex patterns. The present paper aims at applying a recent end-to-end symbolic regression technique, i.e. the equation learner (EQL), to get an analytical equation for wind speed forecasting. We show that it is possible to derive an analytical equation that can achieve reasonable accuracy for short term horizons predictions only using few number of features.
Demystifying Cryptocurrency Mining Attacks: A Semi-supervised Learning Approach Based on Digital Forensics and Dynamic Network Characteristics
Venue: arXiv cs.CR 2021
Year: 2021

Cryptocurrencies have emerged as a new form of digital money that has not escaped the eyes of cyber-attackers. Traditionally, they have been maliciously used as a medium of exchange for proceeds of crime in the cyber dark-market by cyber-criminals. However, cyber-criminals have devised an exploitative technique of directly acquiring cryptocurrencies from benign users' CPUs without their knowledge through a process called crypto mining. The presence of crypto mining activities in a network is often an indicator of compromise of illegal usage of network resources for crypto mining purposes. Crypto mining has had a financial toll on victims such as corporate networks and individual home users. This paper addresses the detection of crypto mining attacks in a generic network environment using dynamic network characteristics. It tackles an in-depth overview of crypto mining operational details and proposes a semi-supervised machine learning approach to detection using various crypto mining features derived from complex network characteristics. The results demonstrate that the integration of semi-supervised learning with complex network theory modeling is effective at detecting crypto mining activities in a network environment. Such an approach is helpful during security mitigation by network security administrators and law enforcement agencies.
Safe Reinforcement Learning Using Robust Action Governor
Venue: arXiv cs.LG 2021
Year: 2021

Reinforcement Learning (RL) is essentially a trial-and-error learning procedure which may cause unsafe behavior during the exploration-and-exploitation process. This hinders the application of RL to real-world control problems, especially to those for safety-critical systems. In this paper, we introduce a framework for safe RL that is based on integration of a RL algorithm with an add-on safety supervision module, called the Robust Action Governor (RAG), which exploits set-theoretic techniques and online optimization to manage safety-related requirements during learning. We illustrate this proposed safe RL framework through an application to automotive adaptive cruise control.
(Almost Full) EFX Exists for Four Agents (and Beyond)
Venue: arXiv cs.GT 2021
Year: 2021

The existence of EFX allocations is a major open problem in fair division, even for additive valuations. The current state of the art is that no setting where EFX allocations are impossible is known, and EFX is known to exist for ($i$) agents with identical valuations, ($ii$) 2 agents, ($iii$) 3 agents with additive valuations, ($iv$) agents with one of two additive valuations and ($v$) agents with two-valued instances. It is also known that EFX exists if one can leave $n-1$ items unallocated, where $n$ is the number of agents. We develop new techniques that allow us to push the boundaries of the enigmatic EFX problem beyond these known results, and, arguably, to simplify proofs of earlier results. Our main results are ($i$) every setting with 4 additive agents admits an EFX allocation that leaves at most a single item unallocated, ($ii$) every setting with $n$ additive valuations has an EFX allocation with at most $n-2$ unallocated items. Moreover, all of our results extend beyond additive valuations to all nice cancelable valuations (a new class, including additive, unit-demand, budget-additive and multiplicative valuations, among others). Furthermore, using our new techniques, we show that previous results for additive valuations extend to nice cancelable valuations.
SigVM: Enabling Event-Driven Execution for Autonomous Smart Contracts
Venue: arXiv cs.PL 2021
Year: 2021

This paper presents SigVM, a novel blockchain virtual machine that supports an event-driven execution model, enabling developers to build autonomous smart contracts. Contracts in SigVM can emit signal events, on which other contracts can listen. Once an event is triggered, corresponding handler functions are automatically executed as signal transactions. We build an end-to-end blockchain platform SigChain and a contract language compiler SigSolid to realize the potential of SigVM. Experimental results show that our benchmark applications can be reimplemented with SigVM in an autonomous way, eliminating the dependency on unreliable mechanisms like off-chain relay servers. The development effort of reimplementing these contracts with SigVM is small, i.e., we modified on average 2.6% of the contract code.
Attention Models for Point Clouds in Deep Learning: A Survey
Venue: arXiv cs.CV 2021
Year: 2021

Recently, the advancement of 3D point clouds in deep learning has attracted intensive research in different application domains such as computer vision and robotic tasks. However, creating feature representation of robust, discriminative from unordered and irregular point clouds is challenging. In this paper, our ultimate goal is to provide a comprehensive overview of the point clouds feature representation which uses attention models. More than 75+ key contributions in the recent three years are summarized in this survey, including the 3D objective detection, 3D semantic segmentation, 3D pose estimation, point clouds completion etc. We provide a detailed characterization (1) the role of attention mechanisms, (2) the usability of attention models into different tasks, (3) the development trend of key technology.
Root Radii and Subdivision for Polynomial Root-Finding
Venue: arXiv cs.SC 2021
Year: 2021

We depart from our approximation of 2000 of all root radii of a polynomial, which has readily extended Sch{ö}nhage's efficient algorithm of 1982 for a single root radius. We revisit this extension, advance it, based on our simple but novel idea, and yield significant practical acceleration of the known near optimal subdivision algorithms for complex and real root-finding of user's choice. We achieve this by means of significant saving of exclusion tests and Taylor's shifts, which are the bottleneck of subdivision root-finders. This saving relies on our novel recipes for the initialization of root-finding iterations of independent interest. We demonstrate our practical progress with numerical tests, provide extensive analysis of the resulting algorithms, and show that, like the preceding subdivision root-finders, they support near optimal Boolean complexity bounds.
Distributed Generative Adversarial Networks for mmWaveChannel Modeling in Wireless UAV Networks
Venue: arXiv cs.IT 2021
Year: 2021

In this paper, a novel framework is proposed to enable air-to-ground channel modeling over millimeter wave (mmWave) frequencies in an unmanned aerial vehicle (UAV) wireless network. First, an effective channel estimation approach is developed to collect mmWave channel information allowing each UAV to train a local channel model via a generative adversarial network (GAN). Next, in order to share the channel information between UAVs in a privacy-preserving manner, a cooperative framework, based on a distributed GAN architecture, is developed to enable each UAV to learn the mmWave channel distribution from the entire dataset in a fully distributed approach. The necessary and sufficient conditions for the optimal network structure that maximizes the learning rate for information sharing in the distributed network are derived. Simulation results show that the learning rate of the proposed GAN approach will increase by sharing more generated channel samples at each learning iteration, but decrease given more UAVs in the network. The results also show that the proposed GAN method yields a higher learning accuracy, compared with a standalone GAN, and improves the average rate for UAV downlink communications by over 10%, compared with a baseline real-time channel estimation scheme.
Contour Loss for Instance Segmentation via k-step Distance Transformation Image
Venue: arXiv cs.CV 2021
Year: 2021

Instance segmentation aims to locate targets in the image and segment each target area at pixel level, which is one of the most important tasks in computer vision. Mask R-CNN is a classic method of instance segmentation, but we find that its predicted masks are unclear and inaccurate near contours. To cope with this problem, we draw on the idea of contour matching based on distance transformation image and propose a novel loss function, called contour loss. Contour loss is designed to specifically optimize the contour parts of the predicted masks, thus can assure more accurate instance segmentation. In order to make the proposed contour loss to be jointly trained under modern neural network frameworks, we design a differentiable k-step distance transformation image calculation module, which can approximately compute truncated distance transformation images of the predicted mask and corresponding ground-truth mask online. The proposed contour loss can be integrated into existing instance segmentation methods such as Mask R-CNN, and combined with their original loss functions without modification of the inference network structures, thus has strong versatility. Experimental results on COCO show that contour loss is effective, which can further improve instance segmentation performances.
Multiple Kernel-Based Online Federated Learning
Venue: arXiv cs.LG 2021
Year: 2021

Online federated learning (OFL) becomes an emerging learning framework, in which edge nodes perform online learning with continuous streaming local data and a server constructs a global model from the aggregated local models. Online multiple kernel learning (OMKL), using a preselected set of P kernels, can be a good candidate for OFL framework as it has provided an outstanding performance with a low-complexity and scalability. Yet, an naive extension of OMKL into OFL framework suffers from a heavy communication overhead that grows linearly with P. In this paper, we propose a novel multiple kernel-based OFL (MK-OFL) as a non-trivial extension of OMKL, which yields the same performance of the naive extension with 1/P communication overhead reduction. We theoretically prove that MK-OFL achieves the optimal sublinear regret bound when compared with the best function in hindsight. Finally, we provide the numerical tests of our approach on real-world datasets, which suggests its practicality.
Co-occurrences using Fasttext embeddings for word similarity tasks in Urdu
Venue: arXiv cs.CL 2021
Year: 2021

Urdu is a widely spoken language in South Asia. Though immoderate literature exists for the Urdu language still the data isn't enough to naturally process the language by NLP techniques. Very efficient language models exist for the English language, a high resource language, but Urdu and other under-resourced languages have been neglected for a long time. To create efficient language models for these languages we must have good word embedding models. For Urdu, we can only find word embeddings trained and developed using the skip-gram model. In this paper, we have built a corpus for Urdu by scraping and integrating data from various sources and compiled a vocabulary for the Urdu language. We also modify fasttext embeddings and N-Grams models to enable training them on our built corpus. We have used these trained embeddings for a word similarity task and compared the results with existing techniques.
Universal Approximation Theorem for Neural Networks
Venue: arXiv cs.LG 2021
Year: 2021

Is there any theoretical guarantee for the approximation ability of neural networks? The answer to this question is the "Universal Approximation Theorem for Neural Networks". This theorem states that a neural network is dense in a certain function space under an appropriate setting. This paper is a comprehensive explanation of the universal approximation theorem for feedforward neural networks, its approximation rate problem (the relation between the number of intermediate units and the approximation error), and Barron space in Japanese.
Towards a Machine Learning-driven Trust Evaluation Model for Social Internet of Things: A Time-aware Approach
Venue: arXiv cs.CR 2021
Year: 2021

The emerging paradigm of the Social Internet of Things (SIoT) has transformed the traditional notion of the Internet of Things (IoT) into a social network of billions of interconnected smart objects by integrating social networking facets into the same. In SIoT, objects can establish social relationships in an autonomous manner and interact with the other objects in the network based on their social behaviour. A fundamental problem that needs attention is establishing of these relationships in a reliable and trusted way, i.e., establishing trustworthy relationships and building trust amongst objects. In addition, it is also indispensable to ascertain and predict an object's behaviour in the SIoT network over a period of time. Accordingly, in this paper, we have proposed an efficient time-aware machine learning-driven trust evaluation model to address this particular issue. The envisaged model deliberates social relationships in terms of friendship and community-interest, and further takes into consideration the working relationships and cooperativeness (object-object interactions) as trust parameters to quantify the trustworthiness of an object. Subsequently, in contrast to the traditional weighted sum heuristics, a machine learning-driven aggregation scheme is delineated to synthesize these trust parameters to ascertain a single trust score. The experimental results demonstrate that the proposed model can efficiently segregates the trustworthy and untrustworthy objects within a network, and further provides the insight on how the trust of an object varies with time along with depicting the effect of each trust parameter on a trust score.
An open access NLP dataset for Arabic dialects : Data collection, labeling, and model construction
Venue: arXiv cs.CL 2021
Year: 2021

Natural Language Processing (NLP) is today a very active field of research and innovation. Many applications need however big sets of data for supervised learning, suitably labelled for the training purpose. This includes applications for the Arabic language and its national dialects. However, such open access labeled data sets in Arabic and its dialects are lacking in the Data Science ecosystem and this lack can be a burden to innovation and research in this field. In this work, we present an open data set of social data content in several Arabic dialects. This data was collected from the Twitter social network and consists on +50K twits in five (5) national dialects. Furthermore, this data was labeled for several applications, namely dialect detection, topic detection and sentiment analysis. We publish this data as an open access data to encourage innovation and encourage other works in the field of NLP for Arabic dialects and social media. A selection of models were built using this data set and are presented in this paper along with their performances.
Jointly Learning Clinical Entities and Relations with Contextual Language Models and Explicit Context
Venue: arXiv cs.CL 2021
Year: 2021

We hypothesize that explicit integration of contextual information into an Multi-task Learning framework would emphasize the significance of context for boosting performance in jointly learning Named Entity Recognition (NER) and Relation Extraction (RE). Our work proves this hypothesis by segmenting entities from their surrounding context and by building contextual representations using each independent segment. This relation representation allows for a joint NER/RE system that achieves near state-of-the-art (SOTA) performance on both NER and RE tasks while beating the SOTA RE system at end-to-end NER & RE with a 49.07 F1.
Learning the Subsystem of Local Planning for Autonomous Racing
Venue: arXiv cs.RO 2021
Year: 2021

The problem of autonomous racing is to navigate through a race course as quickly as possible while not colliding with any obstacles. We approach the autonomous racing problem with the added constraint of not maintaining an updated obstacle map of the environment. Several current approaches to this problem use end-to-end learning systems where an agent replaces the entire navigation pipeline. This paper presents a hierarchical planning architecture that combines a high level planner and path following system with a reinforcement learning agent that learns that subsystem of obstacle avoidance. The novel "modification planner" uses the path follower to track the global plan and the deep reinforcement learning agent to modify the references generated by the path follower to avoid obstacles. Importantly, our architecture does not require an updated obstacle map and only 10 laser range finders to avoid obstacles. The modification planner is evaluated in the context of F1/10th autonomous racing and compared to a end-to-end learning baseline, the Follow the Gap Method and an optimisation based planner. The results show that the modification planner can achieve faster average times compared to the baseline end-to-end planner and a 94% success rate which is similar to the baseline.