OpenRaiser/Intern-Atlas huggingface dataset:
Policy Search with Rare Significant Events: Choosing the Right Partner to Cooperate with
Venue: arXiv cs.LG 2021
Year: 2021

This paper focuses on a class of reinforcement learning problems where significant events are rare and limited to a single positive reward per episode. A typical example is that of an agent who has to choose a partner to cooperate with, while a large number of partners are simply not interested in cooperating, regardless of what the agent has to offer. We address this problem in a continuous state and action space with two different kinds of search methods: a gradient policy search method and a direct policy search method using an evolution strategy. We show that when significant events are rare, gradient information is also scarce, making it difficult for policy gradient search methods to find an optimal policy, with or without a deep neural architecture. On the other hand, we show that direct policy search methods are invariant to the rarity of significant events, which is yet another confirmation of the unique role evolutionary algorithms has to play as a reinforcement learning method.
Understanding the Origin of Information-Seeking Exploration in Probabilistic Objectives for Control
Venue: arXiv cs.LG 2021
Year: 2021

The exploration-exploitation trade-off is central to the description of adaptive behaviour in fields ranging from machine learning, to biology, to economics. While many approaches have been taken, one approach to solving this trade-off has been to equip or propose that agents possess an intrinsic 'exploratory drive' which is often implemented in terms of maximizing the agents information gain about the world -- an approach which has been widely studied in machine learning and cognitive science. In this paper we mathematically investigate the nature and meaning of such approaches and demonstrate that this combination of utility maximizing and information-seeking behaviour arises from the minimization of an entirely difference class of objectives we call divergence objectives. We propose a dichotomy in the objective functions underlying adaptive behaviour between \emph{evidence} objectives, which correspond to well-known reward or utility maximizing objectives in the literature, and \emph{divergence} objectives which instead seek to minimize the divergence between the agent's expected and desired futures, and argue that this new class of divergence objectives could form the mathematical foundation for a much richer understanding of the exploratory components of adaptive and intelligent action, beyond simply greedy utility maximization.
Learning with partially separable data
Venue: arXiv cs.LG 2021
Year: 2021

There are partially separable data types that make classification tasks very hard. In other words, only parts of the data are informative meaning that looking at the rest of the data would not give any distinguishable hint for classification. In this situation, the typical assumption of having the whole labeled data as an informative unit set for classification does not work. Consequently, typical classification methods with the mentioned assumption fail in such a situation. In this study, we propose a framework for the classification of partially separable data types that are not classifiable using typical methods. An algorithm based on the framework is proposed that tries to detect separable subgroups of the data using an iterative clustering approach. Then the detected subgroups are used in the classification process. The proposed approach was tested on a real dataset for autism screening and showed its capability by distinguishing children with autism from normal ones, while the other methods failed to do so.
Adaptation to Unknown Situations as the Holy Grail of Learning-Based Self-Adaptive Systems: Research Directions
Venue: arXiv cs.AI 2021
Year: 2021

Self-adaptive systems continuously adapt to changes in their execution environment. Capturing all possible changes to define suitable behaviour beforehand is unfeasible, or even impossible in the case of unknown changes, hence human intervention may be required. We argue that adapting to unknown situations is the ultimate challenge for self-adaptive systems. Learning-based approaches are used to learn the suitable behaviour to exhibit in the case of unknown situations, to minimize or fully remove human intervention. While such approaches can, to a certain extent, generalize existing adaptations to new situations, there is a number of breakthroughs that need to be achieved before systems can adapt to general unknown and unforeseen situations. We posit the research directions that need to be explored to achieve unanticipated adaptation from the perspective of learning-based self-adaptive systems. At minimum, systems need to define internal representations of previously unseen situations on-the-fly, extrapolate the relationship to the previously encountered situations to evolve existing adaptations, and reason about the feasibility of achieving their intrinsic goals in the new set of conditions. We close discussing whether, even when we can, we should indeed build systems that define their own behaviour and adapt their goals, without involving a human supervisor.
Classical (Co)Recursion: Programming
Venue: arXiv cs.PL 2021
Year: 2021

Our aim here is to illustrate how the benefits of structural corecursion can be found in a broader swath of the programming landscape than previously thought. Beginning from a tutorial on structural corecursion in the total, pure functional language Agda, we show how these same ideas are mapped to familiar concepts in a variety of different languages. We show how corecursion can be done in strict functional languages like Scheme, and even escapes the functional paradigm entirely, showing up in the natural expression of common object-oriented features found in languages like Python and Java. Opening up structural corecursion to a much wider selection of languages and paradigms -- and therefore, also to a much larger audience of programmers -- lets us also ask how corecursion interacts with computational effects. Of note, we demonstrate that combining structural corecursion with effects can increase its expressive power. We show a classical version of corecursion -- using first-class control made possible by Scheme's classical call/cc -- that enables us to write some new stream-processing algorithms that aren't possible in effect-free languages.
DefakeHop: A Light-Weight High-Performance Deepfake Detector
Venue: arXiv cs.CV 2021
Year: 2021

A light-weight high-performance Deepfake detection method, called DefakeHop, is proposed in this work. State-of-the-art Deepfake detection methods are built upon deep neural networks. DefakeHop extracts features automatically using the successive subspace learning (SSL) principle from various parts of face images. The features are extracted by c/w Saab transform and further processed by our feature distillation module using spatial dimension reduction and soft classification for each channel to get a more concise description of the face. Extensive experiments are conducted to demonstrate the effectiveness of the proposed DefakeHop method. With a small model size of 42,845 parameters, DefakeHop achieves state-of-the-art performance with the area under the ROC curve (AUC) of 100%, 94.95%, and 90.56% on UADFV, Celeb-DF v1 and Celeb-DF v2 datasets, respectively.
(Genetically) Improving Novelty in Procedural Story Generation
Venue: arXiv cs.SE 2021
Year: 2021

Procedural story generation (PCG) tailors a unique narrative experience for a player and can be accomplished via multiple techniques, from matching storylets to grammar-based generation. There exists a rich opportunity for evolutionary algorithms to be applied to this domain for intelligently constructing game narratives. This paper describes a conceptual procedure for applying genetic improvement to a grammar-driven procedural narrative within the context of a browser-based game.
Learning Feature Weights using Reward Modeling for Denoising Parallel Corpora
Venue: arXiv cs.CL 2021
Year: 2021

Large web-crawled corpora represent an excellent resource for improving the performance of Neural Machine Translation (NMT) systems across several language pairs. However, since these corpora are typically extremely noisy, their use is fairly limited. Current approaches to dealing with this problem mainly focus on filtering using heuristics or single features such as language model scores or bi-lingual similarity. This work presents an alternative approach which learns weights for multiple sentence-level features. These feature weights which are optimized directly for the task of improving translation performance, are used to score and filter sentences in the noisy corpora more effectively. We provide results of applying this technique to building NMT systems using the Paracrawl corpus for Estonian-English and show that it beats strong single feature baselines and hand designed combinations. Additionally, we analyze the sensitivity of this method to different types of noise and explore if the learned weights generalize to other language pairs using the Maltese-English Paracrawl corpus.
Development of recommendation systems for software engineering: the CROSSMINER experience
Venue: arXiv cs.SE 2021
Year: 2021

To perform their daily tasks, developers intensively make use of existing resources by consulting open-source software (OSS) repositories. Such platforms contain rich data sources, e.g., code snippets, documentation, and user discussions, that can be useful for supporting development activities. Over the last decades, several techniques and tools have been promoted to provide developers with innovative features, aiming to bring in improvements in terms of development effort, cost savings, and productivity. In the context of the EU H2020 CROSSMINER project, a set of recommendation systems has been conceived to assist software programmers in different phases of the development process. The systems provide developers with various artifacts, such as third-party libraries, documentation about how to use the APIs being adopted, or relevant API function calls. To develop such recommendations, various technical choices have been made to overcome issues related to several aspects including the lack of baselines, limited data availability, decisions about the performance measures, and evaluation approaches. This paper is an experience report to present the knowledge pertinent to the set of recommendation systems developed through the CROSSMINER project. We explain in detail the challenges we had to deal with, together with the related lessons learned when developing and evaluating these systems. Our aim is to provide the research community with concrete takeaway messages that are expected to be useful for those who want to develop or customize their own recommendation systems. The reported experiences can facilitate interesting discussions and research work, which in the end contribute to the advancement of recommendation systems applied to solve different issues in Software Engineering.
Quantifying the Efficacy of Logic Locking Methods
Venue: arXiv cs.CR 2021
Year: 2021

The outsourced manufacturing of integrated circuits has increased the risk of intellectual property theft. In response, logic locking techniques have been developed for protecting designs by adding programmable elements to the circuit. These techniques differ significantly in both overhead and resistance to various attacks, leaving designers unable to discern their efficacy. To overcome this critical impediment for the adoption of logic locking, we propose two metrics, key corruption and minimum corruption, that capture the goals of locking under different attack scenarios. We develop a flow for approximating these metrics on generic locked circuits and evaluate several locking techniques.
Interleaving Learning, with Application to Neural Architecture Search
Venue: arXiv cs.LG 2021
Year: 2021

Interleaving learning is a human learning technique where a learner interleaves the studies of multiple topics, which increases long-term retention and improves ability to transfer learned knowledge. Inspired by the interleaving learning technique of humans, in this paper we explore whether this learning methodology is beneficial for improving the performance of machine learning models as well. We propose a novel machine learning framework referred to as interleaving learning (IL). In our framework, a set of models collaboratively learn a data encoder in an interleaving fashion: the encoder is trained by model 1 for a while, then passed to model 2 for further training, then model 3, and so on; after trained by all models, the encoder returns back to model 1 and is trained again, then moving to model 2, 3, etc. This process repeats for multiple rounds. Our framework is based on multi-level optimization consisting of multiple inter-connected learning stages. An efficient gradient-based algorithm is developed to solve the multi-level optimization problem. We apply interleaving learning to search neural architectures for image classification on CIFAR-10, CIFAR-100, and ImageNet. The effectiveness of our method is strongly demonstrated by the experimental results.
Reptile: Aggregation-level Explanations for Hierarchical Data
Venue: arXiv cs.DB 2021
Year: 2021

Recent query explanation systems help users understand anomalies in aggregation results by proposing predicates that describe input records that, if deleted, would resolve the anomalies. However, it can be difficult for users to understand how a predicate was chosen, and these approaches are limited to errors that can be resolved through deletion. In contrast, data errors may be due to group-wise errors, such as missing records or systematic value errors. This paper presents Reptile, an explanation system for hierarchical data. Given an anomalous aggregate query result, Reptile recommends the next drill-down attribute,and ranks the drill-down groups based on the extent repairing the group's statistics to its expected values resolves the anomaly. Reptile efficiently trains a multi-level model that leverages the data's hierarchy to estimate the expected values, and uses a factorised representation of the feature matrix to remove redundancies due to the data's hierarchical structure. We further extend model training to support factorised data, and develop a suite of optimizations that leverage the data's hierarchical structure. Reptile reduces end-to-end runtimes by more than 6 times compared to a Matlab-based implementation, correctly identifies 21/30 data errors in John Hopkin's COVID-19 data, and correctly resolves 20/22 complaints in a user study using data and researchers from Columbia University's Financial Instruments Sector Team.
DynACPD Embedding Algorithm for Prediction Tasks in Dynamic Networks
Venue: arXiv cs.AI 2021
Year: 2021

Classical network embeddings create a low dimensional representation of the learned relationships between features across nodes. Such embeddings are important for tasks such as link prediction and node classification. In the current paper, we consider low dimensional embeddings of dynamic networks, that is a family of time varying networks where there exist both temporal and spatial link relationships between nodes. We present novel embedding methods for a dynamic network based on higher order tensor decompositions for tensorial representations of the dynamic network. In one sense, our embeddings are analogous to spectral embedding methods for static networks. We provide a rationale for our algorithms via a mathematical analysis of some potential reasons for their effectiveness. Finally, we demonstrate the power and efficiency of our approach by comparing our algorithms' performance on the link prediction task against an array of current baseline methods across three distinct real-world dynamic networks.
A Weakly Supervised Approach for Classifying Stance in Twitter Replies
Venue: arXiv cs.CL 2021
Year: 2021

Conversations on social media (SM) are increasingly being used to investigate social issues on the web, such as online harassment and rumor spread. For such issues, a common thread of research uses adversarial reactions, e.g., replies pointing out factual inaccuracies in rumors. Though adversarial reactions are prevalent in online conversations, inferring those adverse views (or stance) from the text in replies is difficult and requires complex natural language processing (NLP) models. Moreover, conventional NLP models for stance mining need labeled data for supervised learning. Getting labeled conversations can itself be challenging as conversations can be on any topic, and topics change over time. These challenges make learning the stance a difficult NLP problem. In this research, we first create a new stance dataset comprised of three different topics by labeling both users' opinions on the topics (as in pro/con) and users' stance while replying to others' posts (as in favor/oppose). As we find limitations with supervised approaches, we propose a weakly-supervised approach to predict the stance in Twitter replies. Our novel method allows using a smaller number of hashtags to generate weak labels for Twitter replies. Compared to supervised learning, our method improves the mean F1-macro by 8\% on the hand-labeled dataset without using any hand-labeled examples in the training set. We further show the applicability of our proposed method on COVID 19 related conversations on Twitter.
An Empirical Study on the Usage of BERT Models for Code Completion
Venue: arXiv cs.SE 2021
Year: 2021

Code completion is one of the main features of modern Integrated Development Environments (IDEs). Its objective is to speed up code writing by predicting the next code token(s) the developer is likely to write. Research in this area has substantially bolstered the predictive performance of these techniques. However, the support to developers is still limited to the prediction of the next few tokens to type. In this work, we take a step further in this direction by presenting a large-scale empirical study aimed at exploring the capabilities of state-of-the-art deep learning (DL) models in supporting code completion at different granularity levels, including single tokens, one or multiple entire statements, up to entire code blocks (e.g., the iterated block of a for loop). To this aim, we train and test several adapted variants of the recently proposed RoBERTa model, and evaluate its predictions from several perspectives, including: (i) metrics usually adopted when assessing DL generative models (i.e., BLEU score and Levenshtein distance); (ii) the percentage of perfect predictions (i.e., the predicted code snippets that match those written by developers); and (iii) the "semantic" equivalence of the generated code as compared to the one written by developers. The achieved results show that BERT models represent a viable solution for code completion, with perfect predictions ranging from ~7%, obtained when asking the model to guess entire blocks, up to ~58%, reached in the simpler scenario of few tokens masked from the same code statement.
Evaluation Framework for Performance Limitation of Autonomous Systems under Sensor Attack
Venue: arXiv cs.CR 2021
Year: 2021

Autonomous systems such as self-driving cars rely on sensors to perceive the surrounding world. Measures must be taken against attacks on sensors, which have been a hot topic in the last few years. For that goal one must first evaluate how sensor attacks affect the system, i.e. which part or whole of the system will fail if some of the built-in sensors are compromised, or will keep safe, etc. Among the relevant safety standards, ISO/PAS 21448 addresses the safety of road vehicles taking into account the performance limitations of sensors, but leaves security aspects out of scope. On the other hand, ISO/SAE 21434 addresses the security perspective during the development process of vehicular systems, but not specific threats such as sensor attacks. As a result the safety of autonomous systems under sensor attack is yet to be addressed. In this paper we propose a framework that combines safety analysis for scenario identification, and scenario-based simulation with sensor attack models embedded. Given an autonomous system model, we identify hazard scenarios caused by sensor attacks, and evaluate the performance limitations in the scenarios. We report on a prototype simulator for autonomous vehicles with radar, cameras and LiDAR along with attack models against the sensors. Our experiments show that our framework can evaluate how the system safety changes as parameters of the attacks and the sensors vary.
Thousand to One: Semantic Prior Modeling for Conceptual Coding
Venue: arXiv cs.CV 2021
Year: 2021

Conceptual coding has been an emerging research topic recently, which encodes natural images into disentangled conceptual representations for compression. However, the compression performance of the existing methods is still sub-optimal due to the lack of comprehensive consideration of rate constraint and reconstruction quality. To this end, we propose a novel end-to-end semantic prior modeling-based conceptual coding scheme towards extremely low bitrate image compression, which leverages semantic-wise deep representations as a unified prior for entropy estimation and texture synthesis. Specifically, we employ semantic segmentation maps as structural guidance for extracting deep semantic prior, which provides fine-grained texture distribution modeling for better detail construction and higher flexibility in subsequent high-level vision tasks. Moreover, a cross-channel entropy model is proposed to further exploit the inter-channel correlation of the spatially independent semantic prior, leading to more accurate entropy estimation for rate-constrained training. The proposed scheme achieves an ultra-high 1000x compression ratio, while still enjoying high visual reconstruction quality and versatility towards visual processing and analysis tasks.
A Risk-taking Broker Model to Optimise User Requests placement on On-demand and Contract VMs
Venue: arXiv cs.DC 2021
Year: 2021

Cloud providers offer end-users various pricing schemes to allow them to tailor VMs to their needs, e.g., a pay-as-you-go billing scheme, called \textit{on-demand}, and a discounted contract scheme, called \textit{reserved instances}. This paper presents a cloud broker which offers users both the flexibility of on-demand instances and some level of discounts found in reserved instances. The broker employs a buy-low-and-sell-high strategy that places user requests into a resource pool of pre-purchased discounted cloud resources. By analysing user request time-series data, the broker takes a risk-oriented approach to dynamically adjust the resource pool. This approach does not require a training process which is useful at processing the large data stream. The broker is evaluated with high-frequency real cloud datasets from Alibaba. The results show that the overall profit of the broker is close to the theoretical optimal scenario where user requests can be perfectly predicted.