OpenRaiser/Intern-Atlas huggingface dataset:
MarioMix: Creating Aligned Playstyles for Bots with Interactive Reinforcement Learning
Venue: arXiv cs.HC 2021
Year: 2021

In this paper, we propose a generic framework that enables game developers without knowledge of machine learning to create bot behaviors with playstyles that align with their preferences. Our framework is based on interactive reinforcement learning (RL), and we used it to create a behavior authoring tool called MarioMix. This tool enables non-experts to create bots with varied playstyles for the game titled Super Mario Bros. The main interaction procedure of MarioMix consists of presenting short clips of gameplay displaying precomputed bots with different playstyles to end-users. Then, end-users can select the bot with the playstyle that behaves as intended. We evaluated MarioMix by incorporating input from game designers working in the industry.
A Survey on Interactive Reinforcement Learning: Design Principles and Open Challenges
Venue: arXiv cs.HC 2021
Year: 2021

Interactive reinforcement learning (RL) has been successfully used in various applications in different fields, which has also motivated HCI researchers to contribute in this area. In this paper, we survey interactive RL to empower human-computer interaction (HCI) researchers with the technical background in RL needed to design new interaction techniques and propose new applications. We elucidate the roles played by HCI researchers in interactive RL, identifying ideas and promising research directions. Furthermore, we propose generic design principles that will provide researchers with a guide to effectively implement interactive RL applications.
Investigating label suggestions for opinion mining in German Covid-19 social media
Venue: arXiv cs.CL 2021
Year: 2021

This work investigates the use of interactively updated label suggestions to improve upon the efficiency of gathering annotations on the task of opinion mining in German Covid-19 social media data. We develop guidelines to conduct a controlled annotation study with social science students and find that suggestions from a model trained on a small, expert-annotated dataset already lead to a substantial improvement - in terms of inter-annotator agreement(+.14 Fleiss' $κ$) and annotation quality - compared to students that do not receive any label suggestions. We further find that label suggestions from interactively trained models do not lead to an improvement over suggestions from a static model. Nonetheless, our analysis of suggestion bias shows that annotators remain capable of reflecting upon the suggested label in general. Finally, we confirm the quality of the annotated data in transfer learning experiments between different annotator groups. To facilitate further research in opinion mining on social media data, we release our collected data consisting of 200 expert and 2,785 student annotations.
A functionality taxonomy for document search engines
Venue: arXiv cs.IR 2021
Year: 2021

In this paper a functionality taxonomy for document search engines is proposed. It can be used to assess the features of a search engine, to position search engines relative to each other, or to select which search engine 'fits' a certain situation. One is able to identify areas for improvement. During development, we were guided by the viewpoint of the user. We use the word `search engine' in the broadest sense possible, including library and web based (meta) search engines. The taxonomy distinguishes seven functionality areas: an indexing service, user profiling, query composition, query execution, result presentation, result refinement, and history keeping. Each of these relates and provides services to other functionality areas. It can be extended whenever necessary. To illustrate the validity of our taxonomy, it has been used for comparing various document search engines existing today (ACM Digital Library, PiCarta, Copernic, AltaVista, Google, and GuideBeam). It appears that the functionality aspects covered by our taxonomy can be used for describing these search engines.
TranSmart: A Practical Interactive Machine Translation System
Venue: arXiv cs.CL 2021
Year: 2021

Automatic machine translation is super efficient to produce translations yet their quality is not guaranteed. This technique report introduces TranSmart, a practical human-machine interactive translation system that is able to trade off translation quality and efficiency. Compared to existing publicly available interactive translation systems, TranSmart supports three key features, word-level autocompletion, sentence-level autocompletion and translation memory. By word-level and sentence-level autocompletion, TranSmart allows users to interactively translate words in their own manners rather than the strict manner from left to right. In addition, TranSmart has the potential to avoid similar translation mistakes by using translated sentences in history as its memory. This report presents major functions of TranSmart, algorithms for achieving these functions, how to use the TranSmart APIs, and evaluation results of some key functions. TranSmart is publicly available at its homepage (https://transmart.qq.com).
Path-based knowledge reasoning with textual semantic information for medical knowledge graph completion
Venue: arXiv cs.AI 2021
Year: 2021

Background Knowledge graphs (KGs), especially medical knowledge graphs, are often significantly incomplete, so it necessitating a demand for medical knowledge graph completion (MedKGC). MedKGC can find new facts based on the exited knowledge in the KGs. The path-based knowledge reasoning algorithm is one of the most important approaches to this task. This type of method has received great attention in recent years because of its high performance and interpretability. In fact, traditional methods such as path ranking algorithm (PRA) take the paths between an entity pair as atomic features. However, the medical KGs are very sparse, which makes it difficult to model effective semantic representation for extremely sparse path features. The sparsity in the medical KGs is mainly reflected in the long-tailed distribution of entities and paths. Previous methods merely consider the context structure in the paths of the knowledge graph and ignore the textual semantics of the symbols in the path. Therefore, their performance cannot be further improved due to the two aspects of entity sparseness and path sparseness. To address the above issues, this paper proposes two novel path-based reasoning methods to solve the sparsity issues of entity and path respectively, which adopts the textual semantic information of entities and paths for MedKGC. By using the pre-trained model BERT, combining the textual semantic representations of the entities and the relationships, we model the task of symbolic reasoning in the medical KG as a numerical computing issue in textual semantic representation.
A framework for data-driven solution and parameter estimation of PDEs using conditional generative adversarial networks
Venue: arXiv cs.LG 2021
Year: 2021

This work is the first to employ and adapt the image-to-image translation concept based on conditional generative adversarial networks (cGAN) towards learning a forward and an inverse solution operator of partial differential equations (PDEs). Even though the proposed framework could be applied as a surrogate model for the solution of any PDEs, here we focus on steady-state solutions of coupled hydro-mechanical processes in heterogeneous porous media. Strongly heterogeneous material properties, which translate to the heterogeneity of coefficients of the PDEs and discontinuous features in the solutions, require specialized techniques for the forward and inverse solution of these problems. Additionally, parametrization of the spatially heterogeneous coefficients is excessively difficult by using standard reduced order modeling techniques. In this work, we overcome these challenges by employing the image-to-image translation concept to learn the forward and inverse solution operators and utilize a U-Net generator and a patch-based discriminator. Our results show that the proposed data-driven reduced order model has competitive predictive performance capabilities in accuracy and computational efficiency as well as training time requirements compared to state-of-the-art data-driven methods for both forward and inverse problems.
Graph-Based Deep Learning for Medical Diagnosis and Analysis: Past, Present and Future
Venue: arXiv cs.LG 2021
Year: 2021

With the advances of data-driven machine learning research, a wide variety of prediction problems have been tackled. It has become critical to explore how machine learning and specifically deep learning methods can be exploited to analyse healthcare data. A major limitation of existing methods has been the focus on grid-like data; however, the structure of physiological recordings are often irregular and unordered which makes it difficult to conceptualise them as a matrix. As such, graph neural networks have attracted significant attention by exploiting implicit information that resides in a biological system, with interactive nodes connected by edges whose weights can be either temporal associations or anatomical junctions. In this survey, we thoroughly review the different types of graph architectures and their applications in healthcare. We provide an overview of these methods in a systematic manner, organized by their domain of application including functional connectivity, anatomical structure and electrical-based analysis. We also outline the limitations of existing techniques and discuss potential directions for future research.
One Step Preference Elicitation in Multi-Objective Bayesian Optimization
Venue: arXiv cs.LG 2021
Year: 2021

We consider a multi-objective optimization problem with objective functions that are expensive to evaluate. The decision maker (DM) has unknown preferences, and so the standard approach is to generate an approximation of the Pareto front and let the DM choose from the generated non-dominated designs. However, especially for expensive to evaluate problems where the number of designs that can be evaluated is very limited, the true best solution according to the DM's unknown preferences is unlikely to be among the small set of non-dominated solutions found, even if these solutions are truly Pareto optimal. We address this issue by using a multi-objective Bayesian optimization algorithm and allowing the DM to select a preferred solution from a predicted continuous Pareto front just once before the end of the algorithm rather than selecting a solution after the end. This allows the algorithm to understand the DM's preferences and make a final attempt to identify a more preferred solution. We demonstrate the idea using ParEGO, and show empirically that the found solutions are significantly better in terms of true DM preferences than if the DM would simply pick a solution at the end.
Concept drift detection and adaptation for federated and continual learning
Venue: arXiv cs.LG 2021
Year: 2021

Smart devices, such as smartphones, wearables, robots, and others, can collect vast amounts of data from their environment. This data is suitable for training machine learning models, which can significantly improve their behavior, and therefore, the user experience. Federated learning is a young and popular framework that allows multiple distributed devices to train deep learning models collaboratively while preserving data privacy. Nevertheless, this approach may not be optimal for scenarios where data distribution is non-identical among the participants or changes over time, causing what is known as concept drift. Little research has yet been done in this field, but this kind of situation is quite frequent in real life and poses new challenges to both continual and federated learning. Therefore, in this work, we present a new method, called Concept-Drift-Aware Federated Averaging (CDA-FedAvg). Our proposal is an extension of the most popular federated algorithm, Federated Averaging (FedAvg), enhancing it for continual adaptation under concept drift. We empirically demonstrate the weaknesses of regular FedAvg and prove that CDA-FedAvg outperforms it in this type of scenario.
Comparing Alternatives to the Fixed Degree Sequence Model for Extracting the Backbone of Bipartite Projections
Venue: arXiv cs.SI 2021
Year: 2021

Projections of bipartite or two-mode networks capture co-occurrences, and are used in diverse fields (e.g., ecology, economics, bibliometrics, politics) to represent unipartite networks. A key challenge in analyzing such networks is determining whether an observed number of co-occurrences between two nodes is significant, and therefore whether an edge exists between them. One approach, the fixed degree sequence model (FDSM), evaluates the significance of an edge's weight by comparison to a null model in which the degree sequences of the original bipartite network are fixed. Although the FDSM is an intuitive null model, it is computationally expensive because it requires Monte Carlo simulation to estimate each edge's $p$-value, and therefore is impractical for large projections. In this paper, we explore four potential alternatives to FDSM: fixed fill model (FFM), fixed row model (FRM), fixed column model (FCM), and stochastic degree sequence model (SDSM). We compare these models to FDSM in terms of accuracy, speed, statistical power, similarity, and ability to recover known communities. We find that the computationally-fast SDSM offers a statistically conservative but close approximation of the computationally-impractical FDSM under a wide range of conditions, and that it correctly recovers a known community structure even when the signal is weak. Therefore, although each backbone model may have particular applications, we recommend SDSM for extracting the backbone of bipartite projections when FDSM is impractical.
Intrusion Detection using Machine Learning Techniques: An Experimental Comparison
Venue: arXiv cs.CR 2021
Year: 2021

Due to an exponential increase in the number of cyber-attacks, the need for improved Intrusion Detection Systems (IDS) is apparent than ever. In this regard, Machine Learning (ML) techniques are playing a pivotal role in the early classification of the attacks in case of intrusion detection within the system. However, due to a large number of algorithms available, the selection of the right method is a challenging task. To resolve this issue, this paper analyses some of the current state-of-the-art intrusion detection methods and discusses their pros and cons. Further, a review of different ML methods is carried out with four methods showing to be the most suitable one for classifying attacks. Several algorithms are selected and investigated to evaluate the performance of IDS. These IDS classifies binary and multiclass attacks in terms of detecting whether or not the traffic has been considered as benign or an attack. The experimental results demonstrate that binary classification has greater consistency in their accuracy results which ranged from 0.9938 to 0.9977, while multiclass ranges from 0.9294 to 0.9983. However, it has been also observed that multiclass provides the best results with the algorithm k-Nearest neighbor giving an accuracy score of 0.9983 while the binary classification highest score is 0.9977 from Random Forest. The experimental results demonstrate that multiclass classification produces better performance in terms of intrusion detection by specifically differentiating between the attacks and allowing a more targeted response to an attack.
Computational modeling of the nonlinear stochastic dynamics of horizontal drillstrings
Venue: arXiv cs.CE 2021
Year: 2021

This work intends to analyze the nonlinear stochastic dynamics of drillstrings in horizontal configuration. For this purpose, it considers a beam theory, with effects of rotatory inertia and shear deformation, which is capable of reproducing the large displacements that the beam undergoes. The friction and shock effects, due to beam/borehole wall transversal impacts, as well as the force and torque induced by bit-rock interaction, are also considered in the model. Uncertainties of bit-rock interaction model are taken into account using a parametric probabilistic approach. Numerical simulations have shown that the mechanical system of interest has a very rich nonlinear stochastic dynamics, which generate phenomena such as bit-bounce, stick-slip, and transverse impacts. A study aiming to maximize the drilling process efficiency, varying drillstring velocities of translation and rotation is presented. Also, the work presents the definition and solution of two optimizations problems, one deterministic and one robust, where the objective is to maximize drillstring rate of penetration into the soil respecting its structural limits.
Learning Model-Based Vehicle-Relocation Decisions for Real-Time Ride-Sharing: Hybridizing Learning and Optimization
Venue: arXiv cs.AI 2021
Year: 2021

Large-scale ride-sharing systems combine real-time dispatching and routing optimization over a rolling time horizon with a model predictive control (MPC) component that relocates idle vehicles to anticipate the demand. The MPC optimization operates over a longer time horizon to compensate for the inherent myopic nature of the real-time dispatching. These longer time horizons are beneficial for the quality of relocation decisions but increase computational complexity. Consequently, the ride-sharing operators are often forced to use a relatively short time horizon. To address this computational challenge, this paper proposes a hybrid approach that combines machine learning and optimization. The machine-learning component learns the optimal solution to the MPC on the aggregated level to overcome the sparsity and high-dimensionality of the solution. The optimization component transforms the machine-learning prediction back to the original granularity through a tractable transportation model. As a consequence, the original NP-hard MPC problem is reduced to a polynomial time prediction and optimization, which allows the ride-sharing operators to consider a longer time horizon. Experimental results show that the hybrid approach achieves significantly better service quality than the MPC optimization in terms of average rider waiting time, due to its ability to model a longer horizon.
Inspecting the concept knowledge graph encoded by modern language models
Venue: arXiv cs.AI 2021
Year: 2021

The field of natural language understanding has experienced exponential progress in the last few years, with impressive results in several tasks. This success has motivated researchers to study the underlying knowledge encoded by these models. Despite this, attempts to understand their semantic capabilities have not been successful, often leading to non-conclusive, or contradictory conclusions among different works. Via a probing classifier, we extract the underlying knowledge graph of nine of the most influential language models of the last years, including word embeddings, text generators, and context encoders. This probe is based on concept relatedness, grounded on WordNet. Our results reveal that all the models encode this knowledge, but suffer from several inaccuracies. Furthermore, we show that the different architectures and training strategies lead to different model biases. We conduct a systematic evaluation to discover specific factors that explain why some concepts are challenging. We hope our insights will motivate the development of models that capture concepts more precisely.
Early Experiences Migrating CUDA codes to oneAPI
Venue: arXiv cs.DC 2021
Year: 2021

The heterogeneous computing paradigm represents a real programming challenge due to the proliferation of devices with different hardware characteristics. Recently Intel introduced oneAPI, a new programming environment that allows code developed in DPC++ to be run on different devices such as CPUs, GPUs, FPGAs, among others. This paper presents our first experiences in porting two CUDA applications to DPC++ using the oneAPI dpct tool. From the experimental work, it was possible to verify that dpct does not achieve 100% of the migration task; however, it performs most of the work, reporting the programmer of possible pending adaptations. Additionally, it was possible to verify the functional portability of the DPC++ code obtained, having successfully executed it on different CPU and GPU architectures.
A Note on Optimal Fees for Constant Function Market Makers
Venue: arXiv cs.GT 2021
Year: 2021

We suggest a framework to determine optimal trading fees for constant function market makers (CFMMs) in order to maximize liquidity provider returns. In a setting of multiple competing liquidity pools, we show that no race to the bottom occurs, but instead pure Nash equilibria of optimal fees exist. We theoretically prove the existence of these equilibria for pools using the constant product trade function used in popular CFMMs like Uniswap. We also numerically compute the equilibria for a number of examples and discuss the effects the equilibrium fees have on capital allocation among pools. Finally, we use our framework to compute optimal fees for real world pools using past trade data.
Stochastic Intervention for Causal Inference via Reinforcement Learning
Venue: arXiv cs.AI 2021
Year: 2021

Causal inference methods are widely applied in various decision-making domains such as precision medicine, optimal policy and economics. Central to causal inference is the treatment effect estimation of intervention strategies, such as changes in drug dosing and increases in financial aid. Existing methods are mostly restricted to the deterministic treatment and compare outcomes under different treatments. However, they are unable to address the substantial recent interest of treatment effect estimation under stochastic treatment, e.g., "how all units health status change if they adopt 50\% dose reduction". In other words, they lack the capability of providing fine-grained treatment effect estimation to support sound decision-making. In our study, we advance the causal inference research by proposing a new effective framework to estimate the treatment effect on stochastic intervention. Particularly, we develop a stochastic intervention effect estimator (SIE) based on nonparametric influence function, with the theoretical guarantees of robustness and fast convergence rates. Additionally, we construct a customised reinforcement learning algorithm based on the random search solver which can effectively find the optimal policy to produce the greatest expected outcomes for the decision-making process. Finally, we conduct an empirical study to justify that our framework can achieve significant performance in comparison with state-of-the-art baselines.