Reviewerly/exHarmony huggingface dataset:
Multi-agent Inverse Reinforcement Learning for Certain General-sum Stochastic Games

This paper addresses the problem of multi-agent inverse reinforcement learning (MIRL) in a two-player general-sum stochastic game framework. Five variants of MIRL are considered: uCS-MIRL, advE-MIRL, cooE-MIRL, uCE-MIRL, and uNE-MIRL, each distinguished by its solution concept. Problem uCS-MIRL is a cooperative game in which the agents employ cooperative strategies that aim to maximize the total game value. In problem uCE-MIRL, agents are assumed to follow strategies that constitute a correlated equilibrium while maximizing total game value. Problem uNE-MIRL is similar to uCE-MIRL in total game value maximization, but it is assumed that the agents are playing a Nash equilibrium. Problems advE-MIRL and cooE-MIRL assume agents are playing an adversarial equilibrium and a coordination equilibrium, respectively. We propose novel approaches to address these five problems under the assumption that the game observer either knows or is able to accurate estimate the policies and solution concepts for players. For uCS-MIRL, we first develop a characteristic set of solutions ensuring that the observed bi-policy is a uCS and then apply a Bayesian inverse learning method. For uCE-MIRL, we develop a linear programming problem subject to constraints that define necessary and sufficient conditions for the observed policies to be correlated equilibria. The objective is to choose a solution that not only minimizes the total game value difference between the observed bi-policy and a local uCS, but also maximizes the scale of the solution. We apply a similar treatment to the problem of uNE-MIRL. The remaining two problems can be solved efficiently by taking advantage of solution uniqueness and setting up a convex optimization problem. Results are validated on various benchmark grid-world games.
Prediction of perioperative transfusions using an artificial neural network

Background Accurate prediction of operative transfusions is essential for resource allocation and identifying patients at risk of postoperative adverse events. This research examines the efficacy of using artificial neural networks (ANNs) to predict transfusions for all inpatient operations. Methods Over 1.6 million surgical cases over a two year period from the NSQIP-PUF database are used. Data from 2014 (750937 records) are used for model development and data from 2015 (885502 records) are used for model validation. ANN and regression models are developed to predict perioperative transfusions for surgical patients. Results Various ANN models and logistic regression, using four variable sets, are compared. The best performing ANN models with respect to both sensitivity and area under the receiver operator characteristic curve outperformed all of the regression models (p < .001) and achieved a performance of 70–80% specificity with a corresponding 75–62% sensitivity. Conclusion ANNs can predict >75% of the patients who will require transfusion and 70% of those who will not. Increasing specificity to 80% still enables a sensitivity of almost 67%. The unique contribution of this research is the utilization of a single ANN model to predict transfusions across a broad range of surgical procedures.
Quantum Poker—a game for quantum computers suitable for benchmarking error mitigation techniques on NISQ devices

Abstract Quantum computers are on the verge of becoming a commercially available reality. They represent a paradigm shift in computing, with a steep learning gradient. The creation of games is a way to ease the transition for beginners. We present a game similar to the Poker variant Texas hold ’em with the intention to serve as an engaging pedagogical tool to learn the basics rules of quantum computing. The concepts of quantum states, quantum operations and measurement can be learned in a playful manner. The difference to the classical variant is that the community cards are replaced by a quantum register that is “randomly” initialized, and the cards for each player are replaced by quantum gates, randomly drawn from a set of available gates. Each player can create a quantum circuit with their cards, with the aim to maximize the number of 1’s that are measured in the computational basis. The basic concepts of superposition, entanglement and quantum gates are employed. We provide a proof-of-concept implementation using Qiskit (Aleksandrowicz et al. in An open-source framework for quantum computing, 2019). A comparison of the results for the created circuits using a simulator and IBM machines is conducted, showing that error rates on contemporary quantum computers are still very high. For the success of noisy intermediate scale quantum (NISQ) computers, improvements on the error rates and error mitigation techniques are necessary, even for simple circuits. We show that quantum error mitigation (QEM) techniques can be used to improve expectation values of observables on real quantum devices.
Towards countering hate speech against journalists on social media

The damaging effects of hate speech on social media are evident during the last few years, and several organizations, researchers and social media platforms tried to harness them in various ways. Despite these efforts, social media users are still affected by hate speech. The problem is even more apparent to social groups that promote public discourse, such as journalists. In this work, we focus on countering hate speech that is targeted to journalistic social media accounts. To accomplish this, a group of journalists assembled a definition of hate speech, taking into account the journalistic point of view and the types of hate speech that are usually targeted against journalists. We then compile a large pool of tweets referring to journalism-related accounts in multiple languages. In order to annotate the pool of unlabeled tweets according to the definition, we follow a concise annotation strategy that involves active learning annotation stages. The outcome of this paper is a novel, publicly available collection of Twitter datasets in five different languages. Additionally, we experiment with state-of-the-art deep learning architectures for hate speech detection and use our annotated datasets to train and evaluate them. Finally, we propose an ensemble detection model that outperforms all individual models.
Learning From Synthetic Images via Active Pseudo-Labeling

Synthetic visual data refers to the data automatically rendered by the mature computer graphic algorithms. With the rapid development of these techniques, we can now collect photo-realistic synthetic images with accurate pixel-level annotations without much effort. However, due to the domain gaps between synthetic data and real data, in terms of not only visual appearance but also label distribution, directly applying models trained on synthetic images to real ones can hardly yield satisfactory performance. Since the collection of accurate labels for real images is very laborious and time-consuming, developing algorithms which can learn from synthetic images is of great significance. In this paper, we propose a novel framework, namely Active Pseudo-Labeling (APL), to reduce the domain gaps between synthetic images and real images. In APL framework, we first predict pseudo-labels for the unlabeled real images in the target domain by actively adapting the style of the real images to source domain. Specifically, the style of real images is adjusted via a novel task guided generative model, and then pseudo-labels are predicted for these actively adapted images. Lastly, we fine-tune the source-trained model in the pseudo-labeled target domain, which helps to fit the distribution of the real data. Experiments on both semantic segmentation and object detection tasks with several challenging benchmark data sets demonstrate the priority of our proposed method compared to the existing state-of-the-art approaches.
A Microservices e-Health System for Ecological Frailty Assessment Using Wearables

The population in developed countries is aging and this fact results in high elderly health costs, as well as a decrease in the number of active working members to support these costs. This could lead to a collapse of the current systems. One of the first insights of the decline in elderly people is frailty, which could be decelerated if it is detected at an early stage. Nowadays, health professionals measure frailty manually through questionnaires and tests of strength or gait focused on the physical dimension. Sensors are increasingly used to measure and monitor different e-health indicators while the user is performing Basic Activities of Daily Life (BADL). In this paper, we present a system based on microservices architecture, which collects sensory data while the older adults perform Instrumental ADLs (IADLs) in combination with BADLs. IADLs involve physical dimension, but also cognitive and social dimensions. With the sensory data we built a machine learning model to assess frailty status which outperforms the previous works that only used BADLs. Our model is accurate, ecological, non-intrusive, flexible and can help health professionals to automatically detect frailty.
t-soft update of target network for deep reinforcement learning

This paper proposes a new robust update rule of target network for deep reinforcement learning (DRL), to replace the conventional update rule, given as an exponential moving average. The target network is for smoothly generating the reference signals for a main network in DRL, thereby reducing learning variance. The problem with its conventional update rule is the fact that all the parameters are smoothly copied with the same speed from the main network, even when some of them are trying to update toward the wrong directions. This behavior increases the risk of generating the wrong reference signals. Although slowing down the overall update speed is a naive way to mitigate wrong updates, it would decrease learning speed. To robustly update the parameters while keeping learning speed, a t-soft update method, which is inspired by student-t distribution, is derived with reference to the analogy between the exponential moving average and the normal distribution. Through the analysis of the derived t-soft update, we show that it takes over the properties of the student-t distribution. Specifically, with a heavy-tailed property of the student-t distribution, the t-soft update automatically excludes extreme updates that differ from past experiences. In addition, when the updates are similar to the past experiences, it can mitigate the learning delay by increasing the amount of updates. In PyBullet robotics simulations for DRL, an online actor-critic algorithm with the t-soft update outperformed the conventional methods in terms of the obtained return and/or its variance. From the training process by the t-soft update, we found that the t-soft update is globally consistent with the standard soft update, and the update rates are locally adjusted for acceleration or suppression.
A Primer on Usability Assessment Approaches for Health-Related Applications of Virtual Reality

Health-related virtual reality (VR) applications for patient treatment, rehabilitation, and medical professional training are on the rise. However, there is little guidance on how to select and perform usability evaluations for VR health interventions compared to the supports that exist for other digital health technologies. The purpose of this viewpoint paper is to present an introductory summary of various usability testing approaches or methods that can be used for VR applications. Along with an overview of each, a list of resources is provided for readers to obtain additionally relevant information. Six categories of VR usability evaluations are described using a previously developed classification taxonomy specific to VR environments: (1) cognitive or task walkthrough, (2) graphical evaluation, (3) post hoc questionnaires or interviews, (4) physical performance evaluation, (5) user interface evaluation, and (6) heuristic evaluation. Given the growth of VR in health care, rigorous evaluation and usability testing is crucial in the development and implementation of novel VR interventions. The approaches outlined in this paper provide a starting point for conducting usability assessments for health-related VR applications; however, there is a need to also move beyond these to adopt those from the gaming industry, where assessments for both usability and user experience are routinely conducted.
Learning a Single Model With a Wide Range of Quality Factors for JPEG Image Artifacts Removal

Lossy compression brings artifacts into the compressed image and degrades the visual quality. In recent years, many compression artifacts removal methods based on convolutional neural network (CNN) have been developed with great success. However, these methods usually train a model based on one specific value or a small range of quality factors. Obviously, if the test images quality factor does not match to the assumed value range, then degraded performance will be resulted. With this motivation and further consideration of practical usage, a highly robust compression artifacts removal network is proposed in this paper. Our proposed network is a single model approach that can be trained for handling a wide range of quality factors while consistently delivering superior or comparable image artifacts removal performance. To demonstrate, we focus on the JPEG compression with quality factors, ranging from 1 to 60. Note that a turnkey success of our proposed network lies in the novel utilization of the quantization tables as part of the training data. Furthermore, it has two branches in parallel-i.e., the restoration branch and the global branch. The former effectively removes the local artifacts, such as ringing artifacts removal. On the other hand, the latter extracts the global features of the entire image that provides highly instrumental image quality improvement, especially effective on dealing with the global artifacts, such as blocking, color shifting. Extensive experimental results performed on color and grayscale images have clearly demonstrated the effectiveness and efficacy of our proposed single-model approach on the removal of compression artifacts from the decoded image.
Universal effectiveness of high-depth circuits in variational eigenproblems

We explore the effectiveness of variational quantum circuits in simulating the ground states of quantum many-body Hamiltonians. We show that generic high-depth circuits, performing a sequence of layer unitaries of the same form, can accurately approximate the desired states. We demonstrate their universal success by using two Hamiltonian systems with very different properties: the transverse field Ising model and the Sachdev-Ye-Kitaev model. The energy landscape of the high-depth circuits has a proper structure for the gradient-based optimization, i.e. the presence of local extrema -- near any random initial points -- reaching the ground level energy. We further test the circuit's capability of replicating random quantum states by minimizing the Euclidean distance.
Measurement of Semantic Textual Similarity in Clinical Texts: Comparison of Transformer-Based Models

Semantic textual similarity (STS) is one of the fundamental tasks in natural language processing (NLP). Many shared tasks and corpora for STS have been organized and curated in the general English domain; however, such resources are limited in the biomedical domain. In 2019, the National NLP Clinical Challenges (n2c2) challenge developed a comprehensive clinical STS dataset and organized a community effort to solicit state-of-the-art solutions for clinical STS.This study presents our transformer-based clinical STS models developed during this challenge as well as new models we explored after the challenge. This project is part of the 2019 n2c2/Open Health NLP shared task on clinical STS.In this study, we explored 3 transformer-based models for clinical STS: Bidirectional Encoder Representations from Transformers (BERT), XLNet, and Robustly optimized BERT approach (RoBERTa). We examined transformer models pretrained using both general English text and clinical text. We also explored using a general English STS dataset as a supplementary corpus in addition to the clinical training set developed in this challenge. Furthermore, we investigated various ensemble methods to combine different transformer models.Our best submission based on the XLNet model achieved the third-best performance (Pearson correlation of 0.8864) in this challenge. After the challenge, we further explored other transformer models and improved the performance to 0.9065 using a RoBERTa model, which outperformed the best-performing system developed in this challenge (Pearson correlation of 0.9010).This study demonstrated the efficiency of utilizing transformer-based models to measure semantic similarity for clinical text. Our models can be applied to clinical applications such as clinical text deduplication and summarization.
Dialogue Response Ranking Training with Large-Scale Human Feedback Data

Existing open-domain dialog models are generally trained to minimize the perplexity of target human responses. However, some human replies are more engaging than others, spawning more followup interactions. Current conversational models are increasingly capable of producing turns that are context-relevant, but in order to produce compelling agents, these models need to be able to predict and optimize for turns that are genuinely engaging. We leverage social media feedback data (number of replies and upvotes) to build a large-scale training dataset for feedback prediction. To alleviate possible distortion between the feedback and engagingness, we convert the ranking problem to a comparison of response pairs which involve few confounding factors. We trained DialogRPT, a set of GPT-2 based models on 133M pairs of human feedback data and the resulting ranker outperformed several baselines. Particularly, our ranker outperforms the conventional dialog perplexity baseline with a large margin on predicting Reddit feedback. We finally combine the feedback prediction models and a human-like scoring model to rank the machine-generated dialog responses. Crowd-sourced human evaluation shows that our ranking method correlates better with real human preferences than baseline models.
Modal dependent type theory and dependent right adjoints

Abstract In recent years, we have seen several new models of dependent type theory extended with some form of modal necessity operator, including nominal type theory, guarded and clocked type theory and spatial and cohesive type theory. In this paper, we study modal dependent type theory : dependent type theory with an operator satisfying (a dependent version of) the K axiom of modal logic. We investigate both semantics and syntax. For the semantics, we introduce categories with families with a dependent right adjoint (CwDRA) and show that the examples above can be presented as such. Indeed, we show that any category with finite limits and an adjunction of endofunctors give rise to a CwDRA via the local universe construction. For the syntax, we introduce a dependently typed extension of Fitch-style modal λ -calculus, show that it can be interpreted in any CwDRA, and build a term model. We extend the syntax and semantics with universes.
Sugeno integral generalization applied to improve adaptive image binarization

Classic adaptive binarization methodologies threshold pixels intensity with respect to adjacent pixels exploiting integral images. In turn, integral images are generally computed optimally by using the summed-area-table algorithm (SAT). This document presents a new adaptive binarization technique based on fuzzy integral images. Which, in turn, this technique is supported by an efficient design of a modified SAT for generalized Sugeno fuzzy integrals. We define this methodology as FLAT (Fuzzy Local Adaptive Thresholding). Experimental results show that the proposed methodology produced a better image quality thresholding than well-known global and local thresholding algorithms. We proposed new generalizations of different fuzzy integrals to improve existing results and reaching an accuracy ≈0.94 on a wide dataset. Moreover, due to high performances, these new generalized Sugeno fuzzy integrals created ad hoc for adaptive binarization, can be used as tools for grayscale processing and more complex real-time thresholding applications.
GLEE: Geometric Laplacian Eigenmap Embedding

Graph embedding seeks to build a low-dimensional representation of a graph G. This low-dimensional representation is then used for various downstream tasks. One popular approach is Laplacian Eigenmaps, which constructs a graph embedding based on the spectral properties of the Laplacian matrix of G. The intuition behind it, and many other embedding techniques, is that the embedding of a graph must respect node similarity: similar nodes must have embeddings that are close to one another. Here, we dispose of this distance-minimization assumption. Instead, we use the Laplacian matrix to find an embedding with geometric properties instead of spectral ones, by leveraging the so-called simplex geometry of G. We introduce a new approach, Geometric Laplacian Eigenmap Embedding (or GLEE for short), and demonstrate that it outperforms various other techniques (including Laplacian Eigenmaps) in the tasks of graph reconstruction and link prediction.
Interdisciplinary Research in Artificial Intelligence: Challenges and Opportunities

The use of artificial intelligence (AI) in a variety of research fields is speeding up multiple digital revolutions, from shifting paradigms in healthcare, precision medicine and wearable sensing, to public services and education offered to the masses around the world, to future cities made optimally efficient by autonomous driving. When a revolution happens, the consequences are not obvious straight away, and to date, there is no uniformly adapted framework to guide AI research to ensure a sustainable societal transition. To answer this need, here we analyze three key challenges to interdisciplinary AI research, and deliver three broad conclusions: 1) future development of AI should not only impact other scientific domains but should also take inspiration and benefit from other fields of science, 2) AI research must be accompanied by decision explainability, dataset bias transparency as well as development of evaluation methodologies and creation of regulatory agencies to ensure responsibility, and 3) AI education should receive more attention, efforts and innovation from the educational and scientific communities. Our analysis is of interest not only to AI practitioners but also to other researchers and the general public as it offers ways to guide the emerging collaborations and interactions toward the most fruitful outcomes.