Reviewerly/exHarmony huggingface dataset:
E.T.: Entity-Transformers. Coreference augmented Neural Language Model for richer mention representations via Entity-Transformer blocks

In the last decade, the field of Neural Language Modelling has witnessed enormous changes, with the development of novel models through the use of Transformer architectures. However, even these models struggle to model long sequences due to memory constraints and increasing computational complexity. Coreference annotations over the training data can provide context far beyond the modelling limitations of such language models. In this paper we present an extension over the Transformer-block architecture used in neural language models, specifically in GPT2, in order to incorporate entity annotations during training. Our model, GPT2E, extends the Transformer layers architecture of GPT2 to Entity-Transformers, an architecture designed to handle coreference information when present. To that end, we achieve richer representations for entity mentions, with insignificant training cost. We show the comparative model performance between GPT2 and GPT2E in terms of Perplexity on the CoNLL 2012 and LAMBADA datasets as well as the key differences in the entity representations and their effects in downstream tasks such as Named Entity Recognition. Furthermore, our approach can be adopted by the majority of Transformer-based language models.
Team Alexa at NADI Shared Task

In this paper, we discuss our team’s work on the NADI Shared Task. The task requires classifying Arabic tweets among 21 dialects. We tested out different approaches, and the best one was the simplest one. Our best submission was using Multinational Naive Bayes (MNB) classifier (Small and Hsiao, 1985) with n-grams as features. Despite its simplicity, this classifier shows better results than complicated models such as BERT. Our best submitted score was 17% F1-score and 35% accuracy.
False Positives Using Social Cognitive Mapping to Identify Children’s Peer Groups

Children and adolescents interact in peer groups, which are known to influence a range of psychological and behavioral outcomes. In developmental psychology and related disciplines, social cognitive mapping (SCM), as implemented with the SCM 4.0 software, is the most commonly used method for identifying peer groups from peer report data. However, in a series of four studies, we demonstrate that SCM has an unacceptably high risk of false positives. Specifically, we show that SCM will identify peer groups even when applied to random data. We introduce backbone extraction and community detection as one promising alternative to SCM, and offer several recommendations for researchers seeking to identify peer groups from peer report data.
Design and Implementation of LED Solar Simulator

In this paper, a complete LED-based solar simulator prototype is designed and implemented. The proposed light source of this solar simulator is formed using only six different types of LEDs to simulate the AM 1.5G spectrum. In addition, the flyback converter is designed to provide the required power of the solar simulator. In order to monitor and adjust the temperature at Standard Test Conditions (STC) of 25 °C, a proposed PID controller is applied and implemented using Arduino Mega 2560. The LED-based solar simulator prototype implementation has been achieved. The performance of the solar simulator prototype has been verified and tested.
Addressing Research Fatigue in Energy Communities: New Tools to Prepare Researchers for Better Community Engagement

Two innovative educational products aimed at improving the quality of human-subjects research in and around energy-impacted communities are introduced here. The educational materials include: (1) a self-paced online education course titled “Understanding and Addressing Research Fatigue in Rural Communities” and (2) an accompanying workbook for research planning titled “Engaging in Energy Communities: The Role of the Researcher.” We explain how two recent studies were instrumental in identifying the need for the online education course and the associated workbook. We conclude the paper with guidance on how to access these new resources, which we believe will be particularly useful for graduate students and early-career researchers. We invite feedback from users as well as from colleagues who may wish to use these educational materials.
Faheem at NADI shared task: Identifying the dialect of Arabic tweet

This paper describes Faheem (adj. of understand), our submission to NADI (Nuanced Arabic Dialect Identification) shared task. With so many Arabic dialects being under-studied due to the scarcity of the resources, the objective is to identify the Arabic dialect used in the tweet, country wise. We propose a machine learning approach where we utilize word-level n-gram (n = 1 to 3) and tf-idf features and feed them to six different classifiers. We train the system using a data set of 21,000 tweets—provided by the organizers—covering twenty-one Arab countries. Our top performing classifiers are: Logistic Regression, Support Vector Machines, and Multinomial Na ive Bayes.
Large Scale Author Obfuscation Using Siamese Variational Auto-Encoder: The SiamAO System.

Author obfuscation is the task of masking the author of a piece of text, with applications in privacy. Recent advances in deep neural networks have boosted author identification performance making author obfuscation more challenging. Existing approaches to author obfuscation are largely heuristic. Obfuscation can, however, be thought of as the construction of adversarial examples to attack author identification, suggesting that the deep learning architectures used for adversarial attacks could have application here. Current architectures are proposed to construct adversarial examples against classification-based models, which in author identification would exclude the high-performing similarity-based models employed when facing large number of authorial classes. In this paper, we propose the first deep learning architecture for constructing adversarial examples against similarity-based learners, and explore its application to author obfuscation. We analyse the output from both success in obfuscation and language acceptability, as well as comparing the performance with some common baselines, and showing promising results in finding a balance between safety and soundness of the perturbed texts.
User Models, Metrics and Measures of Search

Evaluation is central to Information Retrieval, and is how we compare the quality of systems. One important principle of evaluation is that the measured score should reflect the user's experience with the system. Hence, there should be direct connection between how users interact with the system and the characteristics of the metric. In this tutorial we introduce the C/W/L approach to user modeling and show how different user models lead to different metrics. We then describe the recent innovations and approaches to evaluation that it has facilitated. The tutorial is presented as a mix of on-line synchronous lecture, pre-recorded in-depth videos, and hands-on activities using the C/W/L toolkit for participants' own evaluation tasks. A followup consultation session is also provided, to allow extended questions and individual discussion with the four presenters.
Reductions for Strings and Regular Expressions Revisited

The theory of strings supported by solvers in formal methods contains a large number of operators. Instead of implementing a semi-decision procedure that reasons about all the operators directly, string solvers often reduce operators to a core fragment and implement a semi-decision procedure over that fragment. These reductions considerably increase the number of constraints and thus have to be done carefully to achieve good performance. We propose novel reductions from regular expressions to string constraints and a framework for minimizing the introduction of new variables in current reductions of string constraints. The reductions of regular expression constraints enable string solvers to handle a significant fragment of such constraints without using dedicated reasoning over regular expressions. Minimizing the number of variables in the reduced constraints makes those constraints significantly cheaper to solve by the core solver. An experimental evaluation of our implementation of both techniques in cvc4, a state-of-the-art SMT solver with extensive support for the theory of strings, shows that they significantly improve the solver's performance.
Blockchain Technology for Mobile Applications Recommendation Systems

The interest in the blockchain technology has been increasing since its inception and it has been applied to many fields and sectors. The blockchain technology creates a decentralized environment where no third party controls the data and transaction. Mobile apps recommendation has been extensively used to recommend apps to mobile users. For example, Android-based recommendation applications have been developed to recommend other mobile apps for download depending on user’s preferences and mobile context. These recommendations help users discover apps by referring to the experiences of other users. Due to the collection of a large amount of data and user information, there is a problem of insecurity and user’s privacy that are prone to be attacked. To address this issue the blockchain technology can be incorporated to assure cryptographic safety. In this paper, we present a survey of the on-going mobile app recommendations and e-commerce technology trend to address how the blockchain can be incorporated into the collaborative filtering recommendation systems to enable the users to set up a secured data, which implies the importance of user privacy preference on personalized app recommendations.
NNGeometry: Easy and Fast Fisher Information Matrices and Neural Tangent Kernels in PyTorch

Fisher Information Matrices (FIM) and Neural Tangent Kernels (NTK) are useful tools in a number of diverse applications related to neural networks. Yet these theoretical tools are often difficult to implement using current libraries for practical size networks, given that they require per-example gradients, and a large amount of memory since they scale as the number of parameters (for the FIM) or the number of examples x cardinality of the output space (for the NTK). NNGeometry is a PyTorch library that offers a simple interface for computing various linear algebra operations such as matrix-vector products, trace, frobenius norm, and so on, where the matrix is either the FIM or the NTK, leveraging recent advances in approximating these matrices. We here present the library and motivate our design choices, then we demonstrate it on actual deep neural networks.
Application of Fuzzy Logic for Developing Sense and Avoid Techniques for UAV Flight Operations in National Airspace

Abstract In this work, a Fuzzy Inference System (FIS) is proposed to develop a sense and avoid technique for conflict-free Unmanned Aerial Vehicle (UAV) flight operations in the national airspace. The proposed method is implemented alongside the flight control software guiding the UAV towards a predefined goal. The fuzzy system makes decisions for sense and avoid with respect to the state of other intruder aircraft in the airspace. The fuzzy rules are selected under consideration of the heading and position data of the intruder aircraft obtained using ADS-B sensor relative to the controlled UAV. The avoidance controller is implemented alongside a traditional PID-based flight controller to test the effectiveness of the sense-and-avoid technique. The PID controller drives the controlled UAV towards the goal and the fuzzy controller is for conflict avoidance based on an event trigger logic if the controlled UAV is in the intrusion zone of another aircraft. The integration of the dynamic model of the controlled UAV and fuzzy inference system is described. The numerical simulations are shown to evaluate the performance of the proposed method in the presence of moving as well as stationary intruder aircraft.
Industrial Control Malicious Traffic Anomaly Detection System Based on Deep Autoencoder

Industrial control network is a direct interface between information system and physical control process. Due to the lack of authentication, encryption, and other necessary security protection designs, it has become the main target of malicious attacks under the trend of increasing openness. In order to protect the industrial control systems, we examine the detection of abnormal traffic in industrial control network and propose a method of detecting abnormal traffic in industrial control network based on autoencoder technology. What is more, a new deep autoencoder model was designed to reduce the dimensionality of traffic data in industrial control network. In this article, the Kullback–Leibler divergence was added to the loss function to improve the ability of feature extraction and the ability to recover raw data. Finally, this model was compared with the traditional data dimensionality reduction method (principal component analysis (PCA), independent component analysis, and singular value decomposition) on gas pipeline dataset. The results show that the approach designed in this article outperforms the three methods in different scenes in terms of f 1 score.
FX trade execution: complex and highly fragmented

The 2019 Triennial Survey shows that trade execution has undergone rapid change, with more diverse participants, new technologies and increasing complexity. Electronification advanced the fastest in dealer-to-customer trading. Dealers and customers navigated a highly fragmented market by leveraging technology to trade across electronic venues. Aspects of FX intermediation tilted more towards non-banks as new market-makers, albeit brokered by top dealers. Bank dealers continued to attract large flows to their own proprietary liquidity pools. Consequently, even though the market grew bigger as a whole, the share of trading activity 'visible' to the broader market declined.
"Cyborg Justice" and the Risk of Technological-Legal Lock-In

Although Artificial Intelligence (AI) is already of use to litigants and legal practitioners, we must be cautious and deliberate in incorporating AI into the common law judicial process. Human beings and machine systems process information and reach conclusions in fundamentally different ways, with AI being particularly ill-suited for the rule application and value balancing required of human judges. Nor will “cyborg justice”—hybrid human/AI judicial systems that attempt to marry the best of human and machine decisionmaking and minimize the drawbacks of both—be a panacea. While such systems would ideally maximize the strengths of human and machine intelligence, they might also magnify the drawbacks of both. They also raise distinct teaming risks associated with overtrust, undertrust, and interface design errors, as well as second-order structural side effects.One such side effect is “technological–legal lock-in.” Translating rules and decisionmaking procedures into algorithms grants them a new kind of permanency, which creates an additional barrier to legal evolution. In augmenting the common law’s extant conservative bent, hybrid human/AI judicial systems risk fostering legal stagnation and an attendant loss of judicial legitimacy.
Towards a Reliable and Robust Dialogue System for Medical Automatic Diagnosis

Dialogue system for medical automatic diagnosis (DSMAD) aims to learn an agent that mimics the behavior of a human doctor, i.e. inquiring symptoms and informing diseases. Since DSMAD has been formulated as a Markov decision-making process, many studies apply reinforcement learning methods to solve it. Unfortunately, existing works solely rely on simple diagnostic accuracy to justify the effectiveness of their DSMAD agents while ignoring the medical rationality of the inquiring process. From the perspective of medical application, it's critical to develop an agent that is able to produce reliable and convincing diagnosing processes and also is robust in making diagnosis facing noisy interaction with patients. To this end, we propose a novel DSMAD agent, INS-DS (Introspective Diagnosis System) comprising of two separate yet cooperative modules, i.e., an inquiry module for proposing symptom-inquiries and an introspective module for deciding when to inform a disease. INS-DS is inspired by the introspective decision-making process of human, where the inquiry module first proposes the most valuable symptom inquiry, and then the introspective module intervenes the potential responses of this inquiry and decides to inquire only if the diagnoses of these interventions vary.
Algorithmic Fair Use

Author(s): Burk, DL | Abstract: Legal governance and regulation are becoming increasingly reliant on data collection and algorithmic data processing. In the area of copyright, online protection of digitized works is frequently mediated by algorithmic enforcement systems intended to purge illicit content and limit the liability of YouTube, Facebook, and other content platforms. But unauthorized content is not necessarily illicit content. Many unauthorized digital postings may claim legitimacy under statutory exceptions like the legal balancing standard known as fair use. Such exceptions exist to ameliorate the negative effects of copyright on public discourse, personal enrichment, and artistic creativity. Consequently, it may seem desirable to incorporate fair use metrics into copyright policing algorithms, both to protect against automated overdeterrence and to inform users of their compliance with copyright law. In this Essay, I examine the prospects for algorithmic mediation of copyright exceptions, warning that the design values embedded in algorithms will inevitably become embedded in public behavior and consciousness. Thus, algorithmic fair use carries with it the very real possibility of habituating new media participants to its own biases and so progressively altering the fair use standard it attempts to embody.
Augmented obstacle avoidance controller design for mobile robots

For unicycle robots and an arbitrary continuous reference tracking controller, we propose a mildly invasive obstacle avoidance augmentation only activated in an eye-shaped neighborhood around an obstacle and providing avoidance guarantees. Switching between different control phases is orchestrated by a hybrid automaton with hysteresis mechanisms that prevent Zeno behavior. Since the size of the eye-shaped neighborhood and its orientation depend continuously on the orientation and velocity of the robot, the velocity control input is continuous. Numerical simulations illustrate the performance of an avoidance-augmented tracking controller.
Toward a Semisupervised Machine Learning Application to Seismic Facies Classification

Summary Seismic facies classification is a machine learning task that maps seismic attributes to classes. The training data for these problems are commonly governed by wells, which are characteristically sparse. Most seismic facies classification problems utilize supervised machine learning algorithms, but supervised algorithms are prone to overfitting in the presence of minimal training data. However, semisupervised algorithms are designed for problems with small training sets because they incorporate both the labelled and the unlabelled data during training. Semisupervised algorithms are largely unexplored in geoscience applications, and we explore their potential here on a study using the 2D SEAM model. We provide a workflow for performing seismic classification of this synthetic model that consists of four stages. The earlier stages synthesize the seismic data from the model and build the classes for the labelled data using unsupervised learning. A latter stage involves estimating a ground-truth facies model using machine learning where the training data amount to roughly 0.2% of the full dataset. We show that our semisupervised algorithm can capture more detail compared to a popular supervised algorithm, XGBoost, and this supports the hypothesis that semisupervised algorithms can recover better predictions than supervised methods in the context of minimal training data.
A Comparative Study of Gamma Markov Chains for Temporal Non-Negative Matrix Factorization

Non-negative matrix factorization (NMF) has become a well-established class of methods for the analysis of non-negative data. In particular, a lot of effort has been devoted to probabilistic NMF, namely estimation or inference tasks in probabilistic models describing the data, based for example on Poisson or exponential likelihoods. When dealing with time series data, several works have proposed to model the evolution of the activation coefficients as a non-negative Markov chain, most of the time in relation with the Gamma distribution, giving rise to so-called temporal NMF models. In this paper, we review four Gamma Markov chains of the NMF literature, and show that they all share the same drawback: the absence of a well-defined stationary distribution. We then introduce a fifth process, an overlooked model of the time series literature named BGAR(1), which overcomes this limitation. These temporal NMF models are then compared in a MAP framework on a prediction task, in the context of the Poisson likelihood.
Regularization Matters in Policy Optimization - An Empirical Study on Continuous Control

Deep Reinforcement Learning (Deep RL) has been receiving increasingly more attention thanks to its encouraging performance on a variety of control tasks. Yet, conventional regularization techniques in training neural networks (e.g., L2 regularization, dropout) have been largely ignored in RL methods, possibly because agents are typically trained and evaluated in the same environment, and because the deep RL community focuses more on high-level algorithm designs. In this work, we present the first comprehensive study of regularization techniques with multiple policy optimization algorithms on continuous control tasks. Interestingly, we find conventional regularization techniques on the policy networks can often bring large improvement, especially on harder tasks. Our findings are shown to be robust against training hyperparameter variations. We also compare these techniques with the more widely used entropy regularization. In addition, we study regularizing different components and find that only regularizing the policy network is typically the best. We further analyze why regularization may help generalization in RL from four perspectives - sample complexity, reward distribution, weight norm, and noise robustness. We hope our study provides guidance for future practices in regularizing policy optimization algorithms. Our code is available at https://github.com/xuanlinli17/iclr2021_rlreg .
Trade Finance Using Blockchain

A canopy term, Trade Finance, covers the importing and exporting of goods, performing transactions through banks and maintaining a healthy importer-exporter relationship. Trade financing traditionally includes massive involvement of paper-based methodology which made the work tedious and prone to tampering. An introduction of blockchain technology can enhance the current trade financing system. Blockchain, with its features, can enhance the transparency between the transactions happening and encrypt each and every step of the process. This system is an interesting blend of blockchain and trade finance which will yield a secure system and innovate the traditional system to a modernized finance model. Further, the paper explains the concept of Hyperledger Fabric - a blockchain framework which helps in developing modular architectures by providing appropriate solutions for existing trade finance system.