Reviewerly/exHarmony huggingface dataset:
Contrastive Adversarial Learning for Person Independent Facial Emotion Recognition

Since most facial emotion recognition (FER) methods significantly rely on supervision information, they have a limit to analyzing emotions independently of persons. On the other hand, adversarial learning is a well-known approach for generalized representation learning because it never requires supervision information. This paper presents a new adversarial learning for FER. In detail, the proposed learning enables the FER network to better understand complex emotional elements inherent in strong emotions by adversarially learning weak emotion samples based on strong emotion samples. As a result, the proposed method can recognize the emotions independently of persons because it understands facial expressions more accurately. In addition, we propose a contrastive loss function for efficient adversarial learning. Finally, the proposed adversarial learning scheme was theoretically verified, and it was experimentally proven to show state of the art (SOTA) performance.
Invariant Theory and Scaling Algorithms for Maximum Likelihood Estimation

We uncover connections between maximum likelihood estimation in statistics and norm minimization over a group orbit in invariant theory. We focus on Gaussian transformation families, which include matrix normal models and Gaussian graphical models given by transitive directed acyclic graphs. We use stability under group actions to characterize boundedness of the likelihood, and existence and uniqueness of the maximum likelihood estimate. Our approach reveals promising consequences of the interplay between invariant theory and statistics. In particular, existing scaling algorithms from statistics can be used in invariant theory, and vice versa.
Enhancing Entity Boundary Detection for Better Chinese Named Entity Recognition

In comparison with English, due to the lack of explicit word boundary and tenses information, Chinese Named Entity Recognition (NER) is much more challenging.In this paper, we propose a boundary enhanced approach for better Chinese NER.In particular, our approach enhances the boundary information from two perspectives.On one hand, we enhance the representation of the internal dependency of phrases by an additional Graph Attention Network(GAT) layer.On the other hand, taking the entity head-tail prediction (i.e., boundaries) as an auxiliary task, we propose an unified framework to learn the boundary information and recognize the NE jointly.Experiments on both the OntoNotes and the Weibo corpora show the effectiveness of our approach.
DeepSynth: Automata Synthesis for Automatic Task Segmentation in Deep Reinforcement Learning

This paper proposes DeepSynth, a method for effective training of deep Reinforcement Learning (RL) agents when the reward is sparse and non-Markovian, but at the same time progress towards the reward requires achieving an unknown sequence of high-level objectives. Our method employs a novel algorithm for synthesis of compact automata to uncover this sequential structure automatically. We synthesise a human-interpretable automaton from trace data collected by exploring the environment. The state space of the environment is then enriched with the synthesised automaton so that the generation of a control policy by deep RL is guided by the discovered structure encoded in the automaton. The proposed approach is able to cope with both high-dimensional, low-level features and unknown sparse non-Markovian rewards. We have evaluated DeepSynth's performance in a set of experiments that includes the Atari game Montezuma's Revenge. Compared to existing approaches, we obtain a reduction of two orders of magnitude in the number of iterations required for policy synthesis, and also a significant improvement in scalability.
A Multi-Task Approach for Improving Biomedical Named Entity Recognition by Incorporating Multi-Granularity information

Neural biomedical named entity recognition (BioNER) methods usually require a large amount of annotated data, while the annotated BioNER datasets are often difficult to obtain and small in scale due to the limitations of privacy, ethics and high degree of specialization.To alleviate the lack of training samples, unlike conventional methods that only use token-level information, this paper proposes a method that simultaneously utilize the latent multi-granularity information in the dataset.Concretely, the proposed model is based on a multi-task approach, which leverages different training objectives by introducing auxiliary tasks, i.e. binary classification, multi-class and multi-token classification.Experimental results over three BioNER datasets show that the proposed model produces better performance over the BioBERT baseline and can get more than 3% improvements of F1score in low-resource scenarios.Finally, we released our code at https://github.com/zgzjdx/MT-BioNER.
Do Response Selection Models Really Know What’s Next? Utterance Manipulation Strategies for Multi-turn Response Selection

In this paper, we study the task of selecting the optimal response given a user and system utterance history in retrieval-based multi-turn dialog systems. Recently, pre-trained language models (e.g., BERT, RoBERTa, and ELECTRA) showed significant improvements in various natural language processing tasks. This and similar response selection tasks can also be solved using such language models by formulating the tasks as dialog--response binary classification tasks. Although existing works using this approach successfully obtained state-of-the-art results, we observe that language models trained in this manner tend to make predictions based on the relatedness of history and candidates, ignoring the sequential nature of multi-turn dialog systems. This suggests that the response selection task alone is insufficient for learning temporal dependencies between utterances. To this end, we propose utterance manipulation strategies (UMS) to address this problem. Specifically, UMS consist of several strategies (i.e., insertion, deletion, and search), which aid the response selection model towards maintaining dialog coherence. Further, UMS are self-supervised methods that do not require additional annotation and thus can be easily incorporated into existing approaches. Extensive evaluation across multiple languages and models shows that UMS are highly effective in teaching dialog consistency, which leads to models pushing the state-of-the-art with significant margins on multiple public benchmark datasets.
Long-Span Summarization via Local Attention and Content Selection

Transformer-based models have achieved state-of-the-art results in a wide range of natural language processing (NLP) tasks including document summarization.Typically these systems are trained by fine-tuning a large pretrained model to the target task.One issue with these transformer-based models is that they do not scale well in terms of memory and compute requirements as the input length grows.Thus, for long document summarization, it can be challenging to train or fine-tune these models.In this work, we exploit large pre-trained transformer-based models and address long-span dependencies in abstractive summarization using two methods: local self-attention; and explicit content selection.These approaches are compared on a range of network configurations.Experiments are carried out on standard long-span summarization tasks, including Spotify Podcast, arXiv, and PubMed datasets.We demonstrate that by combining these methods, we can achieve state-of-the-art results on all three tasks in the ROUGE scores.Moreover, without a large-scale GPU card, our approach can achieve comparable or better results than existing approaches. 1
Image Change Captioning by Learning from an Auxiliary Task

We tackle the challenging task of image change captioning. The goal is to describe the subtle difference between two very similar images by generating a sentence caption. While the recent methods mainly focus on proposing new model architectures for this problem, we instead focus on an alternative training scheme. Inspired by the success of multi-task learning, we formulate a training scheme that uses an auxiliary task to improve the training of the change captioning network. We argue that the task of composed query image retrieval is a natural choice as the auxiliary task. Given two almost similar images as the input, the primary network generates a caption describing the fine change between those two images. Next, the auxiliary network is provided with the generated caption and one of those two images. It then tries to pick the second image among a set of candidates. This forces the primary network to generate detailed and precise captions via having an extra supervision loss by the auxiliary network. Furthermore, we propose a new scheme for selecting a negative set of candidates for the retrieval task that can effectively improve the performance. We show that the proposed training strategy performs well on the task of change captioning on benchmark datasets.
Lifelong and Continual Learning Dialogue Systems: Learning during Conversation

Dialogue systems, also called chatbots, are now used in a wide range of applications. However, they still have some major weaknesses. One key weakness is that they are typically trained from manually-labeled data and/or written with handcrafted rules, and their knowledge bases (KBs) are also compiled by human experts. Due to the huge amount of manual effort involved, they are difficult to scale and also tend to produce many errors ought to their limited ability to understand natural language and the limited knowledge in their KBs. Thus, the level of user satisfactory is often low. In this paper, we propose to dramatically improve the situation by endowing the chatbots the ability to continually learn (1) new world knowledge, (2) new language expressions to ground them to actions, and (3) new conversational skills, during conversation by themselves so that as they chat more and more with users, they become more and more knowledgeable and are better and better able to understand diverse natural language expressions and to improve their conversational skills.
Structuring clinical text with AI: Old versus new natural language processing techniques evaluated on eight common cardiovascular diseases

Free-text clinical notes in electronic health records are more difficult for data mining while the structured diagnostic codes can be missing or erroneous. To improve the quality of diagnostic codes, this work extracts diagnostic codes from free-text notes: five old and new word vectorization methods were used to vectorize Stanford progress notes and predict eight ICD-10 codes of common cardiovascular diseases with logistic regression. The models showed good performance, with TF-IDF as the best vectorization model showing the highest AUROC (0.9499-0.9915) and AUPRC (0.2956-0.8072). The models also showed transferability when tested on MIMIC-III data with AUROC from 0.7952 to 0.9790 and AUPRC from 0.2353 to 0.8084. Model interpretability was shown by the important words with clinical meanings matching each disease. This study shows the feasibility of accurately extracting structured diagnostic codes, imputing missing codes, and correcting erroneous codes from free-text clinical notes for information retrieval and downstream machine-learning applications.
xMoCo: Cross Momentum Contrastive Learning for Open-Domain Question Answering

Dense passage retrieval has been shown to be an effective approach for information retrieval tasks such as open domain question answering.Under this paradigm, a dual-encoder model is learned to encode questions and passages separately into vector representations, and all the passage vectors are then pre-computed and indexed, which can be efficiently retrieved by vector space search during inference time.In this paper, we propose a new contrastive learning method called cross momentum contrastive learning (xMoCo), for learning a dualencoder model for query-passage matching.Our method efficiently maintains a large pool of negative samples like the original MoCo, and by jointly optimizing question-to-passage and passage-to-question matching, enables using separate encoders for questions and passages.We evaluate our method on various open domain QA datasets, and the experimental results show the effectiveness of the proposed approach.
Graph integration of structured, semistructured and unstructured data for data journalism

Digital data is a gold mine for modern journalism. However, datasets which interest journalists are extremely heterogeneous, ranging from highly structured (relational databases), semi-structured (JSON, XML, HTML), graphs (e.g., RDF), and text. Journalists (and other classes of users lacking advanced IT expertise, such as most non-governmental-organizations, or small public administrations) need to be able to make sense of such heterogeneous corpora, even if they lack the ability to define and deploy custom extract-transform-load workflows, especially for dynamically varying sets of data sources. We describe a complete approach for integrating dynamic sets of heterogeneous datasets along the lines described above: the challenges we faced to make such graphs useful, allow their integration to scale, and the solutions we proposed for these problems. Our approach is implemented within the ConnectionLens system; we validate it through a set of experiments.
A Hybrid Approach for Autonomous Collision-Free UAV Navigation in 3D Partially Unknown Dynamic Environments

In the past decades, unmanned aerial vehicles (UAVs) have emerged in a wide range of applications. Owing to the advances in UAV technologies related to sensing, computing, power, etc., it has become possible to carry out missions autonomously. A key component to achieving this goal is the development of safe navigation methods, which is the main focus of this work. A hybrid navigation approach is proposed to allow safe autonomous operations in three-dimensional (3D) partially unknown and dynamic environments. This method combines a global path planning algorithm, namely RRT-Connect, with a reactive control law based on sliding mode control to provide quick reflex-like reactions to newly detected obstacles. The performance of the suggested approach is validated using simulations.
Cost-Based Heterogeneous Learning Framework for Real-Time Spam Detection in Social Networks With Expert Decisions

With the widespread use of social networks, spam messages against them have become a major issue. Spam detection methods can be broadly divided into expert-based and machine learning-based detection methods. When experts participate in spam detection, the detection accuracy is fairly high. However, this method is highly time-consuming and expensive. Conversely, methods using machine learning have the advantage of automation, but their accuracy is relatively low. This paper proposes a spam-detection framework that combines and fully exploits the advantages of both methods. To reduce the workload of the experts, all messages are first analyzed via a primary machine learning filter, and those that are determined to be normal messages are allowed through, whereas suspicious messages are flagged. The flagged messages are subsequently analyzed by an expert to enhance the overall system accuracy. In the filtering process, cost-based machine learning is used to prevent the fatal error of misidentifying a spam message as a normal message. In addition, to obviate the continuously evolving spam trends, a module that periodically updates the expert-diagnosis results on the training dataset is incorporated into the framework. The results of experiments conducted, on an imbalanced dataset of spam tweets and normal tweets in a ratio similar to the actual situation in real life, indicate that the proposed framework has a spam-detection rate of almost 92.8%, which is higher than that of the conventional machine learning technique. Furthermore, the proposed framework delivered stable high performance even in an environment where social network messages changed continuously, unlike the conventional technique, which exhibited large performance deviations.
A roadmap toward the automatic composition of systematic literature reviews

Objective. This paper presents an overview of existing artificial intelligence tools to produce systematic literature reviews. Furthermore, we propose a general framework resulting from combining these techniques to highlight the challenges and possibilities currently existing in this research area.&#x0D; Design/Methodology/Approach. We undertook a scoping review on the systematic literature review steps to automate them via computational techniques.&#x0D; Results/Discussion. The process of creating a literature review is both creative and technical. The technical part of this process is liable to automation. Based on the literature, we chose to divide this technical part into four steps: searching, screening, extraction, and synthesis. For each one of these steps, we presented practical artificial intelligence techniques to carry them out. In addition, we presented the obstacles encountered in the application of each technique.&#x0D; Conclusion. We proposed a framework for automatically creating systematic literature reviews by combining and placing existing techniques in stages where they possess the greatest potential to be useful. Despite still lacking practical assessment in different areas of knowledge, this proposal indicates ways with the potential to reduce the time-consuming and repetitive work embedded in the systematic literature review process.&#x0D; Originality/Value. The paper presents the current possibilities for automating systematic literature reviews and how they can work together to reduce researchers’ operational workload.
Developing a BIM-Based MUVR Treadmill System for Architectural Design Review and Collaboration

Virtual reality (VR) is quickly becoming the medium of choice for various architecture, engineering, and construction applications, such as design visualization, construction planning, and safety training. In particular, this technology offers an immersive experience to enhance the way architects review their design with team members. Traditionally, VR has used a desktop PC or workstation setup inside a room, yielding the risk of two users bump into each other while using multiuser VR (MUVR) applications. MUVR offers shared experiences that disrupt the conventional single-user VR setup, where multiple users can communicate and interact in the same virtual space, providing more realistic scenarios for architects in the design stage. However, this shared virtual environment introduces challenges regarding limited human locomotion and interactions, due to physical constraints of normal room spaces. This study thus presented a system framework that integrates MUVR applications into omnidirectional treadmills. The treadmills allow users an immersive walking experience in the simulated environment, without space constraints or hurt potentialities. A prototype was set up and tested in several scenarios by practitioners and students. The validated MUVR treadmill system aims to promote high-level immersion in architectural design review and collaboration.
LSTM, VADER and TF-IDF based Hybrid Sentiment Analysis Model

Most sentiment analysis models that use supervised learning algorithms consume a lot of labeled data in the training phase in order to give satisfactory results.This is usually expensive and leads to high labor costs in real-world applications.This work consists in proposing a hybrid sentiment analysis model based on a Long Short-Term Memory network, a rulebased sentiment analysis lexicon and the Term Frequency-Inverse Document Frequency weighting method.These three (input) models are combined in a binary classification model.In the latter, each of these algorithms has been implemented: Logistic Regression, k-Nearest Neighbors, Random Forest, Support Vector Machine and Naive Bayes.Then, the model has been trained on a limited amount of data from the IMDB dataset.The results of the evaluation on the IMDB data show a significant improvement in the Accuracy and F1 score compared to the best scores recorded by the three input models separately.On the other hand, the proposed model was able to transfer the knowledge gained on the IMDB dataset to better handle a new data from Twitter US Airlines Sentiments dataset.
A Framework and Benchmarking Study for Counterfactual Generating Methods on Tabular Data

Counterfactual explanations are viewed as an effective way to explain machine learning predictions. This interest is reflected by a relatively young literature with already dozens of algorithms aiming to generate such explanations. These algorithms are focused on finding how features can be modified to change the output classification. However, this rather general objective can be achieved in different ways, which brings about the need for a methodology to test and benchmark these algorithms. The contributions of this work are manifold: First, a large benchmarking study of 10 algorithmic approaches on 22 tabular datasets is performed, using nine relevant evaluation metrics; second, the introduction of a novel, first of its kind, framework to test counterfactual generation algorithms; third, a set of objective metrics to evaluate and compare counterfactual results; and, finally, insight from the benchmarking results that indicate which approaches obtain the best performance on what type of dataset. This benchmarking study and framework can help practitioners in determining which technique and building blocks most suit their context, and can help researchers in the design and evaluation of current and future counterfactual generation algorithms. Our findings show that, overall, there’s no single best algorithm to generate counterfactual explanations as the performance highly depends on properties related to the dataset, model, score, and factual point specificities.
The relationship between asset and capital structure: a compositional approach with panel vector autoregressive models

&lt;abstract&gt; &lt;p&gt;The companies' investment and financing policies are dynamically interrelated and there is no general consensus about the direction of this relationship. There are theoretical arguments and empirical evidence supporting both possible directions, which makes panel vector autoregressive models an appropriate tool. However, the financial ratios normally used to assess this relationship empirically tend to be asymmetric, and to have extreme outliers and non-linear relationships. The aim of this article is to propose a methodological approach to address these issues by complementing panel vector autoregressive models with compositional data analysis. The usefulness of the proposed methodology is illustrated with real data of Spanish retail companies, while a reanalysis with standard financial ratios is inconclusive.&lt;/p&gt; &lt;/abstract&gt;