Reviewerly/exHarmony huggingface dataset:
Unsupervised Extractive Summarization-Based Representations for Accurate and Explainable Collaborative Filtering

We pioneer the first extractive summarizationbased collaborative filtering model called ES-COFILT.Our proposed model specifically produces extractive summaries for each item and user.Unlike other types of explanations, summary-level explanations closely resemble real-life explanations.The strength of ES-COFILT lies in the fact that it unifies representation and explanation.In other words, extractive summaries both represent and explain the items and users.Our model uniquely integrates BERT, K-Means embedding clustering, and multilayer perceptron to learn sentence embeddings, representation-explanations, and user-item interactions, respectively.We argue that our approach enhances both rating prediction accuracy and user/item explainability.Our experiments illustrate that ESCOFILT's prediction accuracy is better than the other state-of-the-art recommender models.Furthermore, we propose a comprehensive set of criteria that assesses the real-life explainability of explanations.Our explainability study demonstrates the superiority of and preference for summary-level explanations over other explanation types.
Time-Varying Nexus between Investor Sentiment and Cryptocurrency Market: New Insights from a Wavelet Coherence Framework

This study attempts to investigate the nexus between investor sentiment and cryptocurrencies prices. Our empirical investigation merges bivariate and multivariate wavelet tools to examine the investor sentiment nexus to inter-cryptocurrencies prices. The study outcomes show that the Sentix Investor Confidence index provides significant information in explaining long-term changes in Bitcoin and Litecoin prices. Moreover, the findings generated from the multiple wavelet coherence illustrate the simultaneous contribution of cryptocurrencies and the Sentix Investor Confidence index in explaining the Bitcoin index movement across frequencies and over horizons, especially during bubble burst periods. The study also suggests a time-dependent relationship of Bitcoin prices with alternative cryptocurrencies and the Sentix Investor Confidence index, mostly pronounced during the Bitcoin bubble. We discuss our results using GSV-based investor sentiment. Our findings remain robust and confirm the strong predictive power of investor sentiment in cryptocurrencies price movements over time and across scales.
Towards Feature Space Adversarial Attack by Style Perturbation

We propose a new adversarial attack to Deep Neural Networks for image classification. Different from most existing attacks that directly perturb input pixels, our attack focuses on perturbing abstract features, more specifically, features that denote styles, including interpretable styles such as vivid colors and sharp outlines, and uninterpretable ones. It induces model misclassification by injecting imperceptible style changes through an optimization procedure. We show that our attack can generate adversarial samples that are more natural-looking than the state-of-the-art unbounded attacks. The experiment also supports that existing pixel-space adversarial attack detection and defense techniques can hardly ensure robustness in the style-related feature space.
Distance learning practices on the Example of Second Language Learning during Coronavirus Epidemic in Russia

The CoronaVirus (COVID19) made the higher education establishment take on urgent measures when all the students were prohibited from entering the university not to catch the disease, but the educational process was not to be stopped.Distance education tools were to be applied and digital platforms of the universities were to be checked.This paper deals with the experience of the National Research South Ural University.The courses of all the schools and institutes of the University were conducted based on the platform Digital SUSU.The example of the courses for second language learning was discussed.Fifty students of bachelor's and master's degrees were respondents in this research.The students answered the questions of the questionnaire on Google Forms and then were interviewed.The results suggest that the most interesting task on the planform Digital SUSU was a video with tasks and vocabulary exercises.However, among the most useful, according to the students' opinion were grammar, vocabulary, and reading tasks.The students were also asked to discuss the advantages and disadvantages of such remote learning.The most frequently mentioned advantages were the ability to study at home and the absence of the necessity to look smart.However, the disadvantages were more varied, including absence of communication, no face to face contact, too much homework, and too many written tasks.
Explainable Prediction of Text Complexity: The Missing Preliminaries for Text Simplification

Text simplification reduces the language complexity of professional content for accessibility purposes.End-to-end neural network models have been widely adopted to directly generate the simplified version of input text, usually functioning as a blackbox.We show that text simplification can be decomposed into a compact pipeline of tasks to ensure the transparency and explainability of the process.The first two steps in this pipeline are often neglected: 1) to predict whether a given piece of text needs to be simplified, and 2) if yes, to identify complex parts of the text.The two tasks can be solved separately using either lexical or deep learning methods, or solved jointly.Simply applying explainable complexity prediction as a preliminary step, the out-ofsample text simplification performance of the state-of-the-art, black-box simplification models can be improved by a large margin.
Learning Graphons via Structured Gromov-Wasserstein Barycenters

We propose a novel and principled method to learn a nonparametric graph model called graphon, which is defined in an infinite-dimensional space and represents arbitrary-size graphs. Based on the weak regularity lemma from the theory of graphons, we leverage a step function to approximate a graphon. We show that the cut distance of graphons can be relaxed to the Gromov-Wasserstein distance of their step functions. Accordingly, given a set of graphs generated by an underlying graphon, we learn the corresponding step function as the Gromov-Wasserstein barycenter of the given graphs. Furthermore, we develop several enhancements and extensions of the basic algorithm, e.g., the smoothed Gromov-Wasserstein barycenter for guaranteeing the continuity of the learned graphons and the mixed Gromov-Wasserstein barycenters for learning multiple structured graphons. The proposed approach overcomes drawbacks of prior state-of-the-art methods, and outperforms them on both synthetic and real-world data. The code is available at https://github.com/HongtengXu/SGWB-Graphon.
MA-BERT: Learning Representation by Incorporating Multi-Attribute Knowledge in Transformers

Incorporating attribute information such as user and product features into deep neural networks has been shown to be useful in sentiment analysis.Previous works typically accomplished this in two ways: concatenating multiple attributes to word/text representation or treating them as a bias to adjust attention distribution.To leverage the advantages of both methods, this paper proposes a multi-attribute BERT (MA-BERT) to incorporate external attribute knowledge.The proposed method has two advantages.First, it applies multi-attribute transformer (MA-Transformer) encoders to incorporate multiple attributes into both input representation and attention distribution.Second, the MA-Transformer is implemented as a universal layer and stacked on a BERT-based model such that it can be initialized from a pre-trained checkpoint and fine-tuned for the downstream applications without extra pretraining costs.Experiments on three benchmark datasets show that the proposed method outperformed pre-trained BERT models and other methods incorporating external attribute knowledge.
Predicting in-hospital mortality by combining clinical notes with time-series data

In intensive care units (ICUs), patient health is monitored through (1) continuous vital signals from various medical devices, and (2) clinical notes consisting of opinions and summaries from doctors which are recorded in electronic health records (EHR).It is difficult to jointly model these two sources of information because clinical notes, unlike vital signals, are collected at irregular intervals and their contents are relatively unstructured.In this paper, we present a model that combines both sources of information about ICU patients to make accurate in-hospital mortality predictions.We apply a fine-tuned BERT model to each of the patient's clinical notes.The resulting embeddings are then combined to obtain the overall embedding for the entire text part of the data.This is then combined with the output of an LSTM model that encodes patients' vital signals.Our model improves upon the state of the art for mortality prediction, attaining an AUC score of 0.9, compared to the previous 0.87, setting a new standard for mortality prediction on the MIMIC III benchmark. 1
Discovering Dialogue Slots with Weak Supervision

Task-oriented dialogue systems typically require manual annotation of dialogue slots in training data, which is costly to obtain.We propose a method that eliminates this requirement: We use weak supervision from existing linguistic annotation models to identify potential slot candidates, then automatically identify domain-relevant slots by using clustering algorithms.Furthermore, we use the resulting slot annotation to train a neural-network-based tagger that is able to perform slot tagging with no human intervention.This tagger is trained solely on the outputs of our method and thus does not rely on any labeled data.Our model demonstrates state-of-the-art performance in slot tagging without labeled training data on four different dialogue domains.Moreover, we find that slot annotations discovered by our model significantly improve the performance of an end-to-end dialogue response generation model, compared to using no slot annotation at all.
Neural Sentence Ordering Based on Constraint Graphs

Sentence ordering aims at arranging a list of sentences in the correct order. Based on the observation that sentence order at different distances may rely on different types of information, we devise a new approach based on multi-granular orders between sentences. These orders form multiple constraint graphs, which are then encoded by Graph Isomorphism Networks and fused into sentence representations. Finally, sentence order is determined using the order-enhanced sentence representations. Our experiments on five benchmark datasets show that our method outperforms all existing baselines significantly, achieving a new state-of-the-art performance. The results demonstrate the advantage of considering multiple types of order information and using graph neural networks to integrate sentence content and order information for the task. Our code is available at https://github.com/DaoD/ConstraintGraph4NSO.
Using ProtoPNet for Interpretable Alzheimer’s Disease Classification

Early detection of Alzheimer's disease (AD) is significant for identifying of better treatment plans for the patients as the AD is not curable.On the other hand, lack of interpretability for the high performing prediction models might prevent incorporation of such models in clinical usage for AD detection.Accordingly, it is important to develop highly interpretable models which can create trust towards the prediction models by showing the factors that contribute to the models' decisions.In this paper, we use ProtoPNet architecture in combination with popular pretrained deep learning models to add interpretability to the AD classifications over MRI scans from ADNI and OASIS datasets.We find that the ProtoPNet model with DenseNet121 architecture can reach 90 percent accuracy while providing explanatory illustrations of the model's reasonings for the generated predictions.We also note that, in most cases, the performances of the ProtoPNet models are slightly inferior to their black-box counterparts, however, their ability to provide reasoning and transparency in the prediction generation process can contribute to higher adoption of the prediction models in clinical practice.
Right for Better Reasons: Training Differentiable Models by Constraining their Influence Functions

Explaining black-box models such as deep neural networks is becoming increasingly important as it helps to boost trust and debugging. Popular forms of explanations map the features to a vector indicating their individual importance to a decision on the instance-level. They can then be used to prevent the model from learning the wrong bias in data possibly due to ambiguity. For instance, Ross et al.'s ``right for the right reasons'' propagates user explanations backwards to the network by formulating differentiable constraints based on input gradients. Unfortunately, input gradients as well as many other widely used explanation methods form an approximation of the decision boundary and assume the underlying model to be fixed. Here, we demonstrate how to make use of influence functions---a well known robust statistic---in the constraints to correct the model’s behaviour more effectively. Our empirical evidence demonstrates that this ``right for better reasons''(RBR) considerably reduces the time to correct the classifier at training time and boosts the quality of explanations at inference time compared to input gradients. Besides, we also showcase the effectiveness of RBR in correcting "Clever Hans"-like behaviour in real, high-dimensional domain.
Learning Event Graph Knowledge for Abductive Reasoning

Abductive reasoning aims at inferring the most plausible explanation for observed events, which would play critical roles in various NLP applications, such as reading comprehension and question answering.To facilitate this task, a narrative text based abductive reasoning task αNLI is proposed, together with explorations about building reasoning framework using pretrained language models.However, abundant event commonsense knowledge is not well exploited for this task.To fill this gap, we propose a variational autoencoder based model ege-RoBERTa, which employs a latent variable to capture the necessary commonsense knowledge from event graph for guiding the abductive reasoning task.Experimental results show that through learning the external event graph knowledge, our approach outperforms the baseline methods on the αNLI task.
Synthesizing Adversarial Negative Responses for Robust Response Ranking and Evaluation

Open-domain neural dialogue models have achieved high performance in response ranking and evaluation tasks.These tasks are formulated as a binary classification of responses given in a dialogue context, and models generally learn to make predictions based on context-response content similarity.However, over-reliance on content similarity makes the models less sensitive to the presence of inconsistencies, incorrect time expressions and other factors important for response appropriateness and coherence.We propose approaches for automatically creating adversarial negative training data to help ranking and evaluation models learn features beyond content similarity.We propose mask-and-fill and keyword-guided approaches that generate negative examples for training more robust dialogue systems.These generated adversarial responses have high content similarity with the contexts but are either incoherent, inappropriate or not fluent.Our approaches are fully data-driven and can be easily incorporated in existing models and datasets.Experiments on classification, ranking and evaluation tasks across multiple datasets demonstrate that our approaches outperform strong baselines in providing informative negative examples for training dialogue systems. 1 Error category Description Sample responses C-entIncorrect entities or actors (R,G)Incorrect subject or object of verbs or presence of one or more incorrect entities or coreference.Context: I am so happy that you are doing okay.
Have We Solved The Hard Problem? It’s Not Easy! Contextual Lexical Contrast as a Means to Probe Neural Coherence

Lexical cohesion is a fundamental mechanism for text which requires a pair of words to be interpreted as a certain type of lexical relation (e.g., similarity) to understand a coherent context; we refer to such relations as the contextual lexical relation. However, work on lexical cohesion has not modeled context comprehensively in considering lexical relations due to the lack of linguistic resources. In this paper, we take initial steps to address contextual lexical relations by focusing on the contrast relation, as it is a well-known relation though it is more subtle and relatively less resourced. We present a corpus named Cont 2 Lex to make Contextual Lexical Contrast Recognition a computationally feasible task. We benchmark this task with widely-adopted semantic representations; we discover that contextual embeddings (e.g. BERT) generally outperform static embeddings (e.g. Glove), but barely go beyond 70% in accuracy performance. In addition, we ﬁnd that all embeddings perform better when CLC occurs within the same sentence, suggesting possible limitations of current computational coherence models. Another intriguing discovery is the improvement of BERT in CLC is largely attributed to its modeling of CLC word pairs co-occurring with other word repetitions. Such observations imply that the progress made in lexical coherence modeling remains relatively primitive even for semantic representations such as BERT that have been empowering numerous standard NLP tasks to approach human benchmarks. Through presenting our corpus and benchmark, we attempt to seed initial discussions and endeavors in advancing semantic representations from modeling syntactic and semantic levels to coherence and discourse levels.
On the Lack of Robust Interpretability of Neural Text Classifiers

With the ever-increasing complexity of neural language models, practitioners have turned to methods for understanding the predictions of these models.One of the most welladopted approaches for model interpretability is feature-based interpretability, i.e., ranking the features in terms of their impact on model predictions.Several prior studies have focused on assessing the fidelity of feature-based interpretability methods, i.e., measuring the impact of dropping the top-ranked features on the model output.However, relatively little work has been conducted on quantifying the robustness of interpretations.In this work, we assess the robustness of interpretations of neural text classifiers, specifically, those based on pretrained Transformer encoders, using two randomization tests.The first compares the interpretations of two models that are identical except for their initializations.The second measures whether the interpretations differ between a model with trained parameters and a model with random parameters.Both tests show surprising deviations from expected behavior, raising questions about the extent of insights that practitioners may draw from interpretations.
Pengaruh Penggunaan Media Papan Cerdas Perkalian Terhadap Hasil Belajar Matematika Materi Perkalian Siswa Kelas II di SD Negeri 75 Buton

The objective of the research was to know the effect of using multiplication smart board media toward students’ achievement on multiplication subject at grade II of SD Negeri 75 Buton, in academic year 2019/2020. This research was a quantitative research using pretest-posttest control group design. Sample was taken using simple random sampling technique, which was gained from population. There were 2 classes used as the sample, namely group II A as control group, which consisted of 8 students, and group II B as an experiment group, which consisted of 8 students. Based on the result of posttest, it found that students’ score from experiment group was higher than students’ score from control group. Classical completeness from experiment group = 77.5, and it was higher than control group = 73.5. based on the result of hypothesis test using t-test, then it found that tcount = 0,187, with df = 14 sig.(2-tailed) where 0.001 &lt; a (0.05). Therefore, it could be concluded that H0 was rejected and H1 was accepted. This result indicated that there was any effect of using multiplication smart board media toward students’ achievement on multiplication subject at grade II of SD Negeri 75 Buton.
How to Adapt Your Pretrained Multilingual Model to 1600 Languages

Pretrained multilingual models (PMMs) enable zero-shot learning via cross-lingual transfer, performing best for languages seen during pretraining.While methods exist to improve performance for unseen languages, they have almost exclusively been evaluated using amounts of raw text only available for a small fraction of the world's languages.In this paper, we evaluate the performance of existing methods to adapt PMMs to new languages using a resource available for over 1600 languages: the New Testament.This is challenging for two reasons: (1) the small corpus size, and (2) the narrow domain.While performance drops for all approaches, we surprisingly still see gains of up to 17.69% accuracy for part-ofspeech tagging and 6.29 F1 for NER on average over all languages as compared to XLM-R.Another unexpected finding is that continued pretraining, the simplest approach, performs best.Finally, we perform a case study to disentangle the effects of domain and size and to shed light on the influence of the finetuning source language.
PBC4occ: A novel contrast pattern-based classifier for one-class classification

In addition to accuracy, another key desirable characteristic of a classifier is interpretability. While there have been attempts to design contrast pattern-based models that support competitive and understandable classifiers, the utility of contrast patterns on the one-class classification problem is an under-explored area. In this paper, we propose a novel pattern-based classifier for one-class classification problems, PBC4occ. Moreover, we introduce the first contrast pattern mining algorithm utilizing decision trees for one-class classification. We analyze a number of contrast patterns extracted by our proposal and the one-class decision boundary built from an explanatory point. Additionally, we compare the performance of our proposal and those of thirteen (13) other state-of-the-art one-class classifiers on 95 imbalanced databases. Our findings show that PBC4occ achieves the best average value for both AUC and EER metrics. In addition, our proposal achieves the best and the second-best average Friedman’s ranking when evaluated under EER and AUC, respectively.
FIMAP: Feature Importance by Minimal Adversarial Perturbation

Instance-based model-agnostic feature importance explanations (LIME, SHAP, L2X) are a popular form of algorithmic transparency. These methods generally return either a weighting or subset of input features as an explanation for the classification of an instance. An alternative literature argues instead that counterfactual instances, which alter the black-box model's classification, provide a more actionable form of explanation. We present Feature Importance by Minimal Adversarial Perturbation (FIMAP), a neural network based approach that unifies feature importance and counterfactual explanations. We show that this approach combines the two paradigms, recovering the output of feature-weighting methods in continuous feature spaces, whilst indicating the direction in which the nearest counterfactuals can be found. Our method also provides an implicit confidence estimate in its own explanations, something existing methods lack. Additionally, FIMAP improves upon the speed of sampling-based methods, such as LIME, by an order of magnitude, allowing for explanation deployment in time-critical applications. We extend our approach to categorical features using a partitioned Gumbel layer and demonstrate its efficacy on standard datasets.