Reviewerly/exHarmony huggingface dataset:
OpenMEVA: A Benchmark for Evaluating Open-ended Story Generation Metrics

Automatic metrics are essential for developing natural language generation (NLG) models, particularly for open-ended language generation tasks such as story generation.However, existing automatic metrics are observed to correlate poorly with human evaluation.The lack of standardized benchmark datasets makes it difficult to fully evaluate the capabilities of a metric and fairly compare different metrics.Therefore, we propose OpenMEVA, a benchmark for evaluating open-ended story generation metrics.OpenMEVA provides a comprehensive test suite to assess the capabilities of metrics, including (a) the correlation with human judgments, (b) the generalization to different model outputs and datasets, (c) the ability to judge story coherence, and (d) the robustness to perturbations.To this end, Open-MEVA includes both manually annotated stories and auto-constructed test examples.We evaluate existing metrics on OpenMEVA and observe that they have poor correlation with human judgments, fail to recognize discourselevel incoherence, and lack inferential knowledge (e.g., causal order between events), the generalization ability and robustness.Our study presents insights for developing NLG models and metrics in further research.
Computer-Aided Diagnosis Evaluation of the Correlation Between Magnetic Resonance Imaging With Molecular Subtypes in Breast Cancer

Background There is a demand for additional alternative methods that can allow the differentiation of the breast tumor into molecular subtypes precisely and conveniently. Purpose The present study aimed to determine suitable optimal classifiers and investigate the general applicability of computer-aided diagnosis (CAD) to associate between the breast cancer molecular subtype and the extracted MR imaging features. Methods We analyzed a total of 264 patients (mean age: 47.9 ± 9.7 years; range: 19–81 years) with 264 masses (mean size: 28.6 ± 15.86 mm; range: 5–91 mm) using a Unet model and Gradient Tree Boosting for segmentation and classification. Results The tumors were segmented clearly by the Unet model automatically. All the extracted features which including the shape features,the texture features of the tumors and the clinical features were input into the classifiers for classification, and the results showed that the GTB classifier is superior to other classifiers, which achieved F1-Score 0.72, AUC 0.81 and score 0.71. Analyzed the different features combinations, we founded that the texture features associated with the clinical features are the optimal features to different the breast cancer subtypes. Conclusion CAD is feasible to differentiate the breast cancer subtypes, automatical segmentation were feasible by Unet model and the extracted texture features from breast MR imaging with the clinical features can be used to help differentiating the molecular subtype. Moreover, in the clinical features, BPE and age characteristics have the best potential for subtype.
Conformance Constraint Discovery

The reliability of inferences made by data-driven systems hinges on the data's continued conformance to the systems' initial settings and assumptions. When serving data (on which we want to apply inference) deviates from the profile of the initial training data, the outcome of inference becomes unreliable. We introduce conformance constraints, a new data profiling primitive tailored towards quantifying the degree of non-conformance, which can effectively characterize if inference over that tuple is untrustworthy. Conformance constraints are constraints over certain arithmetic expressions (called projections) involving the numerical attributes of a dataset, which existing data profiling primitives such as functional dependencies and denial constraints cannot model. Our key finding is that projections that incur low variance on a dataset construct effective conformance constraints. This principle yields the surprising result that low-variance components of a principal component analysis, which are usually discarded for dimensionality reduction, generate stronger conformance constraints than the high-variance components. Based on this result, we provide a highly scalable and efficient technique--linear in data size and cubic in the number of attributes--for discovering conformance constraints for a dataset. To measure the degree of a tuple's non-conformance with respect to a dataset, we propose a quantitative semantics that captures how much a tuple violates the conformance constraints of that dataset. We demonstrate the value of conformance constraints on two applications: trusted machine learning and data drift. We empirically show that conformance constraints offer mechanisms to (1) reliably detect tuples on which the inference of a machine-learned model should not be trusted, and (2) quantify data drift more accurately than the state of the art.
Can Jellyfish Dream? Conceptual Representations in Unsupervised Generative Learning

Representations play essential role in learning of artificial and biologic systems by identifying characteristic patterns in the observed environment. In this work we examine unsupervised latent representations of image data with a collective of generative neural network models. A convolutional autoencoder with strong redundancy reduction was used to create low-dimensional latent representations of a dataset of geometrical shapes. The structure of the resulting latent representations was studied comprehensively with several methods, including density clustering, direct visualization, generative probing and scanning. It was demonstrated that latent representations have high level of consistency between individual learners and a well-defined geometrical and topological structure correlated with principal patterns in the observable data. The results of this work support the hypothesis that conceptual latent representations can emerge naturally in unsupervised generative learning under certain essential constraints and can be a natural platform for emergence of some intelligent functions and behaviors.
Empower Distantly Supervised Relation Extraction with Collaborative Adversarial Training

With recent advances in distantly supervised (DS) relation extraction (RE), considerable attention is attracted to leverage multi-instance learning (MIL) to distill high-quality supervision from the noisy DS. Here, we go beyond label noise and identify the key bottleneck of DS-MIL to be its low data utilization: as high-quality supervision being refined by MIL, MIL abandons a large amount of training instances, which leads to a low data utilization and hinders model training from having abundant supervision. In this paper, we propose collaborative adversarial training to improve the data utilization, which coordinates virtual adversarial training (VAT) and adversarial training (AT) at different levels. Specifically, since VAT is label-free, we employ the instance-level VAT to recycle instances abandoned by MIL. Besides, we deploy AT at the bag-level to unleash the full potential of the high-quality supervision got by MIL. Our proposed method brings consistent improvements (∼ 5 absolute AUC score) to the previous state of the art, which verifies the importance of the data utilization issue and the effectiveness of our method.
The application of reality simulators for improving the education quality at universities

The article touches upon the problem of necessity to change pedagogical methods and technologies in high school education that are determined by the changes in modern society. The authors point out that the up-to-date society is deeply plunged into digital technologies and the reality is that methods being used in education cannot meet the requirements of the present. In this regard the authors speculate that application of digital technologies, namely laboratory facilities of virtual reality in training process of technical university students is able to improve the education quality. The goal of this research is to prove this theory. The authors give a thorough comparative analysis of academic literature that describes the existing virtual reality facilities which are widely used in higher educational establishments of transport abroad as well as in the Russian Federation. The typology of training facilities is given in the article, all advantages and drawbacks of digital technologies concerning the virtual reality on the example of reality simulators railway complexes have been described. With the help of statistic data processing methods that have been obtained by the experimental way the authors have proved that the application of reality simulators laboratory facilities can improve the quality of training at technical higher educational establishments.
The effects of Audiovisual Media on Students’ Listening Skills

Listening is a fundamental ability to train English.  The purpose of this article is to describe the use of audiovisual media in improving students listening skills. The primary purpose of this research is to know the application of sound learning to sound development in improving listening and understanding the ability to listen to English students after learning how to use visual media. This review is quantitative.  About data analysis, percentage, and point t-test, the sample was used.  The results show that students of English students a significant increase in understanding after learning with videos and students the positive attitude for using videos in listening to listening skills.
Multi-Head Highly Parallelized LSTM Decoder for Neural Machine Translation

One of the reasons Transformer translation models are popular is that self-attention networks for context modelling can be easily parallelized at sequence level.However, the computational complexity of a self-attention network is O(n 2 ), increasing quadratically with sequence length.By contrast, the complexity of LSTM-based approaches is only O(n).In practice, however, LSTMs are much slower to train than self-attention networks as they cannot be parallelized at sequence level: to model context, the current LSTM state relies on the full LSTM computation of the preceding state.This has to be computed n times for a sequence of length n.The linear transformations involved in the LSTM gate and state computations are the major cost factors in this.To enable sequence-level parallelization of LSTMs, we approximate full LSTM context modelling by computing hidden states and gates with the current input and a simple bag-of-words representation of the preceding tokens context.This allows us to compute each input step efficiently in parallel, avoiding the formerly costly sequential linear transformations.We then connect the outputs of each parallel step with computationally cheap element-wise computations.We call this the Highly Parallelized LSTM.To further constrain the number of LSTM parameters, we compute several small HPLSTMs in parallel like multi-head attention in the Transformer.The experiments show that our MHPLSTM decoder achieves significant BLEU improvements, while being even slightly faster than the self-attention network in training, and much faster than the standard LSTM.
Lexically Constrained Neural Machine Translation with Explicit Alignment Guidance

Lexically constrained neural machine translation (NMT), which leverages pre-specified translation to constrain NMT, has practical significance in interactive translation and NMT domain adaption. Previous work either modify the decoding algorithm or train the model on augmented dataset. These methods suffer from either high computational overheads or low copying success rates. In this paper, we investigate Att-Input and Att-Output, two alignment-based constrained decoding methods. These two methods revise the target tokens during decoding based on word alignments derived from encoder-decoder attention weights. Our study shows that Att-Input translates better while Att-Output is more computationally efficient. Capitalizing on both strengths, we further propose EAM-Output by introducing an explicit alignment module (EAM) to a pretrained Transformer. It decodes similarly as EAM-Output, except using alignments derived from the EAM. We leverage the word alignments induced from Att-Input as labels and train the EAM while keeping the parameters of the Transformer frozen. Experiments on WMT16 De-En and WMT16 Ro-En show the effectiveness of our approaches on constrained NMT. In particular, the proposed EAM-Output method consistently outperforms previous approaches in translation quality, with light computational overheads over unconstrained baseline.
Graph Neural Network-Based Anomaly Detection in Multivariate Time Series

Given high-dimensional time series data (e.g., sensor data), how can we detect anomalous events, such as system faults and attacks? More challengingly, how can we do this in a way that captures complex inter-sensor relationships, and detects and explains anomalies which deviate from these relationships? Recently, deep learning approaches have enabled improvements in anomaly detection in high-dimensional datasets; however, existing methods do not explicitly learn the structure of existing relationships between variables, or use them to predict the expected behavior of time series. Our approach combines a structure learning approach with graph neural networks, additionally using attention weights to provide explainability for the detected anomalies. Experiments on two real-world sensor datasets with ground truth anomalies show that our method detects anomalies more accurately than baseline approaches, accurately captures correlations between sensors, and allows users to deduce the root cause of a detected anomaly.
Training of the feed-forward artificial neural networks using butterfly optimization algorithm

Artificial Neural Network (ANN) learns from inputs and outputs. The values of the weights and biases in ANN are updated according to inputs and outputs. Researchers have proposed algorithms to train Multi-Layer Perceptron (MLP). However, classical techniques often face problems in solving this optimization problem. They tend to need large amounts of computing time, large amounts of memory. More importantly, they get stuck within the local optimum and produce poor-quality solutions. To overcome these difficulties, meta-heuristic algorithms have been used to train MLP. In this article, the Butterfly Optimization Algorithm (BOA) which was designed by modeling the behaviors of butterflies was used for the first time to train the multi-layer perceptron. The developed algorithm was named BOA-MLP where the BOA algorithm optimized the values of the weights and biases in the MLP. The success of the BOA-MLP algorithm was tested on five data sets (iris, breast cancer, heart, balloon and xor) which are frequently used in the literature. In the experiments, the BOA-MLP algorithm was compared with the BAT-MLP, SMS-MLP and BP algorithms. The average and standard deviation of the mean squared error, the average classification accuracy, the sensitivity, the specificity, the precision and the F1-score were used as the performance metrics. According to the experimental results, it is seen that the BOA-MLP algorithm surpasses the BAT-MLP, SMS-MLP and BP algorithms on all data sets and shows superior success.
On Positivity Bias in Negative Reviews

Prior work has revealed that positive words occur more frequently than negative words in human expressions, which is typically attributed to positivity bias, a tendency for people to report positive views of reality.But what about the language used in negative reviews?Consistent with prior work, we show that English negative reviews tend to contain more positive words than negative words, using a variety of datasets.We reconcile this observation with prior findings on the pragmatics of negation, and show that negations are commonly associated with positive words in negative reviews.Furthermore, in negative reviews, the majority of sentences with positive words express negative opinions based on sentiment classifiers, indicating some form of negation.
The impact of translation modality on user experience: an eye-tracking study of the Microsoft Word user interface

This paper presents results of the effect of different translation modalities on users when working with the Microsoft Word user interface. An experimental study was set up with 84 Japanese, German, Spanish, and English native speakers working with Microsoft Word in three modalities: the published translated version, a machine translated (MT) version (with unedited MT strings incorporated into the MS Word interface) and the published English version. An eye-tracker measured the cognitive load and usability according to the ISO/TR 16982 guidelines: i.e., effectiveness, efficiency, and satisfaction followed by retrospective think-aloud protocol. The results show that the users' effectiveness (number of tasks completed) does not significantly differ due to the translation modality. However, their efficiency (time for task completion) and self-reported satisfaction are significantly higher when working with the released product as opposed to the unedited MT version, especially when participants are less experienced. The eye-tracking results show that users experience a higher cognitive load when working with MT and with the human-translated versions as opposed to the English original. The results suggest that language and translation modality play a significant role in the usability of software products whether users complete the given tasks or not and even if they are unaware that MT was used to translate the interface.
Multi-stage Pre-training over Simplified Multimodal Pre-training Models

Multimodal pre-training models, such as LXMERT, have achieved excellent results in downstream tasks.However, current pretrained models require large amounts of training data and have huge model sizes, which make them difficult to apply in low-resource situations.How to obtain similar or even better performance than a larger model under the premise of less pre-training data and smaller model size has become an important problem.In this paper, we propose a new Multi-stage Pre-training (MSP) method, which uses information at different granularities from word, phrase to sentence in both texts and images to pre-train the model in stages.We also design several different pre-training tasks suitable for the information granularity in different stage in order to efficiently capture the diverse knowledge from a limited corpus.We take a Simplified LXMERT (LXMERT-S), which has only 45.9% parameters of the original LXMERT model and 11.76% of the original pre-training data as the testbed of our MSP method.Experimental results show that our method achieves comparable performance to the original LXMERT model in all downstream tasks, and even outperforms the original model in Image-Text Retrieval task.
Model-based clustering for random hypergraphs

A probabilistic model for random hypergraphs is introduced to represent unary, binary and higher order interactions among objects in real-world problems. This model is an extension of the latent class analysis model that introduces two clustering structures for hyperedges and captures variation in the size of hyperedges. An expectation maximization algorithm with minorization maximization steps is developed to perform parameter estimation. Model selection using Bayesian Information Criterion is proposed. The model is applied to simulated data and two real-world data sets where interesting results are obtained.
The effect of hyperparameter search on artificial neural network in human activity recognition

Abstract In the last decade, many researchers applied shallow and deep networks for human activity recognition (HAR). Currently, the trending research line in HAR is applying deep learning to extract features and classify activities from raw data. However, we observed that, authors of previous studies have not performed an efficient hyperparameter search on their artificial neural network (shallow or deep)-based classifier. Therefore, in this article, we demonstrate the effect of the random and Bayesian parameter search on a shallow neural network using five HAR databases. The result of this work shows that a shallow neural network with correct parameter optimization can achieve similar or even better recognition accuracy than the previous best deep classifier(s) on all databases. In addition, we draw conclusions about the advantages and disadvantages of the two hyperparameter search techniques according to the results.
Prototyping Machine Learning Through Diffractive Art Practice

In this paper, we outline a diffractive practice of machine learning (ML) in the frame of material-centered interaction design. To this aim, we review related work in ML, HCI, design, new interfaces for musical expression, and computational art, and introduce two practice-based studies of music performance and robotic art based on interactive machine learning tools, with the hope of revealing the computational materiality of ML, and the potential of embodiment to craft prototypes of ML that reconfigure conceptual or technical approaches to ML. We derive five interference conditions for such art-based ML prototypes—situational whole, small data, shallow model, learnable algorithm, and somaesthetic behaviour—and describe their widening of design and engineering practices of ML prototyping. Finally, we sketch how a process of intra-active machine learning could complement that of interactive machine learning to take materiality as an entry point for ML design within HCI.
Dissecting Generation Modes for Abstractive Summarization Models via Ablation and Attribution

Despite the prominence of neural abstractive summarization models, we know little about how they actually form summaries and how to understand where their decisions come from.We propose a two-step method to interpret summarization model decisions.We first analyze the model's behavior by ablating the full model to categorize each decoder decision into one of several generation modes: roughly, is the model behaving like a language model, is it relying heavily on the input, or is it somewhere in between?After isolating decisions that do depend on the input, we explore interpreting these decisions using several different attribution methods.We compare these techniques based on their ability to select content and reconstruct the model's predicted token from perturbations of the input, thus revealing whether highlighted attributions are truly important for the generation of the next token.While this machinery can be broadly useful even beyond summarization, we specifically demonstrate its capability to identify phrases the summarization model has memorized and determine where in the training pipeline this memorization happened, as well as study complex generation phenomena like sentence fusion on a per-instance basis.Conclusion: These doc tokens impacted prediction the most (according to int.grad.)Conclusion: Higher difference means higher dependence on context Ablation mayoral Cameron 0.01 0.01 0.56 0.99 Diff between LM and full model for Khan 0.96 0.99 0.99 0.99 Input Article Speaking at a rally for Tory candidate Zac Goldsmith, the prime minister warned about the dangers of a Labour victory for the capital's economy.Mr Goldsmith said his Labour rival was "Mr Corbyn's man" in City Hall.But