Reviewerly/exHarmony huggingface dataset:
Undesirable Difficulties: Investigating Barriers to Students’ Learning with Ebooks in a Semester-length Course

Our ability to make informed decisions about ebooks is constrained by our limited understanding of how students perceive and use them. A team of librarians and a professor in learning sciences asked graduate students to serve as informants on student experience with ebooks. We analyzed student work in two semester-long studies, focusing on barriers and affordances they identified. In the first cohort, students who chose to explore ebooks uncovered affordances. In the second cohort, student comfort levels with PDF formats increased, while comfort with ebooks decreased. We discuss strategies for minimizing challenges and increasing desirable difficulties to support ebooks as learning tools.
Integrating Personas and Use Case Models

Multidisciplinary design is characterized by phases of distributed work and co-design activities. An effective sharing and integration of design representations that are created by sub-teams from different disciplines is still often challenging and typically requires the reconciliation of diverging design perspectives. This paper investigates an integrated use of personas and use cases - two popular types of design representations among interaction designers and software engineers respectively. The proposed integration is particularly suitable for role-based interactive systems and differs from existing integration approaches in that it is based on a critical examination of the prevalent understandings of the goal concept in persona and use case approaches. In the paper we suggest distinguishing between organizational and user goals (while at the same time acknowledging their interplay). Corresponding adaptations to use case notations and personas are introduced and discussed. These remove the tight coupling between goals and tasks and allow integration of organizational and different persona-specific design perspectives within one use case specification and at the interaction level. As a result, interactive systems can be specified by a more compact sets of use cases. This is illustrated by an example in the context of course management systems in higher education.
Kingsoft’s Neural Machine Translation System for WMT19

This paper describes the Kingsoft AI Lab's submission to the WMT2019 news translation shared task.We participated in two language directions: English→Chinese and Chinese→English.For both language directions, we trained several variants of Transformer models using the provided parallel data enlarged with a large quantity of backtranslated monolingual data.The best translation result was obtained with ensemble and reranking techniques.According to automatic metrics (BLEU) our Chinese→English system reached the second highest score, and our English→Chinese system reached the second highest score for this subtask.
Real-World Oriented Smartphone AR Supported Learning System Based on Planetarium Contents for Seasonal Constellation Observation

A popular astronomical concept covered by projection learning programs in the planetarium is seasonal constellation. However, a planetarium’s learning environment is limited to virtual scenes, where learners can observe seasonal constellations, but there is a significant difference between reality and the learners’ imagination regarding constellations. It is important to create a real-world oriented observation learning environment for observing seasonal constellations. Augmented reality has proved to be a powerful tool for astronomical observation learning. In this paper, augmented reality (AR) contents and 2D contents are used to develop a smartphone-based learning system called the Real-World Oriented Smartphone AR Learning System (R-WOSARLS) for seasonal constellation observation, which is based on the planetarium contents of the planetarium of the Nagoya City Science Museum, for seasonal constellation learning. Two experiments were conducted to evaluate the usefulness, usability, and learner satisfaction of our system in university and junior high school, respectively. The results show that R-WOSARLS is an effective learning tool for constellation observation and learning, and it enhances learners’ motivation to pursue seasonal constellation learning. Moreover, R-WOSARLS could be a teaching tool not only to help students learn more than with traditional instruction, but also to stimulate their interest in astronomical phenomena outside of school.
Correlations between Word Vector Sets

Vitalii Zhelezniak, April Shen, Daniel Busbridge, Aleksandar Savkov, Nils Hammerla. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.
Investigating Sub-Word Embedding Strategies for the Morphologically Rich and Free Phrase-Order Hungarian

For morphologically rich languages, word embeddings provide less consistent semantic representations due to higher variance in word forms.Moreover, these languages often allow for less constrained word order, which further increases variance.For the highly agglutinative Hungarian, semantic accuracy of word embeddings measured on word analogy tasks drops by 50-75% compared to English.We observed that embeddings learn morphosyntax quite well instead.Therefore, we explore and evaluate several sub-word unit based embedding strategiescharacter n-grams, lemmatization provided by an NLP-pipeline, and segments obtained in unsupervised learning (morfessor) -to boost semantic consistency in Hungarian word vectors.The effect of changing embedding dimension and context window size have also been considered.Morphological analysis based lemmatization was found to be the best strategy to improve embeddings' semantic accuracy, whereas adding character n-grams was found consistently counterproductive in this regard.
A Search-based Neural Model for Biomedical Nested and Overlapping Event Detection

We tackle the nested and overlapping event detection task and propose a novel search-based neural network (SBNN) structured prediction model that treats the task as a search problem on a relation graph of trigger-argument structures.Unlike existing structured prediction tasks such as dependency parsing, the task targets to detect DAG structures, which constitute events, from the relation graph.We define actions to construct events and use all the beams in a beam search to detect all event structures that may be overlapping and nested.The search process constructs events in a bottom-up manner while modelling the global properties for nested and overlapping structures simultaneously using neural networks.We show that the model achieves performance comparable to the state-of-the-art model Turku Event Extraction System (TEES) on the BioNLP Cancer Genetics (CG) Shared Task 2013 without the use of any syntactic and hand-engineered features.Further analyses on the development set show that our model is more computationally efficient while yielding higher F1-score performance.
A Test Suite and Manual Evaluation of Document-Level NMT at WMT19

As the quality of machine translation rises and neural machine translation (NMT) is moving from sentence to document level translations, it is becoming increasingly difficult to evaluate the output of translation systems.We provide a test suite for WMT19 aimed at assessing discourse phenomena of MT systems participating in the News Translation Task.We have manually checked the outputs and identified types of translation errors that are relevant to document-level translation.
Bridging the Defined and the Defining: Exploiting Implicit Lexical Semantic Relations in Definition Modeling

Definition modeling includes acquiring word embeddings from dictionary definitions and generating definitions of words.While the meanings of defining words are important in dictionary definitions, it is crucial to capture the lexical semantic relations between defined words and defining words.However, thus far, the utilization of such relations has not been explored for definition modeling.In this paper, we propose definition modeling methods that use lexical semantic relations.To utilize implicit semantic relations in definitions, we use unsupervisedly obtained pattern-based word-pair embeddings that represent semantic relations of word pairs.Experimental results indicate that our methods improve the performance in learning embeddings from definitions, as well as definition generation.
Combining Generative and Discriminative Models for Hybrid Inference

A graphical model is a structured representation of the data generating process. The traditional method to reason over random variables is to perform inference in this graphical model. However, in many cases the generating process is only a poor approximation of the much more complex true data generating process, leading to suboptimal estimation. The subtleties of the generative process are however captured in the data itself and we can ``learn to infer'', that is, learn a direct mapping from observations to explanatory latent variables. In this work we propose a hybrid model that combines graphical inference with a learned inverse model, which we structure as in a graph neural network, while the iterative algorithm as a whole is formulated as a recurrent neural network. By using cross-validation we can automatically balance the amount of work performed by graphical inference versus learned inference. We apply our ideas to the Kalman filter, a Gaussian hidden Markov model for time sequences, and show, among other things, that our model can estimate the trajectory of a noisy chaotic Lorenz Attractor much more accurately than either the learned or graphical inference run in isolation.
Open Source Infrastructure for Health Care Data Integration and Machine Learning Analyses

PURPOSE We have created a cloud-based machine learning system (CLOBNET) that is an open-source, lean infrastructure for electronic health record (EHR) data integration and is capable of extract, transform, and load (ETL) processing. CLOBNET enables comprehensive analysis and visualization of structured EHR data. We demonstrate the utility of CLOBNET by predicting primary therapy outcomes of patients with high-grade serous ovarian cancer (HGSOC) on the basis of EHR data. MATERIALS AND METHODS CLOBNET is built using open-source software to make data preprocessing, analysis, and model training user friendly. The source code of CLOBNET is available in GitHub. The HGSOC data set was based on a prospective cohort of 208 patients with HGSOC who were treated at Turku University Hospital, Finland, from 2009 to 2019 for whom comprehensive clinical and EHR data were available. RESULTS We trained machine learning (ML) models using clinical data, including a herein developed dissemination score that quantifies the disease burden at the time of diagnosis, to identify patients with progressive disease (PD) or a complete response (CR) on the basis of RECIST (version 1.1). The best performance was achieved with a logistic regression model, which resulted in an area under receiver operating characteristic curve (AUROC) of 0.86, with a specificity of 73% and a sensitivity of 89%, when it classified between patients who experienced PD and CR. CONCLUSION We have developed an open-source computational infrastructure, CLOBNET, that enables effective and rapid analysis of EHR and other clinical data. Our results demonstrate that CLOBNET allows predictions to be made on the basis of EHR data to address clinically relevant questions.
Learning Only from Relevant Keywords and Unlabeled Documents

We consider a document classification problem where document labels are absent but only relevant keywords of a target class and unlabeled documents are given.Although heuristic methods based on pseudo-labeling have been considered, theoretical understanding of this problem has still been limited.Moreover, previous methods cannot easily incorporate welldeveloped techniques in supervised text classification.In this paper, we propose a theoretically guaranteed learning framework that is simple to implement and has flexible choices of models, e.g., linear models or neural networks.We demonstrate how to optimize the area under the receiver operating characteristic curve (AUC) effectively and also discuss how to adjust it to optimize other well-known evaluation metrics such as the accuracy and F 1measure.Finally, we show the effectiveness of our framework using benchmark datasets.
Specific and Shared Causal Relation Modeling and Mechanism-Based Clustering

State-of-the-art approaches to causal discovery usually assume a fixed underlying causal model. However, it is often the case that causal models vary across domains or subjects, due to possibly omitted factors that affect the quantitative causal effects. As a typical example, causal connectivity in the brain network has been reported to vary across individuals, with significant differences across groups of people, such as autistics and typical controls. In this paper, we develop a unified framework for causal discovery and mechanism-based group identification. In particular, we propose a specific and shared causal model (SSCM), which takes into account the variabilities of causal relations across individuals/groups and leverages their commonalities to achieve statistically reliable estimation. The learned SSCM gives the specific causal knowledge for each individual as well as the general trend over the population. In addition, the estimated model directly provides the group information of each individual. Experimental results on synthetic and real-world data demonstrate the efficacy of the proposed method.
Tilde’s Machine Translation Systems for WMT 2019

The paper describes the development process of Tilde's NMT systems for the WMT 2019 shared task on news translation.We trained systems for the English-Lithuanian and Lithuanian-English translation directions in constrained and unconstrained tracks.We build upon the best methods of the previous year's competition and combine them with recent advancements in the field.We also present a new method to ensure source domain adherence in back-translated data.Our systems achieved a shared first place in human evaluation.
Exploring Transfer Learning and Domain Data Selection for the Biomedical Translation

Transfer Learning and Selective data training are two of the many approaches being extensively investigated to improve the quality of Neural Machine Translation systems.This paper presents a series of experiments by applying transfer learning and selective data training for participation in the Bio-medical shared task of WMT19.We have used Information Retrieval to selectively choose related sentences from out-of-domain data and used them as additional training data using transfer learning.We also report the effect of tokenization on translation model performance.
Neural and FST-based approaches to grammatical error correction

In this paper, we describe our submission to the BEA 2019 shared task on grammatical error correction.We present a system pipeline that utilises both error detection and correction models.The input text is first corrected by two complementary neural machine translation systems: one using convolutional networks and multi-task learning, and another using a neural Transformer-based system.Training is performed on publicly available data, along with artificial examples generated through back-translation.The n-best lists of these two machine translation systems are then combined and scored using a finite state transducer (FST).Finally, an unsupervised reranking system is applied to the n-best output of the FST.The re-ranker uses a number of error detection features to re-rank the FST nbest list and identify the final 1-best correction hypothesis.Our system achieves 66.75% F 0.5 on error correction (ranking 4th), and 82.52% F 0.5 on token-level error detection (ranking 2nd) in the restricted track of the shared task.
Improved Generalization of Arabic Text Classifiers

While transfer learning for text has been very active in the English language, progress in Arabic has been slow, including the use of Domain Adaptation (DA).Domain Adaptation is used to generalize the performance of any classifier by trying to balance the classifier's accuracy for a particular task among different text domains.In this paper, we propose and evaluate two variants of a domain adaptation technique: the first is a base model called Domain Adversarial Neural Network (DANN), while the second is a variation that incorporates representational learning.Similar to previous approaches, we propose the use of proxy A-distance as a metric to assess the success of generalization.We make use of ArSentD-LEV, a multi-topic dataset collected from the Levantine countries, to test the performance of the models.We show the superiority of the proposed method in accuracy and robustness when dealing with the Arabic language.
Unsupervised Feature Selection and Clustering Optimization Based on Improved Differential Evolution

The feature selection method based on supervised learning has been widely studied and applied to the field of machine learning and data mining. But unsupervised feature selection is still a tricky area of research because the unavailability of the label information, especially for clustering tasks. Irrelevant features and redundant features in the original data seriously block the discovery of clustering structure and weaken the performance of the subsequent classification. In order to address this problem, the unsupervised feature selection and clustering algorithm based on the evolutionary computing framework is proposed in this paper. First, the binary differential evolution algorithm is constructed for unsupervised feature selection. Specifically, the individuals of the population are used to characterize the feature subspaces and the improved Laplacian model is designed to measure the local manifold structure of each individual. Subsequently, the approximate optimal manifold structure and the corresponding feature subset are obtained. Then, the continuous differential evolutionary algorithm is executed on the optimized feature subset, in which the individual representation strategy and the integrated individual measure function are designed for clustering. Moreover, the predicted pseudo-labels are utilized to classify and further verify the validity of clustering. The experimental results demonstrate that the proposed framework outperforms the most state-of-the-art methods.
Using Artificial Neural Network for Multimedia Information Retrieval

Multimedia Information Retrieval (MIR) is an important field due to the great amount of information going through the Internet. Multimedia data can be considered as raw data or the features that compose it. Raw multimedia data consists of data structures with diverse characteristics such as image, audio, video, and text. The big challenge of MIR is a semantic gap, which is the difference between the human perception of a concept and how it can be represented using a machine-level language. The aim of this paper is to use different algorithms through two stages one for training and the other for testing. The first algorithm depends on the nature of the query language to retrieve the text document using two models, Vector Space Model (VSM) and Latent Semantic Index (LSI). The second algorithm is based on the extracted features using curvelet decomposition and the statistic parameters such as mean, standard deviation and energy of signals. The other algorithm is based on the discrete wavelet transform (DWT) and features of signals to retrieve audio signals, then the neural network is applied to describe the information retrieval model which retrieves the information from the multimedia. The neural network model, based on multiplayer perceptron and spreading activation network type, accepts the structure of conceptually and linguistically oriented model.
Provable Non-linear Inductive Matrix Completion

Consider a standard recommendation/retrieval problem where given a query, the goal is to retrieve the most relevant items. Inductive matrix completion (IMC) method is a standard approach for this problem where the given query as well as the items are embedded in a common low-dimensional space. The inner product between a query embedding and an item embedding reflects relevance of the (query, item) pair. Non-linear IMC (NIMC) uses non-linear networks to embed the query as well as items, and is known to be highly effective for a variety of tasks, such as video recommendations for users, semantic web search, etc. Despite its wide usage, existing literature lacks rigorous understanding of NIMC models. A key challenge in analyzing such models is to deal with the non-convexity arising out of non-linear embeddings in addition to the non-convexity arising out of the low-dimensional restriction of the embedding space, which is akin to the low-rank restriction in the standard matrix completion problem. In this paper, we provide the first theoretical analysis for a simple NIMC model in the realizable setting, where the relevance score of a (query, item) pair is formulated as the inner product between their single-layer neural representations. Our results show that under mild assumptions we can recover the ground truth parameters of the NIMC model using standard (stochastic) gradient descent methods if the methods are initialized within a small distance to the optimal parameters. We show that a standard tensor method can be used to initialize the solution within the required distance to the optimal parameters. Furthermore, we show that the number of query-item relevance observations required, a key parameter in learning such models, scales nearly linearly with the input dimensionality thus matching existing results for the standard linear inductive matrix completion.
GTCOM Neural Machine Translation Systems for WMT19

This paper describes the Global Tone Communication Co., Ltd.'s submission of the WMT19 shared news translation task.We participate in six directions: English to (Gujarati, Lithuanian and Finnish) and (Gujarati, Lithuanian and Finnish) to English.Further, we get the best BLEU scores in the directions of English to Gujarati and Lithuanian to English (28.2 and 36.3 respectively) among all the participants.The submitted systems mainly focus on backtranslation, knowledge distillation and reranking to build a competitive model for this task.Also, we apply language model to filter monolingual data, back-translated data and parallel data.The techniques we apply for data filtering include filtering by rules, language models.Besides, We conduct several experiments to validate different knowledge distillation techniques and right-to-left (R2L) reranking.
Classifying the reported ability in clinical mobility descriptions

Assessing how individuals perform different activities is key information for modeling health states of individuals and populations.Descriptions of activity performance in clinical free text are complex, including syntactic negation and similarities to textual entailment tasks.We explore a variety of methods for the novel task of classifying four types of assertions about activity performance: Able, Unable, Unclear, and None (no information).We find that ensembling an SVM trained with lexical features and a CNN achieves 77.9% macro F1 score on our task, and yields nearly 80% recall on the rare Unclear and Unable samples.Finally, we highlight several challenges in classifying performance assertions, including capturing information about sources of assistance, incorporating syntactic structure and negation scope, and handling new modalities at test time.Our findings establish a strong baseline for this novel task, and identify intriguing areas for further research.