pwc-archive/papers-with-abstracts huggingface dataset:
Spatial-Temporal Self-Attention Network for Flow Prediction
Authors: Haoxing Lin, Weijia Jia, Yiping Sun, Yongjian You
Date: 2019-12-13 00:00:00
Tasks: Prediction

Flow prediction (e.g., crowd flow, traffic flow) with features of spatial-temporal is increasingly investigated in AI research field. It is very challenging due to the complicated spatial dependencies between different locations and dynamic temporal dependencies among different time intervals. Although measurements of both dependencies are employed, existing methods suffer from the following two problems. First, the temporal dependencies are measured either uniformly or bias against long-term dependencies, which overlooks the distinctive impacts of short-term and long-term temporal dependencies. Second, the existing methods capture spatial and temporal dependencies independently, which wrongly assumes that the correlations between these dependencies are weak and ignores the complicated mutual influences between them. To address these issues, we propose a Spatial-Temporal Self-Attention Network (ST-SAN). As the path-length of attending long-term dependency is shorter in the self-attention mechanism, the vanishing of long-term temporal dependencies is prevented. In addition, since our model relies solely on attention mechanisms, the spatial and temporal dependencies can be simultaneously measured. Experimental results on real-world data demonstrate that, in comparison with state-of-the-art methods, our model reduces the root mean square errors by 9% in inflow prediction and 4% in outflow prediction on Taxi-NYC data, which is very significant compared to the previous improvement.
It's easy to fool yourself: Case studies on identifying bias and confounding in bio-medical datasets
Authors: Subhashini Venugopalan, Arunachalam Narayanaswamy, Samuel Yang, Anton Geraschenko, Scott Lipnick, Nina Makhortova, James Hawrot, Christine Marques, Joao Pereira, Michael Brenner, Lee Rubin, Brian Wainger, Marc Berndl
Date: 2019-12-12 00:00:00
Tasks: BIG-bench Machine Learning, Prediction

Confounding variables are a well known source of nuisance in biomedical studies. They present an even greater challenge when we combine them with black-box machine learning techniques that operate on raw data. This work presents two case studies. In one, we discovered biases arising from systematic errors in the data generation process. In the other, we found a spurious source of signal unrelated to the prediction task at hand. In both cases, our prediction models performed well but under careful examination hidden confounders and biases were revealed. These are cautionary tales on the limits of using machine learning techniques on raw data from scientific experiments.
Deep Learning based Multiple Regression to Predict Total Column Water Vapor (TCWV) from Physical Parameters in West Africa by using Keras Library
Authors: Daouda Diouf, Awa Niang, Sylvie Thiria
Date: 2019-12-12 00:00:00
Tasks: Deep Learning, regression

Total column water vapor is an important factor for the weather and climate. This study apply deep learning based multiple regression to map the TCWV with elements that can improve spatiotemporal prediction. In this study, we predict the TCWV with the use of ERA5 that is the fifth generation ECMWF atmospheric reanalysis of the global climate. We use an appropriate deep learning based multiple regression algorithm using Keras library to improve nonlinear prediction between Total Column water vapor and predictors as Mean sea level pressure, Surface pressure, Sea surface temperature, 100 metre U wind component, 100 metre V wind component, 10 metre U wind component, 10 metre V wind component, 2 metre dew point temperature, 2 metre temperature. The results obtained permit to build a predictor which modelling TCWV with a mean abs error (MAE) equal to 3.60 kg/m2 and a coefficient of determination R2 equal to 0.90.
Graph Input Representations for Machine Learning Applications in Urban Network Analysis
Authors: Alessio Pagani, Abhinav Mehrotra, Mirco Musolesi
Date: 2019-12-11 00:00:00
Tasks: BIG-bench Machine Learning

Understanding and learning the characteristics of network paths has been of particular interest for decades and has led to several successful applications. Such analysis becomes challenging for urban networks as their size and complexity are significantly higher compared to other networks. The state-of-the-art machine learning (ML) techniques allow us to detect hidden patterns and, thus, infer the features associated with them. However, very little is known about the impact on the performance of such predictive models by the use of different input representations. In this paper, we design and evaluate six different graph input representations (i.e., representations of the network paths), by considering the network's topological and temporal characteristics, for being used as inputs for machine learning models to learn the behavior of urban networks paths. The representations are validated and then tested with a real-world taxi journeys dataset predicting the tips using a road network of New York. Our results demonstrate that the input representations that use temporal information help the model to achieve the highest accuracy (RMSE of 1.42$).
SpaRCe: Improved Learning of Reservoir Computing Systems through Sparse Representations
Authors: Luca Manneschi, Andrew C. Lin, Eleni Vasilaki
Date: 2019-12-04 00:00:00
Tasks: BIG-bench Machine Learning, Decision Making

"Sparse" neural networks, in which relatively few neurons or connections are active, are common in both machine learning and neuroscience. Whereas in machine learning, "sparsity" is related to a penalty term that leads to some connecting weights becoming small or zero, in biological brains, sparsity is often created when high spiking thresholds prevent neuronal activity. Here we introduce sparsity into a reservoir computing network via neuron-specific learnable thresholds of activity, allowing neurons with low thresholds to contribute to decision-making but suppressing information from neurons with high thresholds. This approach, which we term "SpaRCe", optimises the sparsity level of the reservoir without affecting the reservoir dynamics. The read-out weights and the thresholds are learned by an on-line gradient rule that minimises an error function on the outputs of the network. Threshold learning occurs by the balance of two opposing forces: reducing inter-neuronal correlations in the reservoir by deactivating redundant neurons, while increasing the activity of neurons participating in correct decisions. We test SpaRCe on classification problems and find that threshold learning improves performance compared to standard reservoir computing. SpaRCe alleviates the problem of catastrophic forgetting, a problem most evident in standard echo state networks and recurrent neural networks in general, due to increasing the number of task-specialised neurons that are included in the network decisions.
The accuracy vs. coverage trade-off in patient-facing diagnosis models
Authors: Anitha Kannan, Jason Alan Fries, Eric Kramer, Jen Jen Chen, Nigam Shah, Xavier Amatriain
Date: 2019-12-11 00:00:00

A third of adults in America use the Internet to diagnose medical concerns, and online symptom checkers are increasingly part of this process. These tools are powered by diagnosis models similar to clinical decision support systems, with the primary difference being the coverage of symptoms and diagnoses. To be useful to patients and physicians, these models must have high accuracy while covering a meaningful space of symptoms and diagnoses. To the best of our knowledge, this paper is the first in studying the trade-off between the coverage of the model and its performance for diagnosis. To this end, we learn diagnosis models with different coverage from EHR data. We find a 1\% drop in top-3 accuracy for every 10 diseases added to the coverage. We also observe that complexity for these models does not affect performance, with linear models performing as well as neural networks.
Radar Human Motion Recognition Using Motion States and Two-Way Classifications
Authors: Moeness G. Amin, Ronny G. Guendel
Date: 2019-11-08 00:00:00
Tasks: General Classification, Translation, Vocal Bursts Valence Prediction

We perform classification of activities of daily living (ADL) using a Frequency-Modulated Continuous Waveform (FMCW) radar. In particular, we consider contiguous motions that are inseparable in time. Both the micro-Doppler signature and range-map are used to determine transitions from translation (walking) to in-place motions and vice versa, as well as to provide motion onset and the offset times. The possible classes of activities post and prior to the translation motion can be separately handled by forward and background classifiers. The paper describes ADL in terms of states and transitioning actions, and sets a framework to deal with separable and inseparable contiguous motions. It is shown that considering only the physically possible classes of motions stemming from the current motion state improves classification rates compared to incorporating all ADL for any given time.
A Robust Predictive Model for Stock Price Prediction Using Deep Learning and Natural Language Processing
Authors: Sidra Mehtab, Jaydip Sen
Date: 2019-12-09 00:00:00
Tasks: BIG-bench Machine Learning, Prediction, Sentiment Analysis, Stock Price Prediction

Prediction of future movement of stock prices has been a subject matter of many research work. There is a gamut of literature of technical analysis of stock prices where the objective is to identify patterns in stock price movements and derive profit from it. Improving the prediction accuracy remains the single most challenge in this area of research. We propose a hybrid approach for stock price movement prediction using machine learning, deep learning, and natural language processing. We select the NIFTY 50 index values of the National Stock Exchange of India, and collect its daily price movement over a period of three years (2015 to 2017). Based on the data of 2015 to 2017, we build various predictive models using machine learning, and then use those models to predict the closing value of NIFTY 50 for the period January 2018 till June 2019 with a prediction horizon of one week. For predicting the price movement patterns, we use a number of classification techniques, while for predicting the actual closing price of the stock, various regression models have been used. We also build a Long and Short-Term Memory - based deep learning network for predicting the closing price of the stocks and compare the prediction accuracies of the machine learning models with the LSTM model. We further augment the predictive model by integrating a sentiment analysis module on twitter data to correlate the public sentiment of stock prices with the market sentiment. This has been done using twitter sentiment and previous week closing values to predict stock price movement for the next week. We tested our proposed scheme using a cross validation method based on Self Organizing Fuzzy Neural Networks and found extremely interesting results.
Exploring Multi-Banking Customer-to-Customer Relations in AML Context with Poincaré Embeddings
Authors: Lucia Larise Stavarache, Donatas Narbutis, Toyotaro Suzumura, Ray Harishankar, Augustas Žaltauskas
Date: 2019-12-04 00:00:00

In the recent years money laundering schemes have grown in complexity and speed of realization, affecting financial institutions and millions of customers globally. Strengthened privacy policies, along with in-country regulations, make it hard for banks to inner- and cross-share, and report suspicious activities for the AML (Anti-Money Laundering) measures. Existing topologies and models for AML analysis and information sharing are subject to major limitations, such as compliance with regulatory constraints, extended infrastructure to run high-computation algorithms, data quality and span, proving cumbersome and costly to execute, federate, and interpret. This paper proposes a new topology for exploring multi-banking customer social relations in AML context -- customer-to-customer, customer-to-transaction, and transaction-to-transaction -- using a 3D modeling topological algebra formulated through Poincar\'e embeddings.
Towards A Robust Morphological Analyzer for Kunwinjku
Authors: William Lane, Steven Bird
Date: 2019-04-01 00:00:00
Proceeding: ALTA 2019 4

Kunwinjku is an indigenous Australian language spoken in northern Australia which exhibits agglutinative and polysynthetic properties. Members of the community have expressed interest in co-developing language applications that promote their values and priorities. Modeling the morphology of the Kunwinjku language is an important step towards accomplishing the community{'}s goals. Finite State Transducers have long been the go-to method for modeling morphologically rich languages, and in this paper we discuss some of the distinct modeling challenges present in the morphosyntax of verbs in Kunwinjku. We show that a fairly straightforward implementation using standard features of the foma toolkit can account for much of the verb structure. Continuing challenges include robustness in the face of variation and unseen vocabulary, as well as how to handle complex reduplicative processes. Our future work will build off the baseline and challenges presented here.
From Shakespeare to Li-Bai: Adapting a Sonnet Model to Chinese Poetry
Authors: Zhuohan Xie, Jey Han Lau, Trevor Cohn
Date: 2019-04-01 00:00:00
Proceeding: ALTA 2019 4

In this paper, we adapt Deep-speare, a joint neural network model for English sonnets, to Chinese poetry. We illustrate characteristics of Chinese quatrain and explain our architecture as well as training and generation procedure, which differs from Shakespeare sonnets in several aspects. We analyse the generated poetry and find that model works well for Chinese poetry, as it can: (1) generate coherent 4-line quatrains of different topics; and (2) capture rhyme automatically (to a certain extent).
Readability of Twitter Tweets for Second Language Learners
Authors: Patrick Jacob, Alex Uitdenbogerd, ra
Date: 2019-04-01 00:00:00
Proceeding: ALTA 2019 4
Tasks: Language Acquisition

Optimal language acquisition via reading requires the learners to read slightly above their current language skill level. Identifying material at the right level is the essential role of automatic readability measurement. Short message platforms such as Twitter offer the opportunity for language practice while reading about current topics and engaging in conversation in small doses, and can be filtered according to linguistic criteria to suit the learner. In this research, we explore how readable tweets are for English language learners and which factors contribute to their readability. With participants from six language groups, we collected 14,659 data points, each representing a tweet from a pool of 4100 tweets, and a judgement of perceived readability. Traditional readability measures and features failed on the data-set, but demographic data showed that judgements were largely genuine and reflected reported language skill, which is consistent with other recent studies. We report on the properties of the data set and implications for future research.
Red-faced ROUGE: Examining the Suitability of ROUGE for Opinion Summary Evaluation
Authors: Wenyi Tay, Aditya Joshi, Xiuzhen Zhang, Sarvnaz Karimi, Stephen Wan
Date: 2019-04-01 00:00:00
Proceeding: ALTA 2019 4

One of the most common metrics to automatically evaluate opinion summaries is ROUGE, a metric developed for text summarisation. ROUGE counts the overlap of word or word units between a candidate summary against reference summaries. This formulation treats all words in the reference summary equally.In opinion summaries, however, not all words in the reference are equally important. Opinion summarisation requires to correctly pair two types of semantic information: (1) aspect or opinion target; and (2) polarity of candidate and reference summaries. We investigate the suitability of ROUGE for evaluating opin-ion summaries of online reviews. Using three simulation-based experiments, we evaluate the behaviour of ROUGE for opinion summarisation on the ability to match aspect and polarity. We show that ROUGE cannot distinguish opinion summaries of similar or opposite polarities for the same aspect. Moreover,ROUGE scores have significant variance under different configuration settings. As a result, we present three recommendations for future work that uses ROUGE to evaluate opinion summarisation.
Does an LSTM forget more than a CNN? An empirical study of catastrophic forgetting in NLP
Authors: Gaurav Arora, Afshin Rahimi, Timothy Baldwin
Date: 2019-04-01 00:00:00
Proceeding: ALTA 2019 4
Tasks: Continual Learning, Transfer Learning

Catastrophic forgetting {---} whereby a model trained on one task is fine-tuned on a second, and in doing so, suffers a {``}catastrophic{''} drop in performance over the first task {---} is a hurdle in the development of better transfer learning techniques. Despite impressive progress in reducing catastrophic forgetting, we have limited understanding of how different architectures and hyper-parameters affect forgetting in a network. With this study, we aim to understand factors which cause forgetting during sequential training. Our primary finding is that CNNs forget less than LSTMs. We show that max-pooling is the underlying operation which helps CNNs alleviate forgetting compared to LSTMs. We also found that curriculum learning, placing a hard task towards the end of task sequence, reduces forgetting. We analysed the effect of fine-tuning contextual embeddings on catastrophic forgetting and found that using embeddings as feature extractor is preferable to fine-tuning in continual learning setup.
Detecting Chemical Reactions in Patents
Authors: Hiyori Yoshikawa, Dat Quoc Nguyen, Zenan Zhai, Christian Druckenbrodt, Camilo Thorne, Saber A. Akhondi, Timothy Baldwin, Karin Verspoor
Date: 2019-04-01 00:00:00
Proceeding: ALTA 2019 4

Extracting chemical reactions from patents is a crucial task for chemists working on chemical exploration. In this paper we introduce the novel task of detecting the textual spans that describe or refer to chemical reactions within patents. We formulate this task as a paragraph-level sequence tagging problem, where the system is required to return a sequence of paragraphs which contain a description of a reaction. To address this new task, we construct an annotated dataset from an existing proprietary database of chemical reactions manually extracted from patents. We introduce several baseline methods for the task and evaluate them over our dataset. Through error analysis, we discuss what makes the task complex and challenging, and suggest possible directions for future research.
Identifying Patients with Pain in Emergency Departments using Conventional Machine Learning and Deep Learning
Authors: Thanh Vu, Anthony Nguyen, Nathan Brown, James Hughes
Date: 2019-04-01 00:00:00
Proceeding: ALTA 2019 4
Tasks: BIG-bench Machine Learning, Management

Pain is the main symptom that patients present with to the emergency department (ED). Pain management, however, is often poorly done aspect of emergency care and patients with painful conditions can endure long waits before their pain is assessed or treated. To improve pain management quality, identifying whether or not an ED patient presents with pain is an important task and allows for further investigation of the quality of care provided. In this paper, machine learning was utilised to handle the task of automatically detecting patients who present at EDs with pain from retrospective data. Experimental results on a manually annotated dataset show that our proposed machine learning models achieve high performances, in which the highest accuracy and macro-averaged F1 are 91.00{\%} and 90.96{\%}, respectively.
CNL-ER: A Controlled Natural Language for Specifying and Verbalising Entity Relationship Models
Authors: Bayzid Ashik Hossain, Gayathri Rajan, Rolf Schwitter
Date: 2019-04-01 00:00:00
Proceeding: ALTA 2019 4

The first step towards designing an information system is conceptual modelling where domain experts and knowledge engineers identify the necessary information together to build an information system. Entity relationship modelling is one of the most popular conceptual modelling techniques that represents an information system in terms of entities, attributes and relationships. Entity relationship models are constructed graphically but are often difficult to understand by domain experts. To overcome this problem, we suggest to verbalise these models in a controlled natural language. In this paper, we present CNL-ER, a controlled natural language for specifying and verbalising entity relationship (ER) models that not only solves the verbalisation problem for these models but also provides the benefits of automatic verification and validation, and semantic round-tripping which makes the communication process transparent between the domain experts and the knowledge engineers.
Measuring English Readability for Vietnamese Speakers
Authors: Phuoc Nguyen, Alex Uitdenbogerd, ra
Date: 2019-04-01 00:00:00
Proceeding: ALTA 2019 4

Reading is important for any language learner, but the difficulty level of the text needs to match a reader{'}s level to enable efficient learning of new vocabulary. Many widely used traditional readability measures are not effective for those who speak English as a second or additional language. This study examines English readability for Vietnamese native speakers (VL1). A collection of text difficulty judgements of nearly 100 English text passages was obtained from 12 VL1 participants, using a 5-point Likert scale. Using the same basic features found in traditional English readability measures we found that SVMs and Dale-Chall features were slightly better than linear models using either Flesch or Dale-Chall. VL1 participants{'} text judgements were strongly correlated with their past IELTS test scores. This study introduces a first approximation to readability of English text for VL1, with suggestions for further improvements.
Does Multi-Task Learning Always Help?: An Evaluation on Health Informatics
Authors: Aditya Joshi, Sarvnaz Karimi, Ross Sparks, Cecile Paris, C Raina MacIntyre
Date: 2019-04-01 00:00:00
Proceeding: ALTA 2019 4
Tasks: Classification, General Classification, Multi-Task Learning, Relevance Detection

Multi-Task Learning (MTL) has been an attractive approach to deal with limited labeled datasets or leverage related tasks, for a variety of NLP problems. We examine the benefit of MTL for three specific pairs of health informatics tasks that deal with: (a) overlapping symptoms for the same classification problem (personal health mention classification for influenza and for a set of symptoms); (b) overlapping medical concepts for related classification problems (vaccine usage and drug usage detection); and, (c) related classification problems (vaccination intent and vaccination relevance detection). We experiment with a simple neural architecture: a shared layer followed by task-specific dense layers. The novelty of this work is that it compares alternatives for shared layers for these pairs of tasks. While our observations agree with the promise of MTL as compared to single-task learning, for health informatics, we show that the benefit also comes with caveats in terms of the choice of shared layers and the relatedness between the participating tasks.