pwc-archive/papers-with-abstracts huggingface dataset:
Unsupervised Online Feature Selection for Cost-Sensitive Medical Diagnosis
Authors: Arun Verma, Manjesh K. Hanawal, Nandyala Hemachandra
Date: 2019-12-25 00:00:00
Tasks: feature selection, Medical Diagnosis

In medical diagnosis, physicians predict the state of a patient by checking measurements (features) obtained from a sequence of tests, e.g., blood test, urine test, followed by invasive tests. As tests are often costly, one would like to obtain only those features (tests) that can establish the presence or absence of the state conclusively. Another aspect of medical diagnosis is that we are often faced with unsupervised prediction tasks as the true state of the patients may not be known. Motivated by such medical diagnosis problems, we consider a {\it Cost-Sensitive Medical Diagnosis} (CSMD) problem, where the true state of patients is unknown. We formulate the CSMD problem as a feature selection problem where each test gives a feature that can be used in a prediction model. Our objective is to learn strategies for selecting the features that give the best trade-off between accuracy and costs. We exploit the `Weak Dominance' property of problem to develop online algorithms that identify a set of features which provides an `optimal' trade-off between cost and accuracy of prediction without requiring to know the true state of the medical condition. Our empirical results validate the performance of our algorithms on problem instances generated from real-world datasets.
Artificial Intelligence in Surgery
Authors: Xiao-Yun Zhou, Yao Guo, Mali Shen, Guang-Zhong Yang
Date: 2019-12-23 00:00:00

Artificial Intelligence (AI) is gradually changing the practice of surgery with the advanced technological development of imaging, navigation and robotic intervention. In this article, the recent successful and influential applications of AI in surgery are reviewed from pre-operative planning and intra-operative guidance to the integration of surgical robots. We end with summarizing the current state, emerging trends and major challenges in the future development of AI in surgery.
Convolutional Neural Network-based Topology Optimization (CNN-TO) By Estimating Sensitivity of Compliance from Material Distribution
Authors: Yusuke Takahashi, Yoshiro Suzuki, Akira Todoroki
Date: 2019-12-23 00:00:00
Tasks: Sensitivity

This paper proposes a new topology optimization method that applies a convolutional neural network (CNN), which is one deep learning technique for topology optimization problems. Using this method, we acquire a structure with a little higher performance that could not be obtained by the previous topology optimization method. In particular, in this paper, we solve a topology optimization problem aimed at maximizing stiffness with a mass constraint, which is a common type of topology optimization. In this paper, we first formulate the conventional topology optimization by the solid isotropic material with penalization method. Next, we formulate the topology optimization using CNN. Finally, we show the effectiveness of the proposed topology optimization method by solving a verification example, namely a topology optimization problem aimed at maximizing stiffness. In this research, as a result of solving the verification example for a small design area of 16x32 element, we obtain the solution different from the previous topology optimization method. This result suggests that stiffness information of structure can be extracted and analyzed for structural design by analyzing the density distribution using CNN like an image. This suggests that CNN technology can be utilized in the structural design and topology optimization.
BehavDT: A Behavioral Decision Tree Learning to Build User-Centric Context-Aware Predictive Model
Authors: Iqbal H. Sarker, Alan Colman, Jun Han, Asif Irshad Khan, Yoosef B. Abushark, Khaled Salah
Date: 2019-12-17 00:00:00
Tasks: BIG-bench Machine Learning

This paper formulates the problem of building a context-aware predictive model based on user diverse behavioral activities with smartphones. In the area of machine learning and data science, a tree-like model as that of decision tree is considered as one of the most popular classification techniques, which can be used to build a data-driven predictive model. The traditional decision tree model typically creates a number of leaf nodes as decision nodes that represent context-specific rigid decisions, and consequently may cause overfitting problem in behavior modeling. However, in many practical scenarios within the context-aware environment, the generalized outcomes could play an important role to effectively capture user behavior. In this paper, we propose a behavioral decision tree, "BehavDT" context-aware model that takes into account user behavior-oriented generalization according to individual preference level. The BehavDT model outputs not only the generalized decisions but also the context-specific decisions in relevant exceptional cases. The effectiveness of our BehavDT model is studied by conducting experiments on individual user real smartphone datasets. Our experimental results show that the proposed BehavDT context-aware model is more effective when compared with the traditional machine learning approaches, in predicting user diverse behaviors considering multi-dimensional contexts.
Multi-Gradient Descent for Multi-Objective Recommender Systems
Authors: Nikola Milojkovic, Diego Antognini, Giancarlo Bergamin, Boi Faltings, Claudiu Musat
Date: 2019-12-09 00:00:00
Tasks: Recommendation Systems

Recommender systems need to mirror the complexity of the environment they are applied in. The more we know about what might benefit the user, the more objectives the recommender system has. In addition there may be multiple stakeholders - sellers, buyers, shareholders - in addition to legal and ethical constraints. Simultaneously optimizing for a multitude of objectives, correlated and not correlated, having the same scale or not, has proven difficult so far. We introduce a stochastic multi-gradient descent approach to recommender systems (MGDRec) to solve this problem. We show that this exceeds state-of-the-art methods in traditional objective mixtures, like revenue and recall. Not only that, but through gradient normalization we can combine fundamentally different objectives, having diverse scales, into a single coherent framework. We show that uncorrelated objectives, like the proportion of quality products, can be improved alongside accuracy. Through the use of stochasticity, we avoid the pitfalls of calculating full gradients and provide a clear setting for its applicability.
Estimation of the yield curve for Costa Rica using combinatorial optimization metaheuristics applied to nonlinear regression
Authors: Andres Quiros-Granados, Javier Trejos-Zelaya
Date: 2019-11-20 00:00:00
Tasks: Combinatorial Optimization, regression

The term structure of interest rates or yield curve is a function relating the interest rate with its own term. Nonlinear regression models of Nelson-Siegel and Svensson were used to estimate the yield curve using a sample of historical data supplied by the National Stock Exchange of Costa Rica. The optimization problem involved in the estimation process of model parameters is addressed by the use of four well known combinatorial optimization metaheuristics: Ant colony optimization, Genetic algorithm, Particle swarm optimization and Simulated annealing. The aim of the study is to improve the local minima obtained by a classical quasi-Newton optimization method using a descent direction. Good results with at least two metaheuristics are achieved, Particle swarm optimization and Simulated annealing. Keywords: Yield curve, nonlinear regression, Nelson-
Analytic Continued Fractions for Regression: A Memetic Algorithm Approach
Authors: Pablo Moscato, Haoyuan Sun, Mohammad Nazmul Haque
Date: 2019-12-18 00:00:00
Tasks: BIG-bench Machine Learning, regression, Symbolic Regression

We present an approach for regression problems that employs analytic continued fractions as a novel representation. Comparative computational results using a memetic algorithm are reported in this work. Our experiments included fifteen other different machine learning approaches including five genetic programming methods for symbolic regression and ten machine learning methods. The comparison on training and test generalization was performed using 94 datasets of the Penn State Machine Learning Benchmark. The statistical tests showed that the generalization results using analytic continued fractions provides a powerful and interesting new alternative in the quest for compact and interpretable mathematical models for artificial intelligence.
Fairness in Multi-agent Reinforcement Learning for Stock Trading
Authors: Wenhang Bao
Date: 2019-12-14 00:00:00
Tasks: Fairness, Multi-agent Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Unfair stock trading strategies have been shown to be one of the most negative perceptions that customers can have concerning trading and may result in long-term losses for a company. Investment banks usually place trading orders for multiple clients with the same target assets but different order sizes and diverse requirements such as time frame and risk aversion level, thereby total earning and individual earning cannot be optimized at the same time. Orders executed earlier would affect the market price level, so late execution usually means additional implementation cost. In this paper, we propose a novel scheme that utilizes multi-agent reinforcement learning systems to derive stock trading strategies for all clients which keep a balance between revenue and fairness. First, we demonstrate that Reinforcement learning (RL) is able to learn from experience and adapt the trading strategies to the complex market environment. Secondly, we show that the Multi-agent RL system allows developing trading strategies for all clients individually, thus optimizing individual revenue. Thirdly, we use the Generalized Gini Index (GGI) aggregation function to control the fairness level of the revenue across all clients. Lastly, we empirically demonstrate the superiority of the novel scheme in improving fairness meanwhile maintaining optimization of revenue.
Experiments with Game Tree Search in Real-Time Strategy Games
Authors: Santiago Ontanon
Date: 2012-08-09 00:00:00
Tasks: Real-Time Strategy Games

Game tree search algorithms such as minimax have been used with enormous success in turn-based adversarial games such as Chess or Checkers. However, such algorithms cannot be directly applied to real-time strategy (RTS) games because a number of reasons. For example, minimax assumes a turn-taking game mechanics, not present in RTS games. In this paper we present RTMM, a real-time variant of the standard minimax algorithm, and discuss its applicability in the context of RTS games. We discuss its strengths and weaknesses, and evaluate it in two real-time games.
Multiple People Tracking using Body and Joint Detections
Authors: Roberto Henschel, Yunzhe Zou, Bodo Rosenhahn
Date: 2019-06-01 00:00:00
Proceeding: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops 2019 6
Tasks: Multi-Object Tracking, Multiple People Tracking

Most multiple people tracking systems compute trajectories based on the tracking-by-detection paradigm. Consequently, the performance depends to a large extent on the quality of the employed input detections. However, despite an enormous progress in recent years, partially occluded people are still often not recognized. Also, many correct detections are mistakenly discarded when the non-maximum suppression is performed. Improving the tracking performance thus requires to augment the coarse input. Wellsuited for this task are fine-graded body joint detections, as they allow to locate even strongly occluded persons.
Thus in this work, we analyze the suitability of including joint detections for multiple people tracking. We introduce different affinities between the two detection types and evaluate their performances. Tracking is then performed within a near-online framework based on a min cost graph labeling formulation. As a result, our framework can recover heavily occluded persons and solve the data association efficiently. We evaluate our framework on the MOT16/17 benchmark. Experimental results demonstrate that our framework achieves state-of-the-art results.
What does the free energy principle tell us about the brain?
Authors: Samuel J. Gershman
Date: 2019-01-23 00:00:00
Tasks: Active Learning, Bayesian Inference

The free energy principle has been proposed as a unifying account of brain function. It is closely related, and in some cases subsumes, earlier unifying ideas such as Bayesian inference, predictive coding, and active learning. This article clarifies these connections, teasing apart distinctive and shared predictions.
Adapting Deep Learning for Sentiment Classification of Code-Switched Informal Short Text
Authors: Muhammad Haroon Shakeel, Asim Karim
Date: 2020-01-04 00:00:00
Tasks: Classification, General Classification, Lexical Normalization, Sentiment Analysis, Sentiment Classification, Translation

Nowadays, an abundance of short text is being generated that uses nonstandard writing styles influenced by regional languages. Such informal and code-switched content are under-resourced in terms of labeled datasets and language models even for popular tasks like sentiment classification. In this work, we (1) present a labeled dataset called MultiSenti for sentiment classification of code-switched informal short text, (2) explore the feasibility of adapting resources from a resource-rich language for an informal one, and (3) propose a deep learning-based model for sentiment classification of code-switched informal short text. We aim to achieve this without any lexical normalization, language translation, or code-switching indication. The performance of the proposed models is compared with three existing multilingual sentiment classification models. The results show that the proposed model performs better in general and adapting character-based embeddings yield equivalent performance while being computationally more efficient than training word-based domain-specific embeddings.
A Comprehensive Survey of Multilingual Neural Machine Translation
Authors: Raj Dabre, Chenhui Chu, Anoop Kunchukuttan
Date: 2020-01-04 00:00:00
Tasks: Machine Translation, NMT, Survey, Transfer Learning, Translation

We present a survey on multilingual neural machine translation (MNMT), which has gained a lot of traction in the recent years. MNMT has been useful in improving translation quality as a result of translation knowledge transfer (transfer learning). MNMT is more promising and interesting than its statistical machine translation counterpart because end-to-end modeling and distributed representations open new avenues for research on machine translation. Many approaches have been proposed in order to exploit multilingual parallel corpora for improving translation quality. However, the lack of a comprehensive survey makes it difficult to determine which approaches are promising and hence deserve further exploration. In this paper, we present an in-depth survey of existing literature on MNMT. We first categorize various approaches based on their central use-case and then further categorize them based on resource scenarios, underlying modeling principles, core-issues and challenges. Wherever possible we address the strengths and weaknesses of several techniques by comparing them with each other. We also discuss the future directions that MNMT research might take. This paper is aimed towards both, beginners and experts in NMT. We hope this paper will serve as a starting point as well as a source of new ideas for researchers and engineers interested in MNMT.
Transformer-based language modeling and decoding for conversational speech recognition
Authors: Kareem Nassar
Date: 2020-01-04 00:00:00
Tasks: Language Modeling, Language Modelling, speech-recognition, Speech Recognition

We propose a way to use a transformer-based language model in conversational speech recognition. Specifically, we focus on decoding efficiently in a weighted finite-state transducer framework. We showcase an approach to lattice re-scoring that allows for longer range history captured by a transfomer-based language model and takes advantage of a transformer's ability to avoid computing sequentially.
Computationally Efficient NER Taggers with Combined Embeddings and Constrained Decoding
Authors: Brian Lester, Daniel Pressel, Amy Hemmeter, Sagnik Ray Choudhury
Date: 2020-01-05 00:00:00
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER, TAG

Current State-of-the-Art models in Named Entity Recognition (NER) are neural models with a Conditional Random Field (CRF) as the final network layer, and pre-trained "contextual embeddings". The CRF layer is used to facilitate global coherence between labels, and the contextual embeddings provide a better representation of words in context. However, both of these improvements come at a high computational cost. In this work, we explore two simple techniques that substantially improve NER performance over a strong baseline with negligible cost. First, we use multiple pre-trained embeddings as word representations via concatenation. Second, we constrain the tagger, trained using a cross-entropy loss, during decoding to eliminate illegal transitions. While training a tagger on CoNLL 2003 we find a $786$\% speed-up over a contextual embeddings-based tagger without sacrificing strong performance. We also show that the concatenation technique works across multiple tasks and datasets. We analyze aspects of similarity and coverage between pre-trained embeddings and the dynamics of tag co-occurrence to explain why these techniques work. We provide an open source implementation of our tagger using these techniques in three popular deep learning frameworks --- TensorFlow, Pytorch, and DyNet.
Generating Word and Document Embeddings for Sentiment Analysis
Authors: Cem Rıfkı Aydın, Tunga Güngör, Ali Erkan
Date: 2020-01-05 00:00:00
Tasks: Sentiment Analysis

Sentiments of words differ from one corpus to another. Inducing general sentiment lexicons for languages and using them cannot, in general, produce meaningful results for different domains. In this paper, we combine contextual and supervised information with the general semantic representations of words occurring in the dictionary. Contexts of words help us capture the domain-specific information and supervised scores of words are indicative of the polarities of those words. When we combine supervised features of words with the features extracted from their dictionary definitions, we observe an increase in the success rates. We try out the combinations of contextual, supervised, and dictionary-based approaches, and generate original vectors. We also combine the word2vec approach with hand-crafted features. We induce domain-specific sentimental vectors for two corpora, which are the movie domain and the Twitter datasets in Turkish. When we thereafter generate document vectors and employ the support vector machines method utilising those vectors, our approaches perform better than the baseline studies for Turkish with a significant margin. We evaluated our models on two English corpora as well and these also outperformed the word2vec approach. It shows that our approaches are cross-domain and portable to other languages.
Improving Entity Linking by Modeling Latent Entity Type Information
Authors: Shuang Chen, Jinpeng Wang, Feng Jiang, Chin-Yew Lin
Date: 2020-01-06 00:00:00
Tasks: Entity Disambiguation, Entity Embeddings, Entity Linking, Vocal Bursts Type Prediction, Word Embeddings

Existing state of the art neural entity linking models employ attention-based bag-of-words context model and pre-trained entity embeddings bootstrapped from word embeddings to assess topic level context compatibility. However, the latent entity type information in the immediate context of the mention is neglected, which causes the models often link mentions to incorrect entities with incorrect type. To tackle this problem, we propose to inject latent entity type information into the entity embeddings based on pre-trained BERT. In addition, we integrate a BERT-based entity similarity score into the local context model of a state-of-the-art model to better capture latent entity type information. Our model significantly outperforms the state-of-the-art entity linking models on standard benchmark (AIDA-CoNLL). Detailed experiment analysis demonstrates that our model corrects most of the type errors produced by the direct baseline.
Speaker-aware speech-transformer
Authors: Zhiyun Fan, Jie Li, Shiyu Zhou, Bo Xu
Date: 2020-01-02 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), speech-recognition, Speech Recognition

Recently, end-to-end (E2E) models become a competitive alternative to the conventional hybrid automatic speech recognition (ASR) systems. However, they still suffer from speaker mismatch in training and testing condition. In this paper, we use Speech-Transformer (ST) as the study platform to investigate speaker aware training of E2E models. We propose a model called Speaker-Aware Speech-Transformer (SAST), which is a standard ST equipped with a speaker attention module (SAM). The SAM has a static speaker knowledge block (SKB) that is made of i-vectors. At each time step, the encoder output attends to the i-vectors in the block, and generates a weighted combined speaker embedding vector, which helps the model to normalize the speaker variations. The SAST model trained in this way becomes independent of specific training speakers and thus generalizes better to unseen testing speakers. We investigate different factors of SAM. Experimental results on the AISHELL-1 task show that SAST achieves a relative 6.5% CER reduction (CERR) over the speaker-independent (SI) baseline. Moreover, we demonstrate that SAST still works quite well even if the i-vectors in SKB all come from a different data source other than the acoustic training set.
Stance Detection Benchmark: How Robust Is Your Stance Detection?
Authors: Benjamin Schiller, Johannes Daxenberger, Iryna Gurevych
Date: 2020-01-06 00:00:00
Tasks: Fake News Detection, Multi-Task Learning, Stance Detection, Transfer Learning

Stance Detection (StD) aims to detect an author's stance towards a certain topic or claim and has become a key component in applications like fake news detection, claim validation, and argument search. However, while stance is easily detected by humans, machine learning models are clearly falling short of this task. Given the major differences in dataset sizes and framing of StD (e.g. number of classes and inputs), we introduce a StD benchmark that learns from ten StD datasets of various domains in a multi-dataset learning (MDL) setting, as well as from related tasks via transfer learning. Within this benchmark setup, we are able to present new state-of-the-art results on five of the datasets. Yet, the models still perform well below human capabilities and even simple adversarial attacks severely hurt the performance of MDL models. Deeper investigation into this phenomenon suggests the existence of biases inherited from multiple datasets by design. Our analysis emphasizes the need of focus on robustness and de-biasing strategies in multi-task learning approaches. The benchmark dataset and code is made available.