pwc-archive/papers-with-abstracts huggingface dataset:
Whole MILC: generalizing learned dynamics across tasks, datasets, and populations
Authors: Usman Mahmood, Md Mahfuzur Rahman, Alex Fedorov, Noah Lewis, Zening Fu, Vince D. Calhoun, Sergey M. Plis
Date: 2020-07-29 00:00:00
Tasks: Feature Importance

Behavioral changes are the earliest signs of a mental disorder, but arguably, the dynamics of brain function gets affected even earlier. Subsequently, spatio-temporal structure of disorder-specific dynamics is crucial for early diagnosis and understanding the disorder mechanism. A common way of learning discriminatory features relies on training a classifier and evaluating feature importance. Classical classifiers, based on handcrafted features are quite powerful, but suffer the curse of dimensionality when applied to large input dimensions of spatio-temporal data. Deep learning algorithms could handle the problem and a model introspection could highlight discriminatory spatio-temporal regions but need way more samples to train. In this paper we present a novel self supervised training schema which reinforces whole sequence mutual information local to context (whole MILC). We pre-train the whole MILC model on unlabeled and unrelated healthy control data. We test our model on three different disorders (i) Schizophrenia (ii) Autism and (iii) Alzheimers and four different studies. Our algorithm outperforms existing self-supervised pre-training methods and provides competitive classification results to classical machine learning algorithms. Importantly, whole MILC enables attribution of subject diagnosis to specific spatio-temporal regions in the fMRI signal.
Low Dimensional State Representation Learning with Reward-shaped Priors
Authors: Nicolò Botteghi, Ruben Obbink, Daan Geijs, Mannes Poel, Beril Sirmacek, Christoph Brune, Abeje Mersha, Stefano Stramigioli
Date: 2020-07-29 00:00:00
Tasks: Feature Engineering, Representation Learning, Robot Navigation

Reinforcement Learning has been able to solve many complicated robotics tasks without any need for feature engineering in an end-to-end fashion. However, learning the optimal policy directly from the sensory inputs, i.e the observations, often requires processing and storage of a huge amount of data. In the context of robotics, the cost of data from real robotics hardware is usually very high, thus solutions that achieve high sample-efficiency are needed. We propose a method that aims at learning a mapping from the observations into a lower-dimensional state space. This mapping is learned with unsupervised learning using loss functions shaped to incorporate prior knowledge of the environment and the task. Using the samples from the state space, the optimal policy is quickly and efficiently learned. We test the method on several mobile robot navigation tasks in a simulation environment and also on a real robot.
Ensuring the Robustness and Reliability of Data-Driven Knowledge Discovery Models in Production and Manufacturing
Authors: Shailesh Tripathi, David Muhr, Brunner Manuel, Frank Emmert-Streib, Herbert Jodlbauer, Matthias Dehmer
Date: 2020-07-28 00:00:00
Tasks: BIG-bench Machine Learning

The implementation of robust, stable, and user-centered data analytics and machine learning models is confronted by numerous challenges in production and manufacturing. Therefore, a systematic approach is required to develop, evaluate, and deploy such models. The data-driven knowledge discovery framework provides an orderly partition of the data-mining processes to ensure the practical implementation of data analytics and machine learning models. However, the practical application of robust industry-specific data-driven knowledge discovery models faces multiple data-- and model-development--related issues. These issues should be carefully addressed by allowing a flexible, customized, and industry-specific knowledge discovery framework; in our case, this takes the form of the cross-industry standard process for data mining (CRISP-DM). This framework is designed to ensure active cooperation between different phases to adequately address data- and model-related issues. In this paper, we review several extensions of CRISP-DM models and various data-robustness-- and model-robustness--related problems in machine learning, which currently lacks proper cooperation between data experts and business experts because of the limitations of data-driven knowledge discovery models.
Hierarchical BiGraph Neural Network as Recommendation Systems
Authors: Dom Huh
Date: 2020-07-27 00:00:00
Tasks: Recommendation Systems

Graph neural networks emerge as a promising modeling method for applications dealing with datasets that are best represented in the graph domain. In specific, developing recommendation systems often require addressing sparse structured data which often lacks the feature richness in either the user and/or item side and requires processing within the correct context for optimal performance. These datasets intuitively can be mapped to and represented as networks or graphs. In this paper, we propose the Hierarchical BiGraph Neural Network (HBGNN), a hierarchical approach of using GNNs as recommendation systems and structuring the user-item features using a bigraph framework. Our experimental results show competitive performance with current recommendation system methods and transferability.
Greedy Bandits with Sampled Context
Authors: Dom Huh
Date: 2020-07-27 00:00:00
Tasks: Decision Making, Multi-Armed Bandits, reinforcement-learning, Reinforcement Learning (RL), Thompson Sampling

Bayesian strategies for contextual bandits have proved promising in single-state reinforcement learning tasks by modeling uncertainty using context information from the environment. In this paper, we propose Greedy Bandits with Sampled Context (GB-SC), a method for contextual multi-armed bandits to develop the prior from the context information using Thompson Sampling, and arm selection using an epsilon-greedy policy. The framework GB-SC allows for evaluation of context-reward dependency, as well as providing robustness for partially observable context vectors by leveraging the prior developed. Our experimental results show competitive performance on the Mushroom environment in terms of expected regret and expected cumulative regret, as well as insights on how each context subset affects decision-making.
Graph Convolutional Networks using Heat Kernel for Semi-supervised Learning
Authors: Bingbing Xu, Hua-Wei Shen, Qi Cao, Keting Cen, Xue-Qi Cheng
Date: 2020-07-27 00:00:00

Graph convolutional networks gain remarkable success in semi-supervised learning on graph structured data. The key to graph-based semisupervised learning is capturing the smoothness of labels or features over nodes exerted by graph structure. Previous methods, spectral methods and spatial methods, devote to defining graph convolution as a weighted average over neighboring nodes, and then learn graph convolution kernels to leverage the smoothness to improve the performance of graph-based semi-supervised learning. One open challenge is how to determine appropriate neighborhood that reflects relevant information of smoothness manifested in graph structure. In this paper, we propose GraphHeat, leveraging heat kernel to enhance low-frequency filters and enforce smoothness in the signal variation on the graph. GraphHeat leverages the local structure of target node under heat diffusion to determine its neighboring nodes flexibly, without the constraint of order suffered by previous methods. GraphHeat achieves state-of-the-art results in the task of graph-based semi-supervised classification across three benchmark datasets: Cora, Citeseer and Pubmed.
Exclusion and Inclusion -- A model agnostic approach to feature importance in DNNs
Authors: Subhadip Maji, Arijit Ghosh Chowdhury, Raghav Bali, Vamsi M Bhandaru
Date: 2020-07-13 00:00:00
Tasks: Feature Importance, regression

Deep Neural Networks in NLP have enabled systems to learn complex non-linear relationships. One of the major bottlenecks towards being able to use DNNs for real world applications is their characterization as black boxes. To solve this problem, we introduce a model agnostic algorithm which calculates phrase-wise importance of input features. We contend that our method is generalizable to a diverse set of tasks, by carrying out experiments for both Regression and Classification. We also observe that our approach is robust to outliers, implying that it only captures the essential aspects of the input.
BERT Learns (and Teaches) Chemistry
Authors: Josh Payne, Mario Srouji, Dian Ang Yap, Vineet Kosaraju
Date: 2020-07-11 00:00:00
Tasks: Drug Discovery

Modern computational organic chemistry is becoming increasingly data-driven. There remain a large number of important unsolved problems in this area such as product prediction given reactants, drug discovery, and metric-optimized molecule synthesis, but efforts to solve these problems using machine learning have also increased in recent years. In this work, we propose the use of attention to study functional groups and other property-impacting molecular substructures from a data-driven perspective, using a transformer-based model (BERT) on datasets of string representations of molecules and analyzing the behavior of its attention heads. We then apply the representations of functional groups and atoms learned by the model to tackle problems of toxicity, solubility, drug-likeness, and synthesis accessibility on smaller datasets using the learned representations as features for graph convolution and attention models on the graph structure of molecules, as well as fine-tuning of BERT. Finally, we propose the use of attention visualization as a helpful tool for chemistry practitioners and students to quickly identify important substructures in various chemical properties.
Neural Composition: Learning to Generate from Multiple Models
Authors: Denis Filimonov, Ravi Teja Gadde, Ariya Rastrow
Date: 2020-07-10 00:00:00
Tasks: Language Modeling, Language Modelling

Decomposing models into multiple components is critically important in many applications such as language modeling (LM) as it enables adapting individual components separately and biasing of some components to the user's personal preferences. Conventionally, contextual and personalized adaptation for language models, are achieved through class-based factorization, which requires class-annotated data, or through biasing to individual phrases which is limited in scale. In this paper, we propose a system that combines model-defined components, by learning when to activate the generation process from each individual component, and how to combine probability distributions from each component, directly from unlabeled text data.
Reciprocal Recommender Systems: Analysis of State-of-Art Literature, Challenges and Opportunities towards Social Recommendation
Authors: Ivan Palomares, Carlos Porcel, Luiz Pizzato, Ido Guy, Enrique Herrera-Viedma
Date: 2020-07-17 00:00:00
Tasks: Decision Making, Recommendation Systems

There exist situations of decision-making under information overload in the Internet, where people have an overwhelming number of available options to choose from, e.g. products to buy in an e-commerce site, or restaurants to visit in a large city. Recommender systems arose as a data-driven personalized decision support tool to assist users in these situations: they are able to process user-related data, filtering and recommending items based on the users preferences, needs and/or behaviour. Unlike most conventional recommender approaches where items are inanimate entities recommended to the users and success is solely determined upon the end users reaction to the recommendation(s) received, in a Reciprocal Recommender System (RRS) users become the item being recommended to other users. Hence, both the end user and the user being recommended should accept the 'matching' recommendation to yield a successful RRS performance. The operation of an RRS entails not only predicting accurate preference estimates upon user interaction data as classical recommenders do, but also calculating mutual compatibility between (pairs of) users, typically by applying fusion processes on unilateral user-to-user preference information. This paper presents a snapshot-style analysis of the extant literature that summarizes the state-of-the-art RRS research to date, focusing on the algorithms, fusion processes and fundamental characteristics of RRS, both inherited from conventional user-to-item recommendation models and those inherent to this emerging family of approaches. Representative RRS models are likewise highlighted. Following this, we discuss the challenges and opportunities for future research on RRSs, with special focus on (i) fusion strategies to account for reciprocity and (ii) emerging application domains related to social recommendation.
Exploring Swedish & English fastText Embeddings for NER with the Transformer
Authors: Tosin P. Adewumi, Foteini Liwicki, Marcus Liwicki
Date: 2020-07-23 00:00:00
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER

In this paper, our main contributions are that embeddings from relatively smaller corpora can outperform ones from larger corpora and we make the new Swedish analogy test set publicly available. To achieve a good network performance in natural language processing (NLP) downstream tasks, several factors play important roles: dataset size, the right hyper-parameters, and well-trained embeddings. We show that, with the right set of hyper-parameters, good network performance can be reached even on smaller datasets. We evaluate the embeddings at both the intrinsic and extrinsic levels. The embeddings are deployed with the Transformer in named entity recognition (NER) task and significance tests conducted. This is done for both Swedish and English. We obtain better performance in both languages on the downstream task with smaller training data, compared to recently released, Common Crawl versions; and character n-grams appear useful for Swedish, a morphologically rich language.
Multi-task learning for natural language processing in the 2020s: where are we going?
Authors: Joseph Worsham, Jugal Kalita
Date: 2020-07-22 00:00:00
Tasks: Multi-Task Learning, Transfer Learning

Multi-task learning (MTL) significantly pre-dates the deep learning era, and it has seen a resurgence in the past few years as researchers have been applying MTL to deep learning solutions for natural language tasks. While steady MTL research has always been present, there is a growing interest driven by the impressive successes published in the related fields of transfer learning and pre-training, such as BERT, and the release of new challenge problems, such as GLUE and the NLP Decathlon (decaNLP). These efforts place more focus on how weights are shared across networks, evaluate the re-usability of network components and identify use cases where MTL can significantly outperform single-task solutions. This paper strives to provide a comprehensive survey of the numerous recent MTL contributions to the field of natural language processing and provide a forum to focus efforts on the hardest unsolved problems in the next decade. While novel models that improve performance on NLP benchmarks are continually produced, lasting MTL challenges remain unsolved which could hold the key to better language understanding, knowledge discovery and natural language interfaces.
Neural Machine Translation model for University Email Application
Authors: Sandhya Aneja, Siti Nur Afikah Bte Abdul Mazid, Nagender Aneja
Date: 2020-07-20 00:00:00
Tasks: Decoder, Machine Translation, NMT, Translation

Machine translation has many applications such as news translation, email translation, official letter translation etc. Commercial translators, e.g. Google Translation lags in regional vocabulary and are unable to learn the bilingual text in the source and target languages within the input. In this paper, a regional vocabulary-based application-oriented Neural Machine Translation (NMT) model is proposed over the data set of emails used at the University for communication over a period of three years. A state-of-the-art Sequence-to-Sequence Neural Network for ML -> EN and EN -> ML translations is compared with Google Translate using Gated Recurrent Unit Recurrent Neural Network machine translation model with attention decoder. The low BLEU score of Google Translation in comparison to our model indicates that the application based regional models are better. The low BLEU score of EN -> ML of our model and Google Translation indicates that the Malay Language has complex language features corresponding to English.
Word Embeddings: Stability and Semantic Change
Authors: Lucas Rettenmeier
Date: 2020-07-23 00:00:00
Tasks: Word Embeddings

Word embeddings are computed by a class of techniques within natural language processing (NLP), that create continuous vector representations of words in a language from a large text corpus. The stochastic nature of the training process of most embedding techniques can lead to surprisingly strong instability, i.e. subsequently applying the same technique to the same data twice, can produce entirely different results. In this work, we present an experimental study on the instability of the training process of three of the most influential embedding techniques of the last decade: word2vec, GloVe and fastText. Based on the experimental results, we propose a statistical model to describe the instability of embedding techniques and introduce a novel metric to measure the instability of the representation of an individual word. Finally, we propose a method to minimize the instability - by computing a modified average over multiple runs - and apply it to a specific linguistic problem: The detection and quantification of semantic change, i.e. measuring changes in the meaning and usage of words over time.
Toward Givenness Hierarchy Theoretic Natural Language Generation
Authors: Poulomi Pal, Tom Williams
Date: 2020-07-17 00:00:00
Tasks: Text Generation

Language-capable interactive robots participating in dialogues with human interlocutors must be able to naturally and efficiently communicate about the entities in their environment. A key aspect of such communication is the use of anaphoric language. The linguistic theory of the Givenness Hierarchy(GH) suggests that humans use anaphora based on the cognitive statuses their referents have in the minds of their interlocutors. In previous work, researchers presented GH-theoretic approaches to robot anaphora understanding. In this paper we describe how the GH might need to be used quite differently to facilitate robot anaphora generation.
Toward Campus Mail Delivery Using BDI
Authors: Chidiebere Onyedinma, Patrick Gavigan, Babak Esfandiari
Date: 2020-07-23 00:00:00
Tasks: Decision Making, Navigate

Autonomous systems developed with the Belief-Desire-Intention (BDI) architecture are usually mostly implemented in simulated environments. In this project we sought to build a BDI agent for use in the real world for campus mail delivery in the tunnel system at Carleton University. Ideally, the robot should receive a delivery order via a mobile application, pick up the mail at a station, navigate the tunnels to the destination station, and notify the recipient. We linked the Robot Operating System (ROS) with a BDI reasoning system to achieve a subset of the required use cases. ROS handles the low-level sensing and actuation, while the BDI reasoning system handles the high-level reasoning and decision making. Sensory data is orchestrated and sent from ROS to the reasoning system as perceptions. These perceptions are then deliberated upon, and an action string is sent back to ROS for interpretation and driving of the necessary actuator for the action to be performed. In this paper we present our current implementation, which closes the loop on the hardware-software integration, and implements a subset of the use cases required for the full system.
ResNet50_on_Cifar_100_Without_Transfer_Learning
Authors: Batuhan Bayraktar
Date: 2020-08-03 00:00:00
Proceeding: Sorry, no such thing happened 2020 8
Tasks: General Classification, image-classification, Image Classification

I am a young person who is curious about image classification. 
I have an average knowledge and an average computer.
I thought the cifar 100 data for image classification was challenging for me.
It was a data containing 100 classes from 32x32 and 100 images.
I chose resnet as the model due to the low number of data and gradient vanishing problem.
I worked with google colab because my computer is not enough (thank you Google)
Since I used the free version, I could only run Resnet50.
I tried to stick to the original Resnet article. But I made it myself in changes.
I have tried many hyper parameters. 
I found the parameters that gave the best results as soon as possible as fast as I could.
I'm a young man who likes to push his luck. that is all
DIRECT POSITION DETERMINATION OF MULTIPLE RADIO SIGNALS
Authors: Alon Amar, Anthony J. Weiss
Date: 2004-05-17 00:00:00
Proceeding: conferrence 2004 5
Tasks: Position

The most common methods for localization of radio signal 
emitters are based on measuring a specified parameter 
such as signal Angle-of-Arrival (AOA) or Time-of-
Arrival (TOA). The measured parameters are then used to 
estimate the transmitter location. Since the measurements 
are done at each base station independently, without using 
the constraint that all AOA/TOA estimates must 
correspond to the same transmitter they are sub-optimal. 
Moreover, if the number of array elements at each base 
station is M the number of co-channel simultaneous 
transmitters that can be localized is M-1. We propose a 
technique that uses exactly the same data as the common 
AOA/TOA methods but the position determination is 
direct. The proposed method can handle LM-1 co-channel 
simultaneous signals, using L base stations. Although 
there are many stray parameters only two-dimensional 
search is required for a planar geometry and a three-
dimensional search for the general case. The technique 
exploits the principles of the MUSIC algorithm, and 
provides a natural solution to the measurements-sources 
association problem that is encountered in AOA/TOA 
based systems.
Unsupervised 3D Learning for Shape Analysis via Multiresolution Instance Discrimination
Authors: Peng-Shuai Wang, Yu-Qi Yang, Qian-Fang Zou, Zhirong Wu, Yang Liu, Xin Tong
Date: 2020-08-03 00:00:00
Tasks: 3D Point Cloud Linear Classification, 3D Semantic Segmentation

Although unsupervised feature learning has demonstrated its advantages to reducing the workload of data labeling and network design in many fields, existing unsupervised 3D learning methods still cannot offer a generic network for various shape analysis tasks with competitive performance to supervised methods. In this paper, we propose an unsupervised method for learning a generic and efficient shape encoding network for different shape analysis tasks. The key idea of our method is to jointly encode and learn shape and point features from unlabeled 3D point clouds. For this purpose, we adapt HR-Net to octree-based convolutional neural networks for jointly encoding shape and point features with fused multiresolution subnetworks and design a simple-yet-efficient Multiresolution Instance Discrimination (MID) loss for jointly learning the shape and point features. Our network takes a 3D point cloud as input and output both shape and point features. After training, the network is concatenated with simple task-specific back-end layers and fine-tuned for different shape analysis tasks. We evaluate the efficacy and generality of our method and validate our network and loss design with a set of shape analysis tasks, including shape classification, semantic shape segmentation, as well as shape registration tasks. With simple back-ends, our network demonstrates the best performance among all unsupervised methods and achieves competitive performance to supervised methods, especially in tasks with a small labeled dataset. For fine-grained shape segmentation, our method even surpasses existing supervised methods by a large margin.