pwc-archive/papers-with-abstracts huggingface dataset:
Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks
Authors: Wei-Long Zheng, Bao-liang Lu
Date: 2015-05-08 00:00:00
Proceeding: IEEE Transactions on Autonomous Mental Development ( Volume: 7 , Issue: 3 , Sept. 2015 ) 2015 5
Tasks: EEG, Electroencephalogram (EEG), Emotion Recognition

To investigate critical frequency bands and channels, this paper introduces deep belief networks (DBNs) to constructing EEG-based emotion recognition models for three emotions: positive, neutral and negative. We develop an EEG dataset acquired from 15 subjects. Each subject performs the experiments twice at the interval of a few days. DBNs are trained with differential entropy features extracted from multichannel EEG data. We examine the weights of the trained DBNs and investigate the critical frequency bands and channels. Four different profiles of 4, 6, 9, and 12 channels are selected. The recognition accuracies of these four profiles are relatively stable with the best accuracy of 86.65%, which is even better than that of the original 62 channels. The critical frequency bands and channels determined by using the weights of trained DBNs are consistent with the existing observations. In addition, our experiment results show that neural signatures associated with different emotions do exist and they share commonality across sessions and individuals. We compare the performance of deep models with shallow models. The average accuracies of DBN, SVM, LR, and KNN are 86.08%, 83.99%, 82.70%, and 72.60%, respectively.
DisNet: A novel method for distance estimation from monocular camera
Authors: Muhammad Abdul Haseeb, Jianyu Guan, Danijela Ristić-Durrant, Axel Gräser
Date: 2018-10-01 00:00:00
Proceeding: Institute of Automation 2018 10
Tasks: Depth Estimation, Object

In this paper, a machine learning setup that provides the obstacle detection system with a method to estimate the distance from the monocular camera to the object viewed with the camera is presented. In particular, the preliminary results of on-going research to allow the onboard multisensory system, which is under development within H2020 Shift2Rail project SMART, to autonomously learn distances to objects, possible obstacles on the rail tracks ahead of the locomotive are given. The presented distance estimation system is based on Multi Hidden-Layer Neural Network, named DisNet, which is used to learn and predict the distance between the object and the camera sensor. The DisNet was trained using a supervised learning technique where the input features were manually calculated parameters of the object bounding boxes resulted from the YOLO object classifier and outputs were the accurate 3D laser scanner measurements of the distances to objects in the recorded scene. The presented DisNet-based distance estimation system was evaluated on the images of railway scenes as well as on the images of a road scene. Shown results demonstrate a general nature of the proposed DisNet system that enables its use for the estimation of distances to objects imaged with different types of monocular cameras.
Discriminative Graph Autoencoder
Authors: Haifeng Jin, Qingquan Song, Xia Hu
Date: 2018-11-17 00:00:00
Proceeding: 2018 IEEE International Conference on Big Knowledge (ICBK) 2018 11
Tasks: Graph Classification, Graph Representation Learning, Representation Learning

With the abundance of graph-structured data in various applications, graph representation learning has become an effective computational tool for seeking informative vector representations for graphs. Traditional graph kernel approaches are usually frequency-based. Each dimension of a learned vector representation for a graph is the frequency of a certain type of substructure. They encounter high computational cost for counting the occurrence of predefined substructures. The learned vector representations are very sparse, which prohibit the use of inner products. Moreover, the learned vector representations are not in a smooth space since the values can only be integers. The state-of-the-art approaches tackle the challenges by changing kernel functions instead of producing better vector representations. They can only produce kernel matrices for kernel-based methods and not compatible with methods requiring vector representations. Effectively learning smooth vector representations for graphs of various structures and sizes remains a challenging task. Motivated by the recent advances in deep autoencoders, in this paper, we explore the capability of autoencoder on learning representations for graphs. Unlike videos or images, the graphs are usually of various sizes and are not readily prepared for autoencoder. Therefore, a novel framework, namely discriminative graph autoencoder (DGA), is proposed to learn low-dimensional vector representations for graphs. The algorithm decomposes the large graphs into small subgraphs, from which the structural information is sampled. The DGA produces smooth and informative vector representations of graphs efficiently while preserving the discriminative information according to their labels. Extensive experiments have been conducted to evaluate DGA. The experimental results demonstrate the efficiency and effectiveness of DGA comparing with traditional and state-of-the-art approaches on various real-world datasets and applications, e.g.,...
Deep neural networks for learning graph representations
Authors: Shaosheng Cao, Wei Lu, Qiongkai Xu
Date: 2016-02-21 00:00:00
Proceeding: Thirtieth AAAI Conference on Artificial Intelligence 2016 2
Tasks: Clustering, Denoising, Graph Clustering

In this paper, we propose a novel model for learning graph representations, which generates a low-dimensional vector representation for each vertex by capturing the graph structural information. Different from other previous research efforts, we adopt a random surfing model to capture graph structural information directly, instead of using the sampling-based method for generating linear sequences proposed by Perozzi et al. (2014). The advantages of our approach will be illustrated from both theorical and empirical perspectives. We also give a new perspective for the matrix factorization method proposed by Levy and Goldberg (2014), in which the pointwise mutual information (PMI) matrix is considered as an analytical solution to the objective function of the skip-gram model with negative sampling proposed by Mikolov et al. (2013). Unlike their approach which involves the use of the SVD for finding the low-dimensitonal projections from the PMI matrix, however, the stacked denoising autoencoder is introduced in our model to extract complex features and model non-linearities. To demonstrate the effectiveness of our model, we conduct experiments on clustering and visualization tasks, employing the learned vertex representations as features. Empirical results on datasets of varying sizes show that our model outperforms other stat-of-the-art models in such tasks.
Marginalized graph autoencoder for graph clustering
Authors: Chun Wang, Shirui Pan, Guodong Long, Xingquan Zhu, Jing Jiang
Date: 2017-11-06 00:00:00
Proceeding: CIKM '17 Proceedings of the 2017 ACM on Conference on Information and Knowledge Management 2017 11
Tasks: Clustering, Graph Clustering, Graph Representation Learning, Representation Learning

Graph clustering aims to discovercommunity structures in networks, the task being fundamentally challenging mainly because the topology structure and the content of the graphs are difficult to represent for clustering analysis. Recently, graph clustering has moved from traditional shallow methods to deep learning approaches, thanks to the unique feature representation learning capability of deep learning. However, existing deep approaches for graph clustering can only exploit the structure information, while ignoring the content information associated with the nodes in a graph. In this paper, we propose a novel marginalized graph autoencoder (MGAE) algorithm for graph clustering. The key innovation of MGAE is that it advances the autoencoder to the graph domain, so graph representation learning can be carried out not only in a purely unsupervised setting by leveraging structure and content information, it can also be stacked in a deep fashion to learn effective representation. From a technical viewpoint, we propose a marginalized graph convolutional network to corrupt network node content, allowing node content to interact with network features, and marginalizes the corrupted features in a graph autoencoder context to learn graph feature representations. The learned features are fed into the spectral clustering algorithm for graph clustering. Experimental results on benchmark datasets demonstrate the superior performance of MGAE, compared to numerous baselines.
Deep learning with sentence embeddings pre-trained on biomedical corpora improves the performance of finding similar sentences in electronic medical records
Authors: Qingyu Chen, Jingcheng Du, Sun Kim, W. John Wilbur, Zhiyong Lu
Date: 2019-09-06 00:00:00
Tasks: Semantic Textual Similarity, Sentence, Sentence Embeddings, STS

Capturing sentence semantics plays a vital role in a range of text mining applications. Despite continuous efforts on the development of related datasets and models in the general domain, both datasets and models are limited in biomedical and clinical domains. The BioCreative/OHNLP organizers have made the first attempt to annotate 1,068 sentence pairs from clinical notes and have called for a community effort to tackle the Semantic Textual Similarity (BioCreative/OHNLP STS) challenge. We developed models using traditional machine learning and deep learning approaches. For the post challenge, we focus on two models: the Random Forest and the Encoder Network. We applied sentence embeddings pre-trained on PubMed abstracts and MIMIC-III clinical notes and updated the Random Forest and the Encoder Network accordingly. The official results demonstrated our best submission was the ensemble of eight models. It achieved a Person correlation coefficient of 0.8328, the highest performance among 13 submissions from 4 teams. For the post challenge, the performance of both Random Forest and the Encoder Network was improved; in particular, the correlation of the Encoder Network was improved by ~13%. During the challenge task, no end-to-end deep learning models had better performance than machine learning models that take manually-crafted features. In contrast, with the sentence embeddings pre-trained on biomedical corpora, the Encoder Network now achieves a correlation of ~0.84, which is higher than the original best model. The ensembled model taking the improved versions of the Random Forest and Encoder Network as inputs further increased performance to 0.8528. Deep learning models with sentence embeddings pre-trained on biomedical corpora achieve the highest performance on the test set.
Uncertain Natural Language Inference
Authors: Tongfei Chen, Zhengping Jiang, Adam Poliak, Keisuke Sakaguchi, Benjamin Van Durme
Date: 2019-09-06 00:00:00
Proceeding: ACL 2020 6
Tasks: Learning-To-Rank, Natural Language Inference, regression

We introduce Uncertain Natural Language Inference (UNLI), a refinement of Natural Language Inference (NLI) that shifts away from categorical labels, targeting instead the direct prediction of subjective probability assessments. We demonstrate the feasibility of collecting annotations for UNLI by relabeling a portion of the SNLI dataset under a probabilistic scale, where items even with the same categorical label differ in how likely people judge them to be true given a premise. We describe a direct scalar regression modeling approach, and find that existing categorically labeled NLI data can be used in pre-training. Our best models approach human performance, demonstrating models may be capable of more subtle inferences than the categorical bin assignment employed in current NLI tasks.
Improved Hierarchical Patient Classification with Language Model Pretraining over Clinical Notes
Authors: Jonas Kemp, Alvin Rajkomar, Andrew M. Dai
Date: 2019-09-06 00:00:00
Tasks: General Classification, Language Modeling, Language Modelling

Clinical notes in electronic health records contain highly heterogeneous writing styles, including non-standard terminology or abbreviations. Using these notes in predictive modeling has traditionally required preprocessing (e.g. taking frequent terms or topic modeling) that removes much of the richness of the source data. We propose a pretrained hierarchical recurrent neural network model that parses minimally processed clinical notes in an intuitive fashion, and show that it improves performance for discharge diagnosis classification tasks on the Medical Information Mart for Intensive Care III (MIMIC-III) dataset, compared to models that treat the notes as an unordered collection of terms or that conduct no pretraining. We also apply an attribution technique to examples to identify the words that the model uses to make its prediction, and show the importance of the words' nearby context.
Quantized Fisher Discriminant Analysis
Authors: Benyamin Ghojogh, Ali Saheb Pasand, Fakhri Karray, Mark Crowley
Date: 2019-09-06 00:00:00
Tasks: BIG-bench Machine Learning, Dimensionality Reduction, Quantization

This paper proposes a new subspace learning method, named Quantized Fisher Discriminant Analysis (QFDA), which makes use of both machine learning and information theory. There is a lack of literature for combination of machine learning and information theory and this paper tries to tackle this gap. QFDA finds a subspace which discriminates the uniformly quantized images in the Discrete Cosine Transform (DCT) domain at least as well as discrimination of non-quantized images by Fisher Discriminant Analysis (FDA) while the images have been compressed. This helps the user to throw away the original images and keep the compressed images instead without noticeable loss of classification accuracy. We propose a cost function whose minimization can be interpreted as rate-distortion optimization in information theory. We also propose quantized Fisherfaces for facial analysis in QFDA. Our experiments on AT&T face dataset and Fashion MNIST dataset show the effectiveness of this subspace learning method.
Neural Network-Based Modeling of Phonetic Durations
Authors: Xizi Wei, Melvyn Hunt, Adrian Skilling
Date: 2019-09-06 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), speech-recognition, Speech Recognition, text-to-speech, Text to Speech

A deep neural network (DNN)-based model has been developed to predict non-parametric distributions of durations of phonemes in specified phonetic contexts and used to explore which factors influence durations most. Major factors in US English are pre-pausal lengthening, lexical stress, and speaking rate. The model can be used to check that text-to-speech (TTS) training speech follows the script and words are pronounced as expected. Duration prediction is poorer with training speech for automatic speech recognition (ASR) because the training corpus typically consists of single utterances from many speakers and is often noisy or casually spoken. Low probability durations in ASR training material nevertheless mostly correspond to non-standard speech, with some having disfluencies. Children's speech is disproportionately present in these utterances, since children show much more variation in timing.
Agora: A Unified Asset Ecosystem Going Beyond Marketplaces and Cloud Services
Authors: Jonas Traub, Jorge-Arnulfo Quiané-Ruiz, Zoi Kaoudi, Volker Markl
Date: 2019-09-06 00:00:00
Tasks: Management

Data, algorithms, and compute/storage infrastructure are key assets that drive data science and artificial intelligence applications. As providing all these assets requires a huge investment, data science and artificial intelligence technologies are currently dominated by a small number of providers who can afford these investments. This leads to lock-in effects and hinders features that require a flexible exchange of assets among users. In this vision paper, we present Agora, a unified asset ecosystem. The Agora system provides the technical infrastructure that allows for offering and using data and algorithms, as well as physical infrastructure components. Agora is designed as an open ecosystem of asset marketplaces and provides to a broad audience not only data but the entire data value chain (including computational resources and human expertise). Agora (i) leverages a fine-grained exchange of assets, (ii) allows for combining assets to novel applications, and (iii) flexibly executes such applications on available resources. As a result, Agora overcomes lock-in effects and removes entry barriers for new asset providers. In contrast to existing data management systems, Agora operates in a heavily decentralized and dynamic environment: Data, algorithms, and even compute resources are dynamically created, modified, and removed by different stakeholders. Agora presents novel research directions for the data management community as a whole: It requires to combine our traditional expertise in scalable data processing and management with infrastructure provisioning as well as economic and application aspects of data, algorithms, and infrastructure.
Annotating Student Talk in Text-based Classroom Discussions
Authors: Luca Lugini, Diane Litman, Amanda Godley, Christopher Olshefski
Date: 2019-09-06 00:00:00
Proceeding: WS 2018 6
Tasks: Specificity

Classroom discussions in English Language Arts have a positive effect on students' reading, writing and reasoning skills. Although prior work has largely focused on teacher talk and student-teacher interactions, we focus on three theoretically-motivated aspects of high-quality student talk: argumentation, specificity, and knowledge domain. We introduce an annotation scheme, then show that the scheme can be used to produce reliable annotations and that the annotations are predictive of discussion quality. We also highlight opportunities provided by our scheme for education and natural language processing research.
Argument Component Classification for Classroom Discussions
Authors: Luca Lugini, Diane Litman
Date: 2019-09-06 00:00:00
Proceeding: WS 2018 11
Tasks: Argument Mining, Classification, Component Classification, General Classification, Specificity

This paper focuses on argument component classification for transcribed spoken classroom discussions, with the goal of automatically classifying student utterances into claims, evidence, and warrants. We show that an existing method for argument component classification developed for another educationally-oriented domain performs poorly on our dataset. We then show that feature sets from prior work on argument mining for student essays and online dialogues can be used to improve performance considerably. We also provide a comparison between convolutional neural networks and recurrent neural networks when trained under different conditions to classify argument components in classroom discussions. While neural network models are not always able to outperform a logistic regression model, we were able to gain some useful insights: convolutional networks are more robust than recurrent networks both at the character and at the word level, and specificity information can help boost performance in multi-task training.
Approaching Machine Learning Fairness through Adversarial Network
Authors: Xiaoqian Wang, Heng Huang
Date: 2019-09-06 00:00:00
Tasks: BIG-bench Machine Learning, Fairness

Fairness is becoming a rising concern w.r.t. machine learning model performance. Especially for sensitive fields such as criminal justice and loan decision, eliminating the prediction discrimination towards a certain group of population (characterized by sensitive features like race and gender) is important for enhancing the trustworthiness of model. In this paper, we present a new general framework to improve machine learning fairness. The goal of our model is to minimize the influence of sensitive feature from the perspectives of both the data input and the predictive model. In order to achieve this goal, we reformulate the data input by removing the sensitive information and strengthen model fairness by minimizing the marginal contribution of the sensitive feature. We propose to learn the non-sensitive input via sampling among features and design an adversarial network to minimize the dependence between the reformulated input and the sensitive information. Extensive experiments on three benchmark datasets suggest that our model achieve better results than related state-of-the-art methods with respect to both fairness metrics and prediction performance.
One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniques
Authors: Vijay Arya, Rachel K. E. Bellamy, Pin-Yu Chen, Amit Dhurandhar, Michael Hind, Samuel C. Hoffman, Stephanie Houde, Q. Vera Liao, Ronny Luss, Aleksandra Mojsilović, Sami Mourad, Pablo Pedemonte, Ramya Raghavendra, John Richards, Prasanna Sattigeri, Karthikeyan Shanmugam, Moninder Singh, Kush R. Varshney, Dennis Wei, Yunfeng Zhang
Date: 2019-09-06 00:00:00
Tasks: All, Navigate

As artificial intelligence and machine learning algorithms make further inroads into society, calls are increasing from multiple stakeholders for these algorithms to explain their outputs. At the same time, these stakeholders, whether they be affected citizens, government regulators, domain experts, or system developers, present different requirements for explanations. Toward addressing these needs, we introduce AI Explainability 360 (http://aix360.mybluemix.net/), an open-source software toolkit featuring eight diverse and state-of-the-art explainability methods and two evaluation metrics. Equally important, we provide a taxonomy to help entities requiring explanations to navigate the space of explanation methods, not only those in the toolkit but also in the broader literature on explainability. For data scientists and other users of the toolkit, we have implemented an extensible software architecture that organizes methods according to their place in the AI modeling pipeline. We also discuss enhancements to bring research innovations closer to consumers of explanations, ranging from simplified, more accessible versions of algorithms, to tutorials and an interactive web demo to introduce AI explainability to different audiences and application domains. Together, our toolkit and taxonomy can help identify gaps where more explainability methods are needed and provide a platform to incorporate them as they are developed.
RNN Architecture Learning with Sparse Regularization
Authors: Jesse Dodge, Roy Schwartz, Hao Peng, Noah A. Smith
Date: 2019-09-06 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Sentiment Analysis

Neural models for NLP typically use large numbers of parameters to reach state-of-the-art performance, which can lead to excessive memory usage and increased runtime. We present a structure learning method for learning sparse, parameter-efficient NLP models. Our method applies group lasso to rational RNNs (Peng et al., 2018), a family of models that is closely connected to weighted finite-state automata (WFSAs). We take advantage of rational RNNs' natural grouping of the weights, so the group lasso penalty directly removes WFSA states, substantially reducing the number of parameters in the model. Our experiments on a number of sentiment analysis datasets, using both GloVe and BERT embeddings, show that our approach learns neural structures which have fewer parameters without sacrificing performance relative to parameter-rich baselines. Our method also highlights the interpretable properties of rational RNNs. We show that sparsifying such models makes them easier to visualize, and we present models that rely exclusively on as few as three WFSAs after pruning more than 90% of the weights. We publicly release our code.