pwc-archive/papers-with-abstracts huggingface dataset:
Multi-objective Semi-supervised Clustering for Finding Predictive Clusters
Authors: Zahra Ghasemi, Hadi Akbarzadeh Khorshidi, Uwe Aickelin
Date: 2022-01-26 00:00:00
Tasks: Clustering, regression

This study concentrates on clustering problems and aims to find compact clusters that are informative regarding the outcome variable. The main goal is partitioning data points so that observations in each cluster are similar and the outcome variable can be predicated using these clusters simultaneously. We model this semi-supervised clustering problem as a multi-objective optimization problem with considering deviation of data points in clusters and prediction error of the outcome variable as two objective functions to be minimized. For finding optimal clustering solutions, we employ a non-dominated sorting genetic algorithm II approach and local regression is applied as prediction method for the output variable. For comparing the performance of the proposed model, we compute seven models using five real-world data sets. Furthermore, we investigate the impact of using local regression for predicting the outcome variable in all models, and examine the performance of the multi-objective models compared to single-objective models.
A comprehensive review and evaluation on text predictive and entertainment systems
Authors: Hozan K. Hamarashid, Soran A. Saeed, Tarik A. Rashid
Date: 2022-01-08 00:00:00
Tasks: Miscellaneous, Prediction

One of the most important ways to experience communication and interact with the systems is by handling the prediction of the most likely words to happen after typing letters or words. It is helpful for people with disabilities due to disabling people who could type or enter texts at a limited slow speed. Also, it is beneficial for people with dyslexia and those people who are not well with spells of words. Though, an input technology, for instance, the next word suggestion facilitates the typing process in smartphones as an example. This means that when a user types a word, then the system suggests the next words to be chosen in which the necessary word by the user. Besides, it can be used in entertainment as a gam, for example, to determine a target word and reach it or tackle it within 10 attempts of prediction. Generally, the systems depend on a text corpus, which was provided in the system to conduct the prediction. Writing every single word is time-consuming, therefore, it is vitally important to decrease time consumption by reducing efforts to input texts in the systems by offering most probable words for the user to select, this could be done via next word prediction systems. There are several techniques can be found in literature, which is utilized to conduct a variety of next word prediction systems by using different approaches. In this paper, a survey of miscellaneous techniques towards the next word prediction systems will be addressed. Besides, the evaluation of the prediction systems will be discussed. Then, a modal technique will be determined to be utilized for the next word prediction system from the perspective of easiness of implementation and obtaining a good result.
Learnable Wavelet Packet Transform for Data-Adapted Spectrograms
Authors: Frusque Gaetan, Fink Olga
Date: 2022-01-26 00:00:00
Tasks: Anomaly Detection, Feature Engineering

Capturing high-frequency data concerning the condition of complex systems, e.g. by acoustic monitoring, has become increasingly prevalent. Such high-frequency signals typically contain time dependencies ranging over different time scales and different types of cyclic behaviors. Processing such signals requires careful feature engineering, particularly the extraction of meaningful time-frequency features. This can be time-consuming and the performance is often dependent on the choice of parameters. To address these limitations, we propose a deep learning framework for learnable wavelet packet transforms, enabling to learn features automatically from data and optimise them with respect to the defined objective function. The learned features can be represented as a spectrogram, containing the important time-frequency information of the dataset. We evaluate the properties and performance of the proposed approach by evaluating its improved spectral leakage and by applying it to an anomaly detection task for acoustic monitoring.
A Kernel-Based Approach for Modelling Gaussian Processes with Functional Information
Authors: D. Andrew Brown, Peter Kiessler, John Nicholson
Date: 2022-01-26 00:00:00
Tasks: Gaussian Processes, Uncertainty Quantification

Gaussian processes (GPs) are ubiquitous tools for modeling and predicting continuous processes in physical and engineering sciences. This is partly due to the fact that one may employ a Gaussian process as an interpolator while facilitating straightforward uncertainty quantification at other locations. In addition to training data, it is sometimes the case that available information is not in the form of a finite collection of points. For example, boundary value problems contain information on the boundary of a domain, or underlying physics lead to known behavior on an entire uncountable subset of the domain of interest. While an approximation to such known information may be obtained via pseudo-training points in the known subset, such a procedure is ad hoc with little guidance on the number of points to use, nor the behavior as the number of pseudo-observations grows large. We propose and construct Gaussian processes that unify, via reproducing kernel Hilbert space, the typical finite training data case with the case of having uncountable information by exploiting the equivalence of conditional expectation and orthogonal projections in Hilbert space. We show existence of the proposed process and establish that it is the limit of a conventional GP conditioned on an increasing number of training points. We illustrate the flexibility and advantages of our proposed approach via numerical experiments.
Can Old TREC Collections Reliably Evaluate Modern Neural Retrieval Models?
Authors: Ellen M. Voorhees, Ian Soboroff, Jimmy Lin
Date: 2022-01-26 00:00:00
Tasks: Retrieval

Neural retrieval models are generally regarded as fundamentally different from the retrieval techniques used in the late 1990's when the TREC ad hoc test collections were constructed. They thus provide the opportunity to empirically test the claim that pooling-built test collections can reliably evaluate retrieval systems that did not contribute to the construction of the collection (in other words, that such collections can be reusable). To test the reusability claim, we asked TREC assessors to judge new pools created from new search results for the TREC-8 ad hoc collection. These new search results consisted of five new runs (one each from three transformer-based models and two baseline runs that use BM25) plus the set of TREC-8 submissions that did not previously contribute to pools. The new runs did retrieve previously unseen documents, but the vast majority of those documents were not relevant. The ranking of all runs by mean evaluation score when evaluated using the official TREC-8 relevance judgment set and the newly expanded relevance set are almost identical, with Kendall's tau correlations greater than 0.99. Correlations for individual topics are also high. The TREC-8 ad hoc collection was originally constructed using deep pools over a diverse set of runs, including several effective manual runs. Its judgment budget, and hence construction cost, was relatively large. However, it does appear that the expense was well-spent: even with the advent of neural techniques, the collection has stood the test of time and remains a reliable evaluation instrument as retrieval techniques have advanced.
Causality and Correlation Graph Modeling for Effective and Explainable Session-based Recommendation
Authors: Huizi Wu, Cong Geng, Hui Fang
Date: 2022-01-26 00:00:00
Tasks: Graph Neural Network, Session-Based Recommendations

Session-based recommendation which has been witnessed a booming interest recently, focuses on predicting a user's next interested item(s) based on an anonymous session. Most existing studies adopt complex deep learning techniques (e.g., graph neural networks) for effective session-based recommendation. However, they merely address co-occurrence between items, but fail to well distinguish causality and correlation relationship. Considering the varied interpretations and characteristics of causality and correlation relationship between items, in this study, we propose a novel method denoted as CGSR by jointly modeling causality and correlation relationship between items. In particular, we construct cause, effect and correlation graphs from sessions by simultaneously considering the false causality problem. We further design a graph neural network-based method for session-based recommendation. To conclude, we strive to explore the relationship between items from specific ``causality" (directed) and ``correlation" (undirected) perspectives. Extensive experiments on three datasets show that our model outperforms other state-of-the-art methods in terms of recommendation accuracy. Moreover, we further propose an explainable framework on CGSR, and demonstrate the explainability of our model via case studies on Amazon dataset.
Out-of-Domain Semantics to the Rescue! Zero-Shot Hybrid Retrieval Models
Authors: Tao Chen, Mingyang Zhang, Jing Lu, Michael Bendersky, Marc Najork
Date: 2022-01-25 00:00:00
Tasks: Language Modelling, Passage Retrieval, Retrieval

The pre-trained language model (eg, BERT) based deep retrieval models achieved superior performance over lexical retrieval models (eg, BM25) in many passage retrieval tasks. However, limited work has been done to generalize a deep retrieval model to other tasks and domains. In this work, we carefully select five datasets, including two in-domain datasets and three out-of-domain datasets with different levels of domain shift, and study the generalization of a deep model in a zero-shot setting. Our findings show that the performance of a deep retrieval model is significantly deteriorated when the target domain is very different from the source domain that the model was trained on. On the contrary, lexical models are more robust across domains. We thus propose a simple yet effective framework to integrate lexical and deep retrieval models. Our experiments demonstrate that these two models are complementary, even when the deep model is weaker in the out-of-domain setting. The hybrid model obtains an average of 20.4% relative gain over the deep retrieval model, and an average of 9.54% over the lexical model in three out-of-domain datasets.
Online POI Recommendation: Learning Dynamic Geo-Human Interactions in Streams
Authors: Dongjie Wang, Kunpeng Liu, Hui Xiong, Yanjie Fu
Date: 2022-01-19 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

In this paper, we focus on the problem of modeling dynamic geo-human interactions in streams for online POI recommendations. Specifically, we formulate the in-stream geo-human interaction modeling problem into a novel deep interactive reinforcement learning framework, where an agent is a recommender and an action is a next POI to visit. We uniquely model the reinforcement learning environment as a joint and connected composition of users and geospatial contexts (POIs, POI categories, functional zones). An event that a user visits a POI in stream updates the states of both users and geospatial contexts; the agent perceives the updated environment state to make online recommendations. Specifically, we model a mixed-user event stream by unifying all users, visits, and geospatial contexts as a dynamic knowledge graph stream, in order to model human-human, geo-human, geo-geo interactions. We design an exit mechanism to address the expired information challenge, devise a meta-path method to address the recommendation candidate generation challenge, and develop a new deep policy network structure to address the varying action space challenge, and, finally, propose an effective adversarial training method for optimization. Finally, we present extensive experiments to demonstrate the enhanced performance of our method.
Topic Aware Contextualized Embeddings for High Quality Phrase Extraction
Authors: Venktesh V, Mukesh Mohania, Vikram Goyal
Date: 2022-01-17 00:00:00
Tasks: Keyphrase Extraction, Vocal Bursts Intensity Prediction

Keyphrase extraction from a given document is the task of automatically extracting salient phrases that best describe the document. This paper proposes a novel unsupervised graph-based ranking method to extract high-quality phrases from a given document. We obtain the contextualized embeddings from pre-trained language models enriched with topic vectors from Latent Dirichlet Allocation (LDA) to represent the candidate phrases and the document. We introduce a scoring mechanism for the phrases using the information obtained from contextualized embeddings and the topic vectors. The salient phrases are extracted using a ranking algorithm on an undirected graph constructed for the given document. In the undirected graph, the nodes represent the phrases, and the edges between the phrases represent the semantic relatedness between them, weighted by a score obtained from the scoring mechanism. To demonstrate the efficacy of our proposed method, we perform several experiments on open source datasets in the science domain and observe that our novel method outperforms existing unsupervised embedding based keyphrase extraction methods. For instance, on the SemEval2017 dataset, our method advances the F1 score from 0.2195 (EmbedRank) to 0.2819 at the top 10 extracted keyphrases. Several variants of the proposed algorithm are investigated to determine their effect on the quality of keyphrases. We further demonstrate the ability of our proposed method to collect additional high-quality keyphrases that are not present in the document from external knowledge bases like Wikipedia for enriching the document with newly discovered keyphrases. We evaluate this step on a collection of annotated documents. The F1-score at the top 10 expanded keyphrases is 0.60, indicating that our algorithm can also be used for 'concept' expansion using external knowledge.
Alleviating Cold-start Problem in CTR Prediction with A Variational Embedding Learning Framework
Authors: Xiaoxiao Xu, Chen Yang, Qian Yu, Zhiwei Fang, Jiaxing Wang, Chaosheng Fan, Yang He, Changping Peng, Zhangang Lin, Jingping Shao
Date: 2022-01-17 00:00:00
Tasks: Click-Through Rate Prediction, Variational Inference

We propose a general Variational Embedding Learning Framework (VELF) for alleviating the severe cold-start problem in CTR prediction. VELF addresses the cold start problem via alleviating over-fits caused by data-sparsity in two ways: learning probabilistic embedding, and incorporating trainable and regularized priors which utilize the rich side information of cold start users and advertisements (Ads). The two techniques are naturally integrated into a variational inference framework, forming an end-to-end training process. Abundant empirical tests on benchmark datasets well demonstrate the advantages of our proposed VELF. Besides, extended experiments confirmed that our parameterized and regularized priors provide more generalization capability than traditional fixed priors.
Machine Learning for Food Review and Recommendation
Authors: Tan Khang Le, Siu Cheung Hui
Date: 2022-01-15 00:00:00
Tasks: BIG-bench Machine Learning, Deep Learning, Dependency Parsing, POS, POS Tagging, Re-Ranking, Retrieval, Sentiment Analysis, TAG

Food reviews and recommendations have always been important for online food service websites. However, reviewing and recommending food is not simple as it is likely to be overwhelmed by disparate contexts and meanings. In this paper, we use different deep learning approaches to address the problems of sentiment analysis, automatic review tag generation, and retrieval of food reviews. We propose to develop a web-based food review system at Nanyang Technological University (NTU) named NTU Food Hunter, which incorporates different deep learning approaches that help users with food selection. First, we implement the BERT and LSTM deep learning models into the system for sentiment analysis of food reviews. Then, we develop a Part-of-Speech (POS) algorithm to automatically identify and extract adjective-noun pairs from the review content for review tag generation based on POS tagging and dependency parsing. Finally, we also train a RankNet model for the re-ranking of the retrieval results to improve the accuracy in our Solr-based food reviews search system. The experimental results show that our proposed deep learning approaches are promising for the applications of real-world problems.
ThanosNet: A Novel Trash Classification Method Using Metadata
Authors: Alan Sun, Harry Xiao
Date: 2021-03-19 00:00:00
Proceeding: IEEE International Conference on Big Data 2021 3
Tasks: Classification, Image Classification, Transfer Learning

Recent progress in deep neural networks has spurred significant development of image-based trash classification literature. These methods predominately use transfer learning to achieve state-of-the-art results. In this contribution, a new methodology is introduced that uses metadata fields such as location and time-based traffic intensity to assist existing image-based classifiers. We curated ISBNet, a dataset which contains 889 images and their associated metadata, distributed over 5 classes (paper, plastic, cans, tetra pak, and landfill). This dataset was used to develop our model, ThanosNet, which is superior to current state-of-the-art, image-based, trash classification models. Although ISBNet is localized to one user community, the general methodology developed here is applicable to a wide array of consumer contexts.
Confidence intervals for the Cox model test error from cross-validation
Authors: Min Woo Sun, Robert Tibshirani
Date: 2022-01-26 00:00:00

Cross-validation (CV) is one of the most widely used techniques in statistical learning for estimating the test error of a model, but its behavior is not yet fully understood. It has been shown that standard confidence intervals for test error using estimates from CV may have coverage below nominal levels. This phenomenon occurs because each sample is used in both the training and testing procedures during CV and as a result, the CV estimates of the errors become correlated. Without accounting for this correlation, the estimate of the variance is smaller than it should be. One way to mitigate this issue is by estimating the mean squared error of the prediction error instead using nested CV. This approach has been shown to achieve superior coverage compared to intervals derived from standard CV. In this work, we generalize the nested CV idea to the Cox proportional hazards model and explore various choices of test error for this setting.
Combining Experimental and Observational Data for Identification and Estimation of Long-Term Causal Effects
Authors: AmirEmad Ghassami, Alan Yang, David Richardson, Ilya Shpitser, Eric Tchetgen Tchetgen
Date: 2022-01-26 00:00:00
Tasks: Causal Inference

We consider the task of identifying and estimating the causal effect of a treatment variable on a long-term outcome variable using data from an observational domain and an experimental domain. The observational domain is subject to unobserved confounding. Furthermore, subjects in the experiment are only followed for a short period of time; hence, long-term effects of treatment are unobserved but short-term effects will be observed. Therefore, data from neither domain alone suffices for causal inference about the effect of the treatment on the long-term outcome, and must be pooled in a principled way, instead. Athey et al. (2020) proposed a method for systematically combining such data for identifying the downstream causal effect in view. Their approach is based on the assumptions of internal and external validity of the experimental data, and an extra novel assumption called latent unconfoundedness. In this paper, we first review their proposed approach, and then we propose three alternative approaches for data fusion for the purpose of identifying and estimating average treatment effect as well as the effect of treatment on the treated. Our first approach is based on assuming equi-confounding bias for the short-term and long-term outcomes. Our second approach is based on a relaxed version of the equi-confounding bias assumption, where we assume the existence of an observed confounder such that the short-term and long-term potential outcome variables have the same partial additive association with that confounder. Our third approach is based on the proximal causal inference framework, in which we assume the existence of an extra variable in the system which is a proxy of the latent confounder of the treatment-outcome relation. We propose influence function-based estimation strategies for each of our data fusion frameworks and study the robustness properties of the proposed estimators.
Supervised learning of sheared distributions using linearized optimal transport
Authors: Varun Khurana, Harish Kannan, Alexander Cloninger, Caroline Moosmüller
Date: 2022-01-25 00:00:00
Tasks: image-classification, Image Classification

In this paper we study supervised learning tasks on the space of probability measures. We approach this problem by embedding the space of probability measures into $L^2$ spaces using the optimal transport framework. In the embedding spaces, regular machine learning techniques are used to achieve linear separability. This idea has proved successful in applications and when the classes to be separated are generated by shifts and scalings of a fixed measure. This paper extends the class of elementary transformations suitable for the framework to families of shearings, describing conditions under which two classes of sheared distributions can be linearly separated. We furthermore give necessary bounds on the transformations to achieve a pre-specified separation level, and show how multiple embeddings can be used to allow for larger families of transformations. We demonstrate our results on image classification tasks.
Design and Development of an Autonomous Surface Vehicle for Water Quality Monitoring
Authors: MM Rashid, Rupal Roy, Md Manjurul Ahsan, Zahed Siddique
Date: 2022-01-26 00:00:00

Manually monitoring water quality is very exhausting and requires several hours of sampling and laboratory testing for a particular body of water. This article presents a solution to test water properties like electrical conductivity and pH with a remote-controlled floating vehicle that minimizes time intervals. An autonomous surface vehicle (ASV) has been designed mathematically and operated via MATLAB \& Simulink simulation where the Proportional integral derivative (PID) controller has been considered. A PVC model with Small waterplane area twin-hull (SWATH) technology is used to develop this vehicle. Manually collected data is compared to online sensors, suggesting a better solution for determining water properties such as dissolved oxygen (DO), biochemical oxygen demand (BOD), temperature, conductivity, total alkalinity, and bacteria. Preliminary computational results show the promising result, as Sungai Pasu rivers tested water falls in the safe range of pH (~6.8-7.14) using the developed ASV.