pwc-archive/papers-with-abstracts huggingface dataset:
Feature selection revisited in the single-cell era
Authors: Pengyi Yang, Hao Huang, Chunlei Liu
Date: 2021-10-27 00:00:00
Tasks: feature selection

Feature selection techniques are essential for high-dimensional data analysis. In the last two decades, their popularity has been fuelled by the increasing availability of high-throughput biomolecular data where high-dimensionality is a common data property. Recent advances in biotechnologies enable global profiling of various molecular and cellular features at single-cell resolution, resulting in large-scale datasets with increased complexity. These technological developments have led to a resurgence in feature selection research and application in the single-cell field. Here, we revisit feature selection techniques and summarise recent developments. We review their versatile application to a range of single-cell data types including those generated from traditional cytometry and imaging technologies and the latest array of single-cell omics technologies. We highlight some of the challenges and future directions on which feature selection could have a significant impact. Finally, we consider the scalability and make general recommendations on the utility of each type of feature selection method. We hope this review serves as a reference point to stimulate future research and application of feature selection in the single-cell era.
Mathematical Modeling, In-Human Evaluation and Analysis of Volume Kinetics and Kidney Function after Burn Injury and Resuscitation
Authors: Ghazal ArabiDarrehDor, Ali Tivay, Chris Meador, George C. Kramer, Jin-OH Hahn, Jose Salinas
Date: 2021-10-24 00:00:00
Tasks: Kidney Function

Existing burn resuscitation protocols exhibit large variability in treatment efficacy. Hence, they must be further optimized based on comprehensive knowledge of burn pathophysiology. A physics-based mathematical model that can replicate physiological responses in diverse burn patients can serve as an attractive basis to perform non-clinical testing of burn resuscitation protocols and to expand knowledge on burn pathophysiology. We intend to develop, optimize, validate, and analyze a mathematical model to replicate physiological responses in burn patients. Using clinical datasets collected from 233 burn patients receiving burn resuscitation, we developed and validated a mathematical model applicable to computer-aided in-human burn resuscitation trial and knowledge expansion. Using the validated mathematical model, we examined possible physiological mechanisms responsible for the cohort-dependent differences in burn pathophysiology between younger versus older patients, female versus male patients, and patients with versus without inhalational injury. We demonstrated that the mathematical model could replicate physiological responses in burn patients associated with wide demographic characteristics and injury severity and that an increased inflammatory response to injury may be a key contributing factor in increasing the mortality risk of older patients and patients with inhalation injury via an increase in the fluid retention. The mathematical model may provide an attractive platform to conduct non-clinical testing of burn resuscitation protocols and test new hypotheses on burn pathophysiology.
Solution to the Equity Premium Puzzle Using the Sufficiency Factor of the Model
Authors: Atilla Aras
Date: 2021-10-27 00:00:00

This study provides the solution to the equity premium puzzle. The new model was developed by including the behavior of investors toward risk in financial markets in prior studies. The calculations of this newly tested model show that the value of the coefficient of relative risk aversion is 1.033526 by assuming the value of the subjective time discount factor to be 0.99. Since these values are compatible with the existing empirical studies, they confirm the validity of the newly derived model that provides the solution to the equity premium puzzle.
Ask "Who", Not "What": Bitcoin Volatility Forecasting with Twitter Data
Authors: M. Eren Akbiyik, Mert Erkul, Killian Kaempf, Vaiva Vasiliauskaite, Nino Antulov-Fantulin
Date: 2021-10-27 00:00:00
Tasks: Deep Learning

Understanding the variations in trading price (volatility), and its response to exogenous information, is a well-researched topic in finance. In this study, we focus on finding stable and accurate volatility predictors for a relatively new asset class of cryptocurrencies, in particular Bitcoin, using deep learning representations of public social media data obtained from Twitter. For our experiments, we extracted semantic information and user statistics from over 30 million Bitcoin-related tweets, in conjunction with 15-minute frequency price data over a horizon of 144 days. Using this data, we built several deep learning architectures that utilized different combinations of the gathered information. For each model, we conducted ablation studies to assess the influence of different components and feature sets over the prediction accuracy. We found statistical evidences for the hypotheses that: (i) temporal convolutional networks perform significantly better than both classical autoregressive models and other deep learning-based architectures in the literature, and (ii) tweet author meta-information, even detached from the tweet itself, is a better predictor of volatility than the semantic content and tweet volume statistics. We demonstrate how different information sets gathered from social media can be utilized in different architectures and how they affect the prediction results. As an additional contribution, we make our dataset public for future research.
Entropy-based adaptive Hamiltonian Monte Carlo
Authors: Marcel Hirt, Michalis K. Titsias, Petros Dellaportas
Date: 2021-10-27 00:00:00
Proceeding: NeurIPS 2021 12

Hamiltonian Monte Carlo (HMC) is a popular Markov Chain Monte Carlo (MCMC) algorithm to sample from an unnormalized probability distribution. A leapfrog integrator is commonly used to implement HMC in practice, but its performance can be sensitive to the choice of mass matrix used therein. We develop a gradient-based algorithm that allows for the adaptation of the mass matrix by encouraging the leapfrog integrator to have high acceptance rates while also exploring all dimensions jointly. In contrast to previous work that adapt the hyperparameters of HMC using some form of expected squared jumping distance, the adaptation strategy suggested here aims to increase sampling efficiency by maximizing an approximation of the proposal entropy. We illustrate that using multiple gradients in the HMC proposal can be beneficial compared to a single gradient-step in Metropolis-adjusted Langevin proposals. Empirical evidence suggests that the adaptation method can outperform different versions of HMC schemes by adjusting the mass matrix to the geometry of the target distribution and by providing some control on the integration time.
Spike-and-Slab LASSO Generalized Additive Models and Scalable Algorithms for High-Dimensional Data Analysis
Authors: Boyi Guo, Byron C. Jaeger, A. K. M. Fazlur Rahman, D. Leann Long, Nengjun Yi
Date: 2021-10-27 00:00:00
Tasks: Additive models

There are proposals that extend the classical generalized additive models (GAMs) to accommodate high-dimensional data ($p>>n$) using group sparse regularization. However, the sparse regularization may induce excess shrinkage when estimating smooth functions, damaging predictive performance. Moreover, most of these GAMs consider an "all-in-all-out" approach for functional selection, rendering them difficult to answer if nonlinear effects are necessary. While some Bayesian models can address these shortcomings, using Markov chain Monte Carlo algorithms for model fitting creates a new challenge, scalability. Hence, we propose Bayesian hierarchical generalized additive models as a solution: we consider the smoothing penalty for proper shrinkage of curve interpolation via reparameterization. A novel two-part spike-and-slab LASSO prior for smooth functions is developed to address the sparsity of signals while providing extra flexibility to select the linear or nonlinear components of smooth functions. A scalable and deterministic algorithm, EM-Coordinate Descent, is implemented in an open-source R package BHAM. Simulation studies and metabolomics data analyses demonstrate improved predictive and computational performance against state-of-the-art models. Functional selection performance suggests trade-offs exist regarding the effect hierarchy assumption.
A Self-adaptive Weighted Differential Evolution Approach for Large-scale Feature Selection
Authors: Xubin Wang, Yunhe Wang, Ka-Chun Wong, Xiangtao Li
Date: 2021-10-27 00:00:00
Tasks: feature selection

Recently, many evolutionary computation methods have been developed to solve the feature selection problem. However, the studies focused mainly on small-scale issues, resulting in stagnation issues in local optima and numerical instability when dealing with large-scale feature selection dilemmas. To address these challenges, this paper proposes a novel weighted differential evolution algorithm based on self-adaptive mechanism, named SaWDE, to solve large-scale feature selection. First, a multi-population mechanism is adopted to enhance the diversity of the population. Then, we propose a new self-adaptive mechanism that selects several strategies from a strategy pool to capture the diverse characteristics of the datasets from the historical information. Finally, a weighted model is designed to identify the important features, which enables our model to generate the most suitable feature-selection solution. We demonstrate the effectiveness of our algorithm on twelve large-scale datasets. The performance of SaWDE is superior compared to six non-EC algorithms and six other EC algorithms, on both training and test datasets and on subset size, indicating that our algorithm is a favorable tool to solve the large-scale feature selection problem. Moreover, we have experimented SaWDE with six EC algorithms on twelve higher-dimensional data, which demonstrates that SaWDE is more robust and efficient compared to those state-of-the-art methods. SaWDE source code is available on Github at https://github.com/wangxb96/SaWDE.
MEST: Accurate and Fast Memory-Economic Sparse Training Framework on the Edge
Authors: Geng Yuan, Xiaolong Ma, Wei Niu, Zhengang Li, Zhenglun Kong, Ning Liu, Yifan Gong, Zheng Zhan, Chaoyang He, Qing Jin, Siyue Wang, Minghai Qin, Bin Ren, Yanzhi Wang, Sijia Liu, Xue Lin
Date: 2021-10-26 00:00:00
Proceeding: NeurIPS 2021 12

Recently, a new trend of exploring sparsity for accelerating neural network training has emerged, embracing the paradigm of training on the edge. This paper proposes a novel Memory-Economic Sparse Training (MEST) framework targeting for accurate and fast execution on edge devices. The proposed MEST framework consists of enhancements by Elastic Mutation (EM) and Soft Memory Bound (&S) that ensure superior accuracy at high sparsity ratios. Different from the existing works for sparse training, this current work reveals the importance of sparsity schemes on the performance of sparse training in terms of accuracy as well as training speed on real edge devices. On top of that, the paper proposes to employ data efficiency for further acceleration of sparse training. Our results suggest that unforgettable examples can be identified in-situ even during the dynamic exploration of sparsity masks in the sparse training process, and therefore can be removed for further training speedup on edge devices. Comparing with state-of-the-art (SOTA) works on accuracy, our MEST increases Top-1 accuracy significantly on ImageNet when using the same unstructured sparsity scheme. Systematical evaluation on accuracy, training speed, and memory footprint are conducted, where the proposed MEST framework consistently outperforms representative SOTA works. A reviewer strongly against our work based on his false assumptions and misunderstandings. On top of the previous submission, we employ data efficiency for further acceleration of sparse training. And we explore the impact of model sparsity, sparsity schemes, and sparse training algorithms on the number of removable training examples. Our codes are publicly available at: https://github.com/boone891214/MEST.
Comprehensive learning particle swarm optimization enabled modeling framework for multi-step-ahead influenza prediction
Authors: Siyue Yang, Yukun Bao
Date: 2021-10-27 00:00:00
Tasks: Prediction

Epidemics of influenza are major public health concerns. Since influenza prediction always relies on the weekly clinical or laboratory surveillance data, typically the weekly Influenza-like illness (ILI) rate series, accurate multi-step-ahead influenza predictions using ILI series is of great importance, especially, to the potential coming influenza outbreaks. This study proposes Comprehensive Learning Particle Swarm Optimization based Machine Learning (CLPSO-ML) framework incorporating support vector regression (SVR) and multilayer perceptron (MLP) for multi-step-ahead influenza prediction. A comprehensive examination and comparison of the performance and potential of three commonly used multi-step-ahead prediction modeling strategies, including iterated strategy, direct strategy and multiple-input multiple-output (MIMO) strategy, was conducted using the weekly ILI rate series from both the Southern and Northern China. The results show that: (1) The MIMO strategy achieves the best multi-step-ahead prediction, and is potentially more adaptive for longer horizon; (2) The iterated strategy demonstrates special potentials for deriving the least time difference between the occurrence of the predicted peak value and the true peak value of an influenza outbreak; (3) For ILI in the Northern China, SVR model implemented with MIMO strategy performs best, and SVR with iterated strategy also shows remarkable performance especially during outbreak periods; while for ILI in the Southern China, both SVR and MLP models with MIMO strategy have competitive prediction performance
Learning where to learn: Gradient sparsity in meta and continual learning
Authors: Johannes von Oswald, Dominic Zhao, Seijin Kobayashi, Simon Schug, Massimo Caccia, Nicolas Zucchet, João Sacramento
Date: 2021-10-27 00:00:00
Proceeding: NeurIPS 2021 12
Tasks: Continual Learning, Inductive Bias, Meta-Learning, Sparse Learning

Finding neural network weights that generalize well from small datasets is difficult. A promising approach is to learn a weight initialization such that a small number of weight changes results in low generalization error. We show that this form of meta-learning can be improved by letting the learning algorithm decide which weights to change, i.e., by learning where to learn. We find that patterned sparsity emerges from this process, with the pattern of sparsity varying on a problem-by-problem basis. This selective sparsity results in better generalization and less interference in a range of few-shot and continual learning problems. Moreover, we find that sparse learning also emerges in a more expressive model where learning rates are meta-learned. Our results shed light on an ongoing debate on whether meta-learning can discover adaptable features and suggest that learning by sparse gradient descent is a powerful inductive bias for meta-learning systems.
Revisiting the Performance of iALS on Item Recommendation Benchmarks
Authors: Steffen Rendle, Walid Krichene, Li Zhang, Yehuda Koren
Date: 2021-10-26 00:00:00
Tasks: Collaborative Filtering, Recommendation Systems

Matrix factorization learned by implicit alternating least squares (iALS) is a popular baseline in recommender system research publications. iALS is known to be one of the most computationally efficient and scalable collaborative filtering methods. However, recent studies suggest that its prediction quality is not competitive with the current state of the art, in particular autoencoders and other item-based collaborative filtering methods. In this work, we revisit the iALS algorithm and present a bag of tricks that we found useful when applying iALS. We revisit four well-studied benchmarks where iALS was reported to perform poorly and show that with proper tuning, iALS is highly competitive and outperforms any method on at least half of the comparisons. We hope that these high quality results together with iALS's known scalability spark new interest in applying and further improving this decade old technique.
Automated Evaluation of Web Site Accessibility Using A Dynamic Accessibility Measurement Crawler
Authors: Trevor Bostic, Jeffrey Stanley, John Higgins, Daniel Chudnov, Justin Brunelle, Brittany Tracy
Date: 2021-10-27 00:00:00
Tasks: Navigate

Achieving accessibility compliance is extremely important for many government agencies and businesses who wish to improve services for their consumers. With the growing reliance on dynamic web applications many organizations are finding it difficult to implement accessibility standards, often due to the inability of current automated testing tools to test the stateful environments created by dynamic web applications. In this paper, we present mathematical foundations and theory for the Demodocus framework and prototype, and outline its approach to using web science, web crawling,and accessibility testing to automatically navigate and test interactive content for accessibility. Our approach simulates the page interactions of users with and without disabilities, and compares graphs of reachable states from these simulations to determine both the accessibility and the difficulty of content access for these different users.
Don't read, just look: Main content extraction from web pages using visual features
Authors: Geunseong Jung, Sungjae Han, Hansung Kim, Kwanguk Kim, Jaehyuk Cha
Date: 2021-10-27 00:00:00
Tasks: Information Retrieval, Retrieval, TAG

Extracting main content from web pages provides primary informative blocks that remove a web page's minor areas like navigation menu, ads, and site templates. The main content extraction has various applications: information retrieval, search engine optimization, and browser reader mode. We assessed the existing four main content extraction methods (Readability.js, Chrome DOM Distiller, Web2Text, and Boilernet) with the web pages of two English datasets from global websites of 2017 and 2020 and seven non-English datasets by languages of 2020. Its result showed that performance was lower by up to 40% in non-English datasets than in English datasets. Thus, this paper proposes a multilingual main content extraction method using visual features: the elements' positions, size, and distances from three centers. These centers were derived from the browser window, web document, and the first browsing area. We propose this first browsing area, which is the top side of a web document for simulating situations where a user first encountered a web page. Because web page authors placed their main contents in the central area for the web page's usability, we can assume the center of this area is close to the main content. Our grid-centering-expanding (GCE) method suggests the three centroids as hypothetical user foci. Traversing the DOM tree from each of the leaf nodes closest to these centroids, our method inspects which the ancestor node can be the main content candidate. Finally, it extracts the main content by selecting the best among the three main content candidates. Our method performed 14% better than the existing method on average in Longest Common Subsequence F1 score. In particular, it improved performance by up to 25% in the English dataset and 16% in the non-English dataset. Therefore, our method showed the visual and basic HTML features are effective in extracting the main content.
iALS++: Speeding up Matrix Factorization with Subspace Optimization
Authors: Steffen Rendle, Walid Krichene, Li Zhang, Yehuda Koren
Date: 2021-10-26 00:00:00

iALS is a popular algorithm for learning matrix factorization models from implicit feedback with alternating least squares. This algorithm was invented over a decade ago but still shows competitive quality compared to recent approaches like VAE, EASE, SLIM, or NCF. Due to a computational trick that avoids negative sampling, iALS is very efficient especially for large item catalogues. However, iALS does not scale well with large embedding dimensions, d, due to its cubic runtime dependency on d. Coordinate descent variations, iCD, have been proposed to lower the complexity to quadratic in d. In this work, we show that iCD approaches are not well suited for modern processors and can be an order of magnitude slower than a careful iALS implementation for small to mid scale embedding sizes (d ~ 100) and only perform better than iALS on large embeddings d ~ 1000. We propose a new solver iALS++ that combines the advantages of iALS in terms of vector processing with a low computational complexity as in iCD. iALS++ is an order of magnitude faster than iCD both for small and large embedding dimensions. It can solve benchmark problems like Movielens 20M or Million Song Dataset even for 1000 dimensional embedding vectors in a few minutes.
SQALER: Scaling Question Answering by Decoupling Multi-Hop and Logical Reasoning
Authors: Mattia Atzeni, Jasmina Bogojeska, Andreas Loukas
Date: 2021-10-27 00:00:00
Proceeding: NeurIPS 2021 12
Tasks: Knowledge Graphs, Logical Reasoning, Question Answering

State-of-the-art approaches to reasoning and question answering over knowledge graphs (KGs) usually scale with the number of edges and can only be applied effectively on small instance-dependent subgraphs. In this paper, we address this issue by showing that multi-hop and more complex logical reasoning can be accomplished separately without losing expressive power. Motivated by this insight, we propose an approach to multi-hop reasoning that scales linearly with the number of relation types in the graph, which is usually significantly smaller than the number of edges or nodes. This produces a set of candidate solutions that can be provably refined to recover the solution to the original problem. Our experiments on knowledge-based question answering show that our approach solves the multi-hop MetaQA dataset, achieves a new state-of-the-art on the more challenging WebQuestionsSP, is orders of magnitude more scalable than competitive approaches, and can achieve compositional generalization out of the training distribution.
TopicNet: Semantic Graph-Guided Topic Discovery
Authors: Zhibin Duan, Yishi Xu, Bo Chen, Dongsheng Wang, Chaojie Wang, Mingyuan Zhou
Date: 2021-10-27 00:00:00
Proceeding: NeurIPS 2021 12
Tasks: Inductive Bias, Topic Models, Variational Inference

Existing deep hierarchical topic models are able to extract semantically meaningful topics from a text corpus in an unsupervised manner and automatically organize them into a topic hierarchy. However, it is unclear how to incorporate prior beliefs such as knowledge graph to guide the learning of the topic hierarchy. To address this issue, we introduce TopicNet as a deep hierarchical topic model that can inject prior structural knowledge as an inductive bias to influence learning. TopicNet represents each topic as a Gaussian-distributed embedding vector, projects the topics of all layers into a shared embedding space, and explores both the symmetric and asymmetric similarities between Gaussian embedding vectors to incorporate prior semantic hierarchies. With an auto-encoding variational inference network, the model parameters are optimized by minimizing the evidence lower bound and a regularization term via stochastic gradient descent. Experiments on widely used benchmarks show that TopicNet outperforms related deep topic models on discovering deeper interpretable topics and mining better document~representations.