pwc-archive/papers-with-abstracts huggingface dataset:
Refinement of Hottopixx Method for Nonnegative Matrix Factorization Under Noisy Separability
Authors: Tomohiko Mizutani
Date: 2021-09-07 00:00:00

Hottopixx, proposed by Bittorf et al. at NIPS 2012, is an algorithm for solving nonnegative matrix factorization (NMF) problems under the separability assumption. Separable NMFs have important applications, such as topic extraction from documents and unmixing of hyperspectral images. In such applications, the robustness of the algorithm to noise is the key to the success. Hottopixx has been shown to be robust to noise, and its robustness can be further enhanced through postprocessing. However, there is a drawback. Hottopixx and its postprocessing require us to estimate the noise level involved in the matrix we want to factorize before running, since they use it as part of the input data. The noise-level estimation is not an easy task. In this paper, we overcome this drawback. We present a refinement of Hottopixx and its postprocessing that runs without prior knowledge of the noise level. We show that the refinement has almost the same robustness to noise as the original algorithm.
Self-adaptive deep neural network: Numerical approximation to functions and PDEs
Authors: Zhiqiang Cai, Jingshuang Chen, Min Liu
Date: 2021-09-07 00:00:00

Designing an optimal deep neural network for a given task is important and challenging in many machine learning applications. To address this issue, we introduce a self-adaptive algorithm: the adaptive network enhancement (ANE) method, written as loops of the form train, estimate and enhance. Starting with a small two-layer neural network (NN), the step train is to solve the optimization problem at the current NN; the step estimate is to compute a posteriori estimator/indicators using the solution at the current NN; the step enhance is to add new neurons to the current NN. Novel network enhancement strategies based on the computed estimator/indicators are developed in this paper to determine how many new neurons and when a new layer should be added to the current NN. The ANE method provides a natural process for obtaining a good initialization in training the current NN; in addition, we introduce an advanced procedure on how to initialize newly added neurons for a better approximation. We demonstrate that the ANE method can automatically design a nearly minimal NN for learning functions exhibiting sharp transitional layers as well as discontinuous solutions of hyperbolic partial differential equations.
COCO Denoiser: Using Co-Coercivity for Variance Reduction in Stochastic Convex Optimization
Authors: Manuel Madeira, Renato Negrinho, João Xavier, Pedro M. Q. Aguiar
Date: 2021-09-07 00:00:00
Tasks: Stochastic Optimization

First-order methods for stochastic optimization have undeniable relevance, in part due to their pivotal role in machine learning. Variance reduction for these algorithms has become an important research topic. In contrast to common approaches, which rarely leverage global models of the objective function, we exploit convexity and L-smoothness to improve the noisy estimates outputted by the stochastic gradient oracle. Our method, named COCO denoiser, is the joint maximum likelihood estimator of multiple function gradients from their noisy observations, subject to co-coercivity constraints between them. The resulting estimate is the solution of a convex Quadratically Constrained Quadratic Problem. Although this problem is expensive to solve by interior point methods, we exploit its structure to apply an accelerated first-order algorithm, the Fast Dual Proximal Gradient method. Besides analytically characterizing the proposed estimator, we show empirically that increasing the number and proximity of the queried points leads to better gradient estimates. We also apply COCO in stochastic settings by plugging it in existing algorithms, such as SGD, Adam or STRSAGA, outperforming their vanilla versions, even in scenarios where our modelling assumptions are mismatched.
Spectral properties of sample covariance matrices arising from random matrices with independent non identically distributed columns
Authors: Cosme Louart, Romain Couillet
Date: 2021-09-06 00:00:00

Given a random matrix $X= (x_1,\ldots, x_n)\in \mathcal M_{p,n}$ with independent columns and satisfying concentration of measure hypotheses and a parameter $z$ whose distance to the spectrum of $\frac{1}{n} XX^T$ should not depend on $p,n$, it was previously shown that the functionals $\text{tr}(AR(z))$, for $R(z) = (\frac{1}{n}XX^T- zI_p)^{-1}$ and $A\in \mathcal M_{p}$ deterministic, have a standard deviation of order $O(\|A\|_* / \sqrt n)$. Here, we show that $\|\mathbb E[R(z)] - \tilde R(z)\|_F \leq O(1/\sqrt n)$, where $\tilde R(z)$ is a deterministic matrix depending only on $z$ and on the means and covariances of the column vectors $x_1,\ldots, x_n$ (that do not have to be identically distributed). This estimation is key to providing accurate fluctuation rates of functionals of $X$ of interest (mostly related to its spectral properties) and is proved thanks to the introduction of a semi-metric $d_s$ defined on the set $\mathcal D_n(\mathbb H)$ of diagonal matrices with complex entries and positive imaginary part and satisfying, for all $D,D' \in \mathcal D_n(\mathbb H)$: $d_s(D,D') = \max_{i\in[n]} |D_i - D_i'|/ (\Im(D_i) \Im(D_i'))^{1/2}$. Possibly most importantly, the underlying concentration of measure assumption on the columns of $X$ finds an extremely natural ground for application in modern statistical machine learning algorithms where non-linear Lipschitz mappings and high number of classes form the base ingredients.
Adaptive variational Bayes: Optimality, computation and applications
Authors: Ilsang Ohn, Lizhen Lin
Date: 2021-09-07 00:00:00
Tasks: Model Selection

In this paper, we explore adaptive inference based on variational Bayes. Although several studies have been conducted to analyze the contraction properties of variational posteriors, there is still a lack of a general and computationally tractable variational Bayes method that performs adaptive inference. To fill this gap, we propose a novel adaptive variational Bayes framework, which can operate on a collection of models. The proposed framework first computes a variational posterior over each individual model separately and then combines them with certain weights to produce a variational posterior over the entire model. It turns out that this combined variational posterior is the closest member to the posterior over the entire model in a predefined family of approximating distributions. We show that the adaptive variational Bayes attains optimal contraction rates adaptively under very general conditions. We also provide a methodology to maintain the tractability and adaptive optimality of the adaptive variational Bayes even in the presence of an enormous number of individual models, such as sparse models. We apply the general results to several examples, including deep learning and sparse factor models, and derive new and adaptive inference results. In addition, we characterize an implicit regularization effect of variational Bayes and show that the adaptive variational posterior can utilize this.
Mutation frequency time series reveal complex mixtures of clones in the world-wide SARS-CoV-2 viral population
Authors: Hong-Li Zeng, Yue Liu, Vito Dichio, Kaisa Thorell, Rickard Nordén, Erik Aurell
Date: 2021-09-07 00:00:00
Tasks: Time Series, Time Series Analysis

We compute the allele frequencies of the alpha (B.1.1.7), beta (B.1.351) and delta (B.167.2) variants of SARS-CoV-2 from almost two million genome sequences on the GISAID repository. We find that the frequencies of a majority of the defining mutations in alpha rose towards the end of 2020 but drifted apart during spring 2021, a similar pattern being followed by delta during summer of 2021. For beta we find a more complex scenario with frequencies of some mutations rising and some remaining close to zero. Our results point to that what is generally reported as single variants is in fact a collection of variants with different genetic characteristics. For all three variants we further find some alleles with a clearly deviating time series.
Core motifs predict dynamic attractors in combinatorial threshold-linear networks
Authors: Caitlyn Parmelee, Samantha Moore, Katherine Morrison, Carina Curto
Date: 2021-09-06 00:00:00

Combinatorial threshold-linear networks (CTLNs) are a special class of inhibition-dominated TLNs defined from directed graphs. Like more general TLNs, they display a wide variety of nonlinear dynamics including multistability, limit cycles, quasiperiodic attractors, and chaos. In prior work, we have developed a detailed mathematical theory relating stable and unstable fixed points of CTLNs to graph-theoretic properties of the underlying network. Here we find that a special type of fixed points, corresponding to core motifs, are predictive of both static and dynamic attractors. Moreover, the attractors can be found by choosing initial conditions that are small perturbations of these fixed points. This motivates us to hypothesize that dynamic attractors of a network correspond to unstable fixed points supported on core motifs. We tested this hypothesis on a large family of directed graphs of size $n=5$, and found remarkable agreement. Furthermore, we discovered that core motifs with similar embeddings give rise to nearly identical attractors. This allowed us to classify attractors based on structurally-defined graph families. Our results suggest that graphical properties of the connectivity can be used to predict a network's complex repertoire of nonlinear dynamics.
Statistical analysis of locally parameterized shapes
Authors: Mohsen Taheri, Jörn Schulz
Date: 2021-08-18 00:00:00
Tasks: Translation

The alignment of shapes has been a crucial step in statistical shape analysis, for example, in calculating mean shape, detecting locational differences between two shape populations, and classification. Procrustes alignment is the most commonly used method and state of the art. In this work, we uncover that alignment might seriously affect the statistical analysis. For example, alignment can induce false shape differences and lead to misleading results and interpretations. We propose a novel hierarchical shape parameterization based on local coordinate systems. The local parameterized shapes are translation and rotation invariant. Thus, the inherent alignment problems from the commonly used global coordinate system for shape representation can be avoided using this parameterization. The new parameterization is also superior for shape deformation and simulation. The method's power is demonstrated on the hypothesis testing of simulated data as well as the left hippocampi of patients with Parkinson's disease and controls.
Characterizing interdisciplinarity in drug research: a translational science perspective
Authors: Xin Li, Xuli Tang
Date: 2021-09-04 00:00:00
Tasks: Drug Discovery

Despite the significant advances in life science, it still takes decades to translate a basic drug discovery into a cure for human disease. To accelerate the process from bench to bedside, interdisciplinary research (especially research involving both basic research and clinical research) has been strongly recommend by many previous studies. However, the patterns and the roles of the interdisciplinary characteristics in drug research have not been deeply examined in extant studies. The purpose of this study was to characterize interdisciplinary characteristics in drug research from the perspective of translational science, and to examine the role of different kinds of interdisciplinary characteristics in translational research for drugs.
A Scalable AI Approach for Clinical Trial Cohort Optimization
Authors: Xiong Liu, Cheng Shi, Uday Deore, Yingbo Wang, Myah Tran, Iya Khalil, Murthy Devarakonda
Date: 2021-09-07 00:00:00
Tasks: Diversity

FDA has been promoting enrollment practices that could enhance the diversity of clinical trial populations, through broadening eligibility criteria. However, how to broaden eligibility remains a significant challenge. We propose an AI approach to Cohort Optimization (AICO) through transformer-based natural language processing of the eligibility criteria and evaluation of the criteria using real-world data. The method can extract common eligibility criteria variables from a large set of relevant trials and measure the generalizability of trial designs to real-world patients. It overcomes the scalability limits of existing manual methods and enables rapid simulation of eligibility criteria design for a disease of interest. A case study on breast cancer trial design demonstrates the utility of the method in improving trial generalizability.
Net Buying Pressure and the Information in Bitcoin Option Trades
Authors: Carol Alexander, Jun Deng, Jianfen Feng, Huning Wan
Date: 2021-09-06 00:00:00

How do supply and demand from informed traders drive market prices of bitcoin options? Deribit options tick-level data supports the limits-to-arbitrage hypothesis about the market maker's supply. The main demand-side effects are that at-the-money option prices are largely driven by volatility traders and out-of-the-money options are simultaneously driven by volatility traders and those with proprietary information about the direction of future bitcoin price movements. The demand-side trading results contrast with prior studies on established options markets in the US and Asia, but we also show that Deribit is rapidly evolving into a more efficient channel for aggregating information from informed traders.
Examining the Dynamic Asset Market Linkages under the COVID-19 Global Pandemic
Authors: Akihiko Noda
Date: 2021-09-07 00:00:00

This study examines the dynamic asset market linkages under the COVID-19 global pandemic based on market efficiency, in the sense of Fama (1970). Particularly, we estimate the joint degree of market efficiency by applying Ito et al.'s (2014; 2017) Generalized Least Squares-based time-varying vector autoregression model. The empirical results show that (1) the joint degree of market efficiency changes widely over time, as shown in Lo's (2004) adaptive market hypothesis, (2) the COVID-19 pandemic may eliminate arbitrage and improve market efficiency through enhanced linkages between the asset markets; and (3) the market efficiency has continued to decline due to the Bitcoin bubble that emerged at the end of 2020.
Moment Matching Method for Pricing Spread Options with Mean-Variance Mixture Lévy Motions
Authors: Dongdong Hu, Hasanjan Sayit, Svetlozar T. Rachev
Date: 2021-09-07 00:00:00
Tasks: Form

The paper Borovkova et al. [4] uses moment matching method to obtain closed form formulas for spread and basket call option prices under log normal models. In this note, we also use moment matching method to obtain semi-closed form formulas for the price of spread options under exponential L\'evy models with mean-variance mixture. Unlike the semi-closed form formulas in Caldana and Fusai [5], where spread prices were expressed by using Fourier inversion formula for general price dynamics, our formula expresses spread prices in terms of the mixing distribution. Numerical tests show that our formulas give accurate spread prices also
Crash Report Data Analysis for Creating Scenario-Wise, Spatio-Temporal Attention Guidance to Support Computer Vision-based Perception of Fatal Crash Risks
Authors: Yu Li, Muhammad Monjurul Karim, Ruwen Qin
Date: 2021-09-06 00:00:00
Tasks: Clustering

Reducing traffic fatalities and serious injuries is a top priority of the US Department of Transportation. The computer vision (CV)-based crash anticipation in the near-crash phase is receiving growing attention. The ability to perceive fatal crash risks earlier is also critical because it will improve the reliability of crash anticipation. Yet, annotated image data for training a reliable AI model for the early visual perception of crash risks are not abundant. The Fatality Analysis Reporting System contains big data of fatal crashes. It is a reliable data source for learning the relationship between driving scene characteristics and fatal crashes to compensate for the limitation of CV. Therefore, this paper develops a data analytics model, named scenario-wise, Spatio-temporal attention guidance, from fatal crash report data, which can estimate the relevance of detected objects to fatal crashes from their environment and context information. First, the paper identifies five sparse variables that allow for decomposing the 5-year fatal crash dataset to develop scenario-wise attention guidance. Then, exploratory analysis of location- and time-related variables of the crash report data suggests reducing fatal crashes to spatially defined groups. The group's temporal pattern is an indicator of the similarity of fatal crashes in the group. Hierarchical clustering and K-means clustering merge the spatially defined groups into six clusters according to the similarity of their temporal patterns. After that, association rule mining discovers the statistical relationship between the temporal information of driving scenes with crash features, for each cluster. The paper shows how the developed attention guidance supports the design and implementation of a preliminary CV model that can identify objects of a possibility to involve in fatal crashes from their environment and context information.
On the Out-of-distribution Generalization of Probabilistic Image Modelling
Authors: Mingtian Zhang, Andi Zhang, Steven McDonagh
Date: 2021-09-04 00:00:00
Proceeding: NeurIPS 2021 12
Tasks: Out-of-Distribution Generalization, Out of Distribution (OOD) Detection

Out-of-distribution (OOD) detection and lossless compression constitute two problems that can be solved by the training of probabilistic models on a first dataset with subsequent likelihood evaluation on a second dataset, where data distributions differ. By defining the generalization of probabilistic models in terms of likelihood we show that, in the case of image models, the OOD generalization ability is dominated by local features. This motivates our proposal of a Local Autoregressive model that exclusively models local image features towards improving OOD performance. We apply the proposed model to OOD detection tasks and achieve state-of-the-art unsupervised OOD detection performance without the introduction of additional data. Additionally, we employ our model to build a new lossless image compressor: NeLLoC (Neural Local Lossless Compressor) and report state-of-the-art compression rates and model size.
An Empirical Study on Few-shot Knowledge Probing for Pretrained Language Models
Authors: Tianxing He, Kyunghyun Cho, James Glass
Date: 2021-09-06 00:00:00
Tasks: Knowledge Probing, Prompt Engineering, World Knowledge

Prompt-based knowledge probing for 1-hop relations has been used to measure how much world knowledge is stored in pretrained language models. Existing work uses considerable amounts of data to tune the prompts for better performance. In this work, we compare a variety of approaches under a few-shot knowledge probing setting, where only a small number (e.g., 10 or 20) of example triples are available. In addition, we create a new dataset named TREx-2p, which contains 2-hop relations. We report that few-shot examples can strongly boost the probing performance for both 1-hop and 2-hop relations. In particular, we find that a simple-yet-effective approach of finetuning the bias vectors in the model outperforms existing prompt-engineering methods. Our dataset and code are available at \url{https://github.com/cloudygoose/fewshot_lama}.
A new neighborhood structure for job shop scheduling problems
Authors: Jin Xie, Xinyu Li, Liang Gao, Lin Gui
Date: 2021-09-07 00:00:00
Tasks: Combinatorial Optimization, Job Shop Scheduling, Scheduling

Job shop scheduling problem (JSP) is a widely studied NP-complete combinatorial optimization problem. Neighborhood structures play a critical role in solving JSP. At present, there are three state-of-the-art neighborhood structures, i.e., N5, N6, and N7. Improving the upper bounds of some famous benchmarks is inseparable from the role of these neighborhood structures. However, these existing neighborhood structures only consider the movement of critical operations within a critical block. According to our experiments, it is also possible to improve the makespan of a scheduling scheme by moving a critical operation outside its critical block. According to the above finding, this paper proposes a new N8 neighborhood structure considering the movement of critical operations within a critical block and the movement of critical operations outside the critical block. Besides, a neighborhood clipping method is designed to avoid invalid movement, reducing the computational time. Tabu search (TS) is a commonly used algorithm framework combined with neighborhood structures. This paper uses this framework to compare the N8 neighborhood structure with N5, N6, and N7 neighborhood structures on four famous benchmarks. The experimental results verify that the N8 neighborhood structure is more effective and efficient in solving JSP than the other state-of-the-art neighborhood structures.
Readying Medical Students for Medical AI: The Need to Embed AI Ethics Education
Authors: Thomas P Quinn, Simon Coghlan
Date: 2021-09-07 00:00:00
Tasks: Ethics

Medical students will almost inevitably encounter powerful medical AI systems early in their careers. Yet, contemporary medical education does not adequately equip students with the basic clinical proficiency in medical AI needed to use these tools safely and effectively. Education reform is urgently needed, but not easily implemented, largely due to an already jam-packed medical curricula. In this article, we propose an education reform framework as an effective and efficient solution, which we call the Embedded AI Ethics Education Framework. Unlike other calls for education reform to accommodate AI teaching that are more radical in scope, our framework is modest and incremental. It leverages existing bioethics or medical ethics curricula to develop and deliver content on the ethical issues associated with medical AI, especially the harms of technology misuse, disuse, and abuse that affect the risk-benefit analyses at the heart of healthcare. In doing so, the framework provides a simple tool for going beyond the "What?" and the "Why?" of medical AI ethics education, to answer the "How?", giving universities, course directors, and/or professors a broad road-map for equipping their students with the necessary clinical proficiency in medical AI.