pwc-archive/papers-with-abstracts huggingface dataset:
Any Part of Bayesian Network Structure Learning
Authors: Zhaolong Ling, Kui Yu, Hao Wang, Lin Liu, Jiuyong Li
Date: 2021-03-23 00:00:00
Tasks: feature selection

We study an interesting and challenging problem, learning any part of a Bayesian network (BN) structure. In this challenge, it will be computationally inefficient using existing global BN structure learning algorithms to find an entire BN structure to achieve the part of a BN structure in which we are interested. And local BN structure learning algorithms encounter the false edge orientation problem when they are directly used to tackle this challenging problem. In this paper, we first present a new concept of Expand-Backtracking to explain why local BN structure learning methods have the false edge orientation problem, then propose APSL, an efficient and accurate Any Part of BN Structure Learning algorithm. Specifically, APSL divides the V-structures in a Markov blanket (MB) into two types: collider V-structure and non-collider V-structure, then it starts from a node of interest and recursively finds both collider V-structures and non-collider V-structures in the found MBs, until the part of a BN structure in which we are interested are oriented. To improve the efficiency of APSL, we further design the APSL-FS algorithm using Feature Selection, APSL-FS. Using six benchmark BNs, the extensive experiments have validated the efficiency and accuracy of our methods.
Student Network Learning via Evolutionary Knowledge Distillation
Authors: Kangkai Zhang, Chunhui Zhang, Shikun Li, Dan Zeng, Shiming Ge
Date: 2021-03-23 00:00:00
Tasks: Knowledge Distillation, Transfer Learning

Knowledge distillation provides an effective way to transfer knowledge via teacher-student learning, where most existing distillation approaches apply a fixed pre-trained model as teacher to supervise the learning of student network. This manner usually brings in a big capability gap between teacher and student networks during learning. Recent researches have observed that a small teacher-student capability gap can facilitate knowledge transfer. Inspired by that, we propose an evolutionary knowledge distillation approach to improve the transfer effectiveness of teacher knowledge. Instead of a fixed pre-trained teacher, an evolutionary teacher is learned online and consistently transfers intermediate knowledge to supervise student network learning on-the-fly. To enhance intermediate knowledge representation and mimicking, several simple guided modules are introduced between corresponding teacher-student blocks. In this way, the student can simultaneously obtain rich internal knowledge and capture its growth process, leading to effective student network learning. Extensive experiments clearly demonstrate the effectiveness of our approach as well as good adaptability in the low-resolution and few-sample visual recognition scenarios.
RA-BNN: Constructing Robust & Accurate Binary Neural Network to Simultaneously Defend Adversarial Bit-Flip Attack and Improve Accuracy
Authors: Adnan Siraj Rakin, Li Yang, Jingtao Li, Fan Yao, Chaitali Chakrabarti, Yu Cao, Jae-sun Seo, Deliang Fan
Date: 2021-03-22 00:00:00

Recently developed adversarial weight attack, a.k.a. bit-flip attack (BFA), has shown enormous success in compromising Deep Neural Network (DNN) performance with an extremely small amount of model parameter perturbation. To defend against this threat, we propose RA-BNN that adopts a complete binary (i.e., for both weights and activation) neural network (BNN) to significantly improve DNN model robustness (defined as the number of bit-flips required to degrade the accuracy to as low as a random guess). However, such an aggressive low bit-width model suffers from poor clean (i.e., no attack) inference accuracy. To counter this, we propose a novel and efficient two-stage network growing method, named Early-Growth. It selectively grows the channel size of each BNN layer based on channel-wise binary masks training with Gumbel-Sigmoid function. Apart from recovering the inference accuracy, our RA-BNN after growing also shows significantly higher resistance to BFA. Our evaluation of the CIFAR-10 dataset shows that the proposed RA-BNN can improve the clean model accuracy by ~2-8 %, compared with a baseline BNN, while simultaneously improving the resistance to BFA by more than 125 x. Moreover, on ImageNet, with a sufficiently large (e.g., 5,000) amount of bit-flips, the baseline BNN accuracy drops to 4.3 % from 51.9 %, while our RA-BNN accuracy only drops to 37.1 % from 60.9 % (9 % clean accuracy improvement).
Supervised Contrastive Replay: Revisiting the Nearest Class Mean Classifier in Online Class-Incremental Continual Learning
Authors: Zheda Mai, Ruiwen Li, Hyunwoo Kim, Scott Sanner
Date: 2021-03-22 00:00:00
Tasks: class-incremental learning, Class Incremental Learning, Continual Learning

Online class-incremental continual learning (CL) studies the problem of learning new classes continually from an online non-stationary data stream, intending to adapt to new data while mitigating catastrophic forgetting. While memory replay has shown promising results, the recency bias in online learning caused by the commonly used Softmax classifier remains an unsolved challenge. Although the Nearest-Class-Mean (NCM) classifier is significantly undervalued in the CL community, we demonstrate that it is a simple yet effective substitute for the Softmax classifier. It addresses the recency bias and avoids structural changes in the fully-connected layer for new classes. Moreover, we observe considerable and consistent performance gains when replacing the Softmax classifier with the NCM classifier for several state-of-the-art replay methods. To leverage the NCM classifier more effectively, data embeddings belonging to the same class should be clustered and well-separated from those with a different class label. To this end, we contribute Supervised Contrastive Replay (SCR), which explicitly encourages samples from the same class to cluster tightly in embedding space while pushing those of different classes further apart during replay-based training. Overall, we observe that our proposed SCR substantially reduces catastrophic forgetting and outperforms state-of-the-art CL methods by a significant margin on a variety of datasets.
A Novel Methodology For Crowdsourcing AI Models in an Enterprise
Authors: Parthasarathy Suryanarayanan, Sundar Saranathan, Shilpa Mahatma, Divya Pathak
Date: 2021-03-22 00:00:00

The evolution of AI is advancing rapidly, creating both challenges and opportunities for industry-community collaboration. In this work, we present a novel methodology aiming to facilitate this collaboration through crowdsourcing of AI models. Concretely, we have implemented a system and a process that any organization can easily adopt to host AI competitions. The system allows them to automatically harvest and evaluate the submitted models against in-house proprietary data and also to incorporate them as reusable services in a product.
Dynamic Weighted Learning for Unsupervised Domain Adaptation
Authors: Ni Xiao, Lei Zhang
Date: 2021-03-22 00:00:00
Proceeding: CVPR 2021 1
Tasks: Domain Adaptation, Unsupervised Domain Adaptation

Unsupervised domain adaptation (UDA) aims to improve the classification performance on an unlabeled target domain by leveraging information from a fully labeled source domain. Recent approaches explore domain-invariant and class-discriminant representations to tackle this task. These methods, however, ignore the interaction between domain alignment learning and class discrimination learning. As a result, the missing or inadequate tradeoff between domain alignment and class discrimination are prone to the problem of negative transfer. In this paper, we propose Dynamic Weighted Learning (DWL) to avoid the discriminability vanishing problem caused by excessive alignment learning and domain misalignment problem caused by excessive discriminant learning. Technically, DWL dynamically weights the learning losses of alignment and discriminability by introducing the degree of alignment and discriminability. Besides, the problem of sample imbalance across domains is first considered in our work, and we solve the problem by weighing the samples to guarantee information balance across domains. Extensive experiments demonstrate that DWL has an excellent performance in several benchmark datasets.
Fast Approximate Spectral Normalization for Robust Deep Neural Networks
Authors: Zhixin Pan, Prabhat Mishra
Date: 2021-03-22 00:00:00

Deep neural networks (DNNs) play an important role in machine learning due to its outstanding performance compared to other alternatives. However, DNNs are not suitable for safety-critical applications since DNNs can be easily fooled by well-crafted adversarial examples. One promising strategy to counter adversarial attacks is to utilize spectral normalization, which ensures that the trained model has low sensitivity towards the disturbance of input samples. Unfortunately, this strategy requires exact computation of spectral norm, which is computation intensive and impractical for large-scale networks. In this paper, we introduce an approximate algorithm for spectral normalization based on Fourier transform and layer separation. The primary contribution of our work is to effectively combine the sparsity of weight matrix and decomposability of convolution layers. Extensive experimental evaluation demonstrates that our framework is able to significantly improve both time efficiency (up to 60\%) and model robustness (61\% on average) compared with the state-of-the-art spectral normalization.
Conditional inference on the asset with maximum Sharpe ratio
Authors: Steven E. Pav
Date: 2019-06-03 00:00:00

We apply the procedure of Lee et al. to the problem of performing inference on the signal-noise ratio of the asset which displays maximum sample Sharpe ratio over a set of possibly correlated assets. We find a multivariate analogue of the commonly used approximate standard error of the Sharpe ratio to use in this conditional estimation procedure. We also consider several alternative procedures, including the simple Bonferroni correction for multiple hypothesis testing, which we fix for the case of positive common correlation among assets, the chi-bar square test against one-sided alternatives, Follman's test, and Hansen's asymptotic adjustments. Testing indicates the conditional inference procedure achieves nominal type I rate, and does not appear to suffer from non-normality of returns. The conditional estimation test has low power under the alternative where there is little spread in the signal-noise ratios of the assets, and high power under the alternative where a single asset has high signal-noise ratio. Unlike the alternative procedures, it appears to enjoy rejection probabilities montonic in the signal-noise ratio of the selected asset.
A Noise-Aware Memory-Attention Network Architecture for Regression-Based Speech Enhancement
Authors: Yu-Xuan Wang, Jun Du, Li Chai, Chin-Hui Lee, Jia Pan
Date: 2020-10-25 00:00:00
Tasks: regression, Speech Enhancement

We propose a novel noise-aware memory-attention network (NAMAN) for regression-based speech enhancement, aiming at improving quality of enhanced speech in unseen noise conditions. The NAMAN architecture consists of three parts, a main regression network, a memory block and an attention block. First, a long short-term memory recurrent neural network (LSTM-RNN) is adopted as the main network to well model the acoustic context of neighboring frames. Next, the memory block is built with an extensive set of noise feature vectors as the prior noise bases. Finally, the attention block serves as an auxiliary network to improve the noise awareness of the main network by encoding the dynamic noise information at frame level through additional features obtained by weighing the existing noise basis vectors in the memory block. Our experiments show that the proposed NAMAN framework is compact and outperforms the state-of-the-art dynamic noise-aware training approaches in low SNR conditions.
Scaled relative graphs for system analysis
Authors: Thomas Chaffey, Fulvio Forni, Rodolphe Sepulchre
Date: 2021-03-25 00:00:00

Scaled relative graphs were recently introduced to analyze the convergence of optimization algorithms using two dimensional Euclidean geometry. In this paper, we connect scaled relative graphs to the classical theory of input/output systems. It is shown that the Nyquist diagram of an LTI system on $L_2$ is the convex hull of its scaled relative graph under a particular change of coordinates. The SRG may be used to visualize approximations of static nonlinearities such as the describing function and quadratic constraints, allowing system properties to be verified or disproved. Interconnections of systems correspond to graphical manipulations of their SRGs. This is used to provide a simple, graphical proof of the classical incremental passivity theorem.
Putting a price on tenure
Authors: Thiago Marzagao
Date: 2021-03-25 00:00:00

Government employees in Brazil are granted tenure after three years on the job. Firing a tenured government employee is all but impossible, so tenure is a big employee benefit. But exactly how big is it? In other words: how much money is tenure worth to a government employee in Brazil? No one has ever attempted to answer that question. I do that in this paper. I use a modified version of the Sharpe ratio to estimate what the risk-adjusted salaries of government workers should be. The difference between actual salary and risk-adjusted salary gives us an estimate of how much tenure is worth to each employee. I find that in the 2005-2019 period the monthly value of tenure was 3980 reais to the median federal government employee, 1971 reais to the median state government employee, and 500 reais to the median municipal government employee.
Nonlinear Estimation for Position-Aided Inertial Navigation Systems
Authors: Soulaimane Berkane, Abdelhamid Tayebi
Date: 2021-03-25 00:00:00
Tasks: Position

In this work we solve the position-aided 3D navigation problem using a nonlinear estimation scheme. More precisely, we propose a nonlinear observer to estimate the full state of the vehicle (position, velocity, orientation and gyro bias) from IMU and position measurements. The proposed observer does not introduce additional auxiliary states and is shown to guarantee semi-global exponential stability without any assumption on the acceleration of the vehicle. The performance of the observer is shown, through simulation, to overcome the state-of-the-art approach that assumes negligible accelerations.
Comment on Buss et al., Science 2021: An alternative, empirically-supported adjustment for sero-reversion yields a 10 percentage point lower estimate of the cumulative incidence of SARS-CoV-2 in Manaus by October 2020
Authors: Sarah Kadelka, Judith A Bouman, Peter Ashcroft, Roland R Regoes
Date: 2021-03-25 00:00:00

The estimate of the cumulative incidence of SARS-CoV-2 of 76% in Manaus by October 2020 by Buss et al. relies on the assumption of an exponentially-declining probability of sero-reversion over time. We present an alternative, empirically-supported approach that is based on the observed dynamics of antibody titers in sero-positive cases. Through this approach we revise the cumulative incidence estimate to 66% (63.3% - 68.5%) by October 2020. This estimate has implications for the proximity to herd immunity and future estimates of fitness advantages of virus variants, such as the P.1 variant of concern. This methodology is also relevant for any other sero-survey analysis that has to adjust for sero-reversion.
If Loud Aliens Explain Human Earliness, Quiet Aliens Are Also Rare
Authors: Robin Hanson, Daniel Martin, Calvin Mccarter, Jonathan Paulson
Date: 2021-02-01 00:00:00

If life on Earth had to achieve n 'hard steps' to reach humanity's level, then the chance of this event rose as time to the n-th power. Integrating this over habitable star formation and planet lifetime distributions predicts >99% of advanced life appears after today, unless n<3 and max planet duration <50Gyr. That is, we seem early. We offer this explanation: a deadline is set by 'loud' aliens who are born according to a hard steps power law, expand at a common rate, change their volumes' appearances, and prevent advanced life like us from appearing in their volumes. 'Quiet' aliens, in contrast, are much harder to see. We fit this three-parameter model of loud aliens to data: 1) birth power from the number of hard steps seen in Earth history, 2) birth constant by assuming a inform distribution over our rank among loud alien birth dates, and 3) expansion speed from our not seeing alien volumes in our sky. We estimate that loud alien civilizations now control 40-50% of universe volume, each will later control ~10^5 - 3x10^7 galaxies, and we could meet them in ~200Myr - 2Gyr. If loud aliens arise from quiet ones, a depressingly low transition chance (~10^-4) is required to expect that even one other quiet alien civilization has ever been active in our galaxy. Which seems bad news for SETI. But perhaps alien volume appearances are subtle, and their expansion speed lower, in which case we predict many long circular arcs to find in our sky.
A novel weighted approach for time series forecasting based on visibility graph
Authors: Tianxiang Zhan, Fuyuan Xiao
Date: 2021-03-14 00:00:00
Tasks: Time Series, Time Series Analysis, Time Series Forecasting

Time series has attracted a lot of attention in many fields today. Time series forecasting algorithm based on complex network analysis is a research hotspot. How to use time series information to achieve more accurate forecasting is a problem. To solve this problem, this paper proposes a weighted network forecasting method to improve the forecasting accuracy. Firstly, the time series will be transformed into a complex network, and the similarity between nodes will be found. Then, the similarity will be used as a weight to make weighted forecasting on the predicted values produced by different nodes. Compared with the previous method, the proposed method is more accurate. In order to verify the effect of the proposed method, the experimental part is tested on M1, M3 datasets and Construction Cost Index (CCI) dataset, which shows that the proposed method has more accurate forecasting performance.
Impact of the COVID-19 outbreak on Italy's country reputation and stock market performance: a sentiment analysis approach
Authors: Gianpaolo Zammarchi, Francesco Mola, Claudio Conversano
Date: 2021-03-13 00:00:00
Tasks: Sentiment Analysis

During the recent Coronavirus disease 2019 (COVID-19) outbreak, the microblogging service Twitter has been widely used to share opinions and reactions to events. Italy was one of the first European countries to be severely affected by the outbreak and to establish lockdown and stay-at-home orders, potentially leading to country reputation damage. We resort to sentiment analysis to investigate changes in opinions about Italy reported on Twitter before and after the COVID-19 outbreak. Using different lexicons-based methods, we find a breakpoint corresponding to the date of the first established case of COVID-19 in Italy that causes a relevant change in sentiment scores used as proxy of the country reputation. Next, we demonstrate that sentiment scores about Italy are strongly associated with the levels of the FTSE-MIB index, the Italian Stock Exchange main index, as they serve as early detection signals of changes in the values of FTSE-MIB. Finally, we make a content-based classification of tweets into positive and negative and use two machine learning classifiers to validate the assigned polarity of tweets posted before and after the outbreak.
A Physics-Informed Neural Network Framework For Partial Differential Equations on 3D Surfaces: Time-Dependent Problems
Authors: Zhiwei Fang, Justin Zhang, Xiu Yang
Date: 2021-03-19 00:00:00

In this paper, we show a physics-informed neural network solver for the time-dependent surface PDEs. Unlike the traditional numerical solver, no extension of PDE and mesh on the surface is needed. We show a simplified prior estimate of the surface differential operators so that PINN's loss value will be an indicator of the residue of the surface PDEs. Numerical experiments verify efficacy of our algorithm.
Data-driven Aerodynamic Analysis of Structures using Gaussian Processes
Authors: Igor Kavrakov, Allan McRobie, Guido Morgenthal
Date: 2021-03-20 00:00:00
Tasks: Gaussian Processes

An abundant amount of data gathered during wind tunnel testing and health monitoring of structures inspires the use of machine learning methods to replicate the wind forces. This paper presents a data-driven Gaussian Process-Nonlinear Finite Impulse Response (GP-NFIR) model of the nonlinear self-excited forces acting on structures. Constructed in a nondimensional form, the model takes the effective wind angle of attack as lagged exogenous input and outputs a probability distribution of the forces. The nonlinear input/output function is modeled by a GP regression. Consequently, the model is nonparametric, thereby circumventing to set up the function's structure a priori. The training input is designed as random harmonic motion consisting of vertical and rotational displacements. Once trained, the model can predict the aerodynamic forces for both prescribed input motion and aeroelastic analysis. The concept is first verified for a flat plate's analytical solution by predicting the self-excited forces and flutter velocity. Finally, the framework is applied to a streamlined and bluff bridge deck based on Computational Fluid Dynamics (CFD) data. The model's ability to predict nonlinear aerodynamic forces, flutter velocity, and post-flutter behavior are highlighted. Applications of the framework are foreseen in the structural analysis during the design and monitoring of slender line-like structures.
Financial Aspect and Sentiment Predictions with Deep Neural Networks: An Ensemble Approach
Authors: Guangyuan Piao; John G. Breslin
Date: 2018-04-01 00:00:00
Tasks: regression, Sentiment Analysis

In this paper, we describe our ensemble approach for sentiment
and aspect predictions in the financial domain for a given text. This
ensemble approach uses Convolutional Neural Networks (CNNs)
and Recurrent Neural Networks (RNNs) with a ridge regression
and a voting strategy for sentiment and aspect predictions, and
therefore, does not rely on any handcrafted feature. Based on 5-cross
validation on the released training set, the results show that CNNs
overall perform better than RNNs on both tasks, and the ensemble
approach can boost the performance further by leveraging different
types of deep learning approaches.