pwc-archive/papers-with-abstracts huggingface dataset:
A Quantum-like Model for Predicting Human Decisions in the Entangled Social Systems
Authors: Aghdas. Meghdadi, M. R. Akbarzadeh-T., Kourosh Javidan
Date: 2021-11-27 00:00:00
Tasks: Decision Making

Human-centered systems of systems such as social networks, Internet of Things, or healthcare systems are growingly becoming major facets of modern life. Realistic models of human behavior in such systems play a significant role in their accurate modeling and prediction. Yet, human behavior under uncertainty often violates the predictions by the conventional probabilistic models. Recently, quantum-like decision theories have shown a considerable potential to explain the contradictions in human behavior by applying quantum probability. But providing a quantum-like decision theory that could predict, rather than describe the current, state of human behavior is still one of the unsolved challenges. The main novelty of our approach is introducing an entangled Bayesian network inspired by the entanglement concept in quantum information theory, in which each human is a part of the entire society. Accordingly, society's effect on the dynamic evolution of the decision-making process, which is less often considered in decision theories, is modeled by the entanglement measures. The proposed predictive entangled quantum-like Bayesian network (PEQBN) is evaluated on 22 experimental tasks. Results confirm that PEQBN provides more realistic predictions of human decisions under uncertainty, when compared with classical Bayesian networks and three recent quantum-like approaches.
Towards Understanding the Impact of Model Size on Differential Private Classification
Authors: Yinchen Shen, Zhiguo Wang, Ruoyu Sun, Xiaojing Shen
Date: 2021-11-27 00:00:00
Tasks: feature selection, Privacy Preserving

Differential privacy (DP) is an essential technique for privacy-preserving. It was found that a large model trained for privacy preserving performs worse than a smaller model (e.g. ResNet50 performs worse than ResNet18). To better understand this phenomenon, we study high dimensional DP learning from the viewpoint of generalization. Theoretically, we show that for the simple Gaussian model with even small DP noise, if the dimension is large enough, then the classification error can be as bad as the random guessing. Then we propose a feature selection method to reduce the size of the model, based on a new metric which trades off the classification accuracy and privacy preserving. Experiments on real data support our theoretical results and demonstrate the advantage of the proposed method.
Automated Antenna Testing Using Encoder-Decoder-based Anomaly Detection
Authors: Hans Hao-Hsun Hsu, Jiawen Xu, Ravi Sama, Matthias Kovatsch
Date: 2021-11-27 00:00:00
Tasks: Anomaly Detection, Decoder, Time Series Analysis

We propose a new method for testing antenna arrays that records the radiating electromagnetic (EM) field using an absorbing material and evaluating the resulting thermal image series through an AI using a conditional encoder-decoder model. Given the power and phase of the signals fed into each array element, we are able to reconstruct normal sequences through our trained model and compare it to the real sequences observed by a thermal camera. These thermograms only contain low-level patterns such as blobs of various shapes. A contour-based anomaly detector can then map the reconstruction error matrix to an anomaly score to identify faulty antenna arrays and increase the classification F-measure (F-M) by up to 46%. We show our approach on the time series thermograms collected by our antenna testing system. Conventionally, a variational autoencoder (VAE) learning observation noise may yield better results than a VAE with a constant noise assumption. However, we demonstrate that this is not the case for anomaly detection on such low-level patterns for two reasons. First, the baseline metric reconstruction probability, which incorporates the learned observation noise, fails to differentiate anomalous patterns. Second, the area under the receiver operating characteristic (ROC) curve of a VAE with a lower observation noise assumption achieves 11.83% higher than that of a VAE with learned noise.
ML-based Handover Prediction and AP Selection in Cognitive Wi-Fi Networks
Authors: Muhammad Asif Khan, Ridha Hamila, Adel Gastli, Serkan Kiranyaz, Nasser Ahmed Al-Emadi
Date: 2021-11-27 00:00:00

Device mobility in dense Wi-Fi networks offers several challenges. Two well-known problems related to device mobility are handover prediction and access point selection. Due to the complex nature of the radio environment, analytical models may not characterize the wireless channel, which makes the solution of these problems very difficult. Recently, cognitive network architectures using sophisticated learning techniques are increasingly being applied to such problems. In this paper, we propose data-driven machine learning (ML) schemes to efficiently solve these problems in wireless LAN (WLAN) networks. The proposed schemes are evaluated and results are compared with traditional approaches to the aforementioned problems. The results report significant improvement in network performance by applying the proposed schemes. The proposed scheme for handover prediction outperforms traditional methods i.e. received signal strength method and traveling distance method by reducing the number of unnecessary handovers by 60% and 50% respectively. Similarly, in AP selection, the proposed scheme outperforms the strongest signal first and least loaded first algorithms by achieving higher throughput gains up to 9.2% and 8% respectively.
A dual semismooth Newton based augmented Lagrangian method for large-scale linearly constrained sparse group square-root Lasso problems
Authors: Chengjing Wang, Peipei Tang
Date: 2021-11-27 00:00:00
Tasks: regression

Square-root Lasso problems are proven robust regression problems. Furthermore, square-root regression problems with structured sparsity also plays an important role in statistics and machine learning. In this paper, we focus on the numerical computation of large-scale linearly constrained sparse group square-root Lasso problems. In order to overcome the difficulty that there are two nonsmooth terms in the objective function, we propose a dual semismooth Newton (SSN) based augmented Lagrangian method (ALM) for it. That is, we apply the ALM to the dual problem with the subproblem solved by the SSN method. To apply the SSN method, the positive definiteness of the generalized Jacobian is very important. Hence we characterize the equivalence of its positive definiteness and the constraint nondegeneracy condition of the corresponding primal problem. In numerical implementation, we fully employ the second order sparsity so that the Newton direction can be efficiently obtained. Numerical experiments demonstrate the efficiency of the proposed algorithm.
A New Multifractal-based Deep Learning Model for Text Mining
Authors: Zhenhua Wang, Ming Ren, Dong Gao
Date: 2021-11-27 00:00:00
Tasks: Chinese Named Entity Recognition, Deep Learning, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER)

In this world full of uncertainty, where the fabric of existence weaves patterns of complexity, multifractal emerges as beacons of insight, illuminating them. As we delve into the realm of text mining that underpins various natural language processing applications and powers a range of intelligent services, we recognize that behind the veil of text lies a manifestation of human thought and cognition, intricately intertwined with the complexities. Building upon the foundation of perceiving text as a complex system, this study embarks on a journey to unravel the hidden treasures within, armed with the proposed multifractal method that deciphers the multifractal attributes embedded within the text landscape. This endeavor culminates in the birth of our novel model, which also harnesses the power of the proposed activation function to facilitate nonlinear information transmission within its neural network architecture. The success on experiments anchored in real-world technical reports covering the extraction of technical term and classification of hazard events, stands as a testament to our endeavors. This research venture not only expands our understanding of text mining but also opens new horizons for knowledge discovery across various domains.
Why KDAC? A general activation function for knowledge discovery
Authors: Zhenhua Wang, Dong Gao, Haozhe Liu, Fanglin Liu
Date: 2021-11-27 00:00:00
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER)

Deep learning oriented named entity recognition (DNER) has gradually become the paradigm of knowledge discovery, which greatly promotes domain intelligence. However, the current activation function of DNER fails to treat gradient vanishing, no negative output or non-differentiable existence, which may impede knowledge exploration caused by the omission and incomplete representation of latent semantics. To break through the dilemma, we present a novel activation function termed KDAC. Detailly, KDAC is an aggregation function with multiple conversion modes. The backbone of the activation region is the interaction between exponent and linearity, and the both ends extend through adaptive linear divergence, which surmounts the obstacle of gradient vanishing and no negative output. Crucially, the non-differentiable points are alerted and eliminated by an approximate smoothing algorithm. KDAC has a series of brilliant properties, including nonlinear, stable near-linear transformation and derivative, as well as dynamic style, etc. We perform experiments based on BERT-BiLSTM-CNN-CRF model on six benchmark datasets containing different domain knowledge, such as Weibo, Clinical, E-commerce, Resume, HAZOP and People's daily. The evaluation results show that KDAC is advanced and effective, and can provide more generalized activation to stimulate the performance of DNER. We hope that KDAC can be exploited as a promising activation function to devote itself to the construction of knowledge.
Achieving an Accurate Random Process Model for PV Power using Cheap Data: Leveraging the SDE and Public Weather Reports
Authors: Yiwei Qiu, Jin Lin, Zhipeng Zhou, Ningyi Dai, Feng Liu, Yonghua Song
Date: 2021-11-27 00:00:00
Tasks: Time Series, Time Series Analysis, Uncertainty Quantification

The stochastic differential equation (SDE)-based random process models of volatile renewable energy sources (RESs) jointly capture the evolving probability distribution and temporal correlation in continuous time. It has enabled recent studies to remarkably improve the performance of power system dynamic uncertainty quantification and optimization. However, considering the non-homogeneous random process nature of PV, there still remains a challenging question: how can a realistic and accurate SDE model for PV power be obtained that reflects its weather-dependent uncertainty in online operation, especially when high-resolution numerical weather prediction (NWP) is unavailable for many distributed plants? To fill this gap, this article finds that an accurate SDE model for PV power can be constructed by only using the cheap data from low-resolution public weather reports. Specifically, an hourly parameterized Jacobi diffusion process is constructed to recreate the temporal patterns of PV volatility during a day. Its parameters are mapped from the public weather report using an ensemble of extreme learning machines (ELMs) to reflect the varying weather conditions. The SDE model jointly captures intraday and intrahour volatility. Statistical examination based on real-world data collected in Macau shows the proposed approach outperforms a selection of state-of-the-art deep learning-based time-series forecast methods.
Factorized Fourier Neural Operators
Authors: Alasdair Tran, Alexander Mathews, Lexing Xie, Cheng Soon Ong
Date: 2021-11-27 00:00:00

We propose the Factorized Fourier Neural Operator (F-FNO), a learning-based approach for simulating partial differential equations (PDEs). Starting from a recently proposed Fourier representation of flow fields, the F-FNO bridges the performance gap between pure machine learning approaches to that of the best numerical or hybrid solvers. This is achieved with new representations - separable spectral layers and improved residual connections - and a combination of training strategies such as the Markov assumption, Gaussian noise, and cosine learning rate decay. On several challenging benchmark PDEs on regular grids, structured meshes, and point clouds, the F-FNO can scale to deeper networks and outperform both the FNO and the geo-FNO, reducing the error by 83% on the Navier-Stokes problem, 31% on the elasticity problem, 57% on the airfoil flow problem, and 60% on the plastic forging problem. Compared to the state-of-the-art pseudo-spectral method, the F-FNO can take a step size that is an order of magnitude larger in time and achieve an order of magnitude speedup to produce the same solution quality.
Particle Dynamics for Learning EBMs
Authors: Kirill Neklyudov, Priyank Jaini, Max Welling
Date: 2021-11-26 00:00:00

Energy-based modeling is a promising approach to unsupervised learning, which yields many downstream applications from a single model. The main difficulty in learning energy-based models with the "contrastive approaches" is the generation of samples from the current energy function at each iteration. Many advances have been made to accomplish this subroutine cheaply. Nevertheless, all such sampling paradigms run MCMC targeting the current model, which requires infinitely long chains to generate samples from the true energy distribution and is problematic in practice. This paper proposes an alternative approach to getting these samples and avoiding crude MCMC sampling from the current model. We accomplish this by viewing the evolution of the modeling distribution as (i) the evolution of the energy function, and (ii) the evolution of the samples from this distribution along some vector field. We subsequently derive this time-dependent vector field such that the particles following this field are approximately distributed as the current density model. Thereby we match the evolution of the particles with the evolution of the energy function prescribed by the learning procedure. Importantly, unlike Monte Carlo sampling, our method targets to match the current distribution in a finite time. Finally, we demonstrate its effectiveness empirically compared to MCMC-based learning methods.
Failure Modes of Domain Generalization Algorithms
Authors: Tigran Galstyan, Hrayr Harutyunyan, Hrant Khachatrian, Greg Ver Steeg, Aram Galstyan
Date: 2021-11-26 00:00:00
Proceeding: CVPR 2022 1
Tasks: Domain Generalization, Representation Learning

Domain generalization algorithms use training data from multiple domains to learn models that generalize well to unseen domains. While recently proposed benchmarks demonstrate that most of the existing algorithms do not outperform simple baselines, the established evaluation methods fail to expose the impact of various factors that contribute to the poor performance. In this paper we propose an evaluation framework for domain generalization algorithms that allows decomposition of the error into components capturing distinct aspects of generalization. Inspired by the prevalence of algorithms based on the idea of domain-invariant representation learning, we extend the evaluation framework to capture various types of failures in achieving invariance. We show that the largest contributor to the generalization error varies across methods, datasets, regularization strengths and even training lengths. We observe two problems associated with the strategy of learning domain-invariant representations. On Colored MNIST, most domain generalization algorithms fail because they reach domain-invariance only on the training domains. On Camelyon-17, domain-invariance degrades the quality of representations on unseen domains. We hypothesize that focusing instead on tuning the classifier on top of a rich representation can be a promising direction.
Towards Efficient Ansatz Architecture for Variational Quantum Algorithms
Authors: Anbang Wu, Gushu Li, yuke wang, Boyuan Feng, Yufei Ding, Yuan Xie
Date: 2021-11-26 00:00:00

Variational quantum algorithms are expected to demonstrate the advantage of quantum computing on near-term noisy quantum computers. However, training such variational quantum algorithms suffers from gradient vanishing as the size of the algorithm increases. Previous work cannot handle the gradient vanishing induced by the inevitable noise effects on realistic quantum hardware. In this paper, we propose a novel training scheme to mitigate such noise-induced gradient vanishing. We first introduce a new cost function of which the gradients are significantly augmented by employing traceless observables in truncated subspace. We then prove that the same minimum can be reached by optimizing the original cost function with the gradients from the new cost function. Experiments show that our new training scheme is highly effective for major variational quantum algorithms of various tasks.
BCH-NLP at BioCreative VII Track 3: medications detection in tweets using transformer networks and multi-task learning
Authors: Dongfang Xu, Shan Chen, Timothy Miller
Date: 2021-11-26 00:00:00
Tasks: All, Data Augmentation, Multi-Task Learning, text-classification, Text Classification

In this paper, we present our work participating in the BioCreative VII Track 3 - automatic extraction of medication names in tweets, where we implemented a multi-task learning model that is jointly trained on text classification and sequence labelling. Our best system run achieved a strict F1 of 80.4, ranking first and more than 10 points higher than the average score of all participants. Our analyses show that the ensemble technique, multi-task learning, and data augmentation are all beneficial for medication detection in tweets.
SARS-CoV-2 Dissemination using a Network of the United States Counties
Authors: Patrick Urrutia, David Wren, Chrysafis Vogiatzis, Ruriko Yoshida
Date: 2021-11-26 00:00:00
Tasks: Time Series, Time Series Analysis

During 2020 and 2021, severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) transmission has been increasing amongst the world's population at an alarming rate. Reducing the spread of SARS-CoV-2 and other diseases that are spread in similar manners is paramount for public health officials as they seek to effectively manage resources and potential population control measures such as social distancing and quarantines. By analyzing the United States' county network structure, one can model and interdict potential higher infection areas. County officials can provide targeted information, preparedness training, as well as increase testing in these areas. While these approaches may provide adequate countermeasures for localized areas, they are inadequate for the holistic United States. We solve this problem by collecting coronavirus disease 2019 (COVID-19) infections and deaths from the Center for Disease Control and Prevention{\color{black},} and {\color{black} adjacency between all counties obtained} from the United States Census Bureau. Generalized network autoregressive (GNAR) time series models have been proposed as an efficient learning algorithm for networked datasets. This work fuses network science and operations research techniques to univariately model COVID-19 cases, deaths, and current survivors across the United States' county network structure.
A Ubiquitous Unifying Degeneracy in Two-Body Microlensing Systems
Authors: Keming Zhang, B. Scott Gaudi, Joshua S. Bloom
Date: 2021-11-26 00:00:00
Tasks: Vocal Bursts Valence Prediction

While gravitational microlensing by planetary systems provides unique vistas on the properties of exoplanets, observations of a given 2-body microlensing event can often be interpreted with multiple distinct physical configurations. Such ambiguities are typically attributed to the close-wide and inner-outer types of degeneracies that arise from transformation invariances and symmetries of microlensing caustics. However, there remain unexplained inconsistencies between aforementioned theories and observations. Here, leveraging a fast machine learning inference framework, we present the discovery of the offset degeneracy, which concerns a magnification-matching behaviour on the lens-axis and is formulated independent of caustics. This offset degeneracy unifies the close-wide and inner-outer degeneracies, generalises to resonant topologies, and upon reanalysis, not only appears ubiquitous in previously published planetary events with 2-fold degenerate solutions, but also resolves prior inconsistencies. Our analysis demonstrates that degenerate caustics do not strictly result in degenerate magnifications and that the commonly invoked close-wide degeneracy essentially never arises in actual events. Moreover, it is shown that parameters in offset degenerate configurations are related by a simple expression. This suggests the existence of a deeper symmetry in the equations governing 2-body lenses than previously recognised.
Dynamic Network-Assisted D2D-Aided Coded Distributed Learning
Authors: Nikita Zeulin, Olga Galinina, Nageen Himayat, Sergey Andreev, Robert W. Heath Jr
Date: 2021-11-26 00:00:00
Tasks: Federated Learning

Today, various machine learning (ML) applications offer continuous data processing and real-time data analytics at the edge of a wireless network. Distributed real-time ML solutions are highly sensitive to the so-called straggler effect caused by resource heterogeneity and alleviated by various computation offloading mechanisms that seriously challenge the communication efficiency, especially in large-scale scenarios. To decrease the communication overhead, we rely on device-to-device (D2D) connectivity that improves spectrum utilization and allows efficient data exchange between devices in proximity. In particular, we design a novel D2D-aided coded federated learning method (D2D-CFL) for efficient load balancing across devices. The proposed solution captures system dynamics, including data (time-dependent learning model, varied intensity of data arrivals), device (diverse computational resources and volume of training data), and deployment (varied locations and D2D graph connectivity). To minimize the number of communication rounds, we derive an optimal compression rate for achieving minimum processing time and establish its connection with the convergence time. The resulting optimization problem provides suboptimal compression parameters, which improve the total training time. Our proposed method is beneficial for real-time collaborative applications, where the users continuously generate training data resulting in the model drift.