pwc-archive/papers-with-abstracts huggingface dataset:
Paragraph-based Transformer Pre-training for Multi-Sentence Inference
Authors: Luca Di Liello, Siddhant Garg, Luca Soldaini, Alessandro Moschitti
Date: 2022-05-02 00:00:00
Proceeding: NAACL 2022 7
Tasks: Answer Selection, Fact Verification, Question Answering, Sentence

Inference tasks such as answer sentence selection (AS2) or fact verification are typically solved by fine-tuning transformer-based models as individual sentence-pair classifiers. Recent studies show that these tasks benefit from modeling dependencies across multiple candidate sentences jointly. In this paper, we first show that popular pre-trained transformers perform poorly when used for fine-tuning on multi-candidate inference tasks. We then propose a new pre-training objective that models the paragraph-level semantics across multiple input sentences. Our evaluation on three AS2 and one fact verification datasets demonstrates the superiority of our pre-training technique over the traditional ones for transformers used as joint models for multi-candidate inference tasks, as well as when used as cross-encoders for sentence-pair formulations of these tasks. Our code and pre-trained models are released at https://github.com/amazon-research/wqa-multi-sentence-inference .
Robust Subset Selection by Greedy and Evolutionary Pareto Optimization
Authors: Chao Bian, Yawen Zhou, Chao Qian
Date: 2022-05-03 00:00:00

Subset selection, which aims to select a subset from a ground set to maximize some objective function, arises in various applications such as influence maximization and sensor placement. In real-world scenarios, however, one often needs to find a subset which is robust against (i.e., is good over) a number of possible objective functions due to uncertainty, resulting in the problem of robust subset selection. This paper considers robust subset selection with monotone objective functions, relaxing the submodular property required by previous studies. We first show that the greedy algorithm can obtain an approximation ratio of $1-e^{-\beta\gamma}$, where $\beta$ and $\gamma$ are the correlation and submodularity ratios of the objective functions, respectively; and then propose EPORSS, an evolutionary Pareto optimization algorithm that can utilize more time to find better subsets. We prove that EPORSS can also be theoretically grounded, achieving a similar approximation guarantee to the greedy algorithm. In addition, we derive the lower bound of $\beta$ for the application of robust influence maximization, and further conduct experiments to validate the performance of the greedy algorithm and EPORSS.
Toward Robust Spiking Neural Network Against Adversarial Perturbation
Authors: Ling Liang, Kaidi Xu, Xing Hu, Lei Deng, Yuan Xie
Date: 2022-04-12 00:00:00

As spiking neural networks (SNNs) are deployed increasingly in real-world efficiency critical applications, the security concerns in SNNs attract more attention. Currently, researchers have already demonstrated an SNN can be attacked with adversarial examples. How to build a robust SNN becomes an urgent issue. Recently, many studies apply certified training in artificial neural networks (ANNs), which can improve the robustness of an NN model promisely. However, existing certifications cannot transfer to SNNs directly because of the distinct neuron behavior and input formats for SNNs. In this work, we first design S-IBP and S-CROWN that tackle the non-linear functions in SNNs' neuron modeling. Then, we formalize the boundaries for both digital and spike inputs. Finally, we demonstrate the efficiency of our proposed robust training method in different datasets and model architectures. Based on our experiment, we can achieve a maximum $37.7\%$ attack error reduction with $3.7\%$ original accuracy loss. To the best of our knowledge, this is the first analysis on robust training of SNNs.
Automated Learning of Interpretable Models with Quantified Uncertainty
Authors: G. F. Bomarito, P. E. Leser, N. C. M Strauss, K. M. Garbrecht, J. D. Hochhalter
Date: 2022-04-12 00:00:00
Tasks: BIG-bench Machine Learning, Interpretable Machine Learning, regression, scientific discovery, Symbolic Regression, Uncertainty Quantification

Interpretability and uncertainty quantification in machine learning can provide justification for decisions, promote scientific discovery and lead to a better understanding of model behavior. Symbolic regression provides inherently interpretable machine learning, but relatively little work has focused on the use of symbolic regression on noisy data and the accompanying necessity to quantify uncertainty. A new Bayesian framework for genetic-programming-based symbolic regression (GPSR) is introduced that uses model evidence (i.e., marginal likelihood) to formulate replacement probability during the selection phase of evolution. Model parameter uncertainty is automatically quantified, enabling probabilistic predictions with each equation produced by the GPSR algorithm. Model evidence is also quantified in this process, and its use is shown to increase interpretability, improve robustness to noise, and reduce overfitting when compared to a conventional GPSR implementation on both numerical and physical experiments.
On Ranking Consistency of Pre-ranking Stage
Authors: Siyu Gu, Xiangrong Sheng
Date: 2022-05-03 00:00:00
Tasks: Retrieval

Industrial ranking systems, such as advertising systems, rank items by aggregating multiple objectives into one final objective to satisfy user demand and commercial intent. Cascade architecture, composed of retrieval, pre-ranking, and ranking stages, is usually adopted to reduce the computational cost. Each stage may employ various models for different objectives and calculate the final objective by aggregating these models' outputs. The multi-stage ranking strategy causes a new problem - the ranked lists of the ranking stage and previous stages may be inconsistent. For example, items that should be ranked at the top of the ranking stage may be ranked at the bottom of previous stages. In this paper, we focus on the \textbf{ranking consistency} between the pre-ranking and ranking stages. Specifically, we formally define the problem of ranking consistency and propose the Ranking Consistency Score (RCS) metric for evaluation. We demonstrate that ranking consistency has a direct impact on online performance. Compared with the traditional evaluation manner that mainly focuses on the individual ranking quality of every objective, RCS considers the ranking consistency of the fused final objective, which is more proper for evaluation. Finally, to improve the ranking consistency, we propose several methods from the perspective of sample selection and learning algorithms. Experimental results on one of the biggest industrial E-commerce platforms in China validate the efficacy of the proposed metrics and methods.
When Multi-Level Meets Multi-Interest: A Multi-Grained Neural Model for Sequential Recommendation
Authors: Yu Tian, Jianxin Chang, Yannan Niu, Yang song, Chenliang Li
Date: 2022-05-03 00:00:00
Tasks: Sequential Recommendation

Sequential recommendation aims at identifying the next item that is preferred by a user based on their behavioral history. Compared to conventional sequential models that leverage attention mechanisms and RNNs, recent efforts mainly follow two directions for improvement: multi-interest learning and graph convolutional aggregation. Specifically, multi-interest methods such as ComiRec and MIMN, focus on extracting different interests for a user by performing historical item clustering, while graph convolution methods including TGSRec and SURGE elect to refine user preferences based on multi-level correlations between historical items. Unfortunately, neither of them realizes that these two types of solutions can mutually complement each other, by aggregating multi-level user preference to achieve more precise multi-interest extraction for a better recommendation. To this end, in this paper, we propose a unified multi-grained neural model(named MGNM) via a combination of multi-interest learning and graph convolutional aggregation. Concretely, MGNM first learns the graph structure and information aggregation paths of the historical items for a user. It then performs graph convolution to derive item representations in an iterative fashion, in which the complex preferences at different levels can be well captured. Afterwards, a novel sequential capsule network is proposed to inject the sequential patterns into the multi-interest extraction process, leading to a more precise interest learning in a multi-grained manner.
Learning Representations for Incomplete Time Series Clustering
Authors: Qianli Ma, Chuxin Chen, Sen Li, Garrison W. Cottrell
Date: 2021-05-18 00:00:00
Proceeding: AAAI 2021 5
Tasks: Clustering, Imputation, Missing Values, Representation Learning, Time Series, Time Series Analysis, Time Series Clustering

Time-series clustering is an essential unsupervised technique for data analysis, applied to many real-world fields, such as medical analysis and DNA microarray. Existing clustering methods are usually based on the assumption that the data is complete. However, time series in real-world applications often contain missing values. Traditional strategy (imputing first and then clustering) does not optimize the imputation and clustering process as a whole, which not only makes per- formance dependent on the combination of imputation and clustering methods but also fails to achieve satisfactory re- sults. How to best improve the clustering performance on incomplete time series remains a challenge. This paper pro- poses a novel unsupervised temporal representation learning model, named Clustering Representation Learning on Incom- plete time-series data (CRLI). CRLI jointly optimizes the im- putation and clustering process to impute more discrimina- tive values for clustering and make the learned representa- tions possessed good clustering property. Also, to reduce the error propagation from imputation to clustering, we introduce a discriminator to make the distribution of imputation values close to the true one and train CRLI in an alternating train- ing manner. An experiment conducted on eight real-world in- complete time-series datasets shows that CRLI outperforms existing methods. We demonstrates the effectiveness of the learned representations and the convergence of the model through visualization analysis. Moreover, we reveal that the joint training strategy can impute values close to the true ones in those important sub-sequences, and impute more discrim- inative values in those less important sub-sequences at the same time, making the imputed sequence cluster-friendly.
Deep learning for clustering of multivariate clinical patient trajectories with missing values
Authors: Johann de Jong, Mohammad Asif Emon, Ping Wu, Reagon Karki, Meemansa Sood, Patrice Godard, Ashar Ahmad, Henri Vrooman, Martin Hofmann-Apitius, Holger Fröhlich
Date: 2019-11-15 00:00:00
Proceeding: GigaScience 2019 11
Tasks: Clustering, Missing Values, Time Series, Time Series Analysis, Time Series Clustering

Background
Precision medicine requires a stratification of patients by disease presentation that is sufficiently informative to allow for selecting treatments on a per-patient basis. For many diseases, such as neurological disorders, this stratification problem translates into a complex problem of clustering multivariate and relatively short time series because (i) these diseases are multifactorial and not well described by single clinical outcome variables and (ii) disease progression needs to be monitored over time. Additionally, clinical data often additionally are hindered by the presence of many missing values, further complicating any clustering attempts.

Findings
The problem of clustering multivariate short time series with many missing values is generally not well addressed in the literature. In this work, we propose a deep learning–based method to address this issue, variational deep embedding with recurrence (VaDER). VaDER relies on a Gaussian mixture variational autoencoder framework, which is further extended to (i) model multivariate time series and (ii) directly deal with missing values. We validated VaDER by accurately recovering clusters from simulated and benchmark data with known ground truth clustering, while varying the degree of missingness. We then used VaDER to successfully stratify patients with Alzheimer disease and patients with Parkinson disease into subgroups characterized by clinically divergent disease progression profiles. Additional analyses demonstrated that these clinical differences reflected known underlying aspects of Alzheimer disease and Parkinson disease.

Conclusions
We believe our results show that VaDER can be of great value for future efforts in patient stratification, and multivariate time-series clustering in general.
Integration of Behavioral Economic Models to Optimize ML performance and interpretability: a sandbox example
Authors: Emilio Soria-Olivas, José E. Vila Gisbert, Regino Barranquero Cardeñosa, Yolanda Gomez
Date: 2022-05-03 00:00:00

This paper presents a sandbox example of how the integration of models borrowed from Behavioral Economic (specifically Protection-Motivation Theory) into ML algorithms (specifically Bayesian Networks) can improve the performance and interpretability of ML algorithms when applied to Behavioral Data. The integration of Behavioral Economics knowledge to define the architecture of the Bayesian Network increases the accuracy of the predictions in 11 percentage points. Moreover, it simplifies the training process, making unnecessary training computational efforts to identify the optimal structure of the Bayesian Network. Finally, it improves the explicability of the algorithm, avoiding illogical relations among variables that are not supported by previous behavioral cybersecurity literature. Although preliminary and limited to 0ne simple model trained with a small dataset, our results suggest that the integration of behavioral economics and complex ML models may open a promising strategy to improve the predictive power, training costs and explicability of complex ML models. This integration will contribute to solve the scientific issue of ML exhaustion problem and to create a new ML technology with relevant scientific, technological and market implications.
Physical Layer Security for 6G Systems why it is needed and how to make it happen
Authors: Arsenia Chorti
Date: 2022-05-03 00:00:00

Sixth generations (6G) systems will be required to meet diverse constraints in an integrated ground-air-space global network. In particular, meeting overly aggressive latency constraints, operating in massive connectivity regimes, with low energy footprint and low computational effort, while providing explicit security guarantees, can be challenging. In this setting, quality of security (QoSec) is envisioned as a flexible security framework for future networks with highly diverse non-functional requirements. Mirroring the differentiated services (DiffServ) networking paradigm, different security levels could be conceptualized, moving away from static security controls, captured currently in zero-trust security architectures. In parallel, the integration of communications and sensing, along with embedded (on-device) AI, can provide the foundations for building autonomous and adaptive security controls, orchestrated by a vertical security plane in coordination with a vertical semantic plane. It is in this framework, that we envision the incorporation of physical layer security (PLS) schemes in 6G security protocols, introducing security controls at all layers, for the first time.
Attentive activation function for improving end-to-end spoofing countermeasure systems
Authors: Woo Hyun Kang, Jahangir Alam, Abderrahim Fathan
Date: 2022-05-03 00:00:00
Tasks: Speech Synthesis, Voice Conversion

The main objective of the spoofing countermeasure system is to detect the artifacts within the input speech caused by the speech synthesis or voice conversion process. In order to achieve this, we propose to adopt an attentive activation function, more specifically attention rectified linear unit (AReLU) to the end-to-end spoofing countermeasure system. Since the AReLU employs the attention mechanism to boost the contribution of relevant input features while suppressing the irrelevant ones, introducing AReLU can help the countermeasure system to focus on the features related to the artifacts. The proposed framework was experimented on the logical access (LA) task of ASVSpoof2019 dataset, and outperformed the systems using the standard non-learnable activation functions.
A Hybrid Defense Method against Adversarial Attacks on Traffic Sign Classifiers in Autonomous Vehicles
Authors: Zadid Khan, Mashrur Chowdhury, Sakib Mahmud Khan
Date: 2022-04-25 00:00:00
Tasks: Autonomous Vehicles, Navigate, Optical Character Recognition, Optical Character Recognition (OCR), Transfer Learning

Adversarial attacks can make deep neural network (DNN) models predict incorrect output labels, such as misclassified traffic signs, for autonomous vehicle (AV) perception modules. Resilience against adversarial attacks can help AVs navigate safely on the road by avoiding misclassication of signs or objects. This DNN-based study develops a resilient traffic sign classifier for AVs that uses a hybrid defense method. We use transfer learning to retrain the Inception-V3 and Resnet-152 models as traffic sign classifiers. This method also utilizes a combination of three different strategies: random filtering, ensembling, and local feature mapping. We use the random cropping and resizing technique for random filtering, plurality voting as ensembling strategy and an optical character recognition model as a local feature mapper. This DNN-based hybrid defense method has been tested for the no attack scenario and against well-known untargeted adversarial attacks (e.g., Projected Gradient Descent or PGD, Fast Gradient Sign Method or FGSM, Momentum Iterative Method or MIM attack, and Carlini and Wagner or C&W). We find that our hybrid defense method achieves 99% average traffic sign classification accuracy for the no attack scenario and 88% average traffic sign classification accuracy for all attack scenarios. Moreover, the hybrid defense method, presented in this study, improves the accuracy for traffic sign classification compared to the traditional defense methods (i.e., JPEG filtering, feature squeezing, binary filtering, and random filtering) up to 6%, 50%, and 55% for FGSM, MIM, and PGD attacks, respectively.
SynopSet: Multiscale Visual Abstraction Set for Explanatory Analysis of DNA Nanotechnology Simulations
Authors: Deng Luo, Alexandre Kouyoumdjian, Ondřej Strnad, Haichao Miao, Ivan Barišić, Ivan Viola
Date: 2022-04-18 00:00:00

We propose a new abstraction set (SynopSet) that has a continuum of visual representations for the explanatory analysis of molecular dynamics simulations (MDS) in the DNA nanotechnology domain. By re-purposing the commonly used progress bar and designing novel visuals, as well as transforming the data from the domain format to a format that better fits the newly designed visuals, we compose this new set of representations. This set is also designed to be capable of showing all spatial and temporal details, and all structural complexity, or abstracting these to various degrees, enabling both the slow playback of the simulation for detailed examinations or very fast playback for an overview that helps to efficiently identify events of interest, as well as several intermediate levels between these two extremes. For any pair of successive representations, we demonstrate smooth, continuous transitions, enabling users to keep track of relevant information from one representation to the next. By providing multiple representations suited to different temporal resolutions and connected by smooth transitions, we enable time-efficient simulation analysis, giving users the opportunity to examine and present important phases in great detail, or leverage abstract representations to go over uneventful phases much faster. Domain experts can thus gain actionable insight about their simulations and communicate it in a much shorter time. Further, the novel representations are more intuitive and also enable researchers unfamiliar with MDS analysis graphs to better understand the simulation results. We assessed the effectiveness of SynopSet on 12 DNA nanostructure simulations together with a domain expert. We have also shown that our set of representations can be systematically located in a visualization space, dubbed SynopSpace.
AutoFi: Towards Automatic WiFi Human Sensing via Geometric Self-Supervised Learning
Authors: Jianfei Yang, Xinyan Chen, Han Zou, Dazhuo Wang, Lihua Xie
Date: 2022-04-12 00:00:00
Tasks: Activity Recognition, Domain Adaptation, Gait Recognition, Gesture Recognition, Privacy Preserving, Self-Supervised Learning

WiFi sensing technology has shown superiority in smart homes among various sensors for its cost-effective and privacy-preserving merits. It is empowered by Channel State Information (CSI) extracted from WiFi signals and advanced machine learning models to analyze motion patterns in CSI. Many learning-based models have been proposed for kinds of applications, but they severely suffer from environmental dependency. Though domain adaptation methods have been proposed to tackle this issue, it is not practical to collect high-quality, well-segmented and balanced CSI samples in a new environment for adaptation algorithms, but randomly-captured CSI samples can be easily collected. {\color{black}In this paper, we firstly explore how to learn a robust model from these low-quality CSI samples, and propose AutoFi, an annotation-efficient WiFi sensing model based on a novel geometric self-supervised learning algorithm.} The AutoFi fully utilizes unlabeled low-quality CSI samples that are captured randomly, and then transfers the knowledge to specific tasks defined by users, which is the first work to achieve cross-task transfer in WiFi sensing. The AutoFi is implemented on a pair of Atheros WiFi APs for evaluation. The AutoFi transfers knowledge from randomly collected CSI samples into human gait recognition and achieves state-of-the-art performance. Furthermore, we simulate cross-task transfer using public datasets to further demonstrate its capacity for cross-task learning. For the UT-HAR and Widar datasets, the AutoFi achieves satisfactory results on activity recognition and gesture recognition without any prior training. We believe that the AutoFi takes a huge step toward automatic WiFi sensing without any developer engagement.
Efficient dynamic filter for robust and low computational feature extraction
Authors: Donghyeon Kim, Gwantae Kim, Bokyeung Lee, Jeong-gi Kwak, David K. Han, Hanseok Ko
Date: 2022-05-03 00:00:00
Tasks: Keyword Spotting, Speaker Verification

Unseen noise signal which is not considered in a model training process is difficult to anticipate and would lead to performance degradation. Various methods have been investigated to mitigate unseen noise. In our previous work, an Instance-level Dynamic Filter (IDF) and a Pixel Dynamic Filter (PDF) were proposed to extract noise-robust features. However, the performance of the dynamic filter might be degraded since simple feature pooling is used to reduce the computational resource in the IDF part. In this paper, we propose an efficient dynamic filter to enhance the performance of the dynamic filter. Instead of utilizing the simple feature mean, we separate Time-Frequency (T-F) features as non-overlapping chunks, and separable convolutions are carried out for each feature direction (inter chunks and intra chunks). Additionally, we propose Dynamic Attention Pooling that maps high dimensional features as low dimensional feature embeddings. These methods are applied to the IDF for keyword spotting and speaker verification tasks. We confirm that our proposed method performs better in unseen environments (unseen noise and unseen speakers) than state-of-the-art models.