pwc-archive/papers-with-abstracts huggingface dataset:
DCAP: Deep Cross Attentional Product Network for User Response Prediction
Authors: Zekai Chen, Fangtian Zhong, Zhumin Chen, Xiao Zhang, Robert Pless, Xiuzhen Cheng
Date: 2021-05-18 00:00:00
Tasks: Recommendation Systems

User response prediction, which aims to predict the probability that a user will provide a predefined positive response in a given context such as clicking on an ad or purchasing an item, is crucial to many industrial applications such as online advertising, recommender systems, and search ranking. However, due to the high dimensionality and super sparsity of the data collected in these tasks, handcrafting cross features is inevitably time expensive. Prior studies in predicting user response leveraged the feature interactions by enhancing feature vectors with products of features to model second-order or high-order cross features, either explicitly or implicitly. Nevertheless, these existing methods can be hindered by not learning sufficient cross features due to model architecture limitations or modeling all high-order feature interactions with equal weights. This work aims to fill this gap by proposing a novel architecture Deep Cross Attentional Product Network (DCAP), which keeps cross network's benefits in modeling high-order feature interactions explicitly at the vector-wise level. Beyond that, it can differentiate the importance of different cross features in each network layer inspired by the multi-head attention mechanism and Product Neural Network (PNN), allowing practitioners to perform a more in-depth analysis of user behaviors. Additionally, our proposed model can be easily implemented and train in parallel. We conduct comprehensive experiments on three real-world datasets. The results have robustly demonstrated that our proposed model DCAP achieves superior prediction performance compared with the state-of-the-art models. Public codes are available at https://github.com/zachstarkk/DCAP.
ASM2TV: An Adaptive Semi-Supervised Multi-Task Multi-View Learning Framework for Human Activity Recognition
Authors: Zekai Chen, Xiao Zhang, Xiuzhen Cheng
Date: 2021-05-18 00:00:00
Tasks: Activity Recognition, Human Activity Recognition, MULTI-VIEW LEARNING, Time Series Analysis

Many real-world scenarios, such as human activity recognition (HAR) in IoT, can be formalized as a multi-task multi-view learning problem. Each specific task consists of multiple shared feature views collected from multiple sources, either homogeneous or heterogeneous. Common among recent approaches is to employ a typical hard/soft sharing strategy at the initial phase separately for each view across tasks to uncover common knowledge, underlying the assumption that all views are conditionally independent. On the one hand, multiple views across tasks possibly relate to each other under practical situations. On the other hand, supervised methods might be insufficient when labeled data is scarce. To tackle these challenges, we introduce a novel framework ASM2TV for semi-supervised multi-task multi-view learning. We present a new perspective named gating control policy, a learnable task-view-interacted sharing policy that adaptively selects the most desirable candidate shared block for any view across any task, which uncovers more fine-grained task-view-interacted relatedness and improves inference efficiency. Significantly, our proposed gathering consistency adaption procedure takes full advantage of large amounts of unlabeled fragmented time-series, making it a general framework that accommodates a wide range of applications. Experiments on two diverse real-world HAR benchmark datasets collected from various subjects and sources demonstrate our framework's superiority over other state-of-the-arts. The detailed codes are available at https://github.com/zachstarkk/ASM2TV.
Stationary distributions of persistent ecological systems
Authors: Alexandru Hening, Yao Li
Date: 2020-03-09 00:00:00

We analyze ecological systems that are influenced by random environmental fluctuations. We first provide general conditions which ensure that the species coexist and the system converges to a unique invariant probability measure (stationary distribution). Since it is usually impossible to characterize this invariant probability measure analytically, we develop a powerful method for numerically approximating invariant probability measures. This allows us to shed light upon how the various parameters of the ecosystem impact the stationary distribution. We analyze different types of environmental fluctuations. At first we study ecosystems modeled by stochastic differential equations. In the second setting we look at piecewise deterministic Markov processes. These are processes where one follows a system of differential equations for a random time, after which the environmental state changes, and one follows a different set of differential equations -- this procedure then gets repeated indefinitely. Finally, we look at stochastic differential equations with switching, which take into account both the white noise fluctuations and the random environmental switches. As applications of our theoretical and numerical analysis, we look at competitive Lotka--Volterra, Beddington-DeAngelis predator-prey, and rock-paper-scissors dynamics. We highlight new biological insights by analyzing the stationary distributions of the ecosystems and by seeing how various types of environmental fluctuations influence the long term fate of populations.
A parameter refinement method for Ptychography based on Deep Learning concepts
Authors: Francesco Guzzi, George Kourousias, Fulvio Billè, Roberto Pugliese, Alessandra Gianoncelli, Sergio Carrato
Date: 2021-05-18 00:00:00
Tasks: Deep Learning, Position

X-ray Ptychography is an advanced computational microscopy technique which is delivering exceptionally detailed quantitative imaging of biological and nanotechnology specimens. However coarse parametrisation in propagation distance, position errors and partial coherence frequently menaces the experiment viability. In this work we formally introduced these actors, solving the whole reconstruction as an optimisation problem. A modern Deep Learning framework is used to correct autonomously the setup incoherences, thus improving the quality of a ptychography reconstruction. Automatic procedures are indeed crucial to reduce the time for a reliable analysis, which has a significant impact on all the fields that use this kind of microscopy. We implemented our algorithm in our software framework, SciComPty, releasing it as open-source. We tested our system on both synthetic datasets and also on real data acquired at the TwinMic beamline of the Elettra synchrotron facility.
AI-Native Network Slicing for 6G Networks
Authors: Wen Wu, Conghao Zhou, Mushu Li, Huaqing Wu, Haibo Zhou, Ning Zhang, Xuemin, Shen, Weihua Zhuang
Date: 2021-05-18 00:00:00
Tasks: Management

With the global roll-out of the fifth generation (5G) networks, it is necessary to look beyond 5G and envision the 6G networks. The 6G networks are expected to have space-air-ground integrated networks, advanced network virtualization, and ubiquitous intelligence. This article presents an artificial intelligence (AI)-native network slicing architecture for 6G networks to enable the synergy of AI and network slicing, thereby facilitating intelligent network management and supporting emerging AI services. AI-based solutions are first discussed across network slicing lifecycle to intelligently manage network slices, i.e., AI for slicing. Then, network slicing solutions are studied to support emerging AI services by constructing AI instances and performing efficient resource management, i.e., slicing for AI. Finally, a case study is presented, followed by a discussion of open research issues that are essential for AI-native network slicing in 6G networks.
CoTexT: Multi-task Learning with Code-Text Transformer
Authors: Long Phan, Hieu Tran, Daniel Le, Hieu Nguyen, James Anibal, Alec Peltekian, Yanfang Ye
Date: 2021-05-18 00:00:00
Proceeding: ACL (NLP4Prog) 2021 8
Tasks: Code Generation, Code Summarization, Decoder, Defect Detection, Multi-Task Learning, Text Generation

We present CoTexT, a pre-trained, transformer-based encoder-decoder model that learns the representative context between natural language (NL) and programming language (PL). Using self-supervision, CoTexT is pre-trained on large programming language corpora to learn a general understanding of language and code. CoTexT supports downstream NL-PL tasks such as code summarizing/documentation, code generation, defect detection, and code debugging. We train CoTexT on different combinations of available PL corpus including both "bimodal" and "unimodal" data. Here, bimodal data is the combination of text and corresponding code snippets, whereas unimodal data is merely code snippets. We first evaluate CoTexT with multi-task learning: we perform Code Summarization on 6 different programming languages and Code Refinement on both small and medium size featured in the CodeXGLUE dataset. We further conduct extensive experiments to investigate CoTexT on other tasks within the CodeXGlue dataset, including Code Generation and Defect Detection. We consistently achieve SOTA results in these tasks, demonstrating the versatility of our models.
Implementation and Evaluation of a Multivariate Abstraction-Based, Interval-Based Dynamic Time-Warping Method as a Similarity Measure for Longitudinal Medical Records
Authors: Yuval Shahar, Matan Lion
Date: 2021-05-18 00:00:00
Tasks: Dynamic Time Warping

We extended dynamic time warping (DTW) into interval-based dynamic time warping (iDTW), including (A) interval-based representation (iRep): [1] abstracting raw, time-stamped data into interval-based abstractions, [2] comparison-period scoping, [3] partitioning abstract intervals into a given temporal granularity; (B) interval-based matching (iMatch): matching partitioned, abstract-concepts records, using a modified DTW. Using domain knowledge, we abstracted the raw data of medical records, for up to three concepts out of four or five relevant concepts, into two interval types: State abstractions (e.g. LOW, HIGH) and Gradient abstractions (e.g. INCREASING, DECREASING). We created all uni-dimensional (State or Gradient) or multi-dimensional (State and Gradient) abstraction combinations. Tasks: Classifying 161 oncology patients records as autologous or allogenic bone-marrow transplantation; classifying 125 hepatitis patients records as B or C hepatitis; predicting micro- or macro-albuminuria in the next year for 151 Type 2 diabetes patients. We used a k-Nearest-Neighbors majority, k = an odd number from 1 to SQRT(N), N = set size. 75,936 10-fold cross-validation experiments were performed: 33,600 (Oncology), 28,800 (Hepatitis), 13,536 (Diabetes). Measures: Area Under the Curve (AUC), optimal Youden's Index. Paired t-tests compared result vectors for equivalent configurations other than a tested variable, to determine a significant mean accuracy difference (P<0.05). Mean classification and prediction using abstractions was significantly better than using only raw time-stamped data. In each domain, at least one abstraction combination led to a significantly better mean performance than raw data. Increasing feature number and using Multi-dimensional abstractions enhanced performance. Unlike when using raw data, optimal mean performance was often reached with k=5, using abstractions.
Handling Structural Mismatches in Real-time Opera Tracking
Authors: Charles Brazier, Gerhard Widmer
Date: 2021-05-18 00:00:00

Algorithms for reliable real-time score following in live opera promise a lot of useful applications such as automatic subtitles display, or real-time video cutting in live streaming. Until now, such systems were based on the strong assumption that an opera performance follows the structure of the score linearly. However, this is rarely the case in practice, because of different opera versions and directors' cutting choices. In this paper, we propose a two-level solution to this problem. We introduce a real-time-capable, high-resolution (HR) tracker that can handle jumps or repetitions at specific locations provided to it. We then combine this with an additional low-resolution (LR) tracker that can handle all sorts of mismatches that can occur at any time, with some imprecision, and can re-direct the HR tracker if the latter is `lost' in the score. We show that the combination of the two improves tracking robustness in the presence of strong structural mismatches.
Deep Correlation Analysis for Audio-EEG Decoding
Authors: Jaswanth Reddy Katthi, Sriram Ganapathy
Date: 2021-05-18 00:00:00
Tasks: EEG, Eeg Decoding, Electroencephalogram (EEG)

The electroencephalography (EEG), which is one of the easiest modes of recording brain activations in a non-invasive manner, is often distorted due to recording artifacts which adversely impacts the stimulus-response analysis. The most prominent techniques thus far attempt to improve the stimulus-response correlations using linear methods. In this paper, we propose a neural network based correlation analysis framework that significantly improves over the linear methods for auditory stimuli. A deep model is proposed for intra-subject audio-EEG analysis based on directly optimizing the correlation loss. Further, a neural network model with a shared encoder architecture is proposed for improving the inter-subject stimulus response correlations. These models attempt to suppress the EEG artifacts while preserving the components related to the stimulus. Several experiments are performed using EEG recordings from subjects listening to speech and music stimuli. In these experiments, we show that the deep models improve the Pearson correlation significantly over the linear methods (average absolute improvements of 7.4% in speech tasks and 29.3% in music tasks). We also analyze the impact of several model parameters on the stimulus-response correlation.
Relative Positional Encoding for Transformers with Linear Complexity
Authors: Antoine Liutkus, Ondřej Cífka, Shih-Lun Wu, Umut Şimşekli, Yi-Hsuan Yang, Gaël Richard
Date: 2021-05-18 00:00:00
Tasks: Gaussian Processes, Image Classification, Music Generation, Text Classification

Recent advances in Transformer models allow for unprecedented sequence lengths, due to linear space and time complexity. In the meantime, relative positional encoding (RPE) was proposed as beneficial for classical Transformers and consists in exploiting lags instead of absolute positions for inference. Still, RPE is not available for the recent linear-variants of the Transformer, because it requires the explicit computation of the attention matrix, which is precisely what is avoided by such methods. In this paper, we bridge this gap and present Stochastic Positional Encoding as a way to generate PE that can be used as a replacement to the classical additive (sinusoidal) PE and provably behaves like RPE. The main theoretical contribution is to make a connection between positional encoding and cross-covariance structures of correlated Gaussian processes. We illustrate the performance of our approach on the Long-Range Arena benchmark and on music generation.
Parallel and Flexible Sampling from Autoregressive Models via Langevin Dynamics
Authors: Vivek Jayaram, John Thickstun
Date: 2021-05-17 00:00:00
Tasks: Audio Source Separation, Super-Resolution

This paper introduces an alternative approach to sampling from autoregressive models. Autoregressive models are typically sampled sequentially, according to the transition dynamics defined by the model. Instead, we propose a sampling procedure that initializes a sequence with white noise and follows a Markov chain defined by Langevin dynamics on the global log-likelihood of the sequence. This approach parallelizes the sampling process and generalizes to conditional sampling. Using an autoregressive model as a Bayesian prior, we can steer the output of a generative model using a conditional likelihood or constraints. We apply these techniques to autoregressive models in the visual and audio domains, with competitive results for audio source separation, super-resolution, and inpainting.
MUSER: MUltimodal Stress Detection using Emotion Recognition as an Auxiliary Task
Authors: Yiqun Yao, Michalis Papakostas, Mihai Burzo, Mohamed Abouelenien, Rada Mihalcea
Date: 2021-05-17 00:00:00
Proceeding: NAACL 2021 4
Tasks: Emotion Recognition, Multi-Task Learning

The capability to automatically detect human stress can benefit artificial intelligent agents involved in affective computing and human-computer interaction. Stress and emotion are both human affective states, and stress has proven to have important implications on the regulation and expression of emotion. Although a series of methods have been established for multimodal stress detection, limited steps have been taken to explore the underlying inter-dependence between stress and emotion. In this work, we investigate the value of emotion recognition as an auxiliary task to improve stress detection. We propose MUSER -- a transformer-based model architecture and a novel multi-task learning algorithm with speed-based dynamic sampling strategy. Evaluations on the Multimodal Stressed Emotion (MuSE) dataset show that our model is effective for stress detection with both internal and external auxiliary tasks, and achieves state-of-the-art results.
Nonparametric Modeling of Higher-Order Interactions via Hypergraphons
Authors: Krishnakumar Balasubramanian
Date: 2021-05-18 00:00:00

We study statistical and algorithmic aspects of using hypergraphons, that are limits of large hypergraphs, for modeling higher-order interactions. Although hypergraphons are extremely powerful from a modeling perspective, we consider a restricted class of Simple Lipschitz Hypergraphons (SLH), that are amenable to practically efficient estimation. We also provide rates of convergence for our estimator that are optimal for the class of SLH. Simulation results are provided to corroborate the theory.
Divided We Rule: Influencer Polarization on Twitter During Political Crises in India
Authors: Saloni Dash, Dibyendu Mishra, Gazal Shekhawat, Joyojeet Pal
Date: 2021-05-18 00:00:00
Tasks: Sentence

Influencers are key to the nature and networks of information propagation on social media. Influencers are particularly important in political discourse through their engagement with issues, and may derive their legitimacy either solely or in large part through online operation, or have an offline sphere of expertise such as entertainers, journalists etc. To quantify influencers' political engagement and polarity, we use Google's Universal Sentence Encoder (USE) to encode the tweets of 6k influencers and 26k Indian politicians during political crises in India. We then obtain aggregate vector representations of the influencers based on their tweet embeddings, which alongside retweet graphs help compute their stance and polarity with respect to these political issues. We find that influencers engage with the topics in a partisan manner, with polarized influencers being rewarded with increased retweeting and following. Moreover, we observe that specific groups of influencers are consistently polarized across all events. We conclude by discussing how our study provides insights into the political schisms of present-day India, but also offers a means to study the role of influencers in exacerbating political polarization in other contexts.
Independent Asymmetric Embedding for Information Diffusion Prediction on Social Networks
Authors: Wenjin Xie, Xiaomeng Wang, Tao Jia
Date: 2021-05-18 00:00:00
Tasks: Marketing, Network Embedding

The prediction for information diffusion on social networks has great practical significance in marketing and public opinion control. It aims to predict the individuals who will potentially repost the message on the social network. One type of method is based on demographics, complex networks and other prior knowledge to establish an interpretable model to simulate and predict the propagation process, while the other type of method is completely data-driven and maps the nodes to a latent space for propagation prediction. Existing latent space design and embedding methods lack consideration for the intervene among users. In this paper, we propose an independent asymmetric embedding method to embed each individual into one latent influence space and multiple latent susceptibility spaces. Based on the similarity between information diffusion and heat diffusion phenomenon, the heat diffusion kernel is exploited in our model and establishes the embedding rules. Furthermore, our method captures the co-occurrence regulation of user combinations in cascades to improve the calculating effectiveness. The results of extensive experiments conducted on real-world datasets verify both the predictive accuracy and cost-effectiveness of our approach.
Evolutionary Algorithms for Multi-Objective Optimization of Drone Controller Parameters
Authors: Azin Shamshirgaran, Hamed Javidi, Dan Simon
Date: 2021-05-18 00:00:00
Tasks: Drone Controller, Evolutionary Algorithms

Drones are effective for reducing human activity and interactions by performing tasks such as exploring and inspecting new environments, monitoring resources and delivering packages. Drones need a controller to maintain stability and to reach their goal. The most well-known drone controllers are proportional-integral-derivative (PID) and proportional-derivative (PD) controllers. However, the controller parameters need to be tuned and optimized. In this paper, we introduce the use of two evolutionary algorithms, biogeography-based optimization~(BBO) and particle swarm optimization (PSO), for multi-objective optimization (MOO) to tune the parameters of the PD controller of a drone. The combination of MOO, BBO, and PSO results in various methods for optimization: vector evaluated BBO and PSO, denoted as VEBBO and VEPSO; and non-dominated sorting BBO and PSO, denoted as NSBBO and NSPSO. The multi-objective cost function is based on tracking errors for the four states of the system. Two criteria for evaluating the Pareto fronts of the optimization methods, normalized hypervolume and relative coverage, are used to compare performance. Results show that NSBBO generally performs better than the other methods.
Embedded Model Predictive Controller on Low-Cost Low-End Microcontroller for Electrical Drives
Authors: Saket Adhau, Siddesh Dani, Deepak Ingole, Dayaram Sonawane
Date: 2021-05-18 00:00:00
Tasks: C++ code

It is very well-known that the implementation of Model Predictive Controller (MPC) on embedded platforms is challenging due to the computational complexities associated while solving an optimization problem. Although, there are many efficient embedded implementations existing by now, but for faster, more dynamic and non-linear control applications, there is no cost-effective and memory efficient embedded solutions. In this paper, we show the implementation of embedded explicit MPC for a motor speed control application on a lowcost 8 bit PIC 18 series microcontroller which costs only $5. The offset-free explicit MPC is designed for reference tracking, constraints handling, and disturbance rejection. The developed control law is exported to low-level C code and utilized in HIL co-simulations. We present the results of memory demand and control performance under various operating scenarios. The presented results show that the developed embedded MPC utilize about 40% of RAM and 92% of ROM for prediction horizon up to 3 samples. The performance of developed MPC is compared with the conventional PI controller. Overall results show that the presented approach is cost-effective, portable, and gives better performance than the PI controller.
PNLSS Toolbox 1.0
Authors: Jan Decuyper, Koen Tiels, Johan Schoukens
Date: 2021-05-18 00:00:00
Tasks: State Space Models

This is a demonstration of the PNLSS Toolbox 1.0. The toolbox is designed to identify polynomial nonlinear state-space models from data. Nonlinear state-space models can describe a wide range of nonlinear systems. An illustration is provided on experimental data of an electrical system mimicking the forced Duffing oscillator, and on numerical data of a nonlinear fluid dynamics problem.