pwc-archive/papers-with-abstracts huggingface dataset:
Parameter Tuning Strategies for Metaheuristic Methods Applied to Discrete Optimization of Structural Design
Authors: Iván Negrin, Dirk Roose, Ernesto Chagoyén
Date: 2021-10-12 00:00:00

This paper presents several strategies to tune the parameters of metaheuristic methods for (discrete) design optimization of reinforced concrete (RC) structures. A novel utility metric is proposed, based on the area under the average performance curve. The process of modelling, analysis and design of realistic RC structures leads to objective functions for which the evaluation is computationally very expensive. To avoid costly simulations, two types of surrogate models are used. The first one consists of the creation of a database containing all possible solutions. The second one uses benchmark functions to create a discrete sub-space of them, simulating the main features of realistic problems. Parameter tuning of four metaheuristics is performed based on two strategies. The main difference between them is the parameter control established to perform partial assessments. The simplest strategy is suitable to tune good `generalist' methods, i.e., methods with good performance regardless the parameter configuration. The other one is more expensive, but is well suited to assess any method. Tuning results prove that Biogeography-Based Optimization, a relatively new evolutionary algorithm, outperforms other methods such as GA or PSO for such optimization problems, due to its particular approach of applying recombination and mutation operators.
One Timestep is All You Need: Training Spiking Neural Networks with Ultra Low Latency
Authors: Sayeed Shafayet Chowdhury, Nitin Rathi, Kaushik Roy
Date: 2021-10-01 00:00:00
Tasks: All

Spiking Neural Networks (SNNs) are energy efficient alternatives to commonly used deep neural networks (DNNs). Through event-driven information processing, SNNs can reduce the expensive compute requirements of DNNs considerably, while achieving comparable performance. However, high inference latency is a significant hindrance to the edge deployment of deep SNNs. Computation over multiple timesteps not only increases latency as well as overall energy budget due to higher number of operations, but also incurs memory access overhead of fetching membrane potentials, both of which lessen the energy benefits of SNNs. To overcome this bottleneck and leverage the full potential of SNNs, we propose an Iterative Initialization and Retraining method for SNNs (IIR-SNN) to perform single shot inference in the temporal axis. The method starts with an SNN trained with T timesteps (T>1). Then at each stage of latency reduction, the network trained at previous stage with higher timestep is utilized as initialization for subsequent training with lower timestep. This acts as a compression method, as the network is gradually shrunk in the temporal domain. In this paper, we use direct input encoding and choose T=5, since as per literature, it is the minimum required latency to achieve satisfactory performance on ImageNet. The proposed scheme allows us to obtain SNNs with up to unit latency, requiring a single forward pass during inference. We achieve top-1 accuracy of 93.05%, 70.15% and 67.71% on CIFAR-10, CIFAR-100 and ImageNet, respectively using VGG16, with just 1 timestep. In addition, IIR-SNNs perform inference with 5-2500X reduced latency compared to other state-of-the-art SNNs, maintaining comparable or even better accuracy. Furthermore, in comparison with standard DNNs, the proposed IIR-SNNs provide25-33X higher energy efficiency, while being comparable to them in classification performance.
The Terminating-Random Experiments Selector: Fast High-Dimensional Variable Selection with False Discovery Rate Control
Authors: Jasin Machkour, Michael Muma, Daniel P. Palomar
Date: 2021-10-12 00:00:00
Tasks: Variable Selection

We propose the Terminating-Random Experiments (T-Rex) selector, a fast variable selection method for high-dimensional data. The T-Rex selector controls a user-defined target false discovery rate (FDR) while maximizing the number of selected variables. This is achieved by fusing the solutions of multiple early terminated random experiments. The experiments are conducted on a combination of the original predictors and multiple sets of randomly generated dummy predictors. A finite sample proof based on martingale theory for the FDR control property is provided. Numerical simulations confirm that the FDR is controlled at the target level while allowing for high power. We prove that the dummies can be sampled from any univariate probability distribution with finite expectation and variance. The computational complexity of the proposed method is linear in the number of variables. The T-Rex selector outperforms state-of-the-art methods for FDR control in numerical experiments and on a simulated genome-wide association study (GWAS), while its sequential computation time is more than two orders of magnitude lower than that of the strongest benchmark methods. The open source R package TRexSelector containing the implementation of the T-Rex selector is available on CRAN.
Bayesian Regularization for Functional Graphical Models with Applications to Neuroimaging
Authors: Jiajing Niu, Boyoung Hur, John Absher, D. Andrew Brown
Date: 2021-10-11 00:00:00

Graphical models, used to express conditional dependence between random variables observed at various nodes, are used extensively in many fields such as genetics, neuroscience, and social network analysis. While most current statistical methods for estimating graphical models focus on scalar data, there is interest in estimating analogous dependence structures when the data observed at each node are functional, such as signals or images. In this paper, we propose a fully Bayesian regularization scheme for estimating functional graphical models. We first consider a direct Bayesian analog of the functional graphical lasso proposed by Qiao et al. (2019). We then propose a regularization strategy via the graphical horseshoe. We compare these approaches via simulation study and apply our proposed functional graphical horseshoe to two motivating applications, electroencephalography data for comparing brain activation between an alcoholic group and controls, as well as changes in structural connectivity in the presence of traumatic brain injury (TBI). Our results yield insight into how the brain attempts to compensate for disconnected networks after injury.
Efficient Neural Ranking using Forward Indexes
Authors: Jurek Leonhardt, Koustav Rudra, Megha Khosla, Abhijit Anand, Avishek Anand
Date: 2021-10-12 00:00:00
Tasks: CPU, Document Ranking, Retrieval, Semantic Similarity, Semantic Textual Similarity

Neural document ranking approaches, specifically transformer models, have achieved impressive gains in ranking performance. However, query processing using such over-parameterized models is both resource and time intensive. In this paper, we propose the Fast-Forward index -- a simple vector forward index that facilitates ranking documents using interpolation of lexical and semantic scores -- as a replacement for contextual re-rankers and dense indexes based on nearest neighbor search. Fast-Forward indexes rely on efficient sparse models for retrieval and merely look up pre-computed dense transformer-based vector representations of documents and passages in constant time for fast CPU-based semantic similarity computation during query processing. We propose index pruning and theoretically grounded early stopping techniques to improve the query processing throughput. We conduct extensive large-scale experiments on TREC-DL datasets and show improvements over hybrid indexes in performance and query processing efficiency using only CPUs. Fast-Forward indexes can provide superior ranking performance using interpolation due to the complementary benefits of lexical and semantic similarities.
Learning Discrete Representations via Constrained Clustering for Effective and Efficient Dense Retrieval
Authors: Jingtao Zhan, Jiaxin Mao, Yiqun Liu, Jiafeng Guo, Min Zhang, Shaoping Ma
Date: 2021-10-12 00:00:00
Tasks: Clustering, Constrained Clustering, Information Retrieval, Open-Domain Question Answering, Quantization, Retrieval

Dense Retrieval (DR) has achieved state-of-the-art first-stage ranking effectiveness. However, the efficiency of most existing DR models is limited by the large memory cost of storing dense vectors and the time-consuming nearest neighbor search (NNS) in vector space. Therefore, we present RepCONC, a novel retrieval model that learns discrete Representations via CONstrained Clustering. RepCONC jointly trains dual-encoders and the Product Quantization (PQ) method to learn discrete document representations and enables fast approximate NNS with compact indexes. It models quantization as a constrained clustering process, which requires the document embeddings to be uniformly clustered around the quantization centroids and supports end-to-end optimization of the quantization method and dual-encoders. We theoretically demonstrate the importance of the uniform clustering constraint in RepCONC and derive an efficient approximate solution for constrained clustering by reducing it to an instance of the optimal transport problem. Besides constrained clustering, RepCONC further adopts a vector-based inverted file system (IVF) to support highly efficient vector search on CPUs. Extensive experiments on two popular ad-hoc retrieval benchmarks show that RepCONC achieves better ranking effectiveness than competitive vector quantization baselines under different compression ratio settings. It also substantially outperforms a wide range of existing retrieval models in terms of retrieval effectiveness, memory efficiency, and time efficiency.
Contrastive Learning for Representation Degeneration Problem in Sequential Recommendation
Authors: Ruihong Qiu, Zi Huang, Hongzhi Yin, Zijian Wang
Date: 2021-10-12 00:00:00
Tasks: Contrastive Learning, Sequential Recommendation

Recent advancements of sequential deep learning models such as Transformer and BERT have significantly facilitated the sequential recommendation. However, according to our study, the distribution of item embeddings generated by these models tends to degenerate into an anisotropic shape, which may result in high semantic similarities among embeddings. In this paper, both empirical and theoretical investigations of this representation degeneration problem are first provided, based on which a novel recommender model DuoRec is proposed to improve the item embeddings distribution. Specifically, in light of the uniformity property of contrastive learning, a contrastive regularization is designed for DuoRec to reshape the distribution of sequence representations. Given the convention that the recommendation task is performed by measuring the similarity between sequence representations and item embeddings in the same space via dot product, the regularization can be implicitly applied to the item embedding distribution. Existing contrastive learning methods mainly rely on data level augmentation for user-item interaction sequences through item cropping, masking, or reordering and can hardly provide semantically consistent augmentation samples. In DuoRec, a model-level augmentation is proposed based on Dropout to enable better semantic preserving. Furthermore, a novel sampling strategy is developed, where sequences having the same target item are chosen hard positive samples. Extensive experiments conducted on five datasets demonstrate the superior performance of the proposed DuoRec model compared with baseline methods. Visualization results of the learned representations validate that DuoRec can largely alleviate the representation degeneration problem.
A Time-Optimized Content Creation Workflow for Remote Teaching
Authors: Sebastian Hofstätter, Sophia Althammer, Mete Sertkan, Allan Hanbury
Date: 2021-10-11 00:00:00

We describe our workflow to create an engaging remote learning experience for a university course, while minimizing the post-production time of the educators. We make use of ubiquitous and commonly free services and platforms, so that our workflow is inclusive for all educators and provides polished experiences for students. Our learning materials provide for each lecture: 1) a recorded video, uploaded on YouTube, with exact slide timestamp indices, which enables an enhanced navigation UI; and 2) a high-quality flow-text automated transcript of the narration with proper punctuation and capitalization, improved with a student participation workflow on GitHub. All these results could be created by hand in a time consuming and costly way. However, this would generally exceed the time available for creating course materials. Our main contribution is to automate the transformation and post-production between raw narrated slides and our published materials with a custom toolchain. Furthermore, we describe our complete workflow: from content creation to transformation and distribution. Our students gave us overwhelmingly positive feedback and especially liked our use of ubiquitous platforms. The most used feature was YouTube's chapter UI enabled through our automatically generated timestamps. The majority of students, who started using the transcripts, continued to do so. Every single transcript was corrected by students, with an average word-change of 6%. We conclude with the positive feedback that our enhanced content formats are much appreciated and utilized. Important for educators is how our low overhead production workflow was sustainable throughout a busy semester.
Hotel Preference Rank based on Online Customer Review
Authors: Muhammad Apriandito Arya Saputra, Andry Alamsyah, Fajar Ibnu Fatihan
Date: 2021-10-10 00:00:00

Topline hotels are now shifting into the digital way in how they understand their customers to maintain and ensuring satisfaction. Rather than the conventional way which uses written reviews or interviews, the hotel is now heavily investing in Artificial Intelligence particularly Machine Learning solutions. Analysis of online customer reviews changes the way companies make decisions in a more effective way than using conventional analysis. The purpose of this research is to measure hotel service quality. The proposed approach emphasizes service quality dimensions reviews of the top-5 luxury hotel in Indonesia that appear on the online travel site TripAdvisor based on section Best of 2018. In this research, we use a model based on a simple Bayesian classifier to classify each customer review into one of the service quality dimensions. Our model was able to separate each classification properly by accuracy, kappa, recall, precision, and F-measure measurements. To uncover latent topics in the customer's opinion we use Topic Modeling. We found that the common issue that occurs is about responsiveness as it got the lowest percentage compared to others. Our research provides a faster outlook of hotel rank based on service quality to end customers based on a summary of the previous online review.
Two-level monotonic multistage recommender systems
Authors: Ben Dai, Xiaotong Shen, Wei Pan
Date: 2021-10-06 00:00:00
Tasks: Prediction, Recommendation Systems, Vocal Bursts Valence Prediction

A recommender system learns to predict the user-specific preference or intention over many items simultaneously for all users, making personalized recommendations based on a relatively small number of observations. One central issue is how to leverage three-way interactions, referred to as user-item-stage dependencies on a monotonic chain of events, to enhance the prediction accuracy. A monotonic chain of events occurs, for instance, in an article sharing dataset, where a ``follow'' action implies a ``like'' action, which in turn implies a ``view'' action. In this article, we develop a multistage recommender system utilizing a two-level monotonic property characterizing a monotonic chain of events for personalized prediction. Particularly, we derive a large-margin classifier based on a nonnegative additive latent factor model in the presence of a high percentage of missing observations, particularly between stages, reducing the number of model parameters for personalized prediction while guaranteeing prediction consistency. On this ground, we derive a regularized cost function to learn user-specific behaviors at different stages, linking decision functions to numerical and categorical covariates to model user-item-stage interactions. Computationally, we derive an algorithm based on blockwise coordinate descent. Theoretically, we show that the two-level monotonic property enhances the accuracy of learning as compared to a standard method treating each stage individually and an ordinal method utilizing only one-level monotonicity. Finally, the proposed method compares favorably with existing methods in simulations and an article sharing dataset.
Live Multi-Streaming and Donation Recommendations via Coupled Donation-Response Tensor Factorization
Authors: Hsu-Chao Lai, Jui-Yi Tsai, Hong-Han Shuai, Jiun-Long Huang, Wang-Chien Lee, De-Nian Yang
Date: 2021-10-05 00:00:00
Tasks: Recommendation Systems

In contrast to traditional online videos, live multi-streaming supports real-time social interactions between multiple streamers and viewers, such as donations. However, donation and multi-streaming channel recommendations are challenging due to complicated streamer and viewer relations, asymmetric communications, and the tradeoff between personal interests and group interactions. In this paper, we introduce Multi-Stream Party (MSP) and formulate a new multi-streaming recommendation problem, called Donation and MSP Recommendation (DAMRec). We propose Multi-stream Party Recommender System (MARS) to extract latent features via socio-temporal coupled donation-response tensor factorization for donation and MSP recommendations. Experimental results on Twitch and Douyu manifest that MARS significantly outperforms existing recommenders by at least 38.8% in terms of hit ratio and mean average precision.
Game Theory for Adversarial Attacks and Defenses
Authors: Shorya Sharma
Date: 2021-10-08 00:00:00
Tasks: Adversarial Defense, Denoising, Diversity, Super-Resolution

Adversarial attacks can generate adversarial inputs by applying small but intentionally worst-case perturbations to samples from the dataset, which leads to even state-of-the-art deep neural networks outputting incorrect answers with high confidence. Hence, some adversarial defense techniques are developed to improve the security and robustness of the models and avoid them being attacked. Gradually, a game-like competition between attackers and defenders formed, in which both players would attempt to play their best strategies against each other while maximizing their own payoffs. To solve the game, each player would choose an optimal strategy against the opponent based on the prediction of the opponent's strategy choice. In this work, we are on the defensive side to apply game-theoretic approaches on defending against attacks. We use two randomization methods, random initialization and stochastic activation pruning, to create diversity of networks. Furthermore, we use one denoising technique, super resolution, to improve models' robustness by preprocessing images before attacks. Our experimental results indicate that those three methods can effectively improve the robustness of deep-learning neural networks.
EEG functional connectivity and deep learning for automatic diagnosis of brain disorders: Alzheimer's disease and schizophrenia
Authors: Caroline L. Alves, Aruane M. Pineda, Kirstin Roster, Christiane Thielemann, Francisco A. Rodrigues
Date: 2021-10-07 00:00:00
Tasks: EEG, Electroencephalogram (EEG), Functional Connectivity, Time Series, Time Series Analysis

Mental disorders are among the leading causes of disability worldwide. The first step in treating these conditions is to obtain an accurate diagnosis, but the absence of established clinical tests makes this task challenging. Machine learning algorithms can provide a possible solution to this problem, as we describe in this work. We present a method for the automatic diagnosis of mental disorders based on the matrix of connections obtained from EEG time series and deep learning. We show that our approach can classify patients with Alzheimer's disease and schizophrenia with a high level of accuracy. The comparison with the traditional cases, that use raw EEG time series, shows that our method provides the highest precision. Therefore, the application of deep neural networks on data from brain connections is a very promising method to the diagnosis of neurological disorders.
Phase Noise Resilient Three-Level Continuous-Phase Modulation for DFT-Spread OFDM
Authors: Markku Renfors, Ismael Peruga Nasarre, Toni Levanen, Mikko Valkama, Kari Pajukoski
Date: 2021-10-12 00:00:00

A novel OFDM-based waveform with low peak-to-average power ratio (PAPR) and high robustness against phase noise (PN) is presented. It follows the discrete Fourier transform spread orthogonal frequency division multiplexing (DFT-s-OFDM) signal model. 3MSK, is inspired by continuous-phase frequency shift keying (FSK), but it uses three frequencies in the baseband model -- specifically, 0 and $\pm f_{symbol}/4$, where $f_{symbol}$ is the symbol rate -- which effectively constrains the phase transitions between consecutive symbols to 0 and $\pm \pi/2$ rad. Motivated by the phase controlled model of modulation, different degrees of phase continuity can be achieved, while supporting receiver processing with low complexity. The signal characteristics are improved by generating an initial time-domain nearly constant envelope signal at higher than the symbol rate. This helps to reach smooth phase transitions between 3MSK symbols. Also the possibility of using excess bandwidth is investigated by transmitting additional non-zero subcarriers outside active subcarriers of the basic DFT-s-OFDM model, which provides the capability to greatly reduce the PAPR. Due to the fact that the information is encoded in the phase transitions, a receiver model that tracks the phase variations without needing reference signals is developed. To this end, it is shown that this new modulation is well-suited for non-coherent receivers, even under strong phase noise (PN) conditions, thus allowing to reduce the overhead of reference signals. Evaluations of this physical-layer modulation and waveform scheme are performed in terms of transmitter metrics such as PAPR, OOB emissions and achievable output power after the power amplifier (PA). Finally, coded radio link evaluations are also shown and provided, demonstrating that 3MSK has a similar BER performance as that of traditional QPSK.