pwc-archive/papers-with-abstracts huggingface dataset:
Extrinsic Bayesian Optimizations on Manifolds
Authors: Yihao Fang, Mu Niu, Pokman Cheung, Lizhen Lin
Date: 2022-12-21 00:00:00
Tasks: Bayesian Optimization, Gaussian Processes, valid

We propose an extrinsic Bayesian optimization (eBO) framework for general optimization problems on manifolds. Bayesian optimization algorithms build a surrogate of the objective function by employing Gaussian processes and quantify the uncertainty in that surrogate by deriving an acquisition function. This acquisition function represents the probability of improvement based on the kernel of the Gaussian process, which guides the search in the optimization process. The critical challenge for designing Bayesian optimization algorithms on manifolds lies in the difficulty of constructing valid covariance kernels for Gaussian processes on general manifolds. Our approach is to employ extrinsic Gaussian processes by first embedding the manifold onto some higher dimensional Euclidean space via equivariant embeddings and then constructing a valid covariance kernel on the image manifold after the embedding. This leads to efficient and scalable algorithms for optimization over complex manifolds. Simulation study and real data analysis are carried out to demonstrate the utilities of our eBO framework by applying the eBO to various optimization problems over manifolds such as the sphere, the Grassmannian, and the manifold of positive definite matrices.
How Well Do Unsupervised Learning Algorithms Model Human Real-time and Life-long Learning?
Authors: Chengxu Zhuang, Violet Xiang, Yoon Bai, Xiaoxuan Jia, Nicholas Turk-Browne, Kenneth Norman, James J. DiCarlo, Daniel LK Yamins
Date: 2022-11-01 00:00:00
Proceeding: NeurIPS 2022 11
Tasks: Diversity, Self-Supervised Learning

Humans learn from visual inputs at multiple timescales, both rapidly and flexibly acquiring visual knowledge over short periods, and robustly accumulating online learning progress over longer periods. Modeling these powerful learning capabilities is an important problem for computational visual cognitive science, and models that could replicate them would be of substantial utility in real-world computer vision settings. In this work, we establish benchmarks for both real-time and life-long continual visual learning. Our real-time learning benchmark measures a model's ability to match the rapid visual behavior changes of real humans over the course of minutes and hours, given a stream of visual inputs. Our life-long learning benchmark evaluates the performance of models in a purely online learning curriculum obtained directly from child visual experience over the course of years of development. We evaluate a spectrum of recent deep self-supervised visual learning algorithms on both benchmarks, finding that none of them perfectly match human performance, though some algorithms perform substantially better than others. Interestingly, algorithms embodying recent trends in self-supervised learning -- including BYOL, SwAV and MAE -- are substantially worse on our benchmarks than an earlier generation of self-supervised algorithms such as SimCLR and MoCo-v2. We present analysis indicating that the failure of these newer algorithms is primarily due to their inability to handle the kind of sparse low-diversity datastreams that naturally arise in the real world, and that actively leveraging memory through negative sampling -- a mechanism eschewed by these newer algorithms -- appears useful for facilitating learning in such low-diversity environments. We also illustrate a complementarity between the short and long timescales in the two benchmarks, showing how requiring a single learning algorithm to be locally context-sensitive enough to match real-time learning changes while stable enough to avoid catastrophic forgetting over the long term induces a trade-off that human-like algorithms may have to straddle. Taken together, our benchmarks establish a quantitative way to directly compare learning between neural networks models and human learners, show how choices in the mechanism by which such algorithms handle sample comparison and memory strongly impact their ability to match human learning abilities, and expose an open problem space for identifying more flexible and robust visual self-supervision algorithms.
Sparse Structure Search for Delta Tuning
Authors: Shengding Hu, Zhen Zhang, Ning Ding, Yadao Wang, Yasheng Wang, Zhiyuan Liu, Maosong Sun
Date: 2022-11-01 00:00:00
Proceeding: NIPS 2022 11

Adapting large pre-trained models (PTMs) through fine-tuning imposes prohibitive computational and storage burdens. Recent studies of delta tuning (DT), i.e., parameter-efficient tuning,  find that only optimizing a small portion of parameters conditioned on PTMs could yield on-par performance compared to conventional fine-tuning. Generally, DT methods exquisitely design delta modules (DT modules) which could be applied to arbitrary fine-grained positions inside PTMs. However, the effectiveness of these fine-grained positions largely relies on sophisticated manual designation, thereby usually producing sub-optimal results. In contrast to the manual designation, we explore constructing DT modules in an automatic manner. We automatically \textbf{S}earch for the \textbf{S}parse \textbf{S}tructure of \textbf{Delta} Tuning (SDelta).   Based on a unified framework of various DT methods, SDelta conducts the differentiable DT structure search through bi-level optimization and proposes shifted global sigmoid method to explicitly control the number of trainable parameters.  Extensive experiments show that SDelta surpasses manual and random structures with less trainable parameters. The searched structures preserve more than 99\% fine-tuning performance with 0.01\% trainable parameters. Moreover, the advantage of SDelta is amplified with extremely low trainable parameters budgets (0.0009\%0.01\%). The searched structures are transferable and explainable, providing suggestions and guidance for the future design of DT methods.
Investigation of deep learning models on identification of minimum signal length for precise classification of conveyor rubber belt loads
Authors: Tadas Žvirblis, Linas Petkevičius, Damian Bzinkowski, Dominykas Vaitkus, Pranas Vaitkus, Mirosław Rucki, Artūras Kilikevičius
Date: 2022-07-19 00:00:00
Proceeding: Advances in Mechanical Engineering 2022 7
Tasks: Classification

In this paper, long short-term memory (LSTM) and Transformer neural network models were developed for classification of different conveyor belt conditions (loaded and unloaded). Comparative shallow models such as logistic regression, support vector machine and random forest were also developed and summarized. Six different-length belt pressure signals were analyzed: 0.2, 0.4, 0.8, 1.6, 3.2, and 5.0 s. Both LSTM and Transformer models achieved 100% accuracy using pressure raw signal. Furthermore, LSTM model reached the highest classification level with the shortest signals. Accuracy and F1-score of 98% and 100% were reached using only 0.8 and 1.6 s-length signals, respectively. Also, LSTM model performed training and testing procedures faster than Transformer. Random forest model demonstrated the best classification level using aggregated signal data with accuracy of 85% and F1-score for loaded and unloaded conditions of 85% and 69%, respectively. Loaded conveyor belt condition was significantly easier to classify than the unloaded one in all models. Only LSTM showed better classification recall for unloaded conveyor belt condition using short signal. Experimental research dataset CORBEL (Conveyor belt pressure signal dataset) and models are open-sourced and accessible on GitHub https://github.com/TadasZvirblis/CORBEL.
Self-progressive choice models
Authors: Kemal Yildiz
Date: 2022-12-27 00:00:00

Consider a population of heterogenous agents whose choice behaviors are partially comparable according to given primitive orderings. The set of choice functions admissible in the population specifies a choice model. A choice model is self-progressive if each aggregate choice behavior consistent with the model is uniquely representable as a probability distribution over admissible choice functions that are comparable. We establish an equivalence between self-progressive choice models and well-known algebraic structures called lattices. This equivalence provides for a precise recipe to restrict or extend any choice model for unique orderly representation. To prove out, we characterize the minimal self-progressive extension of rational choice functions, explaining why agents might exhibit choice overload. We provide necessary and sufficient conditions for the identification of a (unique) primitive ordering that renders our choice overload representation to a choice model.
Using machine learning algorithms to determine the post-COVID state of a person by his rhythmogram
Authors: Sergey Stasenko, Andrey Kovalchuk, Eremin Evgeny, Natalya Zarechnova, Maria Tsirkova, Sergey Permyakov, Sergey Parin, Sofia Polevaya
Date: 2022-12-16 00:00:00

In this study we applyed machine-learning algorithms to determine the post-COVID state of a person. During the study, a marker of the post-COVID state of a person was found in the electrocardiogram data. We have shown that this marker in the patient's ECG signal can be used to diagnose a post-COVID state.
Using machine learning algorithms to determine the emotional disadaptation of a person by his rhythmogram
Authors: Sergey Stasenko, Olga Shemagina, Eremin Evgeny, Vladimir Yakhno, Sergey Parin, Sofia Polevaya
Date: 2022-12-16 00:00:00

In this study we applyed machine-learning algorithms to determine the emotional disadaptation of a person by his rhythmogram. We used the method of determining a subject level of emotional disadaptation and recording of cardiorhythmography. We show that electrocardiogram (ECG) signals can be used for the registration of the emotional disadaptation of a person.
Calibration-Free Driver Drowsiness Classification based on Manifold-Level Augmentation
Authors: Dong-Young Kim, Dong-Kyun Han, Hye-Bin Shin
Date: 2022-12-14 00:00:00
Tasks: Brain Computer Interface, Data Augmentation, Diversity, Domain Generalization, EEG, Electroencephalogram (EEG)

Drowsiness reduces concentration and increases response time, which causes fatal road accidents. Monitoring drivers' drowsiness levels by electroencephalogram (EEG) and taking action may prevent road accidents. EEG signals effectively monitor the driver's mental state as they can monitor brain dynamics. However, calibration is required in advance because EEG signals vary between and within subjects. Because of the inconvenience, calibration has reduced the accessibility of the brain-computer interface (BCI). Developing a generalized classification model is similar to domain generalization, which overcomes the domain shift problem. Especially data augmentation is frequently used. This paper proposes a calibration-free framework for driver drowsiness state classification using manifold-level augmentation. This framework increases the diversity of source domains by utilizing features. We experimented with various augmentation methods to improve the generalization performance. Based on the results of the experiments, we found that deeper models with smaller kernel sizes improved generalizability. In addition, applying an augmentation at the manifold-level resulted in an outstanding improvement. The framework demonstrated the capability for calibration-free BCI.
Siamese Sleep Transformer For Robust Sleep Stage Scoring With Self-knowledge Distillation and Selective Batch Sampling
Authors: Heon-Gyu Kwak, Young-Seok Kweon, Gi-Hwan Shin
Date: 2022-12-12 00:00:00
Tasks: Knowledge Distillation, Self-Knowledge Distillation

In this paper, we propose a Siamese sleep transformer (SST) that effectively extracts features from single-channel raw electroencephalogram signals for robust sleep stage scoring. Despite the significant advances in sleep stage scoring in the last few years, most of them mainly focused on the increment of model performance. However, other problems still exist: the bias of labels in datasets and the instability of model performance by repetitive training. To alleviate these problems, we propose the SST, a novel sleep stage scoring model with a selective batch sampling strategy and self-knowledge distillation. To evaluate how robust the model was to the bias of labels, we used different datasets for training and testing: the sleep heart health study and the Sleep-EDF datasets. In this condition, the SST showed competitive performance in sleep stage scoring. In addition, we demonstrated the effectiveness of the selective batch sampling strategy with a reduction of the standard deviation of performance by repetitive training. These results could show that SST extracted effective learning features against the bias of labels in datasets, and the selective batch sampling strategy worked for the model robustness in training.
Statistical-CSI-Based Antenna Selection and Precoding in Uplink MIMO
Authors: Chongjun Ouyang, Ali Bereyhi, Saba Asaad, Ralf R. Müller, Hongwen Yang
Date: 2022-12-28 00:00:00

Classical antenna selection schemes require instantaneous channel state information (CSI). This leads to high signaling overhead in the system. This work proposes a novel joint receive antenna selection and precoding scheme for multiuser multiple-input multiple-output uplink transmission that relies only on the long-term statistics of the CSI. The proposed scheme designs the switching network and the uplink precoders, such that the expected throughput of the system in the long term is maximized. Invoking results from the random matrix theory, we derive a closed-form expression for the expected throughput of the system. We then develop a tractable iterative algorithm to tackle the throughput maximization problem, capitalizing on the alternating optimization and majorization-maximization (MM) techniques. Numerical results substantiate the efficiency of the proposed approach and its superior performance as compared with the baseline.
Predictive Precoder Design for OTFS-Enabled URLLC: A Deep Learning Approach
Authors: Chang Liu, Shuangyang Li, Weijie Yuan, Xuemeng Liu, Derrick Wing Kwan Ng
Date: 2022-12-28 00:00:00

This paper investigates the orthogonal time frequency space (OTFS) transmission for enabling ultra-reliable low-latency communications (URLLC). To guarantee excellent reliability performance, pragmatic precoder design is an effective and indispensable solution. However, the design requires accurate instantaneous channel state information at the transmitter (ICSIT) which is not always available in practice. Motivated by this, we adopt a deep learning (DL) approach to exploit implicit features from estimated historical delay-Doppler domain channels (DDCs) to directly predict the precoder to be adopted in the next time frame for minimizing the frame error rate (FER), that can further improve the system reliability without the acquisition of ICSIT. To this end, we first establish a predictive transmission protocol and formulate a general problem for the precoder design where a closed-form theoretical FER expression is derived serving as the objective function to characterize the system reliability. Then, we propose a DL-based predictive precoder design framework which exploits an unsupervised learning mechanism to improve the practicability of the proposed scheme. As a realization of the proposed framework, we design a DDCs-aware convolutional long short-term memory (CLSTM) network for the precoder design, where both the convolutional neural network and LSTM modules are adopted to facilitate the spatial-temporal feature extraction from the estimated historical DDCs to further enhance the precoder performance. Simulation results demonstrate that the proposed scheme facilitates a flexible reliability-latency tradeoff and achieves an excellent FER performance that approaches the lower bound obtained by a genie-aided benchmark requiring perfect ICSI at both the transmitter and receiver.
Audiovisual Database with 360 Video and Higher-Order Ambisonics Audio for Perception, Cognition, Behavior, and QoE Evaluation Research
Authors: Thomas Robotham, Ashutosh Singla, Olli S. Rummukainen, Alexander Raake, Emanuël A. P. Habets
Date: 2022-12-27 00:00:00

Research into multi-modal perception, human cognition, behavior, and attention can benefit from high-fidelity content that may recreate real-life-like scenes when rendered on head-mounted displays. Moreover, aspects of audiovisual perception, cognitive processes, and behavior may complement questionnaire-based Quality of Experience (QoE) evaluation of interactive virtual environments. Currently, there is a lack of high-quality open-source audiovisual databases that can be used to evaluate such aspects or systems capable of reproducing high-quality content. With this paper, we provide a publicly available audiovisual database consisting of twelve scenes capturing real-life nature and urban environments with a video resolution of 7680x3840 at 60 frames-per-second and with 4th-order Ambisonics audio. These 360 video sequences, with an average duration of 60 seconds, represent real-life settings for systematically evaluating various dimensions of uni-/multi-modal perception, cognition, behavior, and QoE. The paper provides details of the scene requirements, recording approach, and scene descriptions. The database provides high-quality reference material with a balanced focus on auditory and visual sensory information. The database will be continuously updated with additional scenes and further metadata such as human ratings and saliency information.
Hierarchical Deep Reinforcement Learning for Age-of-Information Minimization in IRS-aided and Wireless-powered Wireless Networks
Authors: Shimin Gong, Leiyang Cui, Bo Gu, Bin Lyu, Dinh Thai Hoang, Dusit Niyato
Date: 2022-12-27 00:00:00
Tasks: Deep Reinforcement Learning, Fairness, Scheduling

In this paper, we focus on a wireless-powered sensor network coordinated by a multi-antenna access point (AP). Each node can generate sensing information and report the latest information to the AP using the energy harvested from the AP's signal beamforming. We aim to minimize the average age-of-information (AoI) by adapting the nodes' transmission scheduling and the transmission control strategies jointly. To reduce the transmission delay, an intelligent reflecting surface (IRS) is used to enhance the channel conditions by controlling the AP's beamforming vector and the IRS's phase shifting matrix. Considering dynamic data arrivals at different sensing nodes, we propose a hierarchical deep reinforcement learning (DRL) framework to for AoI minimization in two steps. The users' transmission scheduling is firstly determined by the outer-loop DRL approach, e.g. the DQN or PPO algorithm, and then the inner-loop optimization is used to adapt either the uplink information transmission or downlink energy transfer to all nodes. A simple and efficient approximation is also proposed to reduce the inner-loop rum time overhead. Numerical results verify that the hierarchical learning framework outperforms typical baselines in terms of the average AoI and proportional fairness among different nodes.
Electric-Gas Infrastructure Planning for Deep Decarbonization of Energy Systems
Authors: Rahman Khorramfar, Dharik Mallapragada, Saurabh Amin
Date: 2022-12-28 00:00:00
Tasks: Scheduling

The transition to a deeply decarbonized energy system requires coordinated planning of infrastructure investments and operations serving multiple end-uses while considering technology and policy-enabled interactions across sectors. Electricity and natural gas (NG), which are vital vectors of today's energy system, are likely to be coupled in different ways in the future, resulting from increasing electrification, adoption of variable renewable energy (VRE) generation in the power sector and policy factors such as cross-sectoral emissions trading. This paper develops a least-cost investment and operations model for joint planning of electricity and NG infrastructures that considers a wide range of available and emerging technology options across the two vectors, including carbon capture and storage (CCS) equipped power generation, low-carbon drop-in fuels (LCDF) as well as long-duration energy storage (LDES). The model incorporates the main operational constraints of both systems and allows each system to operate under different temporal resolutions consistent with their typical scheduling timescales. We apply the modeling framework to evaluate power-NG system outcomes for the U.S. New England region under different technology, decarbonization goals, and demand scenarios. Under a global emissions constraint, ranging between 80-95\% emissions reduction compared to 1990 levels, the least-cost solution disproportionately relies on using the available emissions budget to serve non-power NG demand and results in the power sector using only 15-43\% of the emissions budget.
Impacts of Earthquakes on Electrical Grid Resilience
Authors: Adam Mate, Travis Hagan, Eduardo Cotilla-Sanchez, Ted K. A. Brekken, Annette Von Jouanne
Date: 2021-01-20 00:00:00

One of the most complex and devastating disaster scenarios that the U.S.~Pacific Northwest region and the state of Oregon faces is a large magnitude Cascadia Subduction Zone earthquake event. The region's electrical grid lacks in resilience against the destruction of a megathrust earthquake, a powerful tsunami, hundreds of aftershocks and increased volcanic activity, all of which are highly probable components of this hazard. This research seeks to catalyze further understanding and improvement of resilience. By systematizing power system related experiences of historical earthquakes, and collecting practical and innovative ideas from other regions on how to enhance network design, construction, and operation, important steps are being taken toward a more resilient, earthquake-resistant grid. This paper presents relevant findings in an effort to be an overview and a useful guideline for those who are also working towards greater electrical grid resilience.
Contraction and $k$-contraction in Lurie systems with applications to networked systems
Authors: Ron Ofir, Alexander Ovseevich, Michael Margaliot
Date: 2022-12-27 00:00:00
Tasks: LEMMA

A Lurie system is the interconnection of a linear time-invariant system and a nonlinear feedback function. We derive a new sufficient condition for $k$-contraction of a Lurie system. For $k=1$, our sufficient condition reduces to the standard stability condition based on the bounded real lemma and a small gain condition. However, Lurie systems often have more than a single equilibrium and are thus not contractive with respect to any norm. For $k=2$, our condition guarantees a well-ordered asymptotic behaviour of the closed-loop system: every bounded solution converges to an equilibrium, which is not necessarily unique. We demonstrate our results by deriving a sufficient condition for $k$-contraction of a general networked system, and then applying it to guarantee $k$-contraction in a Hopfield neural network, a nonlinear opinion dynamics model, and a 2-bus power system.