pwc-archive/papers-with-abstracts huggingface dataset:
Monarch Mixer: A Simple Sub-Quadratic GEMM-Based Architecture
Authors: Daniel Y. Fu, Simran Arora, Jessica Grogan, Isys Johnson, Sabri Eyuboglu, Armin W. Thomas, Benjamin Spector, Michael Poli, Atri Rudra, Christopher Ré
Date: 2023-10-18 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: 4k, image-classification, Image Classification, Language Modeling, Language Modelling

Machine learning models are increasingly being scaled in both sequence length and model dimension to reach longer contexts and better performance. However, existing architectures such as Transformers scale quadratically along both these axes. We ask: are there performant architectures that can scale sub-quadratically along sequence length and model dimension? We introduce Monarch Mixer (M2), a new architecture that uses the same sub-quadratic primitive along both sequence length and model dimension: Monarch matrices, a simple class of expressive structured matrices that captures many linear transforms, achieves high hardware efficiency on GPUs, and scales sub-quadratically. As a proof of concept, we explore the performance of M2 in three domains: non-causal BERT-style language modeling, ViT-style image classification, and causal GPT-style language modeling. For non-causal BERT-style modeling, M2 matches BERT-base and BERT-large in downstream GLUE quality with up to 27% fewer parameters, and achieves up to 9.1$\times$ higher throughput at sequence length 4K. On ImageNet, M2 outperforms ViT-b by 1% in accuracy, with only half the parameters. Causal GPT-style models introduce a technical challenge: enforcing causality via masking introduces a quadratic bottleneck. To alleviate this bottleneck, we develop a novel theoretical view of Monarch matrices based on multivariate polynomial evaluation and interpolation, which lets us parameterize M2 to be causal while remaining sub-quadratic. Using this parameterization, M2 matches GPT-style Transformers at 360M parameters in pretraining perplexity on The PILE--showing for the first time that it may be possible to match Transformer quality without attention or MLPs.
Learning-to-Rank Meets Language: Boosting Language-Driven Ordering Alignment for Ordinal Classification
Authors: Rui Wang, Peipei Li, Huaibo Huang, Chunshui Cao, Ran He, Zhaofeng He
Date: 2023-06-24 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: Age Estimation, Classification, Learning-To-Rank, Ordinal Classification

We present a novel language-driven ordering alignment method for ordinal classification. The labels in ordinal classification contain additional ordering relations, making them prone to overfitting when relying solely on training data. Recent developments in pre-trained vision-language models inspire us to leverage the rich ordinal priors in human language by converting the original task into a visionlanguage alignment task. Consequently, we propose L2RCLIP, which fully utilizes the language priors from two perspectives. First, we introduce a complementary prompt tuning technique called RankFormer, designed to enhance the ordering relation of original rank prompts. It employs token-level attention with residual-style prompt blending in the word embedding space. Second, to further incorporate language priors, we revisit the approximate bound optimization of vanilla cross-entropy loss and restructure it within the cross-modal embedding space. Consequently, we propose a cross-modal ordinal pairwise loss to refine the CLIP feature space, where texts and images maintain both semantic alignment and ordering alignment. Extensive experiments on three ordinal classification tasks, including facial age estimation, historical color image (HCI) classification, and aesthetic assessment demonstrate its promising performance. The code is available at https://github.com/raywang335/L2RCLIP.
A Study on Survival Analysis Methods Using Neural Network to Prevent Cancers
Authors: Chul-Young Bae, Bo-Seon Kim, Sun-Ha Jee, Jong-Hoon Lee, Ngoc-Dung Nguyen
Date: 2023-09-27 00:00:00
Proceeding: Cancers 2023 9
Tasks: regression, Survival Analysis

Background: Cancer is one of the main global health threats. Early personalized prediction of cancer incidence is crucial for the population at risk. This study introduces a novel cancer prediction model based on modern recurrent survival deep learning algorithms. 
Methods: The study includes 160,407 participants from the blood-based cohort of the Korea Cancer Prevention Research-II Biobank, which has been ongoing since 2004. Data linkages were designed to ensure anonymity, and data collection was carried out through nationwide medical examinations. Predictive performance on ten cancer sites, evaluated using the concordance index (c-index), was compared among nDeep and its multitask variation, Cox proportional hazard (PH) regression, DeepSurv, and DeepHit. 
Results: Our models consistently achieved a c-index of over 0.8 for all ten cancers, with a peak of 0.8922 for lung cancer. They outperformed Cox PH regression and other survival deep neural networks. 
Conclusion: This study presents a survival deep learning model that demonstrates the highest predictive performance on censored health dataset, to the best of our knowledge. In the future, we plan to investigate the causal relationship between explanatory variables and cancer to reduce cancer incidence and mortality.
DeepMAO: Deep Multi-scale Aware Overcomplete Network for Building Segmentation in Satellite Imagery
Authors: Aniruddh Sikdar, Sumanth Udupa, Prajwal Gurunath, Suresh Sundaram
Date: 2023-08-14 00:00:00
Proceeding: Computer Vision and Pattern Recognition, Perception Beyond Visible Spectrum Workshop 2023 8
Tasks: Segmentation, The Semantic Segmentation Of Remote Sensing Imagery

Building segmentation in large-scale aerial images is challenging, especially for small buildings in dense and cluttered urban environments. Complex building structures with highly varied geometric footprints pose an additional challenge for the building segmentation task in satellite imagery. In this work, we propose to tackle the issue of detecting and segmenting small and complex-shaped buildings in Electro-Optical (EO) and SAR satellite imagery. A novel architecture Deep Multi-scale Aware Overcomplete Network (DeepMAO), is proposed that comprises an overcomplete branch that focuses on fine structural features and an undercomplete (U-Net) branch tasked to focus on coarse, semantic-rich features. Additionally, a novel self-regulating augmentation strategy, "Loss-Mix," is proposed to increase pixel representation of misclassified pixels. DeepMAO is simple and efficient in accurately identifying small and geometrically complex buildings. Experimental results on SpaceNet 6 dataset, on both EO and SAR modalities, and the INRIA dataset show that DeepMAO achieves state-of-the-art building segmentation performance, including small and complex-shaped buildings with a negligible increase in the parameter count. In addition, the presence of the over-complete branch in DeepMAO helps in handling the speckle noise present in the SAR image modality.
PulsatioMech: An Open-Source MATLAB Toolbox for Seismocardiography Signal Processing
Authors: Nathan Zavanelli
Date: 2024-01-10 00:00:00

This paper summarizes and presents PulsatioMech: an open-source MATLAB toolbox for seismocardiography (SCG) signal processing. The toolbox may be found here: https://github.com/nzavanelli/SCG_master_toolbox PulsatioMech is currently under development as a common tool to promote new studies and discoveries in the use of cardiac mechanical signal for wearable health monitoring. This toolbox is designed to assist users in analyzing SCG signals without the need to devote significant effort into signal processing and coding tasks. Simultaneously, it provides a uniform basis to assess the reproducibility of works based on this toolbox, including those cited here [1-6]. The referenced works contain a great deal more detail regarding the specific algorithms implemented here, whereas this paper will present a short overview of the PulsatioMech Toolbox.
On the existence of funneled orientations for classes of rooted phylogenetic networks
Authors: Janosch Döcker, Simone Linz
Date: 2024-01-11 00:00:00

Recently, there has been a growing interest in the relationships between unrooted and rooted phylogenetic networks. In this context, a natural question to ask is if an unrooted phylogenetic network U can be oriented as a rooted phylogenetic network such that the latter satisfies certain structural properties. In a recent preprint, Bulteau et al. claim that it is computational hard to decide if U has a funneled (resp. funneled tree-child) orientation, for when the internal vertices of U have degree at most 5. Unfortunately, the proof of their funneled tree-child result appears to be incorrect. In this paper, we present a corrected proof and show that hardness remains for other popular classes of rooted phylogenetic networks such as funneled normal and funneled reticulation-visible. Additionally, our results hold regardless of whether U is rooted at an existing vertex or by subdividing an edge with the root.
Slide FFT on a homogeneous mesh in wafer-scale computing
Authors: Maurice H. P. M. van Putten, Leighton Wilson, Adam W. Lavely, Mark Hair
Date: 2024-01-04 00:00:00
Tasks: Astronomy

Searches for signals at low signal-to-noise ratios frequently involve the Fast Fourier Transform (FFT). For high-throughput searches, we here consider FFT on the homogeneous mesh of Processing Elements (PEs) of a wafer-scale engine (WSE). To minimize memory overhead in the inherently non-local FFT algorithm, we introduce a new synchronous slide operation ({\em Slide}) exploiting the fast interconnect between adjacent PEs. Feasibility of compute-limited performance is demonstrated in linear scaling of Slide execution times with varying array size in preliminary benchmarks on the CS-2 WSE. The proposed implementation appears opportune to accelerate and open the full discovery potential of FFT-based signal processing in multi-messenger astronomy.
An impossibility theorem concerning positive involvement in voting
Authors: Wesley H. Holliday
Date: 2024-01-11 00:00:00
Tasks: LEMMA

In social choice theory with ordinal preferences, a voting method satisfies the axiom of positive involvement if adding to a preference profile a voter who ranks an alternative uniquely first cannot cause that alternative to go from winning to losing. In this note, we prove a new impossibility theorem concerning this axiom: there is no ordinal voting method satisfying positive involvement that also satisfies the Condorcet winner and loser criteria, resolvability, and a common invariance property for Condorcet methods, namely that the choice of winners depends only on the ordering of majority margins by size.
Neural Ambisonics encoding for compact irregular microphone arrays
Authors: Mikko Heikkinen, Archontis Politis, Tuomas Virtanen
Date: 2024-01-11 00:00:00

Ambisonics encoding of microphone array signals can enable various spatial audio applications, such as virtual reality or telepresence, but it is typically designed for uniformly-spaced spherical microphone arrays. This paper proposes a method for Ambisonics encoding that uses a deep neural network (DNN) to estimate a signal transform from microphone inputs to Ambisonics signals. The approach uses a DNN consisting of a U-Net structure with a learnable preprocessing as well as a loss function consisting of mean average error, spatial correlation, and energy preservation components. The method is validated on two microphone arrays with regular and irregular shapes having four microphones, on simulated reverberant scenes with multiple sources. The results of the validation show that the proposed method can meet or exceed the performance of a conventional signal-independent Ambisonics encoder on a number of error metrics.
Localizing Acoustic Energy in Sound Field Synthesis by Directionally Weighted Exterior Radiation Suppression
Authors: Yoshihide Tomita, Shoichi Koyama, Hiroshi Saruwatari
Date: 2024-01-11 00:00:00

A method for synthesizing the desired sound field while suppressing the exterior radiation power with directional weighting is proposed. The exterior radiation from the loudspeakers in sound field synthesis systems can be problematic in practical situations. Although several methods to suppress the exterior radiation have been proposed, suppression in all outward directions is generally difficult, especially when the number of loudspeakers is not sufficiently large. We propose the directionally weighted exterior radiation representation to prioritize the suppression directions by incorporating it into the optimization problem of sound field synthesis. By using the proposed representation, the exterior radiation in the prioritized directions can be significantly reduced while maintaining high interior synthesis accuracy, owing to the relaxed constraint on the exterior radiation. Its performance is evaluated with the application of the proposed representation to amplitude matching in numerical experiments.
An Opportunistic Source Synthesis Method for Smart Electromagnetic Environments
Authors: Pietro Da Rù, Arianna Benoni, Marco Salucci, Paolo Rocca, Andrea Massa
Date: 2024-01-11 00:00:00

In the framework of the "Smart ElectroMagnetic Environment" (SEME), an innovative strategy leveraging Equivalence Source concepts is introduced for enhancing the performance of large-scale outdoor wireless communication systems. The proposed Opportunistic Sources Synthesis (OSS) approach is aimed at unconventionally synthesizing the primary source (i.e., the base transceiver station (BTS) antenna array), so that the complex scattering phenomena induced in the surrounding scatterers are profitably exploited to enhance the received power within user-defined regions of interest (RoIs). To yield a computationally feasible synthesis process, an innovative "Embedded-plus-Environment Patterns" (EPEPs) method is introduced. A set of representative numerical examples, concerned with realistic large-scale outdoor scenarios, is presented to assess the effectiveness and the efficiency of the proposed optimization-driven approach for a realistic SEME implementation.
On the Kite-Platform Interactions in Offshore Airborne Wind Energy Systems: Frequency Analysis and Control Approach
Authors: Sofia Trombini, Edoardo Pasta, Lorenzo Fagiano
Date: 2024-01-11 00:00:00

This study investigates deep offshore, pumping Airborne Wind Energy systems, focusing on the kite-platform interaction. The considered system includes a 360 m2 soft-wing kite, connected by a tether to a winch installed on a 10-meter-deep spar with four mooring lines. Wind power is converted into electricity with a feedback controlled periodic trajectory of the kite and corresponding reeling motion of the tether. An analysis of the mutual influence between the platform and the kite dynamics, with different wave regimes, reveals a rather small sensitivity of the flight pattern to the platform oscillations; on the other hand, the frequency of tether force oscillations can be close to the platform resonance peaks, resulting in possible increased fatigue loads and damage of the floating and submerged components. A control design procedure is then proposed to avoid this problem, acting on the kite path planner. Simulation results confirm the effectiveness of the approach.
A Lightweight Energy Management Method for Hybrid PV/Battery/Load Systems
Authors: Mohsen Banaei, Razgar Ebrahimy, Henrik Madsen
Date: 2024-01-11 00:00:00
Tasks: energy management, Management, Model Predictive Control

In this paper, a computationally lightweight algorithm is introduced for hybrid PV/Battery/Load systems that is price responsive, responds fast, does not require powerful hardware, and considers the operational limitations of the system. The method is applied to two buildings equipped with PV and battery. Simulation results show that the method can give results that are up to 3.9% more expensive than the Model predictive control (MPC) approach while the runtime of the program is up to 1000 times less than the MPC. Also, while the runtime of the proposed method is in the range of the self-consumption maximization (SCM) approach as the fastest method, its electricity cost is about 3.2% cheaper than the SCM method. Simulation results also show that in case of providing grid services by the battery the difference between electricity cost of the proposed approach and MPC can reduce which makes the method good for such applications.
Secure Dynamic Event-triggered Consensus Under Asynchronous Denial of Service
Authors: Ali Azarbahram, Amir Amini
Date: 2024-01-11 00:00:00

This article proposes a secure implementation for consensus using a dynamic event-triggered (DET) communication scheme in high-order nonlinear multi-agent systems (MAS) under asynchronous (distributed) denial of service (DoS) attacks. By introducing a linear auxiliary trajectory of the system, the DET data transmission scheme among the neighboring agents is employed to reduce the communication for each agent. The asynchronous DoS attacks can block each communication channel among the cooperative agents independently in an unknown pattern. To guarantee state consensus of auxiliary MAS under DoS, a linear matrix inequality (LMI) based optimization approach is proposed which simultaneously designs all the unknown DET communication parameters as well as the state feedback control gain. In addition to asynchronous DoS attacks over the graph topology, the destructive effects of independent DoS attacks over the communication links between actual and auxiliary states are compensated as an additional layer of resiliency for the system. The output of each agent ultimately tracks the auxiliary state of the system and this results in the output consensus.
Privacy Analysis of Affine Transformations in Cloud-based MPC: Vulnerability to Side-knowledge
Authors: Teimour Hosseinalizadeh, Nils Schlüter, Moritz Schulze Darup, Nima Monshizadeh
Date: 2024-01-11 00:00:00
Tasks: Form, Model Predictive Control, Privacy Preserving

Search for the optimizer in computationally demanding model predictive control (MPC) setups can be facilitated by Cloud as a service provider in cyber-physical systems. This advantage introduces the risk that Cloud can obtain unauthorized access to the privacy-sensitive parameters of the system and cost function. To solve this issue, i.e., preventing Cloud from accessing the parameters while benefiting from Cloud computation, random affine transformations provide an exact yet light weight in computation solution. This research deals with analyzing privacy preserving properties of these transformations when they are adopted for MPC problems. We consider two common strategies for outsourcing the optimization required in MPC problems, namely separate and dense forms, and establish that random affine transformations utilized in these forms are vulnerable to side-knowledge from Cloud. Specifically, we prove that the privacy guarantees of these methods and their extensions for separate form are undermined when a mild side-knowledge about the problem in terms of structure of MPC cost function is available. In addition, while we prove that outsourcing the MPC problem in the dense form inherently leads to some degree of privacy for the system and cost function parameters, we also establish that affine transformations applied to this form are nevertheless prone to be undermined by a Cloud with mild side-knowledge. Numerical simulations confirm our results.
Tracking Consensus of Networked Random Nonlinear Multi-agent Systems with Intermittent Communications
Authors: Ali Azarbahram
Date: 2024-01-11 00:00:00

The paper proposes an intermittent communication mechanism for the tracking consensus of high-order nonlinear multi-agent systems (MASs) surrounded by random disturbances. Each collaborating agent is described by a class of high-order nonlinear uncertain strict-feedback dynamics which is disturbed by a wide stationary process representing the external noise. The resiliency level of this networked control system (NCS) to the failures of physical devices or unreliability of communication channels is analyzed by introducing a linear auxiliary trajectory of the system. More precisely, the unreliability of communication channels sometimes makes an agent incapable of sensing the local information or receiving it from neighboring nodes. Therefore, an intermittent communication scheme is proposed among the follower agents as a consequence of employing the linear auxiliary dynamics. The closed-loop networked system signals are proved to be noise-to-state practically stable in probability (NSpS-P). It has been justified that each agent follows the trajectory of the corresponding local auxiliary virtual system practically in probability. The simulation experiments finally quantify the effectiveness of our proposed approach in terms of providing a resilient performance against unreliability of communication channels and reaching the tracking consensus.
Online input design for discrimination of linear models using concave minimization
Authors: Jacques Noom, Oleg Soloviev, Carlas Smith, Michel Verhaegen
Date: 2024-01-11 00:00:00
Tasks: Fault Diagnosis

Stochastic Closed-Loop Active Fault Diagnosis (CLAFD) aims to select the input sequentially in order to improve the discrimination of different models by minimizing the predicted error probability. As computation of these error probabilities encompasses the evaluation of multidimensional probability integrals, relaxation methods are of interest. This manuscript presents a new method that allows to make an improved trade-off between three factors -- namely maximized accuracy of diagnosis, minimized number of consecutive measurements to achieve that accuracy, and minimized computational effort per time step -- with respect to the state-of-the-art. It relies on minimizing an upper bound on the error probability, which is in the case of linear models with Gaussian noise proven to be concave in the most challenging discrimination conditions. A simulation study is conducted both for open-loop and feedback controlled candidate models. The results demonstrate the favorable trade-off using the new contributions in this manuscript.
Full-State Prescribed Performance-Based Consensus of Double-Integrator Multi-Agent Systems with Jointly Connected Topologies
Authors: Yahui Hou, Bin Cheng
Date: 2024-01-11 00:00:00
Tasks: Position

This paper addresses the full-state prescribed performance-based consensus problem for double-integrator multi-agent systems with jointly connected topologies. To improve the transient performance, a distributed prescribed performance control protocol consisting of the transformed relative position and the transformed relative velocity is proposed, where the communication topology satisfies the jointly connected assumption. Different from the existing literatures, two independent transient performance specifications imposed on relative positions and relative velocities can be guaranteed simultaneously. A numerical example is ultimately used to validate the effectiveness of proposed protocol.