pwc-archive/papers-with-abstracts huggingface dataset:
Emergence of Segmentation with Minimalistic White-Box Transformers
Authors: Yaodong Yu, Tianzhe Chu, Shengbang Tong, Ziyang Wu, Druv Pai, Sam Buchanan, Yi Ma
Date: 2023-08-30 00:00:00
Tasks: Segmentation, Self-Supervised Learning

Transformer-like models for vision tasks have recently proven effective for a wide range of downstream applications such as segmentation and detection. Previous works have shown that segmentation properties emerge in vision transformers (ViTs) trained using self-supervised methods such as DINO, but not in those trained on supervised classification tasks. In this study, we probe whether segmentation emerges in transformer-based models solely as a result of intricate self-supervised learning mechanisms, or if the same emergence can be achieved under much broader conditions through proper design of the model architecture. Through extensive experimental results, we demonstrate that when employing a white-box transformer-like architecture known as CRATE, whose design explicitly models and pursues low-dimensional structures in the data distribution, segmentation properties, at both the whole and parts levels, already emerge with a minimalistic supervised training recipe. Layer-wise finer-grained analysis reveals that the emergent properties strongly corroborate the designed mathematical functions of the white-box network. Our results suggest a path to design white-box foundation models that are simultaneously highly performant and mathematically fully interpretable. Code is at \url{https://github.com/Ma-Lab-Berkeley/CRATE}.
Can Prompt Learning Benefit Radiology Report Generation?
Authors: Jun Wang, Lixing Zhu, Abhir Bhalerao, Yulan He
Date: 2023-08-30 00:00:00
Tasks: Image Captioning, Prompt Engineering, Prompt Learning

Radiology report generation aims to automatically provide clinically meaningful descriptions of radiology images such as MRI and X-ray. Although great success has been achieved in natural scene image captioning tasks, radiology report generation remains challenging and requires prior medical knowledge. In this paper, we propose PromptRRG, a method that utilizes prompt learning to activate a pretrained model and incorporate prior knowledge. Since prompt learning for radiology report generation has not been explored before, we begin with investigating prompt designs and categorise them based on varying levels of knowledge: common, domain-specific and disease-enriched prompts. Additionally, we propose an automatic prompt learning mechanism to alleviate the burden of manual prompt engineering. This is the first work to systematically examine the effectiveness of prompt learning for radiology report generation. Experimental results on the largest radiology report generation benchmark, MIMIC-CXR, demonstrate that our proposed method achieves state-of-the-art performance. Code will be available upon the acceptance.
Robust Principles: Architectural Design Principles for Adversarially Robust CNNs
Authors: Shengyun Peng, Weilin Xu, Cory Cornelius, Matthew Hull, Kevin Li, Rahul Duggal, Mansi Phute, Jason Martin, Duen Horng Chau
Date: 2023-08-30 00:00:00
Tasks: Adversarial Robustness

Our research aims to unify existing works' diverging opinions on how architectural components affect the adversarial robustness of CNNs. To accomplish our goal, we synthesize a suite of three generalizable robust architectural design principles: (a) optimal range for depth and width configurations, (b) preferring convolutional over patchify stem stage, and (c) robust residual block design through adopting squeeze and excitation blocks and non-parametric smooth activation functions. Through extensive experiments across a wide spectrum of dataset scales, adversarial training methods, model parameters, and network design spaces, our principles consistently and markedly improve AutoAttack accuracy: 1-3 percentage points (pp) on CIFAR-10 and CIFAR-100, and 4-9 pp on ImageNet. The code is publicly available at https://github.com/poloclub/robust-principles.
Active Neural Mapping
Authors: Zike Yan, Haoxiang Yang, Hongbin Zha
Date: 2023-08-30 00:00:00
Proceeding: ICCV 2023 1

We address the problem of active mapping with a continually-learned neural scene representation, namely Active Neural Mapping. The key lies in actively finding the target space to be explored with efficient agent movement, thus minimizing the map uncertainty on-the-fly within a previously unseen environment. In this paper, we examine the weight space of the continually-learned neural field, and show empirically that the neural variability, the prediction robustness against random weight perturbation, can be directly utilized to measure the instant uncertainty of the neural map. Together with the continuous geometric information inherited in the neural map, the agent can be guided to find a traversable path to gradually gain knowledge of the environment. We present for the first time an active mapping system with a coordinate-based implicit neural representation for online scene reconstruction. Experiments in the visually-realistic Gibson and Matterport3D environment demonstrate the efficacy of the proposed method.
Information Theoretically Optimal Sample Complexity of Learning Dynamical Directed Acyclic Graphs
Authors: Mishfad Shaikh Veedu, Deepjyoti Deka, Murti V. Salapaka
Date: 2023-08-31 00:00:00

In this article, the optimal sample complexity of learning the underlying interactions or dependencies of a Linear Dynamical System (LDS) over a Directed Acyclic Graph (DAG) is studied. We call such a DAG underlying an LDS as dynamical DAG (DDAG). In particular, we consider a DDAG where the nodal dynamics are driven by unobserved exogenous noise sources that are wide-sense stationary (WSS) in time but are mutually uncorrelated, and have the same {power spectral density (PSD)}. Inspired by the static DAG setting, a metric and an algorithm based on the PSD matrix of the observed time series are proposed to reconstruct the DDAG. It is shown that the optimal sample complexity (or length of state trajectory) needed to learn the DDAG is $n=\Theta(q\log(p/q))$, where $p$ is the number of nodes and $q$ is the maximum number of parents per node. To prove the sample complexity upper bound, a concentration bound for the PSD estimation is derived, under two different sampling strategies. A matching min-max lower bound using generalized Fano's inequality also is provided, thus showing the order optimality of the proposed algorithm.
Non-standard power grid frequency statistics in Asia, Australia, and Europe
Authors: Xinyi Wen, Mehrnaz Anvari, Leonardo Rydin Gorjao, G. Cigdem Yalcin, Veit Hagenmeyer, Benjamin Schafer
Date: 2023-08-31 00:00:00

The power-grid frequency reflects the balance between electricity supply and demand. Measuring the frequency and its variations allows monitoring of the power balance in the system and, thus, the grid stability. In addition, gaining insight into the characteristics of frequency variations and defining precise evaluation metrics for these variations enables accurate assessment of the performance of forecasts and synthetic models of the power-grid frequency. Previous work was limited to a few geographical regions and did not quantify the observed effects. In this contribution, we analyze and quantify the statistical and stochastic properties of self-recorded power-grid frequency data from various synchronous areas in Asia, Australia, and Europe at a resolution of one second. Revealing non-standard statistics of both empirical and synthetic frequency data, we effectively constrain the space of possible (stochastic) power-grid frequency models and share a range of analysis tools to benchmark any model or characterize empirical data. Furthermore, we emphasize the need to analyze data from a large range of synchronous areas to obtain generally applicable models.
Cost-effective Planning of Decarbonized Power-Gas Infrastructure to Meet the Challenges of Heating Electrification
Authors: Rahman Khorramfar, Morgan Santoni-Colvin, Saurabh Amin, Leslie K. Norford, Audun Botterud, Dharik Mallapragada
Date: 2023-08-31 00:00:00

Building heat electrification is central to economy-wide decarbonization efforts and directly affects energy infrastructure planning through increasing electricity demand and reducing the building sector's use of gas infrastructure that also serves the power sector. Here, we develop a modeling framework to quantify end-use demand for electricity and gas in the buildings sector under various electrification pathways and evaluate their impact on co-optimized bulk power-gas infrastructure investments and operations under deep decarbonization scenarios. Applying the framework to study the U.S. New England region in 2050 across 20 weather scenarios, we find high electrification of the residential sector can increase sectoral peak and total electricity demands by up to 56-158% and 41-59% respectively relative to business-as-usual projections. Employing demand-side measures like building envelope improvements under high electrification, however, can reduce the magnitude and weather sensitivity of peak load as well as induce overall efficiency gains, reducing the combined residential sector energy demand for power and gas by 28-30% relative to the present day. Notably, a combination of high electrification and envelope improvements yields the lowest bulk power-gas system cost outcomes. Accounting for \bt{midstream} methane emissions from gas supply chain increase the reliance on low-carbon fuels, which indirectly improves the cost-effectiveness of end-use electrification. Similarly, we find that demand flexibility programs can reduce the total system cost by up to 6.3%.
Security Allocation in Networked Control Systems under Stealthy Attacks
Authors: Anh Tung Nguyen, André M. H. Teixeira, Alexander Medvedev
Date: 2023-08-31 00:00:00

This paper considers the problem of security allocation in a networked control system under stealthy attacks. The system is comprised of interconnected subsystems represented by vertices. A malicious adversary selects a single vertex on which to conduct a stealthy data injection attack with the purpose of maximally disrupting a distant target vertex while remaining undetected. Defense resources against the adversary are allocated by a defender on several selected vertices. First, the objectives of the adversary and the defender with uncertain targets are formulated in a probabilistic manner, resulting in an expected worst-case impact of stealthy attacks. Next, we provide a graph-theoretic necessary and sufficient condition under which the cost for the defender and the expected worst-case impact of stealthy attacks are bounded. This condition enables the defender to restrict the admissible actions to dominating sets of the graph representing the network. Then, the security allocation problem is solved through a Stackelberg game-theoretic framework. Finally, the obtained results are validated through a numerical example of a 50-vertex networked control system.
Enhancing Frequency Control through Rate of Change of Voltage Feedback
Authors: Federico Milano, Bibi Alhanjari, Georgios Tzounas
Date: 2023-08-31 00:00:00

This letter proposes a simple and inexpensive technique to improve the frequency control of distributed energy resources. The proposed control consists in modifying the conventional estimated bus frequency signal with an additional feedback signal that utilizes the rate of change of the voltage magnitude measured at the same bus. The case study showcases the benefits of the proposed control and compares its performance with standard frequency control schemes through time-domain simulations.
Distributed Nonblocking Supervisory Control of Timed Discrete-Event Systems with Communication Delays and Losses
Authors: Yunfeng Hou, Qingdu Li
Date: 2023-08-31 00:00:00

This paper investigates the problem of distributed nonblocking supervisory control for timed discrete-event systems (DESs). The distributed supervisors communicate with each other over networks subject to nondeterministic communication delays and losses. Given that the delays are counted by time, techniques have been developed to model the dynamics of the communication channels. By incorporating the dynamics of the communication channels into the system model, we construct a communication automaton to model the interaction process between the supervisors. Based on the communication automaton, we define the observation mappings for the supervisors, which consider delays and losses occurring in the communication channels. Then, we derive the necessary and sufficient conditions for the existence of a set of supervisors for distributed nonblocking supervisory control. These conditions are expressed as network controllability, network joint observability, and system language closure. Finally, an example of intelligent manufacturing is provided to show the application of the proposed framework.
Multi-Stage Expansion Planning for Decarbonizing Thermal Generation Supported Renewable Power Systems Using Hydrogen and Ammonia Storage
Authors: Zhipeng Yu, Jin Lin, Feng Liu, Jiarong Li, Yingtian Chi, Yonghua Song, Zhengwei Ren
Date: 2023-08-31 00:00:00

Large-scale centralized development of wind and solar energy and peer-to-grid transmission of renewable energy source (RES) via high voltage direct current (HVDC) has been regarded as one of the most promising ways to achieve goals of peak carbon and carbon neutrality in China. Traditionally, large-scale thermal generation is needed to economically support the load demand of HVDC with a given profile, which in turn raises concerns about carbon emissions. To address the issues above, hydrogen energy storage system (HESS) and ammonia energy storage system (AESS) are introduced to gradually replace thermal generation, which is represented as a multi-stage expansion planning (MSEP) problem. Specifically, first, HESS and AESS are established in the MSEP model with carbon emission reduction constraints, and yearly data with hourly time resolution are utilized for each stage to well describe the intermittence of RES. Then, a combined Dantzig-Wolfe decomposition (DWD) and column generation (CG) solution approach is proposed to efficiently solve the large-scale MSEP model. Finally, a real-life system in China is studied. The results indicate that HESS and AESS have the potential to handle the intermittence of RES, as well as the monthly imbalance between RES and load demand. Especially under the goal of carbon neutrality, the contribution of HESS and AESS in reducing levelized cost of energy (LCOE) reaches 12.28% and 14.59%, respectively, which finally leads to a LCOE of 0.4324 RMB/kWh.
PASA: A Priori Adaptive Splitting Algorithm for the Split Delivery Vehicle Routing Problem
Authors: Nariman Torkzaban, Anousheh Gholami, John S. Baras, Bruce Golden
Date: 2023-08-31 00:00:00

The split delivery vehicle routing problem (SDVRP) is a relaxed variant of the capacitated vehicle routing problem (CVRP) where the restriction that each customer is visited precisely once is removed. Compared with CVRP, the SDVRP allows a reduction in the cost of the routes traveled by vehicles. The exact methods to solve the SDVRP are computationally expensive. Moreover, the complexity and difficult implementation of the state-of-the-art heuristic approaches hinder their application in real-life scenarios of the SDVRP. In this paper, we propose an easily understandable and effective approach to solve the SDVPR based on an a priori adaptive splitting algorithm (PASA). The idea of a priori split strategy was first introduced in Chen et al. (2017). In this approach, the demand of the customers is split into smaller values using a fixed splitting rule in advance. Consequently, the original SDVRP instance is converted to a CVRP instance which is solved using an existing CVRP solver. While the proposed a priori splitting rule in Chen et al. (2017) is fixed for all customers regardless of their demand and location, we suggest an adaptive splitting rule that takes into account the distance of the customers to the depot and their demand values. Our experiments show that PASA can generate solutions comparable to the state-of-the-art but much faster. Furthermore, our algorithm outperforms the fixed a priori splitting rule proposed by Chen et al. (2017).
Identifiable specializations for ODE models
Authors: Alexey Ovchinnikov, Anand Pillay, Gleb Pogudin, Thomas Scanlon
Date: 2023-08-30 00:00:00
Tasks: parameter estimation

The parameter identifiability problem for a dynamical system is to determine whether the parameters of the system can be found from data for the outputs of the system. Verifying whether the parameters are identifiable is a necessary first step before a meaningful parameter estimation can take place. Non-identifiability occurs in practical models. To reparametrize a model to achieve identifiability is a challenge. The existing approaches have been shown to be useful for many important examples. However, these approaches are either limited to linear models and scaling parametrizations or are not guaranteed to find a reparametrization even if it exists. In the present paper, we prove that there always exists a locally identifiable model with the same input-output behaviour as the original one obtained from a given one by a partial specialization of the parameters. As an extra feature of our approach, the resulting (at least) locally identifiable reparameterization has the same shape: the monomials in the new state variables in the new model are formed in the same way as in the original model. Furthermore, we give a sufficient observability condition for the existence of a state space transformation from the original model to the new one. Our proof is constructive and can be translated to an algorithm, which we illustrate by several examples.
Prediction of Diblock Copolymer Morphology via Machine Learning
Authors: Hyun Park, Boyuan Yu, Juhae Park, Ge Sun, Emad Tajkhorshid, Juan J. de Pablo, Ludwig Schneider
Date: 2023-08-31 00:00:00
Tasks: Data Augmentation, Explainable artificial intelligence, Prediction

A machine learning approach is presented to accelerate the computation of block polymer morphology evolution for large domains over long timescales. The strategy exploits the separation of characteristic times between coarse-grained particle evolution on the monomer scale and slow morphological evolution over mesoscopic scales. In contrast to empirical continuum models, the proposed approach learns stochastically driven defect annihilation processes directly from particle-based simulations. A UNet architecture that respects different boundary conditions is adopted, thereby allowing periodic and fixed substrate boundary conditions of arbitrary shape. Physical concepts are also introduced via the loss function and symmetries are incorporated via data augmentation. The model is validated using three different use cases. Explainable artificial intelligence methods are applied to visualize the morphology evolution over time. This approach enables the generation of large system sizes and long trajectories to investigate defect densities and their evolution under different types of confinement. As an application, we demonstrate the importance of accessing late-stage morphologies for understanding particle diffusion inside a single block. This work has implications for directed self-assembly and materials design in micro-electronics, battery materials, and membranes.
Experimental validation of an explicit flatness-based MPC design for quadcopter position tracking
Authors: Huu-Thinh Do, Ionela Prodan
Date: 2023-08-30 00:00:00
Tasks: Model Predictive Control, Position

Due to the nonlinearities and operational constraints typical to quadcopter missions, Model Predictive Control (MPC) encounters the major challenge of high computational power necessary for the online implementation. This problem may prove impractical, especially for a hardware-limited or small-scale setup. By removing the need for online solvers while keeping the constraint satisfaction and optimality, Explicit MPC (ExMPC) stands out as a strong candidate for this application. Yet, the formulation was usually hindered by the two main problems: nonlinearity and dimensionality. In this paper, we propose an ExMPC solution for the quadcopter position stabilization by analyzing its description (dynamics and constraints) in the flat output space. With the former issue, the system is exactly linearized into a concatenation of three double integrators at a price of cumbersome constraints in the new coordinates. For the latter, with a suitable characterization of these constraints, the stabilizing ExMPC can be computed for each double integrator separately. The controller is then validated via simulations and experimental tests. The proposed scheme achieves similar performance and guarantees to the state-of-the-art solution but with notably less computational effort, allowing scalability in a centralized manner
Event-Triggered Polynomial Control for Trajectory Tracking by Unicycle Robots
Authors: Harini V, Anusree Rajan, Bharadwaj Amrutur, Pavankumar Tallapragada
Date: 2023-08-30 00:00:00

This paper proposes an event-triggered polynomial control method for trajectory tracking by unicycle robots. In this method, each control input between two consecutive events is a polynomial and its coefficients are chosen to minimize the error in approximating a continuous-time control signal. We design an event-triggering rule that guarantees uniform ultimate boundedness of the tracking error and non-Zeno behavior of inter-event times. We illustrate our results through a suite of numerical simulations and experiments, which indicate that the number of events generated by the proposed controller is significantly less compared to that by a time-triggered controller or a event-triggered controller based on zero-order hold while guaranteeing similar tracking performance.
Round-Trip Energy Efficiency and Energy-Efficiency Fade Estimation for Battery Passport
Authors: Camiel Beckers, Erik Hoedemaekers, Arda Dagkilic, Henk Jan Bergveld
Date: 2023-08-30 00:00:00

The battery passport is proposed as a method to make the use and remaining value of batteries more transparent. The future EU Battery Directive requests this passport to contain the round-trip energy efficiency and its fade. In this paper, an algorithm is presented and demonstrated that estimates the round-trip energy efficiency of a battery pack. The algorithm identifies round trips based on battery current and SoC and characterizes these round trips based on certain conditions. 2D efficiency maps are created as a function of the conditions `temperature' and `RMS C-rate'. The maps are parameterized using multiple linear regression, which allows comparison of the efficiency under the same conditions. Analyzing data from three battery-electric buses over a period of 3.5 years reveals an efficiency fade of up to 0.86 percent point.
Majorization-Minimization for sparse SVMs
Authors: Alessandro Benfenati, Emilie Chouzenoux, Giorgia Franchini, Salla Latva-Aijo, Dominik Narnhofer, Jean-Christophe Pesquet, Sebastian J. Scott, Mahsa Yousefi
Date: 2023-08-31 00:00:00
Tasks: Binary Classification

Several decades ago, Support Vector Machines (SVMs) were introduced for performing binary classification tasks, under a supervised framework. Nowadays, they often outperform other supervised methods and remain one of the most popular approaches in the machine learning arena. In this work, we investigate the training of SVMs through a smooth sparse-promoting-regularized squared hinge loss minimization. This choice paves the way to the application of quick training methods built on majorization-minimization approaches, benefiting from the Lipschitz differentiabililty of the loss function. Moreover, the proposed approach allows us to handle sparsity-preserving regularizers promoting the selection of the most significant features, so enhancing the performance. Numerical tests and comparisons conducted on three different datasets demonstrate the good performance of the proposed methodology in terms of qualitative metrics (accuracy, precision, recall, and F 1 score) as well as computational cost.