pwc-archive/papers-with-abstracts huggingface dataset:
The Linear Representation Hypothesis and the Geometry of Large Language Models
Authors: Kiho Park, Yo Joong Choe, Victor Veitch
Date: 2023-11-07 00:00:00
Tasks: counterfactual, Sentence

Informally, the 'linear representation hypothesis' is the idea that high-level concepts are represented linearly as directions in some representation space. In this paper, we address two closely related questions: What does "linear representation" actually mean? And, how do we make sense of geometric notions (e.g., cosine similarity or projection) in the representation space? To answer these, we use the language of counterfactuals to give two formalizations of "linear representation", one in the output (word) representation space, and one in the input (sentence) space. We then prove these connect to linear probing and model steering, respectively. To make sense of geometric notions, we use the formalization to identify a particular (non-Euclidean) inner product that respects language structure in a sense we make precise. Using this causal inner product, we show how to unify all notions of linear representation. In particular, this allows the construction of probes and steering vectors using counterfactual pairs. Experiments with LLaMA-2 demonstrate the existence of linear representations of concepts, the connection to interpretation and control, and the fundamental role of the choice of inner product.
3DifFusionDet: Diffusion Model for 3D Object Detection with Robust LiDAR-Camera Fusion
Authors: Xinhao Xiang, Simon Dräger, Jiawei Zhang
Date: 2023-11-07 00:00:00
Tasks: 3D Object Detection, Denoising, Object, object-detection, Object Detection

Good 3D object detection performance from LiDAR-Camera sensors demands seamless feature alignment and fusion strategies. We propose the 3DifFusionDet framework in this paper, which structures 3D object detection as a denoising diffusion process from noisy 3D boxes to target boxes. In this framework, ground truth boxes diffuse in a random distribution for training, and the model learns to reverse the noising process. During inference, the model gradually refines a set of boxes that were generated at random to the outcomes. Under the feature align strategy, the progressive refinement method could make a significant contribution to robust LiDAR-Camera fusion. The iterative refinement process could also demonstrate great adaptability by applying the framework to various detecting circumstances where varying levels of accuracy and speed are required. Extensive experiments on KITTI, a benchmark for real-world traffic object identification, revealed that 3DifFusionDet is able to perform favorably in comparison to earlier, well-respected detectors.
ViDa: Visualizing DNA hybridization trajectories with biophysics-informed deep graph embeddings
Authors: Chenwei Zhang, Jordan Lovrod, Boyan Beronov, Khanh Dao Duc, Anne Condon
Date: 2023-11-06 00:00:00
Tasks: Dimensionality Reduction

Visualization tools can help synthetic biologists and molecular programmers understand the complex reactive pathways of nucleic acid reactions, which can be designed for many potential applications and can be modelled using a continuous-time Markov chain (CTMC). Here we present ViDa, a new visualization approach for DNA reaction trajectories that uses a 2D embedding of the secondary structure state space underlying the CTMC model. To this end, we integrate a scattering transform of the secondary structure adjacency, a variational autoencoder, and a nonlinear dimensionality reduction method. We augment the training loss with domain-specific supervised terms that capture both thermodynamic and kinetic features. We assess ViDa on two well-studied DNA hybridization reactions. Our results demonstrate that the domain-specific features lead to significant quality improvements over the state-of-the-art in DNA state space visualization, successfully separating different folding pathways and thus providing useful insights into dominant reaction mechanisms.
A Comprehensive Study on Model Initialization Techniques Ensuring Efficient Federated Learning
Authors: Ishmeet Kaur, Adwaita Janardhan Jadhav
Date: 2023-10-31 00:00:00
Tasks: Federated Learning, Privacy Preserving

Advancement in the field of machine learning is unavoidable, but something of major concern is preserving the privacy of the users whose data is being used for training these machine learning algorithms. Federated learning(FL) has emerged as a promising paradigm for training machine learning models in a distributed and privacy-preserving manner which enables one to collaborate and train a global model without sharing local data. But starting this learning process on each device in the right way, called ``model initialization" is critical. The choice of initialization methods used for models plays a crucial role in the performance, convergence speed, communication efficiency, privacy guarantees of federated learning systems, etc. In this survey, we dive deeper into a comprehensive study of various ways of model initialization techniques in FL.Unlike other studies, our research meticulously compares, categorizes, and delineates the merits and demerits of each technique, examining their applicability across diverse FL scenarios. We highlight how factors like client variability, data non-IIDness, model caliber, security considerations, and network restrictions influence FL model outcomes and propose how strategic initialization can address and potentially rectify many such challenges. The motivation behind this survey is to highlight that the right start can help overcome challenges like varying data quality, security issues, and network problems. Our insights provide a foundational base for experts looking to fully utilize FL, also while understanding the complexities of model initialization.
Some coordination problems are harder than others
Authors: Argyrios Deligkas, Eduard Eiben, Gregory Gutin, Philip R. Neary, Anders Yeo
Date: 2023-11-06 00:00:00

In order to coordinate players in a game must first identify a target pattern of behaviour. In this paper we investigate the difficulty of identifying prominent outcomes in two kinds of binary action coordination problems in social networks: pure coordination games and anti-coordination games. For both environments, we determine the computational complexity of finding a strategy profile that (i) maximises welfare, (ii) maximises welfare subject to being an equilibrium, and (iii) maximises potential. We show that the complexity of these objectives can vary with the type of coordination problem. Objectives (i) and (iii) are tractable problems in pure coordination games, but for anti-coordination games are NP-hard. Objective (ii), finding the best Nash equilibrium, is NP-hard for both. Our results support the idea that environments in which actions are strategic complements (e.g., technology adoption) facilitate successful coordination more readily than those in which actions are strategic substitutes (e.g., public good provision).
Multiplayer Homicidal Chauffeur Reach-Avoid Games: A Pursuit Enclosure Function Approach
Authors: Rui Yan, Xiaoming Duan, Rui Zou, Xin He, Zongying Shi, Francesco Bullo
Date: 2023-11-04 00:00:00
Tasks: ERP

This paper presents a multiplayer Homicidal Chauffeur reach-avoid differential game, which involves Dubins-car pursuers and simple-motion evaders. The goal of the pursuers is to cooperatively protect a planar convex region from the evaders, who strive to reach the region. We propose a cooperative strategy for the pursuers based on subgames for multiple pursuers against one evader and optimal task allocation. We introduce pursuit enclosure functions (PEFs) and propose a new enclosure region pursuit (ERP) winning approach that supports forward analysis for the strategy synthesis in the subgames. We show that if a pursuit coalition is able to defend the region against an evader under the ERP winning, then no more than two pursuers in the coalition are necessarily needed. We also propose a steer-to-ERP approach to certify the ERP winning and synthesize the ERP winning strategy. To implement the strategy, we introduce a positional PEF and provide the necessary parameters, states, and strategies that ensure the ERP winning for both one pursuer and two pursuers against one evader. Additionally, we formulate a binary integer program using the subgame outcomes to maximize the captured evaders in the ERP winning for the pursuit task allocation. Finally, we propose a multiplayer receding-horizon strategy where the ERP winnings are checked in each horizon, the task is allocated, and the strategies of the pursuers are determined. Numerical examples are provided to illustrate the results.
Solving MaxSAT with Matrix Multiplication
Authors: David Warde-Farley, Vinod Nair, Yujia Li, Ivan Lobov, Felix Gimeno, Simon Osindero
Date: 2023-11-01 00:00:00
Tasks: CPU

We propose an incomplete algorithm for Maximum Satisfiability (MaxSAT) specifically designed to run on neural network accelerators such as GPUs and TPUs. Given a MaxSAT problem instance in conjunctive normal form, our procedure constructs a Restricted Boltzmann Machine (RBM) with an equilibrium distribution wherein the probability of a Boolean assignment is exponential in the number of clauses it satisfies. Block Gibbs sampling is used to stochastically search the space of assignments with parallel Markov chains. Since matrix multiplication is the main computational primitive for block Gibbs sampling in an RBM, our approach leads to an elegantly simple algorithm (40 lines of JAX) well-suited for neural network accelerators. Theoretical results about RBMs guarantee that the required number of visible and hidden units of the RBM scale only linearly with the number of variables and constant-sized clauses in the MaxSAT instance, ensuring that the computational cost of a Gibbs step scales reasonably with the instance size. Search throughput can be increased by batching parallel chains within a single accelerator as well as by distributing them across multiple accelerators. As a further enhancement, a heuristic based on unit propagation running on CPU is periodically applied to the sampled assignments. Our approach, which we term RbmSAT, is a new design point in the algorithm-hardware co-design space for MaxSAT. We present timed results on a subset of problem instances from the annual MaxSAT Evaluation's Incomplete Unweighted Track for the years 2018 to 2021. When allotted the same running time and CPU compute budget (but no TPUs), RbmSAT outperforms other participating solvers on problems drawn from three out of the four years' competitions. Given the same running time on a TPU cluster for which RbmSAT is uniquely designed, it outperforms all solvers on problems drawn from all four years.
Safe Collective Control under Noisy Inputs and Competing Constraints via Non-Smooth Barrier Functions
Authors: Clinton Enwerem, John S. Baras
Date: 2023-11-06 00:00:00
Tasks: Model Predictive Control, Stochastic Optimization

We consider the problem of safely coordinating ensembles of identical autonomous agents to conduct complex missions with conflicting safety requirements and under noisy control inputs. Using non-smooth control barrier functions (CBFs) and stochastic model-predictive control as springboards, and by adopting an extrinsic approach where the ensemble is treated as a unified dynamic entity, we devise a method to synthesize safety-aware control inputs for uncertain collectives. Drawing upon stochastic CBF theory and recent developments in Boolean CBF composition, our method proceeds by smoothing a Boolean-composed CBF and solving a stochastic optimization problem where each agent's forcing term is restricted to the affine subspace of control inputs certified by the combined CBF. For the smoothing step, we employ a polynomial approximation scheme, providing evidence for its advantage in generating more conservative yet sufficiently-filtered control inputs than the smoother but more aggressive equivalents produced from an approximation technique based on the log-sum-exp function. To further demonstrate the utility of the proposed method, we present an upper bound for the expected CBF approximation error, along with results from simulations of a single-integrator collective under velocity perturbations. Lastly, we compare these results with those obtained using a naive state-feedback controller lacking safety filters.
Design Of Rubble Analyzer Probe Using ML For Earthquake
Authors: Abhishek Sebastian, R Pragna, K Vishal Vythianathan, Dasaraju Sohan Sai, U Shiva Sri Hari Al, R Anirudh, Apurv Choudhary
Date: 2023-10-24 00:00:00

The earthquake rubble analyzer uses machine learning to detect human presence via ambient sounds, achieving 97.45% accuracy. It also provides real-time environmental data, aiding in assessing survival prospects for trapped individuals, crucial for post-earthquake rescue efforts
Defending a Static Target Point with a Slow Defender
Authors: Goutam Das, Michael Dorothy, Zachary I. Bell, Daigo Shishika
Date: 2023-11-06 00:00:00

This paper studies a target-defense game played between a slow defender and a fast attacker. The attacker wins the game if it reaches the target while avoiding the defender's capture disk. The defender wins the game by preventing the attacker from reaching the target, which includes reaching the target and containing it in the capture disk. Depending on the initial condition, the attacker must circumnavigate the defender's capture disk, resulting in a constrained trajectory. This condition produces three phases of the game, which we analyze to solve for the game of kind. We provide the barrier surface that divides the state space into attacker-win and defender win regions, and present the corresponding strategies that guarantee win for each region. Numerical experiments demonstrate the theoretical results as well as the efficacy of the proposed strategies.
Estimating Primary Substation Boundaries and the Value of Mapping the Electrical Network Infrastructure of Great Britain
Authors: Joseph Day, I. A. Grant Wilson, Daniel L. Donaldson, Edward Barbour, Bruno Cárdenas, Christopher R. Jones, Andrew J. Urquhart, Seamus D. Garvey
Date: 2023-11-06 00:00:00
Tasks: Decision Making

Localised data aggregation in many countries including Great Britain (GB) is typically done to a geographical level with polygon boundaries that have a robust and trusted governance system in place. At a minimum this will mean there is confidence in a process to create a set of polygons that have unique identifiers coupled to geographical areas, and the ability to have these updated through a defined code of practice. Examples found across many countries are in the delivery of post, such as postcodes and zip codes, and of the definition of census areas and municipal boundaries. The confidence in these boundaries allows different data to be aggregated by third parties, which itself provides greater levels of data over comparable geographical areas to enhance wider analysis and decision making. Here we combine publicly available datasets published from the six regional electricity Distribution Network Operators of GB to produce a new geospatial dataset with 4436 unique polygons defining the areas served by electrical primary substations. An example is also presented of the use of these polygons to link postcode level open government datasets on domestic energy consumption (2015-2020) from the Department of Energy Security and Net Zero (DESNZ). This results in another dataset with energy statistics aggregated to the geographical areas served by each primary substation across Great Britain. Therefore, we believe there is a compelling argument for countries to set up processes to create and update polygons that have a meaningful relationship to energy systems. This would allow more accurate energy systems analysis to be performed, ultimately leading to an accelerated or potentially lower cost transition to a net-zero world.
Faster Run-to-Run Feedforward Control of Electromechanical Switching Devices: a Sensitivity-Based Approach
Authors: Edgar Ramirez-Laboreo, Eduardo Moya-Lasheras, Eloy Serrano-Seco
Date: 2023-11-06 00:00:00
Tasks: Sensitivity

Electromechanical switching devices, such as solenoid valves, contactors, and relays, suffer from undesirable phenomena like clicking, mechanical wear, and contact bounce. Despite that, they are still widely used in industry due to their various economic and technical advantages. This has encouraged the development of controllers aimed at reducing the collisions that occur at the end of the switching operations. One of the most successful approaches has been the use of iterative techniques. However, these algorithms typically require a large number of operations to converge, which is definitely a clear drawback. This paper presents a strategy to improve the convergence rate of such controllers. Our proposal, which is based on the sensitivity of the control law with respect to the parameters, assumes that the performance of the system is more heavily affected by some parameters than others. Thus, by avoiding movements in the directions that have less impact, the search algorithm is expected to drive the system to near-optimal behaviors using fewer operations. Results obtained by simulation show significant improvement in the convergence rate of a state-of-the-art run-to-run feedforward controller, which demonstrates the high potential of the proposal.
Gaps in Representations of Hydropower Generation in Steady-State and Dynamic Models
Authors: Bhaskar Mitra, Sohom Datta, Slaven Kincic, Nader Samaan, Abhishek Somani
Date: 2023-11-06 00:00:00

In the evolving power system, where new renewable resources continually displace conventional generation, conventional hydropower resources can be an important asset that helps to maintain reliability and flexibility. Varying climatic patterns do affect the operational pattern of hydropower. This would potentially play a vital role in meeting and delivering energy and meeting climate policy needs. Hydropower is one of the oldest forms of renewable energy resources, however, its dependency on water availability and other constraints are not well represented in power system steady state and dynamic models. This leads to multiple gaps in operations planning especially due to high intermittent renewable generation. Operating constraints and lack of high-quality data often become a barrier to hydropower modeling which leads to inconsistencies in reliability and operational planning studies resulting in unintentional blackouts or unforeseen situations. This paper identifies some of the gaps in hydro-based generation representation in steady-state and dynamic models and provides recommendations for their mitigation.
Congestion-aware Ride-pooling in Mixed Traffic for Autonomous Mobility-on-Demand Systems
Authors: Fabio Paparella, Leonardo Pedroso, Theo Hofman, Mauro Salazar
Date: 2023-11-06 00:00:00

This paper presents a modeling and optimization framework to study congestion-aware ride-pooling Autonomous Mobility-on-Demand (AMoD) systems, whereby self-driving robotaxis are providing on-demand mobility, and users headed in the same direction share the same vehicle for part of their journey. Specifically, taking a mesoscopic time-invariant perspective and on the assumption of a large number of travel requests, we first cast the joint ride-pooling assignment and routing problem as a quadratic program that does not scale with the number of demands and can be solved with off-the-shelf convex solvers. Second, we compare the proposed approach with a significantly simpler decoupled formulation, whereby only the routing is performed in a congestion-aware fashion, whilst the ride-pooling assignment part is congestion-unaware. A case study of Sioux Falls reveals that such a simplification does not significantly alter the solution and that the decisive factor is indeed the congestion-aware routing. Finally, we solve the latter problem accounting for the presence of user-centered private vehicle users in a case study of Manhattan, NYC, characterizing the performance of the car-network as a function of AMoD penetration rate and percentage of pooled rides within it. Our results show that AMoD can significantly reduce congestion and travel times, but only if at least 40% of the users are willing to be pooled together. Otherwise, for higher AMoD penetration rates and low percentage of pooled rides, the effect of the additional rebalancing empty-vehicle trips can be even more detrimental than the benefits stemming from a centralized routing, worsening congestion and leading to an up to 15% higher average travel time.
Modelling and Optimization Based Control for Demand Response in Active Distribution Networks
Authors: Arsam Aryandoust
Date: 2023-11-06 00:00:00
Tasks: Management

We explore how Demand Response (DR) can effectively provide electricity system services such as for the management of bi-directional power flows and the control of voltage deviations in active distribution networks, without compromising consumer comfort or adversely affecting grid operation in the transmission network. By translating intricate power system physics into straightforward control objectives, we design DR control algorithms that can operate within realistic computational time frames at scale. We conduct simulation-based experiments and find that minimizing the Euclidean-Norm of the total residual load at transformer sub-stations is an effective objective for harnessing DR for the dispatch of renewable electricity grids. We show that this control objective can be efficiently pursued in sequential order, without optimal power flow calculations or information about the topology of a grid. Additionally, we find that pursuing this objective reduces the sum of peak power flows along all lines in active distribution networks, and can therefore enable both lower transmission losses and lower voltage deviations.
A contract negotiation scheme for safety verification of interconnected systems
Authors: Xiao Tan, Antonis Papachristodoulou, Dimos V. Dimarogonas
Date: 2023-11-06 00:00:00

This paper proposes a (control) barrier function synthesis and safety verification scheme for interconnected nonlinear systems based on assume-guarantee contracts (AGC) and sum-of-squares (SOS) techniques. It is well-known that the SOS approach does not scale well for barrier function synthesis for high-dimensional systems. In this paper, we show that compositional methods like AGC can mitigate this problem. We formulate the synthesis problem into a set of small-size problems, which constructs local contracts for subsystems, and propose a negotiation scheme among the subsystems at the contract level. The proposed scheme is then implemented numerically on two examples: vehicle platooning and room temperature regulation.