Miracle-dz/UniFAR-FTUs huggingface dataset:
Comparing Differentiable Logics for Learning Systems: A Research Preview

Extensive research on formal verification of machine learning (ML) systems indicates that learning from data alone often fails to capture underlying background knowledge. A variety of verifiers have been developed to ensure that a machine-learnt model satisfies correctness and safety properties, however, these verifiers typically assume a trained network with fixed weights. ML-enabled autonomous systems are required to not only detect incorrect predictions, but should also possess the ability to self-correct, continuously improving and adapting. A promising approach for creating ML models that inherently satisfy constraints is to encode background knowledge as logical constraints that guide the learning process via so-called differentiable logics. In this research preview, we compare and evaluate various logics from the literature in weakly-supervised contexts, presenting our findings and highlighting open problems for future work. Our experimental results are broadly consistent with results reported previously in literature; however, learning with differentiable logics introduces a new hyperparameter that is difficult to tune and has significant influence on the effectiveness of the logics.
Towards Formal Fault Injection for Safety Assessment of Automated Systems

Reasoning about safety, security, and other dependability attributes of autonomous systems is a challenge that needs to be addressed before the adoption of such systems in day-to-day life. Formal methods is a class of methods that mathematically reason about a system's behavior. Thus, a correctness proof is sufficient to conclude the system's dependability. However, these methods are usually applied to abstract models of the system, which might not fully represent the actual system. Fault injection, on the other hand, is a testing method to evaluate the dependability of systems. However, the amount of testing required to evaluate the system is rather large and often a problem. This vision paper introduces formal fault injection, a fusion of these two techniques throughout the development lifecycle to enhance the dependability of autonomous systems. We advocate for a more cohesive approach by identifying five areas of mutual support between formal methods and fault injection. By forging stronger ties between the two fields, we pave the way for developing safe and dependable autonomous systems. This paper delves into the integration's potential and outlines future research avenues, addressing open challenges along the way.
SurgPLAN: Surgical Phase Localization Network for Phase Recognition

Surgical phase recognition is crucial to providing surgery understanding in smart operating rooms. Despite great progress in automatic surgical phase recognition, most existing methods are still restricted by two problems. First, these methods cannot capture discriminative visual features for each frame and motion information with simple 2D networks. Second, the frame-by-frame recognition paradigm degrades the performance due to unstable predictions within each phase, termed as phase shaking. To address these two challenges, we propose a Surgical Phase LocAlization Network, named SurgPLAN, to facilitate a more accurate and stable surgical phase recognition with the principle of temporal detection. Specifically, we first devise a Pyramid SlowFast (PSF) architecture to serve as the visual backbone to capture multi-scale spatial and temporal features by two branches with different frame sampling rates. Moreover, we propose a Temporal Phase Localization (TPL) module to generate the phase prediction based on temporal region proposals, which ensures accurate and consistent predictions within each surgical phase. Extensive experiments confirm the significant advantages of our SurgPLAN over frame-by-frame approaches in terms of both accuracy and stability.
Examining bias perpetuation in academic search engines: an algorithm audit of Google and Semantic Scholar

Researchers rely on academic web search engines to find scientific sources, but search engine mechanisms may selectively present content that aligns with biases embedded in the queries. This study examines whether confirmation-biased queries prompted into Google Scholar and Semantic Scholar will yield skewed results. Six queries (topics across health and technology domains such as"vaccines"or"internet use") were analyzed for disparities in search results. We confirm that biased queries (targeting"benefits"or"risks") affect search results in line with the bias, with technology-related queries displaying more significant disparities. Overall, Semantic Scholar exhibited fewer disparities than Google Scholar. Topics rated as more polarizing did not consistently show more skewed results. Academic search results that perpetuate confirmation bias have strong implications for both researchers and citizens searching for evidence. More research is needed to explore how scientific inquiry and academic search engines interact.
Zenkai - Framework For Exploring Beyond Backpropagation

Zenkai is an open-source framework designed to give researchers more control and flexibility over building and training deep learning machines. It does this by dividing the deep learning machine into layers of semi-autonomous learning machines with their own target and learning algorithm. This is to allow researchers greater exploration such as the use of non-differentiable layers or learning algorithms beyond those based on error backpropagation. Backpropagation Rumelhart et al. [1986] has powered deep learning to become one of the most exciting fields of the 21st century. As a result, a large number of software tools have been developed to support efficient implementation and training of neural networks through the use of backpropa- gation. While these have been critical to the success of deep learning, building frameworks around backpropagation can make it challenging to implement solutions that do not adhere to it. Zenkai aims to make it easier to get around these limitations and help researchers more easily explore new frontiers in deep learning that do not strictly adhere to the backpropagation framework.
Non-injectivity of Bures--Wasserstein barycentres in infinite dimensions

We construct a counterexample to the injectivity conjecture of Masarotto et al (2018). Namely, we construct a class of examples of injective covariance operators on an infinite-dimensional separable Hilbert space for which the Bures--Wasserstein barycentre is highly non injective -- it has a kernel of infinite dimension.
Carbon Dioxide Emission Minimized Virtual Machine (VM) Placement in Cloud-Fog Network Architecture

Cloud computing has provided economies of scale, savings, and efficiency for both individual consumers and enterprises. Its key advantage is its ability to handle increasing amounts of data and provide functionality that gives users the ability to scale their computing resources, including processing, data storage, and networking capabilities. Virtual Machines (VM), enabled via virtualization technology, allow cloud service providers to deliver their services to users. This, however, results in increasing carbon dioxide emissions from increased energy use. This paper introduces a Mixed-Integer Linear Programming (MILP) model that investigates the VM placement, focusing on the British Telecom (BT) network topology, in a cloud-fog network architecture when renewable energy sources are introduced in the fog layer located near traffic-producing sources. VMs can be placed on nodes hosted on the core, metro, and access (fog) layers. We first investigate the effect of varying traffic on IP over WDM power consumption in the backbone network and the number of optical carrier signals to serve the traffic over a period of time. We later extend the model to consider the CO2-minimized optimal virtual machine placement given the sporadic traffic quantity, and the consideration of solar renewable energy sources placed in data centers located in the access (fog) layer throughout the day, imposed on the VM and the minimum workload requirement of the VM to maintain a service level agreement (SLA).
Spatial Bayesian Neural Networks

Statistical models for spatial processes play a central role in statistical analyses of spatial data. Yet, it is the simple, interpretable, and well understood models that are routinely employed even though, as is revealed through prior and posterior predictive checks, these can poorly characterise the spatial heterogeneity in the underlying process of interest. Here, we propose a new, flexible class of spatial-process models, which we refer to as spatial Bayesian neural networks (SBNNs). An SBNN leverages the representational capacity of a Bayesian neural network; it is tailored to a spatial setting by incorporating a spatial"embedding layer"into the network and, possibly, spatially-varying network parameters. An SBNN is calibrated by matching its finite-dimensional distribution at locations on a fine gridding of space to that of a target process of interest. That process could be easy to simulate from or we have many realisations from it. We propose several variants of SBNNs, most of which are able to match the finite-dimensional distribution of the target process at the selected grid better than conventional BNNs of similar complexity. We also show that a single SBNN can be used to represent a variety of spatial processes often used in practice, such as Gaussian processes and lognormal processes. We briefly discuss the tools that could be used to make inference with SBNNs, and we conclude with a discussion of their advantages and limitations.
Alternatives to the Scaled Dot Product for Attention in the Transformer Neural Network Architecture

The transformer neural network architecture uses a form of attention in which the dot product of query and key is divided by the square root of the key dimension before applying softmax. This scaling of the dot product is designed to avoid the absolute value of the dot products becoming so large that applying softmax leads to vanishing gradients. In this paper, we propose some alternative scalings, including dividing the dot product instead by the sum of the key lengths before applying softmax. We use simulated keys and queries to show that in many situations this appears to be more effective at avoiding regions where applying softmax leads to vanishing gradients.
Towards a data-driven model of hadronization using normalizing flows

We introduce a model of hadronization based on invertible neural networks that faithfully reproduces a simplified version of the Lund string model for meson hadronization. Additionally, we introduce a new training method for normalizing flows, termed MAGIC, that improves the agreement between simulated and experimental distributions of high-level (macroscopic) observables by adjusting single-emission (microscopic) dynamics. Our results constitute an important step toward realizing a machine-learning based model of hadronization that utilizes experimental data during training. Finally, we demonstrate how a Bayesian extension to this normalizing-flow architecture can be used to provide analysis of statistical and modeling uncertainties on the generated observable distributions.
SparseAuto: An Auto-Scheduler for Sparse Tensor Computations Using Recursive Loop Nest Restructuring

Automated code generation and performance optimizations for sparse tensor algebra are cardinal since they have become essential in many real-world applications like quantum computing, physics, chemistry, and machine learning. General sparse tensor algebra compilers are not always versatile enough to generate asymptotically optimal code for sparse tensor contractions. This paper shows how to optimize and generate asymptotically better schedules for complex tensor expressions using kernel fission and fusion. We present a generalized loop transformation to achieve loop nesting for minimized memory footprint and reduced asymptotic complexity. Furthermore, we present an auto-scheduler that uses a partially ordered set-based cost model that uses both time and auxiliary memory complexities in its pruning stages. In addition, we highlight the use of SMT solvers in sparse auto-schedulers to prune the Pareto frontier of schedules to the smallest number of possible schedules with user-defined constraints available at compile time. Finally, we show that our auto-scheduler can select asymptotically better schedules that use our compiler transformation to generate optimized code. Our results show that the auto-scheduler achieves orders of magnitude speedup compared to the TACO-generated code for several real-world tensor algebra computations on different real-world inputs.
Stability estimates for an inverse boundary value problem for biharmonic operators with first order perturbation from partial data

In this paper we study an inverse boundary value problem for the biharmonic operator with first order perturbation. Our geometric setting is that of a bounded simply connected domain in the Euclidean space of dimension three or higher. Assuming that the inaccessible portion of the boundary is flat, and we have knowledge of the Dirichlet-to-Neumann map on the complement, we prove logarithmic type stability estimates for both the first and the zeroth order perturbation of the biharmonic operator.
Limits on Optical Counterparts to the Repeating FRB 20180916B from High-speed Imaging with Gemini-N/'Alopeke

We report on contemporaneous optical observations at ~10 ms timescales from the fast radio burst (FRB) 20180916B of two repeat bursts (FRB 20201023, FRB 20220908) taken with the 'Alopeke camera on the Gemini North telescope. These repeats have radio fluences of 2.8 and 3.5 Jy ms, respectively, approximately in the lower 50th percentile for fluence from this repeating burst. The 'Alopeke data reveal no significant optical detections at the FRB position and we place 3-sigma upper limits to the optical fluences of<8.3e-3 and<7.7e-3 Jy ms after correcting for line-of-sight extinction. Together, these yield the most sensitive limits to the optical-to-radio fluence ratio of an FRB on these timescales with eta<3e-3 by roughly an order of magnitude. These measurements rule out progenitor models where FRB 20180916B has a similar fluence ratio to optical pulsars similar to the Crab pulsar or optical emission is produced as inverse Compton radiation in a pulsar magnetosphere or young supernova remnant. Our ongoing program with 'Alopeke on Gemini-N will continue to monitor repeating FRBs, including FRB 20180916B, to search for optical counterparts on ms timescales.
ML-Bench: Large Language Models Leverage Open-source Libraries for Machine Learning Tasks

Large language models have shown promising performance in code generation benchmarks. However, a considerable divide exists between these benchmark achievements and their practical applicability, primarily attributed to real-world programming's reliance on pre-existing libraries. Instead of evaluating LLMs to code from scratch, this work aims to propose a new evaluation setup where LLMs use open-source libraries to finish machine learning tasks. Therefore, we propose ML-Bench, an expansive benchmark developed to assess the effectiveness of LLMs in leveraging existing functions in open-source libraries. Consisting of 10044 samples spanning 130 tasks over 14 notable machine learning GitHub repositories. In this setting, given a specific machine learning task instruction and the accompanying README in a codebase, an LLM is tasked to generate code to accomplish the task. This necessitates the comprehension of long and language-code interleaved documents, as well as the understanding of complex cross-file code structures, introducing new challenges. Notably, while GPT-4 exhibits remarkable improvement over other LLMs, it manages to accomplish only 39.73\% of the tasks, leaving a huge space for improvement. We address these challenges by proposing ML-Agent, designed to effectively navigate the codebase, locate documentation, retrieve code, and generate executable code. Empirical results demonstrate that ML-Agent, built upon GPT-4, results in further improvements. Code, data, and models are available at \url{https://ml-bench.github. io/}.
Network Wide Evacuation Traffic Prediction in a Rapidly Intensifying Hurricane from Traffic Detectors and Facebook Movement Data: A Deep Learning Approach

Traffic prediction during hurricane evacuation is essential for optimizing the use of transportation infrastructures. It can reduce evacuation time by providing information on future congestion in advance. However, evacuation traffic prediction can be challenging as evacuation traffic patterns is significantly different than regular period traffic. A data-driven traffic prediction model is developed in this study by utilizing traffic detector and Facebook movement data during Hurricane Ian, a rapidly intensifying hurricane. We select 766 traffic detectors from Florida's 4 major interstates to collect traffic features. Additionally, we use Facebook movement data collected during Hurricane Ian's evacuation period. The deep-learning model is first trained on regular period (May-August 2022) data to understand regular traffic patterns and then Hurricane Ian's evacuation period data is used as test data. The model achieves 95% accuracy (RMSE = 356) during regular period, but it underperforms with 55% accuracy (RMSE = 1084) during the evacuation period. Then, a transfer learning approach is adopted where a pretrained model is used with additional evacuation related features to predict evacuation period traffic. After transfer learning, the model achieves 89% accuracy (RMSE = 514). Adding Facebook movement data further reduces model's RMSE value to 393 and increases accuracy to 93%. The proposed model is capable to forecast traffic up to 6-hours in advance. Evacuation traffic management officials can use the developed traffic prediction model to anticipate future traffic congestion in advance and take proactive measures to reduce delays during evacuation.
On the evolution of Betti curves in the Cosmic web

In this work, we study the evolution of Betti curves obtained by persistent-homological analysis of pointclouds formed by halos in different cosmological $N$-body simulations. We show that they can be approximated with a scaled log-normal distribution function with reasonable precision. Our analysis shows that the shapes and maximums of Betti curves exhibit dependence on the mass range of the selected subpopulation of halos, but at the same time, the resolution of a simulation does not play any significant role, provided that the mass distribution of simulated halos is complete down to a given mass scale. Besides, we study how Betti curves change with the evolution of the Universe, i.e., their dependence on redshift. Sampling subpopulations of halos within certain mass ranges up to redshift $z=2.5$ yields a surprisingly small difference between corresponding Betti curves. We propose that this may be an indicator of the existence of a new specific topological invariant in the structure of the Universe.
CONNECTIONS BETWEEN CLIMATE INDICES AND CASPIAN SEA SURFACE TEMPERATURE

The relationship between climate indices and the Caspian Sea Surface Temperature (SST) are investigated, making use of the satellite measured SST during 1985-2005. North Sea Caspian Pattern (NCP) and NINO 3.4 index are shown to have a strong influence on the SST among the studied indices, in the studied February to May period. High correlation of SST with the climate indices suggests prompt response of the surface oceanic variables to the atmospheric forcing. Besides the linear relationships, nonlinear relationships are also studied using the Generalized, Unbiased, Interaction Detection and Estimation (GUIDE) algorithm, the results confirming similar predictions of Caspian Sea SST.
The Metallicities of Five Small High-velocity Clouds*

High-velocity clouds (HVCs) are multiphase gas structures whose velocities (∣v LSR∣ ≥ 100 km s−1) are too high to be explained by Galactic disk rotation. While large HVCs are well characterized, compact and small HVCs (with H i angular sizes of a few degrees) are poorly understood. Possible origins for such small clouds include Milky Way (MW) halo gas or fragments of the Magellanic System, but neither their origin nor their connection to the MW halo has been confirmed. We use new Hubble Space Telescope/Cosmic Origins Spectrograph UV spectra and Green Bank Telescope H i spectra to measure the metallicities of five small HVCs in the southern Galactic sky projected near the Magellanic System. We build a set of distance-dependent Cloudy photoionization models for each cloud and calculate their ionization-corrected metallicities. All five small HVCs have oxygen metallicities ≤0.17 Z ⊙, indicating they do not originate in the disk of the MW. Two of the five have metallicities of 0.16–0.17 Z ⊙, similar to the Magellanic Stream, suggesting these clouds are fragments of the Magellanic System. The remaining three clouds have much lower metallicities of 0.02–0.04 Z ⊙. While the origin of these low-metallicity clouds is unclear, they could be gaseous minihalos or gas stripped from dwarf galaxies by ram pressure or tidal interactions. These results suggest that small HVCs do not all reside in the inner MW halo or the Magellanic System, but instead can trace more distant structures.
Demonstrating the Q-Pix front-end using discrete OpAmp and CMOS transistors

Using Commercial Off-The-Shelf (COTS) Operational Amplifiers (OpAmps) and Complementary Metal-Oxide Semiconductor (CMOS) transistors, we present a demonstration of the Q-Pix front-end architecture, a novel readout solution for kiloton-scale Liquid Argon Time Projection Chamber (LArTPC) detectors. The Q-Pix scheme employs a Charge-Integrate/Reset process based on the Least Action principle, enabling pixel-scale self-triggering charge collection and processing, minimizing energy consumption, and maximizing data compression. We examine the architecture's sensitivity, linearity, noise, and other features at the circuit board level and draw comparisons to SPICE simulations. Furthermore, we highlight the resemblance between the Q-Pix front-end and Sigma-Delta modulator, emphasizing that digital data processing techniques for Sigma-Delta can be directly applied to Q-Pix, resulting in enhanced signal-to-noise performance. These insights will inform the development of Q-Pix front-end designs in integrated circuits (IC) and guide data collection and processing for future large-scale LArTPC detectors in neutrino physics and other high-energy physics experiments.
Software-Defined Virtual Synchronous Condenser

Synchronous condensers (SCs) play important roles in integrating wind energy into relatively weak power grids. However, the design of SCs usually depends on specific application requirements and may not be adaptive enough to the frequently-changing grid conditions caused by the transition from conventional to renewable power generation. This paper devises a software-defined virtual synchronous condenser (SDViSC) method to address the challenges. Our contributions are fourfold: 1) design of a virtual synchronous condenser (ViSC) to enable full converter wind turbines to provide built-in SC functionalities; 2) engineering SDViSCs to transfer hardware-based ViSC controllers into software services, where a Tustin transformation-based software-defined control algorithm guarantees accurate tracking of fast dynamics under limited communication bandwidth; 3) a software-defined networking-enhanced SDViSC communication scheme to allow enhanced communication reliability and reduced communication bandwidth occupation; and 4) Prototype of SDViSC on our real-time, cyber-in-the-loop digital twin of large-wind-farm in an RTDS environment. Extensive test results validate the excellent performance of SDViSC to support reliable and resilient operations of wind farms under various physical and cyber conditions.