Miracle-dz/UniFAR-FTUs huggingface dataset:
Open-ended VQA benchmarking of Vision-Language models by exploiting Classification datasets and their semantic hierarchy

The evaluation of text-generative vision-language models is a challenging yet crucial endeavor. By addressing the limitations of existing Visual Question Answering (VQA) benchmarks and proposing innovative evaluation methodologies, our research seeks to advance our understanding of these models' capabilities. We propose a novel VQA benchmark based on well-known visual classification datasets which allows a granular evaluation of text-generative vision-language models and their comparison with discriminative vision-language models. To improve the assessment of coarse answers on fine-grained classification tasks, we suggest using the semantic hierarchy of the label space to ask automatically generated follow-up questions about the ground-truth category. Finally, we compare traditional NLP and LLM-based metrics for the problem of evaluating model predictions given ground-truth answers. We perform a human evaluation study upon which we base our decision on the final metric. We apply our benchmark to a suite of vision-language models and show a detailed comparison of their abilities on object, action, and attribute classification. Our contributions aim to lay the foundation for more precise and meaningful assessments, facilitating targeted progress in the exciting field of vision-language modeling.
Shortest-path percolation on complex networks

We propose a bond-percolation model intended to describe the consumption, and eventual exhaustion, of resources in transport networks. Edges forming minimum-length paths connecting demanded origin-destination nodes are removed if below a certain budget. As pairs of nodes are demanded and edges are removed, the macroscopic connected component of the graph disappears, i.e., the graph undergoes a percolation transition. Here, we study such a shortest-path-percolation transition in homogeneous random graphs where pairs of demanded origin-destination nodes are randomly generated, and fully characterize it by means of finite-size scaling analysis. If budget is finite, the transition is identical to the one of ordinary percolation, where a single giant cluster shrinks as edges are removed from the graph; for infinite budget, the transition becomes more abrupt than the one of ordinary percolation, being characterized by the sudden fragmentation of the giant connected component into a multitude of clusters of similar size.
Decoupling Learning and Decision-Making: Breaking the O(√T) Barrier in Online Resource Allocation with First-Order Methods

Online linear programming plays an important role in both revenue management and resource allocation, and recent research has focused on developing efficient first-order online learning algorithms. Despite the empirical success of first-order methods, they typically achieve a regret no better than $\mathcal{O}(\sqrt{T})$, which is suboptimal compared to the $\mathcal{O}(\log T)$ bound guaranteed by the state-of-the-art linear programming (LP)-based online algorithms. This paper establishes several important facts about online linear programming, which unveils the challenge for first-order-method-based online algorithms to achieve beyond $\mathcal{O}(\sqrt{T})$ regret. To address the challenge, we introduce a new algorithmic framework that decouples learning from decision-making. More importantly, for the first time, we show that first-order methods can attain regret $\mathcal{O}(T^{1/3})$ with this new framework. Lastly, we conduct numerical experiments to validate our theoretical findings.
X-LoRA: Mixture of Low-Rank Adapter Experts, a Flexible Framework for Large Language Models with Applications in Protein Mechanics and Design

We report a mixture of expert strategy to create fine-tuned large language models using a deep layer-wise token-level approach based on low-rank adaptation (LoRA). Starting with a set of pre-trained LoRA adapters, we propose a gating strategy that uses the hidden states to dynamically mix adapted layers, allowing the resulting X-LoRA model to draw upon different capabilities and create never-before-used deep layer-wise combinations of adaptations are established to solve specific tasks. The design is inspired by the biological principles of universality and diversity, where neural network building blocks are reused in different hierarchical manifestations. Hence, the X-LoRA model can be easily implemented for any existing large language model (LLM) without a need for modifications of the underlying structure. We develop a tailored X-LoRA model that offers scientific capabilities including forward/inverse analysis tasks and enhanced reasoning capability, focused on biomaterial analysis, protein mechanics and design. The impact of this work include access to readily expandable, adaptable and changeable models with strong domain knowledge and the capability to integrate across areas of knowledge. With the X-LoRA model featuring experts in biology, mathematics, reasoning, bio-inspired materials, mechanics and materials, chemistry, and protein mechanics we conduct a series of physics-focused case studies. We examine knowledge recall, protein mechanics forward/inverse tasks, protein design, and adversarial agentic modeling including ontological knowledge graphs. The model is capable not only of making quantitative predictions of nanomechanical properties of proteins, but also reasons over the results and correctly predicts likely mechanisms that explain distinct molecular behaviors.
Nesting Particle Filters for Experimental Design in Dynamical Systems

In this paper, we propose a novel approach to Bayesian experimental design for non-exchangeable data that formulates it as risk-sensitive policy optimization. We develop the Inside-Out SMC$^2$ algorithm, a nested sequential Monte Carlo technique to infer optimal designs, and embed it into a particle Markov chain Monte Carlo framework to perform gradient-based policy amortization. Our approach is distinct from other amortized experimental design techniques, as it does not rely on contrastive estimators. Numerical validation on a set of dynamical systems showcases the efficacy of our method in comparison to other state-of-the-art strategies.
Informativeness of Reward Functions in Reinforcement Learning

Reward functions are central in specifying the task we want a reinforcement learning agent to perform. Given a task and desired optimal behavior, we study the problem of designing informative reward functions so that the designed rewards speed up the agent's convergence. In particular, we consider expert-driven reward design settings where an expert or teacher seeks to provide informative and interpretable rewards to a learning agent. Existing works have considered several different reward design formulations; however, the key challenge is formulating a reward informativeness criterion that adapts w.r. t. the agent's current policy and can be optimized under specified structural constraints to obtain interpretable rewards. In this paper, we propose a novel reward informativeness criterion, a quantitative measure that captures how the agent's current policy will improve if it receives rewards from a specific reward function. We theoretically showcase the utility of the proposed informativeness criterion for adaptively designing rewards for an agent. Experimental results on two navigation tasks demonstrate the effectiveness of our adaptive reward informativeness criterion.
Optical Routing with Binary Optimisation and Quantum Annealing

A challenge for scalability of demand-responsive, elastic optical Dense Wavelength Division Multiplexing (DWDM) and Flexgrid networks is the computational complexity of allocating many optical routes on large networks. We demonstrate that demand satisfaction problems in communication networks can be formulated as quadratic unconstrained binary optimisation (QUBO) problems, and solved using a hybrid quantum annealer. Efficient encodings are developed which solve both unicast and multicast multicommodity-flow problems, while also adhering to individual requirements for maximum latency and resilience for each route. We present several QUBO formulations and analyse the qubit scaling. We demonstrate solutions using a hybrid solver, D-Wave Quantum Advantage QPU. Progress in generating optimal solutions with efficient use of computational resources will be beneficial to telecoms operators, enabling them to run dynamic optical network infrastructures which use resources efficiently, are resilient to local faults and cyber-attacks, and can be elastically responsive to demands.
2d (0,2) Gauge Theories from Branes: Recent Progress in Brane Brick Models

We discuss the realization of $2d$ $(0,2)$ gauge theories in terms of branes focusing on Brane Brick Models, which are T-dual to D1-branes probing toric Calabi-Yau 4-folds. These brane setups fully encode the infinite class of $2d$ $(0,2)$ quiver gauge theories on the worldvolume of the D1-branes and substantially streamline their connection to the probed geometries. We review various methods for efficiently generating Brane Brick Models. These algorithms are then used to construct $2d$ $(0,2)$ gauge theories for the cones over all the smooth Fano 3-folds and two infinite families of Sasaki-Einstein 7-manifolds with known metrics. This note is based on the author's talk at the Gauged Linear Sigma Models @ 30 conference at the Simons Center for Geometry and Physics.
PASOA- PArticle baSed Bayesian Optimal Adaptive design

We propose a new procedure named PASOA, for Bayesian experimental design, that performs sequential design optimization by simultaneously providing accurate estimates of successive posterior distributions for parameter inference. The sequential design process is carried out via a contrastive estimation principle, using stochastic optimization and Sequential Monte Carlo (SMC) samplers to maximise the Expected Information Gain (EIG). As larger information gains are obtained for larger distances between successive posterior distributions, this EIG objective may worsen classical SMC performance. To handle this issue, tempering is proposed to have both a large information gain and an accurate SMC sampling, that we show is crucial for performance. This novel combination of stochastic optimization and tempered SMC allows to jointly handle design optimization and parameter inference. We provide a proof that the obtained optimal design estimators benefit from some consistency property. Numerical experiments confirm the potential of the approach, which outperforms other recent existing procedures.
Towards Quantifying the Preconditioning Effect of Adam

There is a notable dearth of results characterizing the preconditioning effect of Adam and showing how it may alleviate the curse of ill-conditioning -- an issue plaguing gradient descent (GD). In this work, we perform a detailed analysis of Adam's preconditioning effect for quadratic functions and quantify to what extent Adam can mitigate the dependence on the condition number of the Hessian. Our key finding is that Adam can suffer less from the condition number but at the expense of suffering a dimension-dependent quantity. Specifically, for a $d$-dimensional quadratic with a diagonal Hessian having condition number $\kappa$, we show that the effective condition number-like quantity controlling the iteration complexity of Adam without momentum is $\mathcal{O}(\min(d, \kappa))$. For a diagonally dominant Hessian, we obtain a bound of $\mathcal{O}(\min(d \sqrt{d \kappa}, \kappa))$ for the corresponding quantity. Thus, when $d<\mathcal{O}(\kappa^p)$ where $p = 1$ for a diagonal Hessian and $p = 1/3$ for a diagonally dominant Hessian, Adam can outperform GD (which has an $\mathcal{O}(\kappa)$ dependence). On the negative side, our results suggest that Adam can be worse than GD for a sufficiently non-diagonal Hessian even if $d \ll \mathcal{O}(\kappa^{1/3})$; we corroborate this with empirical evidence. Finally, we extend our analysis to functions satisfying per-coordinate Lipschitz smoothness and a modified version of the Polyak-\L ojasiewicz condition.
Quantum walks, the discrete wave equation and Chebyshev polynomials

A quantum walk is the quantum analogue of a random walk. While it is relatively well understood how quantum walks can speed up random walk hitting times, it is a long-standing open question to what extent quantum walks can speed up the spreading or mixing rate of random walks on graphs. In this expository paper, inspired by a blog post by Terence Tao, we describe a particular perspective on this question that derives quantum walks from the discrete wave equation on graphs. This yields a description of the quantum walk dynamics as simply applying a Chebyshev polynomial to the random walk transition matrix. This perspective decouples the problem from its quantum origin, and highlights connections to earlier (non-quantum) work and the use of Chebyshev polynomials in random walk theory as in the Varopoulos-Carne bound. We illustrate the approach by proving a weak limit of the quantum walk dynamics on the lattice. This gives a different proof of the quadratically improved spreading behavior of quantum walks on lattices.
Treatment Initiation and Alcohol Use Violations Among Adults With DWI Arrests Who Received a Tailored Brief Intervention

Objective: This program evaluation examined rates of treatment initiation and alcohol-related violations among adults who completed a brief motivational interviewing (BMI) session after a driving while intoxicated (DWI) arrest. Method: Adults attending orientation for pretrial bond supervision were assessed for eligibility to receive services by counselors in an outpatient clinic. A total of 118 male and female adults with DWI arrests completed a single BMI session. Treatment initiation was assessed 1-week after the BMI session and alcohol-related violations were assessed 6 months later. Results: Fifty-three percent of those with DWI arrests who received the BMI session-initiated treatment and of those who initiated treatment, alcohol-related violations were significantly lower than for those who did not initiate treatment. Conclusions: The results provide preliminary support that a single session BMI delivered soon after arrest and before pretrial conditions are implemented has promise for spurring clients to take steps to initiate treatment and is related to better rates of compliance with alcohol-related pretrial conditions.
Understanding the Training Speedup from Sampling with Approximate Losses

It is well known that selecting samples with large losses/gradients can significantly reduce the number of training steps. However, the selection overhead is often too high to yield any meaningful gains in terms of overall training time. In this work, we focus on the greedy approach of selecting samples with large \textit{approximate losses} instead of exact losses in order to reduce the selection overhead. For smooth convex losses, we show that such a greedy strategy can converge to a constant factor of the minimum value of the average loss in fewer iterations than the standard approach of random selection. We also theoretically quantify the effect of the approximation level. We then develop SIFT which uses early exiting to obtain approximate losses with an intermediate layer's representations for sample selection. We evaluate SIFT on the task of training a 110M parameter 12-layer BERT base model and show significant gains (in terms of training hours and number of backpropagation steps) without any optimized implementation over vanilla training. For e.g., to reach 64% validation accuracy, SIFT with exit at the first layer takes ~43 hours compared to ~57 hours of vanilla training.
SemTra: A Semantic Skill Translator for Cross-Domain Zero-Shot Policy Adaptation

This work explores the zero-shot adaptation capability of semantic skills, semantically interpretable experts' behavior patterns, in cross-domain settings, where a user input in interleaved multi-modal snippets can prompt a new long-horizon task for different domains. In these cross-domain settings, we present a semantic skill translator framework SemTra which utilizes a set of multi-modal models to extract skills from the snippets, and leverages the reasoning capabilities of a pretrained language model to adapt these extracted skills to the target domain. The framework employs a two-level hierarchy for adaptation: task adaptation and skill adaptation. During task adaptation, seq-to-seq translation by the language model transforms the extracted skills into a semantic skill sequence, which is tailored to fit the cross-domain contexts. Skill adaptation focuses on optimizing each semantic skill for the target domain context, through parametric instantiations that are facilitated by language prompting and contrastive learning-based context inferences. This hierarchical adaptation empowers the framework to not only infer a complex task specification in one-shot from the interleaved multi-modal snippets, but also adapt it to new domains with zero-shot learning abilities. We evaluate our framework with Meta-World, Franka Kitchen, RLBench, and CARLA environments. The results clarify the framework's superiority in performing long-horizon tasks and adapting to different domains, showing its broad applicability in practical use cases, such as cognitive robots interpreting abstract instructions and autonomous vehicles operating under varied configurations.
Depth Separations in Neural Networks: Separating the Dimension from the Accuracy

We prove an exponential separation between depth 2 and depth 3 neural networks, when approximating an $\mathcal{O}(1)$-Lipschitz target function to constant accuracy, with respect to a distribution with support in $[0,1]^{d}$, assuming exponentially bounded weights. This addresses an open problem posed in \citet{safran2019depth}, and proves that the curse of dimensionality manifests in depth 2 approximation, even in cases where the target function can be represented efficiently using depth 3. Previously, lower bounds that were used to separate depth 2 from depth 3 required that at least one of the Lipschitz parameter, target accuracy or (some measure of) the size of the domain of approximation scale polynomially with the input dimension, whereas we fix the former two and restrict our domain to the unit hypercube. Our lower bound holds for a wide variety of activation functions, and is based on a novel application of an average- to worst-case random self-reducibility argument, to reduce the problem to threshold circuits lower bounds.
Toward an Android Static Analysis Approach for Data Protection

Android applications collecting data from users must protect it according to the current legal frameworks. Such data protection has become even more important since the European Union rolled out the General Data Protection Regulation (GDPR). Since app developers are not legal experts, they find it difficult to write privacy-aware source code. Moreover, they have limited tool support to reason about data protection throughout their app development process. This paper motivates the need for a static analysis approach to diagnose and explain data protection in Android apps. The analysis will recognize personal data sources in the source code, and aims to further examine the data flow originating from these sources. App developers can then address key questions about data manipulation, derived data, and the presence of technical measures. Despite challenges, we explore to what extent one can realize this analysis through static taint analysis, a common method for identifying security vulnerabilities. This is a first step towards designing a tool-based approach that aids app developers and assessors in ensuring data protection in Android apps, based on automated static program analysis.
Combination Therapy for Chronic Hepatitis B Using Capsid Recycling Inhibitor

In this paper, we investigate the dynamics of hepatitis B virus infection taking into account the implementation of combination therapy through mathematical modeling. This model is established considering the interplay between uninfected cells, infected cells, capsids, and viruses. Three drugs are considered for specific roles (i) pegylated interferon (PEG IFN) for immune modulation, (ii) lamivudine (LMV) as a reverse-transcriptase inhibitor, and (iii) entecavir (ETV) to block capsid recycling. Using these drugs, three combination therapies are introduced, specifically CT PEG IFN plus LMV, CT PEG IFN plus ETV, and CT PEG IFN plus LMV plus ETV. As a result, when LMV is used in combination therapy with PEG IFN and ETV, the impacts of ETV become insignificant. In conclusion, if the appropriate drug effectively inhibits reverse transcription, there is no need for an additional inhibitor to block capsid recycling.
Radiant fluence from ray tracing in optical multipass systems

In applications of optical multipass cells in photochemical reactors and laser excitation of weak transitions, estimation of the radiation dose in a volume of interest allows to assess the performance and optimize the design of the cell. We adopt radiant fluence as the figure of merit and employ the radiative transfer equation to derive analytical expressions for average radiant fluence in a given volume of interest. These are used to establish practical approximations and a Monte Carlo ray tracing model of the spatial distribution of fluence. Ray tracing is performed with Zemax OpticsStudio 18.9.
Mapping the Depths: A Stocktake of Underground Power Distribution in United States

A resilient energy infrastructure is crucial for addressing increasing extreme weather and climate risks. The undergrounding of the power system is one approach to building such resiliency. In this study, we introduce Grid Underground Distribution Statistics (GUDS) for the US, the first nationwide comprehensive assessment of underground electricity distribution at a high spatial granularity. In analyzing this dataset, we find regional differences in underground distribution rates, with generally higher rates for east and west coasts and in northern states, and lower rates in the central US. We also observe relationships between underground rates and factors such as household income levels, degree of urbanization, and vulnerability to natural hazards. Notably, regions with higher electricity rates are not associated with greater proportions of underground distribution, highlighting potential equity issues in infrastructure distribution. By presenting this granular information and insights on underground distribution, our study offers valuable guidance for informing planning and decision-making by policymakers, Independent System Operators, utilities, and end-users.
Nontrivial single axiom-schemata and their quasi-nontriviality of Le\'{s}niewski-Ishimoto's propositional ontology $\bf L_1$

On March 8, 1995, was found the following nontrivial single axiom-schema characteristic of Le\'{s}niewski-Ishimoto's propositional ontology $\bf L_1$ (Inou\'{e} \cite{inoue16}). $$(\mbox{$\mathrm{A_{M8}}$)} \enspace \epsilon ab \wedge \epsilon cd . \supset . \epsilon aa \wedge \epsilon cc \wedge (\epsilon bc \supset . \epsilon ad \wedge \epsilon ba). $$ In this paper, we shall present the progress about the above axiom-schema from 1995 and conjectures about it. Here we shall give two criteria \it nontiriviality \rm and \it quasi-nontriviality \rm in order to distinguish two axiom-schemata.