Miracle-dz/UniFAR-FTUs huggingface dataset:
Technical Report: Distributed Asynchronous Large-Scale Mixed-Integer Linear Programming via Saddle Point Computation

We solve large-scale mixed-integer linear programs (MILPs) via distributed asynchronous saddle point computation. This is motivated by the MILPs being able to model problems in multi-agent autonomy, e.g., task assignment problems and trajectory planning with collision avoidance constraints in multi-robot systems. To solve a MILP, we relax it with a nonlinear program approximation whose accuracy tightens as the number of agents increases relative to the number of coupled constraints. Next, we form an equivalent Lagrangian saddle point problem, and then regularize the Lagrangian in both the primal and dual spaces to create a regularized Lagrangian that is strongly-convex-strongly-concave. We then develop a parallelized algorithm to compute saddle points of the regularized Lagrangian. This algorithm partitions problems into blocks, which are either scalars or sub-vectors of the primal or dual decision variables, and it is shown to tolerate asynchrony in the computations and communications of primal and dual variables. Suboptimality bounds and convergence rates are presented for convergence to a saddle point. The suboptimality bound includes (i) the regularization error induced by regularizing the Lagrangian and (ii) the suboptimality gap between solutions to the original MILP and its relaxed form. Simulation results illustrate these theoretical developments in practice, and show that relaxation and regularization together have only a mild impact on the quality of solution obtained.
Sharp Kohn-like phonon anomalies due to charge order can strongly enhance the superconducting T c

: Phonon softening is a ubiquitous phenomenon in condensed matter systems which is usually associated with charge density wave (CDW) instabilities and anharmonicity. The interplay between phonon softening, CDW and superconductivity is a topic of intense debate. In this work, the eﬀects of anomalous soft phonon instabilities on superconductivity are studied based on a recently developed theoretical framework that accounts for phonon damping and softening within the Migdal-Eliashberg theory. Model calculations show that the phonon softening in the form of a sharp dip in the phonon dispersion relation, either acoustic or optical (including the case of Kohn-type anomalies typically associated with CDW), can cause a manifold increase of the electron-phonon coupling constant λ . This, under certain conditions, which are consistent with the concept of optimal frequency introduced by Bergmann and Rainer, can produce a large increase of the superconducting transition temperature T c . In summary, our results suggest the possibility of reaching high-temperature superconductivity by exploiting soft phonon anomalies restricted in momentum space.
Management of adverse events related to endoscopic resection of upper gastrointestinal neoplasms: Review of the literature and recommendations from experts

Prevention therapy is recommended for lesions >1/2 of the esophageal circumference. Locoregional steroid injection is recommended for lesions >1/2–3/4 of the esophageal circumference and oral steroids are recommended for lesions >1/2 of the subtotal circumference. For lesions of the entire circumference, oral steroid combined with injection steroid is considered. Endoscopic balloon dilatation (EBD) is the first choice of treatment for stricture after esophageal endoscopic submucosal dissection (ESD). Radical incision and cutting or self‐expandable metallic stent can be considered for refractory stricture after EBD. In case of intraoperative perforation during esophageal ESD, endoscopic clip closure should be initially attempted. Surgery is considered for treatment of delayed perforation. Current standard practice for prevention of delayed bleeding after gastric ESD includes prophylactic coagulation of vessels on post‐ESD ulcers and giving proton pump inhibitors. Chronic kidney disease stage 4 or 5, multiple antithrombotic drug use, anticoagulant use, and heparin bridging therapy are high‐risk factors for delayed bleeding after gastric ESD. Intraoperative perforation during gastric ESD is initially managed by endoscopic clip closure. If endoscopic clip closure is difficult, other methods such as over‐the‐scope clip (OTSC), polyglycolic acid (PGA) sheet shielding etc. are attempted. Delayed perforation usually requires surgical intervention, but endoscopic closure by OTSC or PGA sheet may be considered. Resection of three‐quarters of the circumference is a risk factor for stenosis after gastric ESD. Giving prophylactic local steroid injection and/or oral steroid is reported, but effectiveness has not been fully verified as has been done for esophageal stricture. The main management method for gastric stenosis is EBD but it may cause perforation.
Casimir energy for spinor fields with δ-shell potentials

This work analyzes the Casimir energy of a massive spinor field propagating in flat space endowed with a spherically symmetric δ-function potential. By utilizing the spectral zeta function regularization method, the Casimir energy is evaluated after performing a suitable analytic continuation. Explicit numerical results are provided for specific cases in which the Casimir energy is unambiguously defined. The results described in this work represent a generalization of the MIT bag model for spinor fields.
Reconstructing high-order sequence features of dynamic functional connectivity networks based on diversified covert attention patterns for Alzheimer's disease classification

Recent studies have applied deep learning methods such as convolutional recurrent neural networks (CRNs) and Transformers to brain disease classification based on dynamic functional connectivity networks (dFCNs), such as Alzheimer's disease (AD), achieving better performance than traditional machine learning methods. However, in CRNs, the continuous convolution operations used to obtain high-order aggregation features may overlook the non-linear correlation between different brain regions due to the essence of convolution being the linear weighted sum of local elements. Inspired by modern neuroscience on the research of covert attention in the nervous system, we introduce the self-attention mechanism, a core module of Transformers, to model diversified covert attention patterns and apply these patterns to reconstruct high-order sequence features of dFCNs in order to learn complex dynamic changes in brain information flow. Therefore, we propose a novel CRN method based on diversified covert attention patterns, DCA-CRN, which combines the advantages of CRNs in capturing local spatio-temporal features and sequence change patterns, as well as Transformers in learning global and high-order correlation features. Experimental results on the ADNI and ADHD-200 datasets demonstrate the prediction performance and generalization ability of our proposed method.
Representing integers as a sum of three cubes

In this article we further develop methods for representing integers as a sum of three cubes. In particular, a barrier to solving the case k = 3, which was outlined in a previous paper of the second author, is overcome. A very recent computation indicates that the method is quite favourable to other methods in terms of time estimates. A hybrid of the method presented here and those in a previous paper is currently underway for unsolved cases.
PESE: Event Structure Extraction using Pointer Network based Encoder-Decoder Architecture

The task of event extraction (EE) aims to find the events and event-related argument information from the text and represent them in a structured format. Most previous works try to solve the problem by separately identifying multiple substructures and aggregating them to get the complete event structure. The problem with the methods is that it fails to identify all the interdependencies among the event participants (event-triggers, arguments, and roles). In this paper, we represent each event record in a unique tuple format that contains trigger phrase, trigger type, argument phrase, and corresponding role information. Our proposed pointer network-based encoder-decoder model generates an event tuple in each time step by exploiting the interactions among event participants and presenting a truly end-to-end solution to the EE task. We evaluate our model on the ACE2005 dataset, and experimental results demonstrate the effectiveness of our model by achieving competitive performance compared to the state-of-the-art methods.
On the Transferability of Visual Features in Generalized Zero-Shot Learning

Generalized Zero-Shot Learning (GZSL) aims to train a classiﬁer that can generalize to unseen classes, using a set of attributes as auxiliary information, and the visual features extracted from a pre-trained convolutional neural network. While recent GZSL methods have explored var-ious techniques to leverage the capacity of these features, there has been an extensive growth of representation learning techniques that remain under-explored. In this work, we investigate the utility of different GZSL methods when using different feature extractors, and examine how these models’ pre-training objectives, datasets, and architecture design af-fect their feature representation ability. Our results indicate that 1) methods using generative components for GZSL provide more advantages when using recent feature extractors; 2) feature extractors pre-trained using self-supervised learning objectives combined with cross-entropy and knowledge distillation provide better feature representations, increasing up to 15% performance when used with recent GZSL techniques; 3) speciﬁc feature extractors pre-trained with larger datasets do not necessarily boost the performance of GZSL methods. In addition, we investigate how GZSL methods fare against CLIP, a more recent multi-modal pre-trained model with strong zero-shot performance. We found that GZSL tasks still beneﬁt from generative-based GZSL methods along with CLIP’s internet-scale pre-training to achieve state-of-the-art performance in ﬁne-grained datasets. We release a modular framework for analyzing representation learning issues in GZSL here: https://github.com/uvavision/TV-GZSL.
HyperTuning: Toward Adapting Large Language Models without Back-propagation

Fine-tuning large language models for different tasks can be costly and inefficient, and even methods that reduce the number of tuned parameters still require full gradient-based optimization. We propose HyperTuning, a novel approach to model adaptation that uses a hypermodel to generate task-specific parameters for a fixed downstream model. We demonstrate a simple setup for hypertuning with HyperT5, a T5-based hypermodel that produces soft prefixes or LoRA parameters for a frozen T5 model from few-shot examples. We train HyperT5 in two stages: first, hyperpretraining with a modified conditional language modeling objective that trains a hypermodel to generate parameters; second, multi-task fine-tuning (MTF) on a large number of diverse language tasks. We evaluate HyperT5 on P3, MetaICL and Super-NaturalInstructions datasets, and show that it can effectively generate parameters for unseen tasks. Moreover, we show that using hypermodel-generated parameters as initializations for further parameter-efficient fine-tuning improves performance. HyperTuning can thus be a flexible and efficient way to leverage large language models for diverse downstream applications.
A Short Survey of Systematic Generalization

This survey includes systematic generalization and a history of how machine learning addresses it. We aim to summarize and organize the related information of both conventional and recent improvements. We first look at the definition of systematic generalization, then introduce Classicist and Connectionist. We then discuss different types of Connectionists and how they approach the generalization. Two crucial problems of variable binding and causality are discussed. We look into systematic generalization in language, vision, and VQA fields. Recent improvements from different aspects are discussed. Systematic generalization has a long history in artificial intelligence. We could cover only a small portion of many contributions. We hope this paper provides a background and is beneficial for discoveries in future work.
Asymptotic Properties of the Synthetic Control Method

This paper provides new insights into the asymptotic properties of the synthetic control method (SCM). We show that the synthetic control (SC) weight converges to a limiting weight that minimizes the mean squared prediction risk of the treatment-effect estimator when the number of pretreatment periods goes to infinity, and we also quantify the rate of convergence. Observing the link between the SCM and model averaging, we further establish the asymptotic optimality of the SC estimator under imperfect pretreatment fit, in the sense that it achieves the lowest possible squared prediction error among all possible treatment effect estimators that are based on an average of control units, such as matching, inverse probability weighting and difference-in-differences. The asymptotic optimality holds regardless of whether the number of control units is fixed or divergent. Thus, our results provide justifications for the SCM in a wide range of applications. The theoretical results are verified via simulations. JEL classification: C13, C21, C23
ArzEn-ST: A Three-way Speech Translation Corpus for Code-Switched Egyptian Arabic-English

We present our work on collecting ArzEn-ST, a code-switched Egyptian Arabic-English Speech Translation Corpus. This corpus is an extension of the ArzEn speech corpus, which was collected through informal interviews with bilingual speakers. In this work, we collect translations in both directions, monolingual Egyptian Arabic and monolingual English, forming a three-way speech translation corpus. We make the translation guidelines and corpus publicly available. We also report results for baseline systems for machine translation and speech translation tasks. We believe this is a valuable resource that can motivate and facilitate further research studying the code-switching phenomenon from a linguistic perspective and can be used to train and evaluate NLP systems.
Conditional negativity of anti-Loewner matrices

Let f be a positive, continuously differentiable function on (0, ∞). We consider matrices K f of the form We show that for such a function f with f(0) = f ′(0) = 0, all K f are conditionally negative definite if and only if all are positive semidefinite (p.s.d.), and give the integral representation of f. We also study functions f with K f (·, ·) conditionally negative definite or p.s. d.
Even Shorter Quantum Circuit for Phase Estimation on Early Fault-Tolerant Quantum Computers with Applications to Ground-State Energy Estimation

We develop a phase estimation method with a distinct feature: its maximal runtime (which determines the circuit depth) is $\delta/\epsilon$, where $\epsilon$ is the target precision, and the preconstant $\delta$ can be arbitrarily close to $0$ as the initial state approaches the target eigenstate. The total cost of the algorithm satisfies the Heisenberg-limited scaling $\widetilde{\mathcal{O}}(\epsilon^{-1})$. As a result, our algorithm may significantly reduce the circuit depth for performing phase estimation tasks on early fault-tolerant quantum computers. The key technique is a simple subroutine called quantum complex exponential least squares (QCELS). Our algorithm can be readily applied to reduce the circuit depth for estimating the ground-state energy of a quantum Hamiltonian, when the overlap between the initial state and the ground state is large. If this initial overlap is small, we can combine our method with the Fourier filtering method developed in [Lin, Tong, PRX Quantum 3, 010318, 2022], and the resulting algorithm provably reduces the circuit depth in the presence of a large relative overlap compared to $\epsilon$. The relative overlap condition is similar to a spectral gap assumption, but it is aware of the information in the initial state and is therefore applicable to certain Hamiltonians with small spectral gaps. We observe that the circuit depth can be reduced by around two orders of magnitude in numerical experiments under various settings.
Conv2Former: A Simple Transformer-Style ConvNet for Visual Recognition

This paper does not attempt to design a state-of-the-art method for visual recognition but investigates a more efficient way to make use of convolutions to encode spatial features. By comparing the design principles of the recent convolutional neural networks ConvNets) and Vision Transformers, we propose to simplify the self-attention by leveraging a convolutional modulation operation. We show that such a simple approach can better take advantage of the large kernels (>=7x7) nested in convolutional layers. We build a family of hierarchical ConvNets using the proposed convolutional modulation, termed Conv2Former. Our network is simple and easy to follow. Experiments show that our Conv2Former outperforms existent popular ConvNets and vision Transformers, like Swin Transformer and ConvNeXt in all ImageNet classification, COCO object detection and ADE20k semantic segmentation.
ModelDiff: A Framework for Comparing Learning Algorithms

We study the problem of (learning) algorithm comparison, where the goal is to find differences between models trained with two different learning algorithms. We begin by formalizing this goal as one of finding distinguishing feature transformations, i.e., input transformations that change the predictions of models trained with one learning algorithm but not the other. We then present ModelDiff, a method that leverages the datamodels framework (Ilyas et al., 2022) to compare learning algorithms based on how they use their training data. We demonstrate ModelDiff through three case studies, comparing models trained with/without data augmentation, with/without pre-training, and with different SGD hyperparameters. Our code is available at https://github.com/MadryLab/modeldiff .
Sampling-Based Optimization for Multi-Agent Model Predictive Control

We systematically review the Variational Optimization, Variational Inference and Stochastic Search perspectives on sampling-based dynamic optimization and discuss their connections to state-of-the-art optimizers and Stochastic Optimal Control (SOC) theory. A general convergence and sample complexity analysis on the three perspectives is provided through the unifying Stochastic Search perspective. We then extend these frameworks to their distributed versions for multi-agent control by combining them with consensus Alternating Direction Method of Multipliers (ADMM) to decouple the full problem into local neighborhood-level ones that can be solved in parallel. Model Predictive Control (MPC) algorithms are then developed based on these frameworks, leading to fully decentralized sampling-based dynamic optimizers. The capabilities of the proposed algorithms framework are demonstrated on multiple complex multi-agent tasks for vehicle and quadcopter systems in simulation. The results compare different distributed sampling-based optimizers and their centralized counterparts using unimodal Gaussian, mixture of Gaussians, and stein variational policies. The scalability of the proposed distributed algorithms is demonstrated on a 196-vehicle scenario where a direct application of centralized sampling-based methods is shown to be prohibitive.
S ELECTION I NDUCED C OLLIDER B IAS : A G ENDER P RONOUN U NCERTAINTY C ASE S TUDY

In this paper, we cast the problem of task underspecification in causal terms, and develop a method for empirical measurement of spurious associations between gender and gender-neutral entities for unmodified large language models, detecting previously unreported spurious correlations. We then describe a lightweight method to exploit the resulting spurious associations for prediction task uncertainty classification, achieving over 90% accuracy on a Winogender Schemas challenge set. Finally, we generalize our approach to address a wider range of prediction tasks and provide open-source demos for each method described here.
On DoF of Active RIS-Assisted MIMO Interference Channel With Arbitrary Antenna Configurations: When Will RIS Help?

An active reconfigurable intelligent surface (RIS) has been shown to be able to enhance the sum-of-degrees-of-freedom (DoF) of a two-user multiple-input multiple-output (MIMO) interference channel (IC) with equal number of antennas at each transmitter and receiver. However, for any number of receive and transmit antennas, when and how an active RIS can help to improve the sum-DoF are still unclear. This paper studies the sum-DoF of an active RIS-assisted two-user MIMO IC with arbitrary antenna configurations. In particular, RIS beamforming, transmit zero-forcing, and interference decoding are integrated together to combat the interference problem. In order to maximize the achievable sum-DoF, an integer optimization problem is formulated to optimize the number of eliminating interference links by RIS beamforming. As a result, the derived achievable sum-DoF can be higher than the sum-DoF of two-user MIMO IC, leading to a RIS gain. Furthermore, a sufficient condition of the RIS gain is given as the relationship between the number of RIS elements and the antenna configuration.
Discrete wavelet analysis of the influence of the North Atlantic Oscillation on Baltic Sea level

ABSTRACT Mean sea level (MSL) in the Baltic Sea is influenced by several factors and therefore presents a complex behaviour over a wide range of time-scales. This work performs a multi-scale analysis of MSL variability in the Baltic Sea using discrete wavelet analysis. The North Atlantic Oscillation (NAO) is well known for having a strong influence in MSL variability over the Baltic; however, the relationship between MSL and NAO at different time-scales is still little understood. In this work a comparison of MSL and NAO variability is performed for a wide range of temporal scales, uncovering distinct relationships in high-frequency and long-term temporal variability. The annual and sub-annual scales are found to account for more than 50% of the total MSL variability. The MSL annual cycle is analysed and a shift from low to high amplitude values is identified in the 1970s for most stations. MSL is found to be anti-correlated to NAO at short time-scales while on the long-term NAO and MSL appear to be positively correlated for most stations. The physical mechanisms behind these distinct relationships deserve deeper study.
Antibody binding reports spatial heterogeneities in cell membrane organization

The spatial organization of cell membrane glycoproteins and glycolipids is critical for mediating the binding of ligands, receptors, and macromolecules on the plasma membrane. However, we currently do not have the methods to quantify the spatial heterogeneities of macromolecular crowding on live cell surfaces. In this work, we combine experiment and simulation to report crowding heterogeneities on reconstituted membranes and live cell membranes with nanometer spatial resolution. By quantifying the effective binding affinity of IgG monoclonal antibodies to engineered antigen sensors, we discovered sharp gradients in crowding within a few nanometers of the crowded membrane surface. Our measurements on human cancer cells support the hypothesis that raft-like membrane domains exclude bulky membrane proteins and glycoproteins. Our facile and high-throughput method to quantify spatial crowding heterogeneities on live cell membranes may facilitate monoclonal antibody design and provide a mechanistic understanding of plasma membrane biophysical organization.