Miracle-dz/UniFAR-FTUs huggingface dataset:
Continuous finite element subgrid basis functions for Discontinuous Galerkin schemes on unstructured polygonal Voronoi meshes

We propose a new high order accurate nodal discontinuous Galerkin (DG) method for the solution of nonlinear hyperbolic systems of partial differential equations (PDE) on unstructured polygonal Voronoi meshes. Rather than using classical polynomials of degree N inside each element, in our new approach the discrete solution is represented by piecewise continuous polynomials of degree N within each Voronoi element, using a continuous finite element basis defined on a subgrid inside each polygon. We call the resulting subgrid basis an agglomerated finite element (AFE) basis for the DG method on general polygons, since it is obtained by the agglomeration of the finite element basis functions associated with the subgrid triangles. The basis functions on each sub-triangle are defined, as usual, on a universal reference element, hence allowing to compute universal mass, flux and stiffness matrices for the subgrid triangles once and for all in a pre-processing stage for the reference element only. Consequently, the construction of an efficient quadrature-free algorithm is possible, despite the unstructured nature of the computational grid. High order of accuracy in time is achieved thanks to the ADER approach, making use of an element-local space-time Galerkin finite element predictor. The novel schemes are carefully validated against a set of typical benchmark problems for the compressible Euler and Navier-Stokes equations. The numerical results have been checked with reference solutions available in literature and also systematically compared, in terms of computational efficiency and accuracy, with those obtained by the corresponding modal DG version of the scheme.
On the Complexity of Maximizing Social Welfare within Fair Allocations of Indivisible Goods

We consider the classical fair division problem which studies how to allocate resources fairly and efficiently. We give a complete landscape on the computational complexity and approximability of maximizing the social welfare within (1) envy-free up to any item (EFX) and (2) envy-free up to one item (EF1) allocations of indivisible goods for both normalized and unnormalized valuations. We show that a partial EFX allocation may have a higher social welfare than a complete EFX allocation, while it is well-known that this is not true for EF1 allocations. Thus, our first group of results focuses on the problem of maximizing social welfare subject to (partial) EFX allocations. For $n=2$ agents, we provide a polynomial time approximation scheme (PTAS) and an NP-hardness result. For a general number of agents $n>2$, we present algorithms that achieve approximation ratios of $O(n)$ and $O(\sqrt{n})$ for unnormalized and normalized valuations, respectively. These results are complemented by the asymptotically tight inapproximability results. We also study the same constrained optimization problem for EF1. For $n=2$, we show a fully polynomial time approximation scheme (FPTAS) and complement this positive result with an NP-hardness result. For general $n$, we present polynomial inapproximability ratios for both normalized and unnormalized valuations. Our results also imply the price of EFX is $\Theta(\sqrt{n})$ for normalized valuations, which is unknown in the previous literature.
Unbiased and efficient estimation of causal treatment effects in crossover trials

We introduce causal inference reasoning to crossover trials, with a focus on thorough QT (TQT) studies. For such trials, we propose different sets of assumptions and consider their impact on the modeling strategy and estimation procedure. We show that unbiased estimates of a causal treatment effect are obtained by a g‐computation approach in combination with weighted least squares predictions from a working regression model. Only a few natural requirements on the working regression and weighting matrix are needed for the result to hold. It follows that a large class of Gaussian linear mixed working models lead to unbiased estimates of a causal treatment effect, even if they do not capture the true data‐generating mechanism. We compare a range of working regression models in a simulation study where data are simulated from a complex data‐generating mechanism with input parameters estimated on a real TQT data set. In this setting, we find that for all practical purposes working models adjusting for baseline QTc measurements have comparable performance. Specifically, this is observed for working models that are by default too simplistic to capture the true data‐generating mechanism. Crossover trials and particularly TQT studies can be analyzed efficiently using simple working regression models without biasing the estimates for the causal parameters of interest.
Unsupervised learning of features and object boundaries from local prediction

A visual system has to learn both which features to extract from images and how to group locations into (proto-)objects. Those two aspects are usually dealt with separately, although predictability is discussed as a cue for both. To incorporate features and boundaries into the same model, we model a layer of feature maps with a pairwise Markov random field model in which each factor is paired with an additional binary variable, which switches the factor on or off. Using one of two contrastive learning objectives, we can learn both the features and the parameters of the Markov random field factors from images without further supervision signals. The features learned by shallow neural networks based on this loss are local averages, opponent colors, and Gabor-like stripe patterns. Furthermore, we can infer connectivity between locations by inferring the switch variables. Contours inferred from this connectivity perform quite well on the Berkeley segmentation database (BSDS500) without any training on contours. Thus, computing predictions across space aids both segmentation and feature learning, and models trained to optimize these predictions show similarities to the human visual system. We speculate that retinotopic visual cortex might implement such predictions over space through lateral connections.
BAN-Cap: A Multi-Purpose English-Bangla Image Descriptions Dataset

As computers have become efficient at understanding visual information and transforming it into a written representation, research interest in tasks like automatic image captioning has seen a significant leap over the last few years. While most of the research attention is given to the English language in a monolingual setting, resource-constrained languages like Bangla remain out of focus, predominantly due to a lack of standard datasets. Addressing this issue, we present a new dataset BAN-Cap following the widely used Flickr8k dataset, where we collect Bangla captions of the images provided by qualified annotators. Our dataset represents a wider variety of image caption styles annotated by trained people from different backgrounds. We present a quantitative and qualitative analysis of the dataset and the baseline evaluation of the recent models in Bangla image captioning. We investigate the effect of text augmentation and demonstrate that an adaptive attention-based model combined with text augmentation using Contextualized Word Replacement (CWR) outperforms all state-of-the-art models for Bangla image captioning. We also present this dataset’s multipurpose nature, especially on machine translation for Bangla-English and English-Bangla. This dataset and all the models will be useful for further research.
A Low-Complexity Brain–Computer Interface for High-Complexity Robot Swarm Control

A brain-computer interface (BCI) is a system that allows a human operator to use only mental commands in controlling end effectors that interact with the world around them. Such a system consists of a measurement device to record the human user’s brain activity, which is then processed into commands that drive a system end effector. BCIs involve either invasive measurements which allow for high-complexity control but are generally infeasible, or noninvasive measurements which offer lower quality signals but are more practical to use. In general, BCI systems have not been developed that efficiently, robustly, and scalably perform high-complexity control while retaining the practicality of noninvasive measurements. Here we leverage recent results from feedback information theory to fill this gap by modeling BCIs as a communications system and deploying a human-implementable interaction algorithm for noninvasive control of a high-complexity robot swarm. We construct a scalable dictionary of robotic behaviors that can be searched simply and efficiently by a BCI user, as we demonstrate through a large-scale user study testing the feasibility of our interaction algorithm, a user test of the full BCI system on (virtual and real) robot swarms, and simulations that verify our results against theoretical models. Our results provide a proof of concept for how a large class of high-complexity effectors (even beyond robotics) can be effectively controlled by a BCI system with low-complexity and noisy inputs.
AGILE Observations of GRB 220101A: A “New Year's Burst” with an Exceptionally Huge Energy Release

We report the AGILE observations of GRB 220101A, which took place at the beginning of 2022 January 1 and was recognized as one of the most energetic gamma-ray bursts (GRBs) ever detected since their discovery. The AGILE satellite acquired interesting data concerning the prompt phase of this burst, providing an overall temporal and spectral description of the event in a wide energy range, from tens of kiloelectronvolts to tens of megaelectronvolts. Dividing the prompt emission into three main intervals, we notice an interesting spectral evolution, featuring a notable hardening of the spectrum in the central part of the burst. The average fluxes encountered in the different time intervals are relatively moderate, with respect to those of other remarkable bursts, and the overall fluence exhibits a quite ordinary value among the GRBs detected by MCAL. However, GRB 220101A is the second farthest event detected by AGILE, and the burst with the highest isotropic equivalent energy of the entire MCAL GRB sample, releasing E iso = 2.54 × 1054 erg and exhibiting an isotropic luminosity of L iso = 2.34 × 1052 erg s−1 (both in the 400 keV–10 MeV energy range). We also analyzed the first 106 s of the afterglow phase, using the publicly available Swift-XRT data, carrying out a theoretical analysis of the afterglow, based on the forward shock model. We notice that GRB 220101A is with high probability surrounded by a wind-like density medium, and that the energy carried by the initial shock shall be a fraction of the total E iso, presumably near ∼50%.
Your Contrastive Learning Is Secretly Doing Stochastic Neighbor Embedding

Contrastive learning, especially self-supervised contrastive learning (SSCL), has achieved great success in extracting powerful features from unlabeled data. In this work, we contribute to the theoretical understanding of SSCL and uncover its connection to the classic data visualization method, stochastic neighbor embedding (SNE), whose goal is to preserve pairwise distances. From the perspective of preserving neighboring information, SSCL can be viewed as a special case of SNE with the input space pairwise similarities specified by data augmentation. The established correspondence facilitates deeper theoretical understanding of learned features of SSCL, as well as methodological guidelines for practical improvement. Specifically, through the lens of SNE, we provide novel analysis on domain-agnostic augmentations, implicit bias and robustness of learned features. To illustrate the practical advantage, we demonstrate that the modifications from SNE to $t$-SNE can also be adopted in the SSCL setting, achieving significant improvement in both in-distribution and out-of-distribution generalization.
Analysis of the ensemble Kalman–Bucy filter for correlated observation noise

Ensemble Kalman--Bucy filters (EnKBFs) are an important tool in Data Assimilation that aim to approximate the posterior distribution for continuous time filtering problems using an ensemble of interacting particles. In this work we extend a previously derived unifying framework for consistent representations of the posterior distribution to correlated observation noise and use these representations to derive an EnKBF suitable for this setting as a constant gain approximation of these optimal filters. Existence and uniqueness results for both the EnKBF and its mean field limit are provided. The existence and uniqueness of solutions to its limiting McKean-Vlasov equation does not seem to be covered by the existing literature. In the correlated noise case the evolution of the ensemble depends also on the pseudoinverse of its empirical covariance matrix, which has to be controlled for global well posedness. These bounds may also be of independent interest. Finally the convergence to the mean field limit is proven. The results can also be extended to other versions of EnKBFs.
Self-Supervised Visual Representation Learning with Semantic Grouping

In this paper, we tackle the problem of learning visual representations from unlabeled scene-centric data. Existing works have demonstrated the potential of utilizing the underlying complex structure within scene-centric data; still, they commonly rely on hand-crafted objectness priors or specialized pretext tasks to build a learning framework, which may harm generalizability. Instead, we propose contrastive learning from data-driven semantic slots, namely SlotCon, for joint semantic grouping and representation learning. The semantic grouping is performed by assigning pixels to a set of learnable prototypes, which can adapt to each sample by attentive pooling over the feature and form new slots. Based on the learned data-dependent slots, a contrastive objective is employed for representation learning, which enhances the discriminability of features, and conversely facilitates grouping semantically coherent pixels together. Compared with previous efforts, by simultaneously optimizing the two coupled objectives of semantic grouping and contrastive learning, our approach bypasses the disadvantages of hand-crafted priors and is able to learn object/group-level representations from scene-centric images. Experiments show our approach effectively decomposes complex scenes into semantic groups for feature learning and significantly benefits downstream tasks, including object detection, instance segmentation, and semantic segmentation. Code is available at: https://github.com/CVMI-Lab/SlotCon.
E2S2: Encoding-Enhanced Sequence-to-Sequence Pretraining for Language Understanding and Generation

Sequence-to-sequence (seq2seq) learning is a popular fashion for large-scale pretraining language models. However, the prior seq2seq pretraining models generally focus on reconstructive objectives on the decoder side and neglect the effect of encoder-side supervision, which we argue may lead to sub-optimal performance. To verify our hypothesis, we first empirically study the functionalities of the encoder and decoder in seq2seq pretrained language models, and find that the encoder takes an important but under-exploitation role than the decoder regarding the downstream performance and neuron activation. Therefore, we propose an encoding-enhanced seq2seq pretraining strategy, namely E2S2, which improves the seq2seq models via integrating more efficient self-supervised information into the encoders. Specifically, E2S2 adopts two self-supervised objectives on the encoder side from two aspects: 1) locally denoising the corrupted sentence (denoising objective); and 2) globally learning better sentence representations (contrastive objective). With the help of both objectives, the encoder can effectively distinguish the noise tokens and capture high-level (i.e., syntactic and semantic) knowledge, thus strengthening the ability of seq2seq model to accurately achieve the conditional generation. On a large diversity of downstream natural language understanding and generation tasks, E2S2 dominantly improves the performance of its powerful backbone models, e.g., BART and T5. For example, upon BART backbone, we achieve +1.1% averaged gain on the general language understanding evaluation (GLUE) benchmark and +1.75% F_0. 5 score improvement on CoNLL2014 dataset. We also provide in-depth analyses to show the improvement stems from better linguistic representation. We hope that our work will foster future self-supervision research on seq2seq language model pretraining.
Gathering despite a linear number of weakly Byzantine agents

We study the gathering problem to make multiple agents initially scattered in arbitrary networks gather at a single node. There exist $k$ agents with unique identifiers (IDs) in the network, and $f$ of them are weakly Byzantine agents, which behave arbitrarily except falsifying their identifiers. The agents behave in synchronous rounds, and each node does not have any memory like a whiteboard. In the literature, two algorithms for solving the gathering problem have been proposed. The first algorithm assumes that the number $n$ of nodes is given to agents and achieves the gathering in $O(n^{4}\cdot\vert \Lambda_{good}\vert \cdot\ X(n))$ rounds, where $\vert \Lambda_{good}\vert$ is the length of the largest ID among non-Byzantine agents, and $X(n)$ is the number of rounds required to explore any network composed of $n$ nodes. The second algorithm assumes that the upper bound $N$ of $n$ is given to agents and at least $4f^{2}+8f+4$ non-Byzantine agents exist, and achieves the gathering in $O((f+\vert \Lambda_{good}\vert)\cdot X(N))$ rounds. Both the algorithms allow agents to start gathering at different times. The first algorithm can terminate agents simultaneously, while the second one not. In this paper, we seek an algorithm that solves the gathering problem efficiently with the intermediate number of non-Byzantine agents since there is a large gap between the numbers of non-Byzantine agents in the previous works. The resultant gathering algorithm works with at least $8f+8$ non-Byzantine agents when agents start the algorithm at the same time, agents may terminate at different times, and $N$ is given to agents. To reduce the number of agents, we propose a new technique to simulate a Byzantine consensus algorithm for synchronous message-passing systems on agent systems. The proposed algorithm achieves the gathering in $o(f\cdot\vert \Lambda_{good}\vert \cdot X(N))$ rounds. This algorithm is faster than the first existing algorithm and requires fewer non-Byzantine agents than the second existing algorithm if $n$ is given to agents, although the guarantees on simultaneous termination and startup delay are not the same.
ComplexGen

We view the reconstruction of CAD models in the boundary representation (B-Rep) as the detection of geometric primitives of different orders, i.e., vertices, edges and surface patches, and the correspondence of primitives, which are holistically modeled as a chain complex, and show that by modeling such comprehensive structures more complete and regularized reconstructions can be achieved. We solve the complex generation problem in two steps. First, we propose a novel neural framework that consists of a sparse CNN encoder for input point cloud processing and a tri-path transformer decoder for generating geometric primitives and their mutual relationships with estimated probabilities. Second, given the probabilistic structure predicted by the neural network, we recover a definite B-Rep chain complex by solving a global optimization maximizing the likelihood under structural validness constraints and applying geometric refinements. Extensive tests on large scale CAD datasets demonstrate that the modeling of B-Rep chain complex structure enables more accurate detection for learning and more constrained reconstruction for optimization, leading to structurally more faithful and complete CAD B-Rep models than previous results.
Task-Agnostic Continual Reinforcement Learning: In Praise of a Simple Baseline

We study task-agnostic continual reinforcement learning (TACRL) in which standard RL challenges are compounded with partial observability stemming from task agnosticism, as well as additional difﬁculties of continual learning (CL), i.e., learning on a non-stationary sequence of tasks. Here we compare TACRL methods with their soft upper bounds prescribed by previous literature: multi-task learning (MTL) methods which do not have to deal with non-stationary data distributions, as well as task-aware methods, which are allowed to operate under full observability . We consider a previously unexplored and straightforward baseline for TACRL, replay-based recurrent RL (3RL), in which we augment an RL algorithm with recurrent mechanisms to address partial observability and experience replay mechanisms to address catastrophic forgetting in CL. Studying empirical performance in a sequence of RL tasks, we ﬁnd surprising occurrences of 3RL matching and overcoming the MTL and task-aware soft upper bounds. We lay out hypotheses that could explain this inﬂection point of continual and task-agnostic learning research. Our hypotheses are empirically tested in continuous control tasks via a large-scale study of the popular multi-task and continual learning benchmark Meta-World. By analyzing different training statistics including gradient conﬂict, we ﬁnd evidence that 3RL’s outperformance stems from its ability to quickly infer how new tasks relate with the previous ones, enabling forward transfer.
Guided Diffusion Model for Adversarial Purification

With wider application of deep neural networks (DNNs) in various algorithms and frameworks, security threats have become one of the concerns. Adversarial attacks disturb DNN-based image classifiers, in which attackers can intentionally add imperceptible adversarial perturbations on input images to fool the classifiers. In this paper, we propose a novel purification approach, referred to as guided diffusion model for purification (GDMP), to help protect classifiers from adversarial attacks. The core of our approach is to embed purification into the diffusion denoising process of a Denoised Diffusion Probabilistic Model (DDPM), so that its diffusion process could submerge the adversarial perturbations with gradually added Gaussian noises, and both of these noises can be simultaneously removed following a guided denoising process. On our comprehensive experiments across various datasets, the proposed GDMP is shown to reduce the perturbations raised by adversarial attacks to a shallow range, thereby significantly improving the correctness of classification. GDMP improves the robust accuracy by 5%, obtaining 90.1% under PGD attack on the CIFAR10 dataset. Moreover, GDMP achieves 70.94% robustness on the challenging ImageNet dataset.
An Optimization-based Algorithm for Non-stationary Kernel Bandits without Prior Knowledge

We propose an algorithm for non-stationary kernel bandits that does not require prior knowledge of the degree of non-stationarity. The algorithm follows randomized strategies obtained by solving optimization problems that balance exploration and exploitation. It adapts to non-stationarity by restarting when a change in the reward function is detected. Our algorithm enjoys a tighter dynamic regret bound than previous work on the non-stationary kernel bandit setting. Moreover, when applied to the non-stationary linear bandit setting by using a linear kernel, our algorithm is nearly minimax optimal, solving an open problem in the non-stationary linear bandit literature. We extend our algorithm to use a neural network for dynamically adapting the feature mapping to observed data. We prove a dynamic regret bound of the extension using the neural tangent kernel theory. We demonstrate empirically that our algorithm and the extension can adapt to varying degrees of non-stationarity.