Miracle-dz/UniFAR-FTUs huggingface dataset:
MetaXLR - Mixed Language Meta Representation Transformation for Low-resource Cross-lingual Learning based on Multi-Armed Bandit

Transfer learning for extremely low resource languages is a challenging task as there is no large scale monolingual corpora for pre training or sufficient annotated data for fine tuning. We follow the work of MetaXL which suggests using meta learning for transfer learning from a single source language to an extremely low resource one. We propose an enhanced approach which uses multiple source languages chosen in a data driven manner. In addition, we introduce a sample selection strategy for utilizing the languages in training by using a multi armed bandit algorithm. Using both of these improvements we managed to achieve state of the art results on the NER task for the extremely low resource languages while using the same amount of data, making the representations better generalized. Also, due to the method ability to use multiple languages it allows the framework to use much larger amounts of data, while still having superior results over the former MetaXL method even with the same amounts of data.
Information trapping by topologically protected edge states: Scrambling and the butterfly velocity

Topological insulators and superconductors have attracted considerable attention, and many different theoretical tools have been used to gain insight into their properties. Here we investigate how perturbations can spread through exemplary one-dimensional topological insulators and superconductors using out-of-time ordered correlators. Out-of-time ordered correlators are often used to consider how information becomes scrambled during quantum dynamics. The wavefront of the out-of-time ordered correlator can be ballistic regardless of the underlying system dynamics, and here we confirm that for topological free fermion systems the wavefront spreads linearly at a characteristic butterfly velocity. We pay special attention to the topologically protected edge states, finding that"information"can become trapped in the edge states and essentially decoupled from the bulk, surviving for relatively long times. We consider different models with multiple possible edge states coexisting on a single edge.
The Mini-batch Stochastic Conjugate Algorithms with the unbiasedness and Minimized Variance Reduction

We firstly propose the new stochastic gradient estimate of unbiasedness and minimized variance in this paper. Secondly, we propose the two algorithms: Algorithml and Algorithm2 which apply the new stochastic gradient estimate to modern stochastic conjugate gradient algorithms SCGA 7and CGVR 8. Then we prove that the proposed algorithms can obtain linearconvergence rate under assumptions of strong convexity and smoothness. Finally, numerical experiments show that the new stochastic gradient estimatecan reduce variance of stochastic gradient effectively. And our algorithms compared SCGA and CGVR can convergent faster in numerical experimentson ridge regression model.
AccMER: Accelerating Multi-Agent Experience Replay with Cache Locality-Aware Prioritization

Multi-Agent Experience Replay (MER) is a key component of off-policy reinforcement learning (RL) algorithms. By remembering and reusing experiences from the past, experience replay significantly improves the stability of RL algorithms and their learning efficiency. In many scenarios, multiple agents interact in a shared environment during online training under centralized training and decentralized execution (CTDE) paradigm. Current multi-agent reinforcement learning (MARL) algorithms consider experience replay with uniform sampling or based on priority weights to improve transition data sample efficiency in the sampling phase. However, moving transition data histories for each agent through the processor memory hierarchy is a performance limiter. Also, as the agents' transitions continuously renew every iteration, the finite cache capacity results in increased cache misses. To this end, we propose AccMER, that repeatedly reuses the transitions (experiences) for a window of $n$ steps in order to improve the cache locality and minimize the transition data movement, instead of sampling new transitions at each step. Specifically, our optimization uses priority weights to select the transitions so that only high-priority transitions will be reused frequently, thereby improving the cache performance. Our experimental results on the Predator- Prey environment demonstrate the effectiveness of reusing the essential transitions based on the priority weights, where we observe an end-to-end training time reduction of 25.4% (for 32 agents) compared to existing prioritized MER algorithms without notable degradation in the mean reward.
A scalable domain decomposition method for FEM discretizations of nonlocal equations of integrable and fractional type

Nonlocal models allow for the description of phenomena which cannot be captured by classical partial differential equations. The availability of efficient solvers is one of the main concerns for the use of nonlocal models in real world engineering applications. We present a domain decomposition solver that is inspired by substructuring methods for classical local equations. In numerical experiments involving finite element discretizations of scalar and vectorial nonlocal equations of integrable and fractional type, we observe improvements in solution time of up to 14.6x compared to commonly used solver strategies.
Balancing Reconstruction and Editing Quality of GAN Inversion for Real Image Editing with StyleGAN Prior Latent Space

The exploration of the latent space in StyleGANs and GAN inversion exemplify impressive real-world image editing, yet the trade-off between reconstruction quality and editing quality remains an open problem. In this study, we revisit StyleGANs' hyperspherical prior $\mathcal{Z}$ and $\mathcal{Z}^+$ and integrate them into seminal GAN inversion methods to improve editing quality. Besides faithful reconstruction, our extensions achieve sophisticated editing quality with the aid of the StyleGAN prior. We project the real images into the proposed space to obtain the inverted codes, by which we then move along $\mathcal{Z}^{+}$, enabling semantic editing without sacrificing image quality. Comprehensive experiments show that $\mathcal{Z}^{+}$ can replace the most commonly-used $\mathcal{W}$, $\mathcal{W}^{+}$, and $\mathcal{S}$ spaces while preserving reconstruction quality, resulting in reduced distortion of edited images.
Rotating Features for Object Discovery

The binding problem in human cognition, concerning how the brain represents and connects objects within a fixed network of neural connections, remains a subject of intense debate. Most machine learning efforts addressing this issue in an unsupervised setting have focused on slot-based methods, which may be limiting due to their discrete nature and difficulty to express uncertainty. Recently, the Complex AutoEncoder was proposed as an alternative that learns continuous and distributed object-centric representations. However, it is only applicable to simple toy data. In this paper, we present Rotating Features, a generalization of complex-valued features to higher dimensions, and a new evaluation procedure for extracting objects from distributed representations. Additionally, we show the applicability of our approach to pre-trained features. Together, these advancements enable us to scale distributed object-centric representations from simple toy to real-world data. We believe this work advances a new paradigm for addressing the binding problem in machine learning and has the potential to inspire further innovation in the field.
RHFedMTL: Resource-Aware Hierarchical Federated Multi-Task Learning

The rapid development of artificial intelligence (AI) over massive applications including Internet-of-things on cellular network raises the concern of technical challenges such as privacy, heterogeneity and resource efficiency. Federated learning is an effective way to enable AI over massive distributed nodes with security. However, conventional works mostly focus on learning a single global model for a unique task across the network, and are generally less competent to handle multi-task learning (MTL) scenarios with stragglers at the expense of acceptable computation and communication cost. Meanwhile, it is challenging to ensure the privacy while maintain a coupled multi-task learning across multiple base stations (BSs) and terminals. In this paper, inspired by the natural cloud-BS-terminal hierarchy of cellular works, we provide a viable resource-aware hierarchical federated MTL (RHFedMTL) solution to meet the heterogeneity of tasks, by solving different tasks within the BSs and aggregating the multi-task result in the cloud without compromising the privacy. Specifically, a primal-dual method has been leveraged to effectively transform the coupled MTL into some local optimization sub-problems within BSs. Furthermore, compared with existing methods to reduce resource cost by simply changing the aggregation frequency, we dive into the intricate relationship between resource consumption and learning accuracy, and develop a resource-aware learning strategy for local terminals and BSs to meet the resource budget. Extensive simulation results demonstrate the effectiveness and superiority of RHFedMTL in terms of improving the learning accuracy and boosting the convergence rate.
On Mixing Rates for Bayesian CART

The success of Bayesian inference with MCMC depends critically on Markov chains rapidly reaching the posterior distribution. Despite the plentitude of inferential theory for posteriors in Bayesian non-parametrics, convergence properties of MCMC algorithms that simulate from such ideal inferential targets are not thoroughly understood. This work focuses on the Bayesian CART algorithm which forms a building block of Bayesian Additive Regression Trees (BART). We derive upper bounds on mixing times for typical posteriors under various proposal distributions. Exploiting the wavelet representation of trees, we provide sufficient conditions for Bayesian CART to mix well (polynomially) under certain hierarchical connectivity restrictions on the signal. We also derive a negative result showing that Bayesian CART (based on simple grow and prune steps) cannot reach deep isolated signals in faster than exponential mixing time. To remediate myopic tree exploration, we propose Twiggy Bayesian CART which attaches/detaches entire twigs (not just single nodes) in the proposal distribution. We show polynomial mixing of Twiggy Bayesian CART without assuming that the signal is connected on a tree. Going further, we show that informed variants achieve even faster mixing. A thorough simulation study highlights discrepancies between spike-and-slab priors and Bayesian CART under a variety of proposals.
The Brain Tumor Segmentation (BraTS-METS) Challenge 2023: Brain Metastasis Segmentation on Pre-treatment MRI

Clinical monitoring of metastatic disease to the brain can be a laborious and timeconsuming process, especially in cases involving multiple metastases when the assessment is performed manually. The Response Assessment in Neuro-Oncology Brain Metastases (RANO-BM) guideline, which utilizes the unidimensional longest diameter, is commonly used in clinical and research settings to evaluate response to therapy in patients with brain metastases. However, accurate volumetric assessment of the lesion and surrounding peri-lesional edema holds significant importance in clinical decision-making and can greatly enhance outcome prediction. The unique challenge in performing segmentations of brain metastases lies in their common occurrence as small lesions. Detection and segmentation of lesions that are smaller than 10 mm in size has not demonstrated high accuracy in prior publications. The brain metastases challenge sets itself apart from previously conducted MICCAI challenges on glioma segmentation due to the significant variability in lesion size. Unlike gliomas, which tend to be larger on presentation scans, brain metastases exhibit a wide range of sizes and tend to include small lesions. We hope that the BraTS-METS dataset and challenge will advance the field of automated brain metastasis detection and segmentation.
Neural Textured Deformable Meshes for Robust Analysis-by-Synthesis

Human vision demonstrates higher robustness than current AI algorithms under out-of-distribution scenarios. It has been conjectured such robustness benefits from performing analysis-by-synthesis. Our paper formulates triple vision tasks in a consistent manner using approximate analysis-by-synthesis by render-and-compare algorithms on neural features. In this work, we introduce Neural Textured Deformable Meshes, which involve the object model with deformable geometry that allows optimization on both camera parameters and object geometries. The deformable mesh is parameterized as a neural field, and covered by whole-surface neural texture maps, which are trained to have spatial discriminability. During inference, we extract the feature map of the test image and subsequently optimize the 3D pose and shape parameters of our model using differentiable rendering to best reconstruct the target feature map. We show that our analysis-by-synthesis is much more robust than conventional neural networks when evaluated on real-world images and even in challenging out-of-distribution scenarios, such as occlusion and domain shift. Our algorithms are competitive with standard algorithms when tested on conventional performance measures.
Cell motility modes are selected by the interplay of mechanosensitive adhesion and membrane tension

The initiation of directional cell motion requires symmetry breaking that can happen both with or without external stimuli. During cell crawling, forces generated by the cytoskeleton and their transmission through mechanosensitive adhesions to the extracellular substrate play a crucial role. In a recently proposed 1D model (Sens, PNAS 2020), a mechanical feedback loop between force-sensitive adhesions and cell tension was shown to be sufficient to explain spontaneous symmetry breaking and multiple motility patterns through stick-slip dynamics, without the need to account for signaling networks or active polar gels. We extended this model to 2D to study the interplay between cell shape and mechanics during crawling. Through a local force balance along a deformable boundary, we show that the membrane tension coupled with shape change can regulate the spatiotemporal evolution of the stochastic binding of mechanosensitive adhesions. Linear stability analysis identified the unstable parameter regimes where spontaneous symmetry breaking can take place. Using simulations to solve the fully coupled nonlinear system of equations, we show that starting from a randomly perturbed circular shape, this instability can lead to keratocyte-like shapes. Simulations predict that different adhesion kinetics and membrane tension can result in different cell motility modes including gliding, zigzag, rotating, and sometimes chaotic movements. Thus, using a minimal model of cell motility, we identify that the interplay between adhesions and tension can select emergent motility modes.
Space Net Optimization

Most metaheuristic algorithms rely on a few searched solutions to guide later searches during the convergence process for a simple reason: the limited computing resource of a computer makes it impossible to retain all the searched solutions. This also reveals that each search of most metaheuristic algorithms is just like a ballpark guess. To help address this issue, we present a novel metaheuristic algorithm called space net optimization (SNO). It is equipped with a new mechanism called space net; thus, making it possible for a metaheuristic algorithm to use most information provided by all searched solutions to depict the landscape of the solution space. With the space net, a metaheuristic algorithm is kind of like having a ``vision'' on the solution space. Simulation results show that SNO outperforms all the other metaheuristic algorithms compared in this study for a set of well-known single objective bound constrained problems in most cases.
Axion helioscopes as solar thermometers

Axions, if discovered, could serve as a powerful new messenger for studying astrophysical objects. In this study we show how the Sun's spatial and spectral “axion image” can be inverted to infer the radial dependence of solar properties in a model-independent way. In particular, the future helioscope IAXO may allow us to accurately reconstruct the Sun's temperature profile T(r) in the region up to about 80% (40%) of the solar radius for an axion-photon coupling g aγγ of 6 × 10-11 GeV-1 (10-11 GeV-1). The statistical fluctuations in the photon data lead to a median precision of better than 10% (16%) in this region, and the corresponding median accuracy was better than 4% (7%). While our approach can simultaneously infer the radial profile of the Debye scale κ s(r), its weaker connection to the axion production rate leads to median accuracy and precision of worse than 30% and 50%, respectively. We discuss possible challenges and improvements for realistic setups, as well as extensions to more general axion models. We also highlight advantages of helioscopes over neutrino detectors.
Accelerated Fingerprint Enhancement: A GPU-Optimized Mixed Architecture Approach

This document presents a preliminary approach to latent fingerprint enhancement, fundamentally designed around a mixed Unet architecture. It combines the capabilities of the Resnet-101 network and Unet encoder, aiming to form a potentially powerful composite. This combination, enhanced with attention mechanisms and forward skip connections, is intended to optimize the enhancement of ridge and minutiae features in fingerprints. One innovative element of this approach includes a novel Fingerprint Enhancement Gabor layer, specifically designed for GPU computations. This illustrates how modern computational resources might be harnessed to expedite enhancement. Given its potential functionality as either a CNN or Transformer layer, this Gabor layer could offer improved agility and processing speed to the system. However, it is important to note that this approach is still in the early stages of development and has not yet been fully validated through rigorous experiments. As such, it may require additional time and testing to establish its robustness and usability in the field of latent fingerprint enhancement. This includes improvements in processing speed, enhancement adaptability with distinct latent fingerprint types, and full validation in experimental approaches such as open-set (identification 1:N) and open-set validation, fingerprint quality evaluation, among others.
Analysis of Flexible Structured Fuzzy Logic Controllers

We suggest a new approach to the construction of fuzzy logic controllers based upon the selection of systems parameters. We first show that the standard fuzzy logic controllers have four basic operations which determine the nature of their functioning, the aggregation process used to combine individual antecedent firing levels to give a rule firing level, the determination of rule output based on antecedent firing level, the aggregation of individual rule outputs to find combined output, and the defuzzification process. The next show how we can parameterize these operations using S-OWA operators. These parameterized models give us a new class of flexible structured fuzzy logic controllers (FS-FLC). We look at the structure and performance of these controllers. We then suggest that one can improve the structure of fuzzy logic controllers by learning the values of the parameters introduced. >
Scattering versus forbidden decay in dark matter freeze-in

It is generically believed that the two-body scattering is suppressed by higher-order weak couplings with respect to the two-body decay. We show that this does not always hold when a heavy particle is produced by forbidden decay in a thermal plasma, where the scattering shares the same order of couplings with the decay. We find that there is a simple and close relation between the forbidden decay and the same-order scattering. To illustrate this point, we consider freeze-in production of heavy dark matter via a light scalar mediator. We point out that, when the Boltzmann (quantum) statistics is used, the forbidden decay can contribute to the dark matter relic density at 5$\%$-24$\%$ (10$\%$-$39\%$) with a weak thermal coupling, while the contribution from the scattering channel can be several orders of magnitude larger than from the forbidden decay if the thermal coupling is much smaller. Such a relative effect between the scattering and the forbidden decay could also exist in other plasma-induced processes, such as the purely thermal generation of the right-handed neutrino dark matter, or of the lepton asymmetry in leptogenesis.
MCDIP‐ADMM: Overcoming overfitting in DIP‐based CT reconstruction

This paper investigates the application of unsupervised learning methods for computed tomography reconstruction. To motivate our work, we review several existing priors, namely the truncated Gaussian prior, the l1 prior, the total variation prior, and the deep image prior (DIP). We find that DIP outperforms the other three priors in terms of representational capability and visual performance. However, the performance of DIP deteriorates when the number of iterations exceeds a certain threshold due to overfitting. To address this issue, we propose a novel method (MCDIP‐ADMM) based on multi‐code deep image prior (MCDIP) and plug‐and‐play alternative direction method of multipliers (ADMM). Specifically, MCDIP utilizes multiple latent codes to generate a series of feature maps at an intermediate layer within a generator model. These maps are then composed with trainable weights, representing the complete image prior. Experimental results demonstrate the superior performance of the proposed MCDIP‐ADMM compared to three existing competitors. In the case of parallel beam projection with Gaussian noise, MCDIP‐ADMM achieves an average improvement of 4.3 dB over DIP, 1.7 dB over ADMM DIP‐weighted total variation (WTV) and 1.2 dB over PnP‐DIP in terms of peak‐signal‐to‐noise ratio (PSNR). Similarly, for fan‐beam projection with Poisson noise, MCDIP‐ADMM achieves an average improvement of 3.09 dB over DIP, 1.86 dB over ADMM DIP‐WTV and 0.84 dB over PnP‐DIP in terms of PSNR.
IQL-TD-MPC: Implicit Q-Learning for Hierarchical Model Predictive Control

Model-based reinforcement learning (RL) has shown great promise due to its sample efficiency, but still struggles with long-horizon sparse-reward tasks, especially in offline settings where the agent learns from a fixed dataset. We hypothesize that model-based RL agents struggle in these environments due to a lack of long-term planning capabilities, and that planning in a temporally abstract model of the environment can alleviate this issue. In this paper, we make two key contributions: 1) we introduce an offline model-based RL algorithm, IQL-TD-MPC, that extends the state-of-the-art Temporal Difference Learning for Model Predictive Control (TD-MPC) with Implicit Q-Learning (IQL); 2) we propose to use IQL-TD-MPC as a Manager in a hierarchical setting with any off-the-shelf offline RL algorithm as a Worker. More specifically, we pre-train a temporally abstract IQL-TD-MPC Manager to predict"intent embeddings", which roughly correspond to subgoals, via planning. We empirically show that augmenting state representations with intent embeddings generated by an IQL-TD-MPC manager significantly improves off-the-shelf offline RL agents' performance on some of the most challenging D4RL benchmark tasks. For instance, the offline RL algorithms AWAC, TD3-BC, DT, and CQL all get zero or near-zero normalized evaluation scores on the medium and large antmaze tasks, while our modification gives an average score over 40.