Miracle-dz/UniFAR-FTUs huggingface dataset:
Contributions to the Domino Problem: Seeding, Recurrence and Satisfiability

We study the seeded domino problem, the recurring domino problem and the $k$-SAT problem on finitely generated groups. These problems are generalization of their original versions on $\mathbb{Z}^2$ that were shown to be undecidable using the domino problem. We show that the seeded and recurring domino problems on a group are invariant under changes in the generating set, are many-one reduced from the respective problems on subgroups, and are positive equivalent to the problems on finite index subgroups. This leads to showing that the recurring domino problem is decidable for free groups. Coupled with the invariance properties, we conjecture that the only groups in which the seeded and recurring domino problems are decidable are virtually free groups. In the case of the $k$-SAT problem, we introduce a new generalization that is compatible with decision problems on finitely generated groups. We show that the subgroup membership problem many-one reduces to the $2$-SAT problem, that in certain cases the $k$-SAT problem many one reduces to the domino problem, and finally that the domino problem reduces to $3$-SAT for the class of scalable groups.
Semi-supervised Semantic Segmentation Meets Masked Modeling: Fine-grained Locality Learning Matters in Consistency Regularization

Semi-supervised semantic segmentation aims to utilize limited labeled images and abundant unlabeled images to achieve label-efficient learning, wherein the weak-to-strong consistency regularization framework, popularized by FixMatch, is widely used as a benchmark scheme. Despite its effectiveness, we observe that such scheme struggles with satisfactory segmentation for the local regions. This can be because it originally stems from the image classification task and lacks specialized mechanisms to capture fine-grained local semantics that prioritizes in dense prediction. To address this issue, we propose a novel framework called \texttt{MaskMatch}, which enables fine-grained locality learning to achieve better dense segmentation. On top of the original teacher-student framework, we design a masked modeling proxy task that encourages the student model to predict the segmentation given the unmasked image patches (even with 30\% only) and enforces the predictions to be consistent with pseudo-labels generated by the teacher model using the complete image. Such design is motivated by the intuition that if the predictions are more consistent given insufficient neighboring information, stronger fine-grained locality perception is achieved. Besides, recognizing the importance of reliable pseudo-labels in the above locality learning and the original consistency learning scheme, we design a multi-scale ensembling strategy that considers context at different levels of abstraction for pseudo-label generation. Extensive experiments on benchmark datasets demonstrate the superiority of our method against previous approaches and its plug-and-play flexibility.
Oscillating edge current in polar active fluid

Dense bacterial suspensions exhibit turbulent behaviour called ``bacterial turbulence''. The behavior of the bulk unconstrained bacterial turbulence is described well by the Toner-Tu-Swift-Hohenberg (TTSH) equation for the velocity field. However, it remains unclear how we should treat boundary conditions on bacterial turbulence in contact with some boundaries (e.g. solid walls). To be more specific, although the importance of the ``edge current'', the flow along the boundary, has been demonstrated in several experimental studies on confined bacterial suspensions, previous numerical studies based on the TTSH equation employ non-slip boundary conditions and do not seem to describe properly the behavior of bacteria near the boundaries. In this study, we impose a slip boundary condition on the TTSH equation to describe the bacterial motion at boundaries. We develop a method to implement the slip boundary condition. Using this method, we have successfully produced edge current and discovered that the direction of the edge current temporally oscillates. The oscillation can be attributable to the advection term in the TTSH equation. Our work demonstrates that boundary conditions could play an important role in the collective dynamics of active systems.
Style Injection in Diffusion: A Training-free Approach for Adapting Large-scale Diffusion Models for Style Transfer

Despite the impressive generative capabilities of diffusion models, existing diffusion model-based style transfer methods require inference-stage optimization (e.g. fine-tuning or textual inversion of style) which is time-consuming, or fails to leverage the generative ability of large-scale diffusion models. To address these issues, we introduce a novel artistic style transfer method based on a pre-trained large-scale diffusion model without any optimization. Specifically, we manipulate the features of self-attention layers as the way the cross-attention mechanism works; in the generation process, substituting the key and value of content with those of style image. This approach provides several desirable characteristics for style transfer including 1) preservation of content by transferring similar styles into similar image patches and 2) transfer of style based on similarity of local texture (e.g. edge) between content and style images. Furthermore, we introduce query preservation and attention temperature scaling to mitigate the issue of disruption of original content, and initial latent Adaptive Instance Normalization (AdaIN) to deal with the disharmonious color (failure to transfer the colors of style). Our experimental results demonstrate that our proposed method surpasses state-of-the-art methods in both conventional and diffusion-based style transfer baselines.
Spin-taste structure of minimally doubled fermions

Minimally doubled fermions realize one degenerate pair of Dirac fermions on the lattice. Similarities to staggered fermions exist, namely, spin and taste degrees of freedom become intertwined, and a remnant, non-singlet chiral symmetry and ultralocality are maintained. However, charge conjugation, isotropy and some space-time reflection symmetries are broken by the cutoff. For two variants, i.e., Karsten-Wilczek (KW) or Borici-Creutz (BC) fermions, a tasted charge conjugation symmetry can be identified, and the respective representations of the spin-taste algebra can be constructed explicitly. In the case of BC fermions, the tasted symmetry indicates that amendments to the published counterterms are necessary. The spin-taste representation on the quark level permits construction of local or extended hadron interpolating operators for any spin-taste combination, albeit with contamination by parity partners and taste-symmetry violation. The few available numerical results for KW fermions are in line with expectations.
Read Between the Layers: Leveraging Intra-Layer Representations for Rehearsal-Free Continual Learning with Pre-Trained Models

We address the Continual Learning (CL) problem, where a model has to learn a sequence of tasks from non-stationary distributions while preserving prior knowledge as it encounters new experiences. With the advancement of foundation models, CL research has shifted focus from the initial learning-from-scratch paradigm to the use of generic features from large-scale pre-training. However, existing approaches to CL with pre-trained models only focus on separating the class-specific features from the final representation layer and neglect the power of intermediate representations that capture low- and mid-level features naturally more invariant to domain shifts. In this work, we propose LayUP, a new class-prototype-based approach to continual learning that leverages second-order feature statistics from multiple intermediate layers of a pre-trained network. Our method is conceptually simple, does not require any replay buffer, and works out of the box with any foundation model. LayUP improves over the state-of-the-art on four of the seven class-incremental learning settings at a considerably reduced memory and computational footprint compared with the next best baseline. Our results demonstrate that fully exhausting the representational capacities of pre-trained models in CL goes far beyond their final embeddings.
Localization with Reconfigurable Intelligent Surface: An Active Sensing Approach

This paper addresses an uplink localization problem in which a base station (BS) aims to locate a remote user with the help of reconfigurable intelligent surfaces (RISs). We propose a strategy in which the user transmits pilots sequentially and the BS adaptively adjusts the sensing vectors, including the BS beamforming vector and multiple RIS reflection coefficients based on the observations already made, to eventually produce an estimated user position. This is a challenging active sensing problem for which finding an optimal solution involves searching through a complicated functional space whose dimension increases with the number of measurements. We show that the long short-term memory (LSTM) network can be used to exploit the latent temporal correlation between measurements to automatically construct scalable state vectors. Subsequently, the state vector is mapped to the sensing vectors for the next time frame via a deep neural network (DNN). A final DNN is used to map the state vector to the estimated user position. Numerical result illustrates the advantage of the active sensing design as compared to non-active sensing methods. The proposed solution produces interpretable results and is generalizable in the number of sensing stages. Remarkably, we show that a network with one BS and multiple RISs can outperform a comparable setting with multiple BSs.
TinyGSM: achieving >80% on GSM8k with small language models

Small-scale models offer various computational advantages, and yet to which extent size is critical for problem-solving abilities remains an open question. Specifically for solving grade school math, the smallest model size so far required to break the 80\% barrier on the GSM8K benchmark remains to be 34B. Our work studies how high-quality datasets may be the key for small language models to acquire mathematical reasoning. We introduce \texttt{TinyGSM}, a synthetic dataset of 12.3M grade school math problems paired with Python solutions, generated fully by GPT-3.5. After finetuning on \texttt{TinyGSM}, we find that a duo of a 1.3B generation model and a 1.3B verifier model can achieve 81.5\% accuracy, outperforming existing models that are orders of magnitude larger. This also rivals the performance of the GPT-3.5 ``teacher'' model (77.4\%), from which our model's training data is generated. Our approach is simple and has two key components: 1) the high-quality dataset \texttt{TinyGSM}, 2) the use of a verifier, which selects the final outputs from multiple candidate generations.
Data and Model Poisoning Backdoor Attacks on Wireless Federated Learning, and the Defense Mechanisms: A Comprehensive Survey

Due to the greatly improved capabilities of devices, massive data, and increasing concern about data privacy, Federated Learning (FL) has been increasingly considered for applications to wireless communication networks (WCNs). Wireless FL (WFL) is a distributed method of training a global deep learning model in which a large number of participants each train a local model on their training datasets and then upload the local model updates to a central server. However, in general, non-independent and identically distributed (non-IID) data of WCNs raises concerns about robustness, as a malicious participant could potentially inject a"backdoor"into the global model by uploading poisoned data or models over WCN. This could cause the model to misclassify malicious inputs as a specific target class while behaving normally with benign inputs. This survey provides a comprehensive review of the latest backdoor attacks and defense mechanisms. It classifies them according to their targets (data poisoning or model poisoning), the attack phase (local data collection, training, or aggregation), and defense stage (local training, before aggregation, during aggregation, or after aggregation). The strengths and limitations of existing attack strategies and defense mechanisms are analyzed in detail. Comparisons of existing attack methods and defense designs are carried out, pointing to noteworthy findings, open challenges, and potential future research directions related to security and privacy of WFL.
On a binary Diophantine inequality involving primes of a special type

Let $1<c<\frac{1787}{1502}$ and $N$ be a sufficiently large real number. In this paper, it is proved that for any arbitrarily large number $E>0$ and for almost all real $R \in (N,2N]$, the Diophantine inequality $$|p_{1}^{c}+p_{2}^{c}-R|<(log N)^{-E}$$ is solvable in prime variables $p_1,p_2$ such that, each of the numbers $p_{1}+2,p_{2}+2$ has at most $[\frac{79606}{35740-30040c}]$ prime factors, counted with multiplicity. Moreover, we prove that the Diophantine inequality $$|p_{1}^{c}+p_{2}^{c}+p_{3}^{c}+p_{4}^{c}-N|<(log N)^{-E}$$ is solvable in prime variables $p_1,p_2,p_3,p_4$ such that, each of the numbers $p_{i}+2(i=1,2,3,4)$ has at most $[\frac{93801402}{35740000-30040000c}]$ prime factors, counted with multiplicity.
Managing the unknown: a survey on Open Set Recognition and tangential areas

In real-world scenarios classification models are often required to perform robustly when predicting samples belonging to classes that have not appeared during its training stage. Open Set Recognition addresses this issue by devising models capable of detecting unknown classes from samples arriving during the testing phase, while maintaining a good level of performance in the classification of samples belonging to known classes. This review comprehensively overviews the recent literature related to Open Set Recognition, identifying common practices, limitations, and connections of this field with other machine learning research areas, such as continual learning, out-of-distribution detection, novelty detection, and uncertainty estimation. Our work also uncovers open problems and suggests several research directions that may motivate and articulate future efforts towards more safe Artificial Intelligence methods.
Accelerating Meta-Learning by Sharing Gradients

The success of gradient-based meta-learning is primarily attributed to its ability to leverage related tasks to learn task-invariant information. However, the absence of interactions between different tasks in the inner loop leads to task-specific over-fitting in the initial phase of meta-training. While this is eventually corrected by the presence of these interactions in the outer loop, it comes at a significant cost of slower meta-learning. To address this limitation, we explicitly encode task relatedness via an inner loop regularization mechanism inspired by multi-task learning. Our algorithm shares gradient information from previously encountered tasks as well as concurrent tasks in the same task batch, and scales their contribution with meta-learned parameters. We show using two popular few-shot classification datasets that gradient sharing enables meta-learning under bigger inner loop learning rates and can accelerate the meta-training process by up to 134%.
Fair Active Learning in Low-Data Regimes

In critical machine learning applications, ensuring fairness is essential to avoid perpetuating social inequities. In this work, we address the challenges of reducing bias and improving accuracy in data-scarce environments, where the cost of collecting labeled data prohibits the use of large, labeled datasets. In such settings, active learning promises to maximize marginal accuracy gains of small amounts of labeled data. However, existing applications of active learning for fairness fail to deliver on this, typically requiring large labeled datasets, or failing to ensure the desired fairness tolerance is met on the population distribution. To address such limitations, we introduce an innovative active learning framework that combines an exploration procedure inspired by posterior sampling with a fair classification subroutine. We demonstrate that this framework performs effectively in very data-scarce regimes, maximizing accuracy while satisfying fairness constraints with high probability. We evaluate our proposed approach using well-established real-world benchmark datasets and compare it against state-of-the-art methods, demonstrating its effectiveness in producing fair models, and improvement over existing methods.
Multi-Modal Learning-based Reconstruction of High-Resolution Spatial Wind Speed Fields

Wind speed at sea surface is a key quantity for a variety of scientific applications and human activities. Due to the non-linearity of the phenomenon, a complete description of such variable is made infeasible on both the small scale and large spatial extents. Methods relying on Data Assimilation techniques, despite being the state-of-the-art for Numerical Weather Prediction, can not provide the reconstructions with a spatial resolution that can compete with satellite imagery. In this work we propose a framework based on Variational Data Assimilation and Deep Learning concepts. This framework is applied to recover rich-in-time, high-resolution information on sea surface wind speed. We design our experiments using synthetic wind data and different sampling schemes for high-resolution and low-resolution versions of original data to emulate the real-world scenario of spatio-temporally heterogeneous observations. Extensive numerical experiments are performed to assess systematically the impact of low and high-resolution wind fields and in-situ observations on the model reconstruction performance. We show that in-situ observations with richer temporal resolution represent an added value in terms of the model reconstruction performance. We show how a multi-modal approach, that explicitly informs the model about the heterogeneity of the available observations, can improve the reconstruction task by exploiting the complementary information in spatial and local point-wise data. To conclude, we propose an analysis to test the robustness of the chosen framework against phase delay and amplitude biases in low-resolution data and against interruptions of in-situ observations supply at evaluation time
Applying Pre-Trained Deep-Learning Model on Wrist Angel Data -- An Analysis Plan

We aim to investigate if we can improve predictions of stress caused by OCD symptoms using pre-trained models, and present our statistical analysis plan in this paper. With the methods presented in this plan, we aim to avoid bias from data knowledge and thereby strengthen our hypotheses and findings. The Wrist Angel study, which this statistical analysis plan concerns, contains data from nine participants, between 8 and 17 years old, diagnosed with obsessive-compulsive disorder (OCD). The data was obtained by an Empatica E4 wristband, which the participants wore during waking hours for 8 weeks. The purpose of the study is to assess the feasibility of predicting the in-the-wild OCD events captured during this period. In our analysis, we aim to investigate if we can improve predictions of stress caused by OCD symptoms, and to do this we have created a pre-trained model, trained on four open-source data for stress prediction. We intend to apply this pre-trained model to the Wrist Angel data by fine-tuning, thereby utilizing transfer learning. The pre-trained model is a convolutional neural network that uses blood volume pulse, heart rate, electrodermal activity, and skin temperature as time series windows to predict OCD events. Furthermore, using accelerometer data, another model filters physical activity to further improve performance, given that physical activity is physiologically similar to stress. By evaluating various ways of applying our model (fine-tuned, non-fine-tuned, pre-trained, non-pre-trained, and with or without activity classification), we contextualize the problem such that it can be assessed if transfer learning is a viable strategy in this domain.
Improving Cross-modal Alignment with Synthetic Pairs for Text-only Image Captioning

Although image captioning models have made significant advancements in recent years, the majority of them heavily depend on high-quality datasets containing paired images and texts which are costly to acquire. Previous works leverage the CLIP's cross-modal association ability for image captioning, relying solely on textual information under unsupervised settings. However, not only does a modality gap exist between CLIP text and image features, but a discrepancy also arises between training and inference due to the unavailability of real-world images, which hinders the cross-modal alignment in text-only captioning. This paper proposes a novel method to address these issues by incorporating synthetic image-text pairs. A pre-trained text-to-image model is deployed to obtain images that correspond to textual data, and the pseudo features of generated images are optimized toward the real ones in the CLIP embedding space. Furthermore, textual information is gathered to represent image features, resulting in the image features with various semantics and the bridged modality gap. To unify training and inference, synthetic image features would serve as the training prefix for the language decoder, while real images are used for inference. Additionally, salient objects in images are detected as assistance to enhance the learning of modality alignment. Experimental results demonstrate that our method obtains the state-of-the-art performance on benchmark datasets.
Spinel LiGa5O8 prospects as ultra-wideband-gap semiconductor: Band structure, optical properties, and doping

LiGa5O8 in the spinel type structure is investigated as a potential ultra-wideband-gap semiconductor. The band structure is determined using the quasiparticle self-consistent GW method, and the optical properties are calculated at the Bethe Salpeter Equation level including electron-hole interaction effects. The optical gap including exciton effects and an estimate of the zero-point motion electron phonon coupling renormalizations is estimated to be about 5.2±0.1 eV with an exciton binding energy of about 0.4 eV. Si doping as potential n-type dopant is investigated and found to be a promising shallow donor.
Self-organisation of auto-phoretic suspensions in confined shear flows

Janus phoretic particles exploit chemical energy stored in their environment to self-propel. These active particles modify and respond to their hydrodynamic and chemical environments, thus giving them a sensibility to external flows and other particles. Furthermore, experimental observations and analysis on biological or synthetic active suspensions indicate that hydro-chemical interparticle interactions lead to non-trivial collective behaviour (e.g., cluster formation of phoretic particles or bacterial swarming) and that the response of the suspensions to shear flows is non-trivial. In fact, it can lead to significant reductions in viscosity due to the energy conversion at microscopic scales. In this work, using simulations of a continuum kinetic model, we analyse the dynamics and response to shear and confinement of dilute suspensions of chemotactic phoretic particles that reorient and drift toward the chemical solutes released by their neighbours. We show that a 1D transient steady distribution driven by the effect of confinement is a common feature considered and analyse its stability for varying confinement strength and shear rate. In the second step, we consider and discuss, more specifically, the feedback effect on the flow by the particle and the resulting effective viscosity of the suspension.