Miracle-dz/UniFAR-FTUs huggingface dataset:
Rethinking Data Heterogeneity in Federated Learning: Introducing a New Notion and Standard Benchmarks

Though successful, federated learning (FL) presents new challenges for machine learning, especially when the issue of data heterogeneity, also known as Non-IID data, arises. To cope with the statistical heterogeneity, previous works incorporated a proximal term in local optimization or modified the model aggregation scheme at the server side or advocated clustered federated learning approaches, where the central server groups agent population into clusters with jointly trainable data distributions to take the advantage of a certain level of personalization. While effective, they lack a deep elaboration on what kind of data heterogeneity and how the data heterogeneity impacts the accuracy performance of the participating clients. In contrast to many of the prior FL approaches, we demonstrate not only the issue of data heterogeneity in current setups is not necessarily a problem but also in fact it can be beneficial for the FL participants. Our observations are intuitive: 1) Dissimilar labels of clients (label skew) are not necessarily considered data heterogeneity, and 2) the principal angle between the clients' data subspaces spanned by their corresponding principal vectors of data is a better estimate of the data heterogeneity.
Shuffled linear regression through graduated convex relaxation

The shuffled linear regression problem aims to recover linear relationships in datasets where the correspondence between input and output is unknown. This problem arises in a wide range of applications including survey data, in which one needs to decide whether the anonymity of the responses can be preserved while uncovering significant statistical connections. In this work, we propose a novel optimization algorithm for shuffled linear regression based on a posterior-maximizing objective function assuming Gaussian noise prior. We compare and contrast our approach with existing methods on synthetic and real data. We show that our approach performs competitively while achieving empirical running-time improvements. Furthermore, we demonstrate that our algorithm is able to utilize the side information in the form of seeds, which recently came to prominence in related problems.
THE RISE TIME OF NORMAL AND SUBLUMINOUS TYPE Ia SUPERNOVAE

We calculate the average stretch-corrected rise time of Type Ia supernovae (SNe Ia) in the Supernova Legacy Survey. We use the aggregate light curves of spectroscopic and photometrically identified SNe Ia to fit the rising part of the light curve with a simple quadratic model. We obtain a light curve shape corrected, i.e., stretch-corrected, fiducial rise time of 17.02+0.18 − 0.28 (stat) days. The measured rise time differs from an earlier finding by the SNLS (Conley et al.) due to the use of different SN Ia templates. We compare it to nearby samples using the same methods and find no evolution in the early part of the light curve of SNe Ia up to z = 1. We search for variations among different populations, particularly subluminous objects, by dividing the sample in stretch. Bright and slow decliners (s > 1.0) have consistent stretch-corrected rise times compared to fainter and faster decliners (0.8 < s ⩽ 1.0); they are shorter by 0.57+0.47 − 0.50 (stat) days. Subluminous SNe Ia (here defined as objects with s ⩽ 0.8), although less constrained, are also consistent, with a rise time of 18.03+0.81 − 1.37 (stat) days. We study several systematic biases and find that the use of different fiducial templates may affect the average rise time but not the intrinsic differences between populations. Based on our results, we estimate that subluminous SNe Ia are powered by 0.05–0.35 M☉ of 56Ni synthesized in the explosion. Our conclusions are the same for the single-stretch and two-stretch parameterizations of the light curve.
AudioGen: Textually Guided Audio Generation

We tackle the problem of generating audio samples conditioned on descriptive text captions. In this work, we propose AaudioGen, an auto-regressive generative model that generates audio samples conditioned on text inputs. AudioGen operates on a learnt discrete audio representation. The task of text-to-audio generation poses multiple challenges. Due to the way audio travels through a medium, differentiating ``objects'' can be a difficult task (e.g., separating multiple people simultaneously speaking). This is further complicated by real-world recording conditions (e.g., background noise, reverberation, etc.). Scarce text annotations impose another constraint, limiting the ability to scale models. Finally, modeling high-fidelity audio requires encoding audio at high sampling rate, leading to extremely long sequences. To alleviate the aforementioned challenges we propose an augmentation technique that mixes different audio samples, driving the model to internally learn to separate multiple sources. We curated 10 datasets containing different types of audio and text annotations to handle the scarcity of text-audio data points. For faster inference, we explore the use of multi-stream modeling, allowing the use of shorter sequences while maintaining a similar bitrate and perceptual quality. We apply classifier-free guidance to improve adherence to text. Comparing to the evaluated baselines, AudioGen outperforms over both objective and subjective metrics. Finally, we explore the ability of the proposed method to generate audio continuation conditionally and unconditionally. Samples: https://felixkreuk.github. io/audiogen
3D UX-Net: A Large Kernel Volumetric ConvNet Modernizing Hierarchical Transformer for Medical Image Segmentation

The recent 3D medical ViTs (e.g., SwinUNETR) achieve the state-of-the-art performances on several 3D volumetric data benchmarks, including 3D medical image segmentation. Hierarchical transformers (e.g., Swin Transformers) reintroduced several ConvNet priors and further enhanced the practical viability of adapting volumetric segmentation in 3D medical datasets. The effectiveness of hybrid approaches is largely credited to the large receptive field for non-local self-attention and the large number of model parameters. In this work, we propose a lightweight volumetric ConvNet, termed 3D UX-Net, which adapts the hierarchical transformer using ConvNet modules for robust volumetric segmentation. Specifically, we revisit volumetric depth-wise convolutions with large kernel size (e.g. starting from $7\times7\times7$) to enable the larger global receptive fields, inspired by Swin Transformer. We further substitute the multi-layer perceptron (MLP) in Swin Transformer blocks with pointwise depth convolutions and enhance model performances with fewer normalization and activation layers, thus reducing the number of model parameters. 3D UX-Net competes favorably with current SOTA transformers (e.g. SwinUNETR) using three challenging public datasets on volumetric brain and abdominal imaging: 1) MICCAI Challenge 2021 FLARE, 2) MICCAI Challenge 2021 FeTA, and 3) MICCAI Challenge 2022 AMOS. 3D UX-Net consistently outperforms SwinUNETR with improvement from 0.929 to 0.938 Dice (FLARE2021) and 0.867 to 0.874 Dice (Feta2021). We further evaluate the transfer learning capability of 3D UX-Net with AMOS2022 and demonstrates another improvement of $2.27\%$ Dice (from 0.880 to 0.900). The source code with our proposed model are available at https://github.com/MASILab/3DUX-Net.
CLUSTERING WITH MULTIVIEWPOINT-BASED SIMILARITY MEASURE USING HIERARCHICAL CLUSTERING

In data mining, clustering technique is an interesting and important technique. The main goal of the clustering is finding the similarity between the data points or similarity between the data within intrinsic data structure and grouping them the data into single groups (or) subgroups in clustering process. The existing system is mainly used for finding the next frequent item set using k-Means method, where this algorithm can reduce the overlapping between the documents in itemset. The result of clustering process is based on the order for choosing the item sets in the k-Means approach; it doesn't follow a sequential order when selecting clusters. This problem will lead to gain less optimal solution for clustering method. To resolve this problem, proposed system which is developing a novel hierarchal algorithm for document clustering produces superlative efficiency and performance. It is mainly focusing on making use of cluster overlapping phenomenon to design cluster merging criteria. Proposing a new way to compute the overlap rate in order to improve time efficiency and accuracy is mainly concentrated. Based on the Hierarchical Clustering Method, the usage of Expectation-Maximization (EM) algorithm in the Gaussian Mixture Model is to count the parameters and make the two sub-clusters combined when their overlap is the largest is reported.
Smallcap: Lightweight Image Captioning Prompted with Retrieval Augmentation

Recent advances in image captioning have focused on scaling the data and model size, substantially increasing the cost of pretraining and finetuning. As an alternative to large models, we present Smallcap, which generates a caption conditioned on an input image and related captions retrieved from a datastore. Our model is lightweight and fast to train, as the only learned parameters are in newly introduced cross-attention layers between a pre-trained CLIP encoder and GPT-2 decoder. Smallcap can transfer to new domains without additional finetuning and can exploit large-scale data in a training-free fashion since the contents of the datastore can be readily replaced. Our experiments show that Smallcap, trained only on COCO, has competitive performance on this benchmark, and also transfers to other domains without retraining, solely through retrieval from target-domain data. Further improvement is achieved through the training-free exploitation of diverse human-labeled and web data, which proves to be effective for a range of domains, including the nocaps benchmark, designed to test generalization to unseen visual concepts.11Code: https://github.com/RitaRamo/smallcap.
The worst approximable rational numbers

We classify and enumerate all rational numbers with approximation constant at least 13 using hyperbolic geometry. Rational numbers correspond to geodesics in the modular torus with both ends in the cusp, and the approximation constant measures how far they stay out of the cusp neighborhood in between. Compared to the original approach, the geometric point of view eliminates the need to discuss the intricate symbolic dynamics of continued fraction representations, and it clariﬁes the distinction between the two types of worst approximable rationals: (1) There is a plane forest of Markov fractions whose denominators are Markov numbers. They correspond to simple geodesics in the modular torus with both ends in the cusp. (2) For each Markov fraction, there are two inﬁnite sequences of companions , which correspond to non-simple geodesics with both ends in the cusp that do not intersect a pair of disjoint simple geodesics, one with both ends in the cusp and one closed. MSC (2010). 10F20, 30F60
Rational Social Learning by Random Sampling

This paper explores rational social learning in which everyone only sees unordered random samples from the action history. In this model, herds need not occur when the distant past can be sampled. If private signal strengths are unbounded and the past is not over-sampled -- not forever affected by any individual -- there is complete learning and a correct proportionate herd. With recursive sampling, welfare almost surely converges under the new proviso that the recent past is not over-sampled. In this case, there is almost surely complete learning with unbounded beliefs and unit sample sizes. The sampling noise in this Polya urn model induces a path-dependent structure, so that re-running the model with identical signals generally produces different outcomes.
Local power coupling as a predictor of high-gradient breakdown performance

A novel quantity for predicting the high-gradient performance of radio frequency accelerating structures is presented. The quantity is motivated, derived and compared with earlier high-gradient limits and experiments. This new method models a nascent RF breakdown as a current-carrying antenna and calculates the coupling of the antenna to an RF power source. With the help of an electron emission model to describe a nascent breakdown, the antenna model describes how a breakdown modiﬁes the local surface electric ﬁeld before it fully develops in any given structure geometry. For the structure geometries that this method was applied to, it was found that the calculated breakdown-loaded electric ﬁeld was well-correlated with observed spatial breakdown dis-tributions, and gave consistent values for the maximum breakdown-limited accelerating gradient between diﬀerent geometries.
Physical Adversarial Attack meets Computer Vision: A Decade Survey

Despite the impressive achievements of Deep Neural Networks (DNNs) in computer vision, their vulnerability to adversarial attacks remains a critical concern. Extensive research has demonstrated that incorporating sophisticated perturbations into input images can lead to a catastrophic degradation in DNNs' performance. This perplexing phenomenon not only exists in the digital space but also in the physical world. Consequently, it becomes imperative to evaluate the security of DNNs-based systems to ensure their safe deployment in real-world scenarios, particularly in security-sensitive applications. To facilitate a profound understanding of this topic, this paper presents a comprehensive overview of physical adversarial attacks. Firstly, we distill four general steps for launching physical adversarial attacks. Building upon this foundation, we uncover the pervasive role of artifacts carrying adversarial perturbations in the physical world. These artifacts influence each step. To denote them, we introduce a new term: adversarial medium. Then, we take the first step to systematically evaluate the performance of physical adversarial attacks, taking the adversarial medium as a first attempt. Our proposed evaluation metric, hiPAA, comprises six perspectives: Effectiveness, Stealthiness, Robustness, Practicability, Aesthetics, and Economics. We also provide comparative results across task categories, together with insightful observations and suggestions for future research directions.
Relative representations enable zero-shot latent space communication

Neural networks embed the geometric structure of a data manifold lying in a high-dimensional space into latent representations. Ideally, the distribution of the data points in the latent space should depend only on the task, the data, the loss, and other architecture-specific constraints. However, factors such as the random weights initialization, training hyperparameters, or other sources of randomness in the training phase may induce incoherent latent spaces that hinder any form of reuse. Nevertheless, we empirically observe that, under the same data and modeling choices, the angles between the encodings within distinct latent spaces do not change. In this work, we propose the latent similarity between each sample and a fixed set of anchors as an alternative data representation, demonstrating that it can enforce the desired invariances without any additional training. We show how neural architectures can leverage these relative representations to guarantee, in practice, invariance to latent isometries and rescalings, effectively enabling latent space communication: from zero-shot model stitching to latent space comparison between diverse settings. We extensively validate the generalization capability of our approach on different datasets, spanning various modalities (images, text, graphs), tasks (e.g., classification, reconstruction) and architectures (e.g., CNNs, GCNs, transformers).
DeLag: Using Multi-Objective Optimization to Enhance the Detection of Latency Degradation Patterns in Service-Based Systems

Performance debugging in production is a fundamental activity in modern service-based systems. The diagnosis of performance issues is often time-consuming, since it requires thorough inspection of large volumes of traces and performance indices. In this paper we present DeLag, a novel automated search-based approach for diagnosing performance issues in service-based systems. DeLag identifies subsets of requests that show, in the combination of their Remote Procedure Call execution times, symptoms of potentially relevant performance issues. We call such symptoms <italic>Latency Degradation Patterns</italic>. DeLag simultaneously searches for multiple <italic>latency degradation patterns</italic> while optimizing precision, recall and latency dissimilarity. Experimentation on 700 datasets of requests generated from two microservice-based systems shows that our approach provides better and more stable effectiveness than three state-of-the-art approaches and general purpose machine learning clustering algorithms. DeLag is more effective than all baseline techniques in at least one case study (with <inline-formula><tex-math notation="LaTeX">$p\leq 0.05$</tex-math><alternatives><mml:math><mml:mrow><mml:mi>p</mml:mi><mml:mo>≤</mml:mo><mml:mn>0</mml:mn><mml:mo>.</mml:mo><mml:mn>05</mml:mn></mml:mrow></mml:math><inline-graphic xlink:href="traini-ieq1-3266041.gif"/></alternatives></inline-formula> and non-negligible effect size). Moreover, DeLag outperforms in terms of efficiency the second and the third most effective baseline techniques on the largest datasets used in our evaluation (up to 22%).
Improved Group Robustness via Classifier Retraining on Independent Splits

Deep neural networks trained by minimizing the average risk can achieve strong average performance. Still, their performance for a subgroup may degrade if the subgroup is underrepresented in the overall data population. Group distributionally robust optimization (Sagawa et al., 2020a), or group DRO in short, is a widely used baseline for learning models with strong worst-group performance. We note that this method requires group labels for every example at training time and can overfit to small groups, requiring strong regularization. Given a limited amount of group labels at training time, Just Train Twice (Liu et al., 2021), or JTT in short, is a two-stage method that infers a pseudo group label for every unlabeled example first, then applies group DRO based on the inferred group labels. The inference process is also sensitive to overfitting, sometimes involving additional hyperparameters. This paper designs a simple method based on the idea of classifier retraining on independent splits of the training data. We find that using a novel sample-splitting procedure achieves robust worst-group performance in the fine-tuning step. When evaluated on benchmark image and text classification tasks, our approach consistently performs favorably to group DRO, JTT, and other strong baselines when either group labels are available during training or are only given in validation sets. Importantly, our method only relies on a single hyperparameter, which adjusts the fraction of labels used for training feature extractors vs. training classification layers. We justify the rationale of our splitting scheme with a generalization-bound analysis of the worst-group loss.
COSMICFLOWS-2: TYPE Ia SUPERNOVA CALIBRATION AND H0

The construction of the Cosmicflows-2 compendium of distances involves the merging of distance measures contributed by the following methods: (Cepheid) period–luminosity, tip of the red giant branch (TRGB), surface brightness fluctuation (SBF), luminosity–linewidth (TF), fundamental plane (FP), and Type Ia supernova (SNIa). The method involving SNIa is at the top of an interconnected ladder, providing accurate distances to well beyond the expected range of distortions to Hubble flow from peculiar motions. In this paper, the SNIa scale is anchored by 36 TF spirals with Cepheid or TRGB distances, 56 SNIa hosts with TF distances, and 61 groups or clusters hosting SNIa with Cepheid, SBF, TF, or FP distances. With the SNIa scale zero-point set, a value of the Hubble constant is evaluated over a range of redshifts 0.03 < z < 0.5, assuming a cosmological model with Ωm = 0.27 and ΩΛ = 0.73. The value determined for the Hubble constant is H0 = 75.9 ± 3.8 km s−1 Mpc−1.
Dissipation in a rotating frame: Master equation, effective temperature, and Lamb shift

Motivated by recent realizations of microwave-driven nonlinear resonators in superconducting circuits, the impact of environmental degrees of freedom is analyzed as seen from a rotating frame. A system plus reservoir model is applied to consistently derive in the weak coupling limit the master equation for the reduced density in the moving frame and near the first bifurcation threshold. The concept of an effective temperature is introduced to analyze to what extent a detailed balance relation exists. Explicit expressions are also found for the Lamb-shift. Results for ohmic baths are in agreement with experimental findings, while for structured environments population inversion is predicted that may qualitatively explain recent observations.
Chaotic star formation and the alignment of stellar rotation with disc and planetary orbital axes

We investigate the evolution of the relative angle between the stellar rotation axis and the circumstellar disc axis of a star that forms in a stellar cluster from the collapse of a turbulent molecular cloud. This is an inherently chaotic environment with variable accretion, both in terms of rate and the angular momentum of the material, and dynamical interactions between stars. We find that the final stellar rotation axis and disc spin axis can be strongly misaligned, but this occurs primarily when the disc is truncated by a dynamical encounter so that the final disc rotation axis depends simply on what fell in last. This may lead to planetary systems with orbits that are misaligned with the stellar rotation axis, but only if the final disc contains enough mass to form planets. We also investigate the time variability of the inner-disc spin axis, which is likely to determine the direction of a protostellar jet. We find that the jet direction varies more strongly for lighter discs, such as those that have been truncated by dynamical interactions or have suffered a period of rapid accretion. Finally, we note that variability of the angular momentum of the material accreted by a star implies that the internal velocity field of such stars may be more complicated than that of aligned differential rotation.
A long‐term follow‐up of early breast cancer patients treated with photobiomodulation during conventional fractionation radiotherapy in the prevention of acute radiation dermatitis

The evidence demonstrating the efficacy of photobiomodulation (PBM) therapy for preventing and managing acute radiation dermatitis (ARD) is growing steadily. The question that arises from many clinicians is, if PBM is safe for oncologic patients. This study aimed to evaluate the disease‐free survival (DFS), cancer‐free survival (CFS), and overall survival (OS) of breast cancer patients treated with PBM for ARD.
Area–angle variables for general relativity

We introduce a modified Regge calculus for general relativity on a triangulated four-dimensional Riemannian manifold where the fundamental variables are areas and a certain class of angles. These variables satisfy constraints which are local in the triangulation. We expect the formulation to have applications to classical discrete gravity and non-perturbative approaches to quantum gravity.