Miracle-dz/UniFAR-FTUs huggingface dataset:
Rethinking Clustering-Based Pseudo-Labeling for Unsupervised Meta-Learning

The pioneering method for unsupervised meta-learning, CACTUs, is a clustering-based approach with pseudo-labeling. This approach is model-agnostic and can be combined with supervised algorithms to learn from unlabeled data. However, it often suffers from label inconsistency or limited diversity, which leads to poor performance. In this work, we prove that the core reason for this is lack of a clustering-friendly property in the embedding space. We address this by minimizing the inter- to intra-class similarity ratio to provide clustering-friendly embedding features, and validate our approach through comprehensive experiments. Note that, despite only utilizing a simple clustering algorithm (k-means) in our embedding space to obtain the pseudo-labels, we achieve significant improvement. Moreover, we adopt a progressive evaluation mechanism to obtain more diverse samples in order to further alleviate the limited diversity problem. Finally, our approach is also model-agnostic and can easily be integrated into existing supervised methods. To demonstrate its generalization ability, we integrate it into two representative algorithms: MAML and EP. The results on three main few-shot benchmarks clearly show that the proposed method achieves significant improvement compared to state-of-the-art models. Notably, our approach also outperforms the corresponding supervised method in two tasks.
A Review of Modern Approaches for Coronary Angiography Imaging Analysis

—Coronary Heart Disease (CHD) is a leading cause of death in the modern world. The development of modern analytical tools for diagnostics and treatment of CHD is receiv- ing substantial attention from the scientiﬁc community. Deep learning-based algorithms, such as segmentation networks and detectors, play an important role in assisting medical professionals by providing timely analysis of a patient’s angiograms. This paper focuses on X-Ray Coronary Angiography (XCA), which is considered to be a “gold standard” in the diagnosis and treatment of CHD. First, we describe publicly available datasets of XCA images. Then, classical and modern techniques of image preprocessing are reviewed. In addition, common frame selection techniques are discussed, which are an important factor of input quality and thus model performance. In the following two chapters we discuss modern vessel segmentation and stenosis detection networks and, ﬁnally, open problems and current limitations of the current state-of-the-art.
Boosting Quantum Fidelity with an Ordered Diverse Ensemble of Clifford Canary Circuits

—On today’s noisy imperfect quantum devices, execu- tion ﬁdelity tends to collapse dramatically for most applications beyond a handful of qubits. It is therefore imperative to employ novel techniques that can boost quantum ﬁdelity in new ways. This paper aims to boost quantum ﬁdelity with Clifford canary circuits by proposing Quancorde: Quantum Canary Ordered D iverse Ensembles , a fundamentally new approach to identifying the correct outcomes of extremely low-ﬁdelity quantum applications. It is based on the key idea of diversity in quantum devices - variations in noise sources, make each (portion of a) device unique, and therefore, their impact on an application’s ﬁdelity, also unique.Quancorde utilizes Clifford canary circuits (which are classically simulable, but also resemble the target application structure and thus suffer similar structural noise impact) to order a diverse ensemble of devices or qubits/mappings approximately along the direction of increasing ﬁdelity of the target application. Quancorde then estimates the correlation of the ensemble-wide probabilities of each output string of the application, with the canary ensemble ordering, and uses this correlation to weight the application’s noisy probability distribution. The correct application outcomes are expected to have higher correlation with the canary ensemble order, and thus their probabilities are boosted in this process. Doing so, Quancorde improves the ﬁdelity of evaluated quantum applications by a mean of 8.9x/4.2x (wrt. different baselines) and up to a maximum of 34x.
Automated Quality Controlled Analysis of 2D Phase Contrast Cardiovascular Magnetic Resonance Imaging

Flow analysis carried out using phase contrast cardiac magnetic resonance imaging (PC-CMR) enables the quantification of important parameters that are used in the assessment of cardiovascular function. An essential part of this analysis is the identification of the correct CMR views and quality control (QC) to detect artefacts that could affect the flow quantification. We propose a novel deep learning based framework for the fully-automated analysis of flow from full CMR scans that first carries out these view selection and QC steps using two sequential convolutional neural networks, followed by automatic aorta and pulmonary artery segmentation to enable the quantification of key flow parameters. Accuracy values of 0.958 and 0.914 were obtained for view classification and QC, respectively. For segmentation, Dice scores were $>$0.969 and the Bland-Altman plots indicated excellent agreement between manual and automatic peak flow values. In addition, we tested our pipeline on an external validation data set, with results indicating good robustness of the pipeline. This work was carried out using multivendor clinical data consisting of 986 cases, indicating the potential for the use of this pipeline in a clinical setting.
The Space Density of Intermediate-redshift, Extremely Compact, Massive Starburst Galaxies

We present a measurement of the intrinsic space density of intermediate-redshift (z ∼ 0.5), massive (M * ∼ 1011 M ⊙), compact (R e ∼ 100 pc) starburst (ΣSFR ∼ 1000 M ⊙ yr−1 kpc−1) galaxies with tidal features indicative of them having undergone recent major mergers. A subset of them host kiloparsec-scale, > 1000 km s−1 outflows and have little indication of AGN activity, suggesting that extreme star formation can be a primary driver of large-scale feedback. The aim for this paper is to calculate their space density so we can place them in a better cosmological context. We do this by empirically modeling the stellar populations of massive, compact starburst galaxies. We determine the average timescale on which galaxies that have recently undergone an extreme nuclear starburst would be targeted and included in our spectroscopically selected sample. We find that massive, compact starburst galaxies targeted by our criteria would be selectable for ∼148−24+27 Myr and have an intrinsic space density nCS∼(1.1−0.3+0.5)×10−6Mpc−3 . This space density is broadly consistent with our z ∼ 0.5 compact starbursts being the most extremely compact and star-forming low-redshift analogs of the compact star-forming galaxies in the early universe, as well as them being the progenitors to a fraction of intermediate-redshift, post-starburst, and compact quiescent galaxies.
A Multi-scale Graph Signature for Persistence Diagrams based on Return Probabilities of Random Walks

Persistence diagrams (PDs), often characterized as sets of death and birth of homology class, have been known for providing a topological representation of a graph structure, which is often useful in machine learning tasks. Prior works rely on a single graph signature to construct PDs. In this paper, we explore the use of a family of multi-scale graph signatures to enhance the robustness of topological features. We propose a deep learning architecture to handle this set input. Experiments on benchmark graph classification datasets demonstrate that our proposed architecture outperforms other persistent homology-based methods and achieves competitive performance compared to state-of-the-art methods using graph neural networks. In addition, our approach can be easily applied to large size of input graphs as it does not suffer from limited scalability which can be an issue for graph kernel methods.
A Survey on Physical Adversarial Attack in Computer Vision

Over the past decade, deep learning has revolutionized conventional tasks that rely on hand-craft feature extraction with its strong feature learning capability, leading to substantial enhancements in traditional tasks. However, deep neural networks (DNNs) have been demonstrated to be vulnerable to adversarial examples crafted by malicious tiny noise, which is imperceptible to human observers but can make DNNs output the wrong result. Existing adversarial attacks can be categorized into digital and physical adversarial attacks. The former is designed to pursue strong attack performance in lab environments while hardly remaining effective when applied to the physical world. In contrast, the latter focus on developing physical deployable attacks, thus exhibiting more robustness in complex physical environmental conditions. Recently, with the increasing deployment of the DNN-based system in the real world, strengthening the robustness of these systems is an emergency, while exploring physical adversarial attacks exhaustively is the precondition. To this end, this paper reviews the evolution of physical adversarial attacks against DNN-based computer vision tasks, expecting to provide beneficial information for developing stronger physical adversarial attacks. Specifically, we first proposed a taxonomy to categorize the current physical adversarial attacks and grouped them. Then, we discuss the existing physical attacks and focus on the technique for improving the robustness of physical attacks under complex physical environmental conditions. Finally, we discuss the issues of the current physical adversarial attacks to be solved and give promising directions.
On Non-stochastic Approximate Information States for Uncertain Systems with Additive Costs

—In this paper, we consider the problem of opti- mizing the worst-case behavior of a partially observed system. All uncontrolled disturbances of the system are modeled as non-stochastic uncertain variables taking values in ﬁnite sets. Using the theory of cost distributions, we present a dynamic program (DP) to derive a control strategy that minimizes the maximum possible total cost over a ﬁnite-time horizon. We also present a general deﬁnition for information states which can improve computational tractability of the DP without loss of optimality. We show that many information states constructed in previous research efforts are special cases of our general deﬁnition. Finally, we present a deﬁnition for approximate information states and an approximate DP that can further improve computational tractability by conceding a bounded performance loss.
Linear estimation of average global effects

We study the problem of estimating the average causal effect of treating every member of a population, as opposed to none, using an experiment that treats only some. This is the policy-relevant estimand when deciding whether to scale up an intervention based on the results of an RCT, for example, but differs from the usual average treatment effect in the presence of spillovers. We consider both estimation and experimental design given a bound (parametrized by $\eta>0$) on the rate at which spillovers decay with the ``distance'' between units, defined in a generalized way to encompass spatial and quasi-spatial settings, e.g. where the economically relevant concept of distance is a gravity equation. Over all estimators linear in the outcomes and all cluster-randomized designs the optimal geometric rate of convergence is $n^{-\frac{1}{2+\frac{1}{\eta}}}$, and this rate can be achieved using a generalized ``Scaling Clusters'' design that we provide. We then introduce the additional assumption, implicit in the OLS estimators used in recent applied studies, that potential outcomes are linear in population treatment assignments. These estimators are inconsistent for our estimand, but a refined OLS estimator is consistent and rate optimal, and performs better than IPW estimators when clusters must be small. Its finite-sample performance can be improved by incorporating prior information about the structure of spillovers. As a robust alternative to the linear approach we also provide a method to select estimator-design pairs that minimize a notion of worst-case risk when the data generating process is unknown. Finally, we provide asymptotically valid inference methods.
Prompt-driven efficient Open-set Semi-supervised Learning

Open-set semi-supervised learning (OSSL) has attracted growing interest, which investigates a more practical scenario where out-of-distribution (OOD) samples are only contained in unlabeled data. Existing OSSL methods like OpenMatch learn an OOD detector to identify outliers, which often update all modal parameters (i.e., full fine-tuning) to propagate class information from labeled data to unlabeled ones. Currently, prompt learning has been developed to bridge gaps between pre-training and fine-tuning, which shows higher computational efficiency in several downstream tasks. In this paper, we propose a prompt-driven efficient OSSL framework, called OpenPrompt, which can propagate class information from labeled to unlabeled data with only a small number of trainable parameters. We propose a prompt-driven joint space learning mechanism to detect OOD data by maximizing the distribution gap between ID and OOD samples in unlabeled data, thereby our method enables the outliers to be detected in a new way. The experimental results on three public datasets show that OpenPrompt outperforms state-of-the-art methods with less than 1% of trainable parameters. More importantly, OpenPrompt achieves a 4% improvement in terms of AUROC on outlier detection over a fully supervised model on CIFAR10.
Towards Multilingual Transitivity and Bidirectional Multilingual Agreement for Multilingual Document-level Machine Translation

Multilingual machine translation has been proved an effective strategy to support translation between multiple languages with a single model. However, most studies focus on multilingual sentence translation without considering generating long documents across different languages, which requires an understanding of multilingual context dependency and is typically harder. In this paper, we ﬁrst spot that naively incorporating auxiliary multilingual data either auxiliary-target or source-auxiliary 1 brings no improvement to the source-target language pair in our interest. Motivated by this observation, we propose a novel framework called M ultilingual Trans itivity (MTrans) to ﬁnd an implicit optimal route via source-auxiliary-target within the multilingual model. 2 To encourage MTrans, we propose a novel method called T riplet P arallel D ata (TPD), which uses parallel triplets that contain (source-auxiliary, auxiliary-target, and source-target) for training. The auxiliary language then serves as a pivot and automatically facilitates the implicit information transition ﬂow which is easier to translate. We further propose a novel framework called Bi directional M ultilingual Agree ment (Bi-MAgree) that encourages the bidirectional agreement be-tween different languages. To encourage Bi-MAgree, we pro-pose a novel method called M ultilingual K ullback– L eibler Divergence (MKL) that forces the output distribution of the inputs with the same meaning but in different languages to be consistent with each other. The experimental results indicate that our methods bring consistent improvements over strong baselines on three document translation tasks: IWSLT2015 Zh-En, De-En, and Vi-En. Our analysis validates the usefulness and existence of MTrans and Bi-MAgree, and our frameworks and methods are effective on synthetic auxiliary data.
Changes in number and intensity of world-wide tropical cyclones

Bayesian statistical models were developed for the number of tropical cyclones and the rate at which these cyclones became hurricanes in the North Atlantic, North and South Indian, and East and West Pacific Oceans. We find that there is small probability that the number of cyclones has increased in the past thirty years. The rate at which these storms become hurricanes appears to be constant. The rate at which hurricanes evolve into category 4 and higher major storms does appear to have increased. We also investigate storm intensity by measuring the distribution of individual storm lifetime in days, storm track length, and Emanuel's power dissiptation index. We find little evidence that, overall, the mean of the distribution of individual storm intensity is changing through time, but the variability of the distribution has increased. The cold tongue index and the North Atlantic oscillation index were found to be strongly associated with storm quality in the Western, and to a smaller extent, the Eastern Pacific oceans. The North Atlantic oscillation index was strongly associated with the increase in the rate of strong storms evolving.
Exploring the Relationship Between Architectural Design and Adversarially Robust Generalization

Adversarial training has been demonstrated to be one of the most effective remedies for defending adversarial examples, yet it often suffers from the huge robustness generalization gap on unseen testing adversaries, deemed as the adversarially robust generalization problem. Despite the preliminary understandings devoted to adversarially robust generalization, little is known from the architectural perspective. To bridge the gap, this paper for the first time systematically investigated the relationship between adversarially robust generalization and architectural design. In particular, we comprehensively evaluated 20 most representative adversarially trained architectures on ImageNette and CIFAR-10 datasets towards multiple $\ell_{p}$-norm adversarial attacks. Based on the extensive experiments, we found that, under aligned settings, Vision Transformers (e.g., PVT, CoAtNet) often yield better adversarially robust generalization while CNNs tend to overfit on specific attacks and fail to generalize on multiple adversaries. To better understand the nature behind it, we conduct theoretical analysis via the lens of Rademacher complexity. We revealed the fact that the higher weight sparsity contributes significantly towards the better adversarially robust generalization of Transformers, which can be often achieved by the specially-designed attention blocks. We hope our paper could help to better understand the mechanism for designing robust DNNs. Our model weights can be found at http://robust.art.
Primordial black hole formation in Starobinsky's linear potential model

We study the power spectrum of the comoving curvature perturbation ℛ in the model that glues two linear potentials of different slopes, originally proposed by Starobinsky. We find that the enhanced power spectrum reaches its maximum at the wavenumber which is π times the junction scale. The peak is ∼ 2.61 times larger than the ultraviolet plateau. We also show that its near-peak behavior can be well approximated by a constant-roll model, once we define the effective ultra-slow-roll e-folding number appropriately by considering the contribution from non-single-clock phase only. Such an abrupt transition to non-attractor phase can leave some interesting characteristic features in the energy spectrum of the scalar-induced gravitational waves, which are detectable in the space-borne interferometers if the primordial black holes generated at such a high peak are all the dark matter.
Isoperimetric inequalities for Neumann eigenvalues on bounded domains in rank-1 symmetric spaces

In this paper, we prove sharp isoperimetric inequalities for lower order eigenvalues of Neumann Laplacian on bounded domains in both compact and noncompact rank-1 symmetric spaces. Our results generalize the work of Wang and Xia for bounded domains in the hyperbolic space [13], and Szeg\"o-Weinberger inequality in rank-1 symmetric spaces obtained by Aithal and Santhanam [1].
System-environment correlations for dephasing two-qubit states coupled to thermal baths

Based on the exact dynamics of a two-qubit system and environment, we investigate system-environment (SE) quantum and classical correlations. The coupling is chosen to represent a dephasing channel for one of the qubits and the environment is a proper thermal bath. First we discuss the general issue of dilation for qubit phase damping. Based on the usual thermal bath of harmonic oscillators, we derive criteria of separability and entanglement between an initial $X$ state and the environment. Applying these criteria to initial Werner states, we find that entanglement between the system and environment is built up in time for temperatures below a certain critical temperature $T_{\mathrm{crit}}$. On the other hand, the total state remains separable during those short times that are relevant for decoherence and loss of entanglement in the two-qubit state. Close to $T_{\mathrm{crit}}$ the SE correlations oscillate between separable and entangled. Even though these oscillations are also observed in the entanglement between the two qubits, no simple relation between the loss of entanglement in the two-qubit system and the build-up of entanglement between the system and environment is found.
Zero-Shot 3D Drug Design by Sketching and Generating

Drug design is a crucial step in the drug discovery cycle. Recently, various deep learning-based methods design drugs by generating novel molecules from scratch, avoiding traversing large-scale drug libraries. However, they depend on scarce experimental data or time-consuming docking simulation, leading to overfitting issues with limited training data and slow generation speed. In this study, we propose the zero-shot drug design method DESERT (Drug dEsign by SkEtching and geneRaTing). Specifically, DESERT splits the design process into two stages: sketching and generating, and bridges them with the molecular shape. The two-stage fashion enables our method to utilize the large-scale molecular database to reduce the need for experimental data and docking simulation. Experiments show that DESERT achieves a new state-of-the-art at a fast speed.
Adaptive Out-Orientations with Applications

We give improved algorithms for maintaining edge-orientations of a fully-dynamic graph, such that the out-degree of each vertex is bounded. On one hand, we show how to orient the edges such that the out-degree of each vertex is proportional to the arboricity $\alpha$ of the graph, in, either, an amortised update time of $O(\log^2 n \log \alpha)$, or a worst-case update time of $O(\log^3 n \log \alpha)$. On the other hand, motivated by applications including dynamic maximal matching, we obtain a different trade-off, namely either $O(\log n \log \alpha)$, amortised, or $O(\log ^2 n \log \alpha)$, worst-case time, for the problem of maintaining an edge-orientation with at most $O(\alpha + \log n)$ out-edges per vertex. Since our algorithms have update times with worst-case guarantees, the number of changes to the solution (i.e. the recourse) is naturally limited. Our algorithms adapt to the current arboricity of the graph, and yield improvements over previous work: Firstly, we obtain an $O(\varepsilon^{-6}\log^3 n \log \rho)$ worst-case update time algorithm for maintaining a $(1+\varepsilon)$ approximation of the maximum subgraph density, $\rho$. Secondly, we obtain an $O(\varepsilon^{-6}\log^3 n \log \alpha)$ worst-case update time algorithm for maintaining a $(1 + \varepsilon) \cdot OPT + 2$ approximation of the optimal out-orientation of a graph with adaptive arboricity $\alpha$. This yields the first worst-case polylogarithmic dynamic algorithm for decomposing into $O(\alpha)$ forests.Thirdly, we obtain arboricity-adaptive fully-dynamic deterministic algorithms for a variety, of problems including maximal matching, $\Delta+1$ coloring, and matrix vector multiplication. All update times are worst-case $O(\alpha+\log^2n \log \alpha)$, where $\alpha$ is the current arboricity of the graph.