Miracle-dz/UniFAR-FTUs huggingface dataset:
Large-scale pretraining on pathological images for fine-tuning of small pathological benchmarks

Pretraining a deep learning model on large image datasets is a standard step before fine-tuning the model on small targeted datasets. The large dataset is usually general images (e.g. imagenet2012) while the small dataset can be specialized datasets that have different distributions from the large dataset. However, this 'large-to-small' strategy is not well-validated when the large dataset is specialized and has a similar distribution to small datasets. We newly compiled three hematoxylin and eosin-stained image datasets, one large (PTCGA200) and two magnification-adjusted small datasets (PCam200 and segPANDA200). Major deep learning models were trained with supervised and self-supervised learning methods and fine-tuned on the small datasets for tumor classification and tissue segmentation benchmarks. ResNet50 pretrained with MoCov2, SimCLR, and BYOL on PTCGA200 was better than imagenet2012 pretraining when fine-tuned on PTCGA200 (accuracy of 83.94%, 86.41%, 84.91%, and 82.72%, respectively). ResNet50 pre-trained on PTCGA200 with MoCov2 exceeded the COCOtrain2017-pretrained baseline and was the best in ResNet50 for the tissue segmentation benchmark (mIoU of 63.53% and 63.22%). We found re-training imagenet-pretrained models (ResNet50, BiT-M-R50x1, and ViT-S/16) on PTCGA200 improved downstream benchmarks.
Simulation-based inference for model parameterization on analog neuromorphic hardware

The BrainScaleS-2 (BSS-2) system implements physical models of neurons as well as synapses and aims for an energy-efficient and fast emulation of biological neurons. When replicating neuroscientific experiments on BSS-2, a major challenge is finding suitable model parameters. This study investigates the suitability of the sequential neural posterior estimation (SNPE) algorithm for parameterizing a multi-compartmental neuron model emulated on the BSS-2 analog neuromorphic system. The SNPE algorithm belongs to the class of simulation-based inference methods and estimates the posterior distribution of the model parameters; access to the posterior allows quantifying the confidence in parameter estimations and unveiling correlation between model parameters. For our multi-compartmental model, we show that the approximated posterior agrees with experimental observations and that the identified correlation between parameters fits theoretical expectations. Furthermore, as already shown for software simulations, the algorithm can deal with high-dimensional observations and parameter spaces when the data is generated by emulations on BSS-2. These results suggest that the SNPE algorithm is a promising approach for automating the parameterization and the analyzation of complex models, especially when dealing with characteristic properties of analog neuromorphic substrates, such as trial-to-trial variations or limited parameter ranges.
Solving Regularized Exp, Cosh and Sinh Regression Problems

In modern machine learning, attention computation is a fundamental task for training large language models such as Transformer, GPT-4 and ChatGPT. In this work, we study exponential regression problem which is inspired by the softmax/exp unit in the attention mechanism in large language models. The standard exponential regression is non-convex. We study the regularization version of exponential regression problem which is a convex problem. We use approximate newton method to solve in input sparsity time. Formally, in this problem, one is given matrix $A \in \mathbb{R}^{n \times d}$, $b \in \mathbb{R}^n$, $w \in \mathbb{R}^n$ and any of functions $\exp, \cosh$ and $\sinh$ denoted as $f$. The goal is to find the optimal $x$ that minimize $ 0.5 \| f(Ax) - b \|_2^2 + 0.5 \| \mathrm{diag}(w) A x \|_2^2$. The straightforward method is to use the naive Newton's method. Let $\mathrm{nnz}(A)$ denote the number of non-zeros entries in matrix $A$. Let $\omega$ denote the exponent of matrix multiplication. Currently, $\omega \approx 2.373$. Let $\epsilon$ denote the accuracy error. In this paper, we make use of the input sparsity and purpose an algorithm that use $\log ( \|x_0 - x^*\|_2 / \epsilon)$ iterations and $\widetilde{O}(\mathrm{nnz}(A) + d^{\omega} )$ per iteration time to solve the problem.
Linking Unserved Energy to Weather Regimes

The integration of renewable energy sources into power systems is expected to increase significantly in the coming decades. This can result in critical situations related to the strong variability in space and time of weather patterns. During these critical situations the power system experiences a structural shortage of energy across multiple time steps and regions, leading to Energy Not Served (ENS) events. Our research explores the relationship between six weather regimes that describe the large scale atmospheric flow and ENS events in Europe by simulating future power systems. Our results indicate that most regions have a specific weather regime that leads to the highest number of ENS events. However, ENS events can still occur during any weather regime, but with a lower probability. In particular, our findings show that ENS events in western and central European countries often coincide with either the positive Scandinavian Blocking (SB+), characterised by cold air penetrating Europe under calm weather conditions from north-eastern regions, or North Atlantic Oscillation (NAO+) weather regime, characterised by westerly flow and cold air in the southern half of Europe. Additionally, we found that the relative impact of one of these regimes reaches a peak 10 days before ENS events in these countries. In Scandinavian and Baltic countries, on the other hand, our results indicate that the relative prevalence of the negative Atlantic Ridge (AR-) weather regime is higher during and leading up to the ENS event.
Generalized Honeycomb-structured Materials in the Subwavelength Regime

Honeycomb structures lead to conically degenerate points on the dispersion surfaces. These spectral points, termed as Dirac points, are responsible for various topological phenomena. In this paper, we investigate the generalized honeycomb-structured materials, which have six inclusions in a hexagonal cell. We obtain the asymptotic band structures and corresponding eigenstates in the subwavelength regime using the layer potential theory. Specifically, we rigorously prove the existence of the double Dirac cones lying on the 2nd-5th bands when the six inclusions satisfy an additional symmetry. This type of inclusions will be referred to as super honeycomb-structured inclusions. Two distinct deformations breaking the additional symmetry, contraction and dilation, are further discussed. We prove that the double Dirac cone disappears, and a local spectral gap opens. The corresponding eigenstates are also obtained to show the topological differences between these two deformations. Direct numerical simulations using finite element methods agree well with our analysis.
Rethinking matching-based few-shot action recognition

Few-shot action recognition, i.e. recognizing new action classes given only a few examples, benefits from incorporating temporal information. Prior work either encodes such information in the representation itself and learns classifiers at test time, or obtains frame-level features and performs pairwise temporal matching. We first evaluate a number of matching-based approaches using features from spatio-temporal backbones, a comparison missing from the literature, and show that the gap in performance between simple baselines and more complicated methods is significantly reduced. Inspired by this, we propose Chamfer++, a non-temporal matching function that achieves state-of-the-art results in few-shot action recognition. We show that, when starting from temporal features, our parameter-free and interpretable approach can outperform all other matching-based and classifier methods for one-shot action recognition on three common datasets without using temporal information in the matching stage. Project page: https://jbertrand89.github. io/matching-based-fsar
Ultrasound-mediated transport of nanoparticles and the influence of particle density

A significant barrier to successful drug delivery in cancer therapy is the limited penetration of nanoscale therapeutics deep into tumors. Ultrasound mediated cavitation has been shown to promote nanoparticle transport, but achieving tumor wide distribution still represents a considerable challenge. The current study investigates the way in which nanoparticle drug-carriers may be designed to enhance penetration under ultrasound exposure. A computational model has been developed to predict the transport of a nanoparticle in an ultrasonic field in the presence of an oscillating microbubble, by a combination of primary and secondary acoustic radiation forces, acoustic streaming, and microstreaming. Experimental investigations were also performed in a tissue-mimicking phantom to study the transport of different types of particle, in the presence or absence of a microbubble ultrasound contrast agent, at ultrasound frequencies of 0.5 MHz and 1.6 MHz with peak pressures in the range of 0–2.0 MPa. Micro- and nano.
Contributions to the theory of the Pitot-tube displacement effect

In an earlier paper the uniform shear flow past a sphere was studied, by investigating how vortex lines are deformed by the ‘primary flow’ (flow in the absence of shear), and deducing the ‘secondary’ vorticity field (first approximation for small shear). In another paper the image system associated with each element of secondary vorticity was found, whence the Biot-Savart law can be used to determine the secondary flow field by integration. The integration is here carried out for the ‘downwash’ (secondary flow component perpendicular to the undisturbed flow, down the velocity gradient) on the dividing streamline. Difficulties due to the infinite domain of integration and singularities of the integrand are overcome by selecting variables of integration carefully and using known analytical properties of the secondary vorticity. From the computation of downwash is inferred the first approximation (for small shear A) to the ‘displacement’ δ (displacement of the dividing streamline, up the velocity gradient, far upstream of the sphere). If U is the upstream flow velocity and a the radius of the sphere, the computed value of lim (Uδ/Aa2) is 0·9. $A \rightarrow 0$ Details of the calculation show that the secondary trailing vorticity is not an important contributor to the displacement, The downwash is due almost entirely to vorticity upstream of the sphere (Hall's earlier simplified theory gave good results, e.g., 1.24 instead of 0.9, because it concentrated on the effect of local vorticity in producing downwash); further, this produces displacement principally through its image vorticity. The relation between theories for a sphere and experimental results on Pitot tubes (beginning with Young & Maas 1936) is discussed. Theoretical evidence on tertiary- and quartary-flow effects is used here in the light of recent work which renders the successive-approximation sequence uniformly valid at infinity. The conclusion is that the theories, taken together, are not inconsistent with the experimental evidence that (i) at values of the ‘shear parameter’ Aa/U at which the displacement is measurable the ratio δ/a seems to have asymptoted to an approximately constant value, and (ii) displacement is greatly reduced in supersonic flow (Johannesen & Mair 1952) or when ‘sharp-lipped’ tubes are used (Livesey 1956).
HISSbot: Sidewinding with a Soft Snake Robot

Snake robots are characterized by their ability to navigate through small spaces and loose terrain by utilizing efficient cyclic forms of locomotion. Soft snake robots are a subset of these robots which utilize soft, compliant actuators to produce movement. Prior work on soft snake robots has primarily focused on planar gaits, such as undulation. More efficient spatial gaits, such as sidewinding, are unexplored gaits for soft snake robots. We propose a novel means of constructing a soft snake robot capable of sidewinding, and introduce the Helical Inflating Soft Snake Robot (HISSbot). We validate this actuation through the physical HISSbot, and demonstrate its ability to sidewind across various surfaces. Our tests show robustness in locomotion through low-friction and granular media.
DisWOT: Student Architecture Search for Distillation WithOut Training

Knowledge distillation (KD) is an effective training strategy to improve the lightweight student models under the guidance of cumbersome teachers. However, the large architecture difference across the teacher-student pairs limits the distillation gains. In contrast to previous adaptive distillation methods to reduce the teacher-student gap, we explore a novel training-free framework to search for the best student architectures for a given teacher. Our work first empirically show that the optimal model under vanilla training cannot be the winner in distillation. Secondly, we find that the similarity of feature semantics and sample relations between random-initialized teacher-student networks have good correlations with final distillation performances. Thus, we efficiently measure similarity matrixs conditioned on the semantic activation maps to select the optimal student via an evolutionary algorithm without any training. In this way, our student architecture search for Distillation WithOut Training (DisWOT) significantly improves the performance of the model in the distillation stage with at least 180 × training acceleration. Additionally, we extend similarity metrics in DisWOT as new distillers and KD-based zero-proxies. Our experiments on CIFAR, ImageNet and NAS-Bench-201 demonstrate that our technique achieves state-of-the-art results on different search spaces. Our project and code are available at https://lilujunai.github. io/DisWot-CvpR20231.
Covariance Matrix Adaptation Evolutionary Strategy with Worst-Case Ranking Approximation for Min–Max Optimization and Its Application to Berthing Control Tasks

In this study, we consider a continuous min–max optimization problem minx ∈ 𝕏 maxy ∈ 𝕐 f(x, y) whose objective function is a black-box. We propose a novel approach to minimize the worst-case objective function F(x) = maxy ∈ 𝕐 f(x, y) directly using a covariance matrix adaptation evolution strategy in which the rankings of solution candidates are approximated by our proposed worst-case ranking approximation mechanism. We develop two variants of worst-case ranking approximation combined with a covariance matrix adaptation evolution strategy and approximate gradient ascent as numerical solvers for the inner maximization problem. Numerical experiments show that our proposed approach outperforms several existing approaches when the objective function is a smooth strongly convex–concave function and the interaction between x and y is strong. We investigate the advantages of the proposed approach for problems where the objective function is not limited to smooth strongly convex–concave functions. The effectiveness of the proposed approach is demonstrated in the robust berthing control problem with uncertainty.
Existence and density of typical Hodge loci

Motivated by a question of Baldi-Klingler-Ullmo, we provide a general sufficient criterion for the existence and analytic density of typical Hodge loci associated to a polarizable $\mathbb{Z}$-variation of Hodge structures $\mathbb{V}$. Our criterion reproves the existing results in the literature on density of Noether-Lefschetz loci. It also applies to understand Hodge loci of subvarieties of $\mathcal{A}_g$ . For instance, we prove that for $g \geq 4$, if a subvariety $S$ of $\mathcal{A}_g$ has dimension at least $g$ then it has an analytically dense typical Hodge locus. This applies for example to the Torelli locus of $\mathcal{A}_g$
Dark Matter in Fractional Gravity. I. Astrophysical Tests on Galactic Scales

We explore the possibility that the dark matter (DM) component in galaxies may originate fractional gravity. In such a framework, the standard law of inertia continues to hold, but the gravitational potential associated with a given DM density distribution is determined by a modified Poisson equation including fractional derivatives (i.e., derivatives of noninteger type) that are meant to describe nonlocal effects. We analytically derive the expression of the potential that in fractional gravity corresponds to various spherically symmetric density profiles, including the Navarro–Frenk–White (NFW) distribution that is usually exploited to describe virialized halos of collisionless DM as extracted from N-body cosmological simulations. We show that in fractional gravity, the dynamics of a test particle moving in a cuspy NFW density distribution is substantially altered with respect to the Newtonian case, mirroring what in Newtonian gravity would instead be sourced by a density profile with an inner core. We test the fractional gravity framework on galactic scales, showing that (i) it can provide accurate fits to the stacked rotation curves of spiral galaxies with different properties, including dwarfs; (ii) it can reproduce to reasonable accuracy the observed shape and scatter of the radial acceleration relation over an extended range of galaxy accelerations; and (iii) it can properly account for the universal surface density and the core radius versus disk scale length scaling relations. Finally, we discuss the possible origin of the fractional gravity behavior as a fundamental or emerging property of the elusive DM component.
Pulse-timing noise reduction of a mode-locked laser diode by incoherent addition

This paper describes the operation principle and the experimental results of a new and simple technique to reduce pulse-timing noise of a mode-locked laser diode using fiber-optic interferometers placed outside the laser cavity. When the delay times of the interferometers are set to be much larger than the laser's coherence time and to be integer multiples of the pulse repetition period, incoherent addition of pulse intensities at the output of the interferometer gives rise to the cancellation of pulse-timing noise at specific Fourier frequencies. The technique does not need electrical or optical actions for the laser diode and is realizable only with passive optical components. Experiments are carried out using a 19.444-GHz hybrid mode-locked laser diode. After passing through the combined ring and Mach-Zehnder interferometers the root-mean-square timing jitter is reduced to 0.489 ps (100 kHz-18 MHz) that is 29% of the direct laser output.
qEUBO: A Decision-Theoretic Acquisition Function for Preferential Bayesian Optimization

Preferential Bayesian optimization (PBO) is a framework for optimizing a decision maker's latent utility function using preference feedback. This work introduces the expected utility of the best option (qEUBO) as a novel acquisition function for PBO. When the decision maker's responses are noise-free, we show that qEUBO is one-step Bayes optimal and thus equivalent to the popular knowledge gradient acquisition function. We also show that qEUBO enjoys an additive constant approximation guarantee to the one-step Bayes-optimal policy when the decision maker's responses are corrupted by noise. We provide an extensive evaluation of qEUBO and demonstrate that it outperforms the state-of-the-art acquisition functions for PBO across many settings. Finally, we show that, under sufficient regularity conditions, qEUBO's Bayesian simple regret converges to zero at a rate $o(1/n)$ as the number of queries, $n$, goes to infinity. In contrast, we show that simple regret under qEI, a popular acquisition function for standard BO often used for PBO, can fail to converge to zero. Enjoying superior performance, simple computation, and a grounded decision-theoretic justification, qEUBO is a promising acquisition function for PBO.
Habitat fragmentation affects climate adaptation in a forest herb

Climate change and the resulting increased drought frequencies pose considerable threats to forest herb populations, particularly where additional environmental challenges jeopardize responses to selection. Specifically, habitat fragmentation may hamper climate adaptation by altering the distribution of adaptive genetic variation and may also induce evolutionary changes in mating systems. To assess how habitat fragmentation disrupts climate adaptation, we conducted a common garden experiment with Primula elatior offspring originating from 24 populations sampled along a latitudinal gradient with varying climate and landscape characteristics. We then quantified a range of vegetative, regulatory and reproductive traits under distinct soil moisture regimes to evaluate imprints of local adaptation and phenotypic plasticity. Additionally, we conducted a more extensive field campaign in 60 populations along the same latitudinal gradient to evaluate the potential evolutionary breakdown of reciprocal herkogamy. For large, connected populations, our results demonstrated an evolutionary shift from a strategy in southern populations that seems aligned with drought avoidance—where plants minimize their exposure to dry conditions and optimize photosynthesis—to a drought tolerance strategy in northern populations, where plants are adapted to function despite water scarcity. However, habitat fragmentation disrupted climate clines and the adaptive responses to drought stress in key traits related to growth, biomass allocation and water regulation. Additionally, our findings indicate the onset of evolutionary breakdown in reciprocal herkogamy and divergence in other key flower traits. The disruption of climate clines, drought responses and adaptations in mating systems contributed to a substantially diminished flowering investment across the distribution range, with the most pronounced effects observed in southern fragmented populations. Synthesis. We present novel empirical evidence of how habitat fragmentation disrupts climate adaptation and drought tolerance in a wide range of traits along the range of the forest herb Primula elatior. These findings emphasize the need to account for habitat fragmentation while designing effective conservation strategies in order to preserve and restore resilient meta‐populations of forest herbs amidst ongoing global changes.
BC-IRL: Learning Generalizable Reward Functions from Demonstrations

How well do reward functions learned with inverse reinforcement learning (IRL) generalize? We illustrate that state-of-the-art IRL algorithms, which maximize a maximum-entropy objective, learn rewards that overfit to the demonstrations. Such rewards struggle to provide meaningful rewards for states not covered by the demonstrations, a major detriment when using the reward to learn policies in new situations. We introduce BC-IRL a new inverse reinforcement learning method that learns reward functions that generalize better when compared to maximum-entropy IRL approaches. In contrast to the MaxEnt framework, which learns to maximize rewards around demonstrations, BC-IRL updates reward parameters such that the policy trained with the new reward matches the expert demonstrations better. We show that BC-IRL learns rewards that generalize better on an illustrative simple task and two continuous robotic control tasks, achieving over twice the success rate of baselines in challenging generalization settings.