Miracle-dz/UniFAR-FTUs huggingface dataset:
Compact Neural Representation Using Attentive Network Pruning

Deep neural networks have evolved to become power demanding and consequently difficult to apply to small-size mobile platforms. Network parameter reduction methods have been introduced to systematically deal with the computational and memory complexity of deep networks. We propose to examine the ability of attentive connection pruning to deal with redundancy reduction in neural networks as a contribution to the reduction of computational demand. In this work, we describe a Top-Down attention mechanism that is added to a Bottom-Up feedforward network to select important connections and subsequently prune redundant ones at all parametric layers. Our method not only introduces a novel hierarchical selection mechanism as the basis of pruning but also remains competitive with previous baseline methods in the experimental evaluation. We conduct experiments using different network architectures on popular benchmark datasets to show high compression ratio is achievable with negligible loss of accuracy.
Learning to hash with semantic similarity metrics and empirical KL divergence

Learning to hash is an efficient paradigm for exact and approximate nearest neighbor search from massive databases. Binary hash codes are typically extracted from an image by rounding output features from a CNN, which is trained on a supervised binary similar/ dissimilar task. Drawbacks of this approach are: (i) resulting codes do not necessarily capture semantic similarity of the input data (ii) rounding results in information loss, manifesting as decreased retrieval performance and (iii) Using only class-wise similarity as a target can lead to trivial solutions, simply encoding classifier outputs rather than learning more intricate relations, which is not detected by most performance metrics. We overcome (i) via a novel loss function encouraging the relative hash code distances of learned features to match those derived from their targets. We address (ii) via a differentiable estimate of the KL divergence between network outputs and a binary target distribution, resulting in minimal information loss when the features are rounded to binary. Finally, we resolve (iii) by focusing on a hierarchical precision metric. Efficiency of the methods is demonstrated with semantic image retrieval on the CIFAR-100, ImageNet and Conceptual Captions datasets, using similarities inferred from the WordNet label hierarchy or sentence embeddings.
Estimating g-Leakage via Machine Learning

This paper considers the problem of estimating the information leakage of a system in the black-box scenario, i.e. when the system's internals are unknown to the learner, or too complicated to analyze, and the only available information are pairs of input-output data samples, obtained by submitting queries to the system or provided by a third party. The frequentist approach relies on counting the frequencies to estimate the input-output conditional probabilities, however this method is not accurate when the domain of possible outputs is large. To overcome this difficulty, the estimation of the Bayes error of the ideal classifier was recently investigated using Machine Learning (ML) models, and it has been shown to be more accurate thanks to the ability of those models to learn the input-output correspondence. However, the Bayes vulnerability is only suitable to describe one-try attacks. A more general and flexible measure of leakage is the g-vulnerability, which encompasses several different types of adversaries, with different goals and capabilities. We propose a novel approach to perform black-box estimation of the g-vulnerability using ML which does not require to estimate the conditional probabilities and is suitable for a large class of ML algorithms. First, we formally show the learnability for all data distributions. Then, we evaluate the performance via various experiments using k-Nearest Neighbors and Neural Networks. Our approach outperform the frequentist one when the observables domain is large.
Universal relations for hybridized 
s
- and 
p
-wave interactions from spin-orbital coupling

In this work, we study the universal relations for 1D spin-orbital-coupled fermions near both $s$- and $p$-wave resonances using effective field theory. Since the spin-orbital coupling mixes different partial waves, a contact matrix is introduced to capture the non-trivial correlation between dimers. We find the signature of the spin-orbital coupling appears at the leading order for the off-diagonal components of the momentum distribution matrix, which is proportional to $1/q^{3}$ ($q$ is the relative momentum). We further derive the large frequency/momentum behavior of the Raman spectroscopy, which serves as an independent measurable quantity for contacts. We finally give an explicit example of contacts by considering a two-body problem.
A New Proof of Newman's Conjecture and a Generalization

Newman's conjecture (proved by Rodgers and Tao in 2018) concerns a certain family of deformations $\{\xi_t(s)\}_{t \in \mathbb{R}}$ of the Riemann xi function for which there exists an associated constant $\Lambda \in \mathbb{R}$ (called the de Bruijn-Newman constant) such that all the zeros of $\xi_t$ lie on the critical line if and only if $t \geq \Lambda$. The Riemann hypothesis is equivalent to the statement that $\Lambda \leq 0$, and Newman's conjecture states that $\Lambda \geq 0$. In this paper we give a new proof of Newman's conjecture which avoids many of the complications in the proof of Rodgers and Tao. Unlike the previous best methods for bounding $\Lambda$, our approach does not require any information about the zeros of the zeta function, and it can be readily be applied to a wide variety of $L$-functions. In particular, we establish that any $L$-function in the extended Selberg class has an associated de Bruijn-Newman constant and that all of these constants are nonnegative. Stated in the Riemann xi function case, our argument proceeds by showing that for every $t < 0$ the function $\xi_t$ can be approximated in terms of a Dirichlet series $\zeta_t(s)=\sum_{n=1}^{\infty}\exp(\frac{t}{4} \log^2 n)n^{-s}$ whose zeros then provide infinitely many zeros of $\xi_t$ off the critical line.
Multi-Scale Zero-Order Optimization of Smooth Functions in an RKHS

We aim to optimize a black-box function $f:\mathcal{X} \mapsto \mathbb{R}$ under the assumption that $f$ is Holder smooth and has bounded norm in the RKHS associated with a given kernel $K$. This problem is known to have an agnostic Gaussian Process (GP) bandit interpretation in which an appropriately constructed GP surrogate model with kernel $K$ is used to obtain an upper confidence bound (UCB) algorithm. In this paper, we propose a new algorithm (\texttt{LP-GP-UCB}) where the usual GP surrogate model is augmented with Local Polynomial (LP) estimators of the Holder smooth function $f$ to construct a multi-scale UCB guiding the search for the optimizer. We analyze this algorithm and derive high probability bounds on its simple and cumulative regret. We then prove that the elements of many common RKHS are Holder smooth and obtain the corresponding Holder smoothness parameters, and hence, specialize our regret bounds for several commonly used kernels. When specialized to the Squared Exponential (SE) kernel, \texttt{LP-GP-UCB} matches the optimal performance, while for the case of Matern kernels $(K_{\nu})_{\nu>0}$, it results in uniformly tighter regret bounds for all values of the smoothness parameter $\nu>0$. Most notably, for certain ranges of $\nu$, the algorithm achieves near-optimal bounds on simple and cumulative regrets, matching the algorithm-independent lower bounds up to polylog factors, and thus closing the large gap between the existing upper and lower bounds for these values of $\nu$. Additionally, our analysis provides the first explicit regret bounds, in terms of the budget $n$, for the Rational-Quadratic (RQ) and Gamma-Exponential (GE). Finally, experiments with synthetic functions as well as a CNN hyperparameter tuning task demonstrate the practical benefits of our multi-scale partitioning approach over some existing algorithms numerically.
Information-Theoretic Generalization Bounds for Meta-Learning and Applications

Meta-learning, or “learning to learn”, refers to techniques that infer an inductive bias from data corresponding to multiple related tasks with the goal of improving the sample efficiency for new, previously unobserved, tasks. A key performance measure for meta-learning is the meta-generalization gap, that is, the difference between the average loss measured on the meta-training data and on a new, randomly selected task. This paper presents novel information-theoretic upper bounds on the meta-generalization gap. Two broad classes of meta-learning algorithms are considered that use either separate within-task training and test sets, like model agnostic meta-learning (MAML), or joint within-task training and test sets, like reptile. Extending the existing work for conventional learning, an upper bound on the meta-generalization gap is derived for the former class that depends on the mutual information (MI) between the output of the meta-learning algorithm and its input meta-training data. For the latter, the derived bound includes an additional MI between the output of the per-task learning procedure and corresponding data set to capture within-task uncertainty. Tighter bounds are then developed for the two classes via novel individual task MI (ITMI) bounds. Applications of the derived bounds are finally discussed, including a broad class of noisy iterative algorithms for meta-learning.
BabyWalk: Going Farther in Vision-and-Language Navigation by Taking Baby Steps

Learning to follow instructions is of fundamental importance to autonomous agents for vision-and-language navigation (VLN). In this paper, we study how an agent can navigate long paths when learning from a corpus that consists of shorter ones. We show that existing state-of-the-art agents do not generalize well. To this end, we propose BabyWalk, a new VLN agent that is learned to navigate by decomposing long instructions into shorter ones (BabySteps) and completing them sequentially. A special design memory buffer is used by the agent to turn its past experiences into contexts for future steps. The learning process is composed of two phases. In the first phase, the agent uses imitation learning from demonstration to accomplish BabySteps. In the second phase, the agent uses curriculum-based reinforcement learning to maximize rewards on navigation tasks with increasingly longer instructions. We create two new benchmark datasets (of long navigation tasks) and use them in conjunction with existing ones to examine BabyWalk’s generalization ability. Empirical results show that BabyWalk achieves state-of-the-art results on several metrics, in particular, is able to follow long instructions better. The codes and the datasets are released on our project page: https://github.com/Sha-Lab/babywalk.
Is the gap in the DS Tau disc hiding a planet?

Recent mm-wavelength surveys performed with the Atacama Large Millimeter Array (ALMA) have revealed protoplanetary discs characterized by rings and gaps. A possible explanation for the origin of such rings is the tidal interaction with an unseen planetary companion. The protoplanetary disc around DS Tau shows a wide gap in the ALMA observation at 1.3 mm. We construct a hydrodynamical model for the dust continuum observed by ALMA assuming the observed gap is carved by a planet between one and five Jupiter masses. We fit the shape of the radial intensity profile along the disc major axis varying the planet mass, the dust disc mass, and the evolution time of the system. The best fitting model is obtained for a planet with $M_{\rm p}=3.5\,M_{\rm Jup}$ and a disc with $M_{\rm dust}= 9.6\cdot10^{-5}\,M_{\odot}$. Starting from this result, we also compute the expected signature of the planet in the gas kinematics, as traced by CO emission. We find that such a signature (in the form of a "kink" in the channel maps) could be observed by ALMA with a velocity resolution between $0.2-0.5\,\rm{kms}^{-1}$ and a beam size between 30 and 50 mas.
Estimation of the intercalving period in Italian Mediterranean buffalo

Abstract Aim of this study was to estimate the average intercalving period of a buffalo herd/population, by using the percentage of subjects that are able to conceive within 90 days post-partum, and that, consequently, have an intercalving period lower than 400 days, in order to give a precocious judgment on its fertility. A total number of 8,845 intercalving periods were recorded in 4 farms and 4 Provinces of Campania and Lazio Regions. In all the farms the out of breeding season mating technique was adopted and a constant diet was administered throughout the year. Values were grouped for primiparous and pluriparous buffaloes and were divided in four calving periods: January-March; April-June; July-Sep-tember; October-December. Within each group/farm ANOVA was performed and the means were analysed by t Student test. In each group the mean value and the percentage of subjects with an intercalving period lower than 400 days (% < 400 days) were calculated and the regression analysis was carried out between the values of the intercalving periods and the % < 400 days. Finally, an all-in regression analysis was performed by using the data of 55 groups, excluding those with less than 25 values. The regression analysis between % < 400 days and the mean value of the intercalving period was always significant, except in one case. The comparison between the real values and those calculated allows to affirm that, if the percentage of subjects with an intercalving period lower than 400 days is higher than 50%, an intercalving period lower than 450 days will be probably recorded.
On asymptotic base loci of relative anti-canonical divisors of algebraic fiber spaces

In this paper, we study the relative anti-canonical divisor − K X / Y -K_{X/Y} of an algebraic fiber space ϕ : X → Y \phi \colon X\to Y , and we reveal relations among positivity conditions of − K X / Y -K_{X/Y} , certain flatness of direct image sheaves, and variants of the base loci including the stable (augmented, restricted) base loci and upper level sets of Lelong numbers. This paper contains three main results: The first result says that all the above base loci are located in the horizontal direction unless they are empty. The second result is an algebraic proof for Campana–Cao–Matsumura’s equality on Hacon–McKernan’s question, whose original proof depends on analytics methods. The third result proves that algebraic fiber spaces with semi-ample relative anti-canonical divisor actually have a product structure via the base change by an appropriate finite étale cover of Y Y . Our proof is based on algebraic as well as analytic methods for positivity of direct image sheaves.
Monopoles and Landau-Ginzburg Models II: Floer Homology

This is the second paper of this series. We define the monopole Floer homology for 3-manifolds with torus boundary, extending the work of Kronheimer-Mrowka for closed 3-manifolds. The Euler characteristic of this Floer homology recovers the Milnor torsion invariant of the 3-manifold by a theorem of Meng-Taubes.
Seasonal incidence of infective trichostrongyle larvae on pasture: The contribution of the ewe and the role of the residual pasture infestation as sources of infection to the lamb

Abstract The contribution made by the ewe and lamb to the trichostrongyle larval population on pasture was investigated for 3 years. Lambs grazing the year round acquire two generations of nematodes. The first is acquired in the spring. , from residual larvae and/or those that develop from eggs passed by the ewe during the post-partum period. The second is acquired in the autumn from eggs passed by the lambs. The results suggest that in most years the residual pasture infestation is the more important source of infection for lambs in the spring The implications of these findings for management are discussed.
Plant Breeding with Genomic Selection: Gain per Unit Time and Cost

ABSTRACTAdvancements in Lorenz, USDA-ARS, R.W. Holley Center for Agriculture and Health, genotyping are rapidly decreasing Cornell Univ., Ithaca, NY 14853. Received 9 Nov. 2009. *Corremarker costs and increasing genome coverage. This is facilitating the use of marker-assisted selection (MAS) in plant breeding. Commonly employed MAS strategies, however, are not well suited for agronomically important complex traits, requiring extra time for ! eld-based phenotyping to identify agro-nomically superior lines. Genomic selection (GS) is an emerging alternative to MAS that uses all marker information to calculate genomic esti-mated breeding values (GEBVs) for complex traits. Selections are made directly on GEBV without further phenotyping. We developed an analytical framework to (i) compare gains from MAS and GS for complex traits and (ii) provide a plant breeding context for interpreting results from studies on GEBV accuracy. We designed MAS and GS breeding strategies with equal budgets for a high-investment maize (Zea mays L.) program and a low-investment winter wheat (Triticum aestivum L.) program. Results indicate that GS can outperform MAS on a per-year basis even at low GEBV accuracies. Using a previ-ously reported GEBV accuracy of 0.53 for net merit in dairy cattle, expected annual gain from GS exceeded that of MAS by about threefold for maize and twofold for winter wheat. We con-clude that if moderate selection accuracies can be achieved, GS could dramatically accelerate genetic gain through its shorter breeding"cycle.E. L. He-ner and M.E. Sorrells, Cornell Univ., Dep. of Plant Breeding and Genetics, Emerson Hall, Ithaca, NY 14853; J.-L. Jannink and A.J. -sponding author (mes12@cornell.edu).
Plasmatic Filaments and Particles in Mature Sieve Elements of Heracleum sphondylium under the Electron Microscope

By using dimethyl sulphoxide in addition to glutaraldehyde on the phloem of Heracleum a speedier fixative for phloem sieve elements has been developed, less disruptive to plasmatic filaments (p.f.). Strands of p.f. under the electron microscope seem to run from sieve element to element, axially through the lumina and the sieve-plate pores. Some of these axial strands give profiles which suggest that they exist as tubular structures in which a series of swellings, 20 to 60 nm in diameter, can be detected as if microperistalsis were passing along them. Marker particles of five different types were noted, often attached to wisps of plasmatic filaments. Many of the p.f. seemed to have a helical substructure. That plasmatic filaments (p.f.), called p-protein by Esau and Cronshaw (1968), are widespread in mature sieve elements of the phloem of plants is now generally accepted, but controversy continues as to their disposition in the living intact element. Some workers such as Anderson and Cronshaw (1969, 1970», b) contend that this material normally exists in the lumina of the sieve elements but hardly at all in the pores. On the other hand, Behnke (1971), Siddiqui and Spanner (1970), and others find that it normally occurs in the sieve-plate pores. These reports may represent the variation which naturally occurs in plants in vivo or they may indicate that electron microscopy still suffers from artifacts of fixation and dehydration. Johnson (1968) has attempted to avoid these fixation difficulties by using a freeze etching technique. We have tried to refine the usual glutaraldehyde fixation by adding dimethyl sulphoxide (DMSO) which has been found by Sandborn (1970) to speed the penetration of fixatives through living membranes. This paper reports the results and advances evidence that in our plants some of the p.f. normally extend through both the lumen and the plate pores of sieve tubes. It also gives additional information about the structure of these filamentous strands which may have important implications for the mechanisms of translocation, and upon the particles which are often associated with them. WITH THREE PLATES IN THE TEXT
A scaling law of multilevel selection on a continuous trait

Living systems are hierarchically structured, whereby replicating components are grouped into reproducing collectives; e.g., organelles are grouped into cells; cells, into multicellular organisms; and multicellular organisms, into colonies. We consider how such hierarchically-structured replicator systems evolve along a continuumbetween cooperation and cheating. Cooperation is selected through competition among collectives, whereas cheating is selected through competition among replicating components within collectives. We determine how the evolution depends on the number of replicating components per collective (N) and the mutation rate of replicating components (m). We demonstrate a scaling relation that holds when selection for cooperation exactly balances out selection for cheating: N ∝ m−α, where α (> 0) increases with selection strength. Increasing N or m drives the system towards a phase where cheating evolves, and the above scaling relation separates this phase from that where cooperation evolves. We develop a simple theory that can account for this scaling relation using moment evolution equations and clarify the limit of applicability of the theory using time-backward observation.
Dynamics of a surfactant-laden bubble bursting through an interface

Abstract We study the effect of surfactant on the dynamics of a bubble bursting through an interface. We perform fully three-dimensional direct numerical simulations using a hybrid interface-tracking/level-set method accounting for surfactant-induced Marangoni stresses, sorption kinetics and diffusive effects. We select an initial bubble shape corresponding to a large Laplace number and a vanishingly small Bond number in order to neglect gravity, and isolate the effects of surfactant on the flow. Our results demonstrate that the presence of surfactant affects the dynamics of the system through Marangoni-induced flow, driving motion from high to low concentration regions, which is responsible for the onset of a recirculation zone close to the free surface. These Marangoni stresses rigidify the interface, delay the cavity collapse and influence the jet breakup process.
Optimizing Deep Learning Recommender Systems Training on CPU Cluster Architectures

During the last two years, the goal of many researchers has been to squeeze the last bit of performance out of HPC system for AI tasks. Often this discussion is held in the context of how fast ResNet50 can be trained. Unfortunately, ResNet50 is no longer a representative workload in 2020. Thus, we focus on Recommender Systems which account for most of the AI cycles in cloud computing centers. More specifically, we focus on Facebook’s DLRM benchmark. By enabling it to run on latest CPU hardware and software tailored for HPC, we are able to achieve up to two-orders of magnitude improvement in performance on a single socket compared to the reference CPU implementation, and high scaling efficiency up to 64 sockets, while fitting ultra-large datasets which cannot be held in single node’s memory. Therefore, this paper discusses and analyzes novel optimization and parallelization techniques for the various operators in DLRM. Several optimizations (e.g. tensor-contraction accelerated MLPs, framework MPI progression, BFLOAT16 training with up to $1.8 \times $ speed-up) are general and transferable to many other deep learning topologies.