Miracle-dz/UniFAR-FTUs huggingface dataset:
HEARTS: Multi-task Fusion of Dense Retrieval and Non-autoregressive Generation for Sponsored Search

Matching user search queries with relevant keywords bid by ad-vertisers in real-time is a crucial problem in sponsored search. In the literature, two broad set of approaches have been explored to solve this problem: (i) Dense Retrieval (DR) - learning dense vector representations for queries and bid keywords in a shared space, and (ii) Natural Language Generation (NLG) - learning to directly generate bid keywords given queries. In this work, we first conduct an empirical study of these two approaches and show that they offer complementary benefits that are additive. In particular, a large fraction of the keywords retrieved from NLG haven’t been retrieved by DR and vice-versa. We then show that it is possible to effectively combine the advantages of these two approaches in one model. Specifically, we propose HEARTS: a novel multi-task fusion framework where we jointly optimize a shared encoder to perform both DR and non-autoregressive NLG. Through extensive experiments on search queries from over 30+ countries spanning 20+ languages, we show that HEARTS retrieves 40.3% more high-quality bid keywords than the baseline approaches with the same GPU compute. We also demonstrate that inferring on a single HEARTS model is as good as inferring on two different DR and NLG baseline models, with 2x the compute. Further, we show that DR models trained with the HEARTS objective are significantly better than those trained with the standard contrastive loss functions. Finally, we show that our HEARTS objective can be adopted to short-text retrieval tasks other than sponsored search and achieve significant performance gains.
Estimation of Average Derivatives of Latent Regressors: With an Application to Inference on Buffer-Stock Saving

This paper proposes a density-weighted average derivative estimator based on two noisy measures of a latent regressor. Both measures have classical errors with possibly asymmetric distributions. We show that the proposed estimator achieves the root- n rate of convergence, and derive its asymptotic normal distribution for statistical inference. Simulation studies demonstrate excellent small-sample performance supporting the root- n asymptotic normality. Based on the proposed estimator, we construct a formal test on the sub-unity of the marginal propensity to consume out of permanent income (MPCP) under a nonparametric consumption model and a permanent-transitory model of income dynamics with nonparametric distribution. Applying the test to four recent waves of U.S. Panel Study of Income Dynamics (PSID), we reject the null hypothesis of the unit MPCP in favor of a sub-unit MPCP, supporting the buﬀer-stock model of saving.
TDB: Breaking All Hop-Constrained Cycles in Billion-Scale Directed Graphs

The feedback vertex set is one of 21 Np-complete problems proposed by Karp, aiming at breaking all cycles in a given graph. It could be used in numerous areas, e.g., program analysis, database systems. In reality, users are concerned the cycles with constraints, e.g., the cycles with a hop constraint. For instance, in the E-commerce networks, the fraud detection team would discard cycles with a high number of hops since they are less relevant and grow exponentially in size. Thus, it is quite reasonable to investigate the feedback vertex set problem with hop-constrained cycles, namely hop-constrained cycle cover problem. It is concerned with determining a vertex set that covers all hop-constrained cycles in a given directed graph. A common method is to use a bottom-up algorithm, where it iteratively selects cover vertices into the result set. Based on this paradigm, the existing works mainly focus on the vertices orders and several heuristic strategies. In this paper, a totally opposite cover process top-down is proposed and bounds are presented on it. Surprisingly, both theoretical and practical performance are improved. On the theoretical side, this work is the first to achieve O(k•n•m) time complexity, whereas the state-of-the-art method achieves time complexity of O(nk). 1 On the practical level, the proposed algorithm, namely TDB++, outperforms the state-of-the-art by 2 to 3 orders of magnitude on average while preserving the minimal property. As a result, the method in this paper outperforms the state-of-the-art approaches in terms of both running time and theoretical time complexity. The hop-constrained cycle cover problem on billion-scale networks has been solved with a minimal2 cover set for k > 3.
Evidence and Strategy on Economic Distance in Spatially Augmented Solow-Swan Growth Model

Economists’ interests in growth theory have a very long history (Harrod, 1939; Domar, 1946; Solow, 1956; Swan 1956; Mankiw, Romer, and Weil, 1992). Recently, starting from the neoclassical growth model, Ertur and Koch (2007) developed the spatially augmented Solow-Swan growth model with the exogenous spatial weights matrices ( W ). While the exogenous W assumption could be true only with the geograph-ical/physical distance, it may not be true when economic/social distances play a role. Using Penn World Table version 7.1, which covers year 1960-2010, I conducted the robust Rao’s score test (Bera, Dogan, and Taspinar, 2018) to determine if W is endogeonus and used the maximum likelihood estimation (Qu and Lee, 2015). The key ﬁnding is that the signiﬁcance and positive eﬀects of physical capital externalities and spatial externalities (technological interdependence) in Ertur and Koch (2007) were no longer found with the exogenous W , but still they were with the endogenous W models. I also found an empirical strategy on which economic distance to use when the data recently has been under heavy shocks of the worldwide ﬁnancial crises during year 1996-2010.
The Bismut-Elworthy-Li formula for semi-linear distribution-dependent SDEs driven by fractional Brownian motion

In this work, we will show the existence, uniqueness, and weak differentiability of the solution of semi-linear mean-ﬁeld stochastic differential equations driven by fractional Brownian motion. We prove an extension of the Bismut-Elworthy-Li formula and show some applications in the sensitivity analysis of variance swaps and the price of derivatives with respect to the initial point. solution by another process whose Malliavin derivative and ﬂow can be easily modiﬁed each other. We showed the application of the Bismut formula in the sensitivity analysis of variance swaps with distribution-dependent models.
Bending the Future: Autoregressive Modeling of Temporal Knowledge Graphs in Curvature-Variable Hyperbolic Spaces

Recently there is an increasing scholarly interest in time-varying knowledge graphs, or temporal knowledge graphs (TKG). Previous research suggests diverse approaches to TKG reasoning that uses historical information. However, less attention has been given to the hierarchies within such information at different timestamps. Given that TKG is a sequence of knowledge graphs based on time, the chronology in the sequence derives hierarchies between the graphs. Furthermore, each knowledge graph has its hierarchical level which may differ from one another. To address these hierarchical characteristics in TKG, we propose HyperVC, which utilizes hyperbolic space that better encodes the hierarchies than Euclidean space. The chronological hierarchies between knowledge graphs at different timestamps are represented by embedding the knowledge graphs as vectors in a common hyperbolic space. Additionally, diverse hierarchical levels of knowledge graphs are represented by adjusting the curvatures of hyperbolic embeddings of their entities and relations. Experiments on four benchmark datasets show substantial improvements, especially on the datasets with higher hierarchical levels.
Contribution à l’histoire des tests laplaciens

Au croisement des recherches historiques de Bernard Bru sur la statistique laplacienne, et des interrogations recurrentes depuis les annees 1960 sur les usages peu satisfaisants des tests statistiques dans les sciences d’observation, nous revisitons la mise en place des tests statistiques dans l’œuvre de Laplace. On trouve deja dans celle-ci un melange entre plusieurs approches, et des audaces interpretatives, dignes de la version aujourd’hui pratiquee, hybride entre les apports de Bayes-Laplace, Fisher et Neyman.
Stability and observability of magnetic primordial black hole-neutron star collisions

The collision of a primordial black hole with a neutron star results in the black hole eventually consuming the entire neutron star. However, if the black hole is magnetically charged, and therefore stable against decay by Hawking radiation, the consequences can be quite different. Upon colliding with a neutron star, a magnetic black hole very rapidly comes to a stop. For large enough magnetic charge, we show that this collision can be detected as a sudden change in the rotation period of the neutron star, a glitch or anti-glitch.We argue that the magnetic primordial black hole, which then settles to the core of the neutron star, does not necessarily devour the entire neutron star; the system can instead reach a long-lived, quasi-stable equilibrium. Because the black hole is microscopic compared to the neutron star, most stellar properties remain unchanged compared to before the collision. However, the neutron star will heat up and its surface magnetic field could potentially change, both effects potentially observable.
Meta-Gradients in Non-Stationary Environments

Meta-gradient methods (Xu et al., 2018; Zahavy et al., 2020) offer a promising solution to the problem of hyperparameter selection and adaptation in non-stationary reinforcement learning problems. However, the properties of meta-gradients in such environments have not been systematically studied. In this work, we bring new clarity to meta-gradients in non-stationary environments. Concretely, we ask: (i) how much information should be given to the learned optimizers, so as to enable faster adaptation and generalization over a lifetime, (ii) what meta-optimizer functions are learned in this process, and (iii) whether meta-gradient methods provide a bigger advantage in highly non-stationary environments. To study the effect of information provided to the meta-optimizer, as in recent works (Flennerhag et al., 2021; Almeida et al., 2021), we replace the tuned meta-parameters of fixed update rules with learned meta-parameter functions of selected context features. The context features carry information about agent performance and changes in the environment and hence can inform learned meta-parameter schedules. We find that adding more contextual information is generally beneficial, leading to faster adaptation of meta-parameter values and increased performance over a lifetime. We support these results with a qualitative analysis of resulting meta-parameter schedules and learned functions of context features. Lastly, we find that without context, meta-gradients do not provide a consistent advantage over the baseline in highly non-stationary environments. Our findings suggest that contextualizing meta-gradients can play a pivotal role in extracting high performance from meta-gradients in non-stationary settings.
Nonlinear focusing of terahertz laser beam using a layered superconductor

We theoretically study the propagation of a Terahertz (THz) Gaussian beam through a thin sample of layered superconductor. We consider the beam axis and the superconducting layers to be perpendicular to the sample interface, while the electric field in the beam is perpendicular to the layers. We show that, in such a geometry, the Josephson current between the superconducting layers supports lensing of the beam instead of divergence on the Rayleigh range. Moreover, due to the nonlinearity, the focal length and waist of the transmitted beam depend on the incident beam intensity. These dependences demonstrate nontrivial hysteresis behavior that can be observed in experiments with THz lasers.
Sample Complexity of an Adversarial Attack on UCB-based Best-arm Identification Policy

In this work I study the problem of adversarial perturbations to rewards, in a Multi-armed bandit (MAB) setting. Speciﬁcally, I focus on an adversarial attack to a UCB type best-arm identiﬁcation policy applied to a stochastic MAB. The UCB attack presented in [1] results in pulling a target arm K very often. I used the attack model of [1] to derive the sample complexity required for selecting target arm K as the best arm. I have proved that the stopping condition of UCB based best-arm identiﬁcation algorithm given in [2], can be achieved by the target arm K in T rounds, where T depends only on the total number of arms and σ parameter of σ 2 − sub-Gaussian random rewards of the arms.
Search for Heavy Right-Handed Neutrinos at the LHC and Beyond in the Same-Sign Leptons Final State

In this study we explore the LHC's Run II potential to discover heavy Majorana neutrinos with luminosities between $30$ and $3000$ fb$^{-1}$ in the $l^{\pm}l^{\prime\pm}j~j$ final state. Given that there are many models for neutrino mass generation, even within the Type I seesaw framework, we use a simplified model approach and assume that a heavy single Majorana neutrino extension of the SM and a limiting case of the left-right symmetric model. We then extend the analysis to a future hadron collider running at $100$ TeV center of mass energies. This extrapolation in energy allows us to study the relative importance of the resonant production versus gauge boson fusion processes in the study of Majorana neutrinos at hadron colliders. We analyze and propose different search strategies designed to maximize the discovery potential in either the resonant production or the gauge boson fusion modes.
Using Dark Matter Haloes to Learn about Cosmic Acceleration: A New Proposal for a Universal Mass Function

Structure formation provides a strong test of any cosmic acceleration model because a successful dark energy model must not inhibit or overpredict the development of observed large-scale structures. Traditional approaches to studies of structure formation in the presence of dark energy or a modified gravity implement a modified Press-Schechter formalism, which relates the linear overdensities to the abundance of dark matter haloes at the same time. We critically examine the universality of the Press-Schechter formalism for different cosmologies, and show that the halo abundance is best correlated with spherical linear overdensity at 94% of collapse (or observation) time. We then extend this argument to ellipsoidal collapse (which decreases the fractional time of best correlation for small haloes), and show that our results agree with deviations from modified Press-Schechter formalism seen in simulated mass functions. This provides a novel universal prescription to measure linear density evolution, based on current and future observations of cluster (or dark matter) halo mass function. In particular, even observations of cluster abundance in a single epoch will constrain the entire history of linear growth of cosmological of perturbations. Subject headings: cosmology: large-scale structure of universe
Numerical rank of singular kernel functions

We study the rank of sub-matrices arising out of kernel functions, F ( xxx,yyy ) : R d × R d (cid:55)→ R , where xxx,yyy ∈ R d , that have a singularity along the diagonal xxx = yyy . Such kernel functions are frequently encountered in a wide range of applications such as N body problems, Green’s functions, integral equations, geostatistics, kriging, Gaussian processes, etc. One of the challenges in dealing with these kernel functions is that the corresponding matrix associated with these kernels is large and dense and thereby, the computational cost of matrix operations is high. In this article, we prove new theorems bounding the numerical rank of sub-matrices arising out of these kernel functions. Under reasonably mild assumptions, we prove that the rank of certain sub-matrices is rank-deﬁcient in ﬁnite precision. This rank depends on the dimension of the ambient space and also on the type of interaction between the hyper-cubes containing the corresponding set of particles. This rank structure can be leveraged to reduce the computational cost of certain matrix operations such as matrix-vector products, solving linear systems, etc. We also present numerical results on the growth of rank of certain sub-matrices in 1 D, 2 D, 3 D and 4 D, which, not surprisingly, agrees with the theoretical results.
Upper bound on the number of resonances for even asymptotically hyperbolic manifolds with real-analytic ends

We prove a polynomial upper bound on the number of resonances in a disc whose radius tends to +∞ for even asymptotically hyperbolic manifolds with real-analytic ends. Our analysis also gives a similar upper bound on the number of quasinormal frequencies for Schwarzschild–de Sitter spacetimes.
Connectivity of a Family of Bilateral Agreement Random Graphs

Bilateral agreement based random undirected graphs were introduced and analyzed by La and Kabkab in 2015. The construction of the graph with $n$ vertices in this model uses a (random) preference order on other $n-1$ vertices and each vertex only prefers the top $k$ other vertices using its own preference order; in general, $k$ can be a function of $n$. An edge is constructed in the ensuing graph if and only if both vertices of a potential edge prefer each other. This random graph is a generalization of the random $k^{th}$-nearest neighbor graphs of Cooper and Frieze that only consider unilateral preferences of the vertices. Moharrami \emph{et al.} studied the emergence of a giant component and its size in this new random graph family in the limit of $n$ going to infinity when $k$ is finite. Connectivity properties of this random graph family have not yet been formally analyzed. In their original paper, La and Kabkab conjectured that for $k(t)=t \log n$, with high probability connectivity happens at $t>1$ and the graph is disconnected for $t<1$. We provide a proof for this conjecture. We will also introduce an asymptotic for the average degree of this graph.
Concealing Sensitive Samples against Gradient Leakage in Federated Learning

Federated Learning (FL) is a distributed learning paradigm that enhances users' privacy by eliminating the need for clients to share raw, private data with the server. Despite the success, recent studies expose the vulnerability of FL to model inversion attacks, where adversaries reconstruct users’ private data via eavesdropping on the shared gradient information. We hypothesize that a key factor in the success of such attacks is the low entanglement among gradients per data within the batch during stochastic optimization. This creates a vulnerability that an adversary can exploit to reconstruct the sensitive data. Building upon this insight, we present a simple, yet effective defense strategy that obfuscates the gradients of the sensitive data with concealed samples. To achieve this, we propose synthesizing concealed samples to mimic the sensitive data at the gradient level while ensuring their visual dissimilarity from the actual sensitive data. Compared to the previous art, our empirical evaluations suggest that the proposed technique provides the strongest protection while simultaneously maintaining the FL performance. Code is located at https://github.com/JingWu321/DCS-2.
Highly reliable tensile strained 810 nm QW laser diode operating at high temperatures

Lasers with tensely strained GaAsP SQW were grown by MOVPE. P-up mounted devices achieved a cw output of nearly 7 W at 15/spl deg/C and >1.5 W at 85/spl deg/C heat sink temperature (aperture 100 /spl mu/m). Degradation rates of 1.2 10/sup -5//h@1 W, 25/spl deg/C have been obtained.
Flash-Cosmos: In-Flash Bulk Bitwise Operations Using Inherent Computation Capability of NAND Flash Memory

Bulk bitwise operations, i. e. , bitwise operations on large bit vectors, are prevalent in a wide range of important application domains, including databases, graph processing, genome analysis, cryptography, and hyper-dimensional computing. In conventional systems, the performance and energy efficiency of bulk bitwise operations are bottlenecked by data movement between the compute units (e.g., CPUs and GPUs) and the memory hierarchy. In-flash processing (i. e., processing data inside NAND flash chips) has a high potential to accelerate bulk bitwise operations by fundamentally reducing data movement through the entire memory hierarchy, especially when the processed data does not fit into main memory. We identify two key limitations of the state-of-the-art in-flash processing technique for bulk bitwise operations; (i) it falls short of maximally exploiting the bit-level parallelism of bulk bitwise operations that could be enabled by leveraging the unique cell-array architecture and operating principles of NAND flash memory; (ii) it is unreliable because it is not designed to take into account the highly error-prone nature of NAND flash memory. We propose Flash-Cosmos (Flash C omputation with-O ne-S hot M ulti-O perand S ensing), a new in-flash processing technique that significantly increases the performance and energy efficiency of bulk bitwise operations while providing high reliability. Flash-Cosmos introduces two key mechanisms that can be easily supported in modern NAND flash chips: (i) M ulti-W ordline S ensing (MWS), which enables bulk bitwise operations on a large number of operands (tens of operands) with a single sensing operation, and (ii) E nhanced S LC-mode P rogramming (ESP), which enables reliable computation inside NAND flash memory. We demonstrate the feasibility of performing bulk bitwise operations with high reliability in Flash-Cosmos by testing 160 real 3D NAND flash chips. Our evaluation shows that Flash-Cosmos improves average performance and energy efficiency by $3.5 \times /32 \times$ and $3.3 \times /95 \times$, respectively, over the state-of-the-art in-flash/outside-storage processing techniques across three real-world applications.
Hyperbolic structures on a toric arrangement complement

This thesis studies the geometric structures on toric arrangement complements. Inspired by the special hypergeometric functions associated with a root system, we consider a family of connections on a total space which is the product of the complement of a toric arrangement (=finite union of hypertori) and $\mathbb{C}^{\times}$. We prove that these connections are torsion free and flat, and hence define a family of projective structures on the toric arrangement complement. We then further show that the projective structure underlies a hyperbolic structure when the multiplicity parameter lies in some certain region so that the complement admits a complex ball structure. This phenomenon could also be viewed as the geometry of the (trigonometric) Calogero-Moser system. Moreover, we prove that there also exists a family of Frobenius algebra structures on the total space, which indicates a connection with Gromov-Witten theory.
AGN astrophysics from comparing radio and Gaia optical astrometry

Gaia will open up a huge volume of new parameter space in which to explore the physics of AGN and black hole evolution. We address the question as to how far along the relativistic jets blazar radio, optical and gamma ray emission originated. In some models the optical centroid wander should be detectable as the relative contributions of thermal and non-thermal optical emission change. Black holes powering AGN do not necessarily reside at the centres of their host galaxies; they can be one member of a binary pair or they could have received a kick after binary coalescence. For radio-loud AGN comparison of astrometric radio and and optical positions can reveal such displacements. It is suggested that it would be feasible to do this using Gaia and e-MERLIN for a sample of thousands of elliptical radio galaxies.