Miracle-dz/UniFAR-FTUs huggingface dataset:
EFloat: Entropy-coded Floating Point Format for Compressing Vector Embedding Models

In a large class of deep learning models, including vector embedding models such as word and database embeddings, we observe that floating point exponent values cluster around a few unique values, permitting entropy based data compression. Entropy coding compresses fixed-length values with variable-length codes, encoding most probable values with fewer bits. We propose the EFloat compressed floating point number format that uses a variable field boundary between the exponent and significand fields. EFloat uses entropy coding on exponent values and signs to minimize the average width of the exponent and sign fields, while preserving the original FP32 exponent range unchanged. Saved bits become part of the significand field increasing the EFloat numeric precision by 4.3 bits on average compared to other reduced-precision floating point formats. EFloat makes 8-bit and even smaller floats practical without sacrificing the exponent range of a 32-bit floating point representation. We currently use the EFloat format for saving memory capacity and bandwidth consumption of large vector embedding models such as those used for database embeddings. Using the RMS error as metric, we demonstrate that EFloat provides higher accuracy than other floating point formats with equal bit budget. The EF12 format with 12-bit budget has less end-to-end application error than the 16-bit BFloat16. EF16 with 16-bit budget has an RMS-error 17 to 35 times less than BF16 RMS-error for a diverse set of embedding models. When making similarity and dissimilarity queries, using the NDCG ranking metric, EFloat matches the result quality of prior floating point representations with larger bit budgets.
Multi-Task Learning as a Bargaining Game

In Multi-task learning (MTL), a joint model is trained to simultaneously make predictions for several tasks. Joint training reduces computation costs and improves data efficiency; however, since the gradients of these different tasks may conflict, training a joint model for MTL often yields lower performance than its corresponding single-task counterparts. A common method for alleviating this issue is to combine per-task gradients into a joint update direction using a particular heuristic. In this paper, we propose viewing the gradients combination step as a bargaining game, where tasks negotiate to reach an agreement on a joint direction of parameter update. Under certain assumptions, the bargaining problem has a unique solution, known as the Nash Bargaining Solution, which we propose to use as a principled approach to multi-task learning. We describe a new MTL optimization procedure, Nash-MTL, and derive theoretical guarantees for its convergence. Empirically, we show that Nash-MTL achieves state-of-the-art results on multiple MTL benchmarks in various domains.
New Insights on Target Speaker Extraction

Speaker extraction (SE) aims to segregate the speech of a target speaker from a mixture of interfering speakers with the help of auxiliary information. Several forms of auxiliary information have been employed in single-channel SE, such as a speech snippet enrolled from the target speaker or visual information corresponding to the spoken utterance. The effectiveness of the auxiliary information in SE is typically evaluated by comparing the extraction performance of SE with uninformed speaker separation (SS) methods. Following this evaluation protocol, many SE studies have reported performance improvement compared to SS, attributing this to the auxiliary information. However, such studies have been conducted on a few datasets and have not considered recent deep neural network architectures for SS that have shown impressive separation performance. In this paper, we examine the role of the auxiliary information in SE for different input scenarios and over multiple datasets. Specifically, we compare the performance of two SE systems (audio-based and video-based) with SS using a common framework that utilizes the recently proposed dual-path recurrent neural network as the main learning machine. Experimental evaluation on various datasets demonstrates that the use of auxiliary information in the considered SE systems does not always lead to better extraction performance compared to the uninformed SS system. Furthermore, we offer insights into the behavior of the SE systems when provided with different and distorted auxiliary information given the same mixture input.
A Bayesian Analysis of Physical Parameters for 783 Kepler Close Binaries: Extreme-mass-ratio Systems and a New Mass Ratio versus Period Lower Limit

Contact binary star systems represent the long-lived penultimate phase of binary evolution. Population statistics of their physical parameters inform an understanding of binary evolutionary pathways and end products. We use light curves and new optical spectroscopy to conduct a pilot study of ten (near) contact systems in the long-period (P > 0.5 days) tail of close binaries in the Kepler field. We use PHOEBE light-curve models to compute Bayesian probabilities on five principal system parameters. Mass ratios and third-light contributions measured from spectra agree well with those inferred from the light curves. Pilot study systems have extreme mass ratios q < 0.32. Most are triples. Analysis of the unbiased sample of 783 0.15 d < P < 2 days (near) contact binaries results in 178 probable contact systems, 114 probable detached systems, and 491 ambiguous systems for which we report best-fitting and 16th-/50th-/84th-percentile parameters. Contact systems are rare at periods P > 0.5 days, as are systems with q > 0.8. There exists an empirical mass ratio lower limit qmin(P) ≈ 0.05–0.15 below which contact systems are absent, supporting a new set of theoretical predictions obtained by modeling the evolution of contact systems under the constraints of mass and angular momentum conservation. Premerger systems should lie at long periods and near this mass ratio lower limit, which rises from q = 0.044 for P = 0.74 days to q = 0.15 at P = 2.0 days. These findings support a scenario whereby nuclear evolution of the primary (more massive) star drives mass transfer to the primary, thus moving systems toward extreme q and larger P until the onset of the Darwin instability at qmin precipitates a merger.
How an $\omega_0$ condensate can spike the speed of sound in cold quarkyonic matter

I describe a novel mechanism where the variation of an $\omega_0$ condensate can generate a"spike"in the speed of sound in hadronic matter. An $\omega_0$ condensate naturally increases the speed of sound; the real problem is how to get the speed of sound to decrease. I suggest this can happen through the appearance of a Quantum Pion Liquid.
Transfer in Reinforcement Learning via Regret Bounds for Learning Agents

We present an approach for the quantification of the usefulness of transfer in reinforcement learning via regret bounds for a multi-agent setting. Considering a number of $\aleph$ agents operating in the same Markov decision process, however possibly with different reward functions, we consider the regret each agent suffers with respect to an optimal policy maximizing her average reward. We show that when the agents share their observations the total regret of all agents is smaller by a factor of $\sqrt{\aleph}$ compared to the case when each agent has to rely on the information collected by herself. This result demonstrates how considering the regret in multi-agent settings can provide theoretical bounds on the benefit of sharing observations in transfer learning.
THE 3 -DIMENSIONAL COMPLEX PROJECTIVE SPACE ADMITS NO SPECIAL GENERIC MAPS

The main theorem of the present paper is that the 3-dimensional complex projective space does not admit special generic maps . Special generic maps are generalized versions of Morse functions on spheres with exactly two singular points. The canonical projections of unit spheres are of the class. The paper mainly focuses on special generic maps on 6-dimensional closed and simply-connected manifolds. The diﬀerentiable structures of spheres admitting special generic maps are known to be restricted strongly in general. Special generic maps on closed and simply-connected manifolds and projective spaces have been studied by various people including the author. The existence or non-existence and construction are main problems. Studies on such maps on closed and simply-connected manifolds whose dimensions are greater than 5 have been diﬃcult.
Improving Screening Processes via Calibrated Subset Selection

Many selection processes such as finding patients qualifying for a medical trial or retrieval pipelines in search engines consist of multiple stages, where an initial screening stage focuses the resources on shortlisting the most promising candidates. In this paper, we investigate what guarantees a screening classifier can provide, independently of whether it is constructed manually or trained. We find that current solutions do not enjoy distribution-free theoretical guarantees -- we show that, in general, even for a perfectly calibrated classifier, there always exist specific pools of candidates for which its shortlist is suboptimal. Then, we develop a distribution-free screening algorithm -- called Calibrated Subset Selection (CSS) -- that, given any classifier and some amount of calibration data, finds near-optimal shortlists of candidates that contain a desired number of qualified candidates in expectation. Moreover, we show that a variant of CSS that calibrates a given classifier multiple times across specific groups can create shortlists with provable diversity guarantees. Experiments on US Census survey data validate our theoretical results and show that the shortlists provided by our algorithm are superior to those provided by several competitive baselines.
FASTER EXACT PERMUTATION TESTING: USING A REPRESENTATIVE SUBGROUP

Non-parametric tests based on permutation, rotation or sign-ﬂipping are examples of so-called group-invariance tests. These tests rely on invariance of the null distribution under a set of transformations that has a group structure, in the algebraic sense. Such groups are often huge, which makes it computationally infeasible to use the entire group. Hence, it is standard practice to test using a randomly sampled set of transformations from the group. This random sample still needs to be substantial to obtain good power and replicability. We improve upon the standard practice by using a well-designed subgroup of transformations instead of a random sample. We show this can yield a more powerful and fully replicable test with the same number of transformations. For a normal location model and a particular design of the subgroup, we show that the power improvement is equivalent to the power difference between a Monte Carlo Z -test and Monte Carlo t -test. In our simulations, we ﬁnd that we can obtain the same power as a test based on sampling with just half the number of transformations, or equivalently, more power for the same computation time. These beneﬁts come entirely ‘for free’, as our methodology relies on an assumption of invariance under the subgroup, which is implied by invariance under the entire group.
A Versatile Dataset of Agile Open Source Software Projects

Agile software development is nowadays a widely adopted practise in both open-source and industrial software projects. Agile teams typically heavily rely on issue management tools to document new issues and keep track of outstanding ones, in addition to storing their technical details, effort estimates, assignment to developers, and more. Previous work utilised the historical information stored in issue management systems for various purposes; however, when researchers make their empirical data public, it is usually relevant solely to the study's objective. In this paper, we present a more holistic and versatile dataset containing a wealth of information on more than half a million issues from 44 open-source Agile software, making it well-suited to several research avenues, and cross-analyses therein, including effort estimation, issue prioritization, issue assignment and many more. We make this data publicly available on GitHub to facilitate ease of use, maintenance, and extensibility.
A Recommender System Based on a Double Feature Allocation Model

A collaborative filtering recommender system predicts user preferences by discovering common features among users and items. We implement such inference using a Bayesian double feature allocation model, that is, a model for random pairs of subsets. We use an Indian buffet process (IBP) to link users and items to features. Here a feature is a subset of users and a matching subset of items. By training feature-specific rating effects, we predict ratings. We use MovieLens Data to demonstrate posterior inference in the model and prediction of user preferences for unseen items compared to items they have previously rated. Part of the implementation is a novel semi-consensus Monte Carlo method to accomodate large numbers of users and items, as is typical for related applications. The proposed approach implements parallel posterior sampling in multiple shards of users while sharing item-related global parameters across shards.
Context Uncertainty in Contextual Bandits with Applications to Recommender Systems

Recurrent neural networks have proven effective in modeling sequential user feedbacks for recommender systems. However, they usually focus solely on item relevance and fail to effectively explore diverse items for users, therefore harming the system performance in the long run. To address this problem, we propose a new type of recurrent neural networks, dubbed recurrent exploration networks (REN), to jointly perform representation learning and effective exploration in the latent space. REN tries to balance relevance and exploration while taking into account the uncertainty in the representations. Our theoretical analysis shows that REN can preserve the rate-optimal sublinear regret even when there exists uncertainty in the learned representations. Our empirical study demonstrates that REN can achieve satisfactory long-term rewards on both synthetic and real-world recommendation datasets, outperforming state-of-the-art models.
Image Forgery Detection with Interpretability

In this work, we present a learning based method focusing on the convolutional neural network (CNN) architecture to detect these forgeries. We consider the detection of both copy-move forgeries and inpainting based forgeries. For these, we synthesize our own large dataset. In addition to classification, the focus is also on interpretability of the forgery detection. As the CNN classification yields the image-level label, it is important to understand if forged region has indeed contributed to the classification. For this purpose, we demonstrate using the Grad-CAM heatmap, that in various correctly classified examples, that the forged region is indeed the region contributing to the classification. Interestingly, this is also applicable for small forged regions, as is depicted in our results. Such an analysis can also help in establishing the reliability of the classification.
Testing Lorentz Invariance with Neutrinos

The search for a theory that unifies general relativity and quantum theory has focused attention on models of physics at the Planck scale. One possible consequence of models such as string theory may be that Lorentz invariance is not an exact symmetry of nature. We discuss here some possible experimental and observational tests of Lorentz invariance involving neutrino physics and astrophysics.
Time delay interferometry without clock synchronisation

Time-delay interferometry (TDI) is a data processing technique for LISA designed to suppress the otherwise overwhelming laser noise by several orders of magnitude. It is widely believed that TDI can only be applied once all phase or frequency measurements from each spacecraft have been synchronized to a common time frame. We demonstrate analytically, using as an example the commonly-used Michelson combination X , that TDI can be computed using the raw, unsynchronized data, thereby avoiding the need for an initial synchronization processing step and signiﬁcantly simplifying the initial noise reduction pipeline. Furthermore, the raw data is free of any potential artifacts introduced by clock synchronization and reference frame transformation algorithms, which allows to operate directly on the MHz beatnotes. As a consequence, in-band clock noise is directly suppressed as part of TDI, in contrast to the approach previously proposed in the literature (in which large trends in the beatnotes are removed before the main laser-noise reduction step, and clock noise is suppressed in an extra processing step). We validate our algorithm with full-scale numerical simulations that use LISA Instrument and PyTDI and show that we reach the same performance levels as the previously proposed methods, ultimately limited by the clock sideband stability.
Quantum Remote Entanglement for Medium-Free Secure Communication?

Present-day quantum communication predominantly depends on trusted relays (e.g., quantum repeaters, low-Earth-orbit satellite) connected by optical fiber cables to transmit information. However, recent evidence supports a decades-old concept that quantum entanglement, harnessed by current quantum communication systems, does not necessarily rely on a physical relay medium. In modern quantum communication networks, this trusted relay infrastructure is (1) susceptible to security attacks, (2) limited by the channel capacity, (3) subject to decoherence loss, and (4) expensive to set up. The instantaneous and faster-than-light activities of quantum entanglement occurring in quantum communication have suggested guidance by some non-locality nature. On the contrary, neither ground nor space-relays have shown or been demonstrated to embody it. It is proposed in this paper that the non-locality nature of quantum theory governs quantum entanglement; elementary particles, components of a universal quantum body, can achieve remote entanglement regardless of a physical medium or spatial proximity. Evidence and theory supporting remote entanglement in superconducting quantum systems (entanglement fidelities for communication in particular) are presented. One such particle, the photon, representing a basic unit of quantum information, qubit |ψ ⟩ = α |0⟩ + β |1⟩, consists of real continuous values in complex numbers (α, β) with infinite precision. These values (α, β) can account for the distinctiveness of qubits and result in an identityQuID that possibly supports remote entanglement. New approaches to medium-free secure quantum communication are suggested by running simulations and actual quantum computations on a quantum circuit.
When do Minimum Wages Increase Social Welfare? A Sufficient Statistics Analysis with Taxes and Transfers∗

This paper characterizes optimal redistribution for a social planner with three instruments: labor income taxes and transfers, corporate income taxes, and a minimum wage. The modeled economy features search-and-matching frictions, generates positive firm profits in equilibrium, and accommodates empirically relevant effects of the minimum wage such as wage spillovers and reallocation effects. I find that minimum wages are more likely to be desirable when corporate taxes are low because minimum wages generate corporate revenue losses and redistribute from capitalists to low-skill workers. Minimum wages can improve welfare even under optimal income taxes by shifting tax incidence when wages bunch at the minimum. I estimate the sufficient statistics that guide the welfare analysis using state-level variation in minimum wages. I find that minimum wages have increased low-skill workers’ welfare with null effects on high-skill workers and negative effects on capitalists. Results suggest that, under current corporate tax rates, weak social preferences for redistribution toward low-skill workers would justify small state-level minimum wages increases. Email: damianvergara@berkeley.edu. I am especially grateful to Pat Kline, Emmanuel Saez, and Danny Yagan, for continuous guidance and encouragement. I also thank Alan Auerbach, Sydnee Caldwell, Patricio Domı́nguez, Cecile Gaubert, Benjamin Glass, Andrés González, Attila Lindner, Cristóbal Otero, Pablo Muñoz, Marco Rojas, Harry Wheeler, Gabriel Zucman, and seminar participants at the NTA 114th Annual Conference on Taxation, the XIII RIDGE Forum Workshop on Public Economics, UC Berkeley, and Universidad Adolfo Ibáñez for very helpful discussions and suggestions. Early versions of this project benefited from discussions with Jakob Brounstein, Sree Kancherla, Maximiliano Lauletta, Michael Love, Billy Morrison, and Mónica Saucedo. I acknowledge financial support from the Center of Equitable Growth at UC Berkeley. Usual disclaimers apply. ar X iv :2 20 2. 00 83 9v 1 [ ec on . G N ] 2 F eb 2 02 2 “The simplest expedient which can be imagined for keeping the wages of labor up to the desirable point would be to fix them by law; the ground of decision being, not the state of the labor market, but natural equity; to provide that the workmen shall have reasonable wages, and the capitalist reasonable profits. ” John Stuart Mill, Principles of Political Economy, 1884.
Eikonal Fields for Refractive Novel-View Synthesis

We tackle the problem of generating novel-view images from collections of 2D images showing refractive and reflective objects. Current solutions assume opaque or transparent light transport along straight paths following the emission-absorption model. Instead, we optimize for a field of 3D-varying index of refraction (IoR) and trace light through it that bends toward the spatial gradients of said IoR according to the laws of eikonal light transport.
From Predictions to Decisions: The Importance of Joint Predictive Distributions

A fundamental challenge for any intelligent system is prediction: given some inputs, can you predict corresponding outcomes? Most work on supervised learning has focused on producing accurate marginal predictions for each input. However, we show that for a broad class of decision problems, accurate joint predictions are required to deliver good performance. In particular, we establish several results pertaining to combinatorial decision problems, sequential predictions, and multi-armed bandits to elucidate the essential role of joint predictive distributions. Our treatment of multi-armed bandits introduces an approximate Thompson sampling algorithm and analytic techniques that lead to a new kind of regret bound.
Learning Linear Polytree Structural Equation Models

We are interested in the problem of learning the directed acyclic graph (DAG) when data are generated from a linear structural equation model (SEM) and the causal structure can be characterized by a polytree. Under the Gaussian polytree models, we study sufficient conditions on the sample sizes for the well-known Chow-Liu algorithm to exactly recover both the skeleton and the equivalence class of the polytree, which is uniquely represented by a CPDAG. On the other hand, necessary conditions on the required sample sizes for both skeleton and CPDAG recovery are also derived in terms of information-theoretic lower bounds, which match the respective sufficient conditions and thereby give a sharp characterization of the difficulty of these tasks. We also consider extensions to the sub-Gaussian case, and then study the estimation of the inverse correlation matrix under such models. Our theoretical findings are illustrated by comprehensive numerical simulations, and experiments on benchmark data also demonstrate the robustness of polytree learning when the true graphical structures can only be approximated by polytrees.