Miracle-dz/UniFAR-FTUs huggingface dataset:
High-Dimensional Censored Regression via the Penalized Tobit Likelihood

Abstract High-dimensional regression and regression with a left-censored response are each well-studied topics. In spite of this, few methods have been proposed which deal with both of these complications simultaneously. The Tobit model—long the standard method for censored regression in economics—has not been adapted for high-dimensional regression at all. To fill this gap and bring up-to-date techniques from high-dimensional statistics to the field of high-dimensional left-censored regression, we propose several penalized Tobit models. We develop a fast algorithm which combines quadratic majorization with coordinate descent to compute the penalized Tobit solution path. Theoretically, we analyze the Tobit lasso and Tobit with a folded concave penalty, bounding the l2 estimation loss for the former and proving that a local linear approximation estimator for the latter possesses the strong oracle property. Through an extensive simulation study, we find that our penalized Tobit models provide more accurate predictions and parameter estimates than other methods on high-dimensional left-censored data. We use a penalized Tobit model to analyze high-dimensional left-censored HIV viral load data from the AIDS Clinical Trials Group and identify potential drug resistance mutations in the HIV genome. A supplementary file contains intermediate theoretical results and technical proofs.
Audio-visual speech separation based on joint feature representation with cross-modal attention

Multi-modal based speech separation has exhibited a specific advantage on isolating the target character in multi-talker noisy environments. Unfortunately, most of current separation strategies prefer a straightforward fusion based on feature learning of each single modality, which is far from sufficient consideration of inter-relationships between modalites. Inspired by learning joint feature representations from audio and visual streams with attention mechanism, in this study, a novel cross-modal fusion strategy is proposed to benefit the whole framework with semantic correlations between different modalities. To further improve audio-visual speech separation, the dense optical flow of lip motion is incorporated to strengthen the robustness of visual representation. The evaluation of the proposed work is performed on two public audio-visual speech separation benchmark datasets. The overall improvement of the performance has demonstrated that the additional motion network effectively enhances the visual representation of the combined lip images and audio signal, as well as outperforming the baseline in terms of all metrics with the proposed cross-modal fusion.
Security of a High Dimensional Two‐Way Quantum Key Distribution Protocol

Two‐way quantum key distribution (QKD) protocols utilize bi‐directional quantum communication to establish a shared secret key. Due to the increased attack surface, security analyses remain challenging. Here a high‐dimensional variant of the Ping Pong protocol is investigated and an information theoretic security analysis in the finite‐key setting is performed. The main contribution in this work is to show a new proof methodology for two‐way quantum key distribution protocols based on the quantum sampling framework of Bouman and Fehr introduced in 2010 and also sampling‐based entropic uncertainty relations introduced by the authors in 2019. The Ping Pong protocol is only investigated here, but these methods may be broadly applicable to other QKD protocols, especially those relying on two‐way channels. Along the way, some fascinating benefits to high‐dimensional quantum states applied to two‐way quantum communication are also showed.
Metric density results for the value distribution of Sudler products

This product was first studied by Erdös and Szekeres [12]. Later, Sudler products appeared in many different areas of mathematics that include, among others, Zagier’s quantum modular forms and hyperbolic knots in algebraic topology [3, 8, 24], restricted partition functions [23], KAM theory [17] and Padé approximants [18]. Furthermore, they were used in the solution of the Ten Martini Problem [5]. Note that by 1 – periodicity of PN(α) and the fact that PN(α) = 0 for rational α and N sufficiently large, it suffices to consider irrational numbers α ∈ [0, 1].
Dynamic ConvNets on Tiny Devices via Nested Sparsity

This work introduces a new training and compression pipeline to build nested sparse convolutional neural networks (ConvNets), a class of dynamic ConvNets suited for inference tasks deployed on resource-constrained devices at the edge of the Internet of Things. A nested sparse ConvNet consists of a single ConvNet architecture, containing $N$ sparse subnetworks with nested weights subsets, like a Matryoshka doll, and can trade accuracy for latency at runtime, using the model sparsity as a dynamic knob. To attain high accuracy at training time, we propose a gradient masking technique that optimally routes the learning signals across the nested weight subsets. To minimize the storage footprint and efficiently process the obtained models at inference time, we introduce a new sparse matrix compression format with dedicated compute kernels that fruitfully exploit the characteristic of the nested weights subsets. Tested on image classification and object detection tasks on an off-the-shelf ARM-M7 microcontroller unit (MCU), nested sparse ConvNets outperform variable-latency solutions naively built assembling single sparse models trained as stand-alone instances, achieving 1) comparable accuracy; 2) remarkable storage savings; and 3) high performance. Moreover, when compared to state-of-the-art dynamic strategies, such as dynamic pruning and layer width scaling, nested sparse ConvNets turn out to be Pareto optimal in the accuracy versus latency space.
Towards Unbiased Multi-Label Zero-Shot Learning With Pyramid and Semantic Attention

Multi-label zero-shot learning extends conventional single-label zero-shot learning to a more realistic scenario that aims at recognizing multiple unseen labels of classes for each input sample. Existing works usually exploit attention mechanism to generate the correlation among different labels. However, most of them are usually biased on several <italic>major classes</italic> while neglect most of the <italic>minor classes</italic> with the same importance in input samples, and may thus result in overly diffused attention maps that cannot sufficiently cover <italic>minor classes</italic>. We argue that disregarding the connection between major and minor classes, i.e., correspond to the global and local information, respectively, is the cause of the problem. In this paper, we propose a novel framework of unbiased multi-label zero-shot learning, by considering various class-specific regions to calibrate the training process of the classifier. Specifically, <italic>Pyramid Feature Attention</italic> (<italic>PFA</italic>) is proposed to build the correlation between global and local information of samples to balance the presence of each class. Meanwhile, for the generated semantic representations of input samples, we propose <italic>Semantic Attention</italic> (<italic>SA</italic>) to strengthen the element-wise correlation among these vectors, which can encourage the coordinated representation of them. Extensive experiments on the large-scale multi-label benchmarks <italic>MS-COCO</italic>, <italic>NUS-WIDE</italic> and <italic>Open-Images</italic> demonstrate that the proposed method surpasses other representative methods by significant margins.
Amorphous Boron Nitride: Ab initio Study of its vibrational properties

Boron nitride (BN) is a structurally versatile insulator since it can be found in several crystalline structures with interesting mechanical and electrical properties, making this material attractive for technological applications. Seeking to improve its features, experimental and simulational studies for the amorphous phase (a-BN) have been carried out by some groups, focusing on the electrical and structural properties, pressure-induced phase transformations, and hydrogenated a-BN. In this work two amorphous structures are computationally generated and studied using ab initio Molecular Dynamics on a 216-atom supercell with two different densities, 2.04 and 2.80 g cm. Our undermelt-quench approach is followed, since it has proven to give good structures for disordered materials and their properties. The topology, the vibrational density of states and some thermodynamic properties of the two samples are reported and compared with existing experiment. Some computational results are also revisited.
Comprehensive Review of Deep Learning-Based 3D Point Cloud Completion Processing and Analysis

Point cloud completion is a generation and estimation issue derived from the partial point clouds, which plays a vital role in the applications of 3D computer vision. The progress of deep learning (DL) has impressively improved the capability and robustness of point cloud completion. However, the quality of completed point clouds is still needed to be further enhanced to meet the practical utilization. Therefore, this work aims to conduct a comprehensive survey on various methods, including point-based, view-based, convolution-based, graph-based, generative model-based, transformer-based approaches, etc. And this survey summarizes the comparisons among these methods to provoke further research insights. Besides, this review sums up the commonly used datasets and illustrates the applications of point cloud completion. Eventually, we also discussed possible research trends in this promptly expanding field.
Floquet engineering the band structure of materials with optimal control theory

We demonstrate that the electronic structure of a material can be deformed into Floquet pseudobands with arbitrarily tailored shapes. We achieve this goal with a combination of quantum optimal control theory and Floquet engineering. The power and versatility of this framework is demonstrated here by utilizing the independent-electron tight-binding description of the π electronic system of graphene. We show several prototype examples focusing on the region around the K (Dirac) point of the Brillouin zone: creation of a gap with opposing ﬂat valence and conduction bands, creation of a gap with opposing concave symmetric valence and conduction bands (which would correspond to a material with an effective negative electron-hole mass), and closure of the gap when departing from a modiﬁed graphene model with a nonzero ﬁeld-free gap. We employ time-periodic drives with several frequency components and polarizations, in contrast to the usual monochromatic ﬁelds, and use control theory to ﬁnd the amplitudes of each component that optimize the shape of the bands as desired. In addition, we use quantum control methods to ﬁnd realistic switch-on pulses that bring the material into the predeﬁned stationary Floquet band structure, i.e., into a state in which the desired Floquet modes of the target bands are fully occupied, so that they should remain stroboscopically stationary, with long lifetimes, when the weak periodic drives are started. Finally, we note that although we have focused on solid state materials, the technique that we propose could be equally used for the Floquet engineering of ultracold atoms in optical lattices and for other nonequilibrium dynamical and correlated systems.
Algorithmic Regularization in Model-free Overparametrized Asymmetric Matrix Factorization

We study the asymmetric matrix factorization problem under a natural nonconvex formulation with arbitrary overparametrization. The model-free setting is considered, with minimal assumption on the rank or singular values of the observed matrix, where the global optima provably overfit. We show that vanilla gradient descent with small random initialization sequentially recovers the principal components of the observed matrix. Consequently, when equipped with proper early stopping, gradient descent produces the best low-rank approximation of the observed matrix without explicit regularization. We provide a sharp characterization of the relationship between the approximation error, iteration complexity, initialization size and stepsize. Our complexity bound is almost dimension-free and depends logarithmically on the approximation error, with significantly more lenient requirements on the stepsize and initialization compared to prior work. Our theoretical results provide accurate prediction for the behavior gradient descent, showing good agreement with numerical experiments.
Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer

Hyperparameter (HP) tuning in deep learning is an expensive process, prohibitively so for neural networks (NNs) with billions of parameters. We show that, in the recently discovered Maximal Update Parametrization (muP), many optimal HPs remain stable even as model size changes. This leads to a new HP tuning paradigm we call muTransfer: parametrize the target model in muP, tune the HP indirectly on a smaller model, and zero-shot transfer them to the full-sized model, i.e., without directly tuning the latter at all. We verify muTransfer on Transformer and ResNet. For example, 1) by transferring pretraining HPs from a model of 13M parameters, we outperform published numbers of BERT-large (350M parameters), with a total tuning cost equivalent to pretraining BERT-large once; 2) by transferring from 40M parameters, we outperform published numbers of the 6.7B GPT-3 model, with tuning cost only 7% of total pretraining cost. A Pytorch implementation of our technique can be found at github.com/microsoft/mup and installable via `pip install mup`.
Negative-Weight Single-Source Shortest Paths in Almost-linear Time

We present a randomized algorithm that computes single-source shortest paths (SSSP) in m 1+ o (1) log W time when edge weights are integral and can be negative. 1 This essentially re-solves classic negative-weight SSSP problem. The previous bounds are ˜ O (( m 1 + n 1 . 5 ) log W ) [BLNPSSSW FOCS’20] and m 4 / 3+ o (1) log W [AMV FOCS’20]. In contrast to all recent developments that rely on sophisticated continuous optimization methods and dynamic algorithms, our algorithm is based on a simple graph decomposition and elementary combinatorial tools. In fact, ours is the ﬁrst combinatorial algorithm for negative-weight SSSP to break through the classic ˜ O ( m √ n log W ) bound from over three decades ago [Gabow and Tarjan SICOMP’89]. Beside being combinatorial, an important feature of our algorithm is in its simplicity: treating our graph decomposition as a black-box, we believe that the reader can reconstruct our algorithm and analysis from our 6-page overview. Independent result. Independently from our result, the recent major breakthrough by Chen, Kyng, Liu, Peng, Gutenberg, and Sachdeva [CKL + 22] achieve an almost-linear time bound for min-cost ﬂow, implying the same bound for our problem. We discuss this result at the end of the introduction.
GeoDiff: a Geometric Diffusion Model for Molecular Conformation Generation

Predicting molecular conformations from molecular graphs is a fundamental problem in cheminformatics and drug discovery. Recently, significant progress has been achieved with machine learning approaches, especially with deep generative models. Inspired by the diffusion process in classical non-equilibrium thermodynamics where heated particles will diffuse from original states to a noise distribution, in this paper, we propose a novel generative model named GeoDiff for molecular conformation prediction. GeoDiff treats each atom as a particle and learns to directly reverse the diffusion process (i.e., transforming from a noise distribution to stable conformations) as a Markov chain. Modeling such a generation process is however very challenging as the likelihood of conformations should be roto-translational invariant. We theoretically show that Markov chains evolving with equivariant Markov kernels can induce an invariant distribution by design, and further propose building blocks for the Markov kernels to preserve the desirable equivariance property. The whole framework can be efficiently trained in an end-to-end fashion by optimizing a weighted variational lower bound to the (conditional) likelihood. Experiments on multiple benchmarks show that GeoDiff is superior or comparable to existing state-of-the-art approaches, especially on large molecules.
Targeted Data Poisoning Attack on News Recommendation System by Content Perturbation

News Recommendation System(NRS) has become a fundamental technology to many online news services. Meanwhile, several studies show that recommendation systems(RS) are vulnerable to data poisoning attacks, and the attackers have the ability to mislead the system to perform as their desires. A widely studied attack approach, injecting fake users, can be applied on the NRS when the NRS is treated the same as the other systems whose items are fixed. However, in the NRS, as each item (i.e. news) is more informative, we propose a novel approach to poison the NRS, which is to perturb contents of some browsed news that results in the manipulation of the rank of the target news. Intuitively, an attack is useless if it is highly likely to be caught, i.e., exposed. To address this, we introduce a notion of the exposure risk and propose a novel problem of attacking a history news dataset by means of perturbations where the goal is to maximize the manipulation of the target news rank while keeping the risk of exposure under a given budget. We design a reinforcement learning framework, called TDP-CP, which contains a two-stage hierarchical model to reduce the searching space. Meanwhile, influence estimation is also applied to save the time on retraining the NRS for rewards. We test the performance of TDP-CP under three NRSs and on different target news. Our experiments show that TDP-CP can increase the rank of the target news successfully with a limited exposure budget.
Choice and Attention across Time

I study how past and future choices are linked in the framework of attention. Attention cannot be observed but past choices are necessarily considered in future decisions. This link connects two types of rationality violations, counterfactual and realized, where the former results from inattention and the latter fully pins down preferences. Results show that the necessary traces of limited attention lie within choice sequences because they enable and compel a decision maker to correct their"mistakes". The framework accommodates different attention structures and extends to framing, introducing choice sequences as an important channel to formulate, identify, and scrutinize limited attention.
Twisted Iwasawa invariants of knots

Let p be a prime number and m an integer coprime to p. In the spirit of arithmetic topology, we introduce the notions of the twisted Iwasawa invariants of GLN‐representations and ‐covers of knots. We prove among other things that the set of Iwasawa invariants determines the genus and the fiberedness of a knot, yielding their profinite rigidity. Several intuitive examples are attached. We further prove the theorem for SL2‐representations of twist knot groups and give some remarks.
The Unsurprising Effectiveness of Pre-Trained Vision Models for Control

Recent years have seen the emergence of pretrained representations as a powerful abstraction for AI applications in computer vision, natural language, and speech. However, policy learning for control is still dominated by a tabula-rasa learning paradigm, with visuo-motor policies often trained from scratch using data from deployment environments. In this context, we revisit and study the role of pre-trained visual representations for control, and in particular representations trained on large-scale computer vision datasets. Through extensive empirical evaluation in diverse control domains (Habitat, DeepMind Control, Adroit, Franka Kitchen), we isolate and study the importance of different representation training methods, data augmentations, and feature hierarchies. Overall, we find that pre-trained visual representations can be competitive or even better than ground-truth state representations to train control policies. This is in spite of using only out-of-domain data from standard vision datasets, without any in-domain data from the deployment environments.
Nonthermal Radio Continuum Emission from Young Nearby Stars

Young and magnetically active low-mass stars often exhibit nonthermal coronal radio emission owing to the gyration of electrons in their magnetized chromospheres. This emission is easily detectable at centimeter wavelengths with the current sensitivity of large radio interferometers like the Very Large Array (VLA). With the aim of identifying nearby stars adequate for future accurate radio astrometric monitoring using very long baseline interferometry (VLBI), we have used the VLA in its B configuration to search for radio emission at ν ≃ 6 GHz (λ ≃ 5 cm) toward a sample of 170 nearby (<130 pc), mostly young (5–500 Myr) stars of spectral types between F4 and M2. At our mean 3σ detection limit of ≃50 μJy, we identify 31 young stars with coronal radio emission (an 18% system detection rate) and more than 600 background (most likely extragalactic) sources. Among the targeted stars, we find a significant decline of the detection rate with age from 56% ± 20% for stars with ages ≤10 Myr to 10% ± 3% for stars with ages 100–200 Myr. No star older than 200 Myr was detected. The detection rate also declines with T eff from 36% ± 10% for stars with T eff < 4000 K to 13% ± 3% for earlier spectral types with T eff > 5000 K. The binarity fraction among the radio-bright stars is at least twice as high as among the radio-quiet stars. The radio-bright nearby young stars identified here provide an interesting sample for future astrometric studies using VLBI arrays aimed at searching for hitherto-unknown tight binary components or even exoplanets.
Why Constant-Composition Codes Reduce Nonlinear Interference Noise

A time-domain perturbation model of the nonlinear Schrödinger equation is used to study wave-length-division multiplexed communication over a single polarization. The model explains (a) why constant-composition codes offer an improvement in signal-to-noise ratio compared with independent and uniform selection of constellation points and (b) why similar gains are obtained using carrier recovery algorithms even without using constant-composition codes.
Compatibility of quantum instruments

Incompatibility of quantum devices is a useful resource in various quantum information theoretical tasks, and it is at the heart of some fundamental features of quantum theory. While the incompatibility of measurements and quantum channels is well-studied, the incompatibility of quantum instruments has not been explored in much detail. In this work, we revise a notion of instrument compatibility introduced in the literature that we call traditional compatibility. Then, we introduce the new notion of parallel compatibility, and show that these two notions are inequivalent. Then, we argue that the notion of traditional compatibility is incomplete, and prove that while parallel compatibility captures measurement and channel compatibility, traditional compatibility does not. Hence, we propose parallel compatibility as the conceptually complete definition of compatibility of quantum instruments.