Miracle-dz/UniFAR-FTUs huggingface dataset:
Robust Neural Networks Outperform Attitude Estimation Filters

—Inertial-sensor-based attitude estimation is a crucial technology in various applications, from human motion tracking to autonomous aerial and ground vehicles. Application scenarios differ in characteristics of the performed motion, presence of disturbances, and environmental conditions. Since state-of-the-art attitude estimators do not generalize well over these characteristics, their parameters must be tuned for the individual motion characteristics and circumstances. We propose RIANN , a real-time-capable neural network for robust IMU-based attitude estimation, which generalizes well across different motion dynamics, environments, and sampling rates, without the need for application-speciﬁc adaptations. We exploit two publicly available datasets for the method development and the training, and we add four completely different datasets for evaluation of the trained neural network in three different test scenarios with varying practical relevance. Results show that RIANN performs at least as well as state-of-the-art attitude estimation ﬁlters and outperforms them in several cases, even if the ﬁlter is tuned on the very same test dataset itself while RIANN has never seen data from that dataset, from the speciﬁc application, the same sensor hardware, or the same sampling frequency before. RIANN is expected to enable plug-and-play solutions in numerous applications, especially when accuracy is crucial but no ground-truth data is available for tuning or when motion and disturbance characteristics are uncertain. We made RIANN publicly available.
Adaptive Active Learning for Coreference Resolution

Training coreference resolution models require comprehensively labeled data. A model trained on one dataset may not successfully transfer to new domains. This paper investigates an approach to active learning for coreference resolution that feeds discrete annotations to an incremental clustering model. The recent developments in incremental coreference resolution allow for a novel approach to active learning in this setting. Through this new framework, we analyze important factors in data acquisition, like sources of model uncertainty and balancing reading and labeling costs. We explore different settings through simulated labeling with gold data. By lowering the data barrier for coreference, coreference resolvers can rapidly adapt to a series of previously unconsidered domains.
Image Super-Resolution via Iterative Refinement

We present SR3, an approach to image Super-Resolution via Repeated Refinement. SR3 adapts denoising diffusion probabilistic models (Ho et al. 2020), (Sohl-Dickstein et al. 2015) to image-to-image translation, and performs super-resolution through a stochastic iterative denoising process. Output images are initialized with pure Gaussian noise and iteratively refined using a U-Net architecture that is trained on denoising at various noise levels, conditioned on a low-resolution input image. SR3 exhibits strong performance on super-resolution tasks at different magnification factors, on faces and natural images. We conduct human evaluation on a standard 8× face super-resolution task on CelebA-HQ for which SR3 achieves a fool rate close to 50%, suggesting photo-realistic outputs, while GAN baselines do not exceed a fool rate of 34%. We evaluate SR3 on a 4× super-resolution task on ImageNet, where SR3 outperforms baselines in human evaluation and classification accuracy of a ResNet-50 classifier trained on high-resolution images. We further show the effectiveness of SR3 in cascaded image generation, where a generative model is chained with super-resolution models to synthesize high-resolution images with competitive FID scores on the class-conditional 256×256 ImageNet generation challenge.
Interplay of charge noise and coupling to phonons in adiabatic electron transfer between quantum dots

Long-distance transfer of quantum information in architectures based on quantum dot spin qubits will be necessary for their scalability. One way of achieving it is to simply move the electron between two quantum registers. Precise control over the electron shuttling through a chain of tunnel-coupled quantum dots is possible when interdot energy detunings are changed adiabatically. Deterministic character of shuttling is however endangered by coupling of the transferred electron to thermal reservoirs: sources of fluctuations of electric fields, and lattice vibrations. We theoretically analyse how the electron transfer between two quantum dots is affected by electron-phonon scattering, and interaction with sources of $1/f$ and Johnson charge noise in both detuning and tunnel coupling. The electron-phonon scattering turns out to be irrelevant in Si quantum dots, while a competition between the effects of charge noise and Landau-Zener effect leads to an existence of optimal detuning sweep rate, at which probability of leaving the electron behind is minimal. In GaAs quantum dots, on the other hand, coupling to phonons is strong enough to make the phonon-assisted processes of interdot transfer dominate over influence of charge noise. The probability of leaving the electron behind depends then monotonically on detuning sweep rate, and values much smaller than in silicon can be obtained for slow sweeps. However, after taking into account limitations on transfer time imposed by need for preservation of electron's spin coherence, minimal probabilities of leaving the electron behind in both GaAs- and Si-based double quantum dots turn out to be of the same order of magnitude. Bringing them down below $10^{-3}$ requires temperatures $\leq \! 100$ mK and tunnel couplings above $20$ $\mu$eV.
Zero-Shot Cross-lingual Semantic Parsing

Recent work in cross-lingual semantic parsing has successfully applied machine translation to localize parsers to new languages. However, these advances assume access to high-quality machine translation systems and word alignment tools. We remove these assumptions and study cross-lingual semantic parsing as a zero-shot problem, without parallel data (i.e., utterance-logical form pairs) for new languages. We propose a multi-task encoder-decoder model to transfer parsing knowledge to additional languages using only English-logical form paired data and in-domain natural language corpora in each new language. Our model encourages language-agnostic encodings by jointly optimizing for logical-form generation with auxiliary objectives designed for cross-lingual latent representation alignment. Our parser performs significantly above translation-based baselines and, in some cases, competes with the supervised upper-bound.
Bondi-Hoyle-Lyttleton Accretion in a Reactive Medium: Detonation Ignition and a Mechanism for SNe Ia

Detonation initiation in a reactive medium can be achieved by an externally created shock wave. Supersonic flow onto a gravitating center, known as Bondi-Hoyle-Lyttleton (BHL) accretion, is a natural shock wave creating process, but, to our knowledge, a reactive medium has never been considered in the literature. Here, we conduct an order of magnitude analysis to investigate under which conditions the shock-induced reaction zone recouples to the shock front. We derive three semi-analytical criteria for self-sustained detonation ignition. We apply these criteria to the special situation where a primordial black hole (PBH) of asteroid mass traverses a carbon-oxygen white dwarf (WD). Since detonations in carbon-oxygen WDs are supposed to produce normal thermonuclear supernovae (SNe Ia), the observed SN Ia rate constrains the fraction of dark matter (DM) in the form of PBHs as log10(fpbh) < 0.8 log10(mbh/g) − 18 in the range 10 − 10g (10 − 10g) from a conservative (optimistic) analysis. Most importantly, these encounters can account for both the rate and the median explosion mass of normal sub-Chandrasekhar SNe Ia if a significant fraction of DM is in the form of PBHs with mass 10g.
Effect of Post-processing on Contextualized Word Representations

Post-processing of static embedding has been shown to improve their performance on both lexical and sequence-level tasks. However, post-processing for contextualized embeddings is an under-studied problem. In this work, we question the usefulness of post-processing for contextualized embeddings obtained from different layers of pre-trained language models. More specifically, we standardize individual neuron activations using z-score, min-max normalization, and by removing top principal components using the all-but-the-top method. Additionally, we apply unit length normalization to word representations. On a diverse set of pre-trained models, we show that post-processing unwraps vital information present in the representations for both lexical tasks (such as word similarity and analogy) and sequence classification tasks. Our findings raise interesting points in relation to the research studies that use contextualized representations, and suggest z-score normalization as an essential step to consider when using them in an application.
Adaptive Sparse Transformer for Multilingual Translation

Multilingual machine translation has attracted much attention recently due to its support of knowledge transfer among languages and the low cost of training and deployment compared with numerous bilingual models. A known challenge of multilingual models is the negative language interference. In order to enhance the translation quality, deeper and wider architectures are applied to multilingual modeling for larger model capacity, which suffers from the increased inference cost at the same time. It has been pointed out in recent studies that parameters shared among languages are the cause of interference while they may also enable positive transfer. Based on these insights, we propose an adaptive and sparse architecture for multilingual modeling, and train the model to learn shared and language-specific parameters to improve the positive transfer and mitigate the interference. The sparse architecture only activates a sub-network which preserves inference efficiency, and the adaptive design selects different sub-networks based on the input languages. Our model outperforms strong baselines across multiple benchmarks. On the large-scale OPUS dataset with $100$ languages, we achieve $+2.1$, $+1.3$ and $+6.2$ BLEU improvements in one-to-many, many-to-one and zero-shot tasks respectively compared to standard Transformer without increasing the inference cost.
True differential superconducting on-chip output amplifier

The true-differential superconductor on-chip amplifier has complementary outputs that float with respect to chip ground. This improves signal integrity and compatibility with the receiving semiconductor stage. Both source-terminated and non-source-terminated designs producing 4 mV demonstrated rejection of a large common mode interference in the package. Measured margins are ±8.5% on the output bias, and ±28% on AC clock amplitude. Waveforms and eye diagrams are taken at 2.9–10 Gb s−1 . Direct measurement of bit-error rates are better than the resolution limit of 1×10−12 at 2.9 Gb s−1 , and better than 1×10−9 at 10 Gb s−1 .
Consistency Training with Virtual Adversarial Discrete Perturbation

Consistency training regularizes a model by enforcing predictions of original and perturbed inputs to be similar. Previous studies have proposed various augmentation methods for the perturbation but are limited in that they are agnostic to the training model. Thus, the perturbed samples may not aid in regularization due to their ease of classification from the model. In this context, we propose an augmentation method of adding a discrete noise that would incur the highest divergence between predictions. This virtual adversarial discrete noise obtained by replacing a small portion of tokens while keeping original semantics as much as possible efficiently pushes a training model’s decision boundary. Experimental results show that our proposed method outperforms other consistency training baselines with text editing, paraphrasing, or a continuous noise on semi-supervised text classification tasks and a robustness benchmark.
Anomalous self-similar solutions of exponential type for the subcritical fast diffusion equation with weighted reaction

We prove existence and uniqueness of the branch of the so-called anomalous eternal solutions in exponential self-similar form for the subcritical fast-diffusion equation with a weighted reaction term ∂tu=Δum+|x|σup, posed in RN with N ⩾ 3, where 1,$? > 01, and the critical value for the weight σ=2(p−1)1−m. The branch of exponential self-similar solutions behaves similarly as the well-established anomalous solutions to the pure fast diffusion equation, but without a finite time extinction or a finite time blow-up, and presenting instead a change of sign of both self-similar exponents at m = m s = (N − 2)/(N + 2), leading to surprising qualitative differences. In this sense, the reaction term we consider realizes a perfect equilibrium in the competition between the fast diffusion and the reaction effects.
Supernova Fallback as Origin of Neutron Star Spins and Spin-kick Alignment

Natal kicks and spins are characteristic properties of neutron stars (NSs) and black holes (BHs). Both offer valuable clues to dynamical processes during stellar core collapse and explosion. Moreover, they influence the evolution of stellar multiple systems and the gravitational-wave signals from their inspiral and merger. Observational evidence of a possibly generic spin-kick alignment has been interpreted as an indication that NS spins are either induced with the NS kicks or inherited from the progenitor rotation, which thus might play a dynamically important role during stellar collapse. Current three-dimensional supernova simulations suggest that NS kicks are transferred in the first seconds of the explosion, mainly by anisotropic mass ejection and, on a secondary level, anisotropic neutrino emission. By contrast, the NS spins are only determined minutes to hours later by the angular momentum associated with the fallback of matter that does not become gravitationally unbound in the supernova. Here, we propose a novel scenario to explain spin-kick alignment as a consequence of tangential vortex flows in the fallback matter that is accreted mostly from the direction of the NS’s motion. For this effect the initial NS kick is crucial, because it produces a growing offset of the NS away from the explosion center, thus promoting one-sided accretion. In this new scenario conclusions based on traditional concepts are reversed. For example, pre-kick NS spins are not required, and rapid progenitor core rotation can hamper spin-kick alignment. We also discuss implications for natal BH kicks and the possibility of tossing the BH’s spin axis during its formation.
Orthogonalizing Convolutional Layers with the Cayley Transform

Recent work has highlighted several advantages of enforcing orthogonality in the weight layers of deep networks, such as maintaining the stability of activations, preserving gradient norms, and enhancing adversarial robustness by enforcing low Lipschitz constants. Although numerous methods exist for enforcing the orthogonality of fully-connected layers, those for convolutional layers are more heuristic in nature, often focusing on penalty methods or limited classes of convolutions. In this work, we propose and evaluate an alternative approach to directly parameterize convolutional layers that are constrained to be orthogonal. Specifically, we propose to apply the Cayley transform to a skew-symmetric convolution in the Fourier domain, so that the inverse convolution needed by the Cayley transform can be computed efficiently. We compare our method to previous Lipschitz-constrained and orthogonal convolutional layers and show that it indeed preserves orthogonality to a high degree even for large convolutions. Applied to the problem of certified adversarial robustness, we show that networks incorporating the layer outperform existing deterministic methods for certified defense against $\ell_2$-norm-bounded adversaries, while scaling to larger architectures than previously investigated. Code is available at https://github.com/locuslab/orthogonal-convolutions.
Demystify Optimization Challenges in Multilingual Transformers

Multilingual Transformer improves parameter ef-ﬁciency and crosslingual transfer. How to effectively train multilingual models has not been well studied. Using multilingual machine translation as a testbed, we study optimization challenges from loss landscape and parameter plasticity perspectives. We found that imbalanced training data poses task interference between high and low resource languages, characterized by nearly orthogonal gradients for major parameters and the optimization trajectory being mostly dominated by high resource. We show that local curvature of the loss surface affects the degree of interference, and existing heuristics of data sampling implicitly reduces the sharpness, although still face a trade-off between high and low resource languages. We propose a principled multi-objective optimization algorithm, C urvature A ware T ask S caling (CATS), which improves both optimization and generalization especially for low resource. Experiments on TED, WMT and OPUS-100 benchmarks demonstrate that CATS advances the Pareto front of accuracy while being efﬁcient to apply to massive multilingual settings at the scale of 100 languages.
An Interpretability Illusion for BERT

We describe an"interpretability illusion"that arises when analyzing the BERT model. Activations of individual neurons in the network may spuriously appear to encode a single, simple concept, when in fact they are encoding something far more complex. The same effect holds for linear combinations of activations. We trace the source of this illusion to geometric properties of BERT's embedding space as well as the fact that common text corpora represent only narrow slices of possible English sentences. We provide a taxonomy of model-learned concepts and discuss methodological implications for interpretability research, especially the importance of testing hypotheses on multiple data sets.
Oxygen hole content, charge-transfer gap, covalency, and cuprate superconductivity

Significance Modern theoretical methods solve a long-standing mystery of cuprate high-temperature superconductivity, identifying crucial quantities that optimize the transition temperature. Superconducting cuprates have very different transition temperatures, and even if the optimal value of the superconducting transition temperature is obtained for a given parent compound by varying doping, there is no correlation between optimal doping and transition temperature. Instead, it has been found experimentally that the optimal transition temperature is controlled by oxygen hole content or by the size of the charge-transfer gap. Our calculations show that these two quantities are correlated and that together with covalency they lead to an effective superexchange interaction between copper atoms that ultimately controls the optimal superconducting order parameter. Experiments have shown that the families of cuprate superconductors that have the largest transition temperature at optimal doping also have the largest oxygen hole content at that doping [D. Rybicki et al., Nat. Commun. 7, 1–6 (2016)]. They have also shown that a large charge-transfer gap [W. Ruan et al., Sci. Bull. (Beijing) 61, 1826–1832 (2016)], a quantity accessible in the normal state, is detrimental to superconductivity. We solve the three-band Hubbard model with cellular dynamical mean-field theory and show that both of these observations follow from the model. Cuprates play a special role among doped charge-transfer insulators of transition metal oxides because copper has the largest covalent bonding with oxygen. Experiments [L. Wang et al., arXiv [Preprint] (2020). https://arxiv.org/abs/2011.05029 (Accessed 10 November 2020)] also suggest that superexchange is at the origin of superconductivity in cuprates. Our results reveal the consistency of these experiments with the above two experimental findings. Indeed, we show that covalency and a charge-transfer gap lead to an effective short-range superexchange interaction between copper spins that ultimately explains pairing and superconductivity in the three-band Hubbard model of cuprates.
Can Artificial Intelligence Make Art?

In two experiments (total N=693) we explored whether people are willing to consider paintings made by AI-driven robots as art, and robots as artists. Across the two experiments, we manipulated three factors: (i) agent type (AI-driven robot v. human agent), (ii) behavior type (intentional creation of a painting v. accidental creation), and (iii) object type (abstract v. representational painting). We found that people judge robot paintings and human painting as art to roughly the same extent. However, people are much less willing to consider robots as artists than humans, which is partially explained by the fact that they are less disposed to attribute artistic intentions to robots.
Fermi Surface Mapping and the Nature of Charge-Density-Wave Order in the Kagome Superconductor 
CsV3Sb5

The recently discovered family of A V 3 Sb 5 ( A : K, Rb Cs) kagome metals possess a unique com-bination of nontrivial band topology, superconducting ground states, and signatures of electron correlations manifest via competing charge density wave order. Little is understood regarding the nature of the charge density wave (CDW) instability inherent to these compounds and the potential correlation with the onset of a large anomalous Hall response. To understand the impact of the CDW order on the electronic structure in these systems, we present quantum oscillation measurements on single crystals of CsV 3 Sb 5 . Our data provide direct evidence that the CDW invokes a substantial reconstruction of the Fermi surface pockets associated with the vanadium orbitals and the kagome lattice framework. In conjunction with density functional theory modeling, we are able to identify split oscillation frequencies originating from reconstructed pockets built from vanadium orbitals and Dirac-like bands. Complementary diﬀraction measurements are further able to demonstrate that the CDW instability has a correlated phasing of distortions between neighboring V 3 Sb 5 planes, and the average structure in the CDW state is proposed. These results provide critical insights into the underlying CDW instability in A V 3 Sb 5 kagome metals and support minimal models of CDW order arising from within the vanadium-based kagome lattice.
Empowering News Recommendation with Pre-trained Language Models

Personalized news recommendation is an essential technique for online news services. News articles usually contain rich textual content, and accurate news modeling is important for personalized news recommendation. Existing news recommendation methods mainly model news texts based on traditional text modeling methods, which is not optimal for mining the deep semantic information in news texts. Pre-trained language models (PLMs) are powerful for natural language understanding, which has the potential for better news modeling. However, there is no public report that shows PLMs have been applied to news recommendation. In this paper, we report our work on pre-trained language models empowered news recommendation (PLM-NR). Offline experimental results on both monolingual and multilingual news recommendation datasets show that leveraging PLMs for news modeling can effectively improve the performance of news recommendation. Our PLM-NR models have been deployed to the Microsoft News platform, and online flight results show that they can achieve significant performance gains in both English-speaking and global markets.