Miracle-dz/UniFAR-FTUs huggingface dataset:
Efficient Differentiable Neural Architecture Search with Meta Kernels

The searching procedure of neural architecture search (NAS) is notoriously time consuming and cost this http URL make the search space continuous, most existing gradient-based NAS methods relax the categorical choice of a particular operation to a softmax over all possible operations and calculate the weighted sum of multiple features, resulting in a large memory requirement and a huge computation burden. In this work, we propose an efficient and novel search strategy with meta kernels. We directly encode the supernet from the perspective on convolution kernels and "shrink" multiple convolution kernel candidates into a single one before these candidates operate on the input feature. In this way, only a single feature is generated between two intermediate nodes. The memory for storing intermediate features and the resource budget for conducting convolution operations are both reduced remarkably. Despite high efficiency, our search strategy can search in a more fine-grained way than existing works and increases the capacity for representing possible networks. We demonstrate the effectiveness of our search strategy by conducting extensive experiments. Specifically, our method achieves 77.0% top-1 accuracy on ImageNet benchmark dataset with merely 357M FLOPs, outperforming both EfficientNet and MobileNetV3 under the same FLOPs constraints. Compared to models discovered by the start-of-the-art NAS method, our method achieves the same (sometimes even better) performance, while faster by three orders of magnitude.
Exact expressions for double descent and implicit regularization via surrogate random design

Double descent refers to the phase transition that is exhibited by the generalization error of unregularized learning models when varying the ratio between the number of parameters and the number of training samples. The recent success of highly over-parameterized machine learning models such as deep neural networks has motivated a theoretical analysis of the double descent phenomenon in classical models such as linear regression which can also generalize well in the over-parameterized regime. We provide the first exact non-asymptotic expressions for double descent of the minimum norm linear estimator. Our approach involves constructing a special determinantal point process which we call surrogate random design, to replace the standard i.i. d. design of the training sample. This surrogate design admits exact expressions for the mean squared error of the estimator while preserving the key properties of the standard design. We also establish an exact implicit regularization result for over-parameterized training samples. In particular, we show that, for the surrogate design, the implicit bias of the unregularized minimum norm estimator precisely corresponds to solving a ridge-regularized least squares problem on the population distribution. In our analysis we introduce a new mathematical tool of independent interest: the class of random matrices for which determinant commutes with expectation.
Upper Limits on Planet Occurrence around Ultracool Dwarfs with K2

The occurrence of planets orbiting ultracool dwarfs is poorly constrained. We present results from a guest observer program on NASA’s K2 spacecraft to search for transiting planets orbiting a sample of 827 ultracool dwarfs. Having found no transiting planets in our sample, we determined an upper limit on the occurrence of planets. We simulated planets orbiting our sample for a range of orbital periods and sizes. For the simulated planets that transit their host, we injected the transit light curve into the real K2 light curves, then attempted to recover the injected planets. For a given occurrence rate, we calculated the probability of seeing no planets, and use the results to place an upper limit on planet occurrence as a function of planet radius and orbital period. We find that short-period, mini-Neptune- and Jupiter-sized planets are rare around ultracool dwarfs, consistent with results for early- and mid-type M dwarf stars. We constrain the occurrence rate η for planets between 0.5 and 10 R⊕ with orbital periods between 1 and 26.3 days.
Josephson current through a ferromagnetic bilayer: Beyond the quasiclassical approximation

Based on the Bogoliubov-de Gennes equations, we provide an exact numerical solution for the critical current of Josephson junctions with a composite ferromagnetic bilayer. We demonstrate that for the anti-parallel orientation of the magnetic moments of the bilayer, the presence of a potential barrier at the bilayer interface results in large oscillations of the critical current as a function of ferromagnet thickness and/or exchange field. Because of this, and remarkably, in the range of small exchange field and thicknesses the magnetism leads to the increase of the critical current. This effect is well pronounced at low temperature but disappears near $T_c$. If the potential barrier is replaced by a spin-active barrier at the bilayer interface the conventional 0-$\pi$ transition, similar to the case of an uniform ferromagnetic Josephson junction, is observed. Strikingly, for a parallel orientation of the magnetic moments of the bilayer, the presence of the spin-active barrier restores the anomalous behavior$-$potential barrier in the anti-parallel case. These behaviors result from the resonant tunnelling of Cooper pairs across the composite barrier$-$an effect related to the spin-dependent of Fermi vector in the presence of the ferromagnets' exchange field.
GeBioToolkit: Automatic Extraction of Gender-Balanced Multilingual Corpus of Wikipedia Biographies

We introduce GeBioToolkit, a tool for extracting multilingual parallel corpora at sentence level, with document and gender information from Wikipedia biographies. Despite the gender inequalities present in Wikipedia, the toolkit has been designed to extract corpus balanced in gender. While our toolkit is customizable to any number of languages (and different domains), in this work we present a corpus of 2,000 sentences in English, Spanish and Catalan, which has been post-edited by native speakers to become a high-quality dataset for machine translation evaluation. While GeBioCorpus aims at being one of the first non-synthetic gender-balanced test datasets, GeBioToolkit aims at paving the path to standardize procedures to produce gender-balanced datasets.
STAGE: Spatio-Temporal Attention on Graph Entities for Video Action Detection

Spatio-temporal action localization is a challenging yet fascinating task that aims to detect and classify human actions in video clips. In this paper, we develop a high-level video understanding module which can encode interactions between actors and objects both in space and time. In our formulation, spatio-temporal relationships are learned by performing self-attention operations on a graph structure connecting entities from consecutive clips. Noticeably, the use of graph learning is unprecedented for this task. From a computational point of view, the proposed module is backbone independent by design and does not need end-to-end training. When tested on the AVA dataset, it demonstrates a 10-16% relative mAP improvement over the baseline. Further, it can outperform or bring performances comparable to state-of-the-art models which require heavy end-to-end and synchronized training on multiple GPUs. Code is publicly available at: this https URL.
Pillar in Pillar: Multi-Scale and Dynamic Feature Extraction for 3D Object Detection in Point Clouds

Sparsity and varied density are two of the main obstacles for 3D detection networks with point clouds. In this paper, we present a multi-scale voxelization method and a decomposable dynamic convolution to solve them. We consider the misalignment problem between voxel representation with different scales and present a center-aligned voxelization strategy. Instead of separating points into individual groups, we use an overlapped partition mechanism to avoid the perception deficiency of edge points in each voxel. Based on this multi-scale voxelization, we are able to build an effective fusion network by one-iteration top-down forward. To handle the variation of density in point cloud data, we propose a decomposable dynamic convolutional layer that considers the shared and dynamic components when applying convolutional filters at different positions of feature maps. By modeling bases in the kernel space, the number of parameters for generating dynamic filters is greatly reduced. With a self-learning network, we can apply dynamic convolutions to input features and deal with the variation in the feature space. We conduct experiments with our PiPNet on KITTI dataset and achieve better results than other voxelization-based methods on 3D detection task.
Learning Depth-Guided Convolutions for Monocular 3D Object Detection

3D object detection from a single image without LiDAR is a challenging task due to the lack of accurate depth information. Conventional 2D convolutions are unsuitable for this task because they fail to capture local object and its scale information, which are vital for 3D object detection. To better represent 3D structure, prior arts typically transform depth maps estimated from 2D images into a pseudo-LiDAR representation, and then apply existing 3D point-cloud based object detectors. However, their results depend heavily on the accuracy of the estimated depth maps, resulting in suboptimal performance. In this work, instead of using pseudo-LiDAR representation, we improve the fundamental 2D fully convolutions by proposing a new local convolutional network (LCN), termed Depth-guided Dynamic-Depthwise-Dilated LCN (D4LCN), where the filters and their receptive fields can be automatically learned from image-based depth maps, making different pixels of different images have different filters. D4LCN overcomes the limitation of conventional 2D convolutions and narrows the gap between image representation and 3D point cloud representation. Extensive experiments show that D4LCN outperforms existing works by large margins. For example, the relative improvement of D4LCN against the state-of-the-art on KITTI is 9.1% in the moderate setting. D4LCN ranks 1st on KITTI monocular 3D object detection benchmark at the time of submission (car, December 2019). The code is available at https://github.com/dingmyu/D4LCN.
Revisiting Zero-Rate Bounds on the Reliability Function of Discrete Memoryless Channels

We present a revised proof of Berlekamp’s zero-rate upper bound on the reliability function of discrete memoryless channels, in its extended form for list-decoding as first proved by Blinovsky. The available proofs are somehow uneasy in that they contain in one form or another some cumbersome "nonstandard" procedures or computations. Here we start from Blinovsky’s ideas and complement them with some missing steps to present a proof which is entirely in the realm of standard information theoretic tools even for general list size.
Derived equivalences between skew-gentle algebras using orbifolds

Skew-gentle algebras are skew-group algebras of gentle algebras equipped with a certain $\Z_2$-action. Building on the bijective correspondence between gentle algebras and dissected surfaces, we obtain in this paper a bijection between skew-gentle algebras and certain dissected orbifolds that admit a double cover. We prove the compatibility of the $\Z_2$-action on the double cover with the skew-group algebra construction. This allows us to investigate the derived equivalence relation between skew-gentle algebras in geometric terms: We associate to each skew-gentle algebra a line field on the orbifold, and on its double cover, and interpret different kinds of derived equivalences of skew-gentle algebras in terms of diffeomorphisms respecting the homotopy class of the line fields associated to the algebras.
Efficient congruencing in ellipsephic sets: the general case

In this paper, we bound the number of solutions to a general Vinogradov system of equations $x_1^j+\dots+x_s^j=y_1^j+\dots+y_s^j$, $(1\leq j\leq k)$, as well as other related systems, in which the variables are required to satisfy digital restrictions in a given base. Specifically, our sets of permitted digits have the property that there are few representations of a natural number as sums of elements of the digit set---the set of squares serving as a key example. We obtain better bounds using this additive structure than could be deduced purely from the size of the set of variables. In particular, when the digits are required to be squares, we obtain diagonal behaviour with $2k(k+1)$ variables.
edBB: Biometrics and Behavior for Assessing Remote Education

We present a platform for student monitoring in remote education consisting of a collection of sensors and software that capture biometric and behavioral data. We define a collection of tasks to acquire behavioral data that can be useful for facing the existing challenges in automatic student monitoring during remote evaluation. Additionally, we release an initial database including data from 20 different users completing these tasks with a set of basic sensors: webcam, microphone, mouse, and keyboard; and also from more advanced sensors: NIR camera, smartwatch, additional RGB cameras, and an EEG band. Information from the computer (e.g. system logs, MAC, IP, or web browsing history) is also stored. During each acquisition session each user completed three different types of tasks generating data of different nature: mouse and keystroke dynamics, face data, and audio data among others. The tasks have been designed with two main goals in mind: i) analyse the capacity of such biometric and behavioral data for detecting anomalies during remote evaluation, and ii) study the capability of these data, i.e. EEG, ECG, or NIR video, for estimating other information about the users such as their attention level, the presence of stress, or their pulse rate.
Large-N behavior of three-dimensional lattice CPN−1 models

We consider a three-dimensional lattice CPN−1 model, which corresponds to the lattice Abelian–Higgs model in the infinite gauge-coupling limit. We investigate its phase diagram and critical behavior in the large-N limit. We obtain numerical evidence that the model undergoes a first-order transition for sufficiently large values of N, i.e. for any N > 2 up to N = 100. The transition becomes stronger—both the latent heat and the surface tension increase—as N increases. Moreover, on the high-temperature side, gauge fields decorrelate on distances of the order of one lattice spacing for all values of N considered. Our results are consistent with a simple scenario, in which the transition is of first order for any N, including . We critically discuss the analytic large-N calculations that predicted a large-N continuous transition, showing that one crucial assumption made in these computations fails for the model we consider.
Thermodynamic costs of Turing machines

Turing Machines (TMs) are the canonical model of computation in computer science and physics. We combine techniques from algorithmic information theory and stochastic thermodynamics to analyze the thermodynamic costs of TMs. We consider two different ways of realizing a given TM with a physical process. The first realization is designed to be thermodynamically reversible when fed with random input bits. The second realization is designed to generate less heat, up to an additive constant, than any realization that is computable (i.e., consistent with the physical Church-Turing thesis). We consider three different thermodynamic costs: the heat generated when the TM is run on each input (which we refer to as the "heat function"), the minimum heat generated when a TM is run with an input that results in some desired output (which we refer to as the "thermodynamic complexity" of the output, in analogy to the Kolmogorov complexity), and the expected heat on the input distribution that minimizes entropy production. For universal TMs, we show for both realizations that the thermodynamic complexity of any desired output is bounded by a constant (unlike the conventional Kolmogorov complexity), while the expected amount of generated heat is infinite. We also show that any computable realization faces a fundamental tradeoff between heat generation, the Kolmogorov complexity of its heat function, and the Kolmogorov complexity of its input-output map. We demonstrate this tradeoff by analyzing the thermodynamics of erasing a long string.
Unravelling the unusually curved X-ray spectrum of RGB J0710 + 591 using AstroSat observations

We report the analysis of simultaneous multiwavelength data of the high-energy-peaked blazar RGB J0710 + 591 from the Large Area X-ray Proportional Counters, Soft X-ray focusing Telescope, and Ultraviolet Imaging Telescope (UVIT) instruments onboard AstroSat. The wide band X-ray spectrum (0.35–30 keV) is modelled as synchrotron emission from a non-thermal distribution of high-energy electrons. The spectrum is unusually curved, with a curvature parameter β_p ∼ 6.4 for a log parabola particle distribution, or a high-energy spectral index p_2 > 4.5 for a broken power-law distribution. The spectrum shows more curvature than an earlier quasi-simultaneous analysis of Swift–XRT/NuSTAR data where the parameters were β_p ∼ 2.2 or p_2 ∼ 4. It has long been known that a power-law electron distribution can be produced from a region where particles are accelerated under Fermi process and the radiative losses in acceleration site decide the maximum attainable Lorentz factor, γ_max. Consequently, this quantity decides the energy at which the spectrum curves steeply. We show that such a distribution provides a more natural explanation for the AstroSat data as well as the earlier XRT/NuSTAR observation, making this as the first well-constrained determination of the photon energy corresponding to γ_max. This in turn provides an estimate of the acceleration time-scale as a function of magnetic field and Doppler factor. The UVIT observations are consistent with earlier optical/UV measurements and reconfirm that they plausibly correspond to a different radiative component than the one responsible for the X-ray emission.
Feedback control of surface roughness in a one-dimensional KPZ growth process

Control of generically scale-invariant systems, i.e., targeting specific cooperative features in non-linear stochastic interacting systems with many degrees of freedom subject to strong fluctuations and correlations that are characterized by power laws, remains an important open problem. We study the control of surface roughness during a growth process described by the Kardar--Parisi--Zhang (KPZ) equation in $(1+1)$ dimensions. We achieve the saturation of the mean surface roughness to a prescribed value using non-linear feedback control. Numerical integration is performed by means of the pseudospectral method, and the results are used to investigate the coupling effects of controlled (linear) and uncontrolled (non-linear) KPZ dynamics during the control process. While the intermediate time kinetics is governed by KPZ scaling, at later times a linear regime prevails, namely the relaxation towards the desired surface roughness. The temporal crossover region between these two distinct regimes displays intriguing scaling behavior that is characterized by non-trivial exponents and involves the number of controlled Fourier modes. Due to the control, the height probability distribution becomes negatively skewed, which affects the value of the saturation width.
Removable and/or Repeated Units Emerge in Overparametrized Deep Neural Networks

Deep neural networks (DNNs) perform well on a variety of tasks despite the fact that most networks used in practice are vastly overparametrized and even capable of perfectly fitting randomly labeled data. Recent evidence suggests that developing compressible representations is key for adjusting the complexity of overparametrized networks to the task at hand. In this paper, we provide new empirical evidence that supports this hypothesis by identifying two types of units that emerge when the network's width is increased: removable units which can be dropped out of the network without significant change to the output and repeated units whose activities are highly correlated with other units. The emergence of these units implies capacity constraints as the function the network represents could be expressed by a smaller network without these units. In a series of experiments with AlexNet, ResNet and Inception networks in the CIFAR-10 and ImageNet datasets, and also using shallow networks with synthetic data, we show that DNNs consistently increase either the number of removable units, repeated units, or both at greater widths for a comprehensive set of hyperparameters. These results suggest that the mechanisms by which networks in the deep learning regime adjust their complexity operate at the unit level and highlight the need for additional research into what drives the emergence of such units.
Closed conformal Killing–Yano initial data

Through an exhaustive search, we produce a five-parameter family of propagation identities for the closed conformal Killing–Yano (cCYK) equation on two-forms, which hold on an Einstein cosmological vacuum spacetime in any dimension n > 4. It is well-known that spacetimes admitting a non-degenerate two-form of this type are exhausted by the Kerr-NUT-(A)dS family of exact higher dimensional black hole solutions. As a consequence, we identify a set of necessary and sufficient conditions ensuring that the cosmological vacuum development of an initial data set for Einstein’s field equations admits a cCYK two-form. We refer to these conditions as closed conformal Killing–Yano initial data equations. The four-dimensional case is special and is treated separately, where we can also handle the conformal Killing–Yano equation without the closed condition.
Towards a full solution of the relativistic Boltzmann equation for quark-gluon matter on GPUs

We have developed a numerical framework for a full solution of the relativistic Boltzmann equations for the quark-gluon matter using the multiple Graphics Processing Units (GPUs) on distributed clusters. Including all the $2 \to 2$ scattering processes of 3-flavor quarks and gluons, we compute the time evolution of distribution functions in both coordinate and momentum spaces for the cases of pure gluons, quarks and the mixture of quarks and gluons. By introducing a symmetrical sampling method on GPUs which ensures the particle number conservation, our framework is able to perform the space-time evolution of quark-gluon system towards thermal equilibrium with high performance. We also observe that the gluons naturally accumulate in the soft region at the early time, which may indicate the gluon condensation.
Asymptotic symmetries and charges at spatial infinity in general relativity

We analyze the asymptotic symmetries and their associated charges at spatial infinity in four-dimensional asymptotically-flat spacetimes. We use the covariant formalism of Ashtekar and Hansen where the asymptotic fields and symmetries live on the three-manifold of spatial directions at spatial infinity, represented by a timelike unit-hyperboloid (or de Sitter space). Using the covariant phase space formalism, we derive formulae for the charges corresponding to asymptotic supertranslations and Lorentz symmetries at spatial infinity. With the motivation of, eventually, proving that these charges match with those defined on null infinity—as has been conjectured by Strominger—we do not impose any restrictions on the choice of conformal factor in contrast to previous work on this problem. Since we work with a general conformal factor we expect that our charge expressions will be more suitable to prove the matching of the Lorentz charges at spatial infinity to those defined on null infinity, as has been recently shown for the supertranslation charges.