Miracle-dz/UniFAR-FTUs huggingface dataset:
Understanding and Improving Early Stopping for Learning with Noisy Labels

The memorization effect of deep neural network (DNN) plays a pivotal role in many state-of-the-art label-noise learning methods. To exploit this property, the early stopping trick, which stops the optimization at the early stage of training, is usually adopted. Current methods generally decide the early stopping point by considering a DNN as a whole. However, a DNN can be considered as a composition of a series of layers, and we find that the latter layers in a DNN are much more sensitive to label noise, while their former counterparts are quite robust. Therefore, selecting a stopping point for the whole network may make different DNN layers antagonistically affected each other, thus degrading the final performance. In this paper, we propose to separate a DNN into different parts and progressively train them to address this problem. Instead of the early stopping, which trains a whole DNN all at once, we initially train former DNN layers by optimizing the DNN with a relatively large number of epochs. During training, we progressively train the latter DNN layers by using a smaller number of epochs with the preceding layers fixed to counteract the impact of noisy labels. We term the proposed method as progressive early stopping (PES). Despite its simplicity, compared with the early stopping, PES can help to obtain more promising and stable results. Furthermore, by combining PES with existing approaches on noisy label training, we achieve state-of-the-art performance on image classification benchmarks.
GPU Based Improved Fast Iterative Algorithm for Eikonal Equation

In this paper we propose an improved fast iterative method to solve the Eikonal equation, which can be implemented in parallel. We improve the fast iterative method for Eikonal equation in two novel ways, in the value update and in the error correction. The new value update is very similar to the fast iterative method in that we selectively update the points in the active list chosen by a convergence measure. However, in order to save time, the improved algorithm does not do a convergence check of the neighboring points of the narrow band as the fast iterative method does. The additional error correction step is to correct the errors that the previous value update step may cause. The error correction step consists of finding and recalculating the point values in a separate remedy list which is quite easy to implement on a GPU. In contrast to the fast marching method and the fast sweeping method for the Eikonal equation, our improved method does not need to compute the solution with any special ordering in either the remedy list or the active list so that it can be implemented in parallel. In our experiments, we implemented our new algorithm in parallel on a GPU and compared the elapsed time with other current algorithms. The improved fast iterative method runs faster than the other algorithms in most cases in our experiments.
Towards a generalized monaural and binaural auditory model for psychoacoustics and speech intelligibility

Auditory perception involves cues in the monaural auditory pathways as well as binaural cues based on differences between the ears. So far auditory models have often focused on either monaural or binaural experiments in isolation. Although binaural models typically build upon stages of (existing) monaural models, only a few attempts have been made to extend a monaural model by a binaural stage using a unified decision stage for monaural and binaural cues. In such approaches, a typical prototype of binaural processing has been the classical equalization-cancelation mechanism, which either involves signal-adaptive delays and provides a single channel output or can be implemented with tapped delays providing a high-dimensional multichannel output. This contribution extends the (monaural) generalized envelope power spectrum model by a non-adaptive binaural stage with only a few, fixed output channels. The binaural stage resembles features of physiologically motivated hemispheric binaural processing, as simplified signal processing stages, yielding a 5-channel monaural and binaural matrix feature"decoder"(BMFD). The back end of the existing monaural model is applied to the 5-channel BMFD output and calculates short-time envelope power and power features. The model is evaluated and discussed for a baseline database of monaural and binaural psychoacoustic experiments from the literature.
Exploring Context Modeling Techniques on the Spatiotemporal Crowd Flow Prediction

In the big data and AI era, context is widely exploited as extra information which makes it easier to learn a more complex pattern in machine learning systems. However, most of the existing related studies seldom take context into account. The difficulty lies in the unknown generalization ability of both context and its modeling techniques across different scenarios. To fill the above gaps, we conduct a large-scale analytical and empirical study on the spatiotemporal crowd prediction (STCFP) problem that is a widely-studied and hot research topic. We mainly make three efforts:(i) we develop new taxonomy about both context features and context modeling techniques based on extensive investigations in prevailing STCFP research; (ii) we conduct extensive experiments on seven datasets with hundreds of millions of records to quantitatively evaluate the generalization ability of both distinct context features and context modeling techniques; (iii) we summarize some guidelines for researchers to conveniently utilize context in diverse applications.
Interventional Assays for the Latent Space of Autoencoders

The encoders and decoders of autoencoders effectively project the input onto learned manifolds in the latent space and data space respectively. We propose a framework, called latent responses, for probing the learned data manifold using interventions in the latent space. Using this framework, we investigate "holes" in the representation to quantitatively ascertain to what extent the latent space of a trained VAE is consistent with the chosen prior. Furthermore, we use the identified structure to improve interpolation between latent vectors. We evaluate how our analyses improve the quality of the generated samples using the VAE on a variety of benchmark datasets.
Spatially Resolving the Kinematics of the ≲100 μas Quasar Broad-line Region Using Spectroastrometry. II. The First Tentative Detection in a Luminous Quasar at z = 2.3

Direct measurements of the masses of supermassive black holes are key to understanding their growth and constraining their symbiotic relationship to their host galaxies. However, current methods used to directly measure black hole masses in active quasars become challenging or impossible beyond z ≳ 0.2. Spectroastrometry (SA) measures the spatial centroid of an object’s spectrum as a function of wavelength, delivering angular resolution far better than the point-spread function (PSF) for high signal-to-noise ratio observations. We observed the luminous quasar SDSS J212329.47–005052.9 at z = 2.279 with the aim of resolving its ∼100 μas Hα broad emission-line region (BLR) and present the first SA constraints on the size and kinematic structure of the BLR. Using a novel pipeline to extract the SA signal and reliable uncertainties, we achieved a centroiding precision of ≃100 μas, or >2000× smaller than the K-band AO-corrected PSF, yielding a tentative 3. 2σ detection of an SA signal from the BLR. Modeling the BLR emission as arising from an inclined rotating disk with a mixture of coherent and random motions we constrain rBLR=454−162+565μas ( 3.71−1.28+4.65pc ), providing a 95% confidence upper limit on the black hole mass MBHsin2i≤1.8×109M⊙ . Our results agree with the r BLR–L relation measured for lower-z quasars but expands its dynamic range by an order of magnitude in luminosity. We did not detect the potentially stronger SA signal from the narrow-line region but discuss in detail why it may be absent. Already with existing instrumentation, SA can deliver ∼6× smaller uncertainties (∼15 μas) than achieved here, enabling ∼10% measurements of supermassive black hole masses in high-z quasars.
On a sinc-type MBE model

We introduce a new sinc-type molecular beam epitaxy model which is derived from a cosine-type energy functional. The landscape of the new functional is remarkably similar to the classical MBE model with double well potential but has the additional advantage that all its derivatives are uniformly bounded. We consider first order IMEX and second order BDF2 discretization schemes. For both cases we quantify explicit time step constraints for the energy dissipation which is in good accord with the practical numerical simulations. Furthermore we introduce a new theoretical framework and prove unconditional uniform energy boundedness with no size restrictions on the time step. This is the first unconditional (i.e. independent of the time step size) result for semi-implicit methods applied to the phase field models without introducing any artificial stabilization terms or fictitious variables.
Single-Step Adversarial Training for Semantic Segmentation

Even though deep neural networks succeed on many different tasks including semantic segmentation, they lack on robustness against adversarial examples. To counteract this exploit, often adversarial training is used. However, it is known that adversarial training with weak adversarial attacks (e.g. using the Fast Gradient Method) does not improve the robustness against stronger attacks. Recent research shows that it is possible to increase the robustness of such single-step methods by choosing an appropriate step size during the training. Finding such a step size, without increasing the computational effort of single-step adversarial training, is still an open challenge. In this work we address the computationally particularly demanding task of semantic segmentation and propose a new step size control algorithm that increases the robustness of single-step adversarial training. The proposed algorithm does not increase the computational effort of single-step adversarial training considerably and also simplifies training, because it is free of meta-parameter. We show that the robustness of our approach can compete with multi-step adversarial training on two popular benchmarks for semantic segmentation.
Ju n 20 21 Ballistic annihilation in one dimension

Soham Biswas Departamento de F́ısica, Universidad de Guadalajara, Guadalajara, Jalisco, Mexico ∗ In this article we have reviewed the problem of reaction annihilation A+A → ∅ on a real lattice in one dimension, where A particles move ballistically in one direction. First we have discussed the case of pure ballistic annihilation, that is ballistic annihilation under the synchronous update rule. Then we have reviewed ballistic annihilation with superimposed diffusion in one dimension. For pure ballistic annihilation the concentration of the particles c(t) decay as t with time. However when the diffusion is superimposed the decay is faster and c(t) ∼ t. We have also discussed other interesting quantities which have been studied over the course of time.
BONuS: Multiple multivariate testing with a data-adaptivetest statistic

We propose a new adaptive empirical Bayes framework, the Bag-Of-Null-Statistics (BONuS) procedure, for multiple testing where each hypothesis testing problem is itself multivariate or nonparametric. BONuS is an adaptive and interactive knockoff-type method that helps improve the testing power while controlling the false discovery rate (FDR), and is closely connected to the"counting knockoffs"procedure analyzed in Weinstein et al. (2017). Contrary to procedures that start with a $p$-value for each hypothesis, our method analyzes the entire data set to adaptively estimate an optimal $p$-value transform based on an empirical Bayes model. Despite the extra adaptivity, our method controls FDR in finite samples even if the empirical Bayes model is incorrect or the estimation is poor. An extension, the Double BONuS procedure, validates the empirical Bayes model to guard against power loss due to model misspecification.
Survivable Robotic Control through Guided Bayesian Policy Search with Deep Reinforcement Learning

Many robot manipulation skills can be represented with deterministic characteristics and there exist efficient techniques for learning parameterized motor plans for those skills. However, one of the active research challenge still remains to sustain manipulation capabilities in situation of a mechanical failure. Ideally, like biological creatures, a robotic agent should be able to reconfigure its control policy by adapting to dynamic adversaries. In this paper, we propose a method that allows an agent to survive in a situation of mechanical loss, and adaptively learn manipulation with compromised degrees of freedom-we call our method Survivable Robotic Learning (SRL). Our key idea is to leverage Bayesian policy gradient by encoding knowledge bias in posterior estimation, which in turn alleviates future policy search explorations, in terms of sample efficiency and when compared to random exploration based policy search methods. SRL represents policy priors as Gaussian process, which allows tractable computation of approximate posterior (when true gradient is intractable), by incorporating guided bias as proxy from prior replays. We evaluate our proposed method against off-the-shelf model free learning algorithm (DDPG), testing on a hexapod robot platform which encounters incremental failure emulation, and our experiments show that our method improves largely in terms of sample requirement and quantitative success ratio in all failure modes. A demonstration video of our experiments can be viewed at: https://sites.google. com/view/survivalrl
SOLO: A Simple Framework for Instance Segmentation

Compared to many other dense prediction tasks, e.g., semantic segmentation, it is the arbitrary number of instances that has made instance segmentation much more challenging. In order to predict a mask for each instance, mainstream approaches either follow the “detect-then-segment” strategy (e.g., Mask R-CNN), or predict embedding vectors first then cluster pixels into individual instances. In this paper, we view the task of instance segmentation from a completely new perspective by introducing the notion of “instance categories”, which assigns categories to each pixel within an instance according to the instance's location. With this notion, we propose segmenting objects by locations (SOLO), a simple, direct, and fast framework for instance segmentation with strong performance. We derive a few SOLO variants (e.g., Vanilla SOLO, Decoupled SOLO, Dynamic SOLO) following the basic principle. Our method directly maps a raw input image to the desired object categories and instance masks, eliminating the need for the grouping post-processing or the bounding box detection. Our approach achieves state-of-the-art results for instance segmentation in terms of both speed and accuracy, while being considerably simpler than the existing methods. Besides instance segmentation, our method yields state-of-the-art results in object detection (from our mask byproduct) and panoptic segmentation. We further demonstrate the flexibility and high-quality segmentation of SOLO by extending it to perform one-stage instance-level image matting. Code is available at: https://git.io/AdelaiDet.
A Differentiable Neural-Network Force Field for Ionic Liquids

We present NeuralIL, a model for the potential energy of an ionic liquid that accurately reproduces first-principles results with orders-of-magnitude savings in computational cost. Built on the basis of a multilayer perceptron and spherical Bessel descriptors of the atomic environments, NeuralIL is implemented in such a way as to be fully automatically differentiable. It can thus be trained on ab initio forces instead of just energies, to make the most out of the available data, and can efficiently predict arbitrary derivatives of the potential energy. Using ethylammonium nitrate as the test system, we obtain out-of-sample accuracies better than 2 meV atom–1 (<0.05 kcal mol–1) in the energies and 70 meV Å–1 in the forces. We show that encoding the element-specific density in the spherical Bessel descriptors is key to achieving this. Harnessing the information provided by the forces drastically reduces the amount of atomic configurations required to train a neural network force field based on atom-centered descriptors. We choose the Swish-1 activation function and discuss the role of this choice in keeping the neural network differentiable. Furthermore, the possibility of training on small data sets allows for an ensemble-learning approach to the detection of extrapolation. Finally, we find that a separate treatment of long-range interactions is not required to achieve a high-quality representation of the potential energy surface of these dense ionic systems.
On the nature of the anomalous event in 2021 in the dwarf nova SS Cygni and its multi-wavelength transition

SS Cyg has long been recognized as the prototype of a group of dwarf novae that show only outbursts. However, this object has entered a quite anomalous event in 2021, which at first appeared to be standstill, i.e., an almost constant luminosity state, observed in Z Cam-type dwarf novae. This unexpected event gives us a great opportunity to reconsider the nature of standstill in cataclysmic variables. We have observed this anomalous event and its forerunner, a gradual and simultaneous increase in the optical and X-ray flux during quiescence, through many optical telescopes and the X-ray telescopes NICER and NuSTAR. We have not found any amplification of the orbital hump during quiescence before the anomalous event, which suggests that the mass transfer rate did not significantly fluctuate on average. The estimated X-ray flux was not enough to explain the increment of the optical flux during quiescence via X-ray irradiation of the disk and the secondary star. It would be natural to consider that viscosity in the quiescent disk was enhanced before the anomalous event, which increased mass accretion rates in the disk and raised not only the optical flux but also the X-ray flux. We suggest that enhanced viscosity also triggered the standstill-like phenomenon in SS Cyg, which is considered to be a series of small outbursts. The inner part of the disk would always stay in the outburst state and only its outer part would be unstable against the thermal-viscous instability during this phenomenon, which is consistent with the observed optical color variations. This scenario is in line with our X-ray spectral analyses which imply that the X-ray emitting inner accretion flow became hotter than usual and vertically expanded and that it became denser and was cooled down after the onset of the standstill-like state.
AI-Based Secure NOMA and Cognitive Radio-Enabled Green Communications: Channel State Information and Battery Value Uncertainties

In this paper, the security-aware robust resource allocation in energy harvesting cognitive radio networks is considered with cooperation between two transmitters while there are uncertainties in channel gains and battery energy value. To be specific, the primary access point harvests energy from the green resource and uses time switching protocol to send the energy and data towards the secondary access point (SAP). Using power-domain non-orthogonal multiple access technique, the SAP helps the primary network to improve the security of data transmission by using the frequency band of the primary network. In this regard, we introduce the problem of maximizing the proportional-fair energy efficiency (PFEE) considering uncertainty in the channel gains and battery energy value subject to the practical constraints. Moreover, the channel gain of the eavesdropper is assumed to be unknown. Employing the decentralized partially observable Markov decision process, we investigate the solution of the corresponding resource allocation problem. We exploit multi-agent with single reward deep deterministic policy gradient (MASRDDPG) and recurrent deterministic policy gradient (RDPG) methods. These methods are compared with the state-of-the-art ones like multi-agent and single-agent DDPG. Simulation results show that both MASRDDPG and RDPG methods, outperform the state-of-the-art methods by providing more PFEE to the network
On the Power of Saturated Transformers: A View from Circuit Complexity

Transformers have become a standard architecture for many NLP problems. This has motivated theoretically analyzing their capabilities as models of language, in order to understand what makes them suc-cessful, and what their potential weaknesses might be. Recent work has shown that transformers with hard attention are quite limited in capacity, and in fact can be simulated by constant-depth circuits. However, hard attention is a restrictive assumption, which may complicate the relevance of these results for practical transformers. In this work, we analyze the circuit complexity of transformers with saturated attention : a generalization of hard attention that more closely captures the attention patterns learnable in practical transformers. We show that saturated transformers transcend the limitations of hard-attention transformers. With some minor assumptions, we prove that the number of bits needed to represent a saturated transformer memory vector is O (log n ) , which implies saturated transformers can be simulated by log-depth circuits. Thus, the jump from hard to saturated attention can be understood as increasing the transformer’s effective circuit depth by a factor of O (log n ) .
A Unified View of Stochastic Hamiltonian Sampling

In this work, we revisit the theoretical properties of Hamiltonian stochastic differential equations ( SDE s) for Bayesian posterior sampling, and we study the two types of errors that arise from numerical SDE simulation: the discretization error and the error due to noisy gradient estimates in the context of data subsampling. We consider overlooked results describing the ergodic convergence rates of numerical integration schemes, and we produce a novel analysis for the effect of mini-batches through the lens of differential operator splitting. In our analysis, the stochastic component of the proposed Hamiltonian SDE is decoupled from the gradient noise, for which we make no normality assumptions. This allows us to derive interesting connections among different sampling schemes, including the original Hamiltonian Monte Carlo ( HMC ) algorithm, and explain their performance. We show that for a careful selection of numerical integrators, both errors vanish at a rate O ( η 2 ) , where η is the integrator step size. Our theoretical results are supported by an empirical study on a variety of regression and classiﬁcation tasks for Bayesian neural networks.