Miracle-dz/UniFAR-FTUs huggingface dataset:
High probability convergence and uniform stability bounds for nonconvex stochastic gradient descent

Stochastic gradient descent (with a mini-batch) is one of the most common iterative algorithms used in machine learning. While being computationally cheap to implement, recent literature suggests that it may also have implicit regularization properties that prevent overfitting. This paper analyzes the properties of stochastic gradient descent from a theoretical standpoint to help bridge the gap between theoretical and empirical results; in particular, we prove bounds that depend explicitly on the number of epochs. Assuming smoothness, the Polyak-Łojasiewicz inequality, and the bounded variation property, we prove high probability bounds on the convergence rate. Assuming Lipschitz continuity and smoothness, we prove high probability bounds on the uniform stability. Putting these together (noting that some of the assumptions imply each other), we bound the true risk of the iterates of stochastic gradient descent. For convergence, our high probability bounds match existing expected bounds. For stability, our high probability bounds extend the nonconvex expected bound in Hardt et al. (2015). We use existing results to bound the generalization error using the stability. Finally, we put the convergence and generalization bounds together. We find that for a certain number of epochs of stochastic gradient descent, the convergence and generalization balance and we get a true risk bound that goes to zero as the number of samples goes to infinity.
Optimal Continual Learning has Perfect Memory and is NP-hard

Continual Learning (CL) algorithms incrementally learn a predictor or representation across multiple sequentially observed tasks. Designing CL algorithms that perform reliably and avoid so-called catastrophic forgetting has proven a persistent challenge. The current paper develops a theoretical approach that explains why. In particular, we derive the computational properties which CL algorithms would have to possess in order to avoid catastrophic forgetting. Our main finding is that such optimal CL algorithms generally solve an NP-hard problem and will require perfect memory to do so. The findings are of theoretical interest, but also explain the excellent performance of CL algorithms using experience replay, episodic memory and core sets relative to regularization-based approaches.
Overcoming the Weight Transport Problem via Spike-Timing-Dependent Weight Inference

We propose a solution to the weight transport problem, which questions the biological plausibility of the backpropagation algorithm. We derive our method based upon a theoretical analysis of the (approximate) dynamics of leaky integrate-and-fire neurons. We show that the use of spike timing alone outcompetes existing biologically plausible methods for synaptic weight inference in spiking neural network models. Furthermore, our proposed method is more flexible, being applicable to any spiking neuron model, is conservative in how many parameters are required for implementation and can be deployed in an online-fashion with minimal computational overhead. These features, together with its biological plausibility, make it an attractive mechanism underlying weight inference at single synapses.
Towards Domain Independence in CNN-based Acoustic Localization using Deep Cross Correlations

Time delay estimation is essential in Acoustic Source Localization (ASL) systems. One of the most used techniques for this purpose is the Generalized Cross Correlation (GCC) between a pair of signals and its use in Steered Response Power (SRP) techniques, which estimate the acoustic power at a specific location. Nowadays, Deep Learning strategies may outperform these methods. However, they are generally dependent on the geometric and sensor configuration conditions that are available during the training phases, thus having limited generalization capabilities when facing new environments if no re-training nor adaptation is applied. In this work, we propose a method based on an encoder-decoder CNN architecture capable of outperforming the well known SRP-PHAT algorithm, and also other Deep Learning strategies when working in mismatched training-testing conditions without requiring a model re-training. Our proposal aims to estimate a smoothed version of the correlation signals, that is then used to generate a refined acoustic power map, which leads to better performance on the ASL task. Our experimental evaluation uses three publicly available realistic datasets and provides a comparison with the SRP-PHAT algorithm and other recent proposals based on Deep Learning.
Feed Intake, Weight Gain, and Behavioral Activity of Heifers on an Electronic or a Tie-Up Feeding System

Abstract Feed intakes, weight changes, and behavioral activity of heifers on either an electronically controlled or tie-stall feeding system were compared in two trials with 16 heifers in each trial. Eight heifers per trial were fed individually by one of the two systems. Trial 1 was for 6 wk; trial 2 for 18 wk. Behavioral activity was observed directly and by closed circuit television. Daily intakes of feed were not different for heifers on electronic doors from those on tie-stall feeding during trial 1. During trial 2 heifers on a switchback experiment consumed more feeding with electronic doors than on tie-stall feeding (10.8 versus 9.9 kg daily). A large interaction of group by treatment indicated higher intake by some heifers on tie-stall feeding. Average daily gain was not measured during trial 1 but was . 49 kg for heifers on electronic feeding and . 67 in tie stalls during trial 2. The correlation between average daily gain and feed consumption was −. 18. Heifers fed electronically spent more time feeding and standing but less time lying down than heifers on tie-stall feeding during trial 1. Television surveillance of heifers on electronic control during trial 2 revealed considerable individuality in feeding habits. Heifers ate alone 47% of the observed feeding time. Feeding with electronic doors can be a useful innovation with monitoring and repair.
Description of a skeleton and supplementary notes on the skull of a rare porpoise Phocoena sinus Norris & McFarland 1958

Summary The skull and incomplete skeleton of a specimen of Phocoena sinus from the Gulf of California are described and compared with specimens of P. phocoena and P. spinipinnis. The skeleton of the specimen is the first to become available for description and from it and from the skull additional specific criteria have been determined that support the status given to this rare species by Norris & McFarland.
Real-time Neural Networks Implementation Proposal for Microcontrollers

The adoption of intelligent systems with Artificial Neural Networks (ANNs) embedded in hardware for real-time applications currently faces a growing demand in fields such as the Internet of Things (IoT) and Machine to Machine (M2M). However, the application of ANNs in this type of system poses a significant challenge due to the high computational power required to process its basic operations. This paper aims to show an implementation strategy of a Multilayer Perceptron (MLP)-type neural network, in a microcontroller (a low-cost, low-power platform). A modular matrix-based MLP with the full classification process was implemented as was the backpropagation training in the microcontroller. The testing and validation were performed through Hardware-In-the-Loop (HIL) of the Mean Squared Error (MSE) of the training process, classification results, and the processing time of each implementation module. The results revealed a linear relationship between the values of the hyperparameters and the processing time required for classification, also the processing time concurs with the required time for many applications in the fields mentioned above. These findings show that this implementation strategy and this platform can be applied successfully in real-time applications that require the capabilities of ANNs.
Neural Physicist: Learning Physical Dynamics from Image Sequences

We present a novel architecture named Neural Physicist (NeurPhy) to learn physical dynamics directly from image sequences using deep neural networks. For any physical system, given the global system parameters, the time evolution of states is governed by the underlying physical laws. How to learn meaningful system representations in an end-to-end way and estimate accurate state transition dynamics facilitating long-term prediction have been long-standing challenges. In this paper, by leveraging recent progresses in representation learning and state space models (SSMs), we propose NeurPhy, which uses variational auto-encoder (VAE) to extract underlying Markovian dynamic state at each time step, neural process (NP) to extract the global system parameters, and a non-linear non-recurrent stochastic state space model to learn the physical dynamic transition. We apply NeurPhy to two physical experimental environments, i.e., damped pendulum and planetary orbits motion, and achieve promising results. Our model can not only extract the physically meaningful state representations, but also learn the state transition dynamics enabling long-term predictions for unseen image sequences. Furthermore, from the manifold dimension of the latent state space, we can easily identify the degree of freedom (DoF) of the underlying physical systems.
Morphology and surface photometry of a sample of isolated early-type galaxies from deep imaging

Isolated early-type galaxies (iETGs) are evolving in unusually poor environments for this morphological family, which is typical of cluster inhabitants. We investigate the mechanisms driving the evolution of these galaxies. Several studies indicate that interactions, accretions, and merging episodes leave their signature on the galaxy structure, from the nucleus down to the faint outskirts. We focus on revealing such signatures, if any, in a sample of iETGs, and we quantitatively revise their galaxy classification. We observed 20 (out of 104) iETGs, selected from the AMIGA catalog, with the 4KCCD camera at the VATT in the SDSS g and r bands. These are the deepest observations of a sample of iETGs so far. The analysis was performed using the AIDA package, providing PSF-corrected 2D surface photometry up to the galaxy outskirts. The package provides a model of the 2D galaxy light distribution, which after model subtraction enhances the fine and peculiar structures in the residual image of the galaxies. Our re-classification suggests that the sample is composed of bona fide ETGs spanning from ellipticals to late-S0s galaxies. Most of the surface brightness profiles are best fitted with a bulge plus disc model, suggesting the presence of an underlying disc structure. The residuals obtained after the model subtraction show the nearly ubiquitous presence of fine structures, such as shells, stellar fans, rings, and tails. Shell systems are revealed in about 60% of these galaxies. Because interaction, accretion, and merging events are widely interpreted as the origin of the fans, ripples, shells and tails in galaxies, we suggest that most of these iETGs have experienced such events. Because they are isolated (after 2-3 Gyr), these galaxies are the cleanest environment in which to study phenomena connected with events like these.
Towards Good Practices for Data Augmentation in GAN Training

Recent successes in Generative Adversarial Networks (GAN) have affirmed the importance of using more data in GAN training. Yet it is expensive to collect data in many domains such as medical applications. Data Augmentation (DA) has been applied in these applications. In this work, we first argue that the classical DA approach could mislead the generator to learn the distribution of the augmented data, which could be different from that of the original data. We then propose a principled framework, termed Data Augmentation Optimized for GAN (DAG), to enable the use of augmented data in GAN training to improve the learning of the original distribution. We provide theoretical analysis to show that using our proposed DAG aligns with the original GAN in minimizing the JS divergence w.r. t. the original distribution and it leverages the augmented data to improve the learnings of discriminator and generator. The experiments show that DAG improves various GAN models. Furthermore, when DAG is used in some GAN models, the system establishes state-of-the-art Frechet Inception Distance (FID) scores.
A Separation-Based Methodology to Consensus Tracking of Switched High-Order Nonlinear Multiagent Systems

This work investigates a reduced-complexity adaptive methodology to consensus tracking for a team of uncertain high-order nonlinear systems with switched (possibly asynchronous) dynamics. It is well known that high-order nonlinear systems are intrinsically challenging as feedback linearization and backstepping methods successfully developed for low-order systems fail to work. Even the adding-one-power-integrator methodology, well explored for the single-agent high-order case, presents some complexity issues and is unsuited for distributed control. At the core of the proposed distributed methodology is a newly proposed definition for separable functions: this definition allows the formulation of a separation-based lemma to handle the high-order terms with reduced complexity in the control design. Complexity is reduced in a twofold sense: the control gain of each virtual control law does not have to be incorporated in the next virtual control law iteratively, thus leading to a simpler expression of the control laws; the power of the virtual and actual control laws increases only proportionally (rather than exponentially) with the order of the systems, dramatically reducing high-gain issues.
Core RNA Polymerase and Promoter DNA Interactions of Purified Domains of σN: Bipartite Functions

Abstract The σ N class of sigma factors confer upon RNA polymerase the requirement for enhancer-binding activator proteins. The sigma-N (σ N ) protein of Klebsiella pneumoniae was analysed by the assay of purified peptides comprising domains or regions of σ N defined by proteolysis or by homology alignment, respectively. The NH 2 -terminal Region I is required for the correct interaction of holoenzyme with the promoter, and promoter complexes forming with a truncated σ N lacking Region I are not activatable. The complexes lack the DNA structure believed to represent nucleated strand separation but still make close contacts with this promoter part. Determinants of specific DNA recognition by σ N were shown to reside in a C-terminal 16 kDa peptide, and core RNA polymerase binding determinants in an adjacent peptide. The latter contacts and appears to pack against the DNA-binding domain. Thus the DNA-binding and core-binding domains are bipartite in function, consistent with core functioning as an allosteric effector of the sigma DNA-binding activity. The DNA-binding and core-binding domains together include Region III of σ N . Although not the primary determinant of core or DNA recognition, the acidic Region II of σ N influenced both activities. Regions I and II in combination with core RNA polymerase thus appear to control the activity of C-terminal DNA contacting surfaces to allow formation of a closed promoter complex that is susceptible to activation.
Transcription Analysis of Differentially Expressed Genes in Insecticide-Resistant Aedes aegypti Mosquitoes After Deltamethrin Exposure

ABSTRACT: Aedes aegypti mosquitoes are resistant to various insecticides, including pyrethroids, throughout Thailand. We previously reported that Ae. aegypti from Mae Wong district, Nakhon Sawan Province in north-central Thailand, were resistant to insecticides, including pyrethroids (deltamethrin and permethrin), organophosphates and carbamates, and that high levels of detoxification enzymes were present. In the present study we used the method of suppression by subtractive hybridization to determine differential expression of genes in Mae Wong Ae. aegypti that survived the exposure to increasing doses (∼ 1.5 - 2 × 10-5M) of deltamethrin beyond the diagnostic dose compared to unexposed mosquitoes. Screening of 350 cDNA clones from the suppression subtractive library by cDNA array hybridization revealed that 58 clones were over-expressed in the mosquito that survived high dose deltamethrin. The over-expressed cDNA insert sequences corresponded to 11 functional genes, five hypothetical protein genes, and five sequences of unknown function that could be located on the supercontig of the Ae. aegypti genome. The functional genes are those coding for cuticular proteins, muscle proteins, proteins related to controlling the release of synaptic vesicles, and other genes such as heat shock protein and small subunit ribosomal RNA. Over-expression of tomosyn and myosin light chain kinase genes was verified using a semiquantitative reverse transcription-polymerase chain reaction (RT-PCR), confirming their increased expression in response to deltamethrin exposure in insecticide-resistant Ae. aegypti.
Fluoxetine inhibits predator avoidance behavior in tadpoles

Fluoxetine is a selective serotonin re-uptake inhibitor used to treat anxiety and depression in humans. It has been detected as a contaminant in the surface waters in many countries. The effects of fluoxetine (0, 0.03, 0.3, and 3 μg L−1) on the swimming and behavioral responses of Bufo arabicus tadpoles to alarm chemicals from predatory dragonfly larvae (Anax imperator) were measured. Fluoxetine significantly reduced swimming speed at 0.3 and 3 μg L−1 in the absence of predator alarm chemicals but had no effect in their presence. Tadpoles exposed to predator alarm chemicals avoided open water and preferred to hide. Exposure to fluoxetine at 3 μg L−1 completely eliminated this predator avoidance response, making the tadpoles more vulnerable to predation.
The genetic structure and microdistribution of shoals of Phoxinus phoxinus, the European minnow

Efficient foraging and a reduction in predation risk have been proposed as reasons for shoal formation. Some behaviours in cyprinid shoals are at first sight altruistic (e.g. predator inspection behaviour, reactions to alarm substance), such that kin selection may have been involved in their evolution. If shoaling behaviour does evolve through kin selection, then genetic differentiation is expected to be greater between shoals than within shoals. Such a hypothesis was tested here by examining shoal integrity and the relatedness of individuals within and between shoals in the European minnow, Phoxinus phoxinus, using nuclear and mitochondrial DNA markers. The breeding structure of 13 minnow shoals collected from Dorset and North Wales, U.K., was examined using allozymes. Genetic affinity within and between shoals was tested using mitochondrial DNA and multi-locus DNA fingerprinting. Shoals consisted of a random assortment of allozyme genotypes, shoal members did not share the same maternal mtDNA lineages and DNA fingerprint profiles were as varied within shoals as between them. The data indicate that it is unlikely that kin selection occurs in P. phoxinus and there is no apparent relationship between shoaling behaviour and genotype distribution in this species.
The ALPINE-ALMA [CII] Survey: nature, luminosity function and star formation history of continuum non-target galaxies up to z~6

We present the detailed characterisation of a sample of 56 sources serendipitously detected in ALMA Band-7, as part of the ALMA Large Program to INvestigate CII at Early Times (ALPINE) in COSMOS and ECDFS. These sources have been used to derive the total infrared luminosity function (LF) and to estimate the cosmic star formation rate density (SFRD) up to z=6. We have looked for counterparts in all the available multi-wavelength and photometric redshift catalogues, and in deeper near- and mid-IR source lists and maps, to identify optically dark sources with no matches in the public catalogues. Our ALMA blind survey allows us to push further the study of the nature and evolution of dusty galaxies at high-z, identifying luminous and massive sources to redshifts and faint luminosities never probed before by any far-infrared surveys. The ALPINE data are the first ones to sample the faint-end of the infrared LF, showing little evolution from z=2.5 to z=6, and a flat slope up to the highest redshifts. The SFRD obtained by integrating the luminosity function remains almost constant between z=2 and 6, and significantly higher than the optical/UV derivations, showing an important contribution of dusty galaxies and obscured star formation up to high-z. About 16. per cent of the ALPINE serendipitous continuum sources are optically+near-IR dark (7 show a counterpart only in the mid-IR and no HST or near-IR identification, while 2 are detected as [CII] emitters at z=5). The 7 HST and near-IR dark galaxies with mid-IR counterpart contribute for about 15 per cent of the total SFRD at z=5 and dominate the high-mass end of the stellar mass function at z>3.
Planning in Markov Decision Processes with Gap-Dependent Sample Complexity

We propose MDP-GapE, a new trajectory-based Monte-Carlo Tree Search algorithm for planning in a Markov Decision Process in which transitions have a finite support. We prove an upper bound on the number of calls to the generative models needed for MDP-GapE to identify a near-optimal action with high probability. This problem-dependent sample complexity result is expressed in terms of the sub-optimality gaps of the state-action pairs that are visited during exploration. Our experiments reveal that MDP-GapE is also effective in practice, in contrast with other algorithms with sample complexity guarantees in the fixed-confidence setting, that are mostly theoretical.
Revisiting Few-sample BERT Fine-tuning

We study the problem of few-sample fine-tuning of BERT contextual representations, and identify three sub-optimal choices in current, broadly adopted practices. First, we observe that the omission of the gradient bias correction in the BERTAdam optimizer results in fine-tuning instability. We also find that parts of the BERT network provide a detrimental starting point for fine-tuning, and simply re-initializing these layers speeds up learning and improves performance. Finally, we study the effect of training time, and observe that commonly used recipes often do not allocate sufficient time for training. In light of these findings, we re-visit recently proposed methods to improve few-sample fine-tuning with BERT and re-evaluate their effectiveness. Generally, we observe a decrease in their relative impact when modifying the fine-tuning process based on our findings.
The Slow Heartbeats of an Ultraluminous X-Ray Source in NGC 3621

We report on the results of X-ray observations of 4XMM J111816.0–324910, a transient ultraluminous X-ray source located in the galaxy NGC 3621. This system is characterized by a transient nature and marked variability with a characteristic timescale of ≈3500 s, in contract with other ultraluminous X-ray sources, which in the vast majority show limited intra-observation variability. Such a behavior is very reminiscent of the so-called heartbeats sometimes observed in the Galactic black hole binary GRS 1915+105, where the variability timescale is ∼10–1000 s. We study the spectral and timing properties of this object and find that overall, once the differences in the variability timescales are taken into account, they match quite closely those of both GRS 1915+105 and of a number of objects showing heartbeats in their light curves, including a confirmed neutron star and a supermassive black hole powering an active galactic nucleus. We investigate the nature of the compact object in 4XMM J111816.0–324910 by searching for typical neutron star signatures and by attempting a mass estimate based on different methods and assumptions. Based on the current available data, we are not able to unambiguously determine the nature of the accreting compact object responsible for the observed phenomenology.