Miracle-dz/UniFAR-FTUs huggingface dataset:
COALA: Co-Aligned Autoencoders for Learning Semantically Enriched Audio Representations

Audio representation learning based on deep neural networks (DNNs) emerged as an alternative approach to hand-crafted features. For achieving high performance, DNNs often need a large amount of annotated data which can be difficult and costly to obtain. In this paper, we propose a method for learning audio representations, aligning the learned latent representations of audio and associated tags. Aligning is done by maximizing the agreement of the latent representations of audio and tags, using a contrastive loss. The result is an audio embedding model which reflects acoustic and semantic characteristics of sounds. We evaluate the quality of our embedding model, measuring its performance as a feature extractor on three different tasks (namely, sound event recognition, and music genre and musical instrument classification), and investigate what type of characteristics the model captures. Our results are promising, sometimes in par with the state-of-the-art in the considered tasks and the embeddings produced with our method are well correlated with some acoustic descriptors.
MSG-GAN: Multi-Scale Gradients for Generative Adversarial Networks

While Generative Adversarial Networks (GANs) have seen huge successes in image synthesis tasks, they are notoriously difficult to adapt to different datasets, in part due to instability during training and sensitivity to hyperparameters. One commonly accepted reason for this instability is that gradients passing from the discriminator to the generator become uninformative when there isn’t enough overlap in the supports of the real and fake distributions. In this work, we propose the Multi-Scale Gradient Generative Adversarial Network (MSG-GAN), a simple but effective technique for addressing this by allowing the flow of gradients from the discriminator to the generator at multiple scales. This technique provides a stable approach for high resolution image synthesis, and serves as an alternative to the commonly used progressive growing technique. We show that MSG-GAN converges stably on a variety of image datasets of different sizes, resolutions and domains, as well as different types of loss functions and architectures, all with the same set of fixed hyperparameters. When compared to state-of-the-art GANs, our approach matches or exceeds the performance in most of the cases we tried.
SE-MelGAN - Speaker Agnostic Rapid Speech Enhancement

Recent advancement in Generative Adversarial Networks in speech synthesis domain[3],[2] have shown, that it's possible to train GANs [8] in a reliable manner for high quality coherent waveform generation from mel-spectograms. We propose that it is possible to transfer the MelGAN's [3] robustness in learning speech features to speech enhancement and noise reduction domain without any model modification tasks. Our proposed method generalizes over multi-speaker speech dataset and is able to robustly handle unseen background noises during the inference. Also, we show that by increasing the batch size for this particular approach not only yields better speech results, but generalizes over multi-speaker dataset easily and leads to faster convergence. Additionally, it outperforms previous state of the art GAN approach for speech enhancement SEGAN [5] in two domains: 1. quality ; 2. speed. Proposed method runs at more than 100x faster than realtime on GPU and more than 2x faster than real time on CPU without any hardware optimization tasks, right at the speed of MelGAN [3].
Search for Optically Dark Infrared Galaxies without Counterparts of Subaru Hyper Suprime-Cam in the AKARI North Ecliptic Pole Wide Survey Field

We present the physical properties of AKARI sources without optical counterparts in optical images from the Hyper Suprime-Cam (HSC) on the Subaru telescope. Using the AKARI infrared (IR) source catalog and HSC optical catalog, we select 583 objects that do not have HSC counterparts in the AKARI North Ecliptic Pole wide survey field (∼5 deg2). Because the HSC limiting magnitude is deep (gAB ∼ 28.6), these are good candidates for extremely red star-forming galaxies (SFGs) and/or active galactic nuclei (AGNs), possibly at high redshifts. We compile multiwavelength data out to 500 μm and use them for fitting the spectral energy distribution with CIGALE to investigate the physical properties of AKARI galaxies without optical counterparts. We also compare their physical quantities with AKARI mid-IR selected galaxies with HSC counterparts. The estimated redshifts of AKARI objects without HSC counterparts range up to z ∼ 4, significantly higher than for AKARI objects with HSC counterparts. We find that (i) 3.6 – 4.5 μm color, (ii) AGN luminosity, (iii) stellar mass, (iv) star formation rate, and (v) V-band dust attenuation in the interstellar medium of AKARI objects without HSC counterparts are systematically larger than those of AKARI objects with counterparts. These results suggest that our sample includes luminous, heavily dust-obscured SFGs/AGNs at z ∼ 1–4 that are missed by previous optical surveys, providing very interesting targets for the coming era of the James Webb Space Telescope.
The Origin and Evolution of Mammals

1. Introduction 2. Time and Classification 3. Evolution of the Mammal like Reptiles 4. Evolution of Mammalian Biology 5. The Mesozoic Mammals 6. Living and Fossil Marsupials 7. Living and Fossil Placentals References Index
Neural gradients are lognormally distributed: understanding sparse and quantized training

Neural gradient compression remains a main bottleneck in improving training efficiency, as most existing neural network compression methods (e.g., pruning or quantization) focus on weights, activations, and weight gradients. However, these methods are not suitable for compressing neural gradients, which have a very different distribution. Specifically, we find that the neural gradients follow a lognormal distribution. Taking this into account, we suggest two methods to reduce the computational and memory burdens of neural gradients. The first one is stochastic gradient pruning, which can accurately set the sparsity level -- up to 85% gradient sparsity without hurting validation accuracy (ResNet18 on ImageNet). The second method determines the floating-point format for low numerical precision gradients (e.g., FP8). Our results shed light on previous findings related to local scaling, the optimal bit-allocation for the mantissa and exponent, and challenging workloads for which low-precision floating-point arithmetic has reported to fail. Reference implementation accompanies the paper.
Fine-Grained Analysis of Stability and Generalization for Stochastic Gradient Descent

Recently there are a considerable amount of work devoted to the study of the algorithmic stability and generalization for stochastic gradient descent (SGD). However, the existing stability analysis requires to impose restrictive assumptions on the boundedness of gradients, strong smoothness and convexity of loss functions. In this paper, we provide a fine-grained analysis of stability and generalization for SGD by substantially relaxing these assumptions. Firstly, we establish stability and generalization for SGD by removing the existing bounded gradient assumptions. The key idea is the introduction of a new stability measure called on-average model stability, for which we develop novel bounds controlled by the risks of SGD iterates. This yields generalization bounds depending on the behavior of the best model, and leads to the first-ever-known fast bounds in the low-noise setting using stability approach. Secondly, the smoothness assumption is relaxed by considering loss functions with Holder continuous (sub)gradients for which we show that optimal bounds are still achieved by balancing computation and stability. To our best knowledge, this gives the first-ever-known stability and generalization bounds for SGD with even non-differentiable loss functions. Finally, we study learning problems with (strongly) convex objectives but non-convex loss functions.
Post-Newtonian Lagrangian of an 
N
-body system with arbitrary mass and spin multipoles

The present paper derives the post-Newtonian Lagrangian of translational motion of N arbitrary-structured bodies with all mass and spin multipoles in a scalar-tensor theory of gravity. The multipoles depend on time and evolve in accordance with their own dynamic equations of motion. The Lagrangian is retrieved from the post-Newtonian equations of motion by solving the inverse problem of the Lagrangian mechanics and generalizes a well-known Lagrangian of pole-dipole-quadrupole massive particles to the particles of higher multipolarity. Analytic treatment of the higher-order multipole contributions is important for more rigorous computation of gravitational waveform of inspiralling compact binaries at the latest stage of their orbital evolution before merger when tidal and rotational deformations of stars are no longer small and rapidly change in time. The Lagrangian of an N-body system with arbitrary mass and spin multipoles is instrumental for formulation of the post-Newtonian conservation laws of energy, momenta and the integrals of the center of mass.
Evidence-Aware Inferential Text Generation with Vector Quantised Variational AutoEncoder

Generating inferential texts about an event in different perspectives requires reasoning over different contexts that the event occurs. Existing works usually ignore the context that is not explicitly provided, resulting in a context-independent semantic representation that struggles to support the generation. To address this, we propose an approach that automatically finds evidence for an event from a large text corpus, and leverages the evidence to guide the generation of inferential texts. Our approach works in an encoderdecoder manner and is equipped with Vector Quantised-Variational Autoencoder, where the encoder outputs representations from a distribution over discrete variables. Such discrete representations enable automatically selecting relevant evidence, which not only facilitates evidence-aware generation, but also provides a natural way to uncover rationales behind the generation. Our approach provides state-of-the-art performance on both Event2mind and Atomic datasets. More importantly, we find that with discrete representations, our model selectively uses evidence to generate different inferential texts.
Standardization of Propagation Models: 800 MHz to 100 GHz -- A Historical Perspective

Propagation models constitute a fundamental building block of wireless communications research. Before we build and operate real systems, we must understand the science of radio propagation, and develop channel models that both reflect the important propagation processes and allow a fair comparison of different systems. In the past five decades, wireless systems have gone through five generations, from supporting voice applications to enhanced mobile broadband. To meet the ever increasing data rate demands of wireless systems, frequency bands covering a wide range from 800 MHz to 100 GHz have been allocated for use. The standardization of these systems started in the early/mid 1980's in Europe by the European Telecommunications Standards Institute with the advent of Global System for Mobile Communications. This motivated the development of the first standardized propagation model by the European Cooperation in Science and Technology (COST) 207 working group. These standardization activities were continued and expanded for the third, fourth, and fifth generations of COST, as well as by the Third Generation Partnership Project, and the International Telecommunication Union. This paper presents a historical overview of the standardized propagation models covering first to fifth-generation systems. In particular, we discuss the evolution and standardization of pathloss models, as well as large and small-scale fading parameters for single antenna and multiple antenna systems. Furthermore, we present insights into the progress of deterministic modelling across the five generations of systems, as well as discuss more advanced modelling components needed for the detailed simulations of millimeter-wave channels. A comprehensive bibliography at the end of the paper will aid the interested reader to dig deeper.
Characterizing the radio continuum nature of sources in the massive star-forming region W75N (B)

The massive star-forming region W75N (B) is thought to host a cluster of massive protostars (VLA 1, VLA 2, and VLA 3) undergoing different evolutionary stages. In this work, we present radio continuum data with the highest sensitivity and angular resolution obtained to date in this region, using the VLA-A and covering a wide range of frequencies (4–48 GHz), which allowed us to study the morphology and the nature of the emission of the different radio continuum sources. We also performed complementary studies with multi-epoch Very Large Array (VLA) data and Atacama Large Millimeter Array (ALMA) archive data at 1.3 mm wavelength. We find that VLA 1 is driving a thermal radio jet at scales of ≈0.1 arcsec (≈130 au), but also shows signs of an incipient hypercompact H ii region at scales of ≲1 arcsec (≲1300 au). VLA 3 is also driving a thermal radio jet at scales of a few tenths of arcsec (few hundred of au). We conclude that this jet is shock exciting the radio continuum sources Bc and VLA 4 (obscured Herbig–Haro objects), which show proper motions moving outward from VLA 3 at velocities of ≈112–118 km s−1. We have also detected three new weak radio continuum sources, two of them associated with millimetre continuum cores observed with ALMA, suggesting that these two sources are also embedded young stellar objects in this massive star-forming region.
Scaling of causal neural avalanches in a neutral model

Neural avalanches are collective firings of neurons that exhibit emergent scale-free behavior. Understanding the nature and distribution of these avalanches is an important element in understanding how the brain functions. We study a model of the brain for which the dynamics are governed by neutral theory. The neural avalanches are defined using causal connections between the firing neurons. We analyze the scaling of causal neural avalanches as the critical point is approached from the absorbing phase. By using cluster analysis tools from percolation theory, we characterize the critical properties of the neural avalanches. We identify the tuning parameters consistent with experiments. The scaling hypothesis provides a unified explanation of the power laws which characterize the critical point. The critical exponents characterizing the avalanche distributions and divergence of the response functions are consistent with the predictions of the scaling hypothesis. We use an universal scaling function for the avalanche profile to find that the firing rate for avalanches of different sizes shows data collapse after appropriate rescaling. We also find data collapse for the avalanche distribution functions, which is a stronger evidence of criticality than just the existence of power laws. Critical slowing-down and power law relaxation of avalanches is observed as the system is tuned to its critical point. We discuss how our results motivate future empirical studies of criticality in the brain.
Bayesian causal inference with some invalid instrumental variables.

In observational studies, instrumental variables estimation is greatly utilized to identify causal effects. One of the key conditions for the instrumental variables estimator to be consistent is the exclusion restriction, which indicates that instruments affect the outcome of interest only via the exposure variable of interest. We propose a likelihood-free Bayesian approach to make consistent inferences about the causal effect when there are some invalid instruments in a way that they violate the exclusion restriction condition. Asymptotic properties of the proposed Bayes estimator, including consistency and normality, are established. A simulation study demonstrates that the proposed Bayesian method produces consistent point estimators and valid credible intervals with correct coverage rates for Gaussian and non-Gaussian data with some invalid instruments. We also demonstrate the proposed method through the real data application.
Continuous Control for Searching and Planning with a Learned Model

Decision-making agents with planning capabilities have achieved huge success in the challenging domain like Chess, Shogi, and Go. In an effort to generalize the planning ability to the more general tasks where the environment dynamics are not available to the agent, researchers proposed the MuZero algorithm that can learn the dynamical model through the interactions with the environment. In this paper, we provide a way and the necessary theoretical results to extend the MuZero algorithm to more generalized environments with continuous action space. Through numerical results on two relatively low-dimensional MuJoCo environments, we show the proposed algorithm outperforms the soft actor-critic (SAC) algorithm, a state-of-the-art model-free deep reinforcement learning algorithm.
Frequency, Timing, and Prediction of Major Bleeding Complications From Percutaneous Renal Biopsy

Background and Objectives: The risk and timing of bleeding events following ultrasound-guided percutaneous renal biopsy are not clearly defined. Design setting, participants, and measurements: We performed a retrospective study of 617 consecutive adult patients who underwent kidney biopsy between 2012 and 2017 at a tertiary academic hospital in London, Canada. We assessed frequency and timing of minor (not requiring intervention) and major (requiring blood transfusion, surgery, or embolization) bleeds and developed a personalized risk calculator for these. Results: Bleeding occurred in 79 patients (12.8%; 95% confidence interval [CI]: 10.4%-15.7%). Minor bleeding occurred in 67 patients (10.9%; 95% CI: 8.6%-13.6%). Major bleeding occurred in 12 patients (1.9%; 95% CI: 1.1%-3.4%); 2 required embolization or surgery (0.3%; 95% CI: 0.09%-1.2%) and 10 required blood transfusion (1.6%; 95% CI: 0.9%-3.0%). Seventy-three of 79 events were identified immediately on post-procedure ultrasound (92.4% of cases; 95% CI: 84.4%-96.5%). Four of 617 patients experienced a minor event not detected immediately (0.6%; 95% CI: 0.3%-1.7%). Two patients (0.3%; 95% CI: 0.09%-1.2%) suffered a major complication that was not recognized immediately; both required blood transfusions only. There were no deaths or nephrectomies. A risk calculator using age, body mass index, platelet count, hemoglobin concentration, size of the target kidney, and whether the kidney is native, or an allograft predicted minor (C-statistic, 0.70) and major bleeding (C-statistic, 0.83). Conclusions: This retrospective study of 617 patients who had percutaneous ultrasound-guided renal biopsies supports the safety of short post-biopsy monitoring for most patients. A risk calculator can further personalize estimates of complication risk (http://perioperativerisk.com/kbrc).
Role of Computed Tomography-guided Biopsies in the Era of Electromagnetic Navigational Bronchoscopy: A Retrospective Study of Factors Predicting Diagnostic Yield in Electromagnetic Navigational Bronchoscopy and Computed Tomography Biopsies

Objectives: Over 25% of the high-risk population screened for lung cancer have an abnormal computed tomography (CT) scan. Conventionally, these lesions have been biopsied with CT guidance with a high diagnostic yield. Electromagnetic navigational bronchoscopy (ENB) with transbronchial biopsy has emerged as a technology that improves the diagnostic sensitivity of conventional bronchoscopic biopsy. It has been used to biopsy lung lesions, due to the low risk of pneumothorax. It is, however, a new technology that is expensive and its role in the diagnosis of the solitary pulmonary nodule (SPN) is yet to be determined. The purpose of this study was to evaluate the diagnostic yield of CT-guided biopsy (CTB) following non-diagnostic ENB biopsy and identify characteristics of the lesion that predicts a low diagnostic yield with ENB, to ensure appropriate use of ENB in the evaluation of SPN. Materials and Methods: One hundred and thirty-five lung lesions were biopsied with ENB from January 2017 to August 2019. Biopsies were considered diagnostic if pathology confirmed malignancy or inflammation in the appropriate clinical and imaging setting. We evaluated lesions for several characteristics including size, lobe, and central/peripheral distribution. The diagnostic yield of CTB in patients who failed ENB biopsies was also evaluated. Logistic regression was used to identify factors likely to predict a non-diagnostic ENB biopsy. Result: Overall, ENB biopsies were performed in 135 patients with solitary lung lesions. ENB biopsies were diagnostic in 52% (70/135) of the patients. In 23 patients with solitary lung lesions, CTBs were performed following a non-diagnostic ENB biopsy. The CTBs were diagnostic in 87% of the patients (20/23). ENB biopsies of lesions <21.5 mm were non-diagnostic in 71% of cases (42/59); 14 of these patients with non-diagnostic ENB biopsies had CTBs, and 86% of them were diagnostic (12/14). ENB biopsies of lesions in the lower lobes were non- diagnostic in 59% of cases (35/59); 12 of these patients with non-diagnostic ENB biopsies had CTBs, and 83% were diagnostic (10/12). ENB biopsies of lesions in the outer 2/3 were non-diagnostic in 57% of cases (50/87); 21 of these patients with non-diagnostic ENB biopsies had CTBs, and 86% were diagnostic (18/21). Conclusion: CTBs have a high diagnostic yield even following non-diagnostic ENB biopsies. Lesions <21.5 mm, in the outer 2/3 of the lung, and in the lower lung have the lowest likelihood of a diagnostic yield with ENB biopsies. Although CTBs have a slightly higher pneumothorax rate, these lesions would be more successfully diagnosed with CTB as opposed to ENB biopsy, in the process expediting the diagnosis and saving valuable medical resources.
Pulmonary pleomorphic carcinoma with pembrolizumab monotherapy

Adenocarcinoma (ADC) is the most common form of lung cancer; however, some other types of lung cancer can sometimes mimic ADC. It then takes experienced pathologists and special stains to make the correct diagnosis. Here, we present a patient with pulmonary pleomorphic carcinoma who was misdiagnosed with ADC, and was treated with pembrolizumab for 13 months. The diagnosis and treatment (including immunotherapy) are also reviewed.