nils-herrmann/clean_scirepeval huggingface dataset:
Deep Bayesian Unsupervised Source Separation Based On A Complex Gaussian Mixture Model

This paper presents an unsupervised method that trains neural source separation by using only multichannel mixture signals. Conventional neural separation methods require a lot of supervised data to achieve excellent performance. Although multichannel methods based on spatial information can work without such training data, they are often sensitive to parameter initialization and degraded with the sources located close to each other. The proposed method uses a cost function based on a spatial model called a complex Gaussian mixture model (cGMM). This model has the time-frequency (TF) masks and direction of arrivals (DoAs) of sources as latent variables and is used for training separation and localization networks that respectively estimate these variables. This joint training solves the frequency permutation ambiguity of the spatial model in a unified deep Bayesian framework. In addition, the pre-trained network can be used not only for conducting monaural separation but also for efficiently initializing a multichannel separation algorithm. Experimental results with simulated speech mixtures showed that our method outperformed a conventional initialization method.
Variational Denoising Network: Toward Blind Noise Modeling and Removal

Blind image denoising is an important yet very challenging problem in computer vision due to the complicated acquisition process of real images. In this work we propose a new variational inference method, which integrates both noise estimation and image denoising into a unique Bayesian framework, for blind image denoising. Specifically, an approximate posterior, parameterized by deep neural networks, is presented by taking the intrinsic clean image and noise variances as latent variables conditioned on the input noisy image. This posterior provides explicit parametric forms for all its involved hyper-parameters, and thus can be easily implemented for blind image denoising with automatic noise estimation for the test noisy image. On one hand, as other data-driven deep learning methods, our method, namely variational denoising network (VDN), can perform denoising efficiently due to its explicit form of posterior expression. On the other hand, VDN inherits the advantages of traditional model-driven approaches, especially the good generalization capability of generative models. VDN has good interpretability and can be flexibly utilized to estimate and remove complicated non-i.i.d. noise collected in real scenarios. Comprehensive experiments are performed to substantiate the superiority of our method in blind image denoising.
Theoretical perspectives on biological machines

By operating out of equilibrium, nanoscale biological machines execute many functions that do not take place in abiotic systems. Statistical physics, physical chemistry, and polymer physics principles are needed in elucidating the rules and constraints governing large-scale structural changes that occur when metabolizing molecular fuel. This paper reviews the advances in theories rooted in coarse graining the complex systems while reminding us that molecular details must be an integral part of a deeper understanding of processes in living systems.
3D Anchor-Free Lesion Detector on Computed Tomography Scans

Lesions are injuries and abnormal tissues in the human body. Detecting lesions in 3D Computed Tomography (CT) scans can be time-consuming even for very experienced physicians and radiologists. In recent years, CNN based lesion detectors have demonstrated huge potentials. Most of current state-of-the-art lesion detectors employ anchors to enumerate all possible bounding boxes with respect to the dataset in process. This anchor mechanism greatly improves the detection performance while also constraining the generalization ability of detectors. In this paper, we propose an anchor-free lesion detector. The anchor mechanism is removed and lesions are formalized as single keypoints. By doing so, we witness a considerable performance gain in terms of both accuracy and inference speed compared with the anchor-based baseline.
Dark matter constraints on low mass and weakly coupled 
B−L
 gauge boson

We investigate constraints on the new $B-L$ gauge boson ($Z_{BL}$) mass and coupling ($g_{BL}$) in a $U(1)_{B-L}$ extension of the standard model (SM) with an SM singlet Dirac fermion ($\zeta$) as dark matter (DM). The DM particle $\zeta$ has an arbitrary $B-L$ charge $Q$ chosen to guarantee its stability. We focus on the small $Z_{BL}$ mass and small $g_{BL}$ regions of the model, and find new constraints for the cases where the DM relic abundance arises from thermal freeze-out as well as freeze-in mechanisms. In the thermal freeze-out case, the DM coupling is given by $g_{\zeta}\equiv g_{BL}Q\simeq0.016\sqrt{m_\zeta[{\rm GeV}]}$ to reproduce the observed DM relic density and $g_{BL}\geq 2.7 \times 10^{-8} \sqrt{m_\zeta[{\rm GeV}]}$ for the DM particle to be in thermal equilibrium prior to freeze-out. Combined with the direct and indirect DM detection constraints, we find that the allowed mass regions are limited to be $m_\zeta \gtrsim 200$ GeV and $M_{Z_{BL}} \gtrsim 10$ GeV. We then discuss the lower $g_{BL}$ values where the freeze-in scenario operates and find the following relic density constraints on parameters depending on the $g_{BL}$ range and dark matter mass: Case (A): for $g_{BL}\geq 2.7\times10^{-8}\sqrt{m_\zeta[{\rm GeV}]}$, one has $g^2_\zeta\,g^2_{BL}+\frac{0.82}{1.2}\,g^4_\zeta\simeq 8.2\times10^{-24}$ and Case (B): for $g_{BL} < 2.7 \times 10^{-8} \sqrt{m_\zeta[{\rm GeV}]}$, there are two separate constraints depending on $m_\zeta$. Case (B1): for $m_\zeta\lesssim 2.5{\rm TeV}$, we find $g_\zeta^2\,g_{BL}^2\simeq 8.2\times10^{-24}\,\left( \frac{m_\zeta}{2.5\,{\rm TeV}} \right)$ and case (B2): for $m_\zeta \gtrsim 2.5$ TeV, we have $g_\zeta^2 \, g_{BL}^2 \simeq 8.2 \times 10^{-24}$. For this case, we display the various parameter regions of the model that can be probed by a variety of ``Lifetime Frontier" experiments such as FASER, FASER2, Belle II, SHiP and LDMX.
Translate and Label! An Encoder-Decoder Approach for Cross-lingual Semantic Role Labeling

We propose a Cross-lingual Encoder-Decoder model that simultaneously translates and generates sentences with Semantic Role Labeling annotations in a resource-poor target language. Unlike annotation projection techniques, our model does not need parallel data during inference time. Our approach can be applied in monolingual, multilingual and cross-lingual settings and is able to produce dependency-based and span-based SRL annotations. We benchmark the labeling performance of our model in different monolingual and multilingual settings using well-known SRL datasets. We then train our model in a cross-lingual setting to generate new SRL labeled data. Finally, we measure the effectiveness of our method by using the generated data to augment the training basis for resource-poor languages and perform manual evaluation to show that it produces high-quality sentences and assigns accurate semantic role annotations. Our proposed architecture offers a flexible method for leveraging SRL data in multiple languages.
An Automated and Robust Image Watermarking Scheme Based on Deep Neural Networks

Digital image watermarking is the process of embedding and extracting a watermark covertly on a cover-image. To dynamically adapt image watermarking algorithms, deep learning–based image watermarking schemes have attracted increased attention during recent years. However, existing deep learning–based watermarking methods neither fully apply the fitting ability to learn and automate the embedding and extracting algorithms, nor achieve the properties of robustness and blindness simultaneously. In this paper, a robust and blind image watermarking scheme based on deep learning neural networks is proposed. To minimize the requirement of domain knowledge, the fitting ability of deep neural networks is exploited to learn and generalize an automated image watermarking algorithm. A deep learning architecture is specially designed for image watermarking tasks, which will be trained in an unsupervised manner to avoid human intervention and annotation. To facilitate flexible applications, the robustness of the proposed scheme is achieved without requiring any prior knowledge or adversarial examples of possible attacks. A challenging case of watermark extraction from phone camera–captured images demonstrates the robustness and practicality of the proposal. The experiments, evaluation, and application cases confirm the superiority of the proposed scheme.
Universal, transferable and targeted adversarial attacks

Deep Neural Network has been found vulnerable recently. A kind of well-designed inputs, which called adversarial examples, can lead the networks to make incorrect predictions. Depending on the different scenarios, goals and capabilities, the difficulty to generate the attack is different. For example, generating a targeted attack is more difficult than a non-targeted attack, a universal attack is more difficult than a non-universal attack, a transferable attack is more difficult than a nontransferable one. The question is: Is there exist an attack that can survival in the most harsh adversity to meet all these requirements. Although many cheap and effective attacks have been proposed, this question is still not completely solved over large models and large scale dataset. In this paper, we learn a universal mapping from the sources to the adversarial examples. These examples can fool classification networks into classifying all of them to one targeted class. Besides, they are also transferable between different models.
Ordered Sets for Data Analysis

This book dwells on mathematical and algorithmic issues of data analysis based on generality order of descriptions and respective precision. To speak of these topics correctly, we have to go some way getting acquainted with the important notions of relation and order theory. On the one hand, data often have a complex structure with natural order on it. On the other hand, many symbolic methods of data analysis and machine learning allow to compare the obtained classifiers w.r.t. their generality, which is also an order relation. Efficient algorithms are very important in data analysis, especially when one deals with big data, so scalability is a real issue. That is why we analyze the computational complexity of algorithms and problems of data analysis. We start from the basic definitions and facts of algorithmic complexity theory and analyze the complexity of various tools of data analysis we consider. The tools and methods of data analysis, like computing taxonomies, groups of similar objects (concepts and n-clusters), dependencies in data, classification, etc., are illustrated with applications in particular subject domains, from chemoinformatics to text mining and natural language processing.
Survey and Benchmarking of Machine Learning Accelerators

Advances in multicore processors and accelerators have opened the flood gates to greater exploration and application of machine learning techniques to a variety of applications. These advances, along with breakdowns of several trends including Moore’s Law, have prompted an explosion of processors and accelerators that promise even greater computational and machine learning capabilities. These processors and accelerators are coming in many forms, from CPUs and GPUs to ASICs, FPGAs, and dataflow accelerators. This paper surveys the current state of these processors and accelerators that have been publicly announced with performance and power consumption numbers. The performance and power values are plotted on a scatter graph and a number of dimensions and observations from the trends on this plot are discussed and analyzed. For instance, there are interesting trends in the plot regarding power consumption, numerical precision, and inference versus training. We then select and benchmark two commercially available low size, weight, and power (SWaP) accelerators as these processors are the most interesting for embedded and mobile machine learning inference applications that are most applicable to the DoD and other SWaP constrained users. We determine how they actually perform with real-world images and neural network models, compare those results to the reported performance and power consumption values and evaluate them against an Intel CPU that is used in some embedded applications.
Human-grounded Evaluations of Explanation Methods for Text Classification

Due to the black-box nature of deep learning models, methods for explaining the models’ results are crucial to gain trust from humans and support collaboration between AIs and humans. In this paper, we consider several model-agnostic and model-specific explanation methods for CNNs for text classification and conduct three human-grounded evaluations, focusing on different purposes of explanations: (1) revealing model behavior, (2) justifying model predictions, and (3) helping humans investigate uncertain predictions. The results highlight dissimilar qualities of the various explanation methods we consider and show the degree to which these methods could serve for each purpose.
PBH in single field inflation: the effect of shape dispersion and non-Gaussianities

Primordial black holes (PBHs) may result from high peaks in a random field of cosmological perturbations. In single field inflationary models, such perturbations can be seeded as the inflaton overshoots a small barrier on its way down the potential. PBHs are then produced through two distinct mechanisms, during the radiation era. The first one is the familiar collapse of large adiabatic overdensities. The second one is the collapse induced by relic bubbles where the inflaton field is trapped in a false vacuum. The latter are due to rare backward fluctuations of the inflaton which prevented it from overshooting the barrier in horizon sized regions. We consider (numerically and analytically) the effect of non-Gaussianities on the threshold for overdensities to collapse into a PBH. Since typical high peaks have some dispersion in their shape or profile, we also consider the effect of such dispersion on the corresponding threshold for collapse. With these results we estimate the most likely channel for PBH production as a function of the non-Gaussianity parameter fNL. We also compare the threshold for collapse coming from the perturbative versus the non perturbative template for the non-Gaussianity arising in this model. We show that i) for fNL ≳ 3.5, the population of PBH coming from false vacuum regions dominates over that which comes from the collapse of large adiabatic overdensities, ii) the non-perturbative template of the non-Gaussianities is important to get accurate results. iii) the effect of the dispersion is small in determining the threshold for the compaction function, although it can be appreciable in determining the threshold amplitude for the curvature perturbation at low fNL. We also confirm that the volume averaged compaction function provides a very accurate universal estimator for the threshold.
Private Heavy Hitters and Range Queries in the Shuffled Model

An exciting new development in differential privacy is the shuffled model, which makes it possible to circumvent the large error lower bounds that are typically incurred in the local model, while relying on much weaker trust assumptions than in the central model. In this work, we study two basic statistical problems, namely, heavy hitters and d-dimensional range counting queries, in the shuffled model of privacy. For both problems we devise algorithms with polylogarithmic communication per user and polylogarithmic error; a direct consequence is an algorithm for approximating the median with similar communication and error. These bounds significantly improve on what is possible in the local model of differential privacy, where the error must provably grow polynomially with the number of users. ∗MIT. Work done while at Google Research. †Visiting from the IT University of Copenhagen. 1 ar X iv :1 90 8. 11 35 8v 1 [ cs .C R ] 2 9 A ug 2 01 9
Improving Deep Transformer with Depth-Scaled Initialization and Merged Attention

The general trend in NLP is towards increasing model capacity and performance via deeper neural networks. However, simply stacking more layers of the popular Transformer architecture for machine translation results in poor convergence and high computational overhead. Our empirical analysis suggests that convergence is poor due to gradient vanishing caused by the interaction between residual connection and layer normalization. We propose depth-scaled initialization (DS-Init), which decreases parameter variance at the initialization stage, and reduces output variance of residual connections so as to ease gradient back-propagation through normalization layers. To address computational cost, we propose a merged attention sublayer (MAtt) which combines a simplified average-based self-attention sublayer and the encoder-decoder attention sublayer on the decoder side. Results on WMT and IWSLT translation tasks with five translation directions show that deep Transformers with DS-Init and MAtt can substantially outperform their base counterpart in terms of BLEU (+1.1 BLEU on average for 12-layer models), while matching the decoding speed of the baseline model thanks to the efficiency improvements of MAtt. Source code for reproduction will be released soon.
A Machine Learning Accelerator In-Memory for Energy Harvesting

There is increasing demand to bring machine learning capabilities to low power devices. By integrating the computational power of machine learning with the deployment capabilities of low power devices, a number of new applications become possible. In some applications, such devices will not even have a battery, and must rely solely on energy harvesting techniques. This puts extreme constraints on the hardware, which must be energy efficient and capable of tolerating interruptions due to power outages. Here, as a representative example, we propose an in-memory support vector machine learning accelerator utilizing non-volatile spintronic memory. The combination of processing-in-memory and non-volatility provides a key advantage in that progress is effectively saved after every operation. This enables instant shut down and restart capabilities with minimal overhead. Additionally, the operations are highly energy efficient leading to low power consumption.
The missing link in gravitational-wave astronomy: discoveries waiting in the decihertz range

The gravitational-wave astronomical revolution began in 2015 with LIGO’s observation of the coalescence of two stellar-mass black holes. Over the coming decades, ground-based detectors like laser interferometer gravitational-wave observatory (LIGO), Virgo and KAGRA will extend their reach, discovering thousands of stellar-mass binaries. In the 2030s, the space-based laser interferometer space antenna (LISA) will enable gravitational-wave observations of the massive black holes in galactic centres. Between ground-based observatories and LISA lies the unexplored dHz gravitational-wave frequency band. Here, we show the potential of a decihertz observatory (DO) which could cover this band, and complement discoveries made by other gravitational-wave observatories. The dHz range is uniquely suited to observation of intermediate-mass (∼102–104 M ⊙) black holes, which may form the missing link between stellar-mass and massive black holes, offering an opportunity to measure their properties. DOs will be able to detect stellar-mass binaries days to years before they merge and are observed by ground-based detectors, providing early warning of nearby binary neutron star mergers, and enabling measurements of the eccentricity of binary black holes, providing revealing insights into their formation. Observing dHz gravitational-waves also opens the possibility of testing fundamental physics in a new laboratory, permitting unique tests of general relativity (GR) and the standard model of particle physics. Overall, a DO would answer outstanding questions about how black holes form and evolve across cosmic time, open new avenues for multimessenger astronomy, and advance our understanding of gravitation, particle physics and cosmology.
Higgs bosons with large couplings to light quarks

A common lore has arisen that beyond the Standard Model (BSM) particles, which can be searched for at current and proposed experiments, should have flavorless or mostly third-generation interactions with Standard Model quarks. This theoretical bias severely limits the exploration of BSM phenomenology, and is especially constraining for extended Higgs sectors. Such limitations can be avoided in the context of Spontaneous Flavor Violation (SFV), a robust and UV complete framework that allows for significant couplings to any up or down-type quark, while suppressing flavor-changing neutral currents via flavor alignment. In this work we study the theory and phenomenology of extended SFV Higgs sectors with large couplings to any quark generation. We perform a comprehensive analysis of flavor and collider constraints of extended SFV Higgs sectors, and demonstrate that new Higgs bosons with large couplings to the light quarks may be found at the electroweak scale. In particular, we find that new Higgses as light as 100 GeV with order $\sim$ 0.1 couplings to first or second generation quarks, which are copiously produced at LHC via quark fusion, are allowed by current constraints. Furthermore, the additional SFV Higgses can mix with the SM Higgs, providing strong theory motivation for an experimental program looking for deviations in the light quark-Higgs couplings. Our work demonstrates the importance of exploring BSM physics coupled preferentially to light quarks, and the need to further develop dedicated experimental techniques for the LHC and future colliders.
Dust accretion in binary systems: implications for planets and transition discs

The presence of planets in binary systems poses interesting problems for planet formation theories, both in cases where planets must have formed in very compact discs around the individual stars and where they are located near the edge of the stable circumbinary region, where in situ formation is challenging. Dust dynamics is expected to play an important role in such systems, since dust trapping at the inner edge of circumbinary discs could aid in situ formation, but would simultaneously starve the circumstellar discs of the solid material needed to form planets. Here we investigate the dynamics of dust in binary systems using Smooth Particle Hydrodynamics. We find that all our simulations tend towards dust trapping in the circumbinary disc, but the timescale on which trapping begins depends on binary mass ratio (q) and eccentricity as well as the angular momentum of the infalling material. For q ≳ 0.1, we find that dust can initially accrete onto the circumstellar discs, but as the circumbinary cavity grows in radius, dust eventually becomes trapped in the circumbinary disc. For q = 0.01, we find that increasing the binary eccentricity increases the time required for dust trapping to begin. However, even this longer timescale is likely to be shorter than the planet formation timescale in the inner disc and is insufficient to explain the observed pre-transitional discs. This indicates that increase in companion eccentricity alone is not enough to allow significant transfer of solids from the outer to the inner disc.