julilili42/papers huggingface dataset:
Exemplary Natural Images Explain CNN Activations Better than State-of-the-Art Feature Visualization
2022-02-24

Feature visualizations such as synthetic maximally activating images are a widely used explanation method to better understand the information processing of convolutional neural networks (CNNs). At the same time, there are concerns that these visualizations might not accurately represent CNNs' inner workings. Here, we measure how much extremely activating images help humans to predict CNN activations. Using a well-controlled psychophysical paradigm, we compare the informativeness of synthetic images by Olah et al. (2017) with a simple baseline visualization, namely exemplary natural images that also strongly activate a specific feature map. Given either synthetic or natural reference images, human participants choose which of two query images leads to strong positive activation. The experiments are designed to maximize participants' performance, and are the first to probe intermediate instead of final layer representations. We find that synthetic images indeed provide helpful information about feature map activations ( accuracy; chance would be ). However, natural images - originally intended as a baseline - outperform synthetic images by a wide margin ( ). Additionally, participants are faster and more confident for natural images, whereas subjective impressions about the interpretability of the feature visualizations are mixed. The higher informativeness of natural images holds across most layers, for both expert and lay participants as well as for hand- and randomly-picked feature visualizations. Even if only a single reference image is given, synthetic images provide less information than natural images ( vs. ). In summary, synthetic images from a popular feature visualization method are significantly less informative for assessing CNN activations than natural images. We argue that visualization methods should improve over this baseline.
Eye Movement Feature Classification for Soccer Goalkeeper Expertise Identification in Virtual Reality
2022-02-24

The latest research in expertise assessment of soccer players has affirmed the importance of perceptual skills (especially for decision making) by focusing either on high experimental control or on a realistic presentation. To assess the perceptual skills of athletes in an optimized manner, we captured omnidirectional in-field scenes and showed these to 12 expert, 10 intermediate and 13 novice soccer goalkeepers on virtual reality glasses. All scenes were shown from the same natural goalkeeper perspective and ended after the return pass to the goalkeeper. Based on their gaze behavior we classified their expertise with common machine learning techniques. This pilot study shows promising results for objective classification of goalkeepers expertise based on their gaze behaviour and provided valuable insight to inform the design of training systems to enhance perceptual skills of athletes.
Efficient Message Passing for 0-1 ILPs with Binary Decision Diagrams
2022-02-24

We present a message passing method for 0-1 integer linear programs. Our algorithm is based on a decomposition of the original problem into subproblems that are represented as binary decision diagrams. The resulting Lagrangean dual is solved iteratively by a series of efficient block coordinate ascent steps. Our method has linear iteration complexity in the size of the decomposition and can be effectively parallelized. The characteristics of our approach are desirable towards solving ever larger problems arising in structured prediction. We present experimental results on combinatorial problems from MAP inference for Markov Random Fields, quadratic assignment, discrete tomography and cell tracking for developmental biology and show promising performance.
Learnable Uncertainty under Laplace Approximations
2022-02-24

Laplace approximations are classic, computationally lightweight means for constructing Bayesian neural networks (BNNs). As in other approximate BNNs, one cannot necessarily expect the induced predictive uncertainty to be calibrated. Here we develop a formalism to explicitly "train" the uncertainty in a decoupled way to the prediction itself. To this end, we introduce uncertainty units for Laplace-approximated networks: Hidden units associated with a particular weight structure that can be added to any pre-trained, point-estimated network. Due to their weights, these units are inactive -- they do not affect the predictions. But their presence changes the geometry (in particular the Hessian) of the loss landscape, thereby affecting the network's uncertainty estimates under a Laplace approximation. We show that such units can be trained via an uncertainty-aware objective, improving standard Laplace approximations' performance in various uncertainty quantification tasks.
Probabilistic Linear Solvers for Machine Learning
2022-02-24

Linear systems are the bedrock of virtually all numerical computation. Machine learning poses specific challenges for the solution of such systems due to their scale, characteristic structure, stochasticity and the central role of uncertainty in the field. Unifying earlier work we propose a class of probabilistic linear solvers which jointly infer the matrix, its inverse and the solution from matrix-vector product observations. This class emerges from a fundamental set of desiderata which constrains the space of possible algorithms and recovers the method of conjugate gradients under certain conditions. We demonstrate how to incorporate prior spectral information in order to calibrate uncertainty and experimentally showcase the potential of such solvers for machine learning.
Strong Rates of Convergence for Space-Time Discretization of the Backward Stochastic Heat Equation, and of a Linear-Quadratic Control Problem for the Stochastic Heat Equation
2022-02-24

We introduce a time-implicit, finite-element based space-time discretization scheme for the backward stochastic heat equation, and for the forward-backward stochastic heat equation from stochastic optimal control, and prove strong rates of convergence. The fully discrete version of the forward-backward stochastic heat equation is then used within a gradient descent algorithm to approximately solve the linear-quadratic control problem for the stochastic heat equation driven by additive noise.
Out-distribution aware Self-training in an Open World Setting
2022-02-24

Deep Learning heavily depends on large labeled datasets which limits further improvements. While unlabeled data is available in large amounts, in particular in image recognition, it does not fulfill the closed world assumption of semi-supervised learning that all unlabeled data are task-related. The goal of this paper is to leverage unlabeled data in an open world setting to further improve prediction performance. For this purpose, we introduce out-distribution aware self-training, which includes a careful sample selection strategy based on the confidence of the classifier. While normal self-training deteriorates prediction performance, our iterative scheme improves using up to 15 times the amount of originally labeled data. Moreover, our classifiers are by design out-distribution aware and can thus distinguish task-related inputs from unrelated ones.
Microchannel-Plate Detector Development for Ultraviolet Missions
2022-02-24

The Institute for Astronomy and Astrophysics in Tübingen (IAAT) has a long-term experience in developing and building space-qualified imaging and photon counting microchannel-plate (MCP) detectors, which are sensitive in the ultraviolet wavelength range. Our goal is to achieve high quantum efficiency and spatial resolution, while maintaining solar blindness and low-noise characteristics. Our flexible detector design is currently tailored to the specific needs of three missions: For the ESBO DS (European Stratospheric Balloon Observatory Design Study) we provide a sealed detector to the STUDIO instrument (Stratospheric Ultraviolet Demonstrator of an Imaging Observatory), a 50 cm telescope with a UV imager for operation at an altitude of 37-41 km. In collaboration with the Indian Institute of Astrophysics we plan a space mission with a CubeSat-sized far-ultraviolet spectroscopic imaging instrument, featuring an open version of our detector. A Chinese mission, led by the Purple Mountain Observatory, comprises a multi-channel imager using open and sealed detector versions. Our MCP detector has a cesium activated p-doped gallium-nitride photocathode. Other photocathode materials like cesium-telluride or potassium-bromide could be used as an alternative. For the sealed version, the photocathode is operated in semi-transparent mode on a MgF window with a cut-off wavelength of about 118 nm. For missions requiring sensitivity below this cut-off, we are planning an open version. We employ a coplanar cross-strip anode and advanced low-power readout electronics with a 128-channel charge-amplifier chip. This publication focuses on the progress concerning the main development challenges: the optimization of the photocathode parameters and the sophisticated detector electronics.
An Infinite-Feature Extension for Bayesian ReLU Nets That Fixes Their Asymptotic Overconfidence
2022-02-24

A Bayesian treatment can mitigate overconfidence in ReLU nets around the training data. But far away from them, ReLU Bayesian neural networks (BNNs) can still underestimate uncertainty and thus be asymptotically overconfident. This issue arises since the output variance of a BNN with finitely many features is quadratic in the distance from the data region. Meanwhile, Bayesian linear models with ReLU features converge, in the infinite-width limit, to a particular Gaussian process (GP) with a variance that grows cubically so that no asymptotic overconfidence can occur. While this may seem of mostly theoretical interest, in this work, we show that it can be used in practice to the benefit of BNNs. We extend finite ReLU BNNs with infinite ReLU features via the GP and show that the resulting model is asymptotically maximally uncertain far away from the data while the BNNs' predictive power is unaffected near the data. Although the resulting model approximates a full GP posterior, thanks to its structure, it can be applied \\emph{post-hoc} to any pre-trained ReLU BNN at a low cost.
A Fourier State Space Model for Bayesian ODE Filters
2022-02-24

Gaussian ODE filtering is a probabilistic numerical method to solve ordinary differential equations (ODEs). It computes a Bayesian posterior over the solution from evaluations of the vector field defining the ODE. Its most popular version, which employs an integrated Brownian motion prior, uses Taylor expansions of the mean to extrapolate forward and has the same convergence rates as classical numerical methods. As the solution of many important ODEs are periodic functions (oscillators), we raise the question whether Fourier expansions can also be brought to bear within the framework of Gaussian ODE filtering. To this end, we construct a Fourier state space model for ODEs and a `hybrid' model that combines a Taylor (Brownian motion) and Fourier state space model. We show by experiments how the hybrid model might become useful in cheaply predicting until the end of the time domain.
Factorized Neural Processes for Neural Processes: $K$-Shot Prediction of Neural Responses
2022-02-24

In recent years, artificial neural networks have achieved state-of-the-art performance for predicting the responses of neurons in the visual cortex to natural stimuli. However, they require a time consuming parameter optimization process for accurately modeling the tuning function of newly observed neurons, which prohibits many applications including real-time, closed-loop experiments. We overcome this limitation by formulating the problem as -shot prediction to directly infer a neuron's tuning function from a small set of stimulus-response pairs using a Neural Process. This required us to developed a Factorized Neural Process, which embeds the observed set into a latent space partitioned into the receptive field location and the tuning function properties. We show on simulated responses that the predictions and reconstructed receptive fields from the Factorized Neural Process approach ground truth with increasing number of trials. Critically, the latent representation that summarizes the tuning function of a neuron is inferred in a quick, single forward pass through the network. Finally, we validate this approach on real neural data from visual cortex and find that the predictive accuracy is comparable to -- and for small even greater than -- optimization based approaches, while being substantially faster. We believe this novel deep learning systems identification framework will facilitate better real-time integration of artificial neural network modeling into neuroscience experiments.
Mapping Diffuse Emission in Lyman UV band
2022-02-24

The CAFE (Census of warm-hot intergalactic medium, Accretion, and Feedback Explorer) and LyRIC (Lyman UV Radiation from Interstellar medium and Circum-galactic medium) have been proposed to the space agencies in China respectively. CAFE was first proposed in 2015 as a joint scientific CAS-ESA small space mission. LyRIC was proposed in 2019 as an independent external payload operating on the Chinese Space Station. Both missions are dedicated to mapping the Lyman UV emission (ionized oxygen (O VI) resonance lines at 103.2 and 103.8 nm, and Lyman series) for the diffuse sources in our Galaxy and the circum-galactic mediums of the nearby galaxies. We present the primary science objectives, mission concepts, the enabling technologies, as well as the current status.
Status of the STUDIO UV balloon mission and platform
2022-02-24

Stratospheric balloons offer accessible and affordable platforms for observations in atmosphere-constrained wavelength ranges. At the same time, they can serve as an effective step for technology demonstration towards future space applications of instruments and other hardware. The Stratospheric UV Demonstrator of an Imaging Observatory (STUDIO) is a balloon-borne platform and mission carrying an imaging micro-channel plate (MCP) detector on a 0.5 m aperture telescope. STUDIO is currently planned to fly during the summer turnaround conditions over Esrange, Sweden, in the 2022 season. For details on the ultraviolet (UV) detector, see the contribution of Conti et al. to this symposium. The scientific goal of the mission is to survey for variable hot compact stars and flaring M-dwarf stars within the galactic plane. At the same time, the mission acts as a demonstrator for a versatile and scalable astronomical balloon platform as well as for the aforementioned MCP instrument. The gondola is designed to allow the use of different instruments or telescopes. Furthermore, it is designed to serve for several, also longer, flights, which are envisioned under the European Stratospheric Balloon Observatory (ESBO) initiative. In this paper, we present the design and current status of manufacturing and testing of the STUDIO platform. We furthermore present the current plans for the flight and observations from Esrange.
Large-stepsize integrators for charged-particle dynamics over multiple time scales
2022-02-24

The Boris algorithm, a closely related variational integrator and a newly proposed filtered variational integrator are studied when they are used to numerically integrate the equations of motion of a charged particle in a non-uniform strong magnetic field, taking step sizes that are much larger than the period of the Larmor rotations. For the Boris algorithm and the standard (unfiltered) variational integrator, satisfactory behaviour is only obtained when the component of the initial velocity orthogonal to the magnetic field is filtered out. The particle motion shows varying behaviour over multiple time scales: fast Larmor rotation, guiding centre motion, slow perpendicular drift, near-conservation of the magnetic moment over very long times and conservation of energy for all times. Using modulated Fourier expansions of the exact and numerical solutions, it is analysed to which extent this behaviour is reproduced by the three numerical integrators used with large step sizes.
Self-organized Criticality in Multi-pulse Gamma-Ray Bursts
2022-02-24

The variability in multi-pulse gamma-ray bursts (GRBs) may help to reveal the mechanism of underlying processes from the central engine. To investigate whether the self-organized criticality (SOC) phenomena exist in the prompt phase of GRBs, we statistically study the properties of GRBs with more than 3 pulses in each burst by fitting the distributions of several observed physical variables with a Markov Chain Monte Carlo approach, including the isotropic energy , the duration time and the peak count rate of each pulse. Our sample consists of 454 pulses in 93 GRBs observed by the CGRO/BATSE satellite. The best-fitting values and uncertainties for these power-law indices of the differential frequency distributions are: , and , while the power-law indices in the cumulative frequency distributions are: , and . We find that these distributions are roughly consistent with the physical framework of a Fractal-Diffusive, Self-Organized Criticality (FD-SOC) system with the spatial dimension and the classical diffusion =1. Our results support that the jet responsible for the GRBs should be magnetically dominated and magnetic instabilities (e.g., kink model, or tearing-model instability) lead the GRB emission region into the SOC state.
Lower regularity assumption for an Euler-Lagrange equation on the contact line of the phase dependent Helfrich energy
2022-02-24

We examine the phase dependent Helfrich energy and show an Euler-Lagrange equation on the phase seperation line. This result has already been observed by e.g. Jülicher-Lipowski and later Elliot-Stinner. Here we are able to lower the regularity assumption for this result down to for the seperation line. In the proof we employ a carefully choosen test function utilising the signed distance function.
Employing distributional semantics to organize task-focused vocabulary learning
2022-02-24

How can a learner systematically prepare for reading a book they are interested in? In this paper,we explore how computational linguistic methods such as distributional semantics, morphological clustering, and exercise generation can be combined with graph-based learner models to answer this question both conceptually and in practice. Based on the highly structured learner model and concepts from network analysis, the learner is guided to efficiently explore the targeted lexical space. They practice using multi-gap learning activities generated from the book focused on words that are central to the targeted lexical space. As such the approach offers a unique combination of computational linguistic methods with concepts from network analysis and the tutoring system domain to support learners in achieving their individual, reading task-based learning goals.
A comparison between repeating bursts of FRB 121102 and giant pulses from Crab pulsar and its applications
2022-02-24

There are some similarities between bursts of repeating fast radio bursts (FRBs) and giant pulses (GPs) of pulsars. To explore possible relations between them, we study the cumulative energy distributions of these two phenomena using the observations of repeating FRB 121102 and the GPs of Crab pulsar. We find that the power-law slope of GPs (with fluence 130 Jy ms) is . The energy distribution of FRB 121102 can be well fitted by a smooth broken power-law function. For the bursts of FRB 121102 above the break energy (1.22 erg), the best-fitting slope is , similar to the index of GPs at the same observing frequency ( 1.4 GHz). We further discuss the physical origin of the repeating FRB 121102 in the framework of the super GPs model. And we find that the super GPs model involving a millisecond pulsar is workable and favored for explaining FRB 121102 despite that the magnetar burst model is more popular.
Calibrated Adaptive Probabilistic ODE Solvers
2022-02-24

Probabilistic solvers for ordinary differential equations assign a posterior measure to the solution of an initial value problem. The joint covariance of this distribution provides an estimate of the (global) approximation error. The contraction rate of this error estimate as a function of the solver's step size identifies it as a well-calibrated worst-case error, but its explicit numerical value for a certain step size is not automatically a good estimate of the explicit error. Addressing this issue, we introduce, discuss, and assess several probabilistically motivated ways to calibrate the uncertainty estimate. Numerical experiments demonstrate that these calibration methods interact efficiently with adaptive step-size selection, resulting in descriptive, and efficiently computable posteriors. We demonstrate the efficiency of the methodology by benchmarking against the classic, widely used Dormand-Prince 4/5 Runge-Kutta method.
Distinct glycerophospholipids potentiate Gsα-activated adenylyl cyclase activity
2022-02-24

Abstract Nine mammalian adenylyl cyclases (AC) are pseudoheterodimers with two hexahelical membrane domains which are isoform-specifically conserved. Previously we proposed that these membrane domains are orphan receptors (10.7554/eLife.13098; 10.1016/j.cellsig.2020.109538). Lipids extracted from fetal bovine serum at pH 1 inhibited several mAC activities. Guided by a lipidomic analysis we tested glycerophospholipids as potential ligands. Contrary to expectations we surprisingly discovered that 1-stearoyl-2-docosahexaenoyl-phosphatidic acid (SDPA) potentiated Gsα-activated activity of human AC isoform 3 seven-fold. The specificity of fatty acyl esters at glycerol positions 1 and 2 was rather stringent. 1-Stearoyl-2-docosahexaenoyl-phosphatidylserine and 1-stearoyl-2-docosahexaenoyl-phosphatidylethanolamine significantly potentiated several Gsα-activated mAC isoforms to different extents. SDPA appears not interact with forskolin activation of AC isoform 3. SDPA enhanced Gsα-activated AC activities in membranes from mouse brain cortex. The action of SDPA was reversible. Unexpectedly, SDPA did not affect cAMP generation in HEK293 cells stimulated by isoproterenol, PGE 2 and adenosine, virtually excluding a role as an extracellular ligand and, instead, suggesting an intracellular role. In summary, we discovered a new dimension of intracellular AC regulation by chemically defined glycerophospholipids.