Miracle-dz/UniFAR-FTUs huggingface dataset:
On the Information Bottleneck Problems: Models, Connections, Applications and Information Theoretic Views

This tutorial paper focuses on the variants of the bottleneck problem taking an information theoretic perspective and discusses practical methods to solve it, as well as its connection to coding and learning aspects. The intimate connections of this setting to remote source-coding under logarithmic loss distortion measure, information combining, common reconstruction, the Wyner–Ahlswede–Korner problem, the efficiency of investment information, as well as, generalization, variational inference, representation learning, autoencoders, and others are highlighted. We discuss its extension to the distributed information bottleneck problem with emphasis on the Gaussian model and highlight the basic connections to the uplink Cloud Radio Access Networks (CRAN) with oblivious processing. For this model, the optimal trade-offs between relevance (i.e., information) and complexity (i.e., rates) in the discrete and vector Gaussian frameworks is determined. In the concluding outlook, some interesting problems are mentioned such as the characterization of the optimal inputs (“features”) distributions under power limitations maximizing the “relevance” for the Gaussian information bottleneck, under “complexity” constraints.
Amphotericin B revisited: reassessment of toxicity.

PURPOSE Although amphotericin B remains the mainstay of therapy for serious fungal infections, numerous acute and chronic toxicities make clinical usage difficult. Nevertheless, utilization of amphotericin B has increased dramatically in recent years. Accordingly, a study to reassess toxicity associated with the use of amphotericin in the modern era and to determine the aggregate impact toxicities have on the overall therapeutic course of patients was undertaken. PATIENTS AND METHODS The charts of 50 patients at our institution who had received amphotericin B were retrospectively reviewed for the occurrence of drug-related acute and chronic toxicities and their impact on therapy. Patients evaluated were at least 21 years of age, had received at least 100 mg of amphotericin B, and were treated for at least 3 days. An analysis to investigate associations between drug administration variables and toxicities was also undertaken. RESULTS The mean age of study subjects were 54 years, the average duration of therapy was 19 days, and the mean cumulative dose of amphotericin B was 582 mg. Acute toxicities accompanying infusion occurred in 66% of all patients. Fever was experienced by 34% of study subjects and chills by 56%, with rates of 2.6 and 3.5 mean episodes per patient per treatment course, respectively. Other acute toxicities including hypotension were rare, and no patients exhibited bronchospasm or anaphylaxis. Nephrotoxicity occurred in 30 patients (60%). Baseline creatinine values in these patients rose by a mean of 1.55 mg/dL (137 mumol/L), with increases in creatinine ranging from 0.2 to 5.2 mg/dL (18 to 460 mumol/L). Analysis of six drug administration parameters by stepwise multiple linear regression failed to reveal any significant associations with nephrotoxicity. The mean nadir potassium value was 3.1 mEq/L (3.1 mmol/L) (range: 2.3 to 4.8 mEq/L [2.3 mmol/L]), with 45 of 50 patients (90%) receiving potassium supplementation (average daily supplement = 69 mEq). Only increasing amphotericin B concentration was correlated with an increased requirement for potassium supplementation (p less than 0.01, stepwise multiple linear regression). CONCLUSION Although clinically important toxicities continue to occur with usage of amphotericin B despite current methods of infusion and premedication, these are usually manageable and rarely limit the course of therapy.
Keeping out the Masses: Understanding the Popularity and Implications of Internet Paywalls

Funding the production of quality online content is a pressing problem for content producers. The most common funding method, online advertising, is rife with well-known performance and privacy harms, and an intractable subject-agent conflict: many users do not want to see advertisements, depriving the site of needed funding. Because of these negative aspects of advertisement-based funding, paywalls are an increasingly popular alternative for websites. This shift to a “pay-for-access” web is one that has potentially huge implications for the web and society. Instead of a system where information (nominally) flows freely, paywalls create a web where high quality information is available to fewer and fewer people, leaving the rest of the web users with less information, that might be also less accurate and of lower quality. Despite the potential significance of a move from an “advertising-but-open” web to a “paywalled” web, we find this issue understudied. This work addresses this gap in our understanding by measuring how widely paywalls have been adopted, what kinds of sites use paywalls, and the distribution of policies enforced by paywalls. A partial list of our findings include that (i) paywall use has increased, and at an increasing rate (2 × more paywalls every 6 months), (ii) paywall adoption differs by country (e.g., 18.75% in US, 12.69% in Australia), (iii) paywall deployment significantly changes how users interact with the site (e.g., higher bounce rates, less incoming links), (iv) the median cost of an annual paywall access is 108 USD per site, and (v) paywalls are in general trivial to circumvent. Finally, we present the design of a novel, automated system for detecting whether a site uses a paywall, through the combination of runtime browser instrumentation and repeated programmatic interactions with the site. We intend this classifier to augment future, longitudinal measurements of paywall use and behavior.
Electrocardiogram Generation and Feature Extraction Using a Variational Autoencoder

We propose a method for generating an electrocardiogram (ECG) signal for one cardiac cycle using a variational autoencoder. Using this method we extracted a vector of new 25 features, which in many cases can be interpreted. The generated ECG has quite natural appearance. The low value of the Maximum Mean Discrepancy metric, 0.00383, indicates good quality of ECG generation too. The extracted new features will help to improve the quality of automatic diagnostics of cardiovascular diseases. Also, generating new synthetic ECGs will allow us to solve the issue of the lack of labeled ECG for use them in supervised learning.
Classification of Computer Models with Labelled Outputs

Classification is a vital tool that is important for modelling many complex numerical models. A model or system may be such that, for certain areas of input space, the output either does not exist, or is not in a quantifiable form. Here, we present a new method for classification where the model outputs are given distinct classifying labels, which we model using a latent Gaussian process (GP). The latent variable is estimated using MCMC sampling, a unique likelihood and distinct prior specifications. Our classifier is then verified by calculating a misclassification rate across the input space. Comparisons are made with other existing classification methods including logistic regression, which models the probability of being classified into one of two regions. To make classification predictions we draw from an independent Bernoulli distribution, meaning that distance correlation is lost from the independent draws and so can result in many misclassifications. By modelling the labels using a latent GP, this problem does not occur in our method. We apply our novel method to a range of examples including a motivating example which models the hormones associated with the reproductive system in mammals, where the two labelled outputs are high and low rates of reproduction.
Sample Complexity of Incentivized Exploration

We consider incentivized exploration: a version of multi-armed bandits where the choice of actions is controlled by self-interested agents, and the algorithm can only issue recommendations. The algorithm controls the flow of information, and the information asymmetry can incentivize the agents to explore. Prior work matches the optimal regret rates for bandits up to "constant" multiplicative factors determined by the Bayesian prior. However, the dependence on the prior in prior work could be arbitrarily large, and the dependence on the number of arms K could be exponential. The optimal dependence on the prior and K is very unclear. We make progress on these issues. Our first result is that Thompson sampling is incentive-compatible if initialized with enough data points. Thus, we reduce the problem of designing incentive-compatible algorithms to that of sample complexity: (i) How many data points are needed to incentivize Thompson sampling? (ii) How many rounds does it take to collect these samples? We address both questions, providing upper bounds on sample complexity that are typically polynomial in K and lower bounds that are polynomially matching.
Sharp Rate of Convergence for Deep Neural Network Classifiers under the Teacher-Student Setting

Classifiers built with neural networks handle large-scale high dimensional data, such as facial images from computer vision, extremely well while traditional statistical methods often fail miserably. In this paper, we attempt to understand this empirical success in high dimensional classification by deriving the convergence rates of excess risk. In particular, a teacher-student framework is proposed that assumes the Bayes classifier to be expressed as ReLU neural networks. In this setup, we obtain a sharp rate of convergence, i.e., $\tilde{O}_d(n^{-2/3})$, for classifiers trained using either 0-1 loss or hinge loss. This rate can be further improved to $\tilde{O}_d(n^{-1})$ when the data distribution is separable. Here, $n$ denotes the sample size. An interesting observation is that the data dimension only contributes to the $\log(n)$ term in the above rates. This may provide one theoretical explanation for the empirical successes of deep neural networks in high dimensional classification, particularly for structured data.
Molecule Property Prediction and Classification with Graph Hypernetworks

Graph neural networks are currently leading the performance charts in learning-based molecule property prediction and classification. Computational chemistry has, therefore, become the a prominent testbed for generic graph neural networks, as well as for specialized message passing methods. In this work, we demonstrate that the replacement of the underlying networks with hypernetworks leads to a boost in performance, obtaining state of the art results in various benchmarks. A major difficulty in the application of hypernetworks is their lack of stability. We tackle this by combining the current message and the first message. A recent work has tackled the training instability of hypernetworks in the context of error correcting codes, by replacing the activation function of the message passing network with a low-order Taylor approximation of it. We demonstrate that our generic solution can replace this domain-specific solution.
Experimental observation of the marginal glass phase in a colloidal glass

Significance Glass is one of the easiest materials to work with, capable of forming complex and sophisticated structures with relative ease in the hands of a glass blower. However, the underlying physical principles that allow glass to slowly melt and flow are among the least understood. Recent theoretical breakthroughs hope to reveal a new kind of underlying simplicity within glasses, one that might extend to all manner of disordered and complex materials. By studying the formation of a model glass system at extremely short time scales and small length scales, we have proven the central prediction of these theories, that glasses undergo an extremely subtle transition while forming, which controls their physical properties. The replica theory of glasses predicts that in the infinite dimensional mean field limit, there exist two distinct glassy phases of matter: stable glass and marginal glass. We have developed a technique to experimentally probe these phases of matter using a colloidal glass. We avoid the difficulties inherent in measuring the long time behavior of glasses by instead focusing on the very short time dynamics of the ballistic to caged transition. We track a single tracer particle within a slowly densifying glass and measure the resulting mean squared displacement (MSD). By analyzing the MSD, we find that upon densification, our colloidal system moves through several states of matter. At lowest densities, it is a subdiffusive liquid. Next, it behaves as a stable glass, marked by the appearance of a plateau in the MSD whose magnitude shrinks with increasing density. However, this shrinking plateau does not shrink to zero; instead, at higher densities, the system behaves as a marginal glass, marked by logarithmic growth in the MSD toward that previous plateau value. Finally, at the highest experimental densities, the system returns to the stable glass phase. This provides direct experimental evidence for the existence of a marginal glass in three dimensions.
Sharp inequalities for the numerical radius of Hilbert space operators and operator matrices

We present new upper and lower bounds for the numerical radius of a bounded linear operator defined on a complex Hilbert space, which improve on the existing bounds. Among many other inequalities proved in this article, we show that for a non-zero bounded linear operator $T$ on a Hilbert space $H,$ $w(T)\geq \frac{\|T\|}{2}+\frac{m(T^2)}{2\|T\|}, $ where $w(T)$ is the numerical radius of $T$ and $m(T^2)$ is the Crawford number of $T^2$. This substantially improves on the existing inequality $w(T)\geq \frac{\|T\|}{2} . $ We also obtain some upper and lower bounds for the numerical radius of operator matrices and illustrate with numerical examples that these bounds are better than the existing bounds.
DiffNet++: A Neural Influence and Interest Diffusion Network for Social Recommendation

Social recommendation has emerged to leverage social connections among users for predicting users’ unknown preferences, which could alleviate the data sparsity issue in collaborative filtering based recommendation. Early approaches relied on utilizing each user’s first-order social neighbors’ interests for better user modeling, and failed to model the social influence diffusion process from the global social network structure. Recently, we propose a preliminary work of a neural influence Diffusion Network (i.e., DiffNet) for social recommendation L. Wu, P. Sun, Y. Fu, R. Hong, X. Wang, and M. Wang, “A neural influence diffusion model for social recommendation,” in Proc. Int. ACM SIGIR Conf. Res. Develop. Inf. Retrieval, 2019, pp. 235–244. DiffNet models the recursive social diffusion process for each user, such that the influence diffusion hidden in the higher-order social network is captured in the user embedding process. Despite the superior performance of DiffNet, we argue that, as users play a central role in both user-user social network and user-item interest network, only modeling the influence diffusion process in the social network would neglect the latent collaborative interests of users hidden in the user-item interest network. To this end, in this paper, we propose DiffNet++, an improved algorithm of DiffNet that models the neural influence diffusion and interest diffusion in a unified framework. By reformulating the social recommendation as a heterogeneous graph with social network and interest network as input, DiffNet++ advances DiffNet by injecting both the higher-order user latent interest reflected in the user-item graph and higher-order user influence reflected in the user-user graph for user embedding learning. This is achieved by iteratively aggregating each user’s embedding from three aspects: the user’s previous embedding, the influence aggregation of social neighbors from the social network, and the interest aggregation of item neighbors from the user-item interest network. Furthermore, we design a multi-level attention network that learns how to attentively aggregate user embeddings from these three aspects. Finally, extensive experimental results on four real-world datasets clearly show the effectiveness of our proposed model. We release the source code at https://github.com/PeiJieSun/diffnet.
Effects of the transverse coherence length in relativistic collisions

Effects of the quantum interference in collisions of particles have a twofold nature: they arise because of the autocorrelation of a complex scattering amplitude and due to spatial coherence of the incoming wave packets. Both these effects are neglected in a conventional scattering theory dealing with the delocalized plane waves, although they sometimes must be taken into account in particle and atomic physics. Here, we study the role of a transverse coherence length of the packets, putting special emphasis on the case in which one of the particles is twisted, that is, it carries an orbital angular momentum lℏ. In ee,ep, and pp collisions the interference results in corrections to the plane-wave cross sections, usually negligible at the energies s≫1 GeV but noticeable for smaller ones, especially if there is a twisted hadron with |l|>103 in initial state. Beyond the perturbative QCD, these corrections become only moderately attenuated allowing one to probe a phase of the hadronic amplitude as a function of s and t. In this regime, the coherence effects can compete with the loop corrections in QED and facilitate testing the phenomenological models of the strong interaction at intermediate and low energies.
The ELM Survey. VIII. Ninety-eight Double White Dwarf Binaries

We present the final sample of 98 detached double white dwarf (WD) binaries found in the Extremely Low Mass (ELM) Survey, a spectroscopic survey targeting <0.3 M⊙ He-core WDs completed in the Sloan Digital Sky Survey footprint. Over the course of the survey we observed ancillary low-mass WD candidates like GD 278, which we show is a P = 0.19 day double WD binary, as well as candidates that turn out to be field blue straggler/subdwarf A-type stars with luminosities too high to be WDs given their Gaia parallaxes. Here, we define a clean sample of ELM WDs that is complete within our target selection and magnitude range 15 < g0 < 20 mag. The measurements are consistent with 100% of ELM WDs being 0.0089 < P < 1.5 day double WD binaries, 35% of which belong to the Galactic halo. We infer that these are mostly He+CO WD binaries given the measurement constraints. The merger rate of the observed He+CO WD binaries exceeds the formation rate of stable mass-transfer AM CVn binaries by a factor of 25, and so the majority of He+CO WD binaries must experience unstable mass transfer and merge. The systems with the shortest periods, such as J0651+2844, are signature LISA verification binaries that can be studied with gravitational waves and light.
Lee-Wald charge and asymptotic behaviors of the Weyl-invariant topologically massive gravity

We apply the Lee–Wald covariant phase space method to the Weyl-invariant topologically massive gravity and compute the corresponding on-shell conserved charges. By using appropriate decay conditions for the existing propagating modes in the near-horizon of a stationary black hole, we obtain the charges generating the asymptotic symmetries. We show that the charges are integrable and the (modified) algebras among the asymptotic generators are closed for the certain choice of central extensions.
Global Texture Enhancement for Fake Face Detection in the Wild

Generative Adversarial Networks (GANs) can generate realistic fake face images that can easily fool human beings. On the contrary, a common Convolutional Neural Network(CNN) discriminator can achieve more than99.9%accuracyin discerning fake/real images. In this paper, we conduct an empirical study on fake/real faces, and have two important observations: firstly, the texture of fake faces is substantially different from real ones; secondly, global texture statistics are more robust to image editing and transferable to fake faces from different GANs and datasets. Motivated by the above observations, we propose a new architecture coined as Gram-Net, which leverages global image texture representations for robust fake image detection. Experimental results on several datasets demonstrate that our Gram-Netoutperforms existing approaches. Especially, our Gram-Netis more robust to image editings, e.g. down-sampling, JPEGcompression, blur, and noise. More importantly, our Gram-Net generalizes significantly better in detecting fake faces from GAN models not seen in the training phase and can perform decently in detecting fake natural images
Radioactive data: tracing through training

We want to detect whether a particular image dataset has been used to train a model. We propose a new technique, \emph{radioactive data}, that makes imperceptible changes to this dataset such that any model trained on it will bear an identifiable mark. The mark is robust to strong variations such as different architectures or optimization methods. Given a trained model, our technique detects the use of radioactive data and provides a level of confidence (p-value). Our experiments on large-scale benchmarks (Imagenet), using standard architectures (Resnet-18, VGG-16, Densenet-121) and training procedures, show that we can detect usage of radioactive data with high confidence (p<10^-4) even when only 1% of the data used to trained our model is radioactive. Our method is robust to data augmentation and the stochasticity of deep network optimization. As a result, it offers a much higher signal-to-noise ratio than data poisoning and backdoor methods.
Decoding Movement Imagination and Execution From Eeg Signals Using Bci-Transfer Learning Method Based on Relation Network

A brain-computer interface (BCI) is used to control external devices for healthy people as well as to rehabilitate motor functions for motor-disabled patients. Decoding movement intention is one of the most significant aspects for performing arm movement tasks using brain signals. Decoding movement execution (ME) from electroencephalogram (EEG) signals have shown high performance in previous works, however movement imagination (MI) paradigm-based intention decoding has so far failed to achieve sufficient accuracy. In this study, we focused on a robust MI decoding method with transfer learning for the ME and MI paradigm. We acquired EEG data related to arm reaching for 3D directions. We proposed a BCI-transfer learning method based on a Relation network (BTRN) architecture. Decoding performances showed the highest performance compared to conventional works. We confirmed the possibility of the BTRN architecture to contribute to continuous decoding of MI using ME datasets.
A frame decomposition of the atmospheric tomography operator

We consider the problem of atmospheric tomography, as it appears for example in adaptive optics systems for extremely large telescopes. We derive a frame decomposition, i.e., a decomposition in terms of a frame, of the underlying atmospheric tomography operator, extending the singular-value-type decomposition results of Neubauer and Ramlau (2017 SIAM J. Appl. Math. 77 838–853) by allowing a mixture of both natural and laser guide stars, as well as arbitrary aperture shapes. Based on both analytical considerations as well as numerical illustrations, we provide insight into the properties of the derived frame decomposition and its building blocks.