Miracle-dz/UniFAR-FTUs huggingface dataset:
Recursive Network Estimation From Binary-Valued Observation Data

This paper studies the problem of recursively estimating the weighted adjacency matrix of a network out of a temporal sequence of binary-valued observations. The observation sequence is generated from nonlinear networked dynamics in which agents exchange and display binary outputs. Sufficient conditions are given to ensure stability of the observation sequence and identifiability of the system parameters. It is shown that stability and identifiability can be guaranteed under the assumption of independent standard Gaussian disturbances. Via a maximum likelihood approach, the estimation problem is transformed into an optimization problem, and it is verified that its solution is the true parameter vector under the independent standard Gaussian assumption. A recursive algorithm for the estimation problem is then proposed based on stochastic approximation techniques. Its strong consistency is established and convergence rate analyzed. Finally, numerical simulations are conducted to illustrate the results and to show that the proposed algorithm is insensitive to small unmodeled factors.
On the Sample Complexity of Learning Sum-Product Networks

Sum-Product Networks (SPNs) can be regarded as a form of deep graphical models that compactly represent deeply factored and mixed distributions. An SPN is a rooted directed acyclic graph (DAG) consisting of a set of leaves (corresponding to base distributions), a set of sum nodes (which represent mixtures of their children distributions) and a set of product nodes (representing the products of its children distributions). In this work, we initiate the study of the sample complexity of PAC-learning the set of distributions that correspond to SPNs. We show that the sample complexity of learning tree structured SPNs with the usual type of leaves (i.e., Gaussian or discrete) grows at most linearly (up to logarithmic factors) with the number of parameters of the SPN. More specifically, we show that the class of distributions that corresponds to tree structured Gaussian SPNs with $k$ mixing weights and $e$ ($d$-dimensional Gaussian) leaves can be learned within Total Variation error $\epsilon$ using at most $\widetilde{O}(\frac{ed^2+k}{\epsilon^2})$ samples. A similar result holds for tree structured SPNs with discrete leaves. We obtain the upper bounds based on the recently proposed notion of distribution compression schemes. More specifically, we show that if a (base) class of distributions $\mathcal{F}$ admits an "efficient" compression, then the class of tree structured SPNs with leaves from $\mathcal{F}$ also admits an efficient compression.
Rademacher complexity and spin glasses: A link between the replica and statistical theories of learning

Statistical learning theory provides bounds of the generalization gap, using in particular the Vapnik-Chervonenkis dimension and the Rademacher complexity. An alternative approach, mainly studied in the statistical physics literature, is the study of generalization in simple synthetic-data models. Here we discuss the connections between these approaches and focus on the link between the Rademacher complexity in statistical learning and the theories of generalization for typical-case synthetic models from statistical physics, involving quantities known as Gardner capacity and ground state energy. We show that in these models the Rademacher complexity is closely related to the ground state energy computed by replica theories. Using this connection, one may reinterpret many results of the literature as rigorous Rademacher bounds in a variety of models in the high-dimensional statistics limit. Somewhat surprisingly, we also show that statistical learning theory provides predictions for the behavior of the ground-state energies in some full replica symmetry breaking models.
Survey of prognostics methods for condition-based maintenance in engineering systems

It is not surprising that the idea of efficient maintenance algorithms (originally motivated by strict emission regulations, and now driven by safety issues, logistics and customer satisfaction) has culminated in the so-called condition-based maintenance program. Condition-based program/monitoring consists of two major tasks, i.e., \textit{diagnostics} and \textit{prognostics} each of which has provided the impetus and technical challenges to the scientists and engineers in various fields of engineering. Prognostics deals with the prediction of the remaining useful life, future condition, or probability of reliable operation of an equipment based on the acquired condition monitoring data. This approach to modern maintenance practice promises to reduce the downtime, spares inventory, maintenance costs, and safety hazards. Given the significance of prognostics capabilities and the maturity of condition monitoring technology, there have been an increasing number of publications on machinery prognostics in the past few years. These publications cover a wide range of issues important to prognostics. Fortunately, improvement in computational resources technology has come to the aid of engineers by presenting more powerful onboard computational resources to make some aspects of these new problems tractable. In addition, it is possible to even leverage connected vehicle information through cloud-computing. Our goal is to review the state of the art and to summarize some of the recent advances in prognostics with the emphasis on models, algorithms and technologies used for data processing and decision making.
Towards understanding residual and dilated dense neural networks via convolutional sparse coding

Abstract Convolutional neural network (CNN) and its variants have led to many state-of-the-art results in various fields. However, a clear theoretical understanding of such networks is still lacking. Recently, a multilayer convolutional sparse coding (ML-CSC) model has been proposed and proved to equal such simply stacked networks (plain networks). Here, we consider the initialization, the dictionary design and the number of iterations to be factors in each layer that greatly affect the performance of the ML-CSC model. Inspired by these considerations, we propose two novel multilayer models: the residual convolutional sparse coding (Res-CSC) model and the mixed-scale dense convolutional sparse coding (MSD-CSC) model. They are closely related to the residual neural network (ResNet) and the mixed-scale (dilated) dense neural network (MSDNet), respectively. Mathematically, we derive the skip connection in the ResNet as a special case of a new forward propagation rule for the ML-CSC model. We also find a theoretical interpretation of dilated convolution and dense connection in the MSDNet by analyzing the MSD-CSC model, which gives a clear mathematical understanding of each. We implement the iterative soft thresholding algorithm and its fast version to solve the Res-CSC and MSD-CSC models. The unfolding operation can be employed for further improvement. Finally, extensive numerical experiments and comparison with competing methods demonstrate their effectiveness.
Spectrally uncorrelated biphotons generated from "the family of BBO crystal"

Spectrally intrinsically uncorrelated biphoton states generated from nonlinear crystals are very important but rare resources for quantum photonics and quantum information applications. Previously, such biphoton states were generated from several kinds of crystals, however, their wavelength ranges and nonlinear efficiencies were still limited for various applications. In order to explore new crystal for wider wavelength range and higher nonlinear efficiency, here we theoretically study the generation of spectrally uncorrelated biphoton states from 14 crystals in the `BBO family', including BBO, CLBO, KABO, KBBF, RBBF, CBBF, BABF, BiBO, LBO, CBO, LRB4, LCB, YCOB, and GdCOB. They satisfy three kinds of group-velocity matching condition from near-infrared to telecom wavelengths. Furthermore, heralded single photons can be generated with a purity as high as 0.98, which is achieved without any narrow filtering. The indistinguishability of photons from independent sources is examined by the Hong-Ou-Mandel interference, which results in a visibility of 98% also without any further filtering, i.e., photons from different heralded single-photon sources are highly indistinguishable. Our study may provide single-photon sources with good performance for quantum information processing at near-infrared and telecom wavelengths.
Gaussian Data-Aided Sensing With Multichannel Random Access and Model Selection

In this article, we study data-aided sensing (DAS) for a system consisting of a base station (BS) and a number of nodes, where the BS becomes a receiver that collects measurements or data sets from the nodes that are distributed over a cell. DAS is an iterative data collection scheme that allows the BS to efficiently estimate a target signal (i.e., all nodes’ measurements) with a small number of measurements (compared to random polling). In DAS, a set of nodes are selected in each round based on the data sets that are already available at the BS from previous rounds for efficient data collection. We consider DAS for measurements that are correlated with Gaussian in this article. The resulting DAS is referred to as Gaussian DAS. Using the mean-squared error (MSE) criterion, in each round, the BS is able to choose a node that has a data set to minimize the MSE of the next round. Furthermore, we generalize Gaussian DAS in two different ways: 1) with multiple parallel channels to upload measurements from nodes using random access and 2) with a model selection, where a multiarmed bandit problem formulation is used to combine the model selection with DAS.
Context Aware Password Guessability via Multi-Dimensional Rank Estimation

Password strength estimators are used to help users avoid picking weak passwords. Existing probabilistic estimators use various approaches to predict how many attempts a password cracker would need until it finds a given password. In this paper we present the first method for estimating the strength of human-chosen text passwords that is able to tweak the estimation according to each user's personal context, without retraining its model. Our method is able to incorporate context such as the user's name, preferred suffix, or previously cracked passwords (if available) when estimating the current password's strength. The tweaking takes only a few seconds per password. Our idea is to cast the question in a probabilistic framework used in side-channel cryptography. We view each password as a point in a d-dimensional search space, and learn the probability distribution of each dimension separately. The a-priori probability of a given password is the product of the d probabilities of its sub-passwords. After a detailed evaluation of leaked password corpora we found that an effective choice is to use d=5 dimensions: base word, prefix, suffix, shift-pattern, and l33t transformation. We coupled this decomposition with a state-of-the-art rank estimation algorithm to create our new PESrank estimator. We show that PESrank is more powerful than previous methods: it can crack more passwords, with fewer attempts, than the password crackers we compared it to. Even without using per-user context, PESrank is more accurate than previous methods for crackable passwords whose rank is smaller than 10^{12}. Furthermore, its training time is drastically shorter than previous methods, taking minutes, rather than days, to train on comparably-sized training sets, and taking a few hours to train on 905 million passwords, which is 8 times more passwords than previously used.
Compact packings of space with three sizes of spheres

A sphere packing of the three-dimensional Euclidean space is compact if it has only tetrahedral holes, that is, any local maximum of the distance to the spheres is at equal distance to exactly four spheres. This papers describes all the compact packings with spheres of three different sizes. They are close-compact packings of unit spheres with holes filled in four different ways by smaller spheres.
Mathematical Programming Models for Mean Computation in Dynamic Time Warping Spaces

The dynamic time warping (dtw) distance is an established tool for mining time series data. The DTW-Mean problem consists of computing a series which minimizes the so-called Frechet function, that is, the sum of squared dtw-distances to a given sample of time series. DTW-Mean is NP-hard and intractable in practice. So far, this challenging problem has been solved by various heuristic approaches without any performance guarantees. We give a polynomial-time algorithm yielding lower bounds on the domain of a mean time series which translate into lower bounds on the Frechet function. We then formulate the problem as a discrete nonlinear optimization problem based on network flows. We introduce several mixed-integer nonlinear programming (MINLP) formulations in order to solve DTW-Mean optimally. Our formulations are based on techniques such as outer approximations and nonlinear reformulations of the well-known big M indicator constraints. Finally, we conduct several computational experiments to compare the different formulations on several instances derived from the UCR Time Series Classification Archive. While in general DTW-Mean remains quite challenging, our fomrulations yield good results in several important specialized probem settings.
On the origin of the asymmetric dwarf galaxy distribution around andromeda

The dwarf galaxy distribution surrounding M31 is significantly anisotropic in nature. Of the 30 dwarf galaxies in this distribution, 15 form a disc-like structure and 23 are contained within the hemisphere facing the Milky Way. Using a realistic local potential, we analyse the conditions required to produce and maintain these asymmetries. We find that some dwarf galaxies are required to have highly eccentric orbits in order to preserve the presence of the hemispherical asymmetry with an appropriately large radial dispersion. Under the assumption that the dwarf galaxies originate from a single association or accretion event, we find that the initial size and specific energy of that association must both be relatively large in order to produce the observed hemispherical asymmetry. However if the association was large in physical size, the very high-energy required would enable several dwarf galaxies to escape from the M31 and be captured by the Milky Way. Furthermore, we find that associations that result in this structure have total specific energies concentrated around $E = V_{\rm esc}^{2} - V_{\rm init}^{2} \sim 200^2$ – $300^2\ \rm {km^2\ s^{-2}}$, implying that the initial velocity and initial position needed to produce the structure are strongly correlated. The overlap of initial conditions required to produce the radial dispersion, angular dispersion, and the planar structure is small and suggests that either they did not originate from a single accretion event, or that these asymmetric structures are short-lived.
3DMAP-VR, A Project to Visualize Three-dimensional Models of Astrophysical Phenomena in Virtual Reality

In this research note, we present 3DMAP-VR,(3-Dimensional Modeling of Astrophysical Phenomena in Virtual Reality), a project aimed at visualizing 3D MHD models of astrophysical simulations, using virtual reality sets of equipment. The models account for all the relevant physical processes in astrophysical phenomena: gravity, magnetic-field-oriented thermal conduction, energy losses due to radiation, gas viscosity, deviations from proton-electron temperature equilibration, deviations from the ionization equilibrium, cosmic rays acceleration, etc. We realized an excellent synergy between our 3DMAP-VR project and Sketchfab (one of the largest open access platforms to publish and share 3D virtual reality and augmented reality content) to promote a wide dissemination of results for both scientific and public outreach purposes.
E2-Capsule Neural Networks for Facial Expression Recognition Using AU-Aware Attention

Capsule neural network is a new and popular technique in deep learning. However, the traditional capsule neural network does not extract features sufficiently before the dynamic routing between the capsules. In this paper, the one Double Enhanced Capsule Neural Network (E2-Capsnet) that uses AU-aware attention for facial expression recognition (FER) is proposed. The E2-Capsnet takes advantage of dynamic routing between the capsules, and has two enhancement modules which are beneficial for FER. The first enhancement module is the convolutional neural network with AU-aware attention, which can help focus on the active areas of the expression. The second enhancement module is the capsule neural network with multiple convolutional layers, which enhances the ability of the feature representation. Finally, squashing function is used to classify the facial expression. We demonstrate the effectiveness of E2-Capsnet on the two public benchmark datasets, RAF-DB and EmotioNet. The experimental results show that our E2-Capsnet is superior to the state-of-the-art methods. Our implementation will be publicly available online.
Indian Buffet Neural Networks for Continual Learning

We place an Indian Buffet Process (IBP) prior over the neural structure of a Bayesian Neural Network (BNN), thus allowing the complexity of the BNN to increase and decrease automatically. We apply this methodology to the problem of resource allocation in continual learning, where new tasks occur and the network requires extra resources. Our BNN exploits online variational inference with relaxations to the Bernoulli and Beta distributions (which constitute the IBP prior), so allowing the use of the reparameterisation trick to learn variational posteriors via gradient-based methods. As we automatically learn the number of weights in the BNN, overfitting and underfitting problems are largely overcome. We show empirically that the method offers competitive results compared to Variational Continual Learning (VCL) in some settings.
Relativistic effective charge model of a multi-electron atom

A relativistic version of the effective charge model (ECM) for computation of observable characteristics of multi-electron atoms and ions is developed. A complete and orthogonal Dirac hydrogen basis set, depending on one parameter—effective nuclear charge Z*—identical for all single-electron wave functions of a given atom or ion, is employed for the construction of the secondary-quantized representation. The effective charge is uniquely determined by the charge of the nucleus and a set of occupied single-electron orbitals for a given state. We thoroughly study the accuracy of the leading-order approximation for the total binding energy and demonstrate that it is independent of the number of electrons of a multi-electron atom. In addition, it is shown that the fully analytical leading-order approximation is especially suited for the description of highly charged ions since our wave functions are almost coincident with the Dirac–Hartree–Fock ones for the complete spectrum. Finally, we evaluate various atomic characteristics, such as scattering factors and photoionization cross-sections, and thus envisage that the ECM can replace other models of comparable complexity, such as the Thomas–Fermi–Dirac model for all applications where it is still utilized.
Self-Supervised Contextual Language Representation of Radiology Reports to Improve the Identification of Communication Urgency

Machine learning methods have recently achieved high-performance in biomedical text analysis. However, a major bottleneck in the widespread application of these methods is obtaining the required large amounts of annotated training data, which is resource intensive and time consuming. Recent progress in self-supervised learning has shown promise in leveraging large text corpora without explicit annotations. In this work, we built a self-supervised contextual language representation model using BERT, a deep bidirectional transformer architecture, to identify radiology reports requiring prompt communication to the referring physicians. We pre-trained the BERT model on a large unlabeled corpus of radiology reports and used the resulting contextual representations in a final text classifier for communication urgency. Our model achieved a precision of 97.0%, recall of 93.3%, and F-measure of 95.1% on an independent test set in identifying radiology reports for prompt communication, and significantly outperformed the previous state-of-the-art model based on word2vec representations.
VersaVIS—An Open Versatile Multi-Camera Visual-Inertial Sensor Suite

Robust and accurate pose estimation is crucial for many applications in mobile robotics. Extending visual Simultaneous Localization and Mapping (SLAM) with other modalities such as an inertial measurement unit (IMU) can boost robustness and accuracy. However, for a tight sensor fusion, accurate time synchronization of the sensors is often crucial. Changing exposure times, internal sensor filtering, multiple clock sources and unpredictable delays from operation system scheduling and data transfer can make sensor synchronization challenging. In this paper, we present VersaVIS, an Open Versatile Multi-Camera Visual-Inertial Sensor Suite aimed to be an efficient research platform for easy deployment, integration and extension for many mobile robotic applications. VersaVIS provides a complete, open-source hardware, firmware and software bundle to perform time synchronization of multiple cameras with an IMU featuring exposure compensation, host clock translation and independent and stereo camera triggering. The sensor suite supports a wide range of cameras and IMUs to match the requirements of the application. The synchronization accuracy of the framework is evaluated on multiple experiments achieving timing accuracy of less than 1 ms. Furthermore, the applicability and versatility of the sensor suite is demonstrated in multiple applications including visual-inertial SLAM, multi-camera applications, multi-modal mapping, reconstruction and object based mapping.
RELICS: A Very Large (θE ∼ 40″) Cluster Lens—RXC J0032.1+1808

Extensive surveys with the Hubble Space Telescope over the past decade, targeting some of the most massive clusters in the sky, have uncovered dozens of galaxy cluster strong lenses. The massive cluster strong-lens scale is typically θE ∼ 10″ to ∼30″–35″, with only a handful of clusters known with Einstein radii θE ∼ 40″ or above (for zsource = 2, nominally). Here we report another very large cluster lens, RXC J0032.1+1808 (z = 0.3956), the second-richest cluster in the redMapper cluster catalog and the 85th most massive cluster in the Planck Sunyaev–Zel’dovich catalog. With our light-traces-mass and fully parametric approaches, we construct strong-lensing models based on 18 multiple images of five background galaxies newly identified in the Hubble data, mainly from the Reionization Lensing Cluster Survey (RELICS), in addition to a known sextuply imaged system in this cluster. Furthermore, we compare these models to Lenstool and GLAFIC models that were produced independently as part of the RELICS program. All models reveal a large effective Einstein radius of θE ≃ 40″ (zsource = 2), owing to the obvious concentration of substructures near the cluster center. Although RXC J0032.1+1808 has a very large critical area and high lensing strength, only three magnified high-redshift candidates are found within the field targeted by RELICS. Nevertheless, we expect many more high-redshift candidates will be seen in wider and deeper observations with Hubble or the James Webb Space Telescope. Finally, the comparison between several algorithms demonstrates that the total error budget is largely dominated by systematic uncertainties.