Miracle-dz/UniFAR-FTUs huggingface dataset:
Efficient Bayes Inference in Neural Networks through Adaptive Importance Sampling

Bayesian neural networks (BNNs) have received an increased interest in the last years. In BNNs, a complete posterior distribution of the unknown weight and bias parameters of the network is produced during the training stage. This probabilistic estimation offers several advantages with respect to point-wise estimates, in particular, the ability to provide uncertainty quantification when predicting new data. This feature inherent to the Bayesian paradigm, is useful in countless machine learning applications. It is particularly appealing in areas where decision-making has a crucial impact, such as medical healthcare or autonomous driving. The main challenge of BNNs is the computational cost of the training procedure since Bayesian techniques often face a severe curse of dimensionality. Adaptive importance sampling (AIS) is one of the most prominent Monte Carlo methodologies benefiting from sounded convergence guarantees and ease for adaptation. This work aims to show that AIS constitutes a successful approach for designing BNNs. More precisely, we propose a novel algorithm PMCnet that includes an efficient adaptation mechanism, exploiting geometric information on the complex (often multimodal) posterior distribution. Numerical results illustrate the excellent performance and the improved exploration capabilities of the proposed method for both shallow and deep neural networks.
Robust Set Stability of Logic Dynamical Systems with respect to Uncertain Switching

This paper proposes several definitions of robust stability for logic dynamical systems (LDSs) with uncertain switching, including robust/uniform robust set stability and asymptotical (or infinitely convergent)/finite-time set stability with ratio one. It is proved herein that an LDS is robustly set stable if and only if the destination set contains all loops (i.e., the paths from each state to itself); an LDS is uniformly robustly set stable, or finite-time set stable with ratio one, if and only if all states outside the destination set are unreachable from any self-reachable state; and an LDS is asymptotically set stable with ratio one if and only if the largest robustly invariant subset (LRIS) in the destination set is reachable from any state. In addition, it is proved that uniform robust set stability implies robust set stability, and robust set stability implies asymptotical set stability with ratio one. However, the inverse claims are not generally true. The relations between robust stability and stability under random switching are revealed, that is, the asymptotical/finite-time set stability with ratio one under uncertain switching is equivalent to asymptotical/finite-time set stability of the LDS under random switching. Furthermore, it is proved that, for uniform set stability and asymptotical/finite-time set stability with ratio one, the set stability is equivalent to the stability with respect to the LRIS in the destination set. However, robust set stability does not imply robust stability with respect to the LRIS in the destination set. This finding corrects a result in a previous study.
The Long Tail of Context: Does it Exist and Matter?

Context has been an important topic in recommender systems over the past two decades. A standard representational approach to context assumes that contextual variables and their structures are known in an application. Most of the prior CARS papers following representational approach manually selected and considered only a few crucial contextual variables in an application, such as time, location, and company of a person. This prior work demonstrated significant recommendation performance improvements when various CARS-based methods have been deployed in numerous applications. However, some recommender systems applications deal with a much bigger and broader types of contexts, and manually identifying and capturing a few contextual variables is not sufficient in such cases. In this paper, we study such ``context-rich'' applications dealing with a large variety of different types of contexts. We demonstrate that supporting only a few most important contextual variables, although useful, is not sufficient. In our study, we focus on the application that recommends various banking products to commercial customers within the context of dialogues initiated by customer service representatives. In this application, we managed to identify over two hundred types of contextual variables. Sorting those variables by their importance forms the Long Tail of Context (LTC). In this paper, we empirically demonstrate that LTC matters and using all these contextual variables from the Long Tail leads to significant improvements in recommendation performance.
Deep Invertible Approximation of Topologically Rich Maps between Manifolds

How can we design neural networks that allow for stable universal approximation of maps between topologically interesting manifolds? The answer is with a coordinate projection. Neural networks based on topological data analysis (TDA) use tools such as persistent homology to learn topological signatures of data and stabilize training but may not be universal approximators or have stable inverses. Other architectures universally approximate data distributions on submanifolds but only when the latter are given by a single chart, making them unable to learn maps that change topology. By exploiting the topological parallels between locally bilipschitz maps, covering spaces, and local homeomorphisms, and by using universal approximation arguments from machine learning, we find that a novel network of the form $\mathcal{T} \circ p \circ \mathcal{E}$, where $\mathcal{E}$ is an injective network, $p$ a fixed coordinate projection, and $\mathcal{T}$ a bijective network, is a universal approximator of local diffeomorphisms between compact smooth submanifolds embedded in $\mathbb{R}^n$. We emphasize the case when the target map changes topology. Further, we find that by constraining the projection $p$, multivalued inversions of our networks can be computed without sacrificing universality. As an application, we show that learning a group invariant function with unknown group action naturally reduces to the question of learning local diffeomorphisms for finite groups. Our theory permits us to recover orbits of the group action. We also outline possible extensions of our architecture to address molecular imaging of molecules with symmetries. Finally, our analysis informs the choice of topologically expressive starting spaces in generative problems.
Large-scale machine-learning-assisted exploration of the whole materials space

Crystal-graph attention networks have emerged recently as remarkable tools for the prediction of thermodynamic stability and materials properties from unrelaxed crystal structures. Previous networks trained on two million materials exhibited, however, strong biases originating from underrepresented chemical elements and structural prototypes in the available data. We tackled this issue computing additional data to provide better balance across both chemical and crystal-symmetry space. Crystal-graph networks trained with this new data show unprecedented generalization accuracy, and allow for reliable, accelerated exploration of the whole space of inorganic compounds. We applied this universal network to perform machine-learning assisted high-throughput materials searches including 2500 binary and ternary structure prototypes and spanning about 1 billion compounds. After validation using density-functional theory, we uncover in total 19512 additional materials on the convex hull of thermodynamic stability and ∼ 150 000 compounds with a distance of less than 50 meV/atom from the hull. Combining again machine learning and ab-initio methods, we ﬁnally evaluate the discovered materials for applications as superconductors, superhard materials, and we look for candidates with large gap deformation potentials, ﬁnding several compounds with extreme values of these properties.
Device-independent uncloneable encryption

Uncloneable encryption, first introduced by Broadbent and Lord (TQC 2020) is a quantum encryption scheme in which a quantum ciphertext cannot be distributed between two non-communicating parties such that, given access to the decryption key, both parties cannot learn the underlying plaintext. In this work, we introduce a variant of uncloneable encryption in which several possible decryption keys can decrypt a particular encryption, and the security requirement is that two parties who receive independently generated decryption keys cannot both learn the underlying ciphertext. We show that this variant of uncloneable encryption can be achieved device-independently, i.e., without trusting the quantum states and measurements used in the scheme, and that this variant works just as well as the original definition in constructing quantum money. Moreover, we show that a simple modification of our scheme yields a single-decryptor encryption scheme, which was a related notion introduced by Georgiou and Zhandry. In particular, the resulting single-decryptor encryption scheme achieves device-independent security with respect to a standard definition of security against random plaintexts. Finally, we derive an"extractor"result for a two-adversary scenario, which in particular yields a single-decryptor encryption scheme for single bit-messages that achieves perfect anti-piracy security without needing the quantum random oracle model.
Measurement of Trustworthiness of the Online Reviews

In electronic commerce (e-commerce)markets, a decision-maker faces a sequential choice problem. Third-party intervention plays an important role in making purchase decisions in this choice process. For instance, while purchasing products/services online, a buyer's choice or behavior is often affected by the overall reviewers' ratings, feedback, etc. Moreover, the reviewer is also a decision-maker. After purchase, the decision-maker would put forth their reviews for the product, online. Such reviews would affect the purchase decision of another potential buyer, who would read the reviews before conforming to his/her final purchase. The question that arises is \textit{how trustworthy are these review reports and ratings? } The trustworthiness of these review reports and ratings is based on whether the reviewer is a rational or an irrational person. Indexing the reviewer's rationality could be a way to quantify a reviewer's rationality but it does not communicate the history of his/her behavior. In this article, the researcher aims at formally deriving a rationality pattern function and thereby, the degree of rationality of the decision-maker or the reviewer in the sequential choice problem in the e-commerce markets. Applying such a rationality pattern function could make it easier to quantify the rational behavior of an agent who participates in the digital markets. This, in turn, is expected to minimize the information asymmetry within the decision-making process and identify the paid reviewers or manipulative reviews.
An experimental study of algorithms for obtaining a singly connected subgraph

A directed graph G = ( V, E ) is singly connected if for any two vertices v, u ∈ V , the directed graph G contains at most one simple path from v to u . In this paper, we study diﬀerent algorithms to ﬁnd a feasible but necessarily optimal solution to the following problem. Given a directed acyclic graph G = ( V, E ), ﬁnd a subset H ⊆ E of minimum size such that the subgraph ( V, E \ H ) is singly connected. Moreover, we prove that this problem can be solved in polynomial time for a special kind of directed graphs.
Sequential Brick Assembly with Efficient Constraint Satisfaction

We address the problem of generating a sequence of LEGO brick assembly with high-ﬁdelity structures, satisfying physical constraints between bricks. The assembly problem is challenging since the number of possible structures increases exponentially with the number of available bricks, complicating the physical constraints to satisfy across bricks. To tackle this problem, our method performs a brick structure assessment to predict the next brick position and its conﬁdence by employing a U-shaped sparse 3D convolutional network. The convolution ﬁlter efﬁciently validates physical constraints in a parallelizable and scalable manner, allowing to process of different brick types. To generate a novel structure, we devise a sampling strategy to determine the next brick position by considering attachable positions under physical constraints. Instead of using handcrafted brick assembly datasets, our model is trained with a large number of 3D objects that allow to create a new high-ﬁdelity structure. We demonstrate that our method successfully generates diverse brick structures while handling two different brick types and outperforms existing methods based on Bayesian optimization, graph generative model, and reinforcement learning, all of which are limited to a single brick type.
Near-Optimal Deployment Efficiency in Reward-Free Reinforcement Learning with Linear Function Approximation

We study the problem of deployment efficient reinforcement learning (RL) with linear function approximation under the \emph{reward-free} exploration setting. This is a well-motivated problem because deploying new policies is costly in real-life RL applications. Under the linear MDP setting with feature dimension $d$ and planning horizon $H$, we propose a new algorithm that collects at most $\widetilde{O}(\frac{d^2H^5}{\epsilon^2})$ trajectories within $H$ deployments to identify $\epsilon$-optimal policy for any (possibly data-dependent) choice of reward functions. To the best of our knowledge, our approach is the first to achieve optimal deployment complexity and optimal $d$ dependence in sample complexity at the same time, even if the reward is known ahead of time. Our novel techniques include an exploration-preserving policy discretization and a generalized G-optimal experiment design, which could be of independent interest. Lastly, we analyze the related problem of regret minimization in low-adaptive RL and provide information-theoretic lower bounds for switching cost and batch complexity.
Information Removal at the bottleneck in Deep Neural Networks

Deep learning models are nowadays broadly deployed to solve an incredibly large variety of tasks. Commonly, leveraging over the availability of"big data", deep neural networks are trained as black-boxes, minimizing an objective function at its output. This however does not allow control over the propagation of some specific features through the model, like gender or race, for solving some an uncorrelated task. This raises issues either in the privacy domain (considering the propagation of unwanted information) and of bias (considering that these features are potentially used to solve the given task). In this work we propose IRENE, a method to achieve information removal at the bottleneck of deep neural networks, which explicitly minimizes the estimated mutual information between the features to be kept ``private'' and the target. Experiments on a synthetic dataset and on CelebA validate the effectiveness of the proposed approach, and open the road towards the development of approaches guaranteeing information removal in deep neural networks.
Hip Fracture Prediction using the First Principal Component Derived from FEA-Computed Fracture Loads

Hip fracture risk assessment is an important but challenging task. Quantitative CT-based patient-specific finite element analysis (FEA) incorporates bone geometry and the three-dimensional distribution of bone density in the proximal femur to compute the force (fracture load) and energy necessary to break the proximal femur in a particular loading condition. Although the fracture loads and energies for different loading conditions are individually associated with incident hip fracture, and are mutually correlated, they each provide different structural information about the proximal femur that can influence a subject’s overall fracture risk. To obtain a more robust measure of fracture risk, we used principal component analysis (PCA) to develop a global FEA-computed fracture risk index that incorporates the FEA-computed yield and ultimate failure loads and energies to failure in four loading conditions (single-limb stance and impact from a fall onto the posterior, posterolateral, and lateral aspects of the greater trochanter) of 110 hip fracture subjects and 235 age- and sex-matched control subjects from the AGES-Reykjavik study. We found that the first PC (PC1) of the FE parameters was the only significant predictor of hip fracture ( p <0.001); we refer to PC1 as the global FEA-computed fracture risk index. To evaluate this fracture risk index, we considered seven covariates including demographic and clinical parameters. Using a logistic regression model, we determined if prediction performance for hip fracture using PC1 differed from that using FE parameters combined by stratified random resampling with respect to hip fracture status. We also compared the performance of predicting hip fracture using the logistic regression model with PC1 and the FRAX model. The results showed that the average of the area under the receive operating characteristic curve (AUC) using PC1 (0.776) was always higher than that using all FE parameters combined (0.737) in the male subjects ( p <0.001). The AUC of PC1 and AUC of the FE parameters combined were not significantly different than that in the female subjects ( p =0.211) or in all subjects ( p =0.159). The AUC values using PC1 (0.754 in all subjects, 0.825 in males and 0.71 in females) were greater than those using FRAX
Approximate Computing and the Efficient Machine Learning Expedition

Approximate computing (AxC) has been long accepted as a design alternative for efficient system implementation at the cost of relaxed accuracy requirements. Despite the AxC research activities in various application domains, AxC thrived the past decade when it was applied in Machine Learning (ML). The by definition approximate notion of ML models but also the increased computational overheads associated with ML applications–that were effectively mitigated by corresponding approximations–led to a perfect matching and a fruitful synergy. AxC for AI/ML has transcended beyond academic prototypes. In this work, we enlighten the synergistic nature of AxC and ML and elucidate the impact of AxC in designing efficient ML systems. To that end, we present an overview and taxonomy of AxC for ML and use two descriptive application scenarios to demonstrate how AxC boosts the efficiency of ML systems.
Cocoon breakout and escape from the ejecta of neutron star mergers

The cocoon is an inevitable product of a jet propagating through ambient matter, and takes a fair fraction of the jet energy. In short gamma-ray bursts (sGRBs), the ambient matter is the ejecta from the merger of neutron stars, expanding with a high velocity ∼0.2c, in contrast to the static stellar envelope in collapsars. Using 2D relativistic hydrodynamic simulations with the ejecta density profile as ρ∝r−2, we find that the expansion makes a big difference; only 0.5–5 per cent of the cocoon mass escapes from (faster than) the ejecta, with an opening angle 20○–30○, while it is $\sim 100{{\%}}$ and spherical in collapsars. We also analytically obtain the shares of mass and energies for the escaped and trapped cocoons. Because the mass of the escaped cocoon is small and the trapped cocoon is concealed by the ejecta and the escaped cocoon, we suggest that it is unlikely that cooling emission from the sGRB-jet heated cocoon was observed as a counterpart to the gravitational wave event GW170817.
Coreshine in L1506C – Evidence for a primitive big-grain component or indication for a turbulent core history?

Context. With the initial steps of the star formation process in the densest part of the interstellar medium (ISM) still under debate, much attention is paid to the formation and evolution of pre-stellar cores. The recently discovered coreshine effect can aid in exploring the core properties and in probing the large grain population of the ISM. Aims. We discuss the implications of the coreshine detected from the molecular cloud core L1506C in the Taurus filament for the history of the core and the existence of a primitive ISM component of large grains becoming visible in cores. Methods. The coreshine surface brightness of L1506C is determined from Spitzer IRAC images at 3.6 µm. We perform grain growth calculations to estimate the grain size distribution in model cores similar in gas density, radius, and turbulent velocity to L1506C. Scattered light intensities at 3.6 µm are calculated for a variety of MRN and grain growth distributions using the DIRBE 3.5 µm all-sky map as external interstellar radiation field, and are compared to the observed coreshine surface brightness. Results. For a core with the overall physical properties of L1506C, no detectable coreshine is predicted with a size distribution following the shape and size limits of an MRN distribution. Extending the distribution to grain radii of about 0.65 µm allows to reproduce the observed surface brightness level in scattered light. Assuming the properties of L1506C to be preserved, models for the growth of grains in cores do not yield sufficient scattered light to account for the coreshine within the lifetime of the Taurus complex. Only increasing the core density and the turbulence amplifies the scattered light intensity to a level consistent with the observed coreshine brightness. Conclusions. The coreshine observed from L1506C requires the presence of grains with sizes exceeding the common MRN distribution. The grains could be part of primitive omni-present large grain population becoming visible in the densest part of the ISM, could grow under the turbulent dense conditions of former cores, or in L1506C itself. In the later case, L1506C must have passed through a period of larger density and stronger turbulence. This would be consistent with the surprisingly strong depletion usually attributed to high column densities, and with the large-scale outward motion of the core envelope observed today.
Frequency Estimation of Evolving Data Under Local Differential Privacy

Collecting and analyzing evolving longitudinal data has become a common practice. One possible approach to protect the users' privacy in this context is to use local differential privacy (LDP) protocols, which ensure the privacy protection of all users even in the case of a breach or data misuse. Existing LDP data collection protocols such as Google's RAPPOR and Microsoft's dBitFlipPM can have longitudinal privacy linear to the domain size k, which is excessive for large domains, such as Internet domains. To solve this issue, in this paper we introduce a new LDP data collection protocol for longitudinal frequency monitoring named LOngitudinal LOcal HAshing (LOLOHA) with formal privacy guarantees. In addition, the privacy-utility trade-off of our protocol is only linear with respect to a reduced domain size $2\leq g \ll k$. LOLOHA combines a domain reduction approach via local hashing with double randomization to minimize the privacy leakage incurred by data updates. As demonstrated by our theoretical analysis as well as our experimental evaluation, LOLOHA achieves a utility competitive to current state-of-the-art protocols, while substantially minimizing the longitudinal privacy budget consumption by up to k/g orders of magnitude.
PromptKG: A Prompt Learning Framework for Knowledge Graph Representation Learning and Application

Knowledge Graphs (KGs) often have two characteristics: heterogeneous graph structure and text-rich entity/relation information. KG representation models should consider graph structures and text semantics, but no comprehensive open-sourced framework is mainly designed for KG regarding informative text description. In this paper, we present PromptKG , a prompt learning framework for KG representation learning and application that equips the cutting-edge text-based methods, integrates a new prompt learning model and supports various tasks (e