Miracle-dz/UniFAR-FTUs huggingface dataset:
Learning Intermediate Representations using Graph Neural Networks for NUMA and Prefetchers Optimization

There is a large space of NUMA and hardware prefetcher configurations that can significantly impact the performance of an application. Previous studies have demonstrated how a model can automatically select configurations based on the dynamic properties of the code to achieve speedups. This paper demonstrates how the static Intermediate Representation (IR) of the code can guide NUMA/prefetcher optimizations without the prohibitive cost of performance profiling. We propose a method to create a comprehensive dataset that includes a diverse set of intermediate representations along with optimum configurations. We then apply a graph neural network model in order to validate this dataset. We show that our static intermediate representation based model achieves 80 % of the performance gains provided by expensive dynamic performance profiling based strategies. We further develop a hybrid model that uses both static and dynamic information. Our hybrid model achieves the same gains as the dynamic models but at a reduced cost by only profiling 30 % of the programs.
Sample Complexity versus Depth: An Information Theoretic Analysis

Deep learning has proven effective across a range of data sets. In light of this, a natural inquiry is: “for what data generating processes can deep learning succeed? ” In this work, we study the sample complexity of learning multilayer data generating processes of a sort for which deep neural networks seem to be suited. We develop general and elegant information-theoretic tools that accommodate analysis of any data generating process – shallow or deep, parametric or nonparametric, noiseless or noisy. We then use these tools to characterize the dependence of sample complexity on the depth of multilayer processes. Our results indicate roughly linear dependence on depth. This is in contrast to previous results that suggest exponential or high-order polynomial dependence.
Privacy-Friendly Flexible IoT Health Data Processing with User-Centric Access Control

—This paper proposes a novel Single and Multiple user(s) data Aggregation (SAMA) scheme designed to support privacy preserving aggregation of health data collected from users’ IoT wearables. SAMA also deploys a user-centric approach to support ﬂexible ﬁne-grain access control. It achieves this by deploying two key ideas. First, it uses multi-key Homo-morphic cryptosystem (variant Paillier) to allow ﬂexibility in accommodating both single and multi-user data processing as well as preserving the privacy of users while processing their IoT health data. Second, it uses ciphertext-policy attribute-based encryption to support ﬂexible access control, which ensures users are able to grant data access securely and selectively. Formal security and privacy analyses have shown that SAMA supports data conﬁdentiality and authorisation. The scheme has also been analysed in terms of computational and communication overheads to demonstrate that it is more efﬁcient than the relevant state-of-the-art solutions.
Unbounded generalization of the Baker-Campbell-Hausdorff formulae

Based on the operator representation on the module over Banach algebra $B(X)$, the Campbell-Baker-Hausdorff formula is generalized to the unbounded situations. In conclusion, by means of the logarithmic representation of generally-unbounded operators, the Campbell-Baker-Hausdorff formula is generalized to be applicable to the unbounded operators.
Measuring the propagation speed of gravitational waves with LISA

The propagation speed of gravitational waves, cT , has been tightly constrained by the binary neutron star merger GW170817 and its electromagnetic counterpart, under the assumption of a frequency-independent cT . Drawing upon arguments from Effective Field Theory and quantum gravity, we discuss the possibility that modifications of General Relativity allow for transient deviations of cT from the speed of light at frequencies well below the band of current ground-based detectors. We motivate two representative Ansätze for cT (f), and study their impact upon the gravitational waveforms of massive black hole binary mergers detectable by the LISA mission. We forecast the constraints on cT (f) obtainable from individual systems and a population of sources, from both inspiral and a full inspiral-merger-ringdown waveform. We show that LISA will enable us to place stringent independent bounds on departures from General Relativity in unexplored low-frequency regimes, even in the absence of an electromagnetic counterpart.
Chemical Variations Across the TMC-1 Boundary: Molecular Tracers from the Translucent Phase to the Dense Phase

We investigated the chemical evolutions of gas-phase and grain-surface species across the Taurus molecular cloud-1 (TMC-1) filament from the translucent phase to the dense phase. By comparing observations with modeling results from an up-to-date chemical network, we examined the conversion processes for the carbon-, oxygen-, nitrogen-, and sulfur-bearing species, i.e., from their initial atomic form to their main molecular reservoir form both in the gas phase and on the grain surface. The conversion processes were found to depend on the species and A V . The effect of initial carbon-to-oxygen elemental abundances ratio (C/O) by varying O on the chemistry was explored, and an initial carbon elemental abundance of 2.5 × 10−4 and a C/O ratio of 0.5 could best reproduce the abundances of most observed molecules at TMC-1 CP, where more than 90 molecules have been identified. Based on the TMC-1 condition, we predicted a varied grain ice composition during the evolutions of molecular clouds, with H2O ice as the dominant ice composition at A V > 4 mag, CO2 ice as the dominant ice composition at A V <4 mag, while CO ice severely decreased at A V around 4–5 mag.
Structure Extraction in Task-Oriented Dialogues with Slot Clustering

Extracting structure information from dialogue data can help us better understand user and system behaviors. In task-oriented dialogues, dialogue structure has often been considered as transition graphs among dialogue states. However, annotating dialogue states manually is expensive and time-consuming. In this paper, we propose a simple yet effective approach for structure extraction in task-oriented dialogues. We first detect and cluster possible slot tokens with a pre-trained model to approximate dialogue ontology for a target domain. Then we track the status of each identified token group and derive a state transition structure. Empirical results show that our approach outperforms unsupervised baseline models by far in dialogue structure extraction. In addition, we show that data augmentation based on extracted structures enriches the surface formats of training data and can achieve a significant performance boost in dialogue response generation.
Automatic Depression Detection via Learning and Fusing Features From Visual Cues

Depression is one of the most prevalent mental disorders, which seriously affects one’s life. Traditional depression diagnostics commonly depend on rating with scales, which can be labor-intensive and subjective. In this context, automatic depression detection (ADD), aiming to assist medical experts in their diagnosis and analysis, has been attracting more attention for its better objectivity and fewer laborious interventions. A typical ADD model detects depression via automatically extracting task-specific features from medical records, such as video sequences, and sending them into a classifier for assistive prediction. However, it remains challenging to effectively extract depression-specific information from long sequences, thereby hindering a satisfying accuracy. In this article, we propose a novel ADD method via learning and fusing features from visual cues. Specifically, we first construct temporal dilated convolutional network (TDCN), in which multiple dilated convolution blocks (DCBs) are designed and stacked, to learn the long-range temporal information from sequences. Then, the featurewise attention (FWA) module is adopted to fuse different features extracted from TDCNs. The module learns to assign weights for the feature channels, aiming to better incorporate different kinds of visual features and further enhance the detection accuracy. Our method achieves the state-of-the-art performance on the Distress Analysis Interview Corpus Wizard-of-Oz (DAIC_WOZ) dataset compared with other visual-feature-based methods, showing its effectiveness.
Learning Neural Hamiltonian Dynamics: A Methodological Overview

The past few years have witnessed an increased interest in learning Hamiltonian dynamics in deep learning frameworks. As an inductive bias based on physical laws, Hamiltonian dynamics endow neural networks with accurate long-term prediction, interpretability, and data-efficient learning. However, Hamiltonian dynamics also bring energy conservation or dissipation assumptions on the input data and additional computational overhead. In this paper, we systematically survey recently proposed Hamiltonian neural network models, with a special emphasis on methodologies. In general, we discuss the major contributions of these models, and compare them in four overlapping directions: 1) generalized Hamiltonian system; 2) symplectic integration, 3) generalized input form, and 4) extended problem settings. We also provide an outlook of the fundamental challenges and emerging opportunities in this area.
Risk-averse controller design against data injection attacks on actuators for uncertain control systems

In this paper, we consider the optimal controller design problem against data injection attacks on actuators for an uncertain control system. We consider attacks that aim at maximizing the attack impact while remaining stealthy in the finite horizon. To this end, we use the Conditional Value-at-Risk to characterize the risk associated with the impact of attacks. The worst-case attack impact is characterized using the recently proposed output-to-output ℓ2-gain (OOG). We formulate the design problem and observe that it is non-convex and hard to solve. Using the framework of scenario-based optimization and a convex proxy for the OOG, we propose a convex optimization problem that approximately solves the design problem with probabilistic certificates. Finally, we illustrate the results through a numerical example.
Gradient Descent Is Optimal Under Lower Restricted Secant Inequality And Upper Error Bound

The study of first-order optimization is sensitive to the assumptions made on the objective functions. These assumptions induce complexity classes which play a key role in worst-case analysis, including the fundamental concept of algorithm optimality. Recent work argues that strong convexity and smoothness, popular assumptions in literature, lead to a pathological definition of the condition number (Guille-Escuret et al., 2021). Motivated by this result, we focus on the class of functions satisfying a lower restricted secant inequality and an upper error bound. On top of being robust to the aforementioned pathological behavior and including some non-convex functions, this pair of conditions displays interesting geometrical properties. In particular, the necessary and sufficient conditions to interpolate a set of points and their gradients within the class can be separated into simple conditions on each sampled gradient. This allows the performance estimation problem (PEP, Drori and Teboulle (2012)) to be solved analytically, leading to a lower bound on the convergence rate that proves gradient descent to be exactly optimal on this class of functions among all first-order algorithms.
NeuRecover: Regression-Controlled Repair of Deep Neural Networks with Training History

Systematic techniques to improve quality of deep neural networks (DNNs) are critical given the increasing demand for practical applications including safety-critical ones. The key challenge comes from the little controllability in updating DNNs. Retraining to fix some behavior often has a destructive impact on other behavior, causing regressions, i.e., the updated DNN fails with inputs correctly handled by the original one. This problem is crucial when engineers are required to investigate failures in intensive assurance activities for safety or trust. Search-based repair techniques for DNNs have potentials to tackle this challenge by enabling localized updates only on “responsible parameters” inside the DNN. However, the potentials have not been explored to realize sufficient controllability to suppress regressions in DNN repair tasks. In this paper, we propose a novel DNN repair method that makes use of the training history for judging which DNN parameters should be changed or not to suppress regressions. We implemented the method into a tool called Neurecover and evaluated it with three datasets. Our method outperformed the existing method by achieving often less than a quarter, even a tenth in some cases, number of regressions. Our method is especially effective when the repair requirements are tight to fix specific failure types. In such cases, our method showed stably low rates (<2 %) of regressions, which were in many cases a tenth of regressions caused by retraining.
Bianchi Type I model of universe with customized scale factors

According to standard cosmology, the universe is homogeneous and isotropic at large scales. However, some anisotropies can be observed at the local scale in the universe through various ways. Here, we have studied the Bianchi Type I model by customizing the scale factors to understand the anisotropic nature of the universe. We have considered two cases with slight modifications of scale factors in different directions in the generalized Bianchi Type I metric equation, and compared the results with the [Formula: see text]CDM model and also with available cosmological observational data. Through this study, we also want to predict the possible degree of anisotropy present in the early universe and its evolution to current time by calculating the value of density parameter for anisotropy [Formula: see text] for both low and high redshift [Formula: see text] along with the possible relative anisotropy that exist among different directions. It is found that there was a relatively higher amount of anisotropy in the early universe and the anisotropic nature of the universe vanishes at the near past and the present epochs. Thus, at near past and present stages of the universe there is no effective distinction between this anisotropic model and the standard [Formula: see text]CDM model.
GraphWorld: Fake Graphs Bring Real Insights for GNNs

Despite advances in the field of Graph Neural Networks (GNNs), only a small number (~5) of datasets are currently used to evaluate new models. This continued reliance on a handful of datasets provides minimal insight into the performance differences between models, and is especially challenging for industrial practitioners who are likely to have datasets which are very different from academic benchmarks. In the course of our work on GNN infrastructure and open-source software at Google, we have sought to develop benchmarks that are robust, tunable, scalable, and generalizable. In this work we introduce GraphWorld, a novel methodology and system for benchmarking GNN models on an arbitrarily-large population ofsynthetic graphs for any conceivable GNN task. GraphWorld allows a user to efficiently generate a world with millions of statistically diverse datasets. It is accessible, scalable, and easy to use. GraphWorld can be run on a single machine without specialized hardware, or it can be easily scaled up to run on arbitrary clusters or cloud frameworks. Using GraphWorld, a user has fine-grained control over graph generator parameters, and can benchmark arbitrary GNN models with built-in hyperparameter tuning. We present insights from GraphWorld experiments regarding the performance characteristics of thirteen GNN models and baselines over millions of benchmark datasets. We further show that GraphWorld efficiently explores regions of benchmark dataset space uncovered by standard benchmarks, revealing comparisons between models that have not been historically obtainable. Using GraphWorld, we also are able to study in-detail the relationship between graph properties and task performance metrics, which is nearly impossible with the classic collection of real-world benchmarks.
Deep Camera Pose Regression Using Pseudo-LiDAR

An accurate and robust large-scale localization system is an integral component for active areas of research such as autonomous vehicles and augmented reality. To this end, many learning algorithms have been proposed that predict 6DOF camera pose from RGB or RGB-D images. However, previous methods that incorporate depth typically treat the data the same way as RGB images, often adding depth maps as additional channels to RGB images and passing them through convolutional neural networks (CNNs). In this paper, we show that converting depth maps into pseudo-LiDAR signals, previously shown to be useful for 3D object detection, is a better representation for camera localization tasks by projecting point clouds that can accurately determine 6DOF camera pose. This is demonstrated by first comparing localization accuracies of a network operating exclusively on pseudo-LiDAR representations, with networks operating exclusively on depth maps. We then propose FusionLoc, a novel architecture that uses pseudo-LiDAR to regress a 6DOF camera pose. FusionLoc is a dual stream neural network, which aims to remedy common issues with typical 2D CNNs operating on RGB-D images. The results from this architecture are compared against various other state-of-the-art deep pose regression implementations using the 7 Scenes dataset. The findings are that FusionLoc performs better than a number of other camera localization methods, with a notable improvement being, on average, 0.33m and 4.35{\deg} more accurate than RGB-D PoseNet. By proving the validity of using pseudo-LiDAR signals over depth maps for localization, there are new considerations when implementing large-scale localization systems.
Deep Potentials for Materials Science

To fill the gap between accurate (and expensive) ab initio calculations and efficient atomistic simulations based on empirical interatomic potentials, a new class of descriptions of atomic interactions has emerged and been widely applied; i.e., machine learning potentials (MLPs). One recently developed type of MLP is the Deep Potential (DP) method. In this review, we provide an introduction to DP methods in computational materials science. The theory underlying the DP method is presented along with a step-by-step introduction to their development and use. We also review materials applications of DPs in a wide range of materials systems. The DP Library provides a platform for the development of DPs and a database of extant DPs. We discuss the accuracy and efficiency of DPs compared with ab initio methods and empirical potentials.
On the Generalization of Representations in Reinforcement Learning

In reinforcement learning, state representations are used to tractably deal with large problem spaces. State representations serve both to approximate the value function with few parameters, but also to generalize to newly encountered states. Their features may be learned implicitly (as part of a neural network) or explicitly (for example, the successor representation of \citet{dayan1993improving}). While the approximation properties of representations are reasonably well-understood, a precise characterization of how and when these representations generalize is lacking. In this work, we address this gap and provide an informative bound on the generalization error arising from a specific state representation. This bound is based on the notion of effective dimension which measures the degree to which knowing the value at one state informs the value at other states. Our bound applies to any state representation and quantifies the natural tension between representations that generalize well and those that approximate well. We complement our theoretical results with an empirical survey of classic representation learning methods from the literature and results on the Arcade Learning Environment, and find that the generalization behaviour of learned representations is well-explained by their effective dimension.
Distributed Alignment Processes With Samples of Group Average

This article studies a stochastic alignment problem assuming that agents can sense the general tendency of the system. More specifically, we consider <inline-formula><tex-math notation="LaTeX">$n$</tex-math></inline-formula> agents, each being associated with a real number. In each round, each agent receives a noisy measurement of the system's average value and then updates its value. This value is then perturbed by random drift. We assume that both noise and drift are Gaussian. We prove that a distributed weighted-average algorithm optimally minimizes the deviation of each agent from the average value, and for every round. Interestingly, this optimality holds even in the centralized setting, where a master agent can gather all the agents' measurements and instruct a move to each one. We find this result surprising since it can be shown that the set of measurements obtained by all agents contains strictly more information about the deviation of Agent <inline-formula><tex-math notation="LaTeX">$i$</tex-math></inline-formula> from the average value, than the information contained in the measurements obtained by Agent <inline-formula><tex-math notation="LaTeX">$i$</tex-math></inline-formula> alone. Although this information is relevant for Agent <inline-formula><tex-math notation="LaTeX">$i$</tex-math></inline-formula>, it is not processed by it when running a weighted-average algorithm. Finally, we also analyze the drift of the center of mass and show that no distributed algorithm can achieve drift that is as small as the one that can be achieved by the best centralized algorithm.
Concentration of measure bounds for matrix-variate data with missing values

We consider the following data perturbation model, where the covariates incur multiplicative errors. For two $n \times m$ random matrices $U, X$, we denote by $U \circ X$ the Hadamard or Schur product, which is defined as $(U \circ X)_{ij} = (U_{ij}) \cdot (X_{ij})$. In this paper, we study the subgaussian matrix variate model, where we observe the matrix variate data $X$ through a random mask $U$: $$ {\mathcal X} = U \circ X \; \; \; \text{ where} \; \; \;X = B^{1/2} {\mathbb{Z}} A^{1/2}, $$ where ${\mathbb{Z}}$ is a random matrix with independent subgaussian entries, and $U$ is a mask matrix with either zero or positive entries, where ${\mathbb E} U_{ij} \in [0, 1]$ and all entries are mutually independent. Subsampling in rows, or columns, or random sampling of entries of $X$ are special cases of this model. Under the assumption of independence between $U$ and $X$, we introduce componentwise unbiased estimators for estimating covariance $A$ and $B$, and prove the concentration of measure bounds in the sense of guaranteeing the restricted eigenvalue($\textsf{RE}$) conditions to hold on the unbiased estimator for $B$, when columns of data matrix $X$ are sampled with different rates. We further develop multiple regression methods for estimating the inverse of $B$ and show statistical rate of convergence. Our results provide insight for sparse recovery for relationships among entities (samples, locations, items) when features (variables, time points, user ratings) are present in the observed data matrix ${\mathcal X}$ with heterogeneous rates. Our proof techniques can certainly be extended to other scenarios. We provide simulation evidence illuminating the theoretical predictions.