Miracle-dz/UniFAR-FTUs huggingface dataset:
Benchmarking the Linear Algebra Awareness of TensorFlow and PyTorch

Linear algebra operations, which are ubiquitous in machine learning, form major performance bottlenecks. The High-Performance Computing community invests significant effort in the development of architecture-specific optimized kernels, such as those provided by the BLAS and LAPACK libraries, to speed up linear algebra operations. However, end users are progressively less likely to go through the error prone and time-consuming process of directly using said kernels; instead, frameworks such as TensorFlow (TF) and PyTorch (PyT), which facilitate the development of machine learning applications, are becoming more and more popular. Although such frameworks link to BLAS and LAPACK, it is not clear whether or not they make use of linear algebra knowledge to speed up computations. For this reason, in this paper we develop benchmarks to investigate the linear algebra optimization capabilities of TF and PyT. Our analyses reveal that a number of linear algebra optimizations are still missing; for instance, reducing the number of scalar operations by applying the distributive law, and automatically identifying the optimal parenthesization of a matrix chain. In this work, we focus on linear algebra computations in TF and PyT; we both expose opportunities for performance enhancement to the benefit of the developers of the frameworks and provide end users with guidelines on how to achieve performance gains.
SOInter: A Novel Deep Energy Based Interpretation Method for Explaining Structured Output Models

We propose a novel interpretation technique to explain the behavior of structured output models, which learn mappings between an input vector to a set of output variables simultaneously. Because of the complex relationship between the computational path of output variables in structured models, a feature can affect the value of output through other ones. We focus on one of the outputs as the target and try to find the most important features utilized by the structured model to decide on the target in each locality of the input space. In this paper, we assume an arbitrary structured output model is available as a black box and argue how considering the correlations between output variables can improve the explanation performance. The goal is to train a function as an interpreter for the target output variable over the input space. We introduce an energy-based training process for the interpreter function, which effectively considers the structural information incorporated into the model to be explained. The effectiveness of the proposed method is confirmed using a variety of simulated and real data sets.
Quantum information scrambling in molecules

Chenghao Zhang, Peter G. Wolynes and Martin Gruebele Department of Physics, University of Illinois at Urbana-Champaign, Illinois 61801, USA, Departments of Chemistry and Physics, and Center for Theoretical Biological Physics, Rice University, 6100 Main Street, Houston, TX 77251, USA, Department of Chemistry, Center for Biophysics and Quantitative Biology, University of Illinois at Urbana-Champaign, Illinois 61801, USA
Towards Enabling Dynamic Convolution Neural Network Inference for Edge Intelligence

Deep learning applications have achieved great success in numerous real-world applications. Deep learning models, especially Convolution Neural Networks (CNN) are often prototyped using FPGA because it offers high power efficiency, and reconfigurability. The deployment of CNNs on FPGAs follows a design cycle that requires saving of model parameters in the on-chip memory during High level synthesis (HLS). Recent advances in edge intelligence requires CNN inference on edge network to increase throughput and reduce latency. To provide flexibility, dynamic parameter allocation to different mobile devices is required to implement either a predefined or defined on-the-fly CNN architecture. In this study, we present novel methodologies for dynamically streaming the model parameters at run-time to implement a traditional CNN architecture. We further propose a library-based approach to design scalable and dynamic distributed CNN inference on the fly leveraging partial-reconfiguration techniques, which is particularly suitable for resource constrained edge devices. The proposed techniques are implemented on the Xilinx PYNQ-Z2 board to prove the concept by utilizing the LeNet-5 CNN model. The results show that the proposed methodologies are effective, with classification accuracy rates of 92%, 86% and 94% respectively.
Performance Analysis of Optimally Coordinated Connected and Automated Vehicles in a Mixed Traffic Environment

Trajectory planning of connected and automated vehicles (CAVs) poses significant challenges in a mixed traffic environment due to the presence of human-driven vehicles (HDVs). In this paper, we apply a framework that allows coordination of CAVs and HDVs traveling through a traffic corridor consisting of an on-ramp merging, a speed reduction zone, and a roundabout. We study the impact of different penetration rates of CAVs and traffic volumes on the efficiency of the corridor. We provide extensive simulation results and report on the benefits in terms of total travel time and fuel economy.
DeepShovel: An Online Collaborative Platform for Data Extraction in Geoscience Literature with AI Assistance

Geoscientists, as well as researchers in many fields, need to read a huge amount of literature to locate, extract, and aggregate relevant results and data to enable future research or to build a scientific database, but there is no existing system to support this use case well. In this paper, based on the findings of a formative study about how geoscientists collaboratively annotate literature and extract and aggregate data, we proposed DeepShovel, a publicly-available AI-assisted data extraction system to support their needs. DeepShovel leverages the state-of-the-art neural network models to support researcher(s) easily and accurately annotate papers (in the PDF format) and extract data from tables, figures, maps, etc. in a human-AI collaboration manner. A follow-up user evaluation with 14 researchers suggested DeepShovel improved users' efficiency of data extraction for building scientific databases, and encouraged teams to form a larger scale but more tightly-coupled collaboration.
Functional Renormalisation for $f(R_{\mu\nu\rho\sigma})$ Quantum Gravity

We derive new functional renormalisation group flows for quantum gravity, in any dimension. The key new achievement is that the equations apply for any theory of gravity whose underlying Lagrangian $\sim f(R_{\mu\nu\rho\sigma})$ is a function of the Riemann tensor and the inverse metric. The results centrally exploit the benefits of maximally symmetric spaces for the evaluation of operator traces. The framework is highly versatile and offers a wide range of new applications to study quantum gravitational effects in extensions of Einstein gravity, many of which have hitherto been out of reach. The phase diagram and sample flows for Einstein-Hilbert gravity, Gauss-Bonnet, and selected higher-order theories of gravity are given. We also provide an algorithm to find the flow for general polynomial Riemann curvature interactions. The setup vastly enhances the reach of fixed point searches, enabling novel types of search strategies including across the operator space spanned by polynomial curvature invariants, and in extensions of general relativity relevant for cosmology. Further implications, and links with unimodular versions of gravity are indicated.
A New Flow‐Kinematics‐Based Model for Time‐Dependent Effective Dispersion in Mixing‐Limited Reactions

A new upscaling procedure that provides 1D representations of 2D mixing‐limited reactive transport systems is developed and applied. A key complication with upscaled models in this setting is that the procedure must differentiate between interface spreading, driven by the spatially variable velocity field, and mixing, through which components contact one another and react. Our model captures the enhanced mixing caused by spreading through use of a time‐dependent effective dispersion term. The early‐time behavior of this dispersion is driven by flow kinematics, while at late times it reaches a Taylor‐dispersion‐like limit. The early‐time behavior is modeled here using a very fast (purely advective) particle tracking procedure. The only free parameter in the model is the late‐time asymptotic effective dispersion. This quantity is estimated using a fit involving a dimensionless grouping of system variables and a few reference results, or by calibrating with the corresponding conservative (non‐reacting) case. Numerical results for bimolecular reaction systems are generated using a pseudo‐spectral approach capable of resolving fronts at high Peclet numbers. Results are presented for three different types of 2D velocity fields over a wide range of parameters. The upscaled model is shown to provide highly accurate results for the conversion factor, along with reasonable approximations of the spatial distribution of reaction occurrence. The model is also shown to be valid to upscale mixing in non‐reacting systems.
Normalized ground states for 3D dipolar Bose-Einstein condensate with attractive three-body interactions

We study the existence of normalized ground states for the 3D dipolar Bose-Einstein condensate equation with attractive three-body interactions: −∆u+ βu+ λ1|u| 2 u+ λ2(K ∗ |u| 2)u− |u|4u = 0. (DBEC) When λ2 = 0 or u is radial, (DBEC) reduces to the cubic-quintic NLS −∆u+ βu+ λ1|u| 2 u− |u|4u = 0, (CQNLS) which has been recently studied by Soave in [31]. In particular, it was shown that for any λ1 < 0 and c > 0, (CQNLS) possesses a radially symmetric ground state solution with mass c and for λ1 ≥ 0, (CQNLS) has no non-trivial solution. We show that by adding a dipole-dipole interaction to (CQNLS), the geometric nature of (CQNLS) changes dramatically and techniques as the ones from [31] cannot be used anymore to obtain similar results. More precisely, due to the axisymmetric nature of the dipole-dipole interaction potential, the energy corresponding to (DBEC) is not stable under symmetric rearrangements, hence conventional arguments based on the radial symmetry of solutions are inapplicable. We will overcome this difficulty by appealing to subtle variational and perturbative methods and prove the following: (i) If the pair (λ1, λ2) is unstable and λ1 < 0, then for any c > 0, (DBEC) has a ground state solution with mass c. (ii) If the pair (λ1, λ2) is unstable and λ1 ≥ 0, then there exists some c ∗ = c∗(λ1, λ2) ≥ 0 such that for all c > c∗, (DBEC) has a ground state solution with mass c. Moreover, any non-trivial solution of (DBEC) in this case must be non-radial. (iii) If the pair (λ1, λ2) is stable, then (DBEC) has no non-trivial solutions.
Avqvc: One-Shot Voice Conversion By Vector Quantization With Applying Contrastive Learning

Voice Conversion(VC) refers to changing the timbre of a speech while retaining the discourse content. Recently, many works have focused on disentangle-based learning techniques to separate the timbre and the linguistic content information from a speech signal. Once successful, voice conversion will be feasible and straightforward. This paper proposed a novel one-shot voice conversion framework based on vector quantization voice conversion (VQVC) and AutoVC, called AVQVC. A new training method is applied to VQVC to separate content and timbre information from speech more effectively. The result shows that this approach has better performance than VQVC in separating content and timbre to improve the sound quality of generated speech.
Navigating Conceptual Space; A new take on Artificial General Intelligence

Edward C. Tolman found reinforcement learning unsatisfactory for explaining intelligence and proposed a clear distinction between learning and behavior. Tolman's ideas on latent learning and cognitive maps eventually led to what is now known as conceptual space, a geometric representation where concepts and ideas can form points or shapes.Active navigation between ideas - reasoning - can be expressed directly as purposive navigation in conceptual space. Assimilating the theory of conceptual space from modern neuroscience, we propose autonomous navigation as a valid approach for emulated cognition. However, achieving autonomous navigation in high-dimensional Euclidean spaces is not trivial in technology. In this work, we explore whether neoRL navigation is up for the task; adopting Kaelbling's concerns for efficient robot navigation, we test whether the neoRL approach is general across navigational modalities, compositional across considerations of experience, and effective when learning in multiple Euclidean dimensions. We find neoRL learning to be more resemblant of biological learning than of RL in AI, and propose neoRL navigation of conceptual space as a plausible new path toward emulated cognition.
Double Thompson Sampling in Finite stochastic Games

We consider the trade-off problem between exploration and exploitation under finite discounted Markov Decision Process, where the state transition matrix of the underlying environment stays unknown. We propose a double Thompson sampling reinforcement learning algorithm(DTS) to solve this kind of problem. This algorithm achieves a total regret bound of $\tilde{\mathcal{O}}(D\sqrt{SAT})$in time horizon $T$ with $S$ states, $A$ actions and diameter $D$. DTS consists of two parts, the first part is the traditional part where we apply the posterior sampling method on transition matrix based on prior distribution. In the second part, we employ a count-based posterior update method to balance between the local optimal action and the long-term optimal action in order to find the global optimal game value. We established a regret bound of $\tilde{\mathcal{O}}(\sqrt{T}/S^{2})$. Which is by far the best regret bound for finite discounted Markov Decision Process to our knowledge. Numerical results proves the efficiency and superiority of our approach.
Streak-less wall-bounded turbulence

The effect of damping the longest streaks in wall-bounded turbulence is explored using numerical experiments. It is found that long streaks are not required for the self-sustenance of the bursting process, which is relatively little affected by their absence. The only perturbations of the streamwise velocity required to sustain turbulence are shown to be those associated with the bursts themselves. This suggests that long streaks are byproducts, rather than active parts of the process.
Alternative Design of DeepPDNet in the Context of Image Restoration

This work designs an image restoration deep network relying on unfolded Chambolle-Pock primal-dual iterations. Each layer of our network is built from Chambolle-Pock iterations when specified for minimizing a sum of a $\ell _2$-norm data-term and an analysis sparse prior. The parameters of our network are the step-sizes of the Chambolle-Pock scheme and the linear operator involved in sparsity-based penalization, including implicitly the regularization parameter. A backpropagation procedure is fully described. Preliminary experiments illustrate the good behavior of such a deep primal-dual network in the context of image restoration on BSD68 database.
A Foundation for Wireless Channel Prediction and Full Ray Makeup Estimation Using an Unmanned Vehicle

In this article, we consider the problem of wireless channel prediction, where we are interested in predicting the channel quality at unvisited locations in an area of interest, based on a small number of prior received power measurements collected by an unmanned vehicle in the area. We propose a new framework for channel prediction that can predict the detailed variations of the received power as well as the detailed makeup of the signal rays (i.e., amplitude, angle-of-arrival (AoA), and phase of the incoming paths). More specifically, we show how an enclosure-based robotic route design ensures that the received measurements at the prior measurement locations can be utilized to fully predict detailed ray parameters at unvisited locations. We then show how to first estimate the detailed ray parameters at the prior measurement route and then extend them to predict the detailed ray makeup at unvisited locations in the area. We validate our proposed framework through extensive real-world experiments in three different areas and show that our approach can accurately predict the received channel power and the detailed makeup of the rays at unvisited locations in an area, considerably outperforming the state-of-the-art in wireless channel prediction.
Delegated Pandora's Box

In delegation problems, a principal does not have the resources necessary to complete a particular task, so they delegate the task to an untrusted agent whose interests may differ from their own. Given any family of such problems and space of mechanisms for the principal to choose from, the delegation gap is the worst-case ratio of the principal's optimal utility when they delegate versus their optimal utility when solving the problem on their own. In this work, we consider the delegation gap of the generalized Pandora's box problem, a search problem in which searching for solutions incurs known costs and solutions are restricted by some downward-closed constraint. First, we show that there is a special case when all random variables have binary support for which there exist constant-factor delegation gaps for matroid constraints. However, there is no constant-factor delegation gap for even simple non-binary instances of the problem. Getting around this impossibility, we consider two variants: the free-agent model, in which the agent doesn't pay the cost of probing elements, and discounted-cost approximations, in which we discount all costs and aim for a bicriteria approximation of the discount factor and delegation gap. We show that there are constant-factor delegation gaps in the free-agent model with discounted-cost approximations for certain downward closed constraints and constant discount factors. However, constant delegation gaps can not be achieved under either variant alone. Finally, we consider another variant called the shared-cost model, in which the principal can choose how costs will be shared between them and the agent before delegating the search problem. We show that the shared-cost model exhibits a constant-factor delegation gap for certain downward closed constraints.
The nature of losses from cyber-related events: risk categories and business sectors

In this study, we examine the nature of losses from cyber-related events across different risk categories and business sectors. Using a leading industry dataset of cyber events, we evaluate the relationship between the frequency and severity of individual cyber-related events and the number of affected records. We find that the frequency of reported cyber-related events has substantially increased between 2008 and 2016. Furthermore, the frequency and severity of losses depend on the business sector and type of cyber threat: the most significant cyber loss event categories, by number of events, were related to data breaches and the unauthorized disclosure of data, while cyber extortion, phishing, spoofing, and other social engineering practices showed substantial growth rates. Interestingly, we do not find a distinct pattern between the frequency of events, the loss severity, and the number of affected records as often alluded to in the literature. We also analyse the severity distribution of cyber-related events across all risk categories and business sectors. This analysis reveals that cyber risks are heavy-tailed, i.e. cyber risk events have a higher probability to produce extreme losses than events whose severity follows an exponential distribution. Furthermore, we find that the frequency and severity of cyber-related losses exhibit a very dynamic and time-varying nature.
Bad priors, their threats to Bayesian optimization and some remedies via prior learning

The performance of deep neural networks can be highly sensitive to the choice of a variety of meta-parameters, such as optimizer parameters and model hyperparameters. Tuning these well, however, often requires extensive and costly experimentation. Bayesian optimization (BO) is a principled approach to solve such expensive hyperparameter tuning problems efﬁciently. Key to the performance of BO is specifying and reﬁning a distribution over functions, which is used to reason about the optima of the underlying function being optimized. In this work, we consider the scenario where we have data from similar functions that allows us to specify a tighter distribution a priori. Speciﬁcally, we focus on the common but potentially costly task of tuning optimizer parameters for training neural networks. Building on the meta BO method from Wang et al. (2018b), we develop practical improvements that (a) boost its performance by leveraging tuning results on multiple tasks without requiring observations for the same meta-parameter points across all tasks, and (b) retain its regret bound for a special case of our method. As a result, we provide a coherent BO solution for iterative optimization of continuous optimizer parameters. To verify our approach in realistic model training setups, we collected a large multi-task hyperparameter tuning dataset by training tens of thousands of conﬁgurations of near-state-of-the-art models on popular image and text datasets, as well as a protein sequence dataset. Our results show that on average, our method is able to locate good hyperparameters at least 3 times more efﬁciently than the best competing methods.
Deep kernel machines: exact inference with representation learning in inﬁnite Bayesian neural networks

Deep neural networks (DNNs) with the ﬂexi-bility to learn good top-layer representations have eclipsed shallow kernel methods without that ﬂexibility. Here, we take inspiration from DNNs to develop the deep kernel machine. Optimizing the deep kernel machine objective is equivalent to exact Bayesian inference (or noisy gradient descent) in an inﬁnitely wide Bayesian neural network or deep Gaussian process, which has been scaled carefully to retain representation learning. Our work thus has important implications for theoretical understanding of neural networks. In addition, we show that the deep kernel machine objec-tive has more desirable properties and better performance than other choices of objective. Finally, we conjecture that the deep kernel machine objective is unimodal. We give a proof of unimodality for linear kernels, and a number of experiments in the nonlinear case in which all deep kernel machines initializa-tions we tried converged to the same solution.
Study on Fault Diagnosis Method Based on Fractal Box Dimension of Indicator Diagram for Rod-less Oil Pumping Machine

By taking the water-base power rod-less oil pumping unit as a research object,based on the theory of fractal box counting dimension,the analysis on property of indicator diagrams was carried out under its 4 different working conditions such as,normal,tubing dropout,fluid insufficient and slow closed of traveling valve,and etc. The changing property of box counting dimension was studied when the signal to noise ratio(SNR) of pressure-flow indicator diagram was varying in scope of 0. 1 ~ 0. 2%,under different working conditions. The results show that the fractal box counting dimension of pressure-flow indicator diagram is little influenced while the system pressure and flow signals' SNR is reciprocated with the scale of 0. 1 ~ 0. 2%,which indicates that the box counting dimension of pressure-flow indicator diagram can be used to the fault diagnosis of water-base power rod-less oil pumping machine. At the same time,this method is favorable to realize automatic intelligent analysis based on the indicator diagram fault diagnosis. The study result is good to the online fast and accurate diagnosis the operation state of the rod-less oil pumping machine,which has values in practical application and theoretical study values.