Miracle-dz/UniFAR-FTUs huggingface dataset:
Skewed Laplace Spectral Mixture kernels for long-term forecasting in Gaussian process

Long-term forecasting involves predicting a horizon that is far ahead of the last observation. It is a problem of highly practical relevance, for instance for companies in order to decide upon expensive long-term investments. Despite recent progress and success of Gaussian Processes (GPs) based on Spectral Mixture kernels, long-term forecasting remains a challenging problem for these kernels because they decay exponentially at large horizons. This is mainly due their use of a mixture of Gaussians to model spectral densities. The challenges underlying long-term forecasting become evident by investigating the distribution of the Fourier coefficients of (the training part of) the signal, which is non-smooth, heavy-tailed, sparse and skewed. Notably the heavy tail and skewness characteristics of such distribution in spectral domain allow to capture long range covariance of the signal in the time domain. Motivated by these observations, we propose to model spectral densities using a Skewed Laplace Spectral Mixture (SLSM) due to the skewness of its peaks, sparsity, non-smoothness, and heavy tail characteristics. By applying the inverse Fourier Transform to this spectral density we obtain a new GP kernel for long-term forecasting. Results of extensive experiments, including a multivariate time series, show the beneficial effect of the proposed SLSM kernel for long-term extrapolation and robustness to the choice of the number of mixture components.
Deep Learning based Monocular Depth Prediction: Datasets, Methods and Applications

Estimating depth from RGB images can facilitate many computer vision tasks, such as indoor localization, height estimation, and simultaneous localization and mapping (SLAM). Recently, monocular depth estimation has obtained great progress owing to the rapid development of deep learning techniques. They surpass traditional machine learning-based methods by a large margin in terms of accuracy and speed. Despite the rapid progress in this topic, there are lacking of a comprehensive review, which is needed to summarize the current progress and provide the future directions. In this survey, we first introduce the datasets for depth estimation, and then give a comprehensive introduction of the methods from three perspectives: supervised learning-based methods, unsupervised learning-based methods, and sparse samples guidance-based methods. In addition, downstream applications that benefit from the progress have also been illustrated. Finally, we point out the future directions and conclude the paper.
Evolution of Artificial Intelligent Plane

With the growth of the internet, it is becoming hard to manage, configure and monitor networks. Recent trends to control and operate them is artificial intelligence based automation to minimize human intervention. Albeit this concept has been introduced since a decade with several different names, but the underlying goal remains the same, which is to make network intelligent enough to assemble, reassemble if configuration changes, and detect a problem on its own and fix it. As a result, in addition to Data Plane, Control Plane and Management Plane, a new plane called Artificial Intelligence (AI) Plane is introduced. Our main objective is to analyze all major AI plane techniques, frameworks and algorithms proposed in various types of networks. We propose a comprehensive and network independent framework to cover all aspects of AI plane, in particular we provide a systematically means of comparison. In conjunction to make AI plane understand simpler, this framework highlights relevant challenges and design considerations for future research. To the best of our knowledge this is the first survey report which represents a complete comparison of AI planes with their investigation issues in several types of networks.
Graph Neural Network with Automorphic Equivalence Filters

Graph neural network (GNN) has recently been established as an effective representation learning framework on graph data. However, the popular message passing models rely on local permutation invariant aggregate functions, which gives rise to the concerns about their representational power. Here, we introduce the concept of automorphic equivalence to theoretically analyze GNN's expressiveness in differentiating node's structural role. We show that the existing message passing GNNs have limitations in learning expressive representations. Moreover, we design a novel GNN class that leverages learnable automorphic equivalence filters to explicitly differentiate the structural roles of each node's neighbors, and uses a squeeze-and-excitation module to fuse various structural information. We theoretically prove that the proposed model is expressive in terms of generating distinct representations for nodes with different structural feature. Besides, we empirically validate our model on eight real-world graph data, including social network, e-commerce co-purchase network and citation network, and show that it consistently outperforms strong baselines.
Machinery for Proving Sum-of-Squares Lower Bounds on Certification Problems

In this paper, we construct general machinery for proving Sum-of-Squares lower bounds on certification problems by generalizing the techniques used by Barak et al. [FOCS 2016] to prove Sum-of-Squares lower bounds for planted clique. Using this machinery, we prove degree $n^{\epsilon}$ Sum-of-Squares lower bounds for tensor PCA, the Wishart model of sparse PCA, and a variant of planted clique which we call planted slightly denser subgraph.
Elastocapillary network model of inhalation

The seemingly simple process of inhalation relies on a complex interplay between muscular contraction in the thorax, elasto-capillary interactions in individual lung branches, propagation of air between different connected branches, and overall air flow into the lungs. These processes occur over considerably different length and time scales; consequently, linking them to the biomechanical properties of the lungs, and quantifying how they together control the spatiotemporal features of inhalation, remains a challenge. We address this challenge by developing a computational model of the lungs as a hierarchical, branched network of connected liquid-lined flexible cylinders coupled to a viscoelastic thoracic cavity. Each branch opens at a rate and a pressure that is determined by input biomechanical parameters, enabling us to test the influence of changes in the mechanical properties of lung tissues and secretions on inhalation dynamics. By summing the dynamics of all the branches, we quantify the evolution of overall lung pressure and volume during inhalation, reproducing the shape of measured breathing curves. Using this model, we demonstrate how changes in lung muscle contraction, mucus viscosity and surface tension, and airway wall stiffness---characteristic of many respiratory diseases, including those arising from COVID-19, cystic fibrosis, chronic obstructive pulmonary disease, asthma, and emphysema---drastically alter inhaled lung capacity and breathing duration. Our work therefore helps to identify the key factors that control breathing dynamics, and provides a way to quantify how disease-induced changes in these factors lead to respiratory distress.
Towards Latency-aware DNN Optimization with GPU Runtime Analysis and Tail Effect Elimination

Despite the superb performance of State-Of-The-Art (SOTA) DNNs, the increasing computational cost makes them very challenging to meet real-time latency and accuracy requirements. Although DNN runtime latency is dictated by model property (e.g., architecture, operations), hardware property (e.g., utilization, throughput), and more importantly, the effective mapping between these two, many existing approaches focus only on optimizing model property such as FLOPS reduction and overlook the mismatch between DNN model and hardware properties. In this work, we show that the mismatch between the varied DNN computation workloads and GPU capacity can cause the idle GPU tail effect, leading to GPU under-utilization and low throughput. As a result, the FLOPs reduction cannot bring effective latency reduction, which causes sub-optimal accuracy versus latency trade-offs. Motivated by this, we propose a GPU runtime-aware DNN optimization methodology to eliminate such GPU tail effect adaptively on GPU platforms. Our methodology can be applied on top of existing SOTA DNN optimization approaches to achieve better latency and accuracy trade-offs. Experiments show 11%-27% latency reduction and 2.5%-4.0% accuracy improvement over several SOTA DNN pruning and NAS methods, respectively
Knowledge Distillation for Singing Voice Detection

Singing Voice Detection (SVD) has been an active area of research in music information retrieval (MIR). Currently, two deep neural network-based methods, one based on CNN and the other on RNN, exist in literature that learn optimized features for the voice detection (VD) task and achieve state-of-the-art performance on common datasets. Both these models have a huge number of parameters (1.4M for CNN and 65.7K for RNN) and hence not suitable for deployment on devices like smartphones or embedded sensors with limited capacity in terms of memory and computation power. The most popular method to address this issue is known as knowledge distillation in deep learning literature (in addition to model compression) where a large pretrained network known as the teacher is used to train a smaller student network. However, to the best of our knowledge, such methods have not been explored yet in the domain of SVD. In this paper, efforts have been made to investigate this issue using both conventional as well as ensemble knowledge distillation techniques. Through extensive experimentation on the publicly available Jamendo dataset, we show that, not only it's possible to achieve comparable accuracies with far smaller models (upto 1000x smaller in terms of parameters), but fascinatingly, in some cases, smaller models trained with distillation, even surpass the current state-of-the-art models on voice detection performance.
Differential Source Count for Gamma-Ray Bursts

Different forms of long gamma-ray burst (GRB) luminosity functions are considered on the basis of an explicit physical model. The inferred flux distributions are compared with the observed ones from two samples of GRBs, Swift and Fermi GBM. The best-fit parameters of the luminosity functions are found, and the physical interpretations are discussed. The results are consistent with the observation of a comparable number of flat-phase afterglows and monotonically decreasing ones.
Motion Prediction on Self-driving Cars: A Review

The autonomous vehicle motion prediction literature is reviewed. Motion prediction is the most challenging task in autonomous vehicles and self-drive cars. These challenges have been discussed. Later on, the state-of-theart has reviewed based on the most recent literature and the current challenges are discussed. The state-of-the-art consists of classical and physical methods, deep learning networks, and reinforcement learning. prons and cons of the methods and gap of the research presented in this review. Finally, the literature surrounding object tracking and motion will be presented. As a result, deep reinforcement learning is the best candidate to tackle self-driving cars.
Frequency-induced Superdiamagnetism in Epoxy/Magnetite Nanocomposites

The epoxy/magnetite nanocomposites express superparamagnetism under a static or a low-frequency electromagnetic field. At the microwave frequency, said the X-band, the nanocomposites reveal an unexpected diamagnetism. To explain the intriguing phenomenon, we revisit the Debye relaxation law with the memory effect. The magnetization vector of the magnetite is unable to synchronize with the rapidly changing magnetic field, and it contributes to superdiamagnetism, a negative magnetic susceptibility for nanoparticles. The model just developed and the fitting result can not only be used to explain the experimental data in the X-band, but also can be used to estimate the transition frequency between superparamagnetism and superdiamagnetism.
Stealthy Hacking and Secrecy of Controlled State Estimation Systems With Random Dropouts

We study the maximum information gain that an adversary may obtain through hacking without being detected. Consider a dynamical process observed by a sensor that transmits a local estimate of the system state to a remote estimator according to some reference transmission policy across a packet-dropping wireless channel equipped with acknowledgments (ACK). An adversary overhears the transmissions and proactively hijacks the sensor to reprogram its transmission policy. We define perfect secrecy as keeping the averaged expected error covariance bounded at the legitimate estimator and unbounded at the adversary. By analyzing the stationary distribution of the expected error covariance, we show that perfect secrecy can be attained for unstable systems only if the ACK channel has no packet dropouts. In other situations, we prove that independent of the reference policy and the detection methods, perfect secrecy is not attainable. For this scenario, we devise a Stackelberg game to derive the optimal defensive reference policy for the legitimate estimator and present a branch-and-bound algorithm with global optimality to solve the proposed game.
ESPnet-SE: End-To-End Speech Enhancement and Separation Toolkit Designed for ASR Integration

We present ESPnet-SE, which is designed for the quick development of speech enhancement and speech separation systems in a single framework, along with the optional downstream speech recognition module. ESPnet-SE is a new project which integrates rich automatic speech recognition related models, resources and systems to support and validate the proposed front-end implementation (i.e. speech enhancement and separation). It is capable of processing both single-channel and multi-channel data, with various functionalities including dereverberation, denoising and source separation. We provide all-in-one recipes including data pre-processing, feature extraction, training and evaluation pipelines for a wide range of benchmark datasets. This paper describes the design of the toolkit, several important functionalities, especially the speech recognition integration, which differentiates ESPnet-SE from other open source toolkits, and experimental results with major benchmark datasets.
Solving the Steiner Tree Problem with few Terminals

The Steiner tree problem is a well-known problem in network design, routing, and VLSI design. Given a graph, edge costs, and a set of dedicated vertices (terminals), the Steiner tree problem asks to output a sub-graph that connects all terminals at minimum cost. A state-of-the-art algorithm to solve the Steiner tree problem by means of dynamic programming is the Dijkstra-Steiner algorithm. The algorithm builds a Steiner tree of the entire instance by systematically searching for smaller instances, based on subsets of the terminals, and combining Steiner trees for these smaller instances. The search heavily relies on a guiding heuristic function in order to prune the search space. However, to ensure correctness, this algorithm allows only for limited heuristic functions, namely, those that satisfy a so-called consistency condition. In this paper, we enhance the Dijkstra-Steiner algorithm and establish a revisited algorithm, called DS*. The DS* algorithm allows for arbitrary lower bounds as heuristics relaxing the previous condition on the heuristic function. Notably, we can now use linear programming based lower bounds. Further, we capture new requirements for a heuristic function in a condition, which we call admissibility. We show that admissibility is indeed weaker than consistency and establish correctness of the DS* algorithm when using an admissible heuristic function. We implement DS* and combine it with modern preprocessing, resulting in an open-source solver (DS*Solve). Finally, we compare its performance on standard benchmarks and observe a competitive behavior.
A Survey of Label-noise Representation Learning: Past, Present and Future

Classical machine learning implicitly assumes that labels of the training data are sampled from a clean distribution, which can be too restrictive for real-world scenarios. However, statistical learning-based methods may not train deep learning models robustly with these noisy labels. Therefore, it is urgent to design Label-Noise Representation Learning (LNRL) methods for robustly training deep models with noisy labels. To fully understand LNRL, we conduct a survey study. We first clarify a formal definition for LNRL from the perspective of machine learning. Then, via the lens of learning theory and empirical study, we figure out why noisy labels affect deep models' performance. Based on the theoretical guidance, we categorize different LNRL methods into three directions. Under this unified taxonomy, we provide a thorough discussion of the pros and cons of different categories. More importantly, we summarize the essential components of robust LNRL, which can spark new directions. Lastly, we propose possible research directions within LNRL, such as new datasets, instance-dependent LNRL, and adversarial LNRL. Finally, we envision potential directions beyond LNRL, such as learning with feature-noise, preference-noise, domain-noise, similarity-noise, graph-noise, and demonstration-noise.
Depthwise Multiception Convolution for Reducing Network Parameters without Sacrificing Accuracy

Deep convolutional neural networks have been proven successful in multiple benchmark challenges in recent years. However, the performance improvements are heavily reliant on increasingly complex network architecture and a high number of parameters, which require ever increasing amounts of storage and memory capacity. Depthwise separable convolution (DSConv) can effectively reduce the number of required parameters through decoupling standard convolution into spatial and cross-channel convolution steps. However, the method causes a degradation of accuracy. To address this problem, we present depthwise multiception convolution, termed Multiception, which introduces layer-wise multiscale kernels to learn multiscale representations of all individual input channels simultaneously. We have carried out the experiment on four benchmark datasets, i.e. Cifar-10, Cifar-100, STL-10 and ImageNet32×32, using five popular CNN models, Multiception achieved accuracy promotion in all models and demonstrated higher accuracy performance compared to related works. Meanwhile, Multiception significantly reduces the number of parameters of standard convolution-based models by 32.48 % on average while still preserving accuracy.
Single- to many-body crossover of a quantum carpet

Strongly interacting many-body system of bosons exhibiting the quantum carpet pattern is investigated exactly by using Gaudin solutions. We show that this highly coherent design usually present in noninteracting, single-body scenarios gets destroyed by weak-to-moderate interatomic interactions in an ultracold bosonic gas trapped in a box potential. However, it becomes revived in a very strongly interacting regime, when the system undergoes fermionization. We track the whole single- to many-body crossover, providing an analysis of de- and recoherence present in the system.
Thermal-nonthermal energy partition in solar flares derived from X-ray, EUV, and bolometric observations

Context. In solar flares, energy is released impulsively and is partly converted into thermal energy of hot plasmas and kinetic energy of accelerated nonthermal particles. It is crucial to constrain the partition of these two energy components to understand energy release and transport as well as particle acceleration in solar flares. Despite numerous efforts, no consensus on quantifying this energy balance has yet been reached. Aims. We aim to understand the reasons for the contradicting results on energy partition obtained by various recent studies. The overarching question we address is whether there is sufficient energy in nonthermal particles to account for the thermal flare component. Methods. We considered five recent studies that address the thermal-nonthermal energy partition in solar flares. Their results are reviewed, and their methods are compared and discussed in detail. Results. The main uncertainties in deriving the energy partition are identified as (a) the derivation of the differential emission measure distribution and (b) the role of the conductive energy loss for the thermal component, as well as (c) the determination of the low-energy cutoff for the injected electrons. The bolometric radiated energy, as a proxy for the total energy released in the flare, is a useful independent constraint on both thermal and nonthermal energetics. In most of the cases, the derived energetics are consistent with this constraint. There are indications that the thermal-nonthermal energy partition changes with flare strength: in weak flares, there appears to be a deficit of energetic electrons, while the injected nonthermal energy is sufficient to account for the thermal component in strong flares. This behavior is identified as the main cause of the dissimilar results in the studies we considered. The changing partition has two important consequences: (a) an additional direct (i.e. non-beam) heating mechanism has to be present, and (b) considering that the bolometric emission originates mainly from deeper atmospheric layers, conduction or waves are required as additional energy transport mechanisms.
Geometric Structure Aided Visual Inertial Localization

Visual Localization is an essential component in autonomous navigation. Existing approaches are either based on the visual structure from SLAM/SfM or the geometric structure from dense mapping. To take the advantages of both, in this work, we present a complete visual inertial localization system based on a hybrid map representation to reduce the computational cost and increase the positioning accuracy. Specially, we propose two modules for data association and batch optimization, respectively. To this end, we develop an efficient data association module to associate map components with local features, which takes only $2$ms to generate temporal landmarks. For batch optimization, instead of using visual factors, we develop a module to estimate a pose prior from the instant localization results to constrain poses. The experimental results on the EuRoC MAV dataset demonstrate a competitive performance compared to the state of the arts. Specially, our system achieves an average position error in 1.7 cm with 100% recall. The timings show that the proposed modules reduce the computational cost by 20-30%. We will make our implementation open source at this http URL