Miracle-dz/UniFAR-FTUs huggingface dataset:
Zeroing Neural Networks, an Introduction to, a Partial Survey of, and Predictive Computations for Discretized Time-varying Matrix Problems

This paper is designed to increase knowledge and understanding of time-varying matrix problems and Zeroing Neural Networks in the numerical analysis community of the West. Zeroing neural networks (ZNN) for time-varying matrix problems were invented 20 years ago in China and almost all of their advances have been made in and still come from its birthplace. These so-called ZNN methods have become a backbone for solving discretized sensor driven time-varying matrix problems in real-time, in theory and in on-chip applications for robots, in control theory and engineering in general. They have become the method of choice for many time-varying matrix problems that beneﬁt from or require efﬁcient, accurate and predictive real-time computations. The typical discretized ZNN algorithm needs seven distinct steps in its initial set-up. Construction of discretized ZNN algorithms starts from a model with an error equation and the stipulation that the error function decrease exponentially fast. The error function differential equation is then mated with a convergent look-ahead ﬁnite difference formula to create a derivative free computer code that predicts the future state of the system reliably from current and earlier state data. Matlab codes for discretized ZNN algorithms typically consist of one linear equations solve and one short recursion of current and previous state data per time step. This makes discretized ZNN based algorithms highly competitive with ordinary differential equation initial value problem path following methods or homotopy methods that are designed to work adaptively. Discretized ZNN methods instead use incoming sensor data with constant sampling gaps or discretized functional equations. To illustrate the easy adaptability of discretized ZNN and further the understanding of ZNN, this paper details the seven step set-up of ZNN for 11 separate time-varying problems and supplies new codes for seven. Open problems are mentioned as well as detailed references to recent work on discretized ZNN and time-varying matrix problems.
Scheduling of Graph Queries: Controlling Intra- and Inter-query Parallelism for a High System Throughput

The vast amounts of data used in social, business or traffic networks, biology and other natural sciences are often managed in graph-based data sets, consisting of a few thousand up to billions and trillions of vertices and edges, respectively. Typical applications utilizing such data either execute one or a few complex queries or many small queries at the same time interactively or as batch jobs. Furthermore, graph processing is inherently complex, as data sets can substantially differ (scale free vs. constant degree), and algorithms exhibit diverse behavior (computational intensity, local or global, push- or pull-based). This work is concerned with multi-query execution by automatically controlling the degree of parallelization, with overall objectives including high system utilization, low synchronization cost, and highly efficient concurrent execution. The underlying concept is three-fold: (1) sampling is used to determine graph statistics, (2) parallelization constraints are derived from algorithm and system properties, and (3) suitable work packages are generated based on the previous two aspects. We evaluate the proposed concept using different algorithms on synthetic and real world data sets, with up to 16 concurrent sessions (queries). The results demonstrate a robust performance in spite of these various configurations, and in particular that the performance is always close to or even slightly ahead of the performance of manually optimized implementations. Furthermore, the similar performance to manually optimized implementations under extreme configurations, which require either a full parallelization (few large queries) or complete sequential execution (many small queries), shows that the proposed concept exhibits a particularly low overhead.
Decentralized Asynchronous Nonconvex Stochastic Optimization on Directed Graphs

In this article, we consider a decentralized stochastic optimization problem over a network of agents, modeled as a directed graph: Agents aim to asynchronously minimize the average of their individual losses (possibly nonconvex), each one having access only to a noisy estimate of the gradient of its own function. We propose an asynchronous distributed algorithm for such a class of problems. The algorithm combines stochastic gradients with tracking in an asynchronous push-sum framework and obtains a sublinear convergence rate, matching the rate of the centralized stochastic gradient descent applied to the nonconvex minimization. Our experiments on a nonconvex image classification task using a convolutional neural network validate the convergence of our proposed algorithm across a different number of nodes and graph connectivity percentages.
Continual Learning in Multilingual NMT via Language-Specific Embeddings

This paper proposes a technique for adding a new source or target language to an existing multilingual NMT model without re-training it on the initial set of languages. It consists in replacing the shared vocabulary with a small language-specific vocabulary and fine-tuning the new embeddings on the new language’s parallel data. Some additional language-specific components may be trained to improve performance (e.g., Transformer layers or adapter modules). Because the parameters of the original model are not modified, its performance on the initial languages does not degrade. We show on two sets of experiments (small-scale on TED Talks, and large-scale on ParaCrawl) that this approach performs as well or better as the more costly alternatives; and that it has excellent zero-shot performance: training on English-centric data is enough to translate between the new language and any of the initial languages.
Effect of venting range hood flow rate on size-resolved ultrafine particle concentrations from gas stove cooking

Abstract Cooking is the main source of ultrafine particles (UFP) in homes. This study investigated the effect of venting range hood flow rate on size-resolved UFP concentrations from gas stove cooking. The same cooking protocol was conducted 60 times using three venting range hoods operated at six flow rates in twin research houses. Size-resolved particle (10–420 nm) concentrations were monitored using a NanoScan scanning mobility particle sizer (SMPS) from 15 min before cooking to 3 h after the cooking had stopped. Cooking increased the background total UFP number concentrations to 1.3 × 103 particles/cm3 on average, with a mean exposure-relevant source strength of 1.8 × 1012 particles/min. Total particle peak reductions ranged from 25% at the lowest fan flow rate of 36 L/s to 98% at the highest rate of 146 L/s. During the operation of a venting range hood, particle removal by deposition was less significant compared to the increasing air exchange rate driven by exhaust ventilation. Exposure to total particles due to cooking varied from 0.9 to 5.8 × 104 particles/cm3·h, 3 h after cooking ended. Compared to the 36 L/s range hood, higher flow rates of 120 and 146 L/s reduced the first-hour post-cooking exposure by 76% and 85%, respectively. © 2018 Crown Copyright. Published with license by Taylor & Francis Group, LLC
Momentum Contrastive Autoencoder: Using Contrastive Learning for Latent Space Distribution Matching in WAE

Wasserstein autoencoder (WAE) shows that matching two distributions is equivalent to minimizing a simple autoencoder (AE) loss under the constraint that the latent space of this AE matches a pre-specified prior distribution. This latent space distribution matching is a core component of WAE, and a challenging task. In this paper, we propose to use the contrastive learning framework that has been shown to be effective for self-supervised representation learning, as a means to resolve this problem. We do so by exploiting the fact that contrastive learning objectives optimize the latent space distribution to be uniform over the unit hyper-sphere, which can be easily sampled from. We show that using the contrastive learning framework to optimize the WAE loss achieves faster convergence and more stable optimization compared with existing popular algorithms for WAE. This is also reflected in the FID scores on CelebA and CIFAR-10 datasets, and the realistic generated image quality on the CelebA-HQ dataset.
R4: A Framework for Route Representation and Route Recommendation

Route recommendation is significant in navigation service. Two major challenges for route recommendation are route representation and user representation. Different from items that can be identified by unique IDs in traditional recommendation, routes are combinations of links (i.e., a road segment and its following action like turning left) and the number of combinations could be close to infinite. Besides, the representation of a route changes under different scenarios. These facts result in severe sparsity of routes, which increases the difficulty of route representation. Moreover, link attribute deficiencies and errors affect preciseness of route representation. Because of the sparsity of routes, the interaction data between users and routes are also sparse. This makes it not easy to acquire user representation from historical user-item interactions as traditional recommendations do. To address these issues, we propose a novel learning framework R4. In R4, we design a sparse & dense network to obtain representations of routes. The sparse unit learns link ID embeddings and aggregates them to represent a route, which captures implicit route characteristics and subsequently alleviates problems caused by link attribute deficiencies and errors. The dense unit extracts implicit local features of routes from link attributes. For user representation, we utilize a series of historical navigation to extract user preference. R4 achieves remarkable performance in both offline and online experiments.
Joint Gaussian graphical model estimation: A survey

Graphs representing complex systems often share a partial underlying structure across domains while retaining individual features. Thus, identifying common structures can shed light on the underlying signal, for instance, when applied to scientific discovery or clinical diagnoses. Furthermore, growing evidence shows that the shared structure across domains boosts the estimation power of graphs, particularly for high‐dimensional data. However, building a joint estimator to extract the common structure may be more complicated than it seems, most often due to data heterogeneity across sources. This manuscript surveys recent work on statistical inference of joint Gaussian graphical models, identifying model structures that fit various data generation processes.
Robust Edge-Direct Visual Odometry based on CNN edge detection and Shi-Tomasi corner optimization

In this paper, we propose a robust edge-direct visual odometry (VO) based on CNN edge detection and Shi-Tomasi corner optimization. Four layers of pyramids were extracted from the image in the proposed method to reduce the motion error between frames. This solution used CNN edge detection and Shi-Tomasi corner optimization to extract information from the image. Then, the pose estimation is performed using the Levenberg-Marquardt (LM) algorithm and updating the keyframes. Our method was compared with the dense direct method, the improved direct method of Canny edge detection, and ORB-SLAM2 system on the RGB-D TUM benchmark. The experimental results indicate that our method achieves better robustness and accuracy.
More Efficient Exploration with Symbolic Priors on Action Sequence Equivalences

Incorporating prior knowledge in reinforcement learning algorithms is mainly an open question. Even when insights about the environment dynamics are available, reinforcement learning is traditionally used in a tabula rasa setting and must explore and learn everything from scratch. In this paper, we consider the problem of exploiting priors about action sequence equivalence: that is, when different sequences of actions produce the same effect. We propose a new local exploration strategy calibrated to minimize collisions and maximize new state visitations. We show that this strategy can be computed at little cost, by solving a convex optimization problem. By replacing the usual epsilon-greedy strategy in a DQN, we demonstrate its potential in several environments with various dynamic structures.
flip-hoisting: Exploiting Repeated Parameters in Discrete Probabilistic Programs

Many of today's probabilistic programming languages (PPLs) have brittle inference performance: the performance of the underlying inference algorithm is very sensitive to the precise way in which the probabilistic program is written. A standard way of addressing this challenge in traditional programming languages is via program optimizations, which seek to unburden the programmer from writing low-level performant code, freeing them to work at a higher-level of abstraction. The arsenal of applicable program optimizations for PPLs to choose from is scarce in comparison to traditional programs; few of today's PPLs offer significant forms of automated program optimization. In this work we develop a new family of program optimizations specific to discrete-valued knowledge compilation based PPLs. We identify a particular form of program structure unique to these PPLs that tangibly affects exact inference performance in these programs: redundant random variables -- variables with repeated parameters and inconsistent path conditions. We develop a new program analysis and associated optimization called flip-hoisting that identifies these redundancies and optimizes them into a single random variable. We show that flip-hoisting yields inference speedups of up to 60% on applications of probabilistic programs such as Bayesian networks and probabilistic verification.
Data splitting improves statistical performance in overparametrized regimes

While large training datasets generally offer improvement in model performance, the training process becomes computationally expensive and time consuming. Distributed learning is a common strategy to reduce the overall training time by exploiting multiple computing devices. Recently, it has been observed in the single machine setting that overparametrization is essential for benign overfitting in ridgeless regression in Hilbert spaces. We show that in this regime, data splitting has a regularizing effect, hence improving statistical performance and computational complexity at the same time. We further provide a unified framework that allows to analyze both the finite and infinite dimensional setting. We numerically demonstrate the effect of different model parameters.
Origin of giant valley splitting in silicon quantum wells induced by superlattice barriers

Enhancing valley splitting in SiGe heterostructures is a crucial task for developing silicon spin qubits. Complex SiGe heterostructures, sharing a common feature of four-monolayer (4ML) Ge layer next to the silicon quantum well (QW), have been computationally designed to have giant valley splitting approaching 9 meV. However, none of them has been fabricated may due to their complexity. Here, we remarkably simplify the original designed complex SiGe heterostructures by laying out the Si QW directly on the Ge substrate followed by capping a (Ge4Si4)n superlattice(SL) barrier with a small sacrifice on VS as it is reduced from a maximum of 8.7 meV to 5.2 meV. Even the smallest number of periods (n = 1) will also give a sizable VS of 1.6 meV, which is large enough for developing stable spin qubits. We also develop an effective Hamiltonian model to reveal the underlying microscopic physics of enhanced valley splitting by (Ge4Si4)n SL barriers. We find that the presence of the SL barrier will reduce the VS instead of enhancing it. Only the (Ge4Si4)n SL barriers with an extremely strong coupling with Si QW valley states provide a remarkable enhancement in VS. These findings lay a solid theoretical foundation for the realization of sufficiently large VS for Si qubits.
Dual Encoding U-Net for Spatio-Temporal Domain Shift Frame Prediction

The landscape of city-wide mobility behaviour has altered significantly over the past 18 months. The ability to make accurate and reliable predictions on such behaviour has likewise changed drastically with COVID-19 measures impacting how populations across the world interact with the different facets of mobility. This raises the question: "How does one use an abundance of pre-covid mobility data to make predictions on future behaviour in a present/post-covid environment? " This paper seeks to address this question by introducing an approach for traffic frame prediction using a lightweight Dual-Encoding U-Net built using only 12 Convolutional layers that incorporates a novel approach to skip-connections between Convolutional LSTM layers. This approach combined with an intuitive handling of training data can model both a temporal and spatio-temporal domain shift (gitlab.com/alchera/alchera-traffic4cast-2021).
Ideal mixing of paraelectric and ferroelectric nematic phases in liquid crystals of distinct molecular species

ABSTRACT The organic mesogens RM734 and DIO are members of separate molecular families featuring distinct molecular structures. These families are the first ones known to exhibit a ferroelectric nematic liquid crystal phase. Here, we present an experimental investigation of the phase diagram and electro-optics of binary mixtures of RM734 and DIO. We observe paraelectric nematic and ferroelectric nematic phases in both materials, each of which exhibits complete miscibility across the phase diagram, showing that the paraelectric and ferroelectric are the same phases in RM734 and DIO. Remarkably, these molecules form ideal mixtures with respect to both the paraelectric–ferroelectric nematic phase behaviour and the ferroelectric polarisation density of the mixtures, the principal order parameter of the transition. Ideal mixing is also manifested in the orientational viscosity, and the onset of glassy dynamics at low temperature. This behaviour is attributable in part to the similarity of their overall molecular shape and net longitudinal dipole moment, and to a common tendency for head-to-tail molecular association. In contrast, the significant difference in molecular structures leads to poor solubility in the crystal phases, enhancing the stability of the ferroelectric nematic phase at low temperature in the mixtures and enabling room-temperature electro-optic effects. GRAPHICAL ABSTRACT
Bristle: Decentralized Federated Learning in Byzantine, Non-i.i.d. Environments

Federated learning (FL) is a privacy-friendly type of machine learning where devices locally train a model on their private data and typically communicate model updates with a server. In decentralized FL (DFL), peers communicate model updates with each other instead. However, DFL is challenging since (1) the training data possessed by different peers is often non-i.i. d. (i.e., distributed differently between the peers) and (2) malicious, or Byzantine, attackers can share arbitrary model updates with other peers to subvert the training process. We address these two challenges and present Bristle, middleware between the learning application and the decentralized network layer. Bristle leverages transfer learning to predetermine and freeze the non-output layers of a neural network, significantly speeding up model training and lowering communication costs. To securely update the output layer with model updates from other peers, we design a fast distance-based prioritizer and a novel performance-based integrator. Their combined effect results in high resilience to Byzantine attackers and the ability to handle non-i.i. d. classes. We empirically show that Bristle converges to a consistent 95% accuracy in Byzantine environments, outperforming all evaluated baselines. In non-Byzantine environments, Bristle requires 83% fewer iterations to achieve 90% accuracy compared to state-of-the-art methods. We show that when the training classes are non-i.i. d. , Bristle significantly outperforms the accuracy of the most Byzantine-resilient baselines by 2.3x while reducing communication costs by 90%.
Resilient time‐varying formation tracking for mobile robot networks under deception attacks on positioning

This paper investigates the resilient control, analysis, recovery, and operation of mobile robot networks in time‐varying formation tracking under deception attacks on global positioning. Local and global tracking control algorithms are presented to ensure redundancy of the mobile robot network and to retain the desired functionality for better resilience. Lyapunov stability analysis is utilized to show the boundedness of the formation tracking error and the stability of the network under various attack modes. A performance index is designed to compare the efficiency of the proposed formation tracking algorithms in situations with or without positioning attacks. Subsequently, a communication‐free decentralized cooperative localization approach based on extended information filters is presented for positioning estimate recovery where the identification of positioning attacks is based on Kullback–Leibler divergence. A gain‐tuning resilient operation is proposed to strategically synthesize formation control and cooperative localization for accurate and rapid system recovery from positioning attacks. The proposed methods are tested using both numerical simulation and experimental validation with a team of quadrotors.
Combining Different V1 Brain Model Variants to Improve Robustness to Image Corruptions in CNNs

While some convolutional neural networks (CNNs) have surpassed human visual abilities in object classification, they often struggle to recognize objects in images corrupted with different types of common noise patterns, highlighting a major limitation of this family of models. Recently, it has been shown that simulating a primary visual cortex (V1) at the front of CNNs leads to small improvements in robustness to these image perturbations. In this study, we start with the observation that different variants of the V1 model show gains for specific corruption types. We then build a new model using an ensembling technique, which combines multiple individual models with different V1 front-end variants. The model ensemble leverages the strengths of each individual model, leading to significant improvements in robustness across all corruption categories and outperforming the base model by 38% on average. Finally, we show that using distillation, it is possible to partially compress the knowledge in the ensemble model into a single model with a V1 front-end. While the ensembling and distillation techniques used here are hardly biologically-plausible, the results presented here demonstrate that by combining the specific strengths of different neuronal circuits in V1 it is possible to improve the robustness of CNNs for a wide range of perturbations.