OpenRaiser/Intern-Atlas huggingface dataset:
Three-Dimensional Dynamic Modeling and Motion Analysis for an Active-Tail-Actuated Robotic Fish with Barycentre Regulating Mechanism
Venue: arXiv cs.RO 2021
Year: 2021

Dynamic modeling has been capturing attention for its fundamentality in precise locomotion analyses and control of underwater robots. However, the existing researches have mainly focused on investigating two-dimensional motion of underwater robots, and little attention has been paid to three-dimensional dynamic modeling, which is just what we focus on. In this article, a three-dimensional dynamic model of an active-tail-actuated robotic fish with a barycentre regulating mechanism is built by combining Newton's second law for linear motion and Euler's equation for angular motion. The model parameters are determined by three-dimensional computer-aided design (CAD) software SolidWorks, HyperFlow-based computational fluid dynamics (CFD) simulation, and grey-box model estimation method. Both kinematic experiments with a prototype and numerical simulations are applied to validate the accuracy of the dynamic model mutually. Based on the dynamic model, multiple three-dimensional motions, including rectilinear motion, turning motion, gliding motion, and spiral motion, are analyzed. The experimental and simulation results demonstrate the effectiveness of the proposed model in evaluating the trajectory, attitude, and motion parameters, including the velocity, turning radius, angular velocity, etc., of the robotic fish.
DanHAR: Dual Attention Network For Multimodal Human Activity Recognition Using Wearable Sensors
Venue: arXiv cs.CV 2021
Year: 2021

Human activity recognition (HAR) in ubiquitous computing has been beginning to incorporate attention into the context of deep neural networks (DNNs), in which the rich sensing data from multimodal sensors such as accelerometer and gyroscope is used to infer human activities. Recently, two attention methods are proposed via combining with Gated Recurrent Units (GRU) and Long Short-Term Memory (LSTM) network, which can capture the dependencies of sensing signals in both spatial and temporal domains simultaneously. However, recurrent networks often have a weak feature representing power compared with convolutional neural networks (CNNs). On the other hand, two attention, i.e., hard attention and soft attention, are applied in temporal domains via combining with CNN, which pay more attention to the target activity from a long sequence. However, they can only tell where to focus and miss channel information, which plays an important role in deciding what to focus. As a result, they fail to address the spatial-temporal dependencies of multimodal sensing signals, compared with attention-based GRU or LSTM. In the paper, we propose a novel dual attention method called DanHAR, which introduces the framework of blending channel attention and temporal attention on a CNN, demonstrating superiority in improving the comprehensibility for multimodal HAR. Extensive experiments on four public HAR datasets and weakly labeled dataset show that DanHAR achieves state-of-the-art performance with negligible overhead of parameters. Furthermore, visualizing analysis is provided to show that our attention can amplifies more important sensor modalities and timesteps during classification, which agrees well with human common intuition.
Teamwise Mean Field Competitions
Venue: arXiv cs.GT 2021
Year: 2021

This paper studies competitions with rank-based reward among a large number of teams. Within each sizable team, we consider a mean-field contribution game in which each team member contributes to the jump intensity of a common Poisson project process; across all teams, a mean field competition game is formulated on the rank of the completion time, namely the jump time of Poisson project process, and the reward to each team is paid based on its ranking. On the layer of teamwise competition game, three optimization problems are introduced when the team size is determined by: (i) the team manager; (ii) the central planner; (iii) the team members' voting as partnership. We propose a relative performance criteria for each team member to share the team's reward and formulate some special cases of mean field games of mean field games, which are new to the literature. In all problems with homogeneous parameters, the equilibrium control of each worker and the equilibrium or optimal team size can be computed in an explicit manner, allowing us to analytically examine the impacts of some model parameters and discuss their economic implications. Two numerical examples are also presented to illustrate the parameter dependence and comparison between different team size decision making.
A metric on directed graphs and Markov chains based on hitting probabilities
Venue: arXiv cs.SI 2021
Year: 2021

The shortest-path, commute time, and diffusion distances on undirected graphs have been widely employed in applications such as dimensionality reduction, link prediction, and trip planning. Increasingly, there is interest in using asymmetric structure of data derived from Markov chains and directed graphs, but few metrics are specifically adapted to this task. We introduce a metric on the state space of any ergodic, finite-state, time-homogeneous Markov chain and, in particular, on any Markov chain derived from a directed graph. Our construction is based on hitting probabilities, with nearness in the metric space related to the transfer of random walkers from one node to another at stationarity. Notably, our metric is insensitive to shortest and average walk distances, thus giving new information compared to existing metrics. We use possible degeneracies in the metric to develop an interesting structural theory of directed graphs and explore a related quotienting procedure. Our metric can be computed in $O(n^3)$ time, where $n$ is the number of states, and in examples we scale up to $n=10,000$ nodes and $\approx 38M$ edges on a desktop computer. In several examples, we explore the nature of the metric, compare it to alternative methods, and demonstrate its utility for weak recovery of community structure in dense graphs, visualization, structure recovering, dynamics exploration, and multiscale cluster detection.
Learning compositional functions via multiplicative weight updates
Venue: arXiv cs.NE 2021
Year: 2021

Compositionality is a basic structural feature of both biological and artificial neural networks. Learning compositional functions via gradient descent incurs well known problems like vanishing and exploding gradients, making careful learning rate tuning essential for real-world applications. This paper proves that multiplicative weight updates satisfy a descent lemma tailored to compositional functions. Based on this lemma, we derive Madam -- a multiplicative version of the Adam optimiser -- and show that it can train state of the art neural network architectures without learning rate tuning. We further show that Madam is easily adapted to train natively compressed neural networks by representing their weights in a logarithmic number system. We conclude by drawing connections between multiplicative weight updates and recent findings about synapses in biology.
Lee-Yang zeros and the complexity of the ferromagnetic Ising model on bounded-degree graphs
Venue: arXiv cs.CC 2021
Year: 2021

We study the computational complexity of approximating the partition function of the ferromagnetic Ising model with the external field parameter $λ$ on the unit circle in the complex plane. Complex-valued parameters for the Ising model are relevant for quantum circuit computations and phase transitions in statistical physics, but have also been key in the recent deterministic approximation scheme for all $|λ|\neq 1$ by Liu, Sinclair, and Srivastava. Here, we focus on the unresolved complexity picture on the unit circle, and on the tantalising question of what happens around $λ=1$, where on one hand the classical algorithm of Jerrum and Sinclair gives a randomised approximation scheme on the real axis suggesting tractability, and on the other hand the presence of Lee-Yang zeros alludes to computational hardness. Our main result establishes a sharp computational transition at the point $λ=1$, and more generally on the entire unit circle. For an integer $Δ\geq 3$ and edge interaction parameter $b\in (0,1)$ we show #P-hardness for approximating the partition function on graphs of maximum degree $Δ$ on the arc of the unit circle where the Lee-Yang zeros are dense. This result contrasts with known approximation algorithms when $|λ|\neq 1$ or when $λ$ is in the complementary arc around $1$ of the unit circle. Our work thus gives a direct connection between the presence/absence of Lee-Yang zeros and the tractability of efficiently approximating the partition function on bounded-degree graphs.
ATOM: Robustifying Out-of-distribution Detection Using Outlier Mining
Venue: arXiv cs.LG 2021
Year: 2021

Detecting out-of-distribution (OOD) inputs is critical for safely deploying deep learning models in an open-world setting. However, existing OOD detection solutions can be brittle in the open world, facing various types of adversarial OOD inputs. While methods leveraging auxiliary OOD data have emerged, our analysis on illuminative examples reveals a key insight that the majority of auxiliary OOD examples may not meaningfully improve or even hurt the decision boundary of the OOD detector, which is also observed in empirical results on real data. In this paper, we provide a theoretically motivated method, Adversarial Training with informative Outlier Mining (ATOM), which improves the robustness of OOD detection. We show that, by mining informative auxiliary OOD data, one can significantly improve OOD detection performance, and somewhat surprisingly, generalize to unseen adversarial attacks. ATOM achieves state-of-the-art performance under a broad family of classic and adversarial OOD evaluation tasks. For example, on the CIFAR-10 in-distribution dataset, ATOM reduces the FPR (at TPR 95%) by up to 57.99% under adversarial OOD inputs, surpassing the previous best baseline by a large margin.
Understanding Gradient Clipping in Private SGD: A Geometric Perspective
Venue: arXiv cs.LG 2021
Year: 2021

Deep learning models are increasingly popular in many machine learning applications where the training data may contain sensitive information. To provide formal and rigorous privacy guarantee, many learning systems now incorporate differential privacy by training their models with (differentially) private SGD. A key step in each private SGD update is gradient clipping that shrinks the gradient of an individual example whenever its L2 norm exceeds some threshold. We first demonstrate how gradient clipping can prevent SGD from converging to stationary point. We then provide a theoretical analysis that fully quantifies the clipping bias on convergence with a disparity measure between the gradient distribution and a geometrically symmetric distribution. Our empirical evaluation further suggests that the gradient distributions along the trajectory of private SGD indeed exhibit symmetric structure that favors convergence. Together, our results provide an explanation why private SGD with gradient clipping remains effective in practice despite its potential clipping bias. Finally, we develop a new perturbation-based technique that can provably correct the clipping bias even for instances with highly asymmetric gradient distributions.
Parallel Weighted Model Counting with Tensor Networks
Venue: arXiv cs.DS 2021
Year: 2021

A promising new algebraic approach to weighted model counting makes use of tensor networks, following a reduction from weighted model counting to tensor-network contraction. Prior work has focused on analyzing the single-core performance of this approach, and demonstrated that it is an effective addition to the current portfolio of weighted-model-counting algorithms. In this work, we explore the impact of multi-core and GPU use on tensor-network contraction for weighted model counting. To leverage multiple cores, we implement a parallel portfolio of tree-decomposition solvers to find an order to contract tensors. To leverage a GPU, we use TensorFlow to perform the contractions. We compare the resulting weighted model counter on 1914 standard weighted model counting benchmarks and show that it significantly improves the virtual best solver.
Graph Convolutional Network for Recommendation with Low-pass Collaborative Filters
Venue: arXiv cs.LG 2021
Year: 2021

\textbf{G}raph \textbf{C}onvolutional \textbf{N}etwork (\textbf{GCN}) is widely used in graph data learning tasks such as recommendation. However, when facing a large graph, the graph convolution is very computationally expensive thus is simplified in all existing GCNs, yet is seriously impaired due to the oversimplification. To address this gap, we leverage the \textit{original graph convolution} in GCN and propose a \textbf{L}ow-pass \textbf{C}ollaborative \textbf{F}ilter (\textbf{LCF}) to make it applicable to the large graph. LCF is designed to remove the noise caused by exposure and quantization in the observed data, and it also reduces the complexity of graph convolution in an unscathed way. Experiments show that LCF improves the effectiveness and efficiency of graph convolution and our GCN outperforms existing GCNs significantly. Codes are available on \url{https://github.com/Wenhui-Yu/LCFN}.
Energy-Efficient UAV Relaying Robust Resource Allocation in Uncertain Adversarial Networks
Venue: arXiv cs.IT 2021
Year: 2021

The mobile relaying technique is a critical enhancing technology in wireless communications due to a higher chance of supporting the remote user from the base station (BS) with better quality of service. This paper investigates energy-efficient (EE) mobile relaying networks, mounted on an unmanned aerial vehicle (UAV), while the unknown adversaries try to intercept the legitimate link. We aim to optimize robust transmit power both UAV and BS along, relay hovering path, speed, and acceleration. The BS sends legitimate information, which is forwarded to the user by the relay. This procedure is defined as information-causality-constraint (ICC). We jointly optimize the worst case secrecy rate (WCSR) and UAV propulsion energy consumption (PEC) for a finite time horizon. We construct the BS-UAV, the UAV-user, and the UAV-adversary channel models. We apply the UAV PEC considering UAV speed and acceleration. At last, we derive EE UAV relay-user maximization problem in the adversarial wireless networks. While the problem is non-convex, we propose an iterative and sub-optimal algorithm to optimize EE UAV relay with constraints, such as ICC, trajectory, speed, acceleration, and transmit power. First, we optimize both BS and UAV transmit power, and hovering speed for known UAV path planning and acceleration. Using the optimal transmit power and speed, we obtain the optimal trajectory and acceleration. We compare our algorithm with existing algorithms and demonstrate the improved EE UAV relaying communication for our model.
Leveraging Subword Embeddings for Multinational Address Parsing
Venue: arXiv cs.CL 2021
Year: 2021

Address parsing consists of identifying the segments that make up an address such as a street name or a postal code. Because of its importance for tasks like record linkage, address parsing has been approached with many techniques. Neural network methods defined a new state-of-the-art for address parsing. While this approach yielded notable results, previous work has only focused on applying neural networks to achieve address parsing of addresses from one source country. We propose an approach in which we employ subword embeddings and a Recurrent Neural Network architecture to build a single model capable of learning to parse addresses from multiple countries at the same time while taking into account the difference in languages and address formatting systems. We achieved accuracies around 99 % on the countries used for training with no pre-processing nor post-processing needed. We explore the possibility of transferring the address parsing knowledge obtained by training on some countries' addresses to others with no further training in a zero-shot transfer learning setting. We achieve good results for 80 % of the countries (33 out of 41), almost 50 % of which (20 out of 41) is near state-of-the-art performance. In addition, we propose an open-source Python implementation of our trained models.
On the Potential of Dynamic Substructuring Methods for Model Updating
Venue: arXiv cs.CE 2021
Year: 2021

While purely data-driven assessment is feasible for the first levels of the Structural Health Monitoring (SHM) process, namely damage detection and arguably damage localization, this does not hold true for more advanced processes. The tasks of damage quantification and eventually residual life prognosis are invariably linked to availability of a representation of the system, which bears physical connotation. In this context, it is often desirable to assimilate data and models, into what is often termed a digital twin of the monitored system. One common take to such an end lies in exploitation of structural mechanics models, relying on use of Finite Element approximations. proper updating of these models, and their incorporation in an inverse problem setting may allow for damage quantification and localization, as well as more advanced tasks, including reliability analysis and fatigue assessment. However, this may only be achieved by means of repetitive analyses of the forward model, which implies considerable computational toll, when the model used is a detailed FE representation. In tackling this issue, reduced order models can be adopted, which retain the parameterisation and link to the parameters regulating the physical properties, albeit greatly reducing the computational burden. In this work a detailed FE model of a wind turbine tower is considered, reduced forms of this model are found using both the Craig Bampton and Dual Craig Bampton methods. These reduced order models are then used and compared in a Transitional Markov Chain Monte Carlo procedure to localise and quantify damage which is introduced to the system.
Constructive Universal High-Dimensional Distribution Generation through Deep ReLU Networks
Venue: arXiv cs.LG 2021
Year: 2021

We present an explicit deep neural network construction that transforms uniformly distributed one-dimensional noise into an arbitrarily close approximation of any two-dimensional Lipschitz-continuous target distribution. The key ingredient of our design is a generalization of the "space-filling" property of sawtooth functions discovered in (Bailey & Telgarsky, 2018). We elicit the importance of depth - in our neural network construction - in driving the Wasserstein distance between the target distribution and the approximation realized by the network to zero. An extension to output distributions of arbitrary dimension is outlined. Finally, we show that the proposed construction does not incur a cost - in terms of error measured in Wasserstein-distance - relative to generating $d$-dimensional target distributions from $d$ independent random variables.
Tusas: A fully implicit parallel approach for coupled phase-field equations
Venue: arXiv cs.CE 2021
Year: 2021

We develop a fully-coupled, fully-implicit approach for phase-field modeling of solidification in metals and alloys. Predictive simulation of solidification in pure metals and metal alloys remains a significant challenge in the field of materials science, as microstructure formation during the solidification process plays a critical role in the properties and performance of the solid material. Our simulation approach consists of a finite element spatial discretization of the fully-coupled nonlinear system of partial differential equations at the microscale, which is treated implicitly in time with a preconditioned Jacobian-free Newton-Krylov method. The approach allows time steps larger than those restricted by the traditional explicit CFL limit and is algorithmically scalable as well as efficient due to an effective preconditioning strategy based on algebraic multigrid and block factorization. We implement this approach in the open-source Tusas framework, which is a general, flexible tool developed in C++ for solving coupled systems of nonlinear partial differential equations. The performance of our approach is analyzed in terms of algorithmic scalability and efficiency, while the computational performance of Tusas is presented in terms of parallel scalability and efficiency on emerging heterogeneous architectures. We demonstrate that modern algorithms, discretizations, and computational science, and heterogeneous hardware provide a robust route for predictive phase-field simulation of microstructure evolution during additive manufacturing.
Causality Learning: A New Perspective for Interpretable Machine Learning
Venue: arXiv cs.LG 2021
Year: 2021

Recent years have witnessed the rapid growth of machine learning in a wide range of fields such as image recognition, text classification, credit scoring prediction, recommendation system, etc. In spite of their great performance in different sectors, researchers still concern about the mechanism under any machine learning (ML) techniques that are inherently black-box and becoming more complex to achieve higher accuracy. Therefore, interpreting machine learning model is currently a mainstream topic in the research community. However, the traditional interpretable machine learning focuses on the association instead of the causality. This paper provides an overview of causal analysis with the fundamental background and key concepts, and then summarizes most recent causal approaches for interpretable machine learning. The evaluation techniques for assessing method quality, and open problems in causal interpretability are also discussed in this paper.
Thinness of product graphs
Venue: arXiv cs.DM 2021
Year: 2021

The thinness of a graph is a width parameter that generalizes some properties of interval graphs, which are exactly the graphs of thinness one. Many NP-complete problems can be solved in polynomial time for graphs with bounded thinness, given a suitable representation of the graph. In this paper we study the thinness and its variations of graph products. We show that the thinness behaves "well" in general for products, in the sense that for most of the graph products defined in the literature, the thinness of the product of two graphs is bounded by a function (typically product or sum) of their thinness, or of the thinness of one of them and the size of the other. We also show for some cases the non-existence of such a function.
MDP Homomorphic Networks: Group Symmetries in Reinforcement Learning
Venue: arXiv cs.LG 2021
Year: 2021

This paper introduces MDP homomorphic networks for deep reinforcement learning. MDP homomorphic networks are neural networks that are equivariant under symmetries in the joint state-action space of an MDP. Current approaches to deep reinforcement learning do not usually exploit knowledge about such structure. By building this prior knowledge into policy and value networks using an equivariance constraint, we can reduce the size of the solution space. We specifically focus on group-structured symmetries (invertible transformations). Additionally, we introduce an easy method for constructing equivariant network layers numerically, so the system designer need not solve the constraints by hand, as is typically done. We construct MDP homomorphic MLPs and CNNs that are equivariant under either a group of reflections or rotations. We show that such networks converge faster than unstructured baselines on CartPole, a grid world and Pong.