Miracle-dz/UniFAR-FTUs huggingface dataset:
Coverage Performance of Aerial-Terrestrial HetNets

Providing seamless coverage in current cellular network technologies is surmountable only through gross over-engineering. Alternatively, as an economically effective solution, the use of unmanned aerial vehicles (UAVs), augmented with the functionalities of terrestrial base stations (BSs), is recently advocated. In this paper we investigate the effect that the incorporation of UAV-mounted BSs (U-BS) poses on the coverage probability of cellular networks. To this end, we focus on the evaluation of the coverage probability of a large-scale aerial-terrestrial heterogenous cellular network (AT-HetNet), in which BSs of each technology/tier can be either ground (G-BS) or U-BS. Our analysis incorporates the impact of Line-of-Sight (LOS) and non-LOS (NLOS) path-loss attenuations of both ground-to-ground (G2G) and Air-to-Ground (A2G) links. Adopting tools of stochastic geometry we then obtain the coverage probability as a function of main system parameters and percentage of BSs in each tier that are aerial. Using simulations we also confirm the accuracy of our analysis. We further observe that for several common communication environments, e.g., high-rise and dense urban environments, the inclusion of U-BSs can be detrimental to the coverage probability. Nevertheless, it is still possible to minimize the coverage cost by turning off a percentage of G-BSs. Interestingly, for urban and sub-urban areas one can adjust the altitude of U-BSs in order to adjust the coverage probability.
An FPT Algorithm for Minimum Additive Spanner Problem

For a positive integer $t$ and a graph $G$, an additive $t$-spanner of $G$ is a spanning subgraph in which the distance between every pair of vertices is at most the original distance plus $t$. Minimum Additive $t$-Spanner Problem is to find an additive $t$-spanner with the minimum number of edges in a given graph, which is known to be NP-hard. Since we need to care about global properties of graphs when we deal with additive $t$-spanners, Minimum Additive $t$-Spanner Problem is hard to handle, and hence only few results are known for it. In this paper, we study Minimum Additive $t$-Spanner Problem from the viewpoint of parameterized complexity. We formulate a parameterized version of the problem in which the number of removed edges is regarded as a parameter, and give a fixed-parameter algorithm for it. We also extend our result to $(\alpha, \beta)$-spanners.
Weakly Supervised Bilinear Attention Network for Fine-Grained Visual Classification

For fine-grained visual classification, objects usually share similar geometric structure but present variant local appearance and different pose. Therefore, localizing and extracting discriminative local features play a crucial role in accurate category prediction. Existing works either pay attention to limited object parts or train isolated networks for locating and classification. In this paper, we propose Weakly Supervised Bilinear Attention Network (WS-BAN) to solve these issues. It jointly generates a set of attention maps (region-of-interest maps) to indicate the locations of object's parts and extracts sequential part features by Bilinear Attention Pooling (BAP). Besides, we propose attention regularization and attention dropout to weakly supervise the generating process of attention maps. WS-BAN can be trained end-to-end and achieves the state-of-the-art performance on multiple fine-grained classification datasets, including CUB-200-2011, Stanford Car and FGVC-Aircraft, which demonstrated its effectiveness.
Recurrent MVSNet for High-Resolution Multi-View Stereo Depth Inference

Deep learning has recently demonstrated its excellent performance for multi-view stereo (MVS). However, one major limitation of current learned MVS approaches is the scalability: the memory-consuming cost volume regularization makes the learned MVS hard to be applied to high-resolution scenes. In this paper, we introduce a scalable multi-view stereo framework based on the recurrent neural network. Instead of regularizing the entire 3D cost volume in one go, the proposed Recurrent Multi-view Stereo Network (R-MVSNet) sequentially regularizes the 2D cost maps along the depth direction via the gated recurrent unit (GRU). This reduces dramatically the memory consumption and makes high-resolution reconstruction feasible. We first show the state-of-the-art performance achieved by the proposed R-MVSNet on the recent MVS benchmarks. Then, we further demonstrate the scalability of the proposed method on several large-scale scenarios, where previous learned approaches often fail due to the memory constraint. Code is available at https://github.com/YoYo000/MVSNet.
Adversarial Attack and Defense on Point Sets

Emergence of the utility of 3D point cloud data in safety-critical vision tasks (e.g., ADAS) urges researchers to pay more attention to the robustness of 3D representations and deep networks. To this end, we develop an attack and defense scheme, dedicated to 3D point cloud data, for preventing 3D point clouds from manipulated as well as pursuing noise-tolerable 3D representation. A set of novel 3D point cloud attack operations are proposed via pointwise gradient perturbation and adversarial point attachment / detachment. We then develop a flexible perturbation-measurement scheme for 3D point cloud data to detect potential attack data or noisy sensing data. Notably, the proposed defense methods are even effective to detect the adversarial point clouds generated by a proof-of-concept attack directly targeting the defense. Transferability of adversarial attacks between several point cloud networks is addressed, and we propose an momentum-enhanced pointwise gradient to improve the attack transferability. We further analyze the transferability from adversarial point clouds to grid CNNs and the inverse. Extensive experimental results on common point cloud benchmarks demonstrate the validity of the proposed 3D attack and defense framework.
NAS-Bench-101: Towards Reproducible Neural Architecture Search

Recent advances in neural architecture search (NAS) demand tremendous computational resources, which makes it difficult to reproduce experiments and imposes a barrier-to-entry to researchers without access to large-scale computation. We aim to ameliorate these problems by introducing NAS-Bench-101, the first public architecture dataset for NAS research. To build NAS-Bench-101, we carefully constructed a compact, yet expressive, search space, exploiting graph isomorphisms to identify 423k unique convolutional architectures. We trained and evaluated all of these architectures multiple times on CIFAR-10 and compiled the results into a large dataset of over 5 million trained models. This allows researchers to evaluate the quality of a diverse range of models in milliseconds by querying the pre-computed dataset. We demonstrate its utility by analyzing the dataset as a whole and by benchmarking a range of architecture optimization algorithms.
ANODE: Unconditionally Accurate Memory-Efficient Gradients for Neural ODEs

Residual neural networks can be viewed as the forward Euler discretization of an Ordinary Differential Equation (ODE) with a unit time step. This has recently motivated researchers to explore other discretization approaches and train ODE based networks. However, an important challenge of neural ODEs is their prohibitive memory cost during gradient backpropogation. Recently a method proposed in arXiv:1806.07366, claimed that this memory overhead can be reduced from LNt, where Nt is the number of time steps, down to O(L) by solving forward ODE backwards in time, where L is the depth of the network. However, we will show that this approach may lead to several problems: (i) it may be numerically unstable for ReLU/non-ReLU activations and general convolution operators, and (ii) the proposed optimize-then-discretize approach may lead to divergent training due to inconsistent gradients for small time step sizes. We discuss the underlying problems, and to address them we propose ANODE, a neural ODE framework which avoids the numerical instability related problems noted above. ANODE has a memory footprint of O(L) + O(Nt), with the same computational cost as reversing ODE solve. We furthermore, discuss a memory efficient algorithm which can further reduce this footprint with a tradeoff of additional computational cost. We show results on Cifar-10/100 datasets using ResNet and SqueezeNext neural networks.
Artificial Constraints and Lipschitz Hints for Unconstrained Online Learning

We provide algorithms that guarantee regret $R_T(u)\le \tilde O(G\|u\|^3 + G(\|u\|+1)\sqrt{T})$ or $R_T(u)\le \tilde O(G\|u\|^3T^{1/3} + GT^{1/3}+ G\|u\|\sqrt{T})$ for online convex optimization with $G$-Lipschitz losses for any comparison point $u$ without prior knowledge of either $G$ or $\|u\|$. Previous algorithms dispense with the $O(\|u\|^3)$ term at the expense of knowledge of one or both of these parameters, while a lower bound shows that some additional penalty term over $G\|u\|\sqrt{T}$ is necessary. Previous penalties were exponential while our bounds are polynomial in all quantities. Further, given a known bound $\|u\|\le D$, our same techniques allow us to design algorithms that adapt optimally to the unknown value of $\|u\|$ without requiring knowledge of $G$.
Graph Adversarial Training: Dynamically Regularizing Based on Graph Structure

Recent efforts show that neural networks are vulnerable to small but intentional perturbations on input features in visual classification tasks. Due to the additional consideration of connections between examples (e.g., articles with citation link tend to be in the same class), graph neural networks could be more sensitive to the perturbations, since the perturbations from connected examples exacerbate the impact on a target example. Adversarial Training (AT), a dynamic regularization technique, can resist the worst-case perturbations on input features and is a promising choice to improve model robustness and generalization. However, existing AT methods focus on standard classification, being less effective when training models on graph since it does not model the impact from connected examples. In this work, we explore adversarial training on graph, aiming to improve the robustness and generalization of models learned on graph. We propose Graph Adversarial Training (GraphAT), which takes the impact from connected examples into account when learning to construct and resist perturbations. We give a general formulation of GraphAT, which can be seen as a dynamic regularization scheme based on the graph structure. To demonstrate the utility of GraphAT, we employ it on a state-of-the-art graph neural network model — Graph Convolutional Network (GCN). We conduct experiments on two citation graphs (Citeseer and Cora) and a knowledge graph (NELL), verifying the effectiveness of GraphAT which outperforms normal training on GCN by 4.51 percent in node classification accuracy. Codes are available via: https://github.com/fulifeng/GraphAT.
PartNet: A Recursive Part Decomposition Network for Fine-Grained and Hierarchical Shape Segmentation

Deep learning approaches to 3D shape segmentation are typically formulated as a multi-class labeling problem. These models are trained for a fixed set of labels, which greatly limits their flexibility and adaptivity. We opt for top-down recursive decomposition and develop the first deep learning model for hierarchical segmentation of 3D shapes, based on recursive neural networks. Starting from a full shape represented as a point cloud, our model performs recursive binary decomposition, where the decomposition network at all nodes in the hierarchy share weights. At each node, a node classifier is trained to determine the type (adjacency or symmetry) and stopping criteria of its decomposition. The features extracted in higher level nodes are recursively propagated to lower level ones. Thus, the meaningful decompositions in higher levels provide strong contextual cues constraining the segmentations in lower levels. Meanwhile, to increase the segmentation accuracy at each node, we enhance the recursive contextual feature with the shape feature extracted for the corresponding part. Our method segments a 3D shape in point cloud into an arbitrary number of parts, depending on the shape complexity, showing strong generality and flexibility. It achieves the state-of-the-art performance, both for fine-grained and semantic segmentation, on the public benchmark and a new benchmark of fine-grained segmentation proposed in this work. We also demonstrate its application for fine-grained part refinements in image-to-shape reconstruction.
Nonlinear generalization of the single index model

Single index model is a powerful yet simple model, widely used in statistics, machine learning, and other scientific fields. It models the regression function as $g( )$, where a is an unknown index vector and x are the features. This paper deals with a nonlinear generalization of this framework to allow for a regressor that uses multiple index vectors, adapting to local changes in the responses. To do so we exploit the conditional distribution over function-driven partitions, and use linear regression to locally estimate index vectors. We then regress by applying a kNN type estimator that uses a localized proxy of the geodesic metric. We present theoretical guarantees for estimation of local index vectors and out-of-sample prediction, and demonstrate the performance of our method with experiments on synthetic and real-world data sets, comparing it with state-of-the-art methods.
Concentration and Esterification of Lactic Acid in Complex Coacervates

The viability of complex coacervates to concentrate lactic acid and further convert it into a valuable compound is evaluated. In the first step, the concentration of lactic acid, butanol, lipases and butyl lactate in complex coacervates is studied. The effect of various parameters such as complex composition, total polyelectrolyte concentration, ionic strength, and temperature on the distribution of the molecules between the dilute phase and complex coacervate phase were discussed. The components presented distribution coefficients greater than one, indicating that the molecules are being incorporated in the complex coacervates. The factor with the greatest influence on the enzymes and lactic acid incorporation was the ionic strength while for butanol distribution the temperature presented the more substantial effect. In the second step, the esterification of lactic acid with butanol was performed in the presence of an enzyme-filled complex coacervate. The results indicate that lactic acid can be converted to butyl lactate at a low yield (7-9%) at the conditions evaluated (50°C, 170 hours). These experiments represent the first approach to assess this possible application of complex coacervates. Several factors can be explored to improve the ester production, such as an increase in enzyme concentration, increase in temperature or concentration of reagents.
Provable Guarantees for Gradient-Based Meta-Learning

We study the problem of meta-learning through the lens of online convex optimization, developing a meta-algorithm bridging the gap between popular gradient-based meta-learning and classical regularization-based multi-task transfer methods. Our method is the first to simultaneously satisfy good sample efficiency guarantees in the convex setting, with generalization bounds that improve with task-similarity, while also being computationally scalable to modern deep learning architectures and the many-task setting. Despite its simplicity, the algorithm matches, up to a constant factor, a lower bound on the performance of any such parameter-transfer method under natural task similarity assumptions. We use experiments in both convex and deep learning settings to verify and demonstrate the applicability of our theory.
Dual Attention Networks for Visual Reference Resolution in Visual Dialog

Visual dialog (VisDial) is a task which requires a dialog agent to answer a series of questions grounded in an image. Unlike in visual question answering (VQA), the series of questions should be able to capture a temporal context from a dialog history and utilizes visually-grounded information. Visual reference resolution is a problem that addresses these challenges, requiring the agent to resolve ambiguous references in a given question and to find the references in a given image. In this paper, we propose Dual Attention Networks (DAN) for visual reference resolution in VisDial. DAN consists of two kinds of attention modules, REFER and FIND. Specifically, REFER module learns latent relationships between a given question and a dialog history by employing a multi-head attention mechanism. FIND module takes image features and reference-aware representations (i.e., the output of REFER module) as input, and performs visual grounding via bottom-up attention mechanism. We qualitatively and quantitatively evaluate our model on VisDial v1.0 and v0.9 datasets, showing that DAN outperforms the previous state-of-the-art model by a significant margin.
Semi-Generative Modelling: Covariate-Shift Adaptation with Cause and Effect Features

Current methods for covariate-shift adaptation use unlabelled data to compute importance weights or domain-invariant features, while the final model is trained on labelled data only. Here, we consider a particular case of covariate shift which allows us also to learn from unlabelled data, that is, combining adaptation and semi-supervised learning. Using ideas from causality, we argue that this requires learning with both causes, X_C, and effects, X_E, of a target variable, Y, and show how this setting leads to what we call a semi-generative model, P(Y,X_E|X_C,θ). Our approach is robust to domain shifts in the distribution of causal features and leverages unlabelled data by learning a direct map from causes to effects. Experiments on synthetic data demonstrate significant improvements in classification over purely-supervised and importance-weighting baselines.
Real-Time 3-D Shape Instantiation for Partially Deployed Stent Segments From a Single 2-D Fluoroscopic Image in Fenestrated Endovascular Aortic Repair

In fenestrated endovascular aortic repair (FEVAR), accurate alignment of stent graft fenestrations or scallops with aortic branches is essential for establishing complete blood flow perfusion. Current navigation is largely based on two-dimensional (2-D) fluoroscopic images, which lacks 3-D anatomical information, thus causing a longer operation time and high risks of radiation exposure. Previously, 3-D shape instantiation frameworks for real-time 3-D shape reconstruction of fully deployed or fully compressed stent grafts from a single 2-D fluoroscopic image have been proposed for 3-D navigation in FEVAR. However, these methods could not instantiate partially deployed stent segments, as the 3-D marker references are unknown. In this letter, an adapted graph convolutional network (GCN) is proposed to predict 3-D partially deployed marker references from 3-D fully deployed marker references. As the original GCN is for classification, in this letter, the coarsening layers are removed and the softmax function at the network end is replaced with linear mapping for regression. The derived 3-D marker references and the 2-D marker positions are used to instantiate the partially deployed stent segment, combined with the previous 3-D shape instantiation framework. Validations were performed on three typical stent grafts and five patient-specific 3-D printed aortic aneurysm phantoms. Reasonable performances with average mesh distance errors from 1.0 to 2.4 mm and average angular errors around $7.2^\circ$ were achieved.
Multiscale Fluctuation-Based Dispersion Entropy and Its Applications to Neurological Diseases

Fluctuation-based dispersion entropy (FDispEn) is a new approach to estimate the dynamical variability of the fluctuations of signals. It is based on Shannon entropy and fluctuation-based dispersion patterns. To quantify the physiological dynamics over multiple time scales, multiscale FDispEn (MFDE) is developed in this paper. MFDE is robust to the presence of baseline wanders or trends in the data. We evaluate MFDE, compared with popular multiscale sample entropy (MSE), multiscale fuzzy entropy (MFE), and the recently introduced multiscale dispersion entropy (MDE), on selected synthetic data and five neurological diseases’ datasets: 1) focal and non-focal electroencephalograms (EEGs); 2) walking stride interval signals for young, elderly, and Parkinson’s subjects; 3) stride interval fluctuations for Huntington’s disease and amyotrophic lateral sclerosis; 4) EEGs for controls and Alzheimer’s disease patients; and 5) eye movement data for Parkinson’s disease and ataxia. The MFDE avoids the problem of the undefined MSE values and, compared with the MFE and MSE, leads to more stable entropy values over the scale factors for white and pink noises. Overall, the MFDE is the fastest and most consistent method for the discrimination of different states of neurological data, especially where the mean value of a time series considerably changes along with the signal (e.g., eye movement data). This paper shows that MFDE is a relevant new metric to gain further insights into the dynamics of neurological diseases’ recordings. The MATLAB codes for the MFDE and its refined composite form are available in Xplore.
Perturbed-History Exploration in Stochastic Multi-Armed Bandits

We propose an online algorithm for cumulative regret minimization in a stochastic multi-armed bandit. The algorithm adds O(t) i.i. d. pseudo-rewards to its history in round t and then pulls the arm with the highest average reward in its perturbed history. Therefore, we call it perturbed-history exploration (PHE). The pseudo-rewards are carefully designed to offset potentially underestimated mean rewards of arms with a high probability. We derive near-optimal gap-dependent and gap-free bounds on the n-round regret of PHE. The key step in our analysis is a novel argument that shows that randomized Bernoulli rewards lead to optimism. Finally, we empirically evaluate PHE and show that it is competitive with state-of-the-art baselines.
Efficient Fully Convolution Neural Network for Generating Pixel Wise Robotic Grasps With High Resolution Images

This paper presents an efficient neural network model to generate robotic grasps with high resolution images. The proposed model uses fully convolution neural network to generate robotic grasps for each pixel using 400 × 400 high resolution RGB-D images. It first down-sample the images to get features and then up-sample those features to the original size of the input as well as combines local and global features from different feature maps. Compared to other regression or classification methods for detecting robotic grasps, our method looks more like the segmentation methods which solves the problem through pixel-wise ways. We use Cornell Grasp Dataset to train and evaluate the model and get high accuracy about 94.42% for image-wise and 91.02% for object-wise and fast prediction time about 8ms. We also demonstrate that without training on the multiple objects dataset, our model can directly output robotic grasps candidates for different objects because of the pixel wise implementation.