OpenRaiser/Intern-Atlas huggingface dataset:
Predicting Attributes of Nodes Using Network Structure
Venue: arXiv cs.LG 2021
Year: 2021

In many graphs such as social networks, nodes have associated attributes representing their behavior. Predicting node attributes in such graphs is an important problem with applications in many domains like recommendation systems, privacy preservation, and targeted advertisement. Attributes values can be predicted by analyzing patterns and correlations among attributes and employing classification/regression algorithms. However, these approaches do not utilize readily available network topology information. In this regard, interconnections between different attributes of nodes can be exploited to improve the prediction accuracy. In this paper, we propose an approach to represent a node by a feature map with respect to an attribute $a_i$ (which is used as input for machine learning algorithms) using all attributes of neighbors to predict attributes values for $a_i$. We perform extensive experimentation on ten real-world datasets and show that the proposed feature map significantly improves the prediction accuracy as compared to baseline approaches on these datasets.
Federated Variance-Reduced Stochastic Gradient Descent with Robustness to Byzantine Attacks
Venue: arXiv cs.LG 2021
Year: 2021

This paper deals with distributed finite-sum optimization for learning over networks in the presence of malicious Byzantine attacks. To cope with such attacks, most resilient approaches so far combine stochastic gradient descent (SGD) with different robust aggregation rules. However, the sizeable SGD-induced stochastic gradient noise makes it challenging to distinguish malicious messages sent by the Byzantine attackers from noisy stochastic gradients sent by the 'honest' workers. This motivates us to reduce the variance of stochastic gradients as a means of robustifying SGD in the presence of Byzantine attacks. To this end, the present work puts forth a Byzantine attack resilient distributed (Byrd-) SAGA approach for learning tasks involving finite-sum optimization over networks. Rather than the mean employed by distributed SAGA, the novel Byrd- SAGA relies on the geometric median to aggregate the corrected stochastic gradients sent by the workers. When less than half of the workers are Byzantine attackers, the robustness of geometric median to outliers enables Byrd-SAGA to attain provably linear convergence to a neighborhood of the optimal solution, with the asymptotic learning error determined by the number of Byzantine workers. Numerical tests corroborate the robustness to various Byzantine attacks, as well as the merits of Byrd- SAGA over Byzantine attack resilient distributed SGD.
An integer programming approach for solving a generalized version of the Grundy domination number
Venue: arXiv cs.DM 2021
Year: 2021

A sequence of vertices in a graph is called a legal dominating sequence if every vertex in the sequence dominates at least one vertex not dominated by those that precede it, and at the end all vertices of the graph are dominated. The Grundy domination number of a graph is the size of a largest legal dominating sequence. In this work, we introduce a generalized version of the Grundy domination problem. We explicitly calculate the corresponding parameter for paths and web graphs. We propose integer programming formulations for the new problem, find families of valid inequalities and perform extensive computational experiments to compare the formulations as well as to test these inequalities as cuts in a branch-and-cut framework. We also design and evaluate the performance of a heuristic for finding good initial lower and upper bounds and a tabu search that improves the initial lower bound. The test instances include randomly generated graphs, structured graphs, classical benchmark instances and two instances from a real application. Our approach is exact for graphs with 20-50 vertices and provides good solutions for graphs up to 10000 vertices.
Stacked DeBERT: All Attention in Incomplete Data for Text Classification
Venue: arXiv cs.CL 2021
Year: 2021

In this paper, we propose Stacked DeBERT, short for Stacked Denoising Bidirectional Encoder Representations from Transformers. This novel model improves robustness in incomplete data, when compared to existing systems, by designing a novel encoding scheme in BERT, a powerful language representation model solely based on attention mechanisms. Incomplete data in natural language processing refer to text with missing or incorrect words, and its presence can hinder the performance of current models that were not implemented to withstand such noises, but must still perform well even under duress. This is due to the fact that current approaches are built for and trained with clean and complete data, and thus are not able to extract features that can adequately represent incomplete data. Our proposed approach consists of obtaining intermediate input representations by applying an embedding layer to the input tokens followed by vanilla transformers. These intermediate features are given as input to novel denoising transformers which are responsible for obtaining richer input representations. The proposed approach takes advantage of stacks of multilayer perceptrons for the reconstruction of missing words' embeddings by extracting more abstract and meaningful hidden feature vectors, and bidirectional transformers for improved embedding representation. We consider two datasets for training and evaluation: the Chatbot Natural Language Understanding Evaluation Corpus and Kaggle's Twitter Sentiment Corpus. Our model shows improved F1-scores and better robustness in informal/incorrect texts present in tweets and in texts with Speech-to-Text error in the sentiment and intent classification tasks.
RBF-FD analysis of 2D time-domain acoustic wave propagation in heterogeneous media
Venue: arXiv cs.CE 2021
Year: 2021

Radial Basis Function-generated Finite Differences (RBF-FD) is a popular variant of local strong-form meshless methods that do not require a predefined connection between the nodes, making it easier to adapt node-distribution to the problem under consideration. This paper investigates an RBF-FD solution of time-domain acoustic wave propagation in the context of seismic modeling in the Earth's subsurface. Through a number of numerical tests, ranging from homogeneous to highly-heterogeneous velocity models including non-smooth irregular topography, we demonstrate that the present approach can be further generalized to solve large-scale seismic modeling and full waveform inversion problems in arbitrarily complex models enabling more robust interpretations of geophysical observations
Trained Trajectory based Automated Parking System using Visual SLAM on Surround View Cameras
Venue: arXiv cs.CV 2021
Year: 2021

Automated Parking is becoming a standard feature in modern vehicles. Existing parking systems build a local map to be able to plan for maneuvering towards a detected slot. Next generation parking systems have an use case where they build a persistent map of the environment where the car is frequently parked, say for example, home parking or office parking. The pre-built map helps in re-localizing the vehicle better when its trying to park the next time. This is achieved by augmenting the parking system with a Visual SLAM pipeline and the feature is called trained trajectory parking in the automotive industry. In this paper, we discuss the use cases, design and implementation of a trained trajectory automated parking system. The proposed system is deployed on commercial vehicles and the consumer application is illustrated in \url{https://youtu.be/nRWF5KhyJZU}. The focus of this paper is on the application and the details of vision algorithms are kept at high level.
Induced odd cycle packing number, independent sets, and chromatic number
Venue: arXiv cs.DM 2021
Year: 2021

The induced odd cycle packing number $iocp(G)$ of a graph $G$ is the maximum integer $k$ such that $G$ contains an induced subgraph consisting of $k$ pairwise vertex-disjoint odd cycles. Motivated by applications to geometric graphs, Bonamy et al.~\cite{indoc} proved that graphs of bounded induced odd cycle packing number, bounded VC dimension, and linear independence number admit a randomized EPTAS for the independence number. We show that the assumption of bounded VC dimension is not necessary, exhibiting a randomized algorithm that for any integers $k\ge 0$ and $t\ge 1$ and any $n$-vertex graph $G$ of induced odd cycle packing number at most $k$ returns in time $O_{k,t}(n^{k+4})$ an independent set of $G$ whose size is at least $α(G)-n/t$ with high probability. In addition, we present $χ$-boundedness results for graphs with bounded odd cycle packing number, and use them to design a QPTAS for the independence number only assuming bounded induced odd cycle packing number.
Positive algorithmic bias cannot stop fragmentation in homophilic networks
Venue: arXiv cs.SI 2021
Year: 2021

Fragmentation, echo chambers, and their amelioration in social networks have been a growing concern in the academic and non-academic world. This paper shows how, under the assumption of homophily, echo chambers and fragmentation are system-immanent phenomena of highly flexible social networks, even under ideal conditions for heterogeneity. We achieve this by finding an analytical, network-based solution to the Schelling model and by proving that weak ties do not hinder the process. Furthermore, we derive that no level of positive algorithmic bias in the form of rewiring is capable of preventing fragmentation and its effect on reducing the fragmentation speed is negligible.
Probabilistic Reasoning across the Causal Hierarchy
Venue: arXiv cs.LO 2021
Year: 2021

We propose a formalization of the three-tier causal hierarchy of association, intervention, and counterfactuals as a series of probabilistic logical languages. Our languages are of strictly increasing expressivity, the first capable of expressing quantitative probabilistic reasoning -- including conditional independence and Bayesian inference -- the second encoding do-calculus reasoning for causal effects, and the third capturing a fully expressive do-calculus for arbitrary counterfactual queries. We give a corresponding series of finitary axiomatizations complete over both structural causal models and probabilistic programs, and show that satisfiability and validity for each language are decidable in polynomial space.
Trajectron++: Dynamically-Feasible Trajectory Forecasting With Heterogeneous Data
Venue: arXiv cs.RO 2021
Year: 2021

Reasoning about human motion is an important prerequisite to safe and socially-aware robotic navigation. As a result, multi-agent behavior prediction has become a core component of modern human-robot interactive systems, such as self-driving cars. While there exist many methods for trajectory forecasting, most do not enforce dynamic constraints and do not account for environmental information (e.g., maps). Towards this end, we present Trajectron++, a modular, graph-structured recurrent model that forecasts the trajectories of a general number of diverse agents while incorporating agent dynamics and heterogeneous data (e.g., semantic maps). Trajectron++ is designed to be tightly integrated with robotic planning and control frameworks; for example, it can produce predictions that are optionally conditioned on ego-agent motion plans. We demonstrate its performance on several challenging real-world trajectory forecasting datasets, outperforming a wide array of state-of-the-art deterministic and generative methods.
Compression of descriptor models for mobile applications
Venue: arXiv cs.CV 2021
Year: 2021

Deep neural networks have demonstrated state-of-the-art performance for feature-based image matching through the advent of new large and diverse datasets. However, there has been little work on evaluating the computational cost, model size, and matching accuracy tradeoffs for these models. This paper explicitly addresses these practical metrics by considering the state-of-the-art HardNet model. We observe a significant redundancy in the learned weights, which we exploit through the use of depthwise separable layers and an efficient Tucker decomposition. We demonstrate that a combination of these methods is very effective, but still sacrifices the top-end accuracy. To resolve this, we propose the Convolution-Depthwise-Pointwise(CDP) layer, which provides a means of interpolating between the standard and depthwise separable convolutions. With this proposed layer, we can achieve an 8 times reduction in the number of parameters on the HardNet model, 13 times reduction in the computational complexity, while sacrificing less than 1% on the overall accuracy across theHPatchesbenchmarks. To further demonstrate the generalisation of this approach, we apply it to the state-of-the-art SuperPoint model, where we can significantly reduce the number of parameters and floating-point operations, with minimal degradation in the matching accuracy.
A storage expansion planning framework using reinforcement learning and simulation-based optimization
Venue: arXiv cs.LG 2021
Year: 2021

In the wake of the highly electrified future ahead of us, the role of energy storage is crucial wherever distributed generation is abundant, such as in microgrid settings. Given the variety of storage options that are becoming more and more economical, determining which type of storage technology to invest in, along with the appropriate timing and capacity becomes a critical research question. It is inevitable that these problems will continue to become increasingly relevant in the future and require strategic planning and holistic and modern frameworks in order to be solved. Reinforcement Learning algorithms have already proven to be successful in problems where sequential decision-making is inherent. In the operations planning area, these algorithms are already used but mostly in short-term problems with well-defined constraints. On the contrary, we expand and tailor these techniques to long-term planning by utilizing model-free algorithms combined with simulation-based models. A model and expansion plan have been developed to optimally determine microgrid designs as they evolve to dynamically react to changing conditions and to exploit energy storage capabilities. We show that it is possible to derive better engineering solutions that would point to the types of energy storage units which could be at the core of future microgrid applications. Another key finding is that the optimal storage capacity threshold for a system depends heavily on the price movements of the available storage units. By utilizing the proposed approaches, it is possible to model inherent problem uncertainties and optimize the whole streamline of sequential investment decision-making.
Classification of Traffic Using Neural Networks by Rejecting: a Novel Approach in Classifying VPN Traffic
Venue: arXiv cs.NI 2021
Year: 2021

In this paper, we introduce a novel end-to-end traffic classification method to distinguish between traffic classes including VPN traffic in three layers of the Open Systems Interconnection (OSI) model. Classification of VPN traffic is not trivial using traditional classification approaches due to its encrypted nature. We utilize two well-known neural networks, namely multi-layer perceptron and recurrent neural network to create our cascade neural network focused on two metrics: class scores and distance from the center of the classes. Such approach combines extraction, selection, and classification functionality into a single end-to-end system to systematically learn the non-linear relationship between input and predicted performance. Therefore, we could distinguish VPN traffics from non-VPN traffics by rejecting the unrelated features of the VPN class. Moreover, we obtain the application type of non-VPN traffics at the same time. The approach is evaluated using the general traffic dataset ISCX VPN-nonVPN, and an acquired dataset. The results demonstrate the efficacy of the framework approach for encrypting traffic classification while also achieving extreme accuracy, $95$ percent, which is higher than the accuracy of the state-of-the-art models, and strong generalization capabilities.
RAN Slicing for Massive IoT and Bursty URLLC Service Multiplexing: Analysis and Optimization
Venue: arXiv cs.NI 2021
Year: 2021

Future wireless networks are envisioned to serve massive Internet of things (mIoT) via some radio access technologies, where the random access channel (RACH) procedure should be exploited for IoT devices to access the networks. However, the theoretical analysis of the RACH procedure for massive IoT devices is challenging. To address this challenge, we first correlate the RACH request of an IoT device with the status of its maintained queue and analyze the evolution of the queue status. Based on the analysis result, we then derive the closed-form expression of the random access (RA) success probability, which is a significant indicator characterizing the RACH procedure of the device. Besides, considering the agreement on converging different services onto a shared infrastructure, we investigate the RAN slicing for mIoT and bursty ultra-reliable and low latency communications (URLLC) service multiplexing. Specifically, we formulate the RAN slicing problem as an optimization one to maximize the total RA success probabilities of all IoT devices and provide URLLC services for URLLC devices in an energy-efficient way. A slice resource optimization (SRO) algorithm exploiting relaxation and approximation with provable tightness and error bound is then proposed to mitigate the optimization problem. Simulation results demonstrate that the proposed SRO algorithm can effectively implement the service multiplexing of mIoT and bursty URLLC traffic.
Self-supervising Action Recognition by Statistical Moment and Subspace Descriptors
Venue: arXiv cs.CV 2021
Year: 2021

In this paper, we build on a concept of self-supervision by taking RGB frames as input to learn to predict both action concepts and auxiliary descriptors e.g., object descriptors. So-called hallucination streams are trained to predict auxiliary cues, simultaneously fed into classification layers, and then hallucinated at the testing stage to aid network. We design and hallucinate two descriptors, one leveraging four popular object detectors applied to training videos, and the other leveraging image- and video-level saliency detectors. The first descriptor encodes the detector- and ImageNet-wise class prediction scores, confidence scores, and spatial locations of bounding boxes and frame indexes to capture the spatio-temporal distribution of features per video. Another descriptor encodes spatio-angular gradient distributions of saliency maps and intensity patterns. Inspired by the characteristic function of the probability distribution, we capture four statistical moments on the above intermediate descriptors. As numbers of coefficients in the mean, covariance, coskewness and cokurtotsis grow linearly, quadratically, cubically and quartically w.r.t. the dimension of feature vectors, we describe the covariance matrix by its leading n' eigenvectors (so-called subspace) and we capture skewness/kurtosis rather than costly coskewness/cokurtosis. We obtain state of the art on five popular datasets such as Charades and EPIC-Kitchens.
Structured Consistency Loss for semi-supervised semantic segmentation
Venue: arXiv cs.CV 2021
Year: 2021

The consistency loss has played a key role in solving problems in recent studies on semi-supervised learning. Yet extant studies with the consistency loss are limited to its application to classification tasks; extant studies on semi-supervised semantic segmentation rely on pixel-wise classification, which does not reflect the structured nature of characteristics in prediction. We propose a structured consistency loss to address this limitation of extant studies. Structured consistency loss promotes consistency in inter-pixel similarity between teacher and student networks. Specifically, collaboration with CutMix optimizes the efficient performance of semi-supervised semantic segmentation with structured consistency loss by reducing computational burden dramatically. The superiority of proposed method is verified with the Cityscapes; The Cityscapes benchmark results with validation and with test data are 81.9 mIoU and 83.84 mIoU respectively. This ranks the first place on the pixel-level semantic labeling task of Cityscapes benchmark suite. To the best of our knowledge, we are the first to present the superiority of state-of-the-art semi-supervised learning in semantic segmentation.
Hydra: Preserving Ensemble Diversity for Model Distillation
Venue: arXiv cs.LG 2021
Year: 2021

Ensembles of models have been empirically shown to improve predictive performance and to yield robust measures of uncertainty. However, they are expensive in computation and memory. Therefore, recent research has focused on distilling ensembles into a single compact model, reducing the computational and memory burden of the ensemble while trying to preserve its predictive behavior. Most existing distillation formulations summarize the ensemble by capturing its average predictions. As a result, the diversity of the ensemble predictions, stemming from each member, is lost. Thus, the distilled model cannot provide a measure of uncertainty comparable to that of the original ensemble. To retain more faithfully the diversity of the ensemble, we propose a distillation method based on a single multi-headed neural network, which we refer to as Hydra. The shared body network learns a joint feature representation that enables each head to capture the predictive behavior of each ensemble member. We demonstrate that with a slight increase in parameter count, Hydra improves distillation performance on classification and regression settings while capturing the uncertainty behavior of the original ensemble over both in-domain and out-of-distribution tasks.
Neural Human Video Rendering by Learning Dynamic Textures and Rendering-to-Video Translation
Venue: arXiv cs.GR 2021
Year: 2021

Synthesizing realistic videos of humans using neural networks has been a popular alternative to the conventional graphics-based rendering pipeline due to its high efficiency. Existing works typically formulate this as an image-to-image translation problem in 2D screen space, which leads to artifacts such as over-smoothing, missing body parts, and temporal instability of fine-scale detail, such as pose-dependent wrinkles in the clothing. In this paper, we propose a novel human video synthesis method that approaches these limiting factors by explicitly disentangling the learning of time-coherent fine-scale details from the embedding of the human in 2D screen space. More specifically, our method relies on the combination of two convolutional neural networks (CNNs). Given the pose information, the first CNN predicts a dynamic texture map that contains time-coherent high-frequency details, and the second CNN conditions the generation of the final video on the temporally coherent output of the first CNN. We demonstrate several applications of our approach, such as human reenactment and novel view synthesis from monocular video, where we show significant improvement over the state of the art both qualitatively and quantitatively.