OpenRaiser/Intern-Atlas huggingface dataset:
A mathematical approach to resilience
Venue: arXiv cs.IT 2020
Year: 2020

In this paper, we evolve from sparsity, a key concept in robust statistics, to concepts and theoretical results of what we call the mathematics of resilience, at the interface between category theory, the theory of dynamical systems, statistical signal processing and biology. We first summarize a recent result on dynamical systems [Beurier, Pastor, Spivak 2019], before presenting the de-generacy paradigm, issued from biology [Edelman, Gally 1973] and mathematically formalized by [Ehresmann Vanbremeersch 2007] [Ehresmann Vanbremeersch 2019] as the Multiplicity Principle (MP). We then make the connection with statistical signal processing by showing that two distinct and structurally different families of tests satisfy the MP.
Towards application-specific query processing systems
Venue: arXiv cs.DB 2020
Year: 2020

Database systems use query processing subsystems for enabling efficient query-based data retrieval. An essential aspect of designing any query-intensive application is tuning the query system to fit the application's requirements and workload characteristics. However, the configuration parameters provided by traditional database systems do not cover the design decisions and trade-offs that arise from the geo-distribution of users and data. In this paper, we present a vision towards a new type of query system architecture that addresses this challenge by enabling query systems to be designed and deployed in a per use case basis. We propose a distributed abstraction called Query Processing Unit that encapsulates primitive query processing tasks, and show how it can be used as a building block for assembling query systems. Using this approach, application architects can construct query systems specialized to their use cases, by controlling the query system's architecture and the placement of its state. We demonstrate the expressiveness of this approach by applying it to the design of a query system that can flexibly place its state in the data center or at the edge, and show that state placement decisions affect the trade-off between query response time and query result freshness.
Contrastive Clustering
Venue: arXiv cs.LG 2020
Year: 2020

In this paper, we propose a one-stage online clustering method called Contrastive Clustering (CC) which explicitly performs the instance- and cluster-level contrastive learning. To be specific, for a given dataset, the positive and negative instance pairs are constructed through data augmentations and then projected into a feature space. Therein, the instance- and cluster-level contrastive learning are respectively conducted in the row and column space by maximizing the similarities of positive pairs while minimizing those of negative ones. Our key observation is that the rows of the feature matrix could be regarded as soft labels of instances, and accordingly the columns could be further regarded as cluster representations. By simultaneously optimizing the instance- and cluster-level contrastive loss, the model jointly learns representations and cluster assignments in an end-to-end manner. Extensive experimental results show that CC remarkably outperforms 17 competitive clustering methods on six challenging image benchmarks. In particular, CC achieves an NMI of 0.705 (0.431) on the CIFAR-10 (CIFAR-100) dataset, which is an up to 19\% (39\%) performance improvement compared with the best baseline.
Privacy-Preserving Machine Learning Training in Aggregation Scenarios
Venue: arXiv cs.CR 2020
Year: 2020

To develop Smart City, the growing popularity of Machine Learning (ML) that appreciates high-quality training datasets generated from diverse IoT devices raises natural questions about the privacy guarantees that can be provided in such settings. Privacy-preserving ML training in an aggregation scenario enables a model demander to securely train ML models with the sensitive IoT data gathered from personal IoT devices. Existing solutions are generally server-aided, cannot deal with the collusion threat between the servers or between the servers and data owners, and do not match the delicate environments of IoT. We propose a privacy-preserving ML training framework named Heda that consists of a library of building blocks based on partial homomorphic encryption (PHE) enabling constructing multiple privacy-preserving ML training protocols for the aggregation scenario without the assistance of untrusted servers and defending the security under collusion situations. Rigorous security analysis demonstrates the proposed protocols can protect the privacy of each participant in the honest-but-curious model and defend the security under most collusion situations. Extensive experiments validate the efficiency of Heda which achieves the privacy-preserving ML training without losing the model accuracy.
Time-to-Provision Evaluation of IoT Devices Using Automated Zero-Touch Provisioning
Venue: arXiv cs.NI 2020
Year: 2020

The Internet of Things (IoT) is being widely adopted in today's society, interconnecting smart embedded devices that are being deployed for indoor and outdoor environments, such as homes, factories and hospitals. Along with the growth in the development and implementation of these IoT devices, their simple and rapid deployment, initial configuration and out-of-the-box operational provisioning are becoming prominent challenges to be circumvented. Considering a large number of heterogeneous devices to be deployed within next generation IoT networks, the amount of time needed for manual provisioning of these IoT devices can significantly delay the deployment and manual provisioning may introduce human-induced failures and errors. By incorporating zero-touch provisioning (ZTP), multiple heterogeneous devices can be provisioned with less effort and without human intervention. In this paper, we propose software-enabled access point (Soft-AP)- and Bluetooth-based ZTP solutions relying only on a single mediator device and evaluate their performances using LOG-A-TEC testbed against manual provisioning in terms of the time required for provisioning (time-to-provision, TTP). We demonstrate that on average, Soft-AP- and Bluetooth-based ZTP solutions outperform manual provisioning with about 154% and 313% when compared to the expert provisioning, and with about 434% and 880% when compared to the non-expert provisioning in terms of TTP performances, respectively.
Modeling the Evolution of Networks as Shrinking Structural Diversity
Venue: arXiv cs.SI 2020
Year: 2020

This article reviews and evaluates models of network evolution based on the notion of structural diversity. We show that diversity is an underlying theme of three principles of network evolution: the preferential attachment model, connectivity and link prediction. We show that in all three cases, a dominant trend towards shrinking diversity is apparent, both theoretically and empirically. In previous work, many kinds of different data have been modeled as networks: social structure, navigational structure, transport infrastructure, communication, etc. Almost all these types of networks are not static structures, but instead dynamic systems that change continuously. Thus, an important question concerns the trends observable in these networks and their interpretation in terms of existing network models. We show in this article that most numerical network characteristics follow statistically significant trends going either up or down, and that these trends can be predicted by considering the notion of diversity. Our work extends previous work observing a shrinking network diameter to measures such as the clustering coefficient, power-law exponent and random walk return probability, and justifies preferential attachment models and link prediction algorithms. We evaluate our hypothesis experimentally using a diverse collection of twenty-seven temporally evolving real-world network datasets.
ServiceNet: A P2P Service Network
Venue: arXiv cs.DC 2020
Year: 2020

Given a large number of online services on the Internet, from time to time, people are still struggling to find out the services that they need. On the other hand, when there are considerable research and development on service discovery and service recommendation, most of the related work are centralized and thus suffers inherent shortages of the centralized systems, e.g., adv-driven, lack at trust, transparence and fairness. In this paper, we propose a ServiceNet - a peer-to-peer (P2P) service network for service discovery and service recommendation. ServiceNet is inspired by blockchain technology and aims at providing an open, transparent and self-growth, and self-management service ecosystem. The paper will present the basic idea, an architecture design of the prototype, and an initial implementation and performance evaluation the prototype design.
Grasp-type Recognition Leveraging Object Affordance
Venue: arXiv cs.RO 2020
Year: 2020

A key challenge in robot teaching is grasp-type recognition with a single RGB image and a target object name. Here, we propose a simple yet effective pipeline to enhance learning-based recognition by leveraging a prior distribution of grasp types for each object. In the pipeline, a convolutional neural network (CNN) recognizes the grasp type from an RGB image. The recognition result is further corrected using the prior distribution (i.e., affordance), which is associated with the target object name. Experimental results showed that the proposed method outperforms both a CNN-only and an affordance-only method. The results highlight the effectiveness of linguistically-driven object affordance for enhancing grasp-type recognition in robot teaching.
Spatio-Temporal Hybrid Graph Convolutional Network for Traffic Forecasting in Telecommunication Networks
Venue: arXiv cs.LG 2020
Year: 2020

Telecommunication networks play a critical role in modern society. With the arrival of 5G networks, these systems are becoming even more diversified, integrated, and intelligent. Traffic forecasting is one of the key components in such a system, however, it is particularly challenging due to the complex spatial-temporal dependency. In this work, we consider this problem from the aspect of a cellular network and the interactions among its base stations. We thoroughly investigate the characteristics of cellular network traffic and shed light on the dependency complexities based on data collected from a densely populated metropolis area. Specifically, we observe that the traffic shows both dynamic and static spatial dependencies as well as diverse cyclic temporal patterns. To address these complexities, we propose an effective deep-learning-based approach, namely, Spatio-Temporal Hybrid Graph Convolutional Network (STHGCN). It employs GRUs to model the temporal dependency, while capturing the complex spatial dependency through a hybrid-GCN from three perspectives: spatial proximity, functional similarity, and recent trend similarity. We conduct extensive experiments on real-world traffic datasets collected from telecommunication networks. Our experimental results demonstrate the superiority of the proposed model in that it consistently outperforms both classical methods and state-of-the-art deep learning models, while being more robust and stable.
Copula-Based Bounds for Multi-User Communications -- Part I: Average Performance
Venue: arXiv cs.IT 2020
Year: 2020

Statistically independent or positively correlated fading models are usually applied to compute the average performance of wireless communications. However, there exist scenarios with negative dependency and it is therefore of interest how different performance metrics behave for different general dependency structures of the channels. Especially best-case and worst-case bounds are practically relevant as a system design guideline. In this two-part letter, we present methods and tools from dependency modeling which can be applied to analyze and design multi-user communications systems exploiting and creating dependencies of the effective fading channels. The first part focuses on fast fading with average performance metrics, while the second part considers slow fading with outage performance metrics.
Heterogeneous Fixed-wing Aerial Vehicles for Resilient Coverage of an Area
Venue: arXiv cs.RO 2020
Year: 2020

This paper presents a distributed approach to provide persistent coverage of an arbitrarily shaped area using heterogeneous coverage of fixed-wing unmanned aerial vehicles (UAVs), and to recover from simultaneous failures of multiple UAVs. The proposed approach discusses level-homogeneous deployment and maintenance of a homogeneous fleet of fixed-wing UAVs given the boundary information and the minimum loitering radius. The UAVs are deployed at different altitude levels to provide heterogeneous coverage and sensing. We use an efficient square packing method to deploy the UAVs, given the minimum loiter radius and the area boundary. The UAVs loiter over the circles inscribed over these packing squares in a synchronized motion to fulfill the full coverage objective. An top-down hierarchy of the square packing, where each outer square (super-square) is partitioned into four equal-sized inner squares (sub-square), is exploited to introduce resilience in the deployed UAV-network. For a failed sub-square UAV, a replacement neighbor is chosen considering the effective coverage and deployed to the corresponding super-square at a higher altitude to recover full coverage, trading-off with the quality of coverage of the sub-area. This is a distributed approach as all the decision making is done within close range of the loss region, and it can be scaled and adapted to various large scale area and UAV configurations. Simulation results have been presented to illustrate and verify the applicability of the approach.
Transparency's Influence on Human-Collective Interactions
Venue: arXiv cs.HC 2020
Year: 2020

Collective robotic systems are biologically inspired and advantageous due to their apparent global intelligence and emergent behaviors. Many applications can benefit from the incorporation of collectives, including environmental monitoring, disaster response missions, and infrastructure support. Transparency research has primarily focused on how the design of the models, visualizations, and control mechanisms influence human-collective interactions. Traditionally most evaluations have focused only on one particular system design element, evaluating its respective transparency. This manuscript analyzed two models and visualizations to understand how the system design elements impacted human-collective interactions, to quantify which model and visualization combination provided the best transparency, and provide design guidance, based on remote supervision of collectives. The consensus decision-making and baseline models, as well as an individual agent and abstract visualizations, were analyzed for sequential best-of-n decision-making tasks involving four collectives, composed of 200 entities each. Both models and visualizations provided transparency and influenced human-collective interactions differently. No single combination provided the best transparency.
WESSA at SemEval-2020 Task 9: Code-Mixed Sentiment Analysis using Transformers
Venue: arXiv cs.CL 2020
Year: 2020

In this paper, we describe our system submitted for SemEval 2020 Task 9, Sentiment Analysis for Code-Mixed Social Media Text alongside other experiments. Our best performing system is a Transfer Learning-based model that fine-tunes "XLM-RoBERTa", a transformer-based multilingual masked language model, on monolingual English and Spanish data and Spanish-English code-mixed data. Our system outperforms the official task baseline by achieving a 70.1% average F1-Score on the official leaderboard using the test set. For later submissions, our system manages to achieve a 75.9% average F1-Score on the test set using CodaLab username "ahmed0sultan".
Copula-Based Bounds for Multi-User Communications -- Part II: Outage Performance
Venue: arXiv cs.IT 2020
Year: 2020

In the first part of this two-part letter, we introduced methods to study the impact of dependency on the expected value of functions of two random variables. In this second part, we present tools to derive worst- and best-case bounds on the outage probability of multi-user communication systems, including multiple access channels, wiretap channels, and reconfigurable intelligent surface-assisted channels.
A Deep Neural Network Tool for Automatic Segmentation of Human Body Parts in Natural Scenes
Venue: arXiv cs.CV 2020
Year: 2020

This short article describes a deep neural network trained to perform automatic segmentation of human body parts in natural scenes. More specifically, we trained a Bayesian SegNet with concrete dropout on the Pascal-Parts dataset to predict whether each pixel in a given frame was part of a person's hair, head, ear, eyebrows, legs, arms, mouth, neck, nose, or torso.
Feature Distillation With Guided Adversarial Contrastive Learning
Venue: arXiv cs.LG 2020
Year: 2020

Deep learning models are shown to be vulnerable to adversarial examples. Though adversarial training can enhance model robustness, typical approaches are computationally expensive. Recent works proposed to transfer the robustness to adversarial attacks across different tasks or models with soft labels.Compared to soft labels, feature contains rich semantic information and holds the potential to be applied to different downstream tasks. In this paper, we propose a novel approach called Guided Adversarial Contrastive Distillation (GACD), to effectively transfer adversarial robustness from teacher to student with features. We first formulate this objective as contrastive learning and connect it with mutual information. With a well-trained teacher model as an anchor, students are expected to extract features similar to the teacher. Then considering the potential errors made by teachers, we propose sample reweighted estimation to eliminate the negative effects from teachers. With GACD, the student not only learns to extract robust features, but also captures structural knowledge from the teacher. By extensive experiments evaluating over popular datasets such as CIFAR-10, CIFAR-100 and STL-10, we demonstrate that our approach can effectively transfer robustness across different models and even different tasks, and achieve comparable or better results than existing methods. Besides, we provide a detailed analysis of various methods, showing that students produced by our approach capture more structural knowledge from teachers and learn more robust features under adversarial attacks.
PESAO: Psychophysical Experimental Setup for Active Observers
Venue: arXiv cs.CV 2020
Year: 2020

Most past and present research in computer vision involves passively observed data. Humans, however, are active observers outside the lab; they explore, search, select what and how to look. Nonetheless, how exactly active observation occurs in humans so that it can inform the design of active computer vision systems is an open problem. PESAO is designed for investigating active, visual observation in a 3D world. The goal was to build an experimental setup for various active perception tasks with human subjects (active observers) in mind that is capable of tracking the head and gaze. While many studies explore human performances, usually, they use line drawings portrayed in 2D, and no active observer is involved. PESAO allows us to bring many studies to the three-dimensional world, even involving active observers. In our instantiation, it spans an area of 400cm x 300cm and can track active observers at a frequency of 120Hz. Furthermore, PESAO provides tracking and recording of 6D head motion, gaze, eye movement-type, first-person video, head-mounted IMU sensor, birds-eye video, and experimenter notes. All are synchronized at microsecond resolution.
Is Each Layer Non-trivial in CNN?
Venue: arXiv cs.CV 2020
Year: 2020

Convolutional neural network (CNN) models have achieved great success in many fields. With the advent of ResNet, networks used in practice are getting deeper and wider. However, is each layer non-trivial in networks? To answer this question, we trained a network on the training set, then we replace the network convolution kernels with zeros and test the result models on the test set. We compared experimental results with baseline and showed that we can reach similar or even the same performances. Although convolution kernels are the cores of networks, we demonstrate that some of them are trivial and regular in ResNet.
Automatic Target Recognition (ATR) from SAR Imaginary by Using Machine Learning Techniques
Venue: arXiv cs.CV 2020
Year: 2020

Automatic Target Recognition (ATR) in Synthetic aperture radar (SAR) images becomes a very challenging problem owing to containing high level noise. In this study, a machine learning-based method is proposed to detect different moving and stationary targets using SAR images. First Order Statistical (FOS) features were obtained from Fast Fourier Transform (FFT), Discrete Cosine Transform (DCT) and Discrete Wavelet Transform (DWT) on gray level SAR images. Gray Level Co-occurrence Matrix (GLCM), Gray Level Run Length Matrix (GLRLM) and Gray Level Size Zone Matrix (GLSZM) algorithms are also used. These features are provided as input for the training and testing stage Support Vector Machine (SVM) model with Gaussian kernels. 4-fold cross-validations were implemented in performance evaluation. Obtained results showed that GLCM + SVM algorithm is the best model with 95.26% accuracy. This proposed method shows that moving and stationary targets in MSTAR database could be recognized with high performance.
Grey-Box Learning of Register Automata
Venue: arXiv cs.FL 2020
Year: 2020

Model learning (a.k.a. active automata learning) is a highly effective technique for obtaining black-box finite state models of software components. Thus far, generalisation to infinite state systems with inputs/outputs that carry data parameters has been challenging. Existing model learning tools for infinite state systems face scalability problems and can only be applied to restricted classes of systems (register automata with equality/inequality). In this article, we show how we can boost the performance of model learning techniques by extracting the constraints on input and output parameters from a run, and making this grey-box information available to the learner. More specifically, we provide new implementations of the tree oracle and equivalence oracle from RALib, which use the derived constraints. We extract the constraints from runs of Python programs using an existing tainting library for Python, and compare our grey-box version of RALib with the existing black-box version on several benchmarks, including some data structures from Python's standard library. Our proof-of-principle implementation results in almost two orders of magnitude improvement in terms of numbers of inputs sent to the software system. Our approach, which can be generalised to richer model classes, also enables RALib to learn models that are out of reach of black-box techniques, such as combination locks.
Depth-Adapted CNN for RGB-D cameras
Venue: arXiv cs.CV 2020
Year: 2020

Conventional 2D Convolutional Neural Networks (CNN) extract features from an input image by applying linear filters. These filters compute the spatial coherence by weighting the photometric information on a fixed neighborhood without taking into account the geometric information. We tackle the problem of improving the classical RGB CNN methods by using the depth information provided by the RGB-D cameras. State-of-the-art approaches use depth as an additional channel or image (HHA) or pass from 2D CNN to 3D CNN. This paper proposes a novel and generic procedure to articulate both photometric and geometric information in CNN architecture. The depth data is represented as a 2D offset to adapt spatial sampling locations. The new model presented is invariant to scale and rotation around the X and the Y axis of the camera coordinate system. Moreover, when depth data is constant, our model is equivalent to a regular CNN. Experiments of benchmarks validate the effectiveness of our model.