OpenRaiser/Intern-Atlas huggingface dataset:
TrouSPI-Net: Spatio-temporal attention on parallel atrous convolutions and U-GRUs for skeletal pedestrian crossing prediction
Venue: arXiv cs.CV 2021
Year: 2021

Understanding the behaviors and intentions of pedestrians is still one of the main challenges for vehicle autonomy, as accurate predictions of their intentions can guarantee their safety and driving comfort of vehicles. In this paper, we address pedestrian crossing prediction in urban traffic environments by linking the dynamics of a pedestrian's skeleton to a binary crossing intention. We introduce TrouSPI-Net: a context-free, lightweight, multi-branch predictor. TrouSPI-Net extracts spatio-temporal features for different time resolutions by encoding pseudo-images sequences of skeletal joints' positions and processes them with parallel attention modules and atrous convolutions. The proposed approach is then enhanced by processing features such as relative distances of skeletal joints, bounding box positions, or ego-vehicle speed with U-GRUs. Using the newly proposed evaluation procedures for two large public naturalistic data sets for studying pedestrian behavior in traffic: JAAD and PIE, we evaluate TrouSPI-Net and analyze its performance. Experimental results show that TrouSPI-Net achieved 0.76 F1 score on JAAD and 0.80 F1 score on PIE, therefore outperforming current state-of-the-art while being lightweight and context-free.
Accurate shape and phase averaging of time series through Dynamic Time Warping
Venue: arXiv cs.CV 2021
Year: 2021

We propose a novel time series averaging method based on Dynamic Time Warping (DTW). In contrast to previous methods, our algorithm preserves durational information and the distinctive durational features of the sequences due to a simple conversion of the output of DTW into a time sequence and an innovative iterative averaging process. We show that it accurately estimates the ground truth mean sequences and mean temporal location of landmarks in synthetic and real-world datasets and outperforms state-of-the-art methods.
Collision avoidance for multiple MAVs using fast centralized NMPC
Venue: arXiv cs.RO 2021
Year: 2021

This article proposes a novel control architecture using a centralized nonlinear model predictive control (CNMPC) scheme for controlling multiple micro aerial vehicles (MAVs). The control architecture uses an augmented state system to control multiple agents and performs both obstacle and collision avoidance. The optimization algorithm used is OpEn, based on the proximal averaged Newton type method for optimal control (PANOC) which provides fast convergence for non-convex optimization problems. The objective is to perform position reference tracking for each individual agent, while nonlinear constrains guarantee collision avoidance and smooth control signals. To produce a trajectory that satisfies all constraints a penalty method is applied to the nonlinear constraints. The efficacy of this proposed novel control scheme is successfully demonstrated through simulation results and comparisons, in terms of computation time and constraint violations, while are provided with respect to the number of agents.
Scene Text recognition with Full Normalization
Venue: arXiv cs.CV 2021
Year: 2021

Scene text recognition has made significant progress in recent years and has become an important part of the work-flow. The widespread use of mobile devices opens up wide possibilities for using OCR technologies in everyday life. However, lack of training data for new research in this area remains relevant. In this article, we present a new dataset consisting of real shots on smartphones and demonstrate the effectiveness of profile normalization in this task. In addition, the influence of various augmentations during the training of models for analyzing document images on smartphones is studied in detail. Our dataset is publicly available.
DAG-Oriented Protocols PHANTOM and GHOSTDAG under Incentive Attack via Transaction Selection Strategy
Venue: arXiv cs.CR 2021
Year: 2021

In response to the bottleneck of processing throughput inherent to single chain PoW blockchains, several proposals have substituted a single chain for Directed Acyclic Graphs (DAGs). In this work, we investigate two notable DAG-oriented designs. We focus on PHANTOM (and its optimization GHOSTDAG), which proposes a custom transaction selection strategy that enables to increase the throughput of the network. However, the related work lacks a thorough investigation of corner cases that deviate from the protocol in terms of transaction selection strategy. Therefore, we build a custom simulator that extends open source simulation tools to support multiple chains and enables us to investigate such corner cases. Our experiments show that malicious actors who diverge from the proposed transaction selection strategy make more profit as compared to honest miners. Moreover, they have a detrimental effect on the processing throughput of the PHANTOM (and GHOSTDAG) due to same transactions being included in more than one block of different chains. Finally, we show that multiple miners not following the transaction selection strategy are incentivized to create a shared mining pool instead of mining independently, which has a negative impact on decentralization.
A Reasoning Engine for the Gamification of Loop-Invariant Discovery
Venue: arXiv cs.AI 2021
Year: 2021

We describe the design and implementation of a reasoning engine that facilitates the gamification of loop-invariant discovery. Our reasoning engine enables students, computational agents and regular software engineers with no formal methods expertise to collaboratively prove interesting theorems about simple programs using browser-based, online games. Within an hour, players are able to specify and verify properties of programs that are beyond the capabilities of fully-automated tools. The hour limit includes the time for setting up the system, completing a short tutorial explaining game play and reasoning about simple imperative programs. Players are never required to understand formal proofs; they only provide insights by proposing invariants. The reasoning engine is responsible for managing and evaluating the proposed invariants, as well as generating actionable feedback.
Cross-platform analysis of user comments in YouTube videos linked on Reddit conspiracy theory forum
Venue: arXiv cs.SI 2021
Year: 2021

We perform a cross-platform analysis in which we study how does linking YouTube content on Reddit conspiracy forum impact language used in user comments on YouTube. Our findings show a slight change in user language in that it becomes more similar to language used on Reddit.
roadscene2vec: A Tool for Extracting and Embedding Road Scene-Graphs
Venue: arXiv cs.CV 2021
Year: 2021

Recently, road scene-graph representations used in conjunction with graph learning techniques have been shown to outperform state-of-the-art deep learning techniques in tasks including action classification, risk assessment, and collision prediction. To enable the exploration of applications of road scene-graph representations, we introduce roadscene2vec: an open-source tool for extracting and embedding road scene-graphs. The goal of roadscene2vec is to enable research into the applications and capabilities of road scene-graphs by providing tools for generating scene-graphs, graph learning models to generate spatio-temporal scene-graph embeddings, and tools for visualizing and analyzing scene-graph-based methodologies. The capabilities of roadscene2vec include (i) customized scene-graph generation from either video clips or data from the CARLA simulator, (ii) multiple configurable spatio-temporal graph embedding models and baseline CNN-based models, (iii) built-in functionality for using graph and sequence embeddings for risk assessment and collision prediction applications, (iv) tools for evaluating transfer learning, and (v) utilities for visualizing scene-graphs and analyzing the explainability of graph learning models. We demonstrate the utility of roadscene2vec for these use cases with experimental results and qualitative evaluations for both graph learning models and CNN-based models. roadscene2vec is available at https://github.com/AICPS/roadscene2vec.
Exploratory Design of a Hands-free Video Game Controller for a Quadriplegic Individual
Venue: arXiv cs.HC 2021
Year: 2021

From colored pixels to hyper-realistic 3D landscapes of virtual reality, video games have evolved immensely over the last few decades. However, video game input still requires two-handed dexterous finger manipulations for simultaneous joystick and trigger or mouse and keyboard presses. In this work, we explore the design of a hands-free game control method using realtime facial expression recognition for individuals with neurological and neuromuscular diseases who are unable to use traditional game controllers. Similar to other Assistive Technologies (AT), our facial input technique is also designed and tested in collaboration with a graduate student who has Spinal Muscular Atrophy. Our preliminary evaluation shows the potential of facial expression recognition for augmenting the lives of quadriplegic individuals by enabling them to accomplish things like walking, running, flying or other adventures that may not be so attainable otherwise.
Log severity levels matter: A multivocal mapping
Venue: arXiv cs.SE 2021
Year: 2021

The choice of log severity level can be challenging and cause problems in producing reliable logging data. However, there is a lack of specifications and practical guidelines to support this challenge. In this study, we present a multivocal systematic mapping of log severity levels from peer-reviewed literature, logging libraries, and practitioners' views. We analyzed 19 severity levels, 27 studies, and 40 logging libraries. Our results show redundancy and semantic similarity between the levels and a tendency to converge the levels for a total of six levels. Our contributions help leverage the reliability of log entries: (i) mapping the literature about log severity levels, (ii) mapping the severity levels in logging libraries, (iii) a set of synthesized six definitions and four general purposes for severity levels. We recommend that developers use a standard nomenclature, and for logging library creators, we suggest providing accurate and unambiguous definitions of log severity levels.
Byzantine Consensus in Directed Hypergraphs
Venue: arXiv cs.DC 2021
Year: 2021

Byzantine consensus is a classical problem in distributed computing. Each node in a synchronous system starts with a binary input. The goal is to reach agreement in the presence of Byzantine faulty nodes. We consider the setting where communication between nodes is modelled via a directed hypergraph. In the classical point-to-point communication model, the communication between nodes is modelled as a simple graph where all messages sent on an edge are private between the two endpoints of the edge. This allows a faulty node to equivocate, i.e., lie differently to its different neighbors. Different models have been proposed in the literature that weaken equivocation. In the local broadcast model, every message transmitted by a node is received identically and correctly by all of its neighbors. In the hypergraph model, every message transmitted by a node on a hyperedge is received identically and correctly by all nodes on the hyperedge. Tight network conditions are known for each of the three cases for undirected (hyper)graphs. For the directed models, tight conditions are known for the point-to-point and local broadcast models. In this work, we consider the directed hypergraph model that encompasses all the models above. Each directed hyperedge consists of a single head (sender) and at least one tail (receiver), This models a local multicast channel where messages transmitted by the sender are received identically by all the receivers in the hyperedge. For this model, we identify tight network conditions for consensus. We observe how the directed hypergraph model reduces to each of the three models above under specific conditions. In each case, we relate our network condition to the corresponding known tight conditions. The directed hypergraph model also encompasses other practical network models of interest that have not been explored previously, as elaborated in the paper.
A Reliable, Self-Adaptive Face Identification Framework via Lyapunov Optimization
Venue: arXiv cs.DC 2021
Year: 2021

Realtime face identification (FID) from a video feed is highly computation-intensive, and may exhaust computation resources if performed on a device with a limited amount of resources (e.g., a mobile device). In general, FID performs better when images are sampled at a higher rate, minimizing false negatives. However, performing it at an overwhelmingly high rate exposes the system to the risk of a queue overflow that hampers the system's reliability. This paper proposes a novel, queue-aware FID framework that adapts the sampling rate to maximize the FID performance while avoiding a queue overflow by implementing the Lyapunov optimization. A preliminary evaluation via a trace-based simulation confirms the effectiveness of the framework.
Performance Health Index for Complex Cyber Infrastructures
Venue: arXiv cs.SE 2021
Year: 2021

Most IT systems depend on a set of configuration variables (CVs), expressed as a name/value pair that collectively define the resource allocation for the system. While the ill-effects of misconfiguration or improper resource allocation are well-known, there is no effective a priori metric to quantify the impact of the configuration on the desired system attributes such as performance, availability, etc. In this paper, we propose a \textit{Configuration Health Index} (CHI) framework specifically attuned to the performance attribute to capture the influence of CVs on the performance aspects of the system. We show how CHI, which is defined as a configuration scoring system, can take advantage of the domain knowledge and the available (but rather limited) performance data to produce important insights into the configuration settings. We compare the CHI with both well-advertised segmented non-linear models and state-of-the-art data-driven models, and show that the CHI not only consistently provides better results but also avoids the dangers of pure data drive approach which may predict incorrect behavior or eliminate some essential configuration variables from consideration.
SMART: A Heterogeneous Scratchpad Memory Architecture for Superconductor SFQ-based Systolic CNN Accelerators
Venue: arXiv cs.AR 2021
Year: 2021

Ultra-fast \& low-power superconductor single-flux-quantum (SFQ)-based CNN systolic accelerators are built to enhance the CNN inference throughput. However, shift-register (SHIFT)-based scratchpad memory (SPM) arrays prevent a SFQ CNN accelerator from exceeding 40\% of its peak throughput, due to the lack of random access capability. This paper first documents our study of a variety of cryogenic memory technologies, including Vortex Transition Memory (VTM), Josephson-CMOS SRAM, MRAM, and Superconducting Nanowire Memory, during which we found that none of the aforementioned technologies made a SFQ CNN accelerator achieve high throughput, small area, and low power simultaneously. Second, we present a heterogeneous SPM architecture, SMART, composed of SHIFT arrays and a random access array to improve the inference throughput of a SFQ CNN systolic accelerator. Third, we propose a fast, low-power and dense pipelined random access CMOS-SFQ array by building SFQ passive-transmission-line-based H-Trees that connect CMOS sub-banks. Finally, we create an ILP-based compiler to deploy CNN models on SMART. Experimental results show that, with the same chip area overhead, compared to the latest SHIFT-based SFQ CNN accelerator, SMART improves the inference throughput by $3.9\times$ ($2.2\times$), and reduces the inference energy by $86\%$ ($71\%$) when inferring a single image (a batch of images).
A Synergetic Attack against Neural Network Classifiers combining Backdoor and Adversarial Examples
Venue: arXiv cs.CR 2021
Year: 2021

In this work, we show how to jointly exploit adversarial perturbation and model poisoning vulnerabilities to practically launch a new stealthy attack, dubbed AdvTrojan. AdvTrojan is stealthy because it can be activated only when: 1) a carefully crafted adversarial perturbation is injected into the input examples during inference, and 2) a Trojan backdoor is implanted during the training process of the model. We leverage adversarial noise in the input space to move Trojan-infected examples across the model decision boundary, making it difficult to detect. The stealthiness behavior of AdvTrojan fools the users into accidentally trust the infected model as a robust classifier against adversarial examples. AdvTrojan can be implemented by only poisoning the training data similar to conventional Trojan backdoor attacks. Our thorough analysis and extensive experiments on several benchmark datasets show that AdvTrojan can bypass existing defenses with a success rate close to 100% in most of our experimental scenarios and can be extended to attack federated learning tasks as well.
Symbol Emergence and The Solutions to Any Task
Venue: arXiv cs.AI 2021
Year: 2021

The following defines intent, an arbitrary task and its solutions, and then argues that an agent which always constructs what is called an Intensional Solution would qualify as artificial general intelligence. We then explain how natural language may emerge and be acquired by such an agent, conferring the ability to model the intent of other individuals labouring under similar compulsions, because an abstract symbol system and the solution to a task are one and the same.
Semantic Segmentation on VSPW Dataset through Aggregation of Transformer Models
Venue: arXiv cs.CV 2021
Year: 2021

Semantic segmentation is an important task in computer vision, from which some important usage scenarios are derived, such as autonomous driving, scene parsing, etc. Due to the emphasis on the task of video semantic segmentation, we participated in this competition. In this report, we briefly introduce the solutions of team 'BetterThing' for the ICCV2021 - Video Scene Parsing in the Wild Challenge. Transformer is used as the backbone for extracting video frame features, and the final result is the aggregation of the output of two Transformer models, SWIN and VOLO. This solution achieves 57.3% mIoU, which is ranked 3rd place in the Video Scene Parsing in the Wild Challenge.
Achieving near native runtime performance and cross-platform performance portability for random number generation through SYCL interoperability
Venue: arXiv cs.DC 2021
Year: 2021

High-performance computing (HPC) is a major driver accelerating scientific research and discovery, from quantum simulations to medical therapeutics. While the increasing availability of HPC resources is in many cases pivotal to successful science, even the largest collaborations lack the computational expertise required for maximal exploitation of current hardware capabilities. The need to maintain multiple platform-specific codebases further complicates matters, potentially adding constraints on machines that can be utilized. Fortunately, numerous programming models are under development that aim to facilitate portable codes for heterogeneous computing. One in particular is SYCL, an open standard, C++-based single-source programming paradigm. Among SYCL's features is interoperability, a mechanism through which applications and third-party libraries coordinate sharing data and execute collaboratively. In this paper, we leverage the SYCL programming model to demonstrate cross-platform performance portability across heterogeneous resources. We detail our NVIDIA and AMD random number generator extensions to the oneMKL open-source interfaces library. Performance portability is measured relative to platform-specific baseline applications executed on four major hardware platforms using two different compilers supporting SYCL. The utility of our extensions are exemplified in a real-world setting via a high-energy physics simulation application. We show the performance of implementations that capitalize on SYCL interoperability are at par with native implementations, attesting to the cross-platform performance portability of a SYCL-based approach to scientific codes.
Access Control Using Spatially Invariant Permutation of Feature Maps for Semantic Segmentation Models
Venue: arXiv cs.CV 2021
Year: 2021

In this paper, we propose an access control method that uses the spatially invariant permutation of feature maps with a secret key for protecting semantic segmentation models. Segmentation models are trained and tested by permuting selected feature maps with a secret key. The proposed method allows rightful users with the correct key not only to access a model to full capacity but also to degrade the performance for unauthorized users. Conventional access control methods have focused only on image classification tasks, and these methods have never been applied to semantic segmentation tasks. In an experiment, the protected models were demonstrated to allow rightful users to obtain almost the same performance as that of non-protected models but also to be robust against access by unauthorized users without a key. In addition, a conventional method with block-wise transformations was also verified to have degraded performance under semantic segmentation models.
Low-Latency and Secure Computation Offloading Assisted by Hybrid Relay-Reflecting Intelligent Surface
Venue: arXiv cs.IT 2021
Year: 2021

Recently, the hybrid relay-reflecting intelligent surface (HRRIS) has been introduced as a spectral- and energy-efficient architecture to assist wireless communication systems. In the HRRIS, a single or few active relay elements are deployed along with a large number of passive reflecting elements, allowing it to not only reflect but also amplify the incident signals. In this work, we investigate the potential of the HRRIS in aiding the computation offloading in a single-user mobile edge computing system. The objective is to minimize the offloading latency while ensuring the secrecy of user data against a malicious eavesdropper. We develop efficient solutions to this latency minimization problem based on alternating optimization. Through numerical results, we show that the deployment of the HRRIS can result in a considerable reduction in latency. Furthermore, the latency reduction gain offered by the HRRIS is much more significant than that of the conventional reconfigurable intelligent surface (RIS).
Joint Management and Analysis of Textual Documents and Tabular Data within the AUDAL Data Lake
Venue: arXiv cs.DB 2021
Year: 2021

In 2010, the concept of data lake emerged as an alternative to data warehouses for big data management. Data lakes follow a schema-on-read approach to provide rich and flexible analyses. However, although trendy in both the industry and academia, the concept of data lake is still maturing, and there are still few methodological approaches to data lake design. Thus, we introduce a new approach to design a data lake and propose an extensive metadata system to activate richer features than those usually supported in data lake approaches. We implement our approach in the AUDAL data lake, where we jointly exploit both textual documents and tabular data, in contrast with structured and/or semi-structured data typically processed in data lakes from the literature. Furthermore, we also innovate by leveraging metadata to activate both data retrieval and content analysis, including Text-OLAP and SQL querying. Finally, we show the feasibility of our approach using a real-word use case on the one hand, and a benchmark on the other hand.