OpenRaiser/Intern-Atlas huggingface dataset:
Being-ahead: Benchmarking and Exploring Accelerators for Hardware-Efficient AI Deployment
Venue: arXiv cs.AR 2021
Year: 2021

Customized hardware accelerators have been developed to provide improved performance and efficiency for DNN inference and training. However, the existing hardware accelerators may not always be suitable for handling various DNN models as their architecture paradigms and configuration tradeoffs are highly application-specific. It is important to benchmark the accelerator candidates in the earliest stage to gather comprehensive performance metrics and locate the potential bottlenecks. Further demands also emerge after benchmarking, which require adequate solutions to address the bottlenecks and improve the current designs for targeted workloads. To achieve these goals, in this paper, we leverage an automation tool called DNNExplorer for benchmarking customized DNN hardware accelerators and exploring novel accelerator designs with improved performance and efficiency. Key features include (1) direct support to popular machine learning frameworks for DNN workload analysis and accurate analytical models for fast accelerator benchmarking; (2) a novel accelerator design paradigm with high-dimensional design space support and fine-grained adjustability to overcome the existing design drawbacks; and (3) a design space exploration (DSE) engine to generate optimized accelerators by considering targeted AI workloads and available hardware resources. Results show that accelerators adopting the proposed novel paradigm can deliver up to 4.2X higher throughput (GOP/s) than the state-of-the-art pipeline design in DNNBuilder and up to 2.0X improved efficiency than the recently published generic design in HybridDNN given the same DNN model and resource budgets. With DNNExplorer's benchmarking and exploration features, we can be ahead at building and optimizing customized AI accelerators and enable more efficient AI applications.
ODE Transformer: An Ordinary Differential Equation-Inspired Model for Neural Machine Translation
Venue: arXiv cs.CL 2021
Year: 2021

It has been found that residual networks are an Euler discretization of solutions to Ordinary Differential Equations (ODEs). In this paper, we explore a deeper relationship between Transformer and numerical methods of ODEs. We show that a residual block of layers in Transformer can be described as a higher-order solution to ODEs. This leads us to design a new architecture (call it ODE Transformer) analogous to the Runge-Kutta method that is well motivated in ODEs. As a natural extension to Transformer, ODE Transformer is easy to implement and parameter efficient. Our experiments on three WMT tasks demonstrate the genericity of this model, and large improvements in performance over several strong baselines. It achieves 30.76 and 44.11 BLEU scores on the WMT'14 En-De and En-Fr test data. This sets a new state-of-the-art on the WMT'14 En-Fr task.
MuSLCAT: Multi-Scale Multi-Level Convolutional Attention Transformer for Discriminative Music Modeling on Raw Waveforms
Venue: arXiv cs.SD 2021
Year: 2021

In this work, we aim to improve the expressive capacity of waveform-based discriminative music networks by modeling both sequential (temporal) and hierarchical information in an efficient end-to-end architecture. We present MuSLCAT, or Multi-scale and Multi-level Convolutional Attention Transformer, a novel architecture for learning robust representations of complex music tags directly from raw waveform recordings. We also introduce a lightweight variant of MuSLCAT called MuSLCAN, short for Multi-scale and Multi-level Convolutional Attention Network. Both MuSLCAT and MuSLCAN model features from multiple scales and levels by integrating a frontend-backend architecture. The frontend targets different frequency ranges while modeling long-range dependencies and multi-level interactions by using two convolutional attention networks with attention-augmented convolution (AAC) blocks. The backend dynamically recalibrates multi-scale and level features extracted from the frontend by incorporating self-attention. The difference between MuSLCAT and MuSLCAN is their backend components. MuSLCAT's backend is a modified version of BERT. While MuSLCAN's is a simple AAC block. We validate the proposed MuSLCAT and MuSLCAN architectures by comparing them to state-of-the-art networks on four benchmark datasets for music tagging and genre recognition. Our experiments show that MuSLCAT and MuSLCAN consistently yield competitive results when compared to state-of-the-art waveform-based models yet require considerably fewer parameters.
SimMBM Channel Simulator for Media-Based Modulation Systems
Venue: arXiv cs.IT 2021
Year: 2021

Media-based modulation (MBM), exploiting rich scattering properties of transmission environments via different radiation patterns of a single reconfigurable antenna (RA), has brought new insights into future communication systems. In this study, considering this innovative transmission principle, we introduce the realistic, two-dimensional (2D), and open-source SimMBM channel simulator to support various applications of MBM systems at sub-6 GHz frequency bands in different environments.
Enabling Novel Interconnection Agreements with Path-Aware Networking Architectures
Venue: arXiv cs.NI 2021
Year: 2021

Path-aware networks (PANs) are emerging as an intriguing new paradigm with the potential to significantly improve the dependability and efficiency of networks. However, the benefits of PANs can only be realized if the adoption of such architectures is economically viable. This paper shows that PANs enable novel interconnection agreements among autonomous systems, which allow to considerably improve both economic profits and path diversity compared to today's Internet. Specifically, by supporting packet forwarding along a path selected by the packet source, PANs do not require the Gao-Rexford conditions to ensure stability. Hence, autonomous systems can establish novel agreements, creating new paths which demonstrably improve latency and bandwidth metrics in many cases. This paper also expounds two methods to set up agreements which are Pareto-optimal, fair, and thus attractive to both parties. We further present a bargaining mechanism that allows two parties to efficiently automate agreement negotiations.
Backdoor Attack in the Physical World
Venue: arXiv cs.CR 2021
Year: 2021

Backdoor attack intends to inject hidden backdoor into the deep neural networks (DNNs), such that the prediction of infected models will be maliciously changed if the hidden backdoor is activated by the attacker-defined trigger. Currently, most existing backdoor attacks adopted the setting of static trigger, $i.e.,$ triggers across the training and testing images follow the same appearance and are located in the same area. In this paper, we revisit this attack paradigm by analyzing trigger characteristics. We demonstrate that this attack paradigm is vulnerable when the trigger in testing images is not consistent with the one used for training. As such, those attacks are far less effective in the physical world, where the location and appearance of the trigger in the digitized image may be different from that of the one used for training. Moreover, we also discuss how to alleviate such vulnerability. We hope that this work could inspire more explorations on backdoor properties, to help the design of more advanced backdoor attack and defense methods.
Leverage Score Sampling for Complete Mode Coverage in Generative Adversarial Networks
Venue: arXiv cs.LG 2021
Year: 2021

Commonly, machine learning models minimize an empirical expectation. As a result, the trained models typically perform well for the majority of the data but the performance may deteriorate in less dense regions of the dataset. This issue also arises in generative modeling. A generative model may overlook underrepresented modes that are less frequent in the empirical data distribution. This problem is known as complete mode coverage. We propose a sampling procedure based on ridge leverage scores which significantly improves mode coverage when compared to standard methods and can easily be combined with any GAN. Ridge leverage scores are computed by using an explicit feature map, associated with the next-to-last layer of a GAN discriminator or of a pre-trained network, or by using an implicit feature map corresponding to a Gaussian kernel. Multiple evaluations against recent approaches of complete mode coverage show a clear improvement when using the proposed sampling strategy.
Accessing HID Devices on the Web With the WebHID API: How to play the Chrome Dino Game by Jumping With a Nintendo Joy-Con Controller in One's Pocket
Venue: arXiv cs.HC 2021
Year: 2021

In this demonstration, we show how special hardware like Nintendo Joy-Con controllers can be made accessible from the Web through the new WebHID API. This novel technology proposal allows developers to write Web drivers in pure JavaScript that talk to Human Interface Device (HID) devices via the HID protocol. One such example of a driver has been realized in the project Joy-Con-WebHID, which allows for fun pastimes like playing the Google Chrome browser's offline dinosaur game by jumping. This works thanks to the accelerometers built into Joy-Con controllers whose signals are read out by the driver and used to control the game character in the browser. A video of the experience is available.
Towards a Computing Platform for the LEO Edge
Venue: arXiv cs.DC 2021
Year: 2021

The new space race is heating up as private companies such as SpaceX and Amazon are building large satellite constellations in low-earth orbit (LEO) to provide global broadband internet access. As the number of subscribers connected to this access network grows, it becomes necessary to investigate if and how edge computing concepts can be applied to LEO satellite networks. In this paper, we discuss the unique characteristics of the LEO edge and analyze the suitability of three organization paradigms for applications considering developer requirements. We conclude that the serverless approach is the most promising solution, opening up the field for future research.
MPC-based Reinforcement Learning for Economic Problems with Application to Battery Storage
Venue: arXiv cs.LG 2021
Year: 2021

In this paper, we are interested in optimal control problems with purely economic costs, which often yield optimal policies having a (nearly) bang-bang structure. We focus on policy approximations based on Model Predictive Control (MPC) and the use of the deterministic policy gradient method to optimize the MPC closed-loop performance in the presence of unmodelled stochasticity or model error. When the policy has a (nearly) bang-bang structure, we observe that the policy gradient method can struggle to produce meaningful steps in the policy parameters. To tackle this issue, we propose a homotopy strategy based on the interior-point method, providing a relaxation of the policy during the learning. We investigate a specific well-known battery storage problem, and show that the proposed method delivers a homogeneous and faster learning than a classical policy gradient approach.
A Distributed Virtual Network Function Placement Approach in Satellite Edge and Cloud Computing
Venue: arXiv cs.NI 2021
Year: 2021

Satellite edge computing has become a promising way to provide computing services for Internet of Things (IoT) devices in remote areas, which are out of the coverage of terrestrial networks, nevertheless, it is not suitable for large-scale IoT devices due to the resource limitation of satellites. Cloud computing can provide sufficient available resources for IoT devices but it does not meet the service requirements of delay sensitive users as high network latency. Collaborative edge and cloud computing is to facilitate flexible service provisioning for numerous IoT devices by incorporating the advantages of edge and cloud computing. In this paper, we investigate the virtual network function (VNF) placement problem in collaborative satellite edge and cloud computing to minimize the satellite network bandwidth usage and the service end-to-end delay. We formulate the VNF placement problem as an integer non-linear programming problem and propose a distributed VNF placement (D-VNFP) algorithm to address it, as the VNF placement problem is NP-hard. The experiments are conducted to evaluate the effectiveness of the proposed D-VNFP algorithm. The results show that the proposed D-VNFP algorithm outperforms the existing baseline algorithms of Greedy and Viterbi for solving the VNF placement problem in satellite edge and cloud computing.
Rearchitecting Kubernetes for the Edge
Venue: arXiv cs.DC 2021
Year: 2021

Recent years have seen Kubernetes emerge as a primary choice for container orchestration. Kubernetes largely targets the cloud environment but new use cases require performant, available and scalable orchestration at the edge. Kubernetes stores all cluster state in etcd, a strongly consistent key-value store. We find that at larger etcd cluster sizes, offering higher availability, write request latency significantly increases and throughput decreases similarly. Coupled with approximately 30% of Kubernetes requests being writes, this directly impacts the request latency and availability of Kubernetes, reducing its suitability for the edge. We revisit the requirement of strong consistency and propose an eventually consistent approach instead. This enables higher performance, availability and scalability whilst still supporting the broad needs of Kubernetes. This aims to make Kubernetes much more suitable for performance-critical, dynamically-scaled edge solutions.
Towards Soft Circuit Breaking in Service Meshes via Application-agnostic Caching
Venue: arXiv cs.NI 2021
Year: 2021

Service meshes factor out code dealing with inter-micro-service communication, such as circuit breaking. Circuit breaking actuation is currently limited to an "on/off" switch, i.e., a tripped circuit breaker will return an application-level error indicating service unavailability to the calling micro-service. This paper proposes a soft circuit breaker actuator, which returns cached data instead of an error. The overall resilience of a cloud application is improved if constituent micro-services return stale data, instead of no data at all. While caching is widely employed for serving web service traffic, its usage in inter-micro-service communication is lacking. Micro-services responses are highly dynamic, which requires carefully choosing adaptive time-to-life caching algorithms. We evaluate our approach through two experiments. First, we quantify the trade-off between traffic reduction and data staleness using a purpose-build service, thereby identifying algorithm configurations that keep data staleness at about 3% or less while reducing network load by up to 30%. Second, we quantify the network load reduction with the micro-service benchmark by Google Cloud called Hipster Shop. Our approach results in caching of about 80% of requests. Results show the feasibility and efficiency of our approach, which encourages implementing caching as a circuit breaking actuator in service meshes.
Improving Cell-Free Massive MIMO by Local Per-Bit Soft Detection
Venue: arXiv cs.IT 2021
Year: 2021

In this letter, we consider the uplink of a cell-free Massive multiple-input multiple-output (MIMO) network where each user is decoded by a subset of access points (APs). An additional step is introduced in the cell-free Massive MIMO processing: each AP in the uplink locally implements soft MIMO detection and then shares the resulting bit log-likelihoods on the front-haul link. The decoding of the data is performed at the central processing unit (CPU), collecting the data from the APs. The non-linear processing at the APs consists of the approximate computation of the posterior density for each received data bit, exploiting only local channel state information. The proposed method offers good performance in terms of frame-error-rate and considerably lower complexity than the optimal maximum-likelihood demodulator.
RFQuack: A Universal Hardware-Software Toolkit for Wireless Protocol (Security) Analysis and Research
Venue: arXiv cs.CR 2021
Year: 2021

Software-defined radios (SDRs) are indispensable for signal reconnaissance and physical-layer dissection, but despite we have advanced tools like Universal Radio Hacker, SDR-based approaches require substantial effort. Contrarily, RF dongles such as the popular Yard Stick One are easy to use and guarantee a deterministic physical-layer implementation. However, they're not very flexible, as each dongle is a static hardware system with a monolithic firmware. We present RFquack, an open-source tool and library firmware that combines the flexibility of a software-based approach with the determinism and performance of embedded RF frontends. RFquack is based on a multi-radio hardware system with swappable RF frontends, and a firmware that exposes a uniform, hardware-agnostic API. RFquack focuses on a structured firmware architecture that allows high- and low-level interaction with the RF frontends. It facilitates the development of host-side scripts and firmware plug-ins, to implement efficient data-processing pipelines or interactive protocols, thanks to the multi-radio support. RFquack has an IPython shell and 9 firmware modules for: spectrum scanning, automatic carrier detection and bitrate estimation, headless operation with remote management, in-flight packet filtering and manipulation, MouseJack, and RollJam (as examples). We used RFquack to setup RF hacking contests, analyze industrial-grade devices and key fobs, on which we found and reported 11 vulnerabilities in their RF protocols.
Prediction of Solar Radiation Using Artificial Neural Network
Venue: arXiv cs.LG 2021
Year: 2021

Most solar applications and systems can be reliably used to generate electricity and power in many homes and offices. Recently, there is an increase in many solar required systems that can be found not only in electricity generation but other applications such as solar distillation, water heating, heating of buildings, meteorology and producing solar conversion energy. Prediction of solar radiation is very significant in order to accomplish the previously mentioned objectives. In this paper, the main target is to present an algorithm that can be used to predict an hourly activity of solar radiation. Using a dataset that consists of temperature of air, time, humidity, wind speed, atmospheric pressure, direction of wind and solar radiation data, an Artificial Neural Network (ANN) model is constructed to effectively forecast solar radiation using the available weather forecast data. Two models are created to efficiently create a system capable of interpreting patterns through supervised learning data and predict the correct amount of radiation present in the atmosphere. The results of the two statistical indicators: Mean Absolute Error (MAE) and Mean Squared Error (MSE) are performed and compared with observed and predicted data. These two models were able to generate efficient predictions with sufficient performance accuracy.
Automatic Large Scale Detection of Red Palm Weevil Infestation using Aerial and Street View Images
Venue: arXiv cs.CV 2021
Year: 2021

The spread of the Red Palm Weevil has dramatically affected date growers, homeowners and governments, forcing them to deal with a constant threat to their palm trees. Early detection of palm tree infestation has been proven to be critical in order to allow treatment that may save trees from irreversible damage, and is most commonly performed by local physical access for individual tree monitoring. Here, we present a novel method for surveillance of Red Palm Weevil infested palm trees utilizing state-of-the-art deep learning algorithms, with aerial and street-level imagery data. To detect infested palm trees we analyzed over 100,000 aerial and street-images, mapping the location of palm trees in urban areas. Using this procedure, we discovered and verified infested palm trees at various locations.
PointShuffleNet: Learning Non-Euclidean Features with Homotopy Equivalence and Mutual Information
Venue: arXiv cs.CV 2021
Year: 2021

Point cloud analysis is still a challenging task due to the disorder and sparsity of samplings of their geometric structures from 3D sensors. In this paper, we introduce the homotopy equivalence relation (HER) to make the neural networks learn the data distribution from a high-dimension manifold. A shuffle operation is adopted to construct HER for its randomness and zero-parameter. In addition, inspired by prior works, we propose a local mutual information regularizer (LMIR) to cut off the trivial path that leads to a classification error from HER. LMIR utilizes mutual information to measure the distance between the original feature and HER transformed feature and learns common features in a contrastive learning scheme. Thus, we combine HER and LMIR to give our model the ability to learn non-Euclidean features from a high-dimension manifold. This is named the non-Euclidean feature learner. Furthermore, we propose a new heuristics and efficiency point sampling algorithm named ClusterFPS to obtain approximate uniform sampling but at faster speed. ClusterFPS uses a cluster algorithm to divide a point cloud into several clusters and deploy the farthest point sampling algorithm on each cluster in parallel. By combining the above methods, we propose a novel point cloud analysis neural network called PointShuffleNet (PSN), which shows great promise in point cloud classification and segmentation. Extensive experiments show that our PSN achieves state-of-the-art results on ModelNet40, ShapeNet and S3DIS with high efficiency. Theoretically, we provide mathematical analysis toward understanding of what the data distribution HER has developed and why LMIR can drop the trivial path by maximizing mutual information implicitly.
A Modified Convolutional Network for Auto-encoding based on Pattern Theory Growth Function
Venue: arXiv cs.CV 2021
Year: 2021

This brief paper reports the shortcoming of a variant of convolutional neural network whose components are developed based on the pattern theory framework.
Microlocation for Smart Buildings in the Era of the Internet of Things: A Survey of Technologies, Techniques, and Approaches
Venue: arXiv cs.NI 2021
Year: 2021

Microlocation plays a key role in the transformation of traditional buildings into smart infrastructure. Microlocation is the process of locating any entity with a very high accuracy, possibly in centimeters. Such technologies require high detection accuracy, energy efficiency, wide reception range, low cost, and availability. In this article, we provide insights into various microlocation-enabling technologies, techniques, and services and discuss how they can accelerate the incorporation of the Internet of Things (IoT) in smart buildings. We cover the challenges and examine some signal processing filtering techniques such that microlocation-enabling technologies and services can be thoroughly integrated with an IoT-equipped smart building. An experiment with Bluetooth Low-Energy (BLE) beacons used for microlocation is also presented.
Building Beyond HLS: Graph Analysis and Others
Venue: arXiv cs.AR 2021
Year: 2021

High-Level Synthesis has introduced reconfigurable logic to a new world -- that of software development. The newest wave of HLS tools has been successful, and the future looks bright. But is HLS the end-all-be-all to FPGA acceleration? Is it enough to allow non-experts to program FPGAs successfully, even when dealing with troublesome data structures and complex control flows -- such as those often encountered in graph algorithms? We take a look at the panorama of adoption of HLS by the software community, focusing on graph analysis in particular in order to generalise to \textit{FPGA-unfriendly} problems. We argue for the existence of shortcomings in current HLS development flows which hinder adoption, and present our perspective on the path forward, including how these issues may be remedied via higher-level tooling.