Reviewerly/exHarmony huggingface dataset:
Tesla-Rapture: A Lightweight Gesture Recognition System From mmWave Radar Sparse Point Clouds

We present Tesla-Rapture, a gesture recognition system for sparse point clouds generated by mmWave Radars. State of the art gesture recognition models are either too resource consuming or not sufficiently accurate for the integration into real-life scenarios using wearable or constrained equipment such as IoT devices (e.g., Raspberry PI), XR hardware (e.g., HoloLens), or smart-phones. To tackle this issue, we have developed Tesla, a Message Passing Neural Network (MPNN) graph convolution approach for mmWave radar point clouds. The model outperforms the state of the art on three datasets in terms of accuracy while reducing the computational complexity and, hence, the execution time. In particular, the approach, is able to predict a gesture almost 8 times faster than the most accurate competitor. Our performance evaluation in different scenarios (environments, angles, distances) shows that Tesla generalizes well and improves the accuracy up to 20% in challenging scenarios, such as a through-wall setting and sensing at extreme angles. Utilizing Tesla, we develop Tesla-Rapture, a real-time implementation using a mmWave Radar on a Raspberry PI 4 and evaluate its accuracy and time-complexity. We also publish the source code, the trained models, and the implementation of the model for embedded devices.
Low-Power Detection and Classification for In-Sensor Predictive Maintenance Based on Vibration Monitoring

In this work, a new custom design of an anomaly detection and classification system is proposed. It is composed of a convolutional Auto-Encoder (AE) hardware design to perform anomaly detection which cooperates with a mixed HW/SW Convolutional Neural Network (CNN) to perform the classification of detected anomalies. The AE features a partial binarization, so that the weights are binarized while the activations, associated to some selected layers, are non-binarized. This has been necessary to meet the severe area and energy constraints that allow it to be integrated on the same die as the MEMS sensors for which it serves as a neural accelerator. The CNN shares the feature extraction module with the AE, whereas a SW classifier is triggered by the AE when a fault is detected, working asynchronously to it. The AE has been mapped on a Xilinx Artix-7 FPGA, featuring an Output Data Rate (ODR) of 365 kHz and achieving a power dissipation of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$333 \boldsymbol {\mu } $ </tex-math></inline-formula> W/MHz. Logic synthesis has targeted TSMC CMOS 65 nm, 90 nm, and 130 nm standard cells. Best results achieved highlight a power consumption of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$138 \boldsymbol {\mu } $ </tex-math></inline-formula> W/MHz with an area occupation of 0.49 mm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> when real-time operations are set. These results enable the integration of the complete neural accelerator in the CMOS circuitry that typically sits with the inertial MEMS on the same silicon die. Comparisons with the related works suggest that the proposed system is capable of state-of-the-art performances and accuracy.
EF-Train: Enable Efficient On-device CNN Training on FPGA through Data Reshaping for Online Adaptation or Personalization

Conventionally, DNN models are trained once in the cloud and deployed in edge devices such as cars, robots, or unmanned aerial vehicles (UAVs) for real-time inference. However, there are many cases that require the models to adapt to new environments, domains, or users. In order to realize such domain adaption or personalization, the models on devices need to be continuously trained on the device. In this work, we design EF-Train, an efficient DNN training accelerator with a unified channel-level parallelism-based convolution kernel that can achieve end-to-end training on resource-limited low-power edge-level FPGAs. It is challenging to implement on-device training on resource-limited FPGAs due to the low efficiency caused by different memory access patterns among forward and backward propagation and weight update. Therefore, we developed a data reshaping approach with intra-tile continuous memory allocation and weight reuse. An analytical model is established to automatically schedule computation and memory resources to achieve high energy efficiency on edge FPGAs. The experimental results show that our design achieves 46.99 GFLOPS and 6.09 GFLOPS/W in terms of throughput and energy efficiency, respectively.
Hybrid CNN-LSTM Architecture for LiDAR Point Clouds Semantic Segmentation

The semantic segmentation of LiDAR in the outdoor environment is still an open problem in the field of automatic driving. Although the emerging development and technological advancements, it remains challenging for three reasons: 1) uneven distribution of LiDAR Points in 3D space; 2) the same object can be represented by different point clouds sequences; 3) no unique geometric information compared with the pixels of 2D RGB images. In order to tackle all the challenges mentioned above, we propose a new LiDAR point clouds semantic segmentation algorithm, Hybrid CNN-LSTM, which is composed of an efficient point clouds feature processing method and a novel neural network structure. Inspired by representing the point clouds as a fixed-length vector in PolarNet, we first convert the 3D point clouds into pseudo image. In order to better represent the features of small objects, we input the pseudo image into long short-term memory (LSTM) network according to the spatial filling curve. We design a new neural network structure, which combines the features of different channels generated by convolutional neural network and long short-term memory network. Experiments show that our method has higher semantic segmentation accuracy in comparison with state-of-the-art algorithms on SemanticKITTI dataset. Theoretically, we provide an analysis to understand why our network can better segment the small objects with sparse point clouds features.
Data Integration and Interoperability: Towards a Model-Driven and Pattern-Oriented Approach

Data integration is one of the core responsibilities of EDM (enterprise data management) and interoperability. It is essential for almost every digitalization project, e.g., during the migration from a legacy ERP (enterprise resource planning) software to a new system. One challenge is the incompatibility of data models, i.e., different software systems use specific or proprietary terminology, data structures, data formats, and semantics. Data need to be interchanged between software systems, and often complex data conversions or transformations are necessary. This paper presents an approach that allows software engineers or data experts to use models and patterns in order to specify data integration: it is based on data models such as ER (entity-relationship) diagrams or UML (unified modeling language) class models that are well-accepted and widely used in practice. Predefined data integration patterns are combined (applied) on the model level leading to formal, precise, and concise definitions of data transformations and conversions. Data integration definitions can then be executed (via code generation) so that a manual implementation is not necessary. The advantages are that existing data models can be reused, standardized data integration patterns lead to fast results, and data integration specifications are executable and can be easily maintained and extended. An example transformation of elements of a relational data model to object-oriented data structures shows the approach in practice. Its focus is on data mappings and relationships.
Cryptocurrencies, Diversification and the COVID-19 Pandemic

This paper features an analysis of cryptocurrencies and the impact of the COVID-19 pandemic on their effectiveness as a portfolio diversification tool and explores the correlations between the continuously compounded returns on Bitcoin, Ethereum and the S&amp;P500 Index using a variety of parametric and non-parametric techniques. These methods include linear standard metrics such as the application of ordinary least squares regression (OLS) and the Pearson, Spearman and Kendall’s tau measures of association. In addition, non-linear, non-parametric measures such as the Generalised Measure of Correlation (GMC) and non-parametric copula estimates are applied. The results across this range of measures are consistent. The metrics suggest that, whilst the shock of the COVID-19 pandemic does not appear to have increased the correlations between the cryptocurrency series, it appears to have increased the correlations between the returns on cryptocurrencies and those on the S&amp;P500 Index. This suggests that investments in cryptocurrencies are not likely to offer key diversification strategies in times of crisis, on the basis of evidence provided by this crisis.
LILLIPUT: a lightweight low-latency lookup-table decoder for near-term Quantum error correction

The error rates of quantum devices are orders of magnitude higher than what is needed to run most quantum applications. To close this gap, Quantum Error Correction (QEC) encodes logical qubits and distributes information using several physical qubits. By periodically executing a syndrome extraction circuit on the logical qubits, information about errors (called syndrome) is extracted while running programs. A decoder uses these syndromes to identify and correct errors in real time, which is necessary to prevent accumulation of errors. Unfortunately, software decoders are slow and hardware decoders are fast but less accurate. Thus, almost all QEC studies so far have relied on offline decoding.
Astraea: towards QoS-aware and resource-efficient multi-stage GPU services

Multi-stage user-facing applications on GPUs are widely-used nowa- days, and are often implemented to be microservices. Prior re- search works are not applicable to ensuring QoS of GPU-based microservices due to the different communication patterns and shared resource contentions. We propose Astraea to manage GPU microservices considering the above factors. In Astraea, a microser- vice deployment policy is used to maximize the supported peak service load while ensuring the required QoS. To adaptively switch the communication methods between microservices according to different deployments, we propose an auto-scaling GPU communi- cation framework. The framework automatically scales based on the currently used hardware topology and microservice location, and adopts global memory-based techniques to reduce intra-GPU communication. Astraea increases the supported peak load by up to 82.3% while achieving the desired 99%-ile latency target compared with state-of-the-art solutions.
Hyperspectral Image Denoising with Realistic Data

The hyperspectral image (HSI) denoising has been widely utilized to improve HSI qualities. Recently, learning-based HSI denoising methods have shown their effectiveness, but most of them are based on synthetic dataset and lack the generalization capability on real testing HSI. Moreover, there is still no public paired real HSI denoising dataset to learn HSI denoising network and quantitatively evaluate HSI methods. In this paper, we mainly focus on how to produce realistic dataset for learning and evaluating HSI denoising network. On the one hand, we collect a paired real HSI denoising dataset, which consists of short-exposure noisy HSIs and the corresponding long-exposure clean HSIs. On the other hand, we propose an accurate HSI noise model which matches the distribution of real data well and can be employed to synthesize realistic dataset. On the basis of the noise model, we present an approach to calibrate the noise parameters of the given hyperspectral camera. The extensive experimental results show that a network learned with only synthetic data generated by our noise model performs as well as it is learned with paired real data. Our code and data are available at: https://github.com/ColinTaoZhang/HSIDwRD.
Motion Adaptive Pose Estimation from Compressed Videos

Human pose estimation from videos has many real-world applications. Existing methods focus on applying models with a uniform computation profile on fully decoded frames, ignoring the freely-available motion signals and motion-compensation residuals from the compressed stream. A novel model, called Motion Adaptive Pose Net is proposed to exploit the compressed streams to efficiently decode pose sequences from videos. The model incorporates a Motion Compensated ConvLSTM to propagate the spatially aligned features, along with an adaptive gate to dynamically determine if the computationally expensive features should be extracted from fully decoded frames to compensate the motion-warped features, solely based on the residual errors. Leveraging the informative yet readily available signals from compressed streams, we propagate the latent features through our Motion Adaptive Pose Net efficiently Our model outperforms the state-of-the-art models in pose-estimation accuracy on two widely used datasets with only around half of the computation complexity.
End-to-end Piece-wise Unwarping of Document Images

Document unwarping attempts to undo physical deformations of the paper and recover a ’flatbed’ scanned document-image for downstream tasks such as OCR. Current state-of-the-art relies on global unwarping of the document which is not robust to local deformation changes. Moreover, a global unwarping often produces spurious warping artifacts in less warped regions to compensate for severe warps present in other parts of the document. In this paper, we propose the first end-to-end trainable piece-wise unwarping <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> method that predicts local deformation fields and stitches them together with global information to obtain an improved unwarping. The proposed piece-wise formulation results in 4% improvement in terms of multi-scale structural similarity (MS-SSIM) and shows better performance in terms of OCR metrics, character error rate (CER) and word error rate (WER) compared to the state-of-the-art.
Efficient learning rate adaptation based on hierarchical optimization approach

This paper proposes a new hierarchical approach to learning rate adaptation in gradient methods, called learning rate optimization (LRO). LRO formulates the learning rate adaption problem as a hierarchical optimization problem that minimizes the loss function with respect to the learning rate for current model parameters and gradients. Then, LRO optimizes the learning rate based on the alternating direction method of multipliers (ADMM). In the process of this learning rate optimization, LRO does not require any second-order information and probabilistic model, so it is highly efficient. Furthermore, LRO does not require any additional hyperparameters when compared to the vanilla gradient method with the simple exponential learning rate decay. In the experiments, we integrated LRO with vanilla SGD and Adam. Then, we compared their optimization performance with the state-of-the-art learning rate adaptation methods and also the most commonly-used adaptive gradient methods. The SGD and Adam with LRO outperformed all the competitors on the benchmark datasets in image classification tasks.
DisBezant: Secure and Robust Federated Learning Against Byzantine Attack in IoT-Enabled MTS

With the intelligentization of Maritime Transportation System (MTS), Internet of Thing (IoT) and machine learning technologies have been widely used to achieve the intelligent control and routing planning for ships. As an important branch of machine learning, federated learning is the first choice to train an accurate joint model without sharing ships' data directly. However, there are still many unsolved challenges while using federated learning in IoT-enabled MTS, such as the privacy preservation and Byzantine attacks. To surmount the above challenges, a novel mechanism, namely DisBezant, is designed to achieve the secure and Byzantine-robust federated learning in IoT-enabled MTS. Specifically, a credibility-based mechanism is proposed to resist the Byzantine attack in non-iid (not independent and identically distributed) dataset which is usually gathered from heterogeneous ships. The credibility is introduced to measure the trustworthiness of uploaded knowledge from ships and is updated based on their shared information in each epoch. Then, we design an efficient privacy-preserving gradient aggregation protocol based on a secure two-party calculation protocol. With the help of a central server, we can accurately recognise the Byzantine attackers and update the global model parameters privately. Furthermore, we theoretically discussed the privacy preservation and efficiency of DisBezant. To verify the effectiveness of our DisBezant, we evaluate it over three real datasets and the results demonstrate that DisBezant can efficiently and effectively achieve the Byzantine-robust federated learning. Although there are 40% nodes are Byzantine attackers in participants, our DisBezant can still recognise them and ensure the accurate model training.
Learning residue-aware correlation filters and refining scale for real-time UAV tracking

• We propose a novel regularization to model the residue between two neighboring frames, resulting in what we call residue-aware correlation filters, which show better convergence properties in filter learning. Meanwhile, we add spatial and temporal regularizations to boot performance with little additional computational cost. • We propose a novel scale estimation approach for DCF-based trackers by using the GrabCut algorithm to refine the discriminative scale estimates, which can be incorporated easily into any tracking method with discriminative scale estimation to improve precision and accuracy. • We demonstrate the proposed methods on four UAV benchmarks, namely, UAV123@10fps, DTB70, UAVDT and Vistrone2018 (VisDrone2018-test-dev). Experimental results show that the proposed approaches achieves state-of-the-art performance. Unmanned aerial vehicle (UAV)-based tracking finds its applications in agriculture, aviation, navigation, transportation and public security, etc and develops rapidly recently. However, due to limitations of computing resources, battery capacity, requirement of low power and maximum load of UAV, the deployment of deep learning-based tracking algorithms in UAV is currently not feasible and therefore discriminative correlation filters (DCF)-based trackers have stood out in UAV tracking community for their high efficiency and appealing robustness on a single CPU. But confronted with difficult challenges the efficiency and accuracy of existing DCF-based approaches is still not satisfying. Inspired by the good optimization properties associated with residue representation, in this paper we exploit the residue nature inherent to videos and propose residue-aware correlation filters which demonstrate better convergence properties in filter learning. In addition, we propose a scale refinement strategy to improve the wildly adopted discriminative scale estimation in DCF-based trackers, which, in fact, greatly impacts the precision and accuracy of the trackers since accumulated scale error degrades the appearance model as online updating goes on. Extensive experiments are conducted on four UAV benchmarks, namely, UAV123@10fps, DTB70, UAVDT and Vistrone2018 (VisDrone2018-test-dev). The results show that our method achieves state-of-the-art performance in UAV tracking.
KNNENS: A <i>k</i>-Nearest Neighbor Ensemble-Based Method for Incremental Learning Under Data Stream With Emerging New Classes

In this brief, we investigate the problem of incremental learning under data stream with emerging new classes (SENC). In the literature, existing approaches encounter the following problems: 1) yielding high false positive for the new class; i) having long prediction time; and 3) having access to true labels for all instances, which is unrealistic and unacceptable in real-life streaming tasks. Therefore, we propose the k -Nearest Neighbor ENSemble-based method (KNNENS) to handle these problems. The KNNENS is effective to detect the new class and maintains high classification performance for known classes. It is also efficient in terms of run time and does not require true labels of new class instances for model update, which is desired in real-life streaming classification tasks. Experimental results show that the KNNENS achieves the best performance on four benchmark datasets and three real-world data streams in terms of accuracy and F1-measure and has a relatively fast run time compared to four reference methods. Codes are available at https://github.com/Ntriver/KNNENS.
Adversarial Attacks On Multi-Agent Communication

Growing at a fast pace, modern autonomous systems will soon be deployed at scale, opening up the possibility for cooperative multi-agent systems. Sharing information and distributing workloads allow autonomous agents to better perform tasks and increase computation efficiency. However, shared information can be modified to execute adversarial attacks on deep learning models that are widely employed in modern systems. Thus, we aim to study the robustness of such systems and focus on exploring adversarial at-tacks in a novel multi-agent setting where communication is done through sharing learned intermediate representations of neural networks. We observe that an indistinguishable adversarial message can severely degrade performance, but becomes weaker as the number of benign agents increases. Furthermore, we show that black-box transfer attacks are more difficult in this setting when compared to directly perturbing the inputs, as it is necessary to align the distribution of learned representations with domain adaptation. Our work studies robustness at the neural network level to con-tribute an additional layer of fault tolerance to modern security protocols for more secure multi-agent systems.
Multi-Robot Path Planning With Due Times

We formulate the problem of multi-robot path planning with due times (MRPP-DT), which is a variant of the classical path planning problem with task due times explicitly considered. The objectives are to obtain collision-free paths with the minimization of maximum lateness, total tardiness or total unit penalties. When task due times are available, formulating the path planning problem as the MRPP-DT is more preferable because of task awareness in path coordination level. This problem is NP-hard to solve optimally for all these three objectives. To solve this problem optimally, we develop integer linear programming (ILP) models for three objectives respectively, and propose reduction-based algorithms. We also extend these algorithms to the problems of anonymous multi-robot path planning with due times. Theoretically, we prove the structure property and computational complexity of these problems, and the completeness of the proposed algorithms. Empirically, we compare their performance on different maps with random obstacles. Computational results demonstrate that these algorithms are suitable for different problem settings, and capable of computing high-quality solutions within allocated time.