Reviewerly/exHarmony huggingface dataset:
ASIP Accelerator for LUT-based Neural Networks Inference

Binarized Neural Networks (BNNs) offer the promise of low power and high throughput, but this is difficult to achieve on regular processors. A considerable amount of research has been devoted to mapping BNNs to specialized hardware, especially FPGAs, setting aside the flexibility of instruction-set processors. This paper introduces a configurable VLIW processor with a specialized instruction set to efficiently compute the inference of Look-Up-Table based artificial binary neurons in a single clock cycle. Our experiments show that the processor achieves an increased throughput of 2994 when compared to the inference done on a base processor. ×
SmartDet: Context-Aware Dynamic Control of Edge Task Offloading for Mobile Object Detection

Mobile devices such as drones and autonomous vehicles increasingly rely on object detection (OD) through deep neural networks (DNNs) to perform critical tasks such as navigation, target-tracking and surveillance, just to name a few. Due to their high complexity, the execution of these DNNs requires excessive time and energy. Low-complexity object tracking (OT) is thus used along with OD, where the latter is periodically applied to generate "fresh" references for tracking. However, the frames processed with OD incur large delays, which does not comply with real-time applications requirements. Offloading OD to edge servers can mitigate this issue, but existing work focuses on the optimization of the offloading process in systems where the wireless channel has a very large capacity. Herein, we consider systems with constrained and erratic channel capacity, and establish parallel OT (at the mobile device) and OD (at the edge server) processes that are resilient to large OD latency. We propose Katch-Up, a novel tracking mechanism that improves the system resilience to excessive OD delay. We show that this technique greatly improves the quality of the reference available to tracking, and boosts performance up to 33%. However, while Katch-Up significantly improves performance, it also increases the computing load of the mobile device. Hence, we design SmartDet, a low-complexity controller based on deep reinforcement learning (DRL) that learns to achieve the right trade-off between resource utilization and OD performance. SmartDet takes as input highly-heterogeneous context-related information related to the current video content and the current network conditions to optimize frequency and type of OD offloading, as well as Katch-Up utilization. We extensively evaluate SmartDet on a real-world testbed composed by a JetSon Nano as mobile device and a GTX 980 Ti as edge server, connected through a Wi-Fi link, to collect several network-related traces, as well as energy measurements. We consider a state-of-the-art video dataset (ILSVRC 2015 - VID) and state-of-the-art OD models (EfficientDet 0, 2 and 4). Experimental results show that SmartDet achieves an optimal balance between tracking performance – mean Average Recall (mAR) and resource usage. With respect to a baseline with full Katch-Up usage and maximum channel usage, we still increase mAR by 4% while using 50% less of the channel and 30% power resources associated with Katch-Up. With respect to a fixed strategy using minimal resources, we increase mAR by 20% while using Katch-Up on 1/3 of the frames.
MEDSCANLATION: A Deep Learning-Based AI Scanner and Translation Device for Doctor's Prescription Medicine

A prescription is a legal document given to a pharmacist by a licensed physician to produce or administer pharmacological agents/medications to diagnose, cure, or treat an illness. A written prescription is the most common method of prescribing medications in the Philippines. The unreadability of a doctor's handwriting when prescribing medication leads to medication errors. Use of an incorrect dose of medication, or even death. As a result, doctors must rely on stenography or shorthand writing to quickly write vital information on prescriptions. The study aims to develop a device that helps pharmacists in performing jobs effectively and helps the pharmacy to translate the sloppily written prescriptions. In data gathering, the researchers visited two doctors to collect data in a handwritten prescription samples and two doctors wrote the names of five antibiotics on a piece of paper 200 times, producing 1000 handwritten prescription samples. This study conducted a data gathering process flow to come up with the final dataset with 11,000 iamges. The study showed also the training process of Custom SSD Mobilenet v2 Detector with the pre-process handwritten antibiotic names. The validation of the detection model are 92.4% accuracy and 57.4 milliseconds speed. The detection model was uploaded to the raspberry pi for the researchers to develop a device that can detect antibiotic names. The performance and functionality of the whole system were tested and evaluated. The system accurately detects the 39 antibiotic names out of 45 with an f-score of 87% accuracy. This shows and validates that the system is efficient and effective in detecting antibiotic's names.
Research on High-definition Image Classification Processing System of Water Target Objects under Computer Image Recognition Technology

Based on computer image recognition technology, this paper designs a classification method based on high-speed features of radiation noise of water targets. The algorithm mainly includes three steps: saturation detection of water surface area, regional consistency GOA detection edge and target object marker location. The results show that the proposed method significantly outperforms the traditional feature methods, with an average accuracy improvement of 16% and an average recall rate improvement of 13.8%. Using HSCQ for classification, the false positive rate of non-high-speed targets is only 4.1%.
Hardening Hardware Accelerartor Based CNN Inference Phase Against Adversarial Noises

Recent research has shown that Convolution Neu-ral Networks (CNNs) are vulnerable to adversarial examples. Many defense techniques like gradient masking etc have been proposed against adversarial attacks. However, these techniques are limited to training methods and do not offer generalizability. Similarly, in a Horizontal Collaborative Environment (HCE) where trained CNN models are partitioned into different layers, models deployed are also subjected to attacks by adversarial inputs. In this work, we develop a defense strategy to harden CNNs in an HCE against adversarial examples through the detection of adversarial inputs. We propose the notion that by obtaining model prediction at different layers of the CNN and noting the prediction inconsistency, an adversarial noise could be detected. In this work, adversarial noises are generated using the Fast Gradient Sign Method (FGSM), Salt and Pepper (S&P), and Gaussian Noise perturbation (GNP) methodologies. We compare predictions at different layers of a CNN and obtain final prediction via coherence in predicted class of the CNN model. The hardware synthesis results on FPGA, validated proposed method showing that obtaining such accuracy inconsistency require reasonable hardware overhead.
A Top-View Multiple People Tracking System based on Newest YOLOv5 and DeepSort Using Depth Data

Nowadays, with more and more video surveillance systems constructed in our society, pedestrian tracking has become an important issue that has always been discussed in the computer vision domain all over the world. How to propose a real-time tracking system with great accuracy has always been the core of this problem, it’s a bit of a paradox because you want to have a precise detection result and a real-time tracking system at the same time, but we know the best method to make target detection is a deep neural network, which is time-consuming to process immense amounts of data. And to solve the ID-switches problem, we introduce the DeepSort algorithm to make the tracking process done. So in this paper, we propose a real-time tracking system using top-view depth data by integrating the newest YOLOv5 with DeepSort that can achieve nearly 40 frames per second of a high-quality video stream. And the orientation of our camera is top-view, which can help the neural network distinguish a bunch of people with occlusion easily, at the same time we choose depth data to avoid privacy leaks problem. Once we have an accurate detection result then we use the Kalman filter and Hungarian algorithm and matching cascade to handle the matching process of multiple detection and tracks. To authenticate the surveillance system we proposed, we conducted a few experiments on different datasets that meet our data requirements, we also recorded two datasets by various cameras in our laboratory and outdoor environment. In addition, the results showed the superior advantages of top-view depth data in tracking by the detection system and improved the tracking accuracy to 99.3% which is the best mAP@0.5 of alike methods. And all experiments conducted on different video streams can reach a real-time level and verify the effectiveness of this system as well.
Combining BERT with Bi-LSTM for Emotion-Cause Pair Extraction

The emotion-cause pair extraction aims to extract emotion clause and the underlying cause clause for the emotion from documents. Existing methods for extracting emotion-cause pairs mostly focus on semantic representations of clauses, ignoring emotion and cause features in the emotional causality, which weakens the representation learning ability of emotion-cause pairs. In this paper, we propose an Emotion-Cause Pair Feature Extraction model (ECPFE) to solve the problem. Pre-training model BERT and graph attention network are used to obtain clause semantic information. Emotion extractor and cause extractor based on Bi-LSTM are utilized to learn valuable features from semantic information. The emotion extractor captures emotion features and cause extractor captures cause features in clauses. And these valuable features are used to design the specific emotion-cause pair representations to improve the model’s representation learning ability. Moreover, relative position information between clauses obtained by functional relative position encoding is embedded in the specific representations to enhance position-awareness of the model. Experimental results demonstrate that our ECPFE model outperforms the existing models by 3.4% in F1 score.
Training Deep Neural Networks with Large-scale Datasets on Sunway High Performance Computer

It’s important to obtain an accurate large-scale dataset for training deep neural networks (DNNs). But manually labeling is a time-consuming process with high labor cost. In this scenario, researchers are concerned about replacing precise data with collecting images from the internet, especially for images recognition tasks. This brings two problems: the labels of images from the web are often imprecise, and the large number of images involves large amount of computation on training. In this paper, we designed a large-scale noisy image training system based on Sunway TaihuLight supercomputer and implemented it using the Caffe framework. The system utilizes parallel processes as well as data prefetching to exploit computing power of the Sunway supercomputer. In addition, the system employs the mutual calibration training method to reduce the impact of noisy labels. Experimental results show that the system can greatly reduce the training time and has good scalability.
Learning to Align Arabic and English Text to Remote Sensing Images Using Transformers

With the rapid increase of multimodal remote sensing (RS) data, cross-modal retrieval maximally benefits us to give us more flexibility in image retrieval tasks. However, image retrieval across different modalities is still an open challenge in RS community. Inspired by the recent achievement of the transformers on natural language processing and computer vision applications, we present a transformer-based method for text-image retrieval tasks, which consists of separate encoders for textual and visual features. Specifically, we adopted Arabic and English captions at the text modality. Afterward, we investigate two paradigms. In the first paradigm. We consider learning each language independently. In the second paradigm, we jointly learned both Arabic and English languages. The experimental results on two cross-modal confirm the promising capabilities of the proposed method.
Anomaly Detection by Unsupervised Adversarial Generative Self-labelling Autoencoder

Conventionally, autoencoder is applied to unsupervised anomaly detection but suffers from over-lapping distributions between reconstruction errors of normal and anomalous samples, which is also known as lack of learnability, as the autoencoder does not know the characteristic difference between normal and anomalous samples (autoencoder is trained on the normal samples, without any anomalous samples). Recently, the generative adversarial network is proposed, provided as another method instead of the autoencoder. However, the training process is time-consuming and unstable compared to the autoencoder. We propose an unsupervised adversarial generative self-labeling autoencoder (AGSA), optimizing the discriminator and autoencoder by adversarial training, and integrating the supervised learning into unsupervised learning: The discriminator evolves by including pseudo-labeled samples for training, then the autoencoder is required to decorate itself such that the reconstructed samples by the autoencoder score high in the discriminator, therefore resembling the normal samples and avoiding being similar to anomalous ones. In this case, reconstruction errors of anomalous samples are maximized and that of normal ones are minimized, compared to the previous autoencoder. Therefore, the overlapping part is lessened, and we are safe and confident to self-label more samples for training discriminators. An adversarial cycle is formed. AGSA replaces the conventional decision criteria using an anomaly score which highly relies on the prior knowledge of the dataset and is sensitive to the extreme and noisy samples of the dataset, with a well-trained discriminator which is less affected by the extreme and noisy samples. Transfer learning is used for constructing the discriminator for accelerating the training and avoid overfitting. AGSA has been applied to the clouds dataset and marble dataset, achieving satisfying results.
Research on Optoelectronic Accelerator based on AI Computing

The progress of all kinds of special computing accelerators has provided the possibility for greater exploration and application of computing based on artificial intelligence. These special accelerators have various forms, which are mainly divided into solutions based on traditional electronic chips and solutions based on new physical theory architectures such as optoelectronic hybrid architecture. This paper investigates the representative research chips in the above two solutions and their present situations. The performance and energy consumption are drawn. Compared with the traditional electronic chip accelerator, the accelerator based on the new physical architectures has more advantages in unit energy consumption. At last, this study provides a possible direction for the future research and development of the special accelerator based on artificial intelligence computing.
Recurrent Neural Network controllers learned using Virtual Reference Feedback Tuning with application to an Electronic Throttle Body

In this paper the application of Virtual Reference Feedback Tuning (VRFT) for control of nonlinear systems with regulators defined by Echo State Networks (ESNs) and Long Short Term Memory (LSTM) networks is investigated. The capability of this class of regulators of constraining the control variable is pointed out and a control scheme that allows to achieve zero steady-state error is presented. The developed scheme is validated on a benchmark example that consists of an Electronic Throttle Body (ETB).
Long-Horizon Motion Planning via Sampling and Segmented Trajectory Optimization

This paper presents a hybrid robot motion planner that generates long-horizon motion plans for robot navigation in environments with obstacles. We propose a hybrid planner, RRT* with segmented trajectory optimization (RRT*-sOpt), which combines the merits of sampling-based planning, optimization-based planning, and trajectory splitting to quickly plan for a collision-free and dynamically-feasible motion plan. When generating a plan, the RRT* layer quickly samples a semi-optimal path and sets it as an initial reference path. Then, the sOpt layer splits the reference path and performs optimization on each segment. It then splits the new trajectory again and repeats the process until the whole trajectory converges. We also propose to reduce the number of segments before convergence with the aim of further reducing computation time. Simulation results show that RRT*-sOpt benefits from the hybrid structure with trajectory splitting and performs robustly in various robot platforms and scenarios.
SpecPref: High Performing Speculative Attacks Resilient Hardware Prefetchers

With the inception of the Spectre attack in 2018, microarchitecture mitigation strategies propose secure cache hi-erarchies that do not leak the speculative state. Among many mitigation strategies, MuonTrap, proposes an efficient, secure cache hierarchy that provides speculative attack resiliency with minimum performance slowdown. Hardware prefetchers play a significant role in improving application performance by fetching and bringing data and instructions into caches before time. To prevent hardware prefetchers from leaking information about the speculative blocks brought into the cache, MuonTrap trains and triggers hardware prefetchers on the committed instruction streams, eliminating speculative state leakage. We find that on-commit prefetching can lead to significant performance slowdown as high as 20.46 % (primarily because of prefetch timeliness issues), making hardware prefetchers less effective. We propose Speculative yet Secure Prefetching (SpecPref), enhancements on top of the MuonTrap hierarchy that allows prefetching both on-commit and speculatively. We focus on improving the performance slowdown with the state-of-the-art hardware prefetchers without compromising the security guarantee provided by the MuonTrap implementation and provide an average performance slowdown of 1.17%.
An Image Denoising Model using Deep Learning for Digital Breast Tomosynthesis Images

Breast cancer refers to a group of breast tumor subtypes with different genetic and cellular origins and clinical characteristics. Digital Breast Tomosynthesis (DBT) is a new technology that can help radiologists identify breast cancer more accurately. DBT method produces a series of low-dose full-field projection images with a lower acquisition angle, reconstruction noise and artifacts are expected. Thus, this study proposes a technique to denoise the DBT images using a deep learning model. The DBT images are obtained from the website, VICTRE trial, that host the clinical trial image for evaluating DBT. The neural network that is being studied will use a multiscale context aggregation network (MS-CAN) to learn to denoising the images. The deep learning method that is used is compared to the conventional method which is bilateral filtering. Statistical measures such as peak signal to noise ratio (PSNR), structural similarity index (SSIM), and Naturalness Image Quality Evaluator (NIQE) were used to evaluate the performance of the method.
Enhancing Privacy in Ride-Sharing Applications Through POIs Selection

The problem of data privacy preservation is of central importance in ride-sharing applications, because in order to efficiently match passengers with vehicles, these services rely on exact location information. Yet, transportation and location data can reveal personal habits, preferences and behaviors, and users may prefer not to share their exact location. Masking location data in order to avoid the identification of users in case of data leakage, and/or misusage would help protect user privacy, but could also lead to poorer system performance, in terms of efficiency and quality of service as perceived by users.In this paper, we compare classic data masking techniques, namely obfuscation, k-anonymity, and l-diversity, applied to users’ location data, before sending it to a carpooling system. While the first two techniques use randomly generated points to mask the actual location, l-diversity uses actual points of interest, having the additional benefit of ensuring that the disclosed location is always an accessible and safe pickup or drop-off location. Given that users in a real ride-sharing system could choose to protect or not protect their location data when using the system, we also evaluate the effect of privacy preservation penetration rate, by varying the percentage of users choosing to have their location data protected. The results show that l-diversity performance is better than the others’ even when the privacy penetration rate is high, suggesting that this technique has the potential to meet both users’ and system’s needs, and thus being a better option to provide privacy within carpooling systems.
An Secure and Effective Copy Move Detection Based on Pretrained Model

The amount of counterfeit or faked photographs that depict inaccurate or incorrect information has increased. Hence, the Digital image forgery has become a serious problem. Copy move forgery is more risky since it involves copying and pasting a portion of an image into another region of the same image to hide information. Conventional Copy Move Forgery Detection (C-MFD) approaches have limitations in terms of performance. The explanation for this is that the discriminative ability and partially invariant to specific transformations of manual-crafted features are insufficient. Therefore, in this paper we used the AlexNet deep learning model to extract the image features and applies the ReliefF feature selection algorithm to get effective features. The logistic classifier is then given selected features to determine whether the image is forged or not. On the publicly accessible benchmark datasets MICC-F600 and MICC-F2000, the proposed method is tested. The precision rate of the presented method based on AlexNet model is equal to 94 %.
Efficient Targeted Bit-Flip Attack Against the Local Binary Pattern Network

Deep neural networks (DNNs) have shown their great capability of surpassing human performance in many areas. With the help of quantization, artificial intelligence (AI) powered devices are ubiquitously deployed. Yet, the easily accessible AI-powered edge devices become the target of malicious users who can deteriorate the privacy and integrity of the inference process. This paper proposes two adversarial attack scenarios, including three threat models, which crush local binary pattern networks (LBPNet). These attacks can be applied maliciously to flip a limited number of susceptible bits in kernels within the system's shared memory. The threat could be driven through the Row-Hammer attack and significantly drops the model's accuracy. Our preliminary simulation results demonstrate flipping only the most significant bit of the first LBP layer decreases the accuracy from 99.51 % down to 18 % on the MNIST data-set. We then briefly discuss potential hardware/software -oriented defense mechanisms as countermeasures to such attacks.