Reviewerly/exHarmony huggingface dataset:
Object-Augmented Skeleton-Based Action Recognition

Human Skeleton-based Action Recognition (SAR) methods have made great advances in different applications in recent years. However, most existing SAR models mainly focus on the joint/limb pose estimation. They ignore the skeleton-object interaction, thereby resulting in underutilization of information available. For instance, drinking and eating actions may exhibit similar skeleton movements but different object interactions. Hence relying on joint/limb pose estimation alone may lead to incorrect action predictions; but leveraging skeleton-object interaction will help to discriminate such similar actions. In view of this, we propose a new effective method called Object-Augmented Skeleton-based Action Recognition (OA-SAR) to integrate skeleton-object interactions into action recognition. In specific, OA-SAR extracts the object and joint/limb heatmaps, and then integrates this information for subsequent action recognition. We evaluate the OA-SAR method on two benchmarks, NTU-RGB+D-60, and Drive&Act. Experimental results show that OA-SAR can achieve strong performance on both action datasets.
An Intelligent Tender Management System using Block Chain and IPFS

Tenders must be included in the procurement process for companies and government organizations. The primary purpose of these bids is to get the products and services necessary to effectively complete operations. Poor management of these bids, on the other hand, might have serious financial consequences. Contractor favoritism, inadequate record-keeping systems, a lack of transparency, hacking, data manipulation, and other obstacles may be among these issues. Improper bid and contract management may cause substantial financial loses and harm to the organisation’s reputation. When deficient record-keeping practices result in the loss of critical information, it can be difficult to trace the progress of tenders and contracts and evaluate performance. Additionally, the possibility of hacking and data tampering can lead to the theft of sensitive information, resulting in financial losses and reputational harm. To solve these issues and eliminate the dangers connected with them, this study has used secure blockchain technology as well as encryption to ensure the safety and security of transactions during the tendering process. Blockchain technology is a secure and decentralized ledger system that allows for secure and tamper-proof transaction recording. Encryption, on the other hand, is a means of transforming plain text into unreadable code that requires a decryption key to decrypt. Tender specifications, bid submissions, business profiles, historical performance records, approval authority details, and rejection information are all protected by its cutting-edge technology. The proposed method assures a completely transparent and safe tendering process by utilizing this technology, lowering the risks associated with tender and contract mismanagement.
Multi-Head Attention based Bi-LSTM for Anomaly Detection in Multivariate Time-Series of WSN

Anomaly detection is a widely utilized technique in the field of wireless sensor networks (WSNs) data stream analysis, aimed at identifying unusual events or anomalies in an early stage. However, the constraints of WSN applications pose a significant challenge in achieving effective and efficient anomaly detection. In this work, we proposes a new multi-head attention-based Bi-LSTM approach for anomaly detection in multivariate time-series. Rather than modeling the time series of individual sensor independently, the proposed approach models the time series of multiple sensors concurrently, taking into account potential latent interactions among them, thereby enhancing the accuracy of anomaly detection. The proposed approach does not require labeled data and can be directly applied in real-world scenarios where labeling a large stream of data from heterogeneous sensors is both difficult and time-consuming. Finally, empirical evaluations using a real-world WSN demonstrate effectiveness and robustness of the proposed approach, outperforming traditional deep learning approaches.
PPT-KP: Pruning Point Training-based Kernel Pruning for Deep Convolutional Neural Networks

Pruning, which is a representative method for compressing huge convolutional neural network (CNN) models, has been mainly studied in two directions: weight pruning and filter pruning, with both approaches having clear limitations caused by their intrinsic characteristics. To solve this problem, research on kernel pruning, which has the advantages of both methods, has recently advanced. In this study, pruning point training-based kernel pruning (PPT-KP) is proposed to address the problems of existing kernel pruning methods. With PPT-KP, the L1 norm of the kernel converges to zero through an adaptive regularizer that applies L1 regularization of different intensities depending on the size of the L1 norm of the kernel to secure network sparsity and obtain multiple margin spaces for pruning. Thus, outstanding kernel pruning is possible because several pruning points can be created. PPT-KP outperformed several existing filter pruning and kernel pruning methods on various networks and datasets in terms of the trade-off between FLOPs reduction and accuracy drops. In particular, PPT-KP reduced parameters and FLOPs by 77.2% and 68.9%, respectively, in ResNet-56 on the CIFAR-10 dataset with only a 0.05% accuracy degradation.
A 1W8R 20T SRAM Codebook for 20% Energy Reduction in Mixed-Precision Deep-Learning Inference Processor System

This study introduces a 1W8R 20T multiport memory for codebook quantization in deep-learning processors. We manufactured the memory in a 40 nm process and achieved memory read-access time at 2.75 ns and 2.7-pj/byte power consumption. In addition, we used NVDLA, which was NVIDIA’s deep-learning processor, as a motif and simulated it based on the power obtained from the actual proposed memory. The obtained power and area reduction results are 20.24% and 26.24%, respectively.
KP2Dtiny: Quantized Neural Keypoint Detection and Description on the Edge

Detection and description of keypoints in images is a fundamental component of a wide range of tasks such as Simultaneous Localization And Mapping (SLAM), image alignment and structure from motion (SfM). Efficient computation of these features is crucial for real-time applications and has been addressed by multiple handcrafted algorithms and, recently, by deep neural network-based detectors. Learned detectors achieve high detection performance, but pose high computational requirements, making them slow and impractical for low-power resource constraint platforms. This paper presents a quantized neural keypoint detector and descriptor optimized for edge devices exploiting two recent AI platforms such as MAX78000 by Analog Devices and the Coral AI USB accelerator from Google. To accommodate the diverse constraints and requirements of various applications, we propose and evaluate two model architectures (KP2DtinySmall and KP2DtinyFast) and deploy them on the aforementioned platforms using full 8-bit integer quantization. Furthermore, we extensively evaluate these models in terms of power, latency and accuracy, reporting results on three image sizes (88x88, 320x240 and 640x480), evaluating both quantized and non-quantized models. Fully quantized, KP2DtinySmall reduces network size by a factor of 54x while improving homographic estimation accuracy on 88x88 images on the most stringent threshold (Correctness d1) by 32.4% (0.550) and on 320x240 images by 10.7% (0.648) compared to the KeypointNet architecture by Yang You et. al. This result is achieved by designing a new network with low power platforms in mind, particularly addressing the lower resolution by increasing the density of detectable features. Deployed on the MAX78000 MCU, inference of low-resolution images is run at 59 FPS, consuming 1.1 mJ per image. On the Coral usb accelerator, KP2DtinyFast runs inference on low-resolution images at 527 FPS consuming 3.1 mJ, on high resolution it achieves 70 FPS at 19.9 mJ per inference.
Securing Decision Tree Inference Using Order-Preserving Cryptography

In machine learning (ML) inference, two parties, Alice and Bob, are engaged in a transaction where Alice is the owner of a decision tree model but does not want to reveal its parameters to Bob, who has private data. Bob wants to use Alice’s model for inference, but does not want to reveal his data. Knowing the heavy computational cost of fully homomorphic encryption, Alice and Bob agree to use order-preserving encryption (OPE) for running the inference engine in full confidence without revealing either the decision tree model or the private data. In this paper, we describe how such an OPE computation is established between Alice and Bob. Specifically, we demonstrate how to secure full confidentiality in decision tree inference on an FPGA accelerator embodying an OPE protocol. A finite-state machine design of the encrypted decision tree is evaluated for throughput and resource utilization on an Intel Cyclone V FPGA using the MNIST dataset.
Grand Challenge on Software and Hardware Co-Optimization for E-Commerce Recommendation System

E-commerce has become an indispensable part of the whole commodity economy with rapid expansion. A great deal of time is required for customers to search products by manual work. A good automatic recommendation system can not only bring the customers good shopping experience, but also help companies gain profit growth. In the IEEE AICAS 2023 conference, we have organized the grand challenge on software and hardware co-optimization for e-commerce recommendation system. The desensitized data from Alibaba Group which recorded online purchase behaviors of online shopping users in China are provided. We organize two rounds of the challenge with two different parts of data, separately encouraging participating teams to propose novel ideas for the recommendation algorithm design and deployment. In the preliminary round, participating teams are required to design a recommendation system with high accuracy performance. In the final round, the qualified teams from the preliminary round will be offered with an ARM-based multi-core Yitian 710 CPU cloud server, the teams are required to design an acceleration scheme for the hardware resolution. In the final, 6 best teams will be awarded by using standard evaluation criteria.
LG-LSQ: Learned Gradient Linear Symmetric Quantization for Low-Precision Integer Hardware

To design a deep learning accelerator hardware in edge devices, lower precision weights and activation have advantages regarding area and power. We propose learned gradient linear symmetric quantization (LG-LSQ) as a method for quantizing weights and activation functions to low bit-widths with high accuracy and an integer-only accelerator as hardware for LG-LSQ. First, we introduce the scaling simulated gradient (SSG) method for determining the appropriate gradient for the scaling factor of the linear quantizer during the training process. Second, we introduce the arctangent soft round (ASR) method, which differs from the straight-through estimator (STE) method in its ability to prevent the gradient from becoming zero, thereby solving the discrete problem caused by the rounding process. Finally, to bridge the gap between full-precision and low-bit quantization networks, we propose the minimize discretization error (MDE) method to determine an accurate gradient in backpropagation. In our evaluation on ImageNet, the proposed quantizer achieved full-precision baseline accuracy in various 3-bit networks, including ResNet18, ResNet34, and ResNet50, and an accuracy drop of less than 1% in the quantization of 4-bit weights and 4-bit activations in lightweight models such as MobileNetV2 and ShuffleNetV2.
Universal Smart Lighting System

An innovative universal device for energy-efficient and intelligent lighting control. The device offers advanced features such as automatic human detection, smart turn on/off, and dynamic lighting adjustments. It can detect natural factors and respond based on ambient light. The proposal explores the device’s features, applications, and versatility in managing LED lights, emphasizing energy savings and cost reduction. Its WIFI compatibility enables convenient control via smartphones. The device’s ability to dynamically adjust lighting promotes a natural and comfortable environment, contributing to sustainability. The proposal addresses challenges and limitations to ensure adaptability and maximize benefits. Overall, this universal discussing device presents a practical and sustainable solution with revolutionary potential for lighting systems.
Reducing Overhead of Feature Importance Visualization via Static GradCAM Computation

Explainable AI (XAI) methods provide insights into the operation of black-box Deep Neural Network (DNN) models. GradCAM, an XAI algorithm, provides an explanation by highlighting regions in the input feature space that were relevant to the model’s output. It involves a gradient computation step that adds a significant overhead compared to inference and hinders providing explanations to end-users. In this work, we identify the root cause of the problem to be the dynamic run-time automatic differentiation. To overcome this issue, we propose to offload the gradient computation step to compile time via analytic evaluation. We validate the idea by designing an FPGA implementation of GradCAM that schedules the entire computation graph statically. For a TinyML ResNet18 model, we achieve a reduction in the explanation generation overhead from > 2× using software frameworks on CPU/GPU systems to < 0.01× on the FPGA using our designed hardware and static scheduling.
Description based person retrieval using outfit characteristics and human gender

Nowadays, video surveillance is widely deployed for security. Additionally, they serve for person retrieval. However, person retrieval manually can take long times. Thus, person retrieval using deep learning methods is analyzed in the research community. So we submit a person retrieval system using outfit characteristics and human gender. Our system includes YOLACT++ for clothes segmentation, YOLOR for human detection, and Efficientnet-B2 for clothes types and color classification. Experimental results are demonstrated on customized dataset and achieve an accuracy of 80.65% for the overall system.
Automatic Domain Classification of Text using Machine Learning

The exponential growth of data in recent years has presented a challenge in extracting insights due to its unstructured nature. Efficient management of this vast information requires an effective automatic text classification or text mining method. Text classification plays a crucial role in labeling and tagging textual units such as sentences, queries, paragraphs, and documents, and finds diverse applications. In this paper, we propose an efficient method for classifying sentences into specific domains, including Chemistry, Computer Science, Law, and Mathematics. Our approach combines the TF-IDF (Term Frequency - Inverse Document Frequency) algorithm to calculate the weight of feature words, along with the well-known machine learning algorithm, Multinomial Naïve Bayes, for classification purposes. We hope that our research contributes to the field of text classification and assists researchers, professionals, and organizations in effectively harnessing the vast amount of available information in diverse domains.
Twitter Sentiment Analysis of COVID-19 In India: VADER Perspective

This paper aims to explore the sentiments of the Indian population towards COVID-specific events that occurred between 2020 and 2021, and investigate possible correlations. Using the COV19Tweets dataset from IEEE and the VADER sentiment analysis tool, the study finds that while some COVID-related events align with the sentiment score, others do not relate as expected. The paper presents a detailed discussion of the approach, dataset, tools, and findings, offering a visual representation of how COVID-19 unfolded in India. The research makes notable contributions, including highlighting the superior performance of VADER over TextBlob’s sentiment ratings in categorizing COVID-19-related tweets, demonstrating a significant association between the events and general sentiment during COVID-19, and providing a user-friendly web application to visualize and compare sentiment scores of COVID-19 events in India with a time-series feature. The paper concludes that this framework could be applied in crisis situations to understand them differently.
Feeling of Control Evaluation: Movement Gain for Virtual Object Manipulation in Virtual Reality

Virtual reality (VR) relies heavily on virtual object manipulation due to its definition. Considering that the physical environment limits the size of the real world, previous research suggested the use of amplified movement gains to enhance actual actions when manipulating objects from a distance. In another instance, scholars proposed reducing movement gains to reduce actual actions for precise manipulation in order to improve task precision. VR research has previously focused on the user experience, such as sense of agency (SoA), which emphasizes users’ feeling of control. However, little was known in the existing literature about how movement gains affect SoA in manipulation tasks. Our study examined users’ feeling of control with different movement gains for manipulating virtual objects in VR using self-report and performance measures. This research contributes to the existing research by providing a novel insight into the association between movement gains and the feeling of control perceived by users in manipulation tasks, as well as proposing design considerations for future application design when considering movement gains in virtual object manipulation.
Hierarchical Label Text Classification Method with Deep-Level Label-Assisted Classification

Hierarchical label text classification is a challenging task in the field of natural language processing, where each document needs to be correctly classified into multiple labels with hierarchical structure. However, in the label set, due to the insufficient semantic information contained in the labels and the small number of documents classified under deep-level labels, the training of deep-level labels is insufficient, leading to a significant imbalance in label training. To address this, a hierarchical label text classification method with deep-level label-assisted classification (DLAC) is proposed. The method proposes a deep-level label-assisted classifier, which effectively utilizes text features and rich features of shallow label nodes corresponding to deep label nodes (i.e., shallow label's rich features) on the basis of enhanced label semantics to enhance the classification performance of deep labels. The comparison experiment results with eleven algorithms on three datasets show that the model can effectively improve the classification performance of deep-level labels and achieve good results.
Short-Term Forecasting for Photovoltaic Power Based on Successive Variational Modal Decomposition and Cascaded Deep Learning Model

Along with the improvement of new energy technology, the proportion of photovoltaic power generation is increasing. The accuracy of photovoltaic power generation prediction plays a key role in the stable operation of the power system. Hence, a hybrid deep learning framework based on successive variational modal decomposition, convolutional neural network and gated recurrent unit is presented. Firstly, the photovoltaic power is decomposed by successive variational modal decomposition. Secondly, convolutional neural layer is employed to extract the local key features between photovoltaic power data subsequences and weather data, and gated recurrent further explores the coupling time characteristics in each subsequence. Thirdly, the eventual photovoltaic power forecasting results are obtained via reconstructing the predicted values of each subsequence. The experimental results show that the proposed prediction model has lower computational complexity in data decomposition and exhibits higher prediction accuracy against other models.
Exploring Artificial Neural Networks in Virtual Reality: A Unity and Leap Motion-Based Visualization of ONNX Models

There is a rapidly increasing need for learning resources in the field of Artificial Intelligence aimed at newcomers. To this end, we have attempted to make an intuitive visualization of Artificial Neural Networks in Virtual Reality, while trying to make it as general and extensible as possible to allow for future work to build upon it to create a series of highly intuitive and interactive learning resources. Our implementation results in a framework built using careful design considerations that we think results in a solid building block for work that aims to interactively educate individuals in fields such as Artificial Intelligence and other such algorithms.
DDPG-Based Path Planning Approach for Autonomous Driving

The present work develops a DDPG-based path planning algorithm that integrates the artificial potential field method into reinforcement learning to learn and generate an obstacle-free path quickly and autonomously. The vehicle kinematic model is adopted to describe the motion of autonomous vehicles, and the potential field function of obstacles, road boundaries as well as reference waypoints are considered to construct rewards of reinforcement learning, which enables the vehicle to realize the tradeoff between avoiding obstacles, preventing driving off the road and following the reference route. In contrast to the existent path planning algorithms, the proposed approach is able to learn autonomously in different driving environments, which is more suitable to autonomous vehicles. Moreover, simulations are provided to further demonstrate the effectiveness and adaptability of the proposed algorithm.
Global-Local Attention Mechanism Based Small Object Detection

A small object detection method based on the combination of global and local attention mechanism is proposed in this paper to detect small objects distributed in images. object detection model based on local attention mechanism has good detection accuracy and speed. However, its performance will be reduced due to the smaller size of the object, especially in the case of missed detection and false detection, and the proposed Global Local Detection Model (GLD) can solve this problem. Specifically, a model solution of the Global and Local Combined Attention Block (GL-CAB) combining deep global features and shallow local features of the network is proposed to solve the problem of small object missed detection. On the one hand, the model focuses on small object s in the local and global ranges, and on the other hand, it supplements the small object information lost during the down-sampling process. Aiming at the situation of pseudo-information generated by small object feature fusion, a multi-branch feature pyramid network (MB-FPN) is proposed. Multi-input is used to form multi-scale feature maps for multi-feature fusion on different branches, which reduces the formation of pseudo-information and enhances the extraction of detailed features of small object by the network. Then, the AU-AIR and VOC2007 datasets are selected for experimental training, and the object detection evaluation indicators (AP, AR, F1, mAP, and FPS) are introduced for comparative analysis. Finally, the simulation results show that the proposed method has better performance to solve the problem of missed detection and false detection of small object.