Reviewerly/exHarmony huggingface dataset:
A Full-Stack Approach for Side-Channel Secure ML Hardware

Machine learning (ML) has recently emerged as an application with confidentiality needs. A trained ML model is indeed a high-value intellectual property (IP), making it a lucrative target for notorious side-channel attacks. Recent works have already shown the possibility of reverse engineering the model internals by exploiting the side channels like timing and power consumption. But the defenses are largely unexplored. Preventing ML IP theft is highly relevant given that the demand for ML will only increase in the coming years. Securing ML hardware against side-channel attacks requires analyzing the vulnerabilities in the current ML applications and developing full-stack countermeasures from the ground up, covering cryptographic proofs, circuit design, firmware support, architecture/microarchitecture integration, compiler extensions, software design, and physical testing. There is a need to work on all abstraction levels because focusing on just one or few level(s) cannot provide a complete solution to this nascent problem. Our research achieves four key objectives to realize the first complete solution for side-channel protected ML. First, we analyze the side-channel vulnerabilities in the various hardware blocks of an ML accelerator and assess the feasibility of model parameter extraction. Second, we design provably-secure gadgets, implement them on FPGA, and empirically validate possible countermeasures. Third, we add usability and flexibility to the solution-the ability to support multiple ML architectures via secure software APIs and compiler extensions on a RISC-V core. Fourth, we fabricate the final solution at Skywater 130nm node.
An Optimized Evolutionary Algorithm Applied to Mobile Robots For Finding The Shortest Path in A Known Environment

Mobile robots are used to accomplish various kinds of tasks nowadays. Path planning is one of the important processes that a mobile robot requires when there is a need for an automated navigation system. The path which is an output of the path planning should be a collision-free and optimized to increase the efficiency of the robot in various manners. Recently, meta-heuristic optimization techniques which have been inspired by the nature of the biosphere are used for finding the shortest path in path planning. This research mainly focused on developing a problem-specific evolutionary algorithm to generate the shortest path from a given initial position to a destination in a known environment. The improved algorithm consists of a combination of random mutation and windowed dynamic mutation operators that improve the intelligent path-searching process The MATLAB programming tool is used to develop the algorithm. Several case studies are conducted to find the shortest path between start and end points that are selected considering various distances. The Proposed EA can provide an optimized path with fewer iterations when compared to genetic algorithms. Thus, the proposed evolutionary algorithm is more computationally efficient than genetic algorithms for this specific application.
Prompting or Fine-tuning? A Comparative Study of Large Language Models for Taxonomy Construction

Taxonomies represent hierarchical relations between entities, frequently applied in various software modeling and natural language processing (NLP) activities. They are typically subject to a set of structural constraints restricting their content. However, manual taxonomy construction can be time-consuming, incomplete, and costly to maintain. Recent studies of large language models (LLMs) have demonstrated that appropriate user inputs (called prompting) can effectively guide LLMs, such as GPT-3, in diverse NLP tasks without explicit (re-)training. However, existing approaches for automated taxonomy construction typically involve fine-tuning a language model by adjusting model parameters. In this paper, we present a general framework for taxonomy construction that takes into account structural constraints. We subsequently conduct a systematic comparison between the prompting and fine-tuning approaches performed on a hypernym taxonomy and a novel computer science taxonomy dataset. Our result reveals the following: (1) Even without explicit training on the dataset, the prompting approach outperforms fine-tuning-based approaches. Moreover, the performance gap between prompting and fine-tuning widens when the training dataset is small. However, (2) taxonomies generated by the fine-tuning approach can be easily post-processed to satisfy all the constraints, whereas handling violations of the taxonomies produced by the prompting approach can be challenging. These evaluation findings provide guidance on selecting the appropriate method for taxonomy construction and highlight potential enhancements for both approaches.
Developing Mask R-CNN Framework for Real-Time Object Detection

In the field of computer vision, achieving real-time object detection through deep learning holds significant importance. Notable strides have been made in real-time object detection methods, particularly due to the rapid progress of deep convolutional neural networks (CNNs) compared to traditional approaches. It has been observed that existing real-time deep CNN-based object detectors face performance limitations, primarily stemming from the architecture of the underlying base network. This study introduces an improved framework for real-time object detection based on the Mask R-CNN model. To address the challenge of enhancing performance under stricter localization criteria, we replace the original Mask R-CNN’s Region of Interest Align (RoIAlign) with spatial interpolation. Additionally, in the final phase of the Mask R-CNN framework, we utilize the depthwise separable convolution architecture from EfficientNet-B7 to construct a classifier for proposal categorization and to adjust bounding boxes for detected objects. Experimental findings on both the COCO dataset and the ImageNet dataset demonstrate that our proposed approach surpasses the original Mask R-CNN in terms of detection accuracy and inference speed. Categorically, our method outperforms the original Mask R-CNN framework by 51.5% on the COCO test set and 46.2% on the ImageNet test set.
Quantum codes over F<sub>q</sub> from α+βu+γv+δuv+ηu<sup>2</sup>+θv<sup>2</sup>+λu<sup>2</sup>v+μuv<sup>2</sup>+νu<sup>2</sup>v<sup>2</sup>- constacyclic codes

The goal of this work is the construction of quantum codes over $\mathbb{F}_{q}$ by using $\alpha+\beta u+\gamma v+\delta u v+\eta u^{2}+\theta v^{2}+\lambda u^{2} v+$ $\mu u v^{2}+\nu u^{2} v^{2}$-constacyclic codes over the ring $R=\mathbb{F}_{q}+\mathbb{F}_{q} u+$ $\mathbb{F}_{q} v+\mathbb{F}_{q} u v+\mathbb{F}_{q} u^{2}+\mathbb{F}_{q} v^{2}+\mathbb{F}_{q} u^{2} v+\mathbb{F}_{q} u v^{2}+\mathbb{F}_{q} u^{2} v^{2}$. We give the structure of $\alpha+\beta u+\gamma v+\delta u v+\eta u^{2}+\theta v^{2}+\lambda u^{2} v+\mu u v^{2}+\nu u^{2} v^{2}$. constacyclic and obtain self-orthogonal codes. We decompose a constacyclic code over $\mathbb{F}_{q}+\mathbb{F}_{q} u+\mathbb{F}_{q} v+\mathbb{F}_{q} u v+\mathbb{F}_{q} u^{2}+\mathbb{F}_{q} v^{2}+$ $\mathbb{F}_{q} u^{2} v+\mathbb{F}_{q} u v^{2}+\mathbb{F}_{q} u^{2} v^{2}$ into constacyclic codes over $\mathbb{F}_{q}$. This decomposition makes it possible to give the parameters of the corresponding quantum code.
Channel Impulse Response Peak Clustering Using Neural Networks

This paper introduces an approach to process channel sounder data acquired from Channel Impulse Response (CIR) of 60GHz and 80GHz channel sounder systems, through the integration of Long Short-Term Memory (LSTM) Neural Network (NN) and Fully Connected Neural Network (FCNN). The primary goal is to enhance and automate cluster detection within peaks from noised CIR data. The study initially compares the performance of LSTM NN and FCNN across different input sequence lengths. Notably, LSTM surpasses FCNN due to its incorporation of memory cells, which prove beneficial for handling longer series.Additionally, the paper investigates the robustness of LSTM NN through various architectural configurations. The findings suggest that robust neural networks tend to closely mimic the input function, whereas smaller neural networks are better at generalizing trends in time series data, which is desirable for anomaly detection, where function peaks are regarded as anomalies.Finally, the selected LSTM NN is compared with traditional signal filters, including Butterworth, Savitzky-Golay, Bessel/Thomson, and median filters. Visual observations indicate that the most effective methods for peak detection within channel impulse response data are either the LSTM NN or median filter, as they yield similar results.
A Lightweight Approach Towards Speaker Authentication Systems

In a world where traditional authentication systems are constrained, the introduction of voice-based authentication provides a viable option. This cutting-edge biometric security method uses unique vocal traits including pitch, tone, and speech patterns to create a unique voiceprint. Its sophisticated and dependable verification procedure, when combined with voice-activated services and secure access systems, not only solves au-thentication issues but also improves user experience overall. Our suggested method provides a thorough blueprint for developing a lightweight voice authentication system that is based on text. This system leverages an effective encoder model to convert voice spectrograms, making deployment easy even on edge devices with limited resources. Various dimensionality reduction techniques are explored to obtain optimal voice embeddings that capture speaker uniqueness while minimizing model complexity. A key novelty is the application of model compression techniques including lightweight architectures and Siamese Networks to obtain highly condensed voice embeddings for each user, reducing storage and infrastructure costs compared to traditional voice authentication methods. The proposed lightweight spectrogram embeddings leverage language-agnostic acoustic features, enabling language-independent speaker verification. Additionally, dimensionality reduction applied during voice registration allows the capturing of discriminative voice characteristics in a low-dimensional compact feature space. This significantly cuts down the storage requirements and infrastructure costs per user compared to standard voice biometric approaches, while retaining competitive verification performance. The architecture is optimized for responsiveness by leveraging lightweight frameworks. The proposed system delivers competitive voice authentication capabilities while minimizing memory, computational, and energy footprints. This makes the system useful for integration into smart devices and paves the way for ubiquitous voice biometrics.
DeFCN-nano: An End-to-End Real-Time Object Detection for Insulator Defects

It is important to achieve real-time and accurate detection of the insulator defects via the unmanned aerial vehicles (UAVs) so as to improve the inspection efficiency of the large-scale power grids. This paper studies the You Only Look Once version 8 (YOLOv8) and DeFCN object detection algorithms based on the deep learning techniques, then an end-to-end real-time insulator defect detection method is proposed based on the DeFCN. The DeFCN model is lightweighted referring to the design of YOLOv8-n, which can balance the accuracy and real-time performance in the model structure. The proposed DeFCN-nano is validated on an open-source dataset and the experimental results demonstrate that the mean Average Precision (mAP) of the insulator detection is 97.51%, the mAP for detecting defects is 99.26% and the overall mAP is 98.39%. Compared with the baseline models, the proposed model has higher detection speed with a real-time detection speed of 58 frames per second.
EVEN: An Event-Based Framework for Monocular Depth Estimation at Adverse Night Conditions

Accurate depth estimation under adverse night conditions has practical impact and applications, such as on autonomous driving and rescue robots. In this work, we studied monocular depth estimation at night time in which various adverse weather, light, and different road conditions exist, with data captured in both RGB and event modalities. Event camera can better capture intensity changes by virtue of its high dynamic range (HDR), which is particularly suitable to be applied at adverse night conditions in which the amount of light is limited in the scene. Although event data can retain visual perception that conventional RGB camera may fail to capture, the lack of texture and color information of event data hinders its applicability to accurately estimate depth alone. To tackle this problem, we propose an event-vision based framework that integrates low-light enhancement for the RGB source, and exploits the complementary merits of RGB and event data. A dataset that includes paired RGB and event streams, and ground truth depth maps has been constructed. Comprehensive experiments have been conducted, and the impact of different adverse weather combinations on the performance of framework has also been investigated. The results have shown that our proposed framework can better estimate monocular depth at adverse nights than six baselines.
A Modular Framework for Robot Embodied Instruction Following by Large Language Model

In the ALFRED challenge for robot simulation, the robot still faces a challenge to schedule a task in the embodied instruction following (EIF) tasks. These tasks require the robot to accurately perceive visual features and understand language instructions. However, the previous approaches typically employed end-to-end structures that utilize a shallow understanding of language instructions, while EIF tasks demand a deeper understanding of the semantic relationships in these instructions. To overcome these limitations, we propose a method which named REIF. The method incorporates modules for visual perception, language understanding, semantic search, closed container prediction, navigation, and operation to form a modular framework based on visual language multi-modal learning. The semantic search module supports more efficient object search, while the closed container prediction module enables deeper language understanding. Through learning multiple task instructions, the robot can efficiently and accurately complete EIF tasks in unseen scenes under certain step lengths. Our framework performs significantly well on unseen scene tasks within the ALFRED benchmark, achieving state-of-the-art accuracy and efficiency rates of 50.83% and 23.06% respectively. These results demonstrate that our method is capable of efficiently and accurately inferring the presence of closed container in unseen scenes, and can successfully execute a series of actions to interact with target object within closed container. Our method has achieved the first place in the ALFRED data-set competition. You can find our submissions and results at the following link: https://leaderboard.allenai.org/alfred/submissions/public.
Robust Multi-Agent Coverage Path Planning for Unmanned Airial Vehicles (UAVs) in Complex 3D Environments with Deep Reinforcement Learning*

Multi-agent coverage path planning (MACPP) with unmanned aerial vehicles (UAVs) is an essential task in several real-world applications, such as environmental monitoring, surveillance, and search and rescue missions. Existing MACPP algorithms suffer from limitations, such as the inability to generalize over all system parameters, long computational time, and limited applicability in dynamic environments. In this study, we propose a novel MACPP framework based on proximal policy optimization (PPO), a model-free reinforcement learning algorithm. Our framework provides a dynamic 3D machine learning-based approach that can generalize over all system parameters, such as target zones, agent positions, battery levels, and other critical factors. Hence, the framework offers several advantages, including efficient and effective path planning, real-time application on hardware, and applicability in complex 3D structures such as buildings or alpine terrain. We demonstrate the feasibility and effectiveness of our approach by successfully applying it in real-time on hardware. Our experimental results show that our proposed framework can effectively handle 3D structures and react to changes mid-flight.
Adaptive Polynomial Predictive Filter: Solving Inconsistent and Interrupted Sensor Data Challenges

This article discusses the challenges faced by sensor fusion systems owing to inconsistent and interrupted sensor data and proposes an adaptive polynomial predictive filter-based sensor fusion approach to address this problem. This approach incorporates an adaptive polynomial filter based on polynomial extrapolation that can effectively capture the temporal dynamics of the sensor data and provide a robust estimate of the underlying signal. This paper provides a detailed analysis of the proposed approach, including its theoretical foundation, implementation, and performance evaluation using simulated sensor data. The simulation results demonstrate that the adaptive polynomial sensor fusion approach outperforms several state-of-the-art sensor fusion techniques, particularly in scenarios in which the sensor data are inconsistent or interrupted. The article concludes that the proposed approach offers an effective solution to the problem of inconsistent and interrupted sensor data in sensor fusion, with potential applications in various fields including robotics, autonomous vehicles, and healthcare.
A Model for Cognitively Valid Lifelong Learning

In continual learning, usually a sequence of tasks are given to a learning agent and the performance of the agent after learning is measured in terms of resistance to catastrophic forgetting, efficacy of knowledge transfer and overall performance on the individual tasks. On the other hand, in multi-task learning, the system is designed to simultaneously acquire knowledge in multiple tasks, often through offline batch learning. A more cognitively valid scenario for lifelong robot learning would be to have a robotic agent to autonomously decide which task to engage and disengage while leveraging many-to-many knowledge transfer ability among tasks during online learning. In this study, we propose a novel lifelong robot learning architecture to fulfill the aforementioned desiderata, and show its validity in an environment where a robot learns the effects of its actions in different task settings. To realize the proposed model, we adopt learning progress measure for task selection, and have the tasks learn by independent neural networks with special structure that allows access to the neural layers of the non-selected tasks. The experiments conducted with a simulated robot arm in an object interaction scenario show that the proposed architecture yields better knowledge transfer and facilitates faster learning compared to baselines of fixed sequence task learning and isolated task learners with no knowledge transfer.
Multi Agent Coordination Strategy for Collaborative Exploration of GPS-denied Environments

Autonomous exploration and mapping of unknown subterranean areas using fleet of Micro Aerial Vehicles (MAVs) have achieved great interest in the past decade. The application of such research direction is extended beyond mining industry for example the exploration of potential lava tubes and caves on Mars using Martian MAVs. In this article we present a Collaborative Exploration and Mapping approach for exploration of large scale multi-branched virtual cave environment using multiple MAVs. An important aspect of such approach is to establish a decent collaboration with minimal information exchange among agents such that the exploration area can be maximized by preventing multiple agents covering overlapping parts of the environment. The collaborative exploration approach in this article is based on sharing local traversable frontier points as information among agents such that the each agent can identify the target areas in order to extend global map in an efficient manner. This work proposes a novel baseline collaborative exploration approach for optimal target area selection with a resource constrained MAVs in order to efficiently as well as rapidly explore subterranean environment. Extensive simulations are performed to validate the proposed collaborative strategy in order to establish a multi agent coordinated exploration-coverage mission.
ANMIP: Adaptive Navigation based on Mutual Information Perception in Uncertain Environments

Navigation in uncertain environment has become a hot research topic. Some path planning algorithms have been proposed to address the uncertainty problem, such as Canadian Traveller’s Problem (CTP) algorithm. However, these algorithms usually require accurate environmental information, which is often difficult to obtain. In addition, their decisions are based on robot current perceptions and the previous navigation experience is usually ignored, whereas the past experience is an important reference for people to optimize policies. In order to address these issues, we propose a mutual information perception based navigation method for efficient navigation in uncertain environment. First, an efficient CTP solver is proposed to quickly generate policy based on the obtained environment information. Second, a topological map representation method is proposed for map decomposition. In order to perceive environmental information, a block judgment interface module is proposed. Then, the door edge resolver algorithm is proposed to absorb the experience of the previous navigation. Finally, we design a complete information updating mechanism based on Wilson confidence interval, so that the robot can update its perception of the environment and realize adaptive navigation in uncertain environments. The experimental results show that by comparing with the existing move_base navigation system, our method has better performance in average navigation cost and navigation success rate.
Double-Layer Multi-robot Path Planner based on Deep Multiple Dueling Network Agent

With its advantages of high efficiency and robustness, the multiple robots system has better application prospects in dynamic and complex environments, such as smart factories. However, when the number of robots increases, it is critical to efficiently coordinate congestion and jams among robots. To address this problem, we design the Double-Layer Multi-robot Path Planner to coordinate the robot conflicts. In the Collaboration Layer, we propose a Deep Multiple Dueling Network Agent( DMDNA) for node distribution based on the idea of forward node distribution to coordinate the path conflict problem of multiple robots. In DMDNA, we design the Multiple Dueling Network(MDN) model based on the DDQN network structure to address the high dimensional discrete action space problem in multiple robots. In addition, to overcome the characteristics of discrete and sparse rewards of reinforcement learning, we add the Hindsight Experience Replay(HER) experience replay strategy in the training of DMDNA. The experience of primary sequences is cut and reused to improve the utilization of samples. In the Motion Layer, we propose the Adaptive-DWA(A-DWA) algorithm, which adds the Target Function. According to the Target Function, the forward simulation time and weight of the evaluation function can be dynamically changed to improve the efficiency of local path planning and dynamic obstacle avoidance. The final experimental results show that the Double-Layer Multi-robot Path Planner based on DMDNA has better success rate and time cost than traditional reinforcement learning methods in solving multi-robot path planning problems. Relevant experimental code: https://github.com/WILL-ZHAO-1/DMDNACODE.git.