Reviewerly/exHarmony huggingface dataset:
Vessel Detection from Optical Remote Sensing Images with Deep Learning Methods

Vessel detection from remote sensing images is becoming exponentially crucial component in marine surveillance applications including maritime traffic control, anti-illegal fishing applications, oil discharge control, marine pollution and safety. Applying deep learning methods to vessel detection applications ineluctably improve the detection results and overcome unforeseen errors that could be made by analysts. Publicly available datasets play vital role for development and evaluation process of deep learning models. In this paper, open source DOTA dataset has been revised and trained with single-staged deep learning methods. The results show that YOLOv8 model has the most efficient value on detecting ships and fastest to detect instances from given test images on inference.
Feature Engineering in Machine Learning-Based Intrusion Detection Systems for OT Networks

This paper evaluates the importance of feature exploration and engineering when applying machine learning for intrusion detection in OT (Operational Technology) networks. Data used consisted of raw network traffic captures from a simulated OT environment communicating over the Modbus/TCP protocol. Feature engineering efforts identified thirty eight attributes of interest at the different layers of the network stack. The Random Forest algorithm was used to analyze the importance of each feature for the detection of anomalous network behavior. Both supervised and unsupervised learning methods were evaluated including Random Forest, Support Vector Machines, K-Nearest Neighbors, K-Means Clustering, and Isolation Forest. Results indicate that statistical based features as well as features derived from the protocol and application layers contained information best suited for detecting anomalous OT behavior. Additionally, variable importance-based feature selection helped reduce complexity and improved detection rate when compared with models trained on the original high dimensional data. Random Forest and Support Vector Machines had the best detection performance but required a large amount of labeled data for training and validation. Notably, Isolation Forest shows potential for anomaly detection in OT networks as it requires no labeled data and produced promising results.
KissLoc: A Spatio-temporal Kissing Recognition System Using Commercial Smart Glasses

In this paper, we propose KissLoc: a system that leverages onboard micro-size sensors of consumer eyewear devices for the dual purpose of activity recognition and localization. Specifically, the system trains a deep learning model for recognizing kissing activity and simultaneously identifying the timestamped location of its occurrence. Consequently, several predefined actions could be taken, including logging or controlling the smart environment. The evaluation shows that KissLoc can recognize the kissing activity with 82% accuracy while locating its occurrence with a median localization error of 1.25m.
Qkd@Edge: Online Admission Control of Edge Applications with QKD-secured Communications

Quantum Key Distribution (QKD) enables secure communications via the exchange of cryptographic keys exploiting the properties of quantum mechanics. Nowadays the related technology is mature enough for production systems, thus field deployments of QKD networks are expected to appear in the near future, starting from local/metropolitan settings, where edge computing is already a thriving reality. In this paper, we investigate the interplay of resource allocation in the QKD network vs. edge nodes, which creates unique research challenges. After modeling mathematically the problem, we propose practical online policies for admitting edge application requests, which also select the edge node for processing and the path in the QKD network. Our simulation results provide initial insights into this emerging topic and lead the way to upcoming studies on the subject.
A Novel Context Aware Paths Recommendation Approach for the Cultural Heritage Enhancement

The will to travel leads humans to discover new places and enjoy new adventures. However, tourists usually need help knowing what to visit and, avoiding time issues, in which order to explore several Points of Interest (POIs). In this field, new technologies can help tourists to improve their experiences and select the visiting path according to personal preferences. Therefore, the employment of Recommender Systems allows the personalization of the experience through the appropriate POIs’ selection. Moreover, RSs’ analysis could take advantage of contextual information that suits the personalization in the specific environment where the elaboration happens, providing users with even more specific and tailored paths. This paper aims to design personalized visiting paths combining a Context-Aware Recommender System (CARSs) and a mathematical model to maximize the number of visited POIs in the available time. The proposed approach is tested through a prototype, obtaining promising results.
Optimizing IoT-based Human Activity Recognition on Extreme Edge Devices

Wearable Internet of Things (IoT) devices with inertial sensors can enable personalized and fine-grained Human Activity Recognition (HAR). While activity classification on the Extreme Edge (EE) can reduce latency and maximize user privacy, it must tackle the unique challenges posed by the constrained environment. Indeed, Deep Learning (DL) techniques may not be applicable, and data processing can become burdensome due to the lack of input systems. In this paper, we address those issues by proposing, implementing, and validating an EE-aware HAR system. Our system incorporates a feature selection mechanism to reduce the data dimensionality in input, and an unsupervised feature separation and classification technique based on Self-Organizing Maps (SOMs). We developed the system on an M5Stack IoT prototype board and implemented a new SOM library for the Arduino SDK. Experimental results on two HAR datasets show that our proposed solution is able to overcome other unsupervised approaches and achieve performance close to state-of-art DL techniques while generating a model small enough to fit the limited memory capabilities of EE devices.
Elixir: A System to Enhance Data Quality for Multiple Analytics on a Video Stream

IoT sensors, especially video cameras, are ubiquitously deployed around the world to perform a variety of computer vision tasks in several verticals including retail, health-care, safety and security, transportation, manufacturing, etc. To amortize their high deployment effort and cost, it is desirable to perform multiple video analytics tasks, which we refer to as Analytical Units (AUs), off the video feed coming out of every camera. As AUs typically use deep-learning based AI/ML models, their performances depend on the quality of the input video. The most recent work has shown that dynamically adjusting the camera setting exposed by popular network cameras can help improve the quality of the video feed and hence the AU accuracy, in a single AU setting. In this paper, we first show that in a multi-AU setting, changing the camera setting has disproportionate impact on different AUs performance. In particular, the optimal setting for one AU may severely degrade the performance for another AU, and further, the impact on different AUs varies as the environmental condition changes. We then present Elixir, a system to enhance the video stream quality for multiple analytics on a video stream. Elixir leverages Multi-Objective Reinforcement Learning (MORL), where the RL agent caters to the objectives from different AUs and adjusts the camera setting to simultaneously enhance the performance of all AUs. To define the multiple objectives in MORL, we develop new AU-specific quality estimator values for each individual AU. We evaluate Elixir through real-world experiments on a testbed with three cameras deployed next to each other (overlooking a large enterprise parking lot) running Elixir and two baseline approaches, respectively. Elixir correctly detects 7.1% (22,068) and 5.0% (15,731) more cars, 94% (551) and 72% (478) more faces, and 670.4% (4975) and 158.6% (3507) more persons than the default-setting and time-sharing approaches, respectively. It also detects 115 license plates, far more than the time-sharing approach (7) and the default setting (0).
Knowledge-embedded Prompt Learning for Zero-shot Social Media Text Classification

Social media plays an irreplaceable role in shaping the way information is created shared and consumed. While it provides access to a vast amount of data, extracting and analyzing useful insights from complex and dynamic social media data can be challenging. Deep learning models have shown promise in social media analysis tasks, but such models require a massive amount of labelled data which is usually unavailable in real-world settings. Additionally, these models lack common-sense knowledge which can limit their ability to generate comprehensive results. To address these challenges, we propose a knowledge-embedded prompt learning model for zero-shot social media text classification. Our experimental results on four social media datasets demonstrate that our proposed approach outperforms other well-known baselines.
E-ADDA: Unsupervised Adversarial Domain Adaptation Enhanced by a New Mahalanobis Distance Loss for Smart Computing

In smart computing, the labels of training samples for a specific task are not always abundant. However, the labels of samples in a relevant but different dataset are available. As a result, researchers have relied on unsupervised domain adaptation to leverage the labels in a dataset (the source domain) to perform better classification in a different, unlabeled dataset (target domain). Existing non-generative adversarial solutions for UDA aim at achieving domain confusion through adversarial training. The ideal scenario is that perfect domain confusion is achieved, but this is not guaranteed to be true. To further enforce domain confusion on top of the adversarial training, we propose a novel UDA algorithm, E-ADDA, which uses both a novel variation of the Mahalanobis distance loss and an out-of-distribution detection subroutine. The Mahalanobis distance loss minimizes the distribution-wise distance between the encoded target samples and the distribution of the source domain, thus enforcing additional domain confusion on top of adversarial training. Then, the OOD subroutine further eliminates samples on which the domain confusion is unsuccessful. We have performed extensive and comprehensive evaluations of E-ADDA in the acoustic and computer vision modalities. In the acoustic modality, E-ADDA outperforms several state-of-the-art UDA algorithms by up to 29.8%, measured in the f1 score. In the computer vision modality, the evaluation results suggest that we achieve new state-of-the-art performance on popular UDA benchmarks such as Office-31 and Office-Home, outperforming the second best-performing algorithms by up to 17.9%.
Multi-modal AI Systems for Human and Animal Pose Estimation in Challenging Conditions

This paper explores the development of multi-modal AI systems for pose estimation in challenging conditions for both humans and animals. Existing single-modality approaches struggle in challenging scenarios such as emergency response and wildlife observation due to factors like smoke, low light, obstacles, and long-distance observations. To address these challenges, this research proposes integrating multiple sensor modalities and leveraging the strengths of different sensors to enhance accuracy and robustness in pose estimation.
Vision Transformer-based Real-Time Camouflaged Object Detection System at Edge

Camouflaged object detection is a challenging task in computer vision that involves identifying objects that are intentionally or unintentionally hidden in their surrounding environment.Vision Transformer mechanisms play a critical role in improving the performance of deep learning models by focusing on the most relevant features that help object detection under camouflaged conditions.In this paper, we utilized a vision transformer (VT) in two phases, a) By integrating VT with a deep learning architecture for efficient monocular depth map generation for camouflaged objects and b) By embedding VT multiclass object detection model with multimodal feature input (RGB with RGB-D) that increases the visual cues and provides more representational information to the model for performance enhancement.Additionally, we performed an ablation study to understand the role of the vision transformer in camouflaged object detection and incorporated GRAD-CAM on top of the model to visualize the performance improvement achieved by embedding the VT in the model architecture.We deployed the model on resource-constrained edge devices for real-time object detection to realistically test the performance of the trained model.
An Online Continuous Semantic Segmentation Framework With Minimal Labeling Efforts

The annotation load for a new dataset has been greatly decreased using domain adaptation based semantic segmentation, which iteratively constructs pseudo labels on unlabeled target data and retrains the network. However, realistic segmentation datasets are often imbalanced, with pseudo-labels tending to favor certain "head" classes while neglecting other "tail" classes. This can lead to an inaccurate and noisy mask. To address this issue, we propose a novel hard sample mining strategy for an active domain adaptation based semantic segmentation network, with the aim of automatically selecting a small subset of labeled target data to fine-tune the network. By calculating class-wise entropy, we are able to rank the difficulty level of different samples. We use a fusion of focal loss and regional mutual information loss instead of cross-entropy loss for the domain adaptation based semantic segmentation network. Our entire framework has been implemented in real-time using the Robotics Operating System (ROS) with a server PC and a small Unmanned Ground Vehicle (UGV) known as the ROSbot2.0 Pro. This implementation allows ROSbot2.0 Pro to access any type of data at any time, enabling it to perform a variety of tasks with ease. Our approach has been thoroughly evaluated through a series of extensive experiments, which demonstrate its superior performance compared to existing state-of-the-art methods. Remarkably, by using just 20% of hard samples for fine-tuning, our network has achieved a level of performance that is comparable (≈88%) to that of a fully supervised approach, with mIOU scores of 60.51% in the In-house dataset.
µ-FF: On-Device Forward-Forward Training Algorithm for Microcontrollers

Deliver intelligence into low-cost hardware e.g., Microcontroller Units (MCUs) for the realization of low-power tailored applications nowadays is an emerging research area. However, the training of deep learning models on embedded systems is still challenging mainly due to their low amount of memory, available energy, and computing power which significantly limit the complexity of the tasks that can be executed, thus making impossible use of traditional training algorithms such as backpropagation (BP). During these years techniques such as weights compression and quantization have emerged as solutions, but they only address the inference phase. Forward-Forward (FF) is a novel training algorithm that has been recently proposed as a possible alternative to BP when the available resources are limited. This is achieved by training the layers of a neural network separately, thus reducing the required energy and memory. In this paper, we propose µ-FF, a variation of the original FF which tackles the training process with a multivariate Ridge regression approach and allows to find closed-form solution by using the Mean Squared Error (MSE) as loss function. Such an approach does not use BP and does not need to compute gradients, thus saving memory and computing resources to enable the on-device training directly on MCUs of the STM32 family. Experimental results conducted on the Fashion-MNIST dataset demonstrate the effectiveness of the proposed approach in terms of memory and accuracy.
HeteroSys: Heterogeneous and Collaborative Sensing in the Wild

Advances in Internet-of-Things, artificial intelligence, and ubiquitous computing technologies have contributed to building the next generation of context-aware heterogeneous systems with robust interoperability to control and monitor the environmental variables of smart environments. Motivated by this, we propose HeteroSys, an end-to-end multi-functional smart IoT-based system prototype for heterogeneous and collaborative sensing in a smart IoT-based environment. A unique characteristic of HeteroSys is that it relies on Home Assistant (HA) to collate heterogeneous sensors (e.g., passive infrared sensors (PIR), reed (door) switches, object tags, wearable wrist-mounted, water leak sensors, and internet protocol cameras), and uses a variety of networking protocols such as Zigbee open standard for mesh networking, WiFi, and Bluetooth Low Energy (BLE) for communication. The reliance on HA (and its broad community support) makes HeteroSys ideal for various applications such as object detection, human activity recognition and behavior patterns. We articulated the development phase, integration, testing challenges and evaluation of the HeteroSys. We conducted an extensive 24-hour longitudinal data collection from 5 participants performing 6 activities by deploying in an indoor home environment. Our assessment of the acquired dataset reveals that the representations learned using deep learning architecture aid in improving the detection of activities to 83.1% accuracy.
Phishing Email’s Detection Using Machine Learning and Deep Learning

In today’s world, all of us are dependent on emails. Emails are a very efficient and fast way of sending a message to someone. But malicious users often use it to send fraudulent emails with fake links that steal user credentials like credit card details, login-id, passwords, etc. These emails are called phishing emails. These emails constitute identity fraud as the emails are interpreted to be from banks or other multinational companies. Many existing solutions require the user to check for grammar errors, check the email-id, or avoid clicking any links. But all these actions require human involvement. In this paper, I have implemented and compared current Machine Learning and Deep Learning techniques used with Natural Language Processing to detect phishing emails and achieved an accuracy of 98%.
Malayalam Handwritten Character Recognition using Transfer Learning and Fine Tuning of Deep Convolutional Neural Networks

In the digitization of Malayalam handwritten documents, recognition of handwritten characters is a difficult task. This is due to the non availability of a labeled benchmark Malayalam handwritten character dataset. The state of the art technique using deep convolutional neural networks demands large amount of labeled dataset. Therefore, this paper aims to develop a pre-trained convolutional neural network (CNN) model for recognizing Malayalam handwritten characters using small sized dataset. Two approaches namely transfer learning and fine tuning of pre-trained Deep Convolutional Neural Network (DCNN) architecture ResNet50 are used to develop models for recognizing Malayalam handwritten characters. Model design is optimized by varying parameters like learning rate, batch size and optimization algorithm. From the experiments, it is found that highest testing accuracy of 78.05% is obtained for the model using fine tuning approach when it is trained with a batch size of 16 using RMSProp optimization algorithm and a learning rate of 0.000001. A testing accuracy of 78.05% is obtained with ResNet50 for binary images even though ResNet50 is pre-trained using colour images.
ArmVision: An Approach to Improve Police Officers Response Times to Gun Violence and Put a Stop to Armed Crimes Before They Occur

An estimated 19,223 people lost their lives due to gun violence in 2020. ArmVision prevents arm related crimes before they occur and brings quicker attention to them if they do with its efficient notification system. Its wide range of applicability expresses a bright future for its development. ArmVision uses a complex machine learning (ML) algorithm called You Only Look Once (YOLO) and Convolutional Neural Networks (CNNs) to optimize its performance.
Duplicate Quora Questions Pair Detection using Siamese Bert and Ma-LSTM

One of the most well-known online communities for question and answer exchanges is the Quora platform, with millions of users asking and answering questions on a wide range of topics. However, a major issue faced by the Quora community is the high quantity of questions that are duplicates that are posted on the platform. These duplicate questions not only clutter the platform but also affect the quality of content, making it difficult for users to find relevant information. Hence, there is a need to automatically identify and remove duplicate question pairs in the Quora community. Duplicate question pair detection is a a difficult issue because of the considerable fluctuation and complexity of natural language. Traditional rule-based approaches are often insufficient for capturing the nuanced meaning and context of questions. Therefore, machine learning-based methods have gained popularity in recent years for detecting duplicate question pairs. This paper proposes a framework for detecting duplicate question pairs on the Quora platform using Siamese Neural Network, BERT, MaLSTM, and BiLSTM models. Each model's effectiveness is evaluated using a variety of evaluation criteria, including accuracy, precision, recall, and F1-score, on a dataset of Quora question pairs. The experimental outcomes demonstrate that the proposed framework detects duplicate question pairs with high accuracy. with the BERT model outperforming the other models in terms of overall performance. This suggests that pretrained transformer networks can effectively capture the semantic meaning of questions and enhance the performance of duplicate question pair detection