Reviewerly/exHarmony huggingface dataset:
A One-Shot Learning Method for Human Activity Recognition Using Extracted Essential Poses from Push-Fixed Adaptive Resonance Theory

The utilization of Human Activity Recognition (HAR) through surveillance cameras has the potential to enhance the daily monitoring of elderly individuals living alone. However, existing research in this area predominantly relies on deep learning methods that necessitate large amounts of data for effective training. The acquisition of new data on older adults can be both challenging and costly. To address these limitations, this paper proposes a one-shot learning HAR method that extracts human poses from video data. The study introduces three key techniques: firstly, the PF-ART algorithm is employed to identify the essential poses for each activity. Secondly, a scoring system is designed to provide similarity scores for input activities. Finally, two decision rules are established to distinguish between upper and lower activities. The proposed system is evaluated using the KTH dataset, and the obtained results demonstrate its superior performance compared to previous methods.
Voice Conversion for Stuttered Speech, Instruments, Unseen Languages and Textually Described Voices

Voice conversion aims to convert source speech into a target voice using recordings of the target speaker as a reference. Newer models are producing increasingly realistic output. But what happens when models are fed with non-standard data, such as speech from a user with a speech impairment? We investigate how a recent voice conversion model performs on non-standard downstream voice conversion tasks. We use a simple but robust approach called k-nearest neighbors voice conversion (kNN-VC). We look at four non-standard applications: stuttered voice conversion, cross-lingual voice conversion, musical instrument conversion, and text-to-voice conversion. The latter involves converting to a target voice specified through a text description, e.g. “a young man with a high-pitched voice”. Compared to an established baseline, we find that kNN-VC retains high performance in stuttered and cross-lingual voice conversion. Results are more mixed for the musical instrument and text-to-voice conversion tasks. E.g., kNN-VC works well on some instruments like drums but not on others. Nevertheless, this shows that voice conversion models – and kNN-VC in particular – are increasingly applicable in a range of non-standard downstream tasks. But there are still limitations when samples are very far from the training distribution. Code, samples, trained models: https://rf5.github.io/sacair2023-knnvc-demo/ .
TinyMM: Multimodal-Multitask Machine Learning on Low-Power MCUs for Smart Glasses

Maintaining a sufficient battery life-time is crucial for an everyday wearable instrumented frame. Therefore, the use of a low-power integrated circuit, such as a microcontroller unit (MCU) is essential. However, MCUs are hardware and software limited (e.g., memory limitations, limited multiprocessing capabilities, real-time constraints, etc.). These limitations restrict the efficient execution of multiple machine learning inferences within the frame. In this regard, we propose in this work, a multimodal multitask machine learning approach for low power micro controllers, called TinyMM. Our proposal has been assessed through a real-world use case for smart glasses, which involves processing data from an accelerometer and a digital microphone to perform three distinct applications: (1) detecting the wearer's activity, (2) identifying the wearer's mode of transportation and (3) locating the wearer's scene. Furthermore, when compared to standard approaches, our proposal showed a 2.7 times reduction in the inference energy and runtime, as well as a 2.1 times reduction in the required storage memory footprint.
Adaptive Resonance Theory-Based Topological Clustering with Node Deletion Mechanism for Evolving Stream Data

Concept drift where the data distribution changes over time deteriorates the performance of machine learning models. Adaptive Resonance Theory (ART)-based clustering algorithms are capable of learning continually and achieving fast and stable clustering. However, concept drift has not been actively discussed in ART-based clustering algorithms. In this paper, we propose an ART-based clustering algorithm that can deal with concept drift. Through computational experiments, we confirmed the performance of the proposed algorithm in synthetic and real-world datasets.
Metafoemer Super-Resolution Network with Recursive Gated Attention for the Meteorological Satellite Cloud Image

Increasing the resolution of satellite cloud images with super-resolution (SR) methods can help meteorological systems process geographic information more accurately. In this paper, we verify several SR methods on the natural color cloud image (NCCI) dataset. On one hand, we find that the residuals between the interpolated up-sampled image and the ground truth are beneficial to the SR effect of the satellite cloud image. On the other hand, we propose the MetaFormer super-resolution network (MetaSR), which adopts the MetaFormer architecture to extract satellite cloud features more efficiently. The recursive gated convolution and the channel-space attention are also embedded in MetaSR to achieve arbitrary-order spatial interactions and feature filtering. The experimental results indicate that MetaSR achieves excellent SR results for satellite cloud images on the NCCI dataset. In detail, the MetaSR takes only 34% parameters of EDSR and improves the PSNR by about 0.045dB on NCCI for ×4 SR.
Skeleton Based Hand Missing Score Evaluation System

In this paper, a skeleton-based method is proposed for evaluating hand defects. Specifically, we take use of the existing hand skeleton extraction method and correct the extracted skeleton points, making them applicable for a hand with defects. Then, we combine the method above with the existing human hand defect evaluation system to automatically evaluate the concrete situation of the hand. The proposed method is tested on a dataset constructed by ourselves, which includes various cases where fingers and palm are missing. Experimental results illustrate that our method achieves excellent performance and can be applied to scenarios where the hand is partially missing with high accuracy. Our method can be helpful to forensic agencies in detecting and evaluating hand mutilations.
Bridging Resource Prediction and System Management: A Case Study in Cloud Systems

In recent years, there has been a significant amount of research focused on predicting resources in order to enhance the performance of cloud systems. Many researchers believe that the more accurate the prediction, the more effective resource management will be in ensuring reliable performance. However, our study in this paper demonstrates that there is a gap between resource demand prediction and system performance. Furthermore, our experiment results have demonstrated that the accurate and fine-grained prediction helps to achieve a more reliable and efficient system performance, especially in CPU utilization rate.
A Critical Study of Few-Shot Learning for Encrypted Traffic Classification

Over the past twenty years, a plethora of methods have been proposed for encrypted traffic classification (ETC), while the Server name indication (SNI) is deemed to solve the problem of classification for TLS traffic. However, SNI-based classification has its pitfalls and the SNI will likely be pushed into the encrypted tunnel in the future. In this work, we envision a futuristic scenario in which encrypted SNI is the norm and labeled traffic flows are scarce. In such settings, we tackle the problem of traffic classification at ISP level using few-shot learning. By means of six real-world ISP-level datasets collected between 2019 and 2021 and two publicly available client-side datasets, we study the performance of a few-shot learner on TLS data, including its cross-dataset generalizability. We further investigate the effect of the number of required labeled samples on the learner's performance. Our experiments show that the dataset-specificity of deep learners carries over to few-shot meta-learning, and calls for addressing the problem of generalizability for deep learning architectures.
Integrated Multiscopic CPSS For Physical and Cognitive Measurement in an Independent Block-Design Test

This paper presents an integrated multiscopic cyber-physical-social system (CPSS) for evaluating physical and cognitive aspects in an independent block-design test (BDT). First, we utilize a hand tracker to extract physical features at the microscopic level, such as hand posture during manipulation. Second, an egocentric vision incorporating an eye tracker describes perceptual representation at the mesoscopic level. This system represents hand-eye coordination by measuring the distance between the hand and the center of visual attention during the test. Thirdly, an upper table vision system uses color feature recognition in each block to evaluate cognition at the macroscopic level. This system detects the blocks and recognizes that the design matches the task given. By evaluating eight designs of WAIS-IV BDT, our system has demonstrated its ability to measure physical and cognitive variables during the BDT successfully. Additional research and development are required to support BDT development in the future. Code and datasets are available online at https://github.com/anom-tmu/bdt-multiscopic.
Robust Bayesian Abstraction of Neural Networks

Much recent research has focused on constructing abstractions of complex, high-dimensional deep neural networks, with the expectation that the abstractions are as close as possible to the original neural network while being interpretable and preservative of key properties. We extend a method proposed by Berthier et al. [1] that abstracts the latent representations of a Deep Neural Network as a Bayesian Network (BN). To this end, we first construct a classifier based on the abstract BN by including a new node and some edges from the nodes of the BN to the new node. We then analyse the robustness of the resulting BN prediction models using a test dataset with crafted perturbations. Empirical experiments confirm that traditional BN inference approximates the Deep Neural Network prediction performance while outperforming it in an adversarial setting. Since the created classifiers are abstractions of the original neural network, our results enable the construction of meaningful abstractions with decent accuracy and robustness.
Mitigating Signaling Storms in 5G with Blockchain-assisted 5GAKA

This paper examines the registration signaling storm attack in 5G networks. This attack is initiated by massive registration requests and targets the 5G core network functions, leading to large-scale user service disruption. To mitigate this problem, we review the 5G Authentication and Key Agreement (5GAKA) protocol and highlight the impact of the exposure of the 5G core network (CN) to massive registration requests. We propose a blockchain-based authentication protocol to effectively block adversarial registration requests and secure the 5G network at a small cost. Our experiments reveal that our protocol is resistant to attacks and prove its superiority compared to a baseline mitigation approach.
Preliminary Results on Exploring Data Exhaust of Consumer Internet of Things Devices

In this paper, we apply a machine learning classifier to the publicly available consumer Internet of Things (IoT) traffic traces to explore the nature and extent of any potential data exhaust. To this end, we propose two feature sets and compare them against the baseline flow feature set and the results from the previous works. Evaluations show the improvement in performance obtained using the proposed feature sets and the variety of information that can be extracted from the captured IoT traffic regardless of encryption.
Multi-Label Oversampling Based on Borderline

Learning from imbalanced multi-label data is a challenging task. Resampling is one of the most commonly used techniques for handling the imbalance issue by preprocessing the multi-label dataset. Previous studies in traditional single-label learning show that the minority samples located at the borderline are easy to be misclassified, thus cloning them can help improving the performance of the classifier. However, the importance of these samples are overlooked in existing multi-label resampling methods. This paper proposes a multi-label oversampling method based on borderline. We provide a delineation of the minority borderline samples based on their nearest neighbors, and propose a dangerous rate to reflect the risk of a borderline sample being misclassified, which is used to determine the frequency of cloning. Experimental results demonstrate that the proposed method is competitive, which significantly enhances the classification performance.
TinyG: Accurate IP Geolocation Using a Tiny Number of Probers

IP geolocation is essential for various applications. However, the reliability of IP geolocation databases has been proven to be inadequate. In recent years, the growing number of public probers has offered the potential for more accurate geolocation results through active measurement. The conventional practice is to probe the target IP address using all available probers and feed the measurement results to the active geolocation method. However, this practice is cost-inefficient and may trigger the anti-flood mechanism. Moreover, public probers typically impose user-level limits on the frequency and quantity of measurements. Therefore, it is important to reduce the average number of probers (ANP) selected for successfully probing each target. Researchers have discovered that geolocation accuracy primarily depends on the minimum delay between probers and the target. Inspired by that, we propose TinyG, a prober selection algorithm designed to reduce the ANP needed to find probers within a sufficiently small delay from the target. TinyG divides the probing process into multiple rounds and leverages previous measurement results to guide the selection of probers for subsequent rounds. Experimental results show that when the associated minimum delay is within 2 ms, various active geolocation methods can provide credible geolocation results. TinyG outperforms other algorithms in reducing the ANP needed to obtain credible results. Compared to using more than 1,300 probers, TinyG can achieve an ANP of 6.7 with only a 6% coverage loss of credible results.
A Testbed for Automating and Analysing Mobile Devices and Their Applications

The need for improved network situational awareness has been highlighted by the growing complexity and severity of cyber-attacks. Mobile phones pose a significant risk to network situational awareness due to their dynamic behaviour and lack of visibility on a network. Machine learning techniques enhance situational awareness by providing administrators insight into the devices and activities which form their network. Developing machine learning techniques for situational awareness requires a testbed to generate and label network traffic. Current testbeds, however, are unable to automate the generation and labelling of realistic network traffic. To address this, we describe a testbed which automates applications on mobile devices to generate and label realistic traffic. From this testbed, two labelled datasets of network traffic have been created. We provide an analysis of the testbed automation reliability and benchmark the datasets for the task of application classification.
Clustering Analysis of a Spatiotemporal Dataset with a Novel Kernel Density Estimator

A vast number of spatiotemporal datasets collected from a wide range of sources has motivated scientists to develop effective approaches to identify interesting patterns hidden in these datasets. In this respect, kernel density estimators, which belong to a class of non-parametric estimators in statistics, have been widely exploited in recent years. With this background, we have developed a novel kernel density estimator aiming to provide accurate analysis results. According to the evaluation with a real spatiotemporal dataset, which collected emergency medical service records in a county in the United States, the proposed kernel density estimator can approximate the probability density function significantly more accurately than a conventional kernel density estimator. Furthermore, we have exploited the proposed kernel density estimator to identify interesting patterns hidden in the real spatiotemporal dataset.
A Privacy-Aware Federated Learning Framework for Distributed Energy Resource Analytics in Constrained Environments

To be resilient against extreme weather events, the rural communities in Puerto Rico are leveraging distributed energy resources (DER). However, computing frameworks sup-porting the grid in critical decision-making are still largely centralized. Sensitive consumer data are transmitted over the Internet or cellular networks to a secondary or tertiary node. It guarantees better situational awareness at the cost of a wider attack surface, jeopardizing user privacy, as more DER come online. Cloud, Edge, and Fog computing all require data aggregation at some level. This paper introduces a privacy-aware federated learning framework that leverages the Fog model by pushing analytics all the way to the DER and load assets. These local models train on individual asset data and transmit only learned parameters (such as weights) over secure communications to a global decision-maker. By abstracting personally identifiable consumer data without impacting decision optimality, this framework better aligns with distributed power generation paradigm.
Data Driven Fuzzy and Neural Dynamic Systems Modeling

The paper addresses data driven fuzzy modeling and long-short term memory structures, and evaluate their performance in modeling nonlinear dynamic systems. These are representative and powerful modeling paradigms representative of the state of the art of fuzzy and deep neural modeling approaches. Models complexity and approximation capabilities are evaluated, and their usefulness is discussed in front of the predictions they produce. Overall, the results suggest that they are competitive with alternative methods such as convolutional and Gaussian process.
Fine-Tuned Heterogeneous Graph Convolutional Network Embedding

Heterogeneous network representation learning, which can understand and capture the complex interactions between different nodes and edges in heterogeneous networks, is a research field that has garnered widespread attention. However, existing models often overlook the varying effects of meta-paths on target nodes, even when they share the same relationship type. This limitation makes it difficult to accurately represent heterogeneous structures. To solve this issue, we introduce FHGCN, a novel Fine-tuned Heterogeneous Graph Convolutional Network that leverages structural information and attribute semantics to enrich the learned node embedding and improve the final performance. The model consists of two parts: A fine-tuned multi-relation aggregation module which automatically learns the weights of heterogeneous relations and further deduces the significance of meta-paths by considering the distinct fluctuation in the impact of meta-paths generated by the same relation. And a multilayer convolutional module which can learn valuable interactions between heterogeneous meta-paths of different lengths to obtain favorable node representation node embedding. Node classification experiments on two heterogeneous graph datasets show that FHGCN has a good predictive ability.
DTL-IDS: Deep Transfer Learning-Based Intrusion Detection System in 5G Networks

In the complex landscape of modern networks, the necessity of Intrusion Detection System (IDS) has become paramount. An IDS is a crucial cybersecurity tool that plays a pivotal role in safeguarding networks against a wide array of threats and attacks. The application of deep learning models for intrusion detection is becoming popular among research communities due to its success in many other domains. However, deep learning models require a significant amount of labeled data to achieve effective training. Obtaining labeled data for intrusion detection can be challenging and costly. To address it, Deep Transfer Learning (DTL) can be employed. This research introduces an innovative traffic classification method tailored for 5G networks. The approach leverages deep transfer learning by utilizing pre-trained models and fine-tuning them. We evaluate several deep-learning models in a transfer learning setting. The Inception model being identified as the top-performing model shows an improvement of approximately 10% in terms of F1-score between IDS-based DTL and the same scheme without DTL.
An Asymmetric Radar-Camera Fusion Framework for Autonomous Driving

Object detection plays a pivotal role in achieving reliable and accurate perception for autonomous driving systems, encompassing tasks such as estimating object location, size, category, and other features from sensory inputs. The prevailing sensor modalities in this domain are LiDAR, camera, and radar, with multi-modal fusion being widely acknowledged as a means to optimize object detection outcomes. Among these modalities, camera and radar exhibit complementary characteristics and hold the potential for comprehensive object profile recognition. Furthermore, they are cost-effective compared to LiDAR solutions. However, the orthogonal nature of camera and radar data poses significant challenges for effective fusion. This paper introduces a novel framework that integrates camera and radar inputs to enhance perception robustness. Our approach involves fusing 2D detection proposals derived from camera imagery and radar points, enabling reliable object detection. To support this fusion framework, we present a calibration algorithm and demonstrate its performance through extensive evaluation on real-world dataset.
Feature Enhancement Based Multi-Feature Fusion Network for Video Anomaly Detection in Offshore Surbeillance

With the advancement of deep learning techniques and the widespread use of surveillance systems, there is an increasing demand for detecting anomalous events in offshore video scenes. This paper proposes a novel Multi-Feature fusion network (MFFN) based on video features. Firstly, we enhance the information entropy feature extracted from the video to generate the Video-Momentum-Feature (VMF). The VMF enlarges the difference between the entropy feature of normal and anomalous videos, and allows the network to focus on the most anomalous parts of the video. Moreover, the network employs I3D to extract the RGB and Optical flow characteristics separately. Then, the VMF is fused with these RGB and Optical flow characteristics, respectively. Finally, the fused features are utilized for anomaly detection. Experimental results on the modified UCF -Crime dataset and the offshore ferryboat dataset demonstrate that our proposed method achieves significant performance.