Reviewerly/exHarmony huggingface dataset:
Demo: The Future of Dog Walking – Four-Legged Robots and Augmented Reality

New generations of mobile networks are opening up novel possibilities for controlling robots remotely in real-time. With 5G’s requirement to support use cases that demand low latencies at a high reliability from the communication network, wireless control applications become feasible. A remote operator typically uses a handheld device with buttons or joysticks to control a mobile robot. Joysticks are widely used today. The limitations of two-dimensional controlling and displaying of camera data can cause difficulties. Augmented Reality (AR)-based control provides the ability to control in three dimensional space. Therefore, new user-friendly Human Machine Interfaces (HMIs) can improve the interaction with these robots. In this demonstration, we present a human-in-the-loop application with a novel HMI. With our HMI a remote operator controls a four-legged Boston Dynamics Spot robot with gestures while wearing the AR-Headset Microsoft HoloLens 2. The remote operator receives feedback from the robot in the form of live camera streams visualised on holographic screens.
Evaluating ChatGPT for Smart Contracts Vulnerability Correction

The growing number of exploits and hacks on the Ethereum blockchain has led to the development of powerful smart contract vulnerability detection tools and the frequent patching of the smart contract’s programming languages (such as Solidity). At the same time, an ever-increasing number of people are interested in blockchain and smart contract-related topics and willing to build and deploy their own Decentralized Applications (dApp). However, learning a new programming language and its best practices as long as how to actually deploy a smart contract on the blockchain is a difficult task even for experienced developers. Recently, ChatGPT, a new user-friendly deep learning tool, has been released to improve the ability of non-skilled users to write high-quality code and in general, to boost the performances of developers in key tasks related to code writing (i.e., writing functions, explaining runtime errors, fixing bugs, etc.). This paper aims to measure the capabilities of ChatGPT in fixing vulnerable smart contracts and to assess the effectiveness of this tool, determining whether it can be a valuable aid for those who want to correct their own smart contract or want to reuse existing ones by first checking their status and eventually fix their vulnerability. In particular, we asked ChatGPT to fix 143 smart contracts with well-known labeled vulnerabilities. We considered a vulnerability as "fixed" if the code corrected by ChatGPT no longer contained the vulnerability (for this purpose, we exploited Slither, one of the state-of-the-art tools for smart contracts vulnerability detection to check the status of the original and the corrected smart contracts). As a result we obtained that ChatGPT was able to fix bugs and vulnerable smart contracts on average the 57.1% of the time with an increase of +1.4% when a description of the bug was provided in addition to the smart contract’s source code.
Leveraging Large Language Models With Vocabulary Sharing For Sign Language Translation

Sign language translation (SLT) is a task that provides translation between spoken and sign languages used in the same country, which tend to show high lexical similarity but low syntactic similarity. The recent emergence of large language models (LLMs) has been remarkable for all downstream tasks in natural language processing, but they have yet to be applied to SLT. In this paper, we explore how to use an LLM with vocabulary sharing for two gloss-based SLT tasks (text-to-gloss (T2G) and gloss-to-text (G2T)) on the NIASL2021 dataset, which consists of 180,848 preprocessed Korean and Korean Sign Language (KSL) sentence pairs. The experimental results showed that Ko-GPT-Trinity-1.2B+VS, a GPT-3-based SLT model with vocabulary sharing, outperformed other SLT models, achieving BLEU-4 scores of 22.06 and 45.89 on T2G and G2T tasks, respectively. We expect that the adoption of an LLM with vocabulary sharing will significantly lessen the resource scarcity problem of SLT.
PFED-AGG: A Personalized Private Federated Learning Aggregation Algorithm

Federated learning is a special kind of distributed machine learning, in which multiple clients work together to solve a machine learning problem with the collaboration of a central server, and the clients only need to upload parameters for server aggregation instead of uploading raw data, so the privacy of the clients can be protected. However, existing research shows that an attacker who obtains the parameters uploaded by the client can reverse the privacy information of the client, and Federated Learning applies a local differential privacy approach to protect the information of the parameters uploaded by the client from being leaked. However, this privacy protection approach assumes the same level of privacy protection for all clients. To the best of our knowledge, there needs to be work that satisfies the personalized privacy needs of clients. To address this problem, we propose a personalized local differential privacy-based federation framework that satisfies the personalized privacy needs of clients and better protects the privacy of clients by making the specific privacy needs of clients inaccessible to the server. We have conducted extensive experiments on six benchmark datasets, and our approach works better and achieves personalized privacy protection compared to the same privacy-preserving method.
A Sequence Agnostic Multimodal Pre-Processing for Clogged Blood Vessel Detection in Alzheimer’s Diagnosis

Successful identification of blood vessel blockage is a crucial step for Alzheimer’s disease diagnosis. These blocks can be identified from the spatial and time-depth variable Two-Photon Excitation Microscopy (TPEF) images of the brain blood vessels using machine learning methods. In this study, we propose several pre-processing schemes to improve the performance of these methods. Our method includes 3D-point cloud data extraction from image modality and their feature-space fusion to leverage complementary information inherent in different modalities. We also enforce the learned representation to be sequence-order invariant by utilizing bidirection dataflow. Experimental results on The Clog Loss dataset <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> show that our proposed method consistently outperforms the state-of-the-art pre-processing methods in stalled and non-stalled vessel classification. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> https://www.drivendata.org/competitions/65/clog-loss-alzheimersresearch/data/
Time Series Adaptation Network for Sensor-Based Cross Domain Human Activity Recognition

Domain adaptation can apply knowledge learned from the source domain to the target domain by reducing data distribution discrepancy inter domains. However, existing domain adaptation algorithms do not do as well on sensor datasets as on image datasets because of the neglect of intra data distribution discrepancy. The long time of collecting a raw data segment on sensors will lead to a shift with time, and the shift distribution will change with the variety of sensors and wearing positions, causing the time series distribution discrepancy intra and inter domains. To solve this problem, we design a new model, Time Series Adaptation Network (TSAN), and a new loss, Time series Contrastive Loss (TCL). TSAN uses a siamese network and “pack” the samples divided from the same segment into the network. Furthermore, TCL is defined as the similarity of “unpack” network output, which leads the model to learn time-independent features. In particular, TSAN can be used as a plug-in to combine with existing domain adaptation algorithms, so the intra and inter distribution discrepancies can be considered simultaneously. We conduct extensive experiments with eight existing domain adaptation algorithms on sensor-based cross domain human activity recognition (HAR) tasks, including three Routine Activity Recognition (RAR) datasets and four Parkinson's tremor Detection (PD) datasets. The results show that all the existing algorithms are improved by an average of 5.4% (RAR), 2.2% (PD) with TSAN.
Prototype based linear sub-manifold learning

Sub-manifold learning has been widely used to project high-dimensional data into a low-dimensional manifold, preserving the structure of the original data as much as possible. Existing sub-manifold learning methods either learn an embedding manifold that may have different geometric properties as the original data space, or learn a sub-manifold without considering the nonlinear structure of the data. In this paper, we learn a sub-manifold of the original data based on learned prototypes which represent prior knowledge about the intrinsic features of the data. This allows to incorporate the prior knowledge existing in the prototypes to find the suitable sub-manifold. The sub-manifold and the prototypes are jointly learned in a unified cost function via the gradient descent algorithm. The learned prototypes are obtained in the original high-dimensional space and subsequently used to learn a projection matrix to map the high-dimensional data into a lower-dimensional subspace with better separability. The prototypes are relearned in the projected subspace. The relearned low-dimensional prototypes are then working as prior knowledge to induce the learning of a better projection matrix, leading to a better subspace. The proposed subspace learning is realized for data points living in the Riemannian space of symmetric positive definite (SPD) matrices via the generalized learning Riemannian space quantization (GLRSQ) method. Experiments on both synthetic and real-world data sets show the effectiveness of the proposed dimension reduction scheme.
An expert system for automatic cyber risk assessment and its AI-based improvements

Evaluating risks against IT Systems is a complex yet crucial process that requires significant resources and competencies. This paper proposes RiskMan, an expert system for the automatic assessment of cyber risks that computes a risk score using information gathering and vulnerability assessment tools, public databases, and leaks from the dark web without involving cybersecurity experts. Moreover, RiskMan uses AI-driven techniques to determine risks also when only partial information is available.
Social Media, Market Sentiment and Meme Stocks

In this study, we focus on the process of how social interaction among retail investors impacts market behavior based on information collected from social media platforms. The analysis shows social media activity from retail investors is not the main contributing factor that has caused the meme stock mania. More specifically, two primary social media metrics, volume and sentiment of discussion for stocks, are examined by applying several machine learning methods. The resulting model indicates volume merely rises or falls contemporaneously with stock prices, carrying almost no information about identifying potential future meme stocks; On the other hand, social media sentiment indeed can help to forecast such regime changes to some degree. Nevertheless, unlike conclusions from some previous investigations, our study shows that positive and extremely negative sentiments do not increase the probability of a stock turning into a meme. Instead, modestly negative sentiment could do. Retail investors are followers and simply pile on momentum.
Adversarial Human Context Recognition: Evasion Attacks and Defenses

Human Context Recognition (HCR) from smartphone sensor data is a crucial task for Context-Aware (CA) systems, such as those targeting the healthcare and security domains. HCR models deployed in the wild are susceptible to adversarial attacks, wherein an adversary perturbs input sensor values to cause malicious mis-classifications. In this study, we demonstrate evasion attacks that can be perpetuated during model inference, particularly input perturbations that are adversarially calibrated to fool classifiers. In contrast to white-box methods that require impractical levels of system access, black-box evasion attacks merely require the ability to query the model with arbitrary inputs. Specifically, we generate adversarial perturbations using only class confidence scores, as in the Zoo attack, or only class decisions, as in the HopSkipJump (HSJ) attack that correspond with plausible scenarios of possible adversarial attacks. We empirically demonstrate that sophisticated adversarial evasion attacks can significantly impair the accuracy of HCR models, resulting in a performance drop of up to 60% in f1-score. We also propose RobustHCR, an innovative framework for demonstrating and defending against black box evasion threats using a provable defense based on a duality-based network. RobustHCR is able to make reliable predictions regardless of whether its input is under attack or not, effectively mitigating the potential negative impacts caused by adversarial attacks. Rigorous evaluation on both scripted and in-the-wild smartphone HCR datasets demonstrates that RobustHCR can significantly improve the HCR model’s robustness and protect it from possible evasion attacks while maintaining acceptable performance on "clean" inputs. In particular, an HCR model with integrated RobustHCR defenses experienced an f1-score reduction of about 3% as opposed to a reduction of over 50% for an HCR model without a defense.
Dynamic Analysis for Detection of Self-Destructive Smart Contracts

Research in Ethereum BlockChain has resulted in the growth of several tools for vulnerability detection. As a typical example, the Vandal tool detects selfdestruct vulnerability.Even though Vandal is a static analysis tool, its approach is also employed by several dynamic analysis tools. There is a need for a different approach for dynamic analysis tools to detect vulnerabilities such as selfdestruct so that dynamic analysis tools can maintain their individuality. This paper uses dynamic analysis to detect the selfdestruct (or self-destructive) vulnerability. Our work balances the developmental pace of the static and dynamic analysis approaches. The novelty of the work presented in this paper is that we use an Ether transfer-based approach and name it as "terminating transfer" to detect the selfdestruct vulnerability using the tool developed by us called "TechyTech".
PrefixCDD: Effective Online Concept Drift Detection over Event Streams using Prefix Trees

Process mining focuses on applying data mining techniques over business process data. Recently, with the improvements in sensoring, collection, and storage of event data, a big demand for both shorter mining time and adaptive models of streaming process events arose. This increased the interest in streaming process mining. Some techniques within this field attempt to identify drifts (change points) from evolving process data streams. Existing work on supervised and unsupervised-learning approaches over data streams have several limitations with regards to the nature of the drifts, the excessive storage required to store and process the stream, and the performance over real-world datasets. This paper contributes PrefixCDD, an efficient unsupervised-learning novel approach for online concept drift detection (CDD) over event streams. Our proposed approach utilizes a data structure, where the data stream components are stored in a set of prefix-trees. It transforms then the discrete data into continuous one using a Principal Component Analysis (PCA) approach over the trees. Then, ADWIN is used to focus on up-to-date information, making it appealing to work with the decaying mechanism logic behind our algorithm. Using six artificial and three real-life datasets, PrefixCDD outperforms state-of-the-art techniques in terms of detecting existing drifts of different natures, discovering them shortly after they appear, and the overall execution time.
On the Connection between Concept Drift and Uncertainty in Industrial Artificial Intelligence

AI-based digital twins are at the leading edge of the Industry 4.0 revolution, which are technologically empowered by the Internet of Things and real-time data analysis. Information collected from industrial assets is produced in a continuous fashion, yielding data streams that must be processed under stringent timing constraints. Such data streams are usually subject to non-stationary phenomena, causing that the data distribution of the streams may change, and thus the knowledge captured by models used for data analysis may become obsolete (leading to the so-called concept drift effect). The early detection of the change (drift) is crucial for updating the model’s knowledge, which is challenging especially in scenarios where the ground truth associated to the stream data is not readily available. Among many other techniques, the estimation of the model’s confidence has been timidly suggested in a few studies as a criterion for detecting drifts in unsupervised settings. The goal of this manuscript is to confirm and expose solidly the connection between the model’s confidence in its output and the presence of a concept drift, showcasing it experimentally and advocating for a major consideration of uncertainty estimation in comparative studies to be reported in the future.
Application Recommendation based on Metagraphs: Combining Behavioral and Published Information

Faced with so many mobile applications in the app store, users have difficulties finding their preferred mobile applications. Existing studies do not comprehensively consider implicit feedback in mobile applications and thus do not combine behavioral information and published information together to make recommendations. This paper proposes a novel method to recommend mobile applications based on metagraph embedding using the combination of behavioral information and published information. Specifically, this paper constructed a conceptual model using the combinations of behavioral information and published information that could well portray users and mobile applications. Based on this conceptual model, six metagraphs are designed to interpret the multidimensional relationships between users and mobile applications in the model. By random walking guided by each metagraph, a series of node sequences that could express node neighborhood are obtained. Finally, the similarity between users and apps is calculated using the embedded vector of each node, and the recommendations are given to the user. Based on a real-world dataset, we evaluate the performance of our method. The experimental result shows that our method outperforms existing models and methods in all metrics, in which the average F1-measure increases by 19.21%, and the average NDCG increases by 4.99%.
Towards the Formal Analysis of UML Activity Diagrams in a Calculus of Context-aware Ambients

The Unified Modeling Language (UML) is the industrial de-facto standard for designing systems. It has been used widely in many industrial applications. However, the lack of formal semantics for UML makes it unsuitable for formal verification. As such, UML is limited when it comes to the design of safety/security critical systems where faults can cause damages to people, properties, or the environment. This paper proposes an attempt to define a formal semantics for the UML activity diagrams. An algorithm is proposed that translates an activity diagram into a process in a Calculus of Context-aware Ambients (CCA). This process can then be formally analysed using the tool support for CCA. Hence, errors can be detected and fixed early during the system development life-cycle. The pragmatics of the proposed approach is demonstrated using a case study in e-commerce.
Evaluations of Semi-supervised Methods for Hepatocellular Carcinoma Segmentation from Pathological Images

Early diagnosing hepatocellular carcinoma (HCC) helps reduce the mortality. In this paper, we apply and compare semi-supervised segmentation methods to solve the HCC segmentation problem of pathological whole slide images. We show that the performance of the semi-supervised methods can even outperform supervised methods for HCC segmentation.
Monitoring the Energy Consumption of Docker Containers

Containers are an increasingly used mechanism for providing low-cost, lightweight, portable, standalone application deployments, particularly for service orchestration. Docker provides container technology that enables a single host to isolate several applications and deploy them rapidly in different environments. The increasing demand for container applications and the growing popularity of Docker has motivated extensive research into evaluating the performance, energy consumption, and running cost of Docker-based computation. This paper investigates the energy footprint of Docker containers and workloads. To motivate research in energy-efficient container development, this paper takes a practical approach to measure the energy consumption in common Docker containers under various workloads.
Distortion Reduction for Off-Center Perspective Projection of Panoramas

A key assumption of perspective projection is that linear features in 3D shall remain linear after being projected to the 2D screen. This assumption is preserved when we draw a spherical panorama perspectively in arbitrary viewing directions and field-of-views as long as the camera position is fixed at the center. However, when the camera moves away from the center, barrel-like distortions appear and such assumption breaks. To address this issue, we propose modifications to the equirectangular-to-perspective (E2P) projection that significantly reduced distortion of linear features when the camera position is away from the center. We compared with other common methods that aim to augment panoramas with 3D information including: 1) building a point cloud augmented with per-pixel depth and 2) building a 3D room model according to room layouts, and found that our method produced rendering results with less linearity distortion measured quantitatively and qualitatively.
Examining Feasibility and Efficacy of Traditional Stream Clustering Algorithms on Complex Human Activity Recognition Data

Modern adoption of network-connected devices have led to an abundance of high speed data streams and a call for the development of analytical models capable of processing them. Much of the real world data is unlabeled which requires development and application of unsupervised learning algorithms for advanced machine perception. However, the datasets that are used to train and validate these algorithms are often synthetic or simulated for the study and too simple, which leads to an inflation of reported performance. Based on the recent literature, there is a lack in research on developing deep learning models for clustering highly complex Human Activity Recognition (HAR) data streams. We, therefore, explore the area and apply three modern high-performing stream clustering algorithms on a complex HAR data stream. We evaluate the ability of these models to adapt and extract the volatile knowledge that is contained within the HAR data streams. Our goal is to set a baseline of performance for clustering timestamped HAR data streams. The results show that despite the reported performance of these algorithms in the literature, they lack the analytical depth to effectively extract sufficient knowledge from the complex HAR streams to construct viable clusters. So, new thoughts are needed to develop models that can efficiently handle HAR data streams.