Reviewerly/exHarmony huggingface dataset:
Strategic Peer Selection Using Transaction Value and Latency

Many blockchains utilize public peer-to-peer networks to communicate transactions. As activity on blockchain-based DeFi protocols has increased, there has been a sharp rise in strategic behaviour from bots and miners, commonly captured by the notion of Maximal Extractable Value (MEV). While many works have focused on MEV arising from the smart contract layer or consensus layer, in this work we study how a strategic agent can maximise realisable MEV through the optimal choice of network peers. Specifically, we study how existing definitions and algorithms for latency optimization can be augmented with information about the transactions themselves in order to optimize peering algorithms. We formally model this optimization objective for two classes of consensus protocols : 1) time-based ("fair ordering") protocols and 2) single leader-based protocols. We present an efficient local algorithm for choosing peers strategically, and evaluate our algorithm on real world data to show that it outperforms benchmark algorithms that either choose peers randomly or do not exploit information about blockchain transactions.
Exposing Side-Channel Leakage of SEAL Homomorphic Encryption Library

This paper reveals a new side-channel leakage of Microsoft SEAL homomorphic encryption library. The proposed attack exploits the leakage of ternary value assignments made during the Number Theoretic Transform (NTT) sub-routine. Notably, the attack can steal the secret key coefficients from a single power/electromagnetic measurement trace. To achieve high accuracy with a single-trace, we build a novel machine-learning based side-channel profiler. Moreover, we implement a defense based on random delay insertion based defense mechanism to mitigate the shown leakage. The results on an ARM Cortex-M4F processor show that our attack extracts secret key coefficients with 98.3% accuracy and random delay insertion defense does not reduce the success rate of our attack.
A Brief History of Digital Forensics and Digital Evidence

The history of digital forensics and digital evidence is fascinating. For this discussion of the history of digital forensics and digital evidence, we have identified six time periods: (1) Foundation (pre-1980, 1980–1990), (2) Pre-automated Tools & Grassroot efforts (1991–2000), (3) Automated Tools/Acquisition (2001–2010), (4) Analysis & Examination (2011–2015), (5) Meaning & Context (2016–2019), (6) Autonomous Tools (2020-Present). These periods or eras were selected based on the clustering of common operational, legal, and scientific needs and demands. While we use the six eras as a framework for our discussion, these are conceptual and not meant to indicate actual hard demarcation points.
Frequentist Approach to Forensic Evidence Interpretation

The frequentist approach to evidence interpretation generally means the use of a frequentist hypothesis-testing framework to make statements of common source. That is, if summary statistics, such as means from control and recovered samples, are unable to be distinguished from variation due to chance, then the recovered samples are said to have come from the control source. This approach is often criticized by proponents of the Bayesian, or likelihood ratio, approach because it only considers the probability of the evidence under one proposition, rather than the two before the court. However, use of formal statistical techniques in evidence evaluation is a considerable step forward over making subjective statements such as “the evidence is consistent with …”
Digital Evidence: Overview

Digital evidence, while commencing as a capability around the 1990s, has grown remarkably quickly. Its value is no longer restricted to law enforcement investigations but is equably useful from “war room to board room”. The examination of digital evidence is also an indispensable capability in private and public sector security environments. Criminal activities in the electronic world are largely similar to those of the physical world. Criminals have access to greater abilities to obscure their conduct with the use of anonymising technology. Digital evidence, while commencing as a capability around the 1990s, has grown remarkably quickly. Its value is no longer restricted to law enforcement investigations but is equably useful from “war room to board room”. The examination of digital evidence is also an indispensable capability in private and public sector security environments. Criminal activities in the electronic world are largely similar to those of the physical world. Criminals have access to greater abilities to obscure their conduct with the use of anonymising technology. Digital evidence intelligence is becoming a priority in order to understand inter-jurisdictional criminal activity. A digital mesh is a human centered theme that refers to the collection of devices, information, apps, services, businesses and other people that exist around an individual. The future digital evidence practitioner will need to be equipped with the knowledge and skills to address forensic questions in a given case. The impact of cloud computing and the rapidly growing prevalence of Internet of Things systems continues to challenge forensic analysts and their employing organizations. However, there are deficiencies in the accuracy and the reliability of automated technologies when applied to digital evidence. The need for explainability has come under renewed focus with an increasing reliance on automated tools. The digital evidence expert’s job involves dealing with text, images, video and audio of people in distress. Workplace features can mitigate the possibility of severe stress reactions. The digital evidence expert’s job involves dealing with text, images, video and audio of people in distress. Workplace features can mitigate the possibility of severe stress reactions.
Artificial Intelligence in Digital Forensics

Since its inception as a forensic discipline, digital evidence has experienced rapid growth in its contribution to the justice system. Accompanying that growth in utility has been the rapid increase in volume and complexity of the subject evidence. The existent analytical methods do not scale to meet the growing demand and practitioners are looking to artificial intelligence to automate some of the tasks in digital evidence analysis. Machine Learning and Deep Learning are described and the ways in which they contribute to automation and the future possibilities. Included are descriptions for training models, validation, explainability and bias.
Rave: A Modular and Extensible Framework for Program State Re-Randomization

Dynamic software diversification is an effective way to boost software security. Existing diversification-based approaches often target a single node environment and leverage in-process agents to diversify code and data, resulting in an unnecessary attack surface on a fixed software/hardware stack. This paper presents Rave, a practical system designed to enable out-of-bound program state shuffling on a moving target environment, avoiding any sensitive agent code invoked within the running target. Rave relies on a user-space page fault handling mechanism introduced in the latest Linux kernel and seamlessly integrates with CRIU, the battle-tested process migration tool for Linux.
Commensality or Reverie in Eating? Exploring the Solo Dining Experience

Commensality, the act of eating together, is commonly associated with many benefits. Dining solo, in contrast, is frequently connected to adverse effects on a person. There is a growing interest in human-computer interaction (HCI) and design in how innovations can enhance eating experiences by, for example, facilitating commensality. The steadily growing number of people eating alone and the associated risks beg the question of how HCI and design could contribute to and improve the solo dining experience and whether or not mimicking or facilitating commensality is what solo diners want. This two-phased study reports on the context exploration of the multimodal solitary dining experience. In the first phase, we scoped the literature describing the benefits and drawbacks of solo dining and commensality. For the second phase, a digital food diary was developed and completed by six solo diners to collect further insights and user requirements. Photos and annotations collected as part of the food diary were analyzed using content analysis. The results indicate several advantages of eating alone, including feeling relaxed, perceiving solo dining as a moment of self-pampering, and appreciating the cooking experience. Overall, it seems that solo dining is not merely a lack of commensality, but a unique experience in and of itself, where people seem to strive towards finding reverie in eating.
Personalized Productive Engagement Recognition in Robot-Mediated Collaborative Learning

In this paper, we propose and compare personalized models for Productive Engagement (PE) recognition. PE is defined as the level of engagement that maximizes learning. Previously, in the context of robot-mediated collaborative learning, a framework of productive engagement was developed by utilizing multimodal data of 32 dyads and learning profiles, namely, Expressive Explorers (EE), Calm Tinkerers (CT), and Silent Wanderers (SW) were identified which categorize learners according to their learning gain. Within the same framework, a PE score was constructed in a non-supervised manner for real-time evaluation. Here, we use these profiles and the PE score within an AutoML deep learning framework to personalize PE models. We investigate two approaches for this purpose: (1) Single-task Deep Neural Architecture Search (ST-NAS), and (2) Multitask NAS (MT-NAS). In the former approach, personalized models for each learner profile are learned from multimodal features and compared to non-personalized models. In the MT-NAS approach, we investigate whether jointly classifying the learners’ profiles with the engagement score through multi-task learning would serve as an implicit personalization of PE. Moreover, we compare the predictive power of two types of features: incremental and non-incremental features. Non-incremental features correspond to features computed from the participant’s behaviours in fixed time windows. Incremental features are computed by accounting to the behaviour from the beginning of the learning activity till the time window where productive engagement is observed. Our experimental results show that (1) personalized models improve the recognition performance with respect to non-personalized models when training models for the gainer vs. non-gainer groups, (2) multitask NAS (implicit personalization) also outperforms non-personalized models, (3) the speech modality has high contribution towards prediction, and (4) non-incremental features outperform the incremental ones overall.
A Spatio-temporal Learning for Music Conditioned Dance Generation

The music-conditioned dance generation, i.e., dancing to music, is a usage scenario of multi-modality human motion synthesis. Typically, it is a challenge to choreograph continuous motions coinciding with the melody and rhythm of the music. This paper proposes a position-wise encoding-decoding framework for spatio-temporal learning of motions and long-term skeleton-based dance generation oriented on music. Given the positional embedding of the frames in 1-minute video clips, firstly, we modularize a regional attention-based feed-forward mechanism to encode the music features. Secondly, based on the skeleton of each frame and the joint trajectories across motion frames, we formalize a graph topology to represent each dance sequence's spatial and temporal knowledge. Specifically, we propose a graph convolutional network (GCN) based blocks to process long-term dependencies of motions and leverage the spatial and temporal features. Both music and motion paths are learned fully in positional embedding schemes and constructed by repeating the corresponding blocks. Finally, as the task of dance generation is inherently the consistency between music and motions, we proposed a cross-modality feature fusion for multimodal interaction and music-conditioned dance generation. Experimental results demonstrate that our method outperforms state-of-art methods in motion quality and motion-music correlation metrics.
NetNDP: Nonoverlapping (delta, gamma)-approximate pattern matching

Pattern matching can be used to calculate the support of patterns, and is a key issue in sequential pattern mining (or sequence pattern mining). Nonoverlapping pattern matching means that two occurrences cannot use the same character in the sequence at the same position. Approximate pattern matching allows for some data noise, and is more general than exact pattern matching. At present, nonoverlapping approximate pattern matching is based on Hamming distance, which cannot be used to measure the local approximation between the subsequence and pattern, resulting in large deviations in matching results. To tackle this issue, we present a Nonoverlapping Delta and gamma approximate Pattern matching (NDP) scheme that employs the (δ,γ)-distance to give an approximate pattern matching, where the local and the global distances do not exceed δ and γ, respectively. We first transform the NDP problem into a local approximate Nettree and then construct an efficient algorithm, called the local approximate Nettree for NDP (NetNDP). We propose a new approach called the Minimal Root Distance which allows us to determine whether or not a node has root paths that satisfy the global constraint and to prune invalid nodes and parent-child relationships. NetNDP finds the rightmost absolute leaf of the max root, searches for the rightmost occurrence from the rightmost absolute leaf, and deletes this occurrence. We iterate the above steps until there are no new occurrences. Numerous experiments are used to verify the performance of the proposed algorithm.
Inclusive Multimodal Voice Interaction for Code Navigation

Navigation of source code typically requires extensive use of a traditional mouse and keyboard which can present significant barriers for developers with physical impairments. We present research exploring how commonly used code navigation approaches (e.g. locating references to user-defined identifiers, jumping to function definitions, conducting a search for specific syntax, etc.) can be optimized for multimodal voice interaction. An exploratory study was initially conducted with five developers who have physical impairments to elicit insights around their experiences in navigating code within existing voice-controlled development environments. Findings from this study informed the design of a code editor integrating different navigation features tailored for multimodal speech input. A user evaluation with 14 developers with physical impairments was conducted with results demonstrating that all participants were able to successfully complete a series of standard navigation tasks. Participants also highlighted that the code navigation techniques were intuitive to use and provided a high-level of usability.
RGBDGaze: Gaze Tracking on Smartphones with RGB and Depth Data

Tracking a user’s gaze on smartphones offers the potential for accessible and powerful multimodal interactions. However, phones are used in a myriad of contexts and state-of-the-art gaze models that use only the front-facing RGB cameras are too coarse and do not adapt adequately to changes in context. While prior research has showcased the efficacy of depth maps for gaze tracking, they have been limited to desktop-grade depth cameras, which are more capable than the types seen in smartphones, that must be thin and low-powered. In this paper, we present a gaze tracking system that makes use of today’s smartphone depth camera technology to adapt to the changes in distance and orientation relative to the user’s face. Unlike prior efforts that used depth sensors, we do not constrain the users to maintain a fixed head position. Our approach works across different use contexts in unconstrained mobile settings. The results show that our multimodal ML model has a mean gaze error of 1.89 cm; a 16.3% improvement over using RGB data alone (2.26 cm error). Our system and dataset offer the first benchmark of gaze tracking on smartphones using RGB+Depth data under different use contexts.
GazeScale: Towards General Gaze-Based Interaction in Public Places

Gaze-based interaction has until now been almost an exclusive prerogative of the assistive field, as it is considered not sufficiently performing compared to traditional communication methods based on keyboards, pointing devices, and touch screens. However, situations such as the one we are experiencing now due to the COVID-19 pandemic highlight the importance of touchless communication, to minimize the spread of the disease. In this paper, as an example of the potential pervasive use of eye tracking technology in public contexts, we propose and study five interfaces for a gaze-controlled scale, to be used in supermarkets to weigh fruits and vegetables. Given the great heterogeneity of potential users, the interaction must be as simple and intuitive as possible and occur without the need for calibration. The experiments carried out confirm that this goal is achievable and show strengths and weaknesses of the five interfaces.
Assessing Daily Activity Routines Using an Unsupervised Approach in a Smart Home Environment

When the mental acuity of older adults deteriorates (e.g., dementia), irregular patterns manifest within their activities of daily living (ADL), which renders an effective opportunity for healthcare providers to monitor patients' mental status. Although successful, such studies depended on supervised learning approaches to recognize ADLs, which require tedious human observation and manual annotation of data. To establish a more efficient alternative, this study develops an unsupervised data segmentation process by modifying a Superpixels Extracted via Energy Driven Sampling (SEEDS) algorithm and a hierarchical clustering method effective for high-dimensional temporal sensor data. The proposed approaches consider the spatiotemporal features (e.g., start time, duration, location, and sequence) and activity-oriented features (e.g., motion intensity and appliance usages) to identify ADL routines without necessitating predefined rules or limiting the scope of features. The results showed that the proposed approaches have comparable accuracy (0.788) to benchmark models that require a priori knowledge (e.g., ontology). Our proposed methodology can be extended to high-dimensional, nonintrusive sensing data to capture the variability of ADL routines in the future. This study contributes a methodological advance for efficiently assessing ADL routines via high-dimensional sensor data and supports future opportunities for capitalizing on smart home technologies that enable older adults to live alone safely, aging-in-place.
Comparative Analysis of Entity Identification and Classification of Indian Epics

Despite the relevance and accessibility of Indian epics such as the Mahabharata, little research has been done on the corpus in Natural Language Processing (NLP). To revitalize NLP research on the Indian epics, we present a computational analysis of SOTA supervised Machine Learning (ML) methods for Named Entity Recognition (NER) using our annotated dataset of labeled tokens of the Mahabharata text. The text contains English and Sanskrit words, offering a different vocabulary than modern literature. The characters also change their nature throughout the storyline, challenging many SOTA NER methods as tokens with the same name can have different entity types. For example, we discover that NLTK inaccurately identifies 95% tokens as named entities. We also note that spaCy and Stanford NER perform adequately only after training. However, since they produce one embedding per token, they fail to distinguish between entities with the same name. In contrast, context-driven methods such as BERT and ELMO tackle the issue as they produce different embeddings. Yet, ELMO delivers poor results due to its character-based embeddings and pseudo-bidirectionality. BERT performs the best with a 0.98 micro-avg F1 score but overfits the dataset. Therefore, the current SOTA techniques are unsuitable for NER on the Indian epics, and more research is needed to build a universally suited NER agent capable of recognizing named entities from diverse cultural contexts.
Discrete Wavelet Transform in digital audio signal processing: A case study of programming languages performance analysis

The time spent processing the test set is a crucial factor in research projects involving audio signal transforms. The present work analyzes the performance of a set of programming languages commonly used in this type of scientific research. The results showed the C and C++ languages as faster in a test suite with 8 different programming languages implementing the discrete Wavelet transform.
Exploring the Benefits of Spatialized Multimodal Psychophysiological Insights for User Experience Research

Conducting psychophysiological investigations outside of lab settings has a lot of potential for academic applications as well as for industries concerned about the quality of their user and customer experience. Prior work employing in-the-wild methodologies often focuses on a limited set of biometrics, which constrains the depth of the insights generated by such investigations. In this work, we field tested a new system for multimodal data acquisition and present exploratory results and insights from two ambulatory data collection sessions, held during public events. Through a spatially grounded analysis, we investigate the feasibility, and practicality of multimodal ambulatory psychophysiological inquiries, and hypothesize on areas of added values for public event organizers.
Masked Face Recognition via Self-Attention Based Local Consistency Regularization

With the COVID-19 pandemic, one critical measure against infection is wearing masks. This measure poses a huge challenge to the existing face recognition systems by introducing heavy occlusions. In this paper, we propose an effective masked face recognition system. To alleviate the challenge of mask occlusion, we first exploit RetinaFace to achieve robust masked face detection and alignment. Secondly, we propose a deep CNN network for masked face recognition trained by minimizing ArcFace loss together with a local consistency regularization (LCR) loss. This facilitates the network to simultaneously learn globally discriminative face representations of different identities together with locally consistent representations between the non-occluded faces and their counterparts wearing synthesized facial masks. The experiments on the masked LFW dataset demonstrate that the proposed system can produce superior masked face recognition performance over multiple state-of-the-art methods. The proposed method is implemented in a portable Jetson Nano device which can achieve real-time masked face recognition.
Hiding Images Into Images with Real-World Robustness

The existing image embedding networks are basically vulnerable to malicious attacks such as JPEG compression and noise adding, not applicable for real-world copyright protection tasks. To solve this problem, we introduce a generative deep network based method for hiding images into images while assuring high-quality extraction from the destructive synthesized images. An embedding network is sequentially concatenated with an attack layer, a decoupling network and an image extraction network. The addition of decoupling network learns to extract the embedded secret image from the attacked image. We also pinpoint the weaknesses of the adversarial training for robustness in previous works and build our improved real-world attack simulator. Experimental results demonstrate the superiority of the proposed method against typical digital attacks by a large margin, as well as the performance boost of the recovered images with the aid of progressive recovery strategy. Besides, we are the first to robustly hide three secret images.