Reviewerly/exHarmony huggingface dataset:
A Dynamic Convolution Framework for Session-Independent Speaker Embedding Learning

Speaker verification (SV) has suffered from session variability in complex acoustic scenarios, and learning session independent speaker representations remains a challenging problem. To tackle this, we propose a dynamic convolution framework for SV in this article, which dynamically adapts the model parameters to each input feature during inference, such that the model can flexibly extract robust speaker characteristics under different acoustic conditions. Specifically, we combine an adaptive context vector extraction (ACVE) module and a sub-kernel scaling (SKS) module in a cascaded manner. The ACVE uses a moving weighted sum and a sub-band self-attention in parallel to extract global-local information, which is then passed through the subsequent SKS module for efficient dynamic kernel generation. The proposed method can be easily implemented on the backbone network by replacing the conventional static counterparts. Experimental results on five public SV datasets show significant and consistent improvements over comparison approaches, and ablation studies and visual analysis further demonstrate the effectiveness of the proposed method.
Automatic Creation of Tiled Maps in Two-Dimensional Top-Down Digital Role Playing Games

Game level maps are fundamental for modern Digital Role Playing Computer Games. Creating such maps by hand is a time-consuming process. We propose an approach that can be applied to create such maps automatically. The approach is explained with clearly defined steps. The map creation process divides a map into four distinct layers. First, we explain the idea of converting input in the form of a rough sketch of a layer into the suitable form — a colour grid. Next, the issue of map creation from the initial stage of the colour grid is presented. We then introduce methods used to correct the inputs in order to create layers without defects. We apply specific methods to each layer, explaining the method in detail in subsequent sections. The layers are then merged to create the final map. As the next step, we discuss the improvements applied to the process that we developed. The complete approach is then tested in practice with a questionnaire, which — following a careful analysis — has showed that there are no significant differences between handcrafted maps and the maps created using our approach.
Decoupled Real-Time Trajectory Planning for Multiple Autonomous Mining Trucks in Unloading Areas

Cooperative trajectory planning for autonomous vehicles has garnered significant attention in structured environments, but corresponding methodologies for unstructured environments remains relatively underexplored. The unloading area, an integral component of open-pit mines, exemplifies a quintessential unstructured environment. Implementing cooperative planning for autonomous mining trucks (AMTs) within these unloading areas is crucial as the optimization of processes in these areas substantially enhances the overarching safety, productivity, and cost-effectiveness of mining operations. Hence, enhancing the operational efficiency of AMTs in the unloading area can considerably elevate productivity levels of open-pit mines. This article focuses on the real-time cooperative trajectory planning problem for AMTs in such areas, which is challenging due to i) small and irregular space ii) complex operations iii) need for path stability and speed flexibility. We propose a decoupled multi-vehicle trajectory planning (MVTP) method that decomposes trajectory planning into path planning and speed planning. Specifically, we present driving behavior enhanced path planning and sequential real-time cooperative speed planning methods. Our method is compared with several state-of-the-art MVTP methods and proves to be both secure and efficient.
PerceptGuide: A Perception Driven Assistive Mobility Aid Based on Self-Attention and Multi-Scale Feature Fusion

The paper introduces a novel wearable aid, PerceptGuide to help for visually impaired individuals to perceive scene around them. It is designed as a wearable, light weight chest rig bag, that incorporates a monocular camera, ultrasonic sensors, vibration motors, and a mono-earphone, powered by an embedded Nvidia Jetson development board. The system provides directional obstacle alerts through the vibration motors, allowing users to avoid obstacles on their path. A user-friendly push-button enables user to inquire about scene information in front of them. The scene details are effectively conveyed through a novel scene understanding approach, that combines multi-scale feature fusion, self-attention models, and a multilayer GRU (Gated Recurrent Unit) architecture on the ResNet50 backbone. The proposed system generates coherent and descriptive captions by capturing image features at different scales, enhancing the quality and contextual understanding of the scene details. The self-attention in both the encoder (ResNet50 + Feature fusion model) and decoder (multilayer GRU), effectively captures long-range dependencies and attend to relevant image regions. The quantitative evaluations conducted on the MSCOCO and Flicker8k datasets show the effectiveness of the model with improved Bleu-67.7, RougeL – 47.6, Meteor – 22.7 and CIEDR-67.4 scores. The PerceptGuide system exhibits exceptional real-time performance, generating audible captions in just 1.5 to 2 seconds. This rapid response time significantly aids visually impaired individuals in understanding the scenes around them. The qualitative evaluation of the aid emphasizes its real-time performance, demonstrating the generation of context-aware, semantically meaningful captions. This validates its potential as a wearable assistive aid for visually impaired people, with the added advantages of low power consumption, compactness, and a lightweight design.
Blockchain for Energy Management: Smart Meters, Home Automation, and Electric Vehicles

Today, more than ever, reducing energy consumption, air pollution, and other causes and effects associated with global warming are topics of interest to researchers. Traditionally, when making transactions with trust between parties, the help of banks or through an intermediary institutional framework is used. However, blockchain technology can change this system by imposing a decentralized architecture without an intermediary between parties when making a transaction. The applicability of blockchain technology in the present case is applied in the energy sector. The architectural principles, events and benefits that can be obtained in this direction are analyzed, such as the adoption of renewable resources, the influence produced due to climate changes for the technical balancing of the network, the creation of neighborhood microgrids (tenants' associations) in order to lower energy prices by using photovoltaic panels (PV) and micro-wind turbines networks at the community level, and the elimination of intermediaries between producers and consumers. It can be said that this sector can be addressed in applications aimed at smart meters, the smart home and electric vehicle charging. In the energy system's architecture, a prosumers hybrid appears between the producer and the consumer. In this way, through the additional quantity achieved through renewable systems, an increase in the generated energy and the corresponding decrease in the purchase price are obtained. This chapter proposes to create an association of green energy producers to manage the entire production and respective energy consumption and surplus. If the energy produced is not consumed or sold, the association will find storage solutions to make consumption available within the imposed limits, and the purchase price is low. Through a mobile application, the entire process will be followed by each association member and globally by its board of directors.
Forensic analysis of manipulation chains: A deep residual network for detecting JPEG-manipulation-JPEG

Due to the proliferation of image editing software, the problem of identifying doctored photos has risen to prominence in information forensics. Even though existing image processing history detection techniques could provide reasonably acceptable outcomes on publicly available databases, most of them generally fail to accomplish this when the doctored images are lossy compressed, which is commonly accomplished in social media. To tackle this issue, a dual-stream residual network (ReMReNet) using ResNet as a backbone is proposed for JPEG-resistant image operator chain detection. ReMReNet consists of noise residual extraction (NRE) and compression feature extraction (CFE) streams for better results against JPEG compression. The CFE stream is trained on discrete cosine transform (DCT) residual coefficients by producing clues left by JPEG-agent images with generic JPEG compression artefacts. In addition, the NRE stream is utilized to extract noise residuals to suppress the image content and enhance the manipulation clues. Hence, the domain transformation between uncompressed and post-JPEG compressed images is reduced by training the backbone network with a domain adaption approach, which helps forensic investigators pinpoint the exact location of the tampering. The proposed framework outperforms other state-of-the-art algorithms in image multiple manipulations and operator chain detection, notably for lossy JPEG compression, as shown by extensive experimental findings on numerous heterogeneous datasets.
Multi-objective joint optimization for concurrent execution of design-construction tasks in design-build mode

PERT (Program Evaluation and Review Technique) is suitable for evaluating schedule optimization of DBB (Design-Bid-Build) projects based on serial execution of design and construction. However, it is not suitable for evaluating the multi-objective joint optimization of DB (Design-Build) projects based on concurrent execution of design and construction. Therefore, focusing on the unique construction method and construction objective under DB mode, this study adopts the multi-parameter grey GERT (Graphical Evaluation and Review Technique) based on z-tag to construct a multi-objective joint optimization model of concurrent execution of design-construction tasks. The research methods adopted are as follows: Firstly, based on the modeling idea of PERT, a multi-parameter grey GERT was constructed, which was composed of sub-GERTs representing the execution flow of design-construction tasks. Second, based on discussing the functional relationship between time, cost and quality in normal execution and rework execution, a method of constructing a moment generating function for concurrent execution time, cost and quality to characterize the uncertainty of input and output parameters is proposed. Thirdly, on the basis of analyzing the solving methods of each parameter of the multi-parameter GERT, the solving methods for starting execution time, information transmission duration, and total execution duration of the critical time nodes are given. Finally, according to the requirements of DB projects, the multi-objective joint optimization models based on progress oriented and quality oriented are constructed respectively, and the gamultiobj algorithm is used to solve the models. The proposed approach provides a scientific quantitative analysis method for evaluating the progress, cost and quality objectives of the concurrent execution of design-construction tasks, and also lays theoretical guidance for further efficiency popularization of DB mode.
Time-Enhanced Neighbor-Aware network on irregular time series for sentiment prediction in social networks

Sentiment prediction is useful for scientific decision-making and reliable assessments in various fields. One significant challenge of sentiment prediction is the difficulty in dynamically capturing changing dependency patterns along the sentiment timeline. This is because individuals’ sentiments are often influenced by neighbors’ sentiment states. Moreover, the prevalence of missing values in real-world datasets significantly restricts the data’s ability to be modeled for prediction tasks. To address this issue, we model sentiment prediction as a task of time series prediction and propose a Time-Enhanced Neighbor-Aware network on irregular time series (TENA) to comprehensively utilize both the temporal and neighbors’ influence information. Specifically, TENA focuses on capturing the sentimental temporal correlations of the target user and the effect of neighbors. Next, TENA uses a memory-based attention mechanism to incorporate temporal influence and social influence into the prediction task. Furthermore, TENA introduces a time-enhanced Long Short-Term Memory (LSTM) network to extract the underlying features from irregular spaced sequential data with missing values by considering the time intervals. Experiments on three real-world datasets demonstrate the superiority of our model over several state-of-the-art methods. We use the macro F-score to measure performance in predicting sentiments. The TENA approach has improved by about 5.31%–18.18% on Twitter, 5.25%–15.66% on Flickr, and 1.70%–18.06% on Truth. In addition, we explore the interpretability of the proposed model in understanding individual sentimental change patterns by visualizing attention scores.
Uncertainty Quantification for Semantic Segmentation Models via Evidential Reasoning

Deep learning models typically render decisions based on probabilistic outputs. However, in safety-critical applications such as environment perception for autonomous vehicles, erroneous decisions made by semantic segmentation models may lead to catastrophic results. Consequently, it would be beneficial if these models could explicitly indicate the reliability of their predictions. Essentially, stakeholders anticipate that deep learning models will convey the degree of uncertainty associated with their decisions. In this paper, we introduce EviSeg, a predictive uncertainty quantification method for semantic segmentation models, based on Dempster-Shafer (DS) theory. Specifically, we extract the discriminative information, i.e., the parameters and the output features from the last convolution layer of a semantic segmentation model. Subsequently, we model this multi-source evidence to the evidential weights, thereby estimating the predictive uncertainty of the semantic segmentation model with the Dempster’s rule of combination. Our proposed method does not require any changes to the model architecture, training process, or loss function. Thus, this uncertainty quantification process does not compromise the model performance. Validated on the urban road scene dataset CamVid, the proposed method enhanced computational efficiency by three to four times compared to the baseline method, while maintaining comparable performance with baseline methods. This improvement is critical for real-time applications.
Tamper-Proof Privacy Auditing for Artificial Intelligence Systems

&lt;p&gt;Privacy audit logs are used to capture the actions of participants in a data sharing environment in order for auditors to check compliance with privacy policies. However, collusion may occur between the auditors and participants to obfuscate actions that should be recorded in the audit logs. In this paper, we propose a Linked Data based method of utilizing blockchain technology to create tamper-proof audit logs that provide proof of log manipulation and non-repudiation.&lt;/p&gt; &lt;p&gt; &lt;/p&gt;
Heterogeneous Patient Graph Embedding in Readmission Prediction

&lt;p&gt;The number of youth seeking mental health services has been increasing in the past decade. Accurate prediction of hospital readmission is a contributing factor in addressing youth mental health problem and healthcare service utilization. Medical records are an important source of information for readmission prediction, however utilizing such records in the context of mental care, requires overcoming two impeding challenges: the diversity of service utilization (e.g., using psychiatric vs. overdose vs. trauma clinics all for the same underlying mental health reason) and the heterogeneity of data associated with each service. Graph Neural Network (GNN) is shown to be promising in performing classification or regression tasks when input data bears a complex structure. In this research, we propose using graph embedding to first generate patient graph that captures episodic emergency department visits and the complex service utilizations of the patient. We then use GNN for readmission prediction. For embedding and training purposes, we utilize more than 4,000 unique mental health patients data over 19 years. To evaluate our approach we systematically compare a variety of of GNN models with four RNN models, namely LSTM, Bi-LSTM, GRU, and Bi-GRU. Our experimental evaluation demonstrates that encoding the complex interrelationship between features of a patient using graph embedding and GNN improves the performance of the predictive model compared to RNN counterparts. &lt;/p&gt;
SPLIT: QoS-Aware DNN Inference on Shared GPU via Evenly-Sized Model Splitting

Improving QoS by simultaneously reducing the latency violation rate and jitter in the presence of multiple deep learning inference (DLI) tasks sharing a single edge computing processor remains a challenge. However, existing DLI systems at the edge, designed to maximize throughput, face performance challenges when confronted with requests with varying QoS.
Mercury: Fast and Optimal Device Placement for Large Deep Learning Models

The rapidly expanding neural network models are becoming increasingly challenging to run on a single device. Hence, model parallelism over multiple devices is critical to guaranteeing the efficiency of training large models. Recent proposals either have long processing time or poor performance. Therefore, we propose Mercury, a fast framework for optimizing device placement for large models. Mercury employs a simple but efficient model parallelization strategy in the baseline measurement, and generates placement policies through a series of scheduling algorithms. We conduct experiments to deploy and evaluate Mercury on numerous large models. The results show that Mercury not only reduces the placement policy generation time by 26.4% but also improves the model throughput by 218.5% compared to the most advanced methods.
The computational complexity of classical knot recognition

The classical knot recognition problem is the problem of determining whether the virtual knot represented by a given diagram is classical. We prove that this problem is in NP, and we give an exponential time algorithm for the problem.
Co-Designing with Users the Explanations for a Proactive Auto-Response Messaging Agent

Explanations of AI Agents' actions are considered to be an important factor in improving users' trust in the decisions made by autonomous AI systems. However, as these autonomous systems evolve from reactive, i.e., acting on user input, to proactive, i.e., acting without requiring user intervention, there is a need to explore how the explanation for the actions of these agents should evolve. In this work, we explore the design of explanations through participatory design methods for a proactive auto-response messaging agent that can reduce perceived obligations and social pressure to respond quickly to incoming messages by providing unavailability-related context. We recruited 14 participants who worked in pairs during collaborative design sessions where they reasoned about the agent's design and actions. We qualitatively analyzed the data collected through these sessions and found that participants' reasoning about agent actions led them to speculate heavily on its design. These speculations significantly influenced participants' desire for explanations and the controls they sought to inform the agents' behavior. Our findings indicate a need to transform users' speculations into accurate mental models of agent design. Further, since the agent acts as a mediator in human-human communication, it is also necessary to account for social norms in its explanation design. Finally, user expertise in understanding their habits and behaviors allows the agent to learn from the user their preferences when justifying its actions.
Tap to Sign: Towards using American Sign Language for Text Entry on Smartphones

Soon, smartphones may be capable of allowing American Sign Language (ASL) signing and/or fingerspelling for text entry. To explore the usefulness of this approach, we compared emulated fingerspelling recognition with a virtual keyboard for 12 Deaf participants. With practice, fingerspelling is faster (42.5 wpm), potentially has fewer errors (4.02% corrected error rate) and higher throughput (14.2 bits/second), and is as desired as virtual keyboard texting (31.9 wpm; 6.46% corrected error rate; 10.9 bits/second throughput). Our second study recruits another 12 Deaf users at the 2022 National Association for the Deaf conference to compare the walk-up usability of fingerspelling alone, signing, and virtual keyboard text entry for interacting with an emulated mobile assistant. Both signing and virtual keyboard text entry were preferred over fingerspelling.
Frappé: An Ultra Lightweight Mobile UI Framework for Rapid API-based Prototyping and Environmental Deployment

QR codes have been used as an inexpensive means to connect users to digital platforms such as websites and mobile applications. However, despite their ubiquity, QR codes are limited in purpose and can only redirect users to the URL contained within it, thereby making their use heavily network dependent which can be unsuitable for use in ephemeral scenarios and areas with limited connectivity. In this paper, we introduce Frappé, a framework capable of deploying ultra lightweight UIs to mobile devices directly through QR codes, without requiring any network connectivity. This is achieved by decomposing the UI into metadata and storing it inside the QR code, while offloading the UI functionality to API calls. We also introduce enFrappé, a WYSIWYG tool for building Frappé UIs. We demonstrate the lightweight nature of our framework through a technical evaluation, whereas the usability of our UI builder tool is demonstrated through a user study.
[Don't] Let The Bodies HIIT The Floor: Fostering Body Awareness in Fast-Paced Physical Activity Using Body-Worn Sensors

Technologies have become an integral part of physical activity. Yet, the majority of popular programs do not focus on promoting a genuine understanding of how sport affects our bodies. As apps and trackers persuade users to exercise more, lack of body awareness can be detrimental to health. In this work, we propose and evaluate the concept of in-session reflective feedback as a means to support informed exercise routines by design. We designed and implemented REPLAY, a system which presents users with a visualization of physiological signals (heart rate, movement) from body-worn sensors during high-intensity interval training (HIIT). Our evaluation showed that participants gained a better understanding of how their body reacted to physical activity, allowing them to understand its effect and recognize own weaknesses. Further, our work demonstrates how the type of feedback can significantly moderate a user's perceived exhaustion. We highlight how in-session reflective feedback using bodily signals can promote healthy and effective workouts through creating a deeper understanding of one's own body physiology and limits.