Reviewerly/exHarmony huggingface dataset:
Hand Gesture Recognition for Human-Computer Interaction Using Computer Vision

We use gestures to communicate with friends, family, and colleagues every day. Gestures have always been a natural and intuitive form of interaction and communication, from waving our hands across the hallway to signal someone to keep the elevator doors open or greet friends from afar. Gestures are the universally understood extension of our body language and comprise a core part of everyday interaction. When we consider this idea for vision-based interaction with computers, we get hand gesture recognition for human-computer interaction using computer vision. We perform a predefined set of gestures in front of a camera and assign individual actions to them. Hence, a computer can recognize the said gestures and complete the appropriate actions. We use this exact method to develop a way to interact with computers with little to no contact. The advent of computer vision and machine learning data libraries makes this dream of contactless interactive technology possible. In light of the pandemic, We wish to create an AI virtual computer mouse. We recognize predefined gestures with a camera and assign appropriate actions that a mouse commonly performs to them. Since the limitations of a physical mouse no longer bind us, we can redefine the conventions of operating a computer with a new set of convenient and easy to grasp rules. We are starting this by developing a virtual gesture-based volume controller. We can perform these feats due to the marvelous works done by academics and researchers that have come before us. The main goal is to use the advantages of computer vision and machine learning to combat the pandemic’s challenges and push forward the future of interactive technology. Thus, we get Human-computer interaction using hand recognition technology.
Towards Engineering AI Planning Functionalities as Services

As Artificial Intelligence (AI) planning is utilised in industry to address complex real-world problems, the need for the construction of advanced and deployable AI planning systems emerges. This task, however, proves to be difficult due to a general lack of established mechanisms in AI planning for system design, interoperability, and deployment. In this context, we provide an overview of key engineering challenges that one is faced with when developing and using systems that incorporate AI planning functionalities. To help address the challenges, we propose to leverage service-orientation together with architectural patterns. In particular, we identify a set of planning functionalities, and we present an initial concept for engineering the planning functionalities as services. Having planning services would enable not only to quickly compose and deploy service-oriented planning systems but also expand them with more features as required by application domains.
Time Constrained DL8.5 Using Limited Discrepancy Search

Decision trees that minimize the error on the training set with a depth limit have been found to be generally superior to those found by more standard greedy algorithms. However, when the search space to be explored is too large, the depth-first search used by exact algorithms can get trapped in left most branches. Consequently, when the user stops the algorithm, the best tree found so far may be unbalanced and poorly minimize the error. Our work aims to improve the anytime behavior by introducing the limited discrepancy search ingredient in these algorithms. This allows to explore the search space by waves increasingly deviating from standard heuristics such as information gain. Our experimental results show that the anytime behavior of the state-of-the-art exact method DL8.5 is greatly improved.
Alternative schemes for twin-field quantum key distribution with discrete-phase-randomized sources

The twin-field quantum key distribution (TF-QKD) protocol and its variants can overcome the well-known rate-loss bound without quantum repeaters, which have attracted significant attention. Generally, to ensure the security of these protocols, weak coherent states with continuous randomized phases are always assumed in the test mode. However, this assumption is difficult to meet in practice. To bridge the gap between theory and practice, we propose two alternative discrete-phase-randomized (DPR)-twin-field quantum key distribution protocols, which remove the phase sifting procedure in the code mode. Simulation results show that when compared with previous discrete-phase-randomized-twin-field quantum key distribution protocols, our modified protocols can significantly improve the secret key rate in the low channel loss range, which is very promising for practical twin-field quantum key distribution systems.
ImbalancedLearningRegression - A Python Package to Tackle the Imbalanced Regression Problem

This package helps Python users address imbalanced regression problems. Popular Python packages exist for imbalanced classification. However, there is still little Python support for imbalanced regression. Imbalanced regression is a well-known problem that occurs across domains, where a continuous target variable is poorly represented on ranges that are important to the end-user. Here, a re-sampling strategy is applied to modify the distribution of the target variable, biasing it towards the end-user interests so that downstream learning algorithms can be trained on the most relevant cases. The package provides an easy-to-use and extensible implementation of eight state-of-the-art re-sampling methods for regression, including four under-sampling and four over-sampling techniques. Code related to this paper is available at: https://github.com/paobranco/ImbalancedLearningRegression .
Political views and organizational distrust affect rural residents’ willingness to share personal data for COVID-19 contact tracing: A cross-sectional survey study

We aimed to examine the attitudes of Pennsylvania rural residents toward data sharing in the setting of the COVID-19 pandemic. Specifically, we were interested in better understanding their willingness to provide personal information for contact tracing to public health staff investigating COVID-19 cases, as well as their concerns. We used a validated scale to describe the influence of distrust of healthcare organizations on their attitudes.We mailed 4000 surveys to rural residents identified from the electronic medical record of a healthcare system in central Pennsylvania. Data were entered into a REDCap database and analyzed using descriptive summaries, and both binomial and multivariable logistic regression.Binomial logistic regression showed that both distrust in healthcare organizations and political values influence respondents' willingness to share information with contact tracers as well as their concerns about sharing personal data. When our multivariable model was applied, political values remained and were consistently associated with willingness to share and concerns about sharing their data.This study is a first step in eliciting rural residents' willingness to share personal data for contact tracing by public health officials. Understanding and addressing rural residents' willingness to share personal data and their concerns about sharing those data will help public health officials identify effective strategies for managing COVID-19 and future pandemics in rural communities. By involving community members at the ground level, public health staff can ensure residents' buy-in for the need to collect their personal data, thereby helping to mitigate the public health crises.
Differentiable channel pruning guided via attention mechanism: a novel neural network pruning approach

Abstract Neural network pruning offers great prospects for facilitating the deployment of deep neural networks on computational resource limited devices. Neural architecture search (NAS) provides an efficient way to automatically seek appropriate neural architecture design for compressed model. It is observed that, for existing NAS-based pruning methods, there is usually a lack of layer information when searching the optimal neural architecture. In this paper, we propose a new NAS approach, namely, differentiable channel pruning method guided via attention mechanism (DCP-A), where the adopted attention mechanism is able to provide layer information to guide the optimization of the pruning policy. The training process is differentiable with Gumbel-softmax sampling, while parameters are optimized under a two-stage training procedure. The neural network block with the shortcut is dedicatedly designed, which is of help to prune the network not only on its width but also on its depth. Extensive experiments are performed to verify the applicability and superiority of the proposed method. Detailed analysis with visualization of the pruned model architecture shows that our proposed DCP-A learns explainable pruning policies.
The digitization of historical astrophysical literature with highly localized figures and figure captions

Scientific articles published prior to the "age of digitization" in the late 1990s contain figures which are "trapped" within their scanned pages. While progress to extract figures and their captions has been made, there is currently no robust method for this process. We present a YOLO-based method for use on scanned pages, after they have been processed with Optical character recognition (OCR), which uses both grayscale and OCR features. We focus our efforts on translating the intersection-over-union (IOU) metric from the field of object detection to document layout analysis and quantify "high localization" levels as an IOU of 0.9. When applied to the astrophysics literature holdings of the NASA astrophysics data system, we find F1 scores of 90.9% (92.2%) for figures (figure captions) with the IOU cut-off of 0.9 which is a significant improvement over other state-of-the-art methods.
Modeling for policy and technology assessment: Challenges from computerbased simulations and artificial intelligence

Modeling for policy has become an integral part of policy making and technology assessment. This became particularly evident to the general public when, during the COVID-19 pandemic, forecasts of infection dynamics based on computer simulations were used to evaluate and justify policy containment measures. Computer models are also playing an increasing role in technology assessment (TA). Computer simulations are used to explore possible futures related to specific technologies, for example, in the area of energy systems analysis. Artificial intelligence (AI) models are also becoming increasingly important. The results is a mix of methods where computer simulations and machine learning converge, posing particular challenges and opening up new research questions. This Special topic brings together case studies from different fields to explore the current state of computational models in general and AI methods in particular for policy and TA.
Large Data Oriented to Image Information Fusion Spark and Improved Fruit Fly Optimization Based on the Density Clustering Algorithm

The density-based applied spatial clustering algorithm is an algorithm based on high-density interconnected regions, which discovers class clusters of arbitrary shapes in noisy data sets and is widely used. However, it suffers from slow computation speed due to large-scale disk I/O and clustering bias due to uneven density class clusters and poor parameter search ability. To address these problems, a parallel density clustering algorithm based on an improved fruit fly optimization algorithm and Spark memory iteration is proposed. The proposed algorithm first divides the data grid using an irregular dynamic density region partitioning strategy. Then, a hybrid fruit fly particle swarm algorithm based on a genetic optimization mechanism is proposed to achieve dynamic optimization seeking for parameters in local clustering to improve the clustering effect of local clustering. Finally, the local merging of samples in irregularly bounded grid cells under each partition is achieved by designing a custom clustering merging strategy. The experiments show that the improved algorithm is generally applicable to the clustering of different shape class clusters and larger scale data and has obvious improvement in accuracy and parallel efficiency.
Efficient key frame extraction and hybrid wavelet convolutional manta ray foraging for sports video classification

Sports video classification (SVC) is now considered a challenging topic, therefore, developing an automatic sports scene classification technique has received tremendous interest. This research develops an efficient key frame extraction method and hybrid Wavelet Convolution Neural Network (WCNN) framework with optimization scheme to classify sports videos. Initially, input videos are converted into number of frames, and keyframes are extracted using Enhanced threshold with Discrete Wavelet Transform (ETDWT) method. Then, Cross Guided Bilateral Filter (CGBF) method eliminates the noise from the keyframe. After that, segmentation process is performed by the Fuzzy Equilibrium Optimizer (FEO) algorithm, and then motions are detected using the Farneback optical flow (OF) method. Finally, classification process is performed using Hybrid Wavelet Convolutional Manta Ray Foraging Optimization (HWCMRFO) algorithm to categorize different sports videos. The overall work is implemented using Python language. Simulation results proved that the proposed work achieved the highest accuracy (93.17%) compared to existing approaches.
Follow the Successful Herd: Towards Explanations for Improved Use and Mental Models of Natural Language Systems

While natural language systems continue improving, they are still imperfect. If a user has a better understanding of how a system works, they may be able to better accomplish their goals even in imperfect systems. We explored whether explanations can support effective authoring of natural language utterances and how those explanations impact users' mental models in the context of a natural language system that generates small programs. Through an online study (n=252), we compared two main types of explanations: 1) system-focused, which provide information about how the system processes utterances and matches terms to a knowledge base, and 2) social, which provide information about how other users have successfully interacted with the system. Our results indicate that providing social suggestions of terms to add to an utterance helped users to repair and generate correct flows more than system-focused explanations or social recommendations of words to modify. We also found that participants commonly understood some mechanisms of the natural language system, such as the matching of terms to a knowledge base, but they often lacked other critical knowledge, such as how the system handled structuring and ordering. Based on these findings, we make design recommendations for supporting interactions with and understanding of natural language systems.
Human-Centered Deferred Inference: Measuring User Interactions and Setting Deferral Criteria for Human-AI Teams

Although deep learning holds the promise of novel and impactful interfaces, realizing such promise in practice remains a challenge: since dataset-driven deep-learned models assume a one-time human input, there is no recourse when they do not understand the input provided by the user. Works that address this via deferred inference—soliciting additional human input when uncertain—show meaningful improvement, but ignore key aspects of how users and models interact. In this work, we focus on the role of users in deferred inference and argue that the deferral criteria should be a function of the user and model as a team, not simply the model itself. In support of this, we introduce a novel mathematical formulation, validate it via an experiment analyzing the interactions of 25 individuals with a deep learning-based visiolinguistic model, and identify user-specific dependencies that are under-explored in prior work. We conclude by demonstrating two human-centered procedures for setting deferral criteria that are simple to implement, applicable to a wide variety of tasks, and perform equal to or better than equivalent procedures that use much larger datasets.
IRIS: Interpretable Rubric-Informed Segmentation for Action Quality Assessment

AI-driven Action Quality Assessment (AQA) of sports videos can mimic Olympic judges to help score performances as a second opinion or for training. However, these AI methods are uninterpretable and do not justify their scores, which is important for algorithmic accountability. Indeed, to account for their decisions, instead of scoring subjectively, sports judges use a consistent set of criteria — rubric — on multiple actions in each performance sequence. Therefore, we propose IRIS to perform Interpretable Rubric-Informed Segmentation on action sequences for AQA. We investigated IRIS for scoring videos of figure skating performance. IRIS predicts (1) action segments, (2) technical element score differences of each segment relative to base scores, (3) multiple program component scores, and (4) the summed final score. In a modeling study, we found that IRIS performs better than non-interpretable, state-of-the-art models. In a formative user study, practicing figure skaters agreed with the rubric-informed explanations, found them useful, and trusted AI judgments more. This work highlights the importance of using judgment rubrics to account for AI decisions.
TransASL: A Smart Glass based Comprehensive ASL Recognizer in Daily Life

Sign language is a primary language used by deaf and hard-of-hearing (DHH) communities. However, existing sign language translation solutions primarily focus on recognizing manual markers. The non-manual markers, such as negative head shaking, question markers, and mouthing, are critical grammatical and semantic components of sign language for better usability and generalizability. Considering the significant role of non-manual markers, we propose the TransASL, a real-time, end-to-end system for sign language recognition and translation. TransASL extracts feature from both manual markers and non-manual markers via a customized eyeglasses-style wearable device with two parallel sensing modalities. Manual marker information is collected by two pairs of outward-facing microphones and speakers mounted to the legs of the eyeglasses. In contrast, non-manual marker information is acquired from a pair of inward-facing microphones and speakers connected to the eyeglasses. Both manual and non-manual marker features undergo a multi-modal, multi-channel fusion network and are eventually recognized as comprehensible ASL content. We evaluate the recognition performance of various sign language expressions at both the word and sentence levels. Given 80 frequently used ASL words and 40 meaningful sentences consisting of manual and non-manual markers, TransASL can achieve the WER of 8.3% and 7.1%, respectively. Our proposed work reveals a great potential for convenient ASL recognition in daily communications between ASL signers and hearing people.
SoundToons: Exemplar-Based Authoring of Interactive Audio-Driven Animation Sprites

Animations can come to life when they are synchronized with relevant sounds. Yet, synchronizing animations to audio requires tedious key-framing or programming, which is difficult for novice creators. There are existing tools that support audio-driven live animation, but they focus primarily on speech and have little or no support for non-speech sounds. We present SoundToons, an exemplar-based authoring tool for interactive, audio-driven animation focusing on non-speech sounds. Our tool enables novice creators to author live animations to a wide variety of non-speech sounds, such as clapping and instrumental music. We support two types of audio interactions: (1) discrete interaction, which triggers animations when a discrete sound event is detected, and (2) continuous, which synchronizes an animation to continuous audio parameters. By employing an exemplar-based iterative authoring approach, we empower novice creators to design and quickly refine interactive animations. User evaluations demonstrate that novice users can author and perform live audio-driven animation intuitively. Moreover, compared to other input modalities such as trackpads or foot pedals, users preferred using audio as an intuitive way to drive animation.
Improved Bilinear Pooling With Pseudo Square-Rooted Matrix

Bilinear pooling is a feature aggregation step applied after the convolutional layers of a deep network and encodes a matrix of local features into a fixed-size bilinear representation. It improves performance in many image classification tasks. Since its emergence, this pooling has seen two major improvements: Compact Bilinear Pooling (CBP) and square-root normalization. Recently, the combination of these two elements has been widely studied. However, due to the lack of good normalization solutions, existing combination approaches showed less efficiency when they are plugged into different networks and less compatibility when they work with existing CBP techniques. To solve this problem, in this paper, we propose to apply Newton iterations, a fast square-root normalization method, to produce a new normalized matrix called <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">pseudo square-rooted matrix</i> . Subsequently, the new matrix allows a CBP technique to encode itself into a compact and normalized bilinear representation. In order to further accelerate the normalization process, our approach has two variants which can handle feature matrix extracted by different networks. Tested on three fine-grained image classification datasets, it provides competitive classification performance while consuming less computational time than other prior works.
Communication-Efficient Federated Learning for Power Load Forecasting in Electric IoTs

With the construction of the modern power system, power load forecasting is significant to keep the electric Internet of Things in operation. However, it usually needs to collect massive power load data on the server and may face the problem of privacy leakage of raw data. Federated learning can enhance the privacy of the raw power load data of clients by frequently transmitting model updates. Concerning the increasing communication burden of resource-heterogeneous clients resulting from frequent communication with the server, a communication-efficient federated learning algorithm based on Compressed Model Updates and Lazy uploAd (CMULA-FL) was proposed to reduce the communication cost. CMULA-FL also integrates the error compensation strategy to improve the model utility. First, the compression operator is used to compress the transmitted model updates, of which large norms are uploaded to reduce the communication cost of each epoch and transmission frequency. Second, by measuring the error of compression and lazy upload, the error is accumulated to the next epoch to improve the model utility. Finally, based on simulation experiments on the benchmark power load data, the results show that the communication cost decreases at least 60% with controlled loss of model prediction compared with baseline.