Reviewerly/exHarmony huggingface dataset:
Understanding Gesture and Speech Multimodal Interactions for Manipulation Tasks in Augmented Reality Using Unconstrained Elicitation

This research establishes a better understanding of the syntax choices in speech interactions and of how speech, gesture, and multimodal gesture and speech interactions are produced by users in unconstrained object manipulation environments using augmented reality. The work presents a multimodal elicitation study conducted with 24 participants. The canonical referents for translation, rotation, and scale were used along with some abstract referents (create, destroy, and select). In this study time windows for gesture and speech multimodal interactions are developed using the start and stop times of gestures and speech as well as the stoke times for gestures. While gestures commonly precede speech by 81 ms we find that the stroke of the gesture is commonly within 10 ms of the start of speech. Indicating that the information content of a gesture and its co-occurring speech are well aligned to each other. Lastly, the trends across the most common proposals for each modality are examined. Showing that the disagreement between proposals is often caused by a variation of hand posture or syntax. Allowing us to present aliasing recommendations to increase the percentage of users' natural interactions captured by future multimodal interactive systems.
Lightweight Deep Learning Model for Automatic Modulation Classification in Cognitive Radio Networks

Automatic modulation classification (AMC) used in cognitive radio networks is an important class of methods apt to utilize spectrum resources efficiently. However, conventional likelihood-based approaches have high computational complexity. Thus, this paper proposes a novel convolutional neural network architecture for AMC. A bottleneck and asymmetric convolution structure are employed in the proposed model, which can reduce the computational complexity. The skip connection technique is used to solve the vanishing gradient problem and improve the classification accuracy. The dataset DeepSig:RadioML, which is composed of 24 modulation classes, is used for the performance analysis. Simulation results show that the classification accuracy performance of the proposed model is outstanding in the signal-to-noise ratio (SNR) range from -4 dB to 20 dB compared with MCNet that is the best model in the conventional models, where the proposed model achieves 5.52% and 5.92% improvement regarding classification accuracy at the SNRs of 0 dB and 10 dB, respectively. In terms of the computational complexity, the proposed model not only saves the trainable parameters by more than 67% but also reduces the prediction time for a signal by more than 54.4% compared with those of MCNet.
Performance Evaluation of Supercomputer Fugaku using Breadth-First Search Benchmark in Graph500

There is increasing demand for the high-speed processing of large-scale graphs in various fields. However, such graph processing requires irregular calculations, making it difficult to scale performance on large-scale distributed memory systems. Against this background, Graph500, a competition for evaluating the performance of large-scale graph processing, has been held. We developed breadth-first search (BFS), which is one of the benchmark kernels used in Graph500, and took the top spot a total of 10 times using the K computer. In this paper, we tune BFS performance and evaluate it using the supercomputer Fugaku, which is the successor to the K computer. The results of evaluating BFS for a large-scale graph composed of about 1.1 trillion vertices and 17.6 trillion edges using 92,160 nodes of Fugaku indicate that Fugaku has 2.27 times the performance of the K computer. Fugaku took the top spot on Graph500 in June 2020.
Enhanced Group Sparse Regularized Nonconvex Regression for Face Recognition

Regression analysis based methods have shown strong robustness and achieved great success in face recognition. In these methods, convex l1-norm and nuclear norm are usually utilized to approximate the l0-norm and rank function. However, such convex relaxations may introduce a bias and lead to a suboptimal solution. In this paper, we propose a novel Enhanced Group Sparse regularized Nonconvex Regression (EGSNR) method for robust face recognition. An upper bounded nonconvex function is introduced to replace l1-norm for sparsity, which alleviates the bias problem and adverse effects caused by outliers. To capture the characteristics of complex errors, we propose a mixed model by combining γ-norm and matrix γ-norm induced from the nonconvex function. Furthermore, an l2,γ-norm based regularizer is designed to directly seek the interclass sparsity or group sparsity instead of traditional l2,1-norm. The locality of data, i.e., the distance between the query sample and multi-subspaces, is also taken into consideration. This enhanced group sparse regularizer enables EGSNR to learn more discriminative representation coefficients. Comprehensive experiments on several popular face datasets demonstrate that the proposed EGSNR outperforms the state-of-the-art regression based methods for robust face recognition.
Query Expansion Based on Top-Ranked Images for Content-Based Medical Image Retrieval

Recently the collection of varied digital image databases has increased.Many users have found that searching and retrieving required images from large collections are very difficult tasks, and successful and effective retrieval methods were developed to provide an effective and efficient search and retrieval process.In most content-based image retrieval (CBIR) methods, various visual features have been considered indirect to retrieve the images from databases.Content-based medical image retrieval (CBMIR), like any CBIR method, is a technique for retrieving medical images on the basis of automatically derived image features, such as colour and texture.Although a number of methods and approaches have been suggested, retrieval performance remains one of the most challenging problems in current CBMIR studies, this due to the well-known 'semantic gap' issue that exists between machine-captured low-level image features and human-perceived high-level semantic concepts.To bridge this gap, much research has been proposed.This study proposes two expansion methods to increase and enhance precision of the retrieval model; both of our proposed methods depend on top-ranked images.Our first expansion method is to reformulate the new expand query image based on mean values for features of top-ranked images, while the second method is based on selecting the most important features only.The expansion process was performed on eighteen colour features and twelve texture features extracted from two common medical images datasets, Kvasir and PH2.Experimental results show that our proposed methods performed better and have a precision of 95.8% and 80.7% for Kvasir and PH2 data set, respectively.
Decolonizing Learning Spaces for Sociotechnical Research and Design

As spaces for learning about Computer-Supported Collaborative Work (CSCW) research and practice (e.g., university classes, academic and industry labs, conferences) become more diverse, there is a pressing need to revise the universal collaborative and pedagogical structures supporting them. Specifically, it has become urgent to explore how to 'de-center' dominant assumptions about who learns in these environments. The goal of this workshop is to explore collectively how to craft learning spaces that resist universality by recognizing and valuing other perspectives and realities. We build on the scholarship of decolonial thinkers, which provides useful theoretical scaffolding on how to start working towards inclusivity and 'pluriversality'. That is, learning spaces where all views can co-exist as equally valid, albeit contradicting. Our workshop will be led by researchers and designers who have both guided and participated in academic and industry-based CSCW learning spaces across domains like Social Computing, Human-Computer Interaction (HCI), Information and Communication Technologies for Development (ICTD), Critical Data Studies, and Participatory Design (PD). We invite a broad range of participants from research and practice interested in learning about or deepening their understanding of how to make of CSCW a more 'pluriversal' site for learning and practicing.
Contribution-Based Cooperative Co-Evolution for Nonseparable Large-Scale Problems With Overlapping Subcomponents

Cooperative co-evolutionary algorithms have addressed many large-scale problems successfully, but the nonseparable large-scale problems with overlapping subcomponents are still a serious difficulty that has not been conquered yet. First, the existence of shared variables makes the problem hard to be decomposed. Second, existing cooperative co-evolutionary frameworks usually cannot maintain the two crucial factors: high cooperation frequency and effective computing resource allocation, simultaneously when optimizing the overlapping subcomponents. Aiming at these two issues, this article proposes a new contribution-based cooperative co-evolutionary algorithm to decompose and optimize nonseparable large-scale problems with overlapping subcomponents effectively and efficiently: 1) a contribution-based decomposition method is proposed to assign the shared variables. Among all the subcomponents containing a shared variable, the one that contributes the most to the entire problem will include the shared variable and 2) to achieve the two crucial factors at the same time, a new contribution-based optimization framework is designed to award the important subcomponents based on the round-robin structure. Experimental studies show that the proposed algorithm performs significantly better than the state-of-the-art algorithms due to the effective grouping structure generated by the proposed decomposition method and the fast optimizing speed provided by the new optimization framework.
Robust Deep Learning Method to Detect Face Masks

With the outbreak of novel coronavirus (2019-nCoV), wearing masks has become an effective way to prevent the transmission of the virus. But in public places, people are often reluctant to wear face masks and cause the virus to spread widely. This paper uses an efficient and robust object detection algorithm to automatically detect the faces with masks or without masks, making the epidemic prevention work more intelligent. Specifically, we collected an extensive database of 9886 images of people with and without face masks and manually labeled them, then use multi-scale training and image mixup methods to improve YOLOv3, an object detection algorithm, to automatically detect whether a face is wearing a mask. Our experiment results demonstrate that the mean Average Precision (mAP) of the improved YOLOv3 algorithm model reached 86.3%. This work can effectively and automatically detect whether people are wearing masks, which reduces the pressure of deploying human resources for checking masks in public places and has high practical application value.
Implementing the Exponential Mechanism with Base-2 Differential Privacy

Despite excellent theoretical support, Differential Privacy (DP) can still be a challenge to implement in practice. In part, this challenge is due to the concerns associated with translating arbitrary- or infinite-precision theoretical mechanisms to the reality of floating point or fixed-precision. Beginning with the troubling result of Mironov demonstrating the security issues of using floating point for implementing the Laplace mechanism, there have been many reasonable questions raised concerning the vulnerabilities of real-world implementations of DP. In this work, we examine the practicalities of implementing the exponential mechanism of McSherry and Talwar. We demonstrate that naive or malicious implementations can result in catastrophic privacy failures. To address these problems, we show that the mechanism can be implemented exactly for a rich set of utility functions and values of the privacy parameter epsilon with limited practical overhead in running time and minimal code complexity. How do we achieve this result? We employ a simple trick of switching from base-e to base 2, allowing us to perform precise base-2 arithmetic. A short, precise expression is always available for epsilon, and the only approximation error we incur is the conversion of the base-2 privacy parameter back to base-e for reporting purposes. The core base-2 arithmetic of the mechanism can be simply and efficiently implemented using open-source high precision arithmetic libraries. Furthermore, the exact nature of the implementation lends itself to simple monitoring of correctness and proofs of privacy.
Structure-Guided Deep Video Inpainting

A fundamental challenge in video inpainting is the difficulty of generating video contents with fine details, while keeping spatio-temporal coherence in the missing region. Recent studies focus on synthesizing temporally smooth pixels by exploiting the flow information, while ignoring maintaining the semantic structural coherence between frames. This makes them suffer from over-smoothing and blurry contours, which significantly reduce the visual quality of inpainting results. To address this issue, we present a novel structure-guided video inpainting approach that enhances temporal structure coherence to improve video inpainting results. In contrast to directly synthesizing the missing pixel colors, we first complete edges in the missing regions to depict scene structures and object shapes via an edge inpainting network with 3D convolutions. Then, we replenish textures using a coarse-to-fine synthesis network with a structure attention module (SAM), under the guidance of the synthesized edges. Specifically, our SAM is designed to model the semantic correlation between video textures and structural edges to generate more realistic content. Besides, motion flows between neighboring frames are employed to enhance temporal consistency for self-supervision during training the edge inpainting and texture inpainting modules. Consequently, the inpainting results using our approach are visually pleasing with fine details and temporal coherence. Experiments on the YouTubeVOS, DAVIS, and 300VW datasets show that our method obtains state-of-the-art performance under diverse video inpainting settings.
Cloud-Based Monitoring of Thermal Anomalies in Industrial Environments Using AI and the Internet of Robotic Things

Recent advancements in cloud computing, artificial intelligence, and the internet of things (IoT) create new opportunities for autonomous industrial environments monitoring. Nevertheless, detecting anomalies in harsh industrial settings remains challenging. This paper proposes an edge-fog-cloud architecture with mobile IoT edge nodes carried on autonomous robots for thermal anomalies detection in aluminum factories. We use companion drones as fog nodes to deliver first response services and a cloud back-end for thermal anomalies analysis. We also propose a self-driving deep learning architecture and a thermal anomalies detection and visualization algorithm. Our results show our robot surveyors are low-cost, deliver reduced response time, and more accurately detect anomalies compared to human surveyors or fixed IoT nodes monitoring the same industrial area. Our self-driving architecture has a root mean square error of 0.19 comparable to VGG-19 with a significantly reduced complexity and three times the frame rate at 60 frames per second. Our thermal to visual registration algorithm maximizes mutual information in the image-gradient domain while adapting to different resolutions and camera frame rates.
A Bayesian end-to-end model with estimated uncertainties for simple question answering over knowledge bases

Existing methods for question answering over knowledge bases (KBQA) ignore the consideration of the model prediction uncertainties. We argue that estimating such uncertainties is crucial for the reliability and interpretability of KBQA systems. Therefore, we propose a novel end-to-end KBQA model based on Bayesian Neural Network (BNN) to estimate uncertainties arose from both model and data. To our best knowledge, we are the first to consider the uncertainty estimation problem for the KBQA task using BNN. The proposed end-to-end model integrates entity detection and relation prediction into a unified framework, and employs BNN to model entity and relation under the given question semantics, transforming network weights into distributions. Therefore, predictive distributions can be estimated by sampling weights and forward inputs through the network multiple times. Uncertainties can be further quantified by calculating the variances of predictive distributions. The experimental results demonstrate the effectiveness of uncertainties in both the misclassification detection task and cause of error detection task. Furthermore, the proposed model also achieves comparable performance compared to the existing state-of-the-art approaches on SimpleQuestions dataset.
TRUSTORE: Side-Channel Resistant Storage for SGX using Intel Hybrid CPU-FPGA

Intel SGX is a security solution promising strong and practical security guarantees for trusted computing. However, recent reports demonstrated that such security guarantees of SGX are broken due to access pattern based side-channel attacks, including page fault, cache, branch prediction, and speculative execution. In order to stop these side-channel attackers, Oblivious RAM (ORAM) has gained strong attention from the security community as it provides cryptographically proven protection against access pattern based side-channels. While several proposed systems have successfully applied ORAM to thwart side-channels, those are severely limited in performance and its scalability due to notorious performance issues of ORAM. This paper presents TrustOre, addressing these issues that arise when using ORAM with Intel SGX. TrustOre leverages an external device, FPGA, to implement a trusted storage service within a completed isolated environment secure from side-channel attacks. TrustOre tackles several challenges in achieving such a goal: extending trust from SGX to FPGA without imposing architectural changes, providing a verifiably-secure connection between SGX applications and FPGA, and seamlessly supporting various access operations from SGX applications to FPGA.We implemented TrustOre on the commodity Intel Hybrid CPU-FPGA architecture. Then we evaluated with three state-of-the-art ORAM-based SGX applications, ZeroTrace, Obliviate, and Obfuscuro, as well as an end-to-end key-value store application. According to our evaluation, TrustOre-based applications outperforms ORAM-based original applications ranging from 10x to 43x, while also showing far better scalability than ORAM-based ones. We emphasize that since TrustOre can be deployed as a simple plug-in to SGX machine's PCIe slot, it is readily used to thwart side-channel attacks in SGX, arguably one of the most cryptic and critical security holes today.
A Cognitive Particle Filter for Collaborative DGNSS Positioning

The advances in low-latency communications networks and the ever-growing amount of devices offering localization and navigation capabilities opened a number of opportunities to develop innovative network-based collaborative solutions to satisfy the increasing demand for positioning accuracy and precision. Recent research works indeed, have fostered the concept of networked Global Navigation Satellite System (GNSS) receivers supporting the sharing of raw measurements with other receivers within the same network. Such measurements (i.e. pseudorange and Doppler) can be processed through Differential GNSS (DGNSS) techniques to retrieve inter-agent distances which can be in turn integrated to improve positioning performance. This article investigates an improved Bayesian estimation algorithm for a sensorless, tight-integration of DGNSS-based collaborative measurements through a modified Particle Filter (PF), namely Cognitive PF. Differently from Extended Kalman Filter and Uscented Kalman Filter indeed, a PF natively support the non-Gaussian noise distribution which characterizes DGNSS-based inter-agent distances. The proposed Cognitive PF is hence designed, implemented and optimized according to the architecture of a proprietary Inertial Navigation System (INS)-free Global Navigation Satellite System (GNSS) software receiver. Experimental tests performed through realistic radio-frequency GNSS signals showed a remarkable improvement in positioning accuracy w.r.t. reference PF and EKF architectures.
HIBOG: Improving the clustering accuracy by ameliorating dataset with gravitation

Clustering is an important technology applied in many fields. Most researchers focus on only clustering algorithms when they want more accurate results. However, this is not an optimal strategy because each algorithm has its unique advantages and disadvantages. Furthermore, a given algorithm cannot get satisfactory results on all datasets. In this paper, focusing on datasets, a method called HIBOG is proposed to improve the clustering accuracy by ameliorating datasets with gravitation. HIBOG can help many clustering algorithms acquire better results on more datasets by ameliorating datasets so that similar objects get closer and dissimilar objects separate further apart. As a result, ameliorated datasets are friendlier to many clustering algorithms than original datasets. Though datasets are diverse, HIBOG can cope with the diversity to some extent due to its robustness to high dimensional datasets, Gaussian distribution datasets, shaped datasets, and datasets with high overlap clusters. We have conducted numerous experiments on real-world datasets to verify the effectiveness of HIBOG. The experiments demonstrated that HIBOG successfully improves the accuracy of different clustering algorithms, and accuracy increases by an average of 113.4% (except maximum and minimum). Moreover, compared with other similar methods, HIBOG improves much higher clustering accuracy and dramatically shortens the running time. At the same time, we conducted 360 experiments, each of which selected different parameter values. The experiments show that most values enable HIBOG to ameliorate datasets, and HIBOG has strong robustness to the parameter adjustment.
Residual Attention SiameseRPN for Visual Tracking

Visual tracking demands to perform the accurate object location given the object state of the first frame. The existing methods have proposed various ways to handle the challenging problems, yet few of them take the relationship between shallow features and deep semantic features into account. Based on an extensive analysis, we first propose a residual attention SiameseRPN visual tracking method for accurate object state estimation, which introduces the correlation filter in a Siamese network framework. A novel loss function is presented to enhance the discriminative capability. Our approach is derived from three different loss terms that is capable of training a model in a few iterations. Second, we present channel attention mechanism to improve the tracking performance, which is offline trained to capture the general features in the tracking. Third, the proposed tracking model is trained in end-to-end manner and takes full advantage of both low-level representation for correlation filter and high-level semantic features for deep object representation by using multi-task learning strategy which can mine the relationship from both levels. Our approach benefits from two complementary effects. Finally, extensive evaluation and ablation studies demonstrate the effectiveness of the proposed tracking approach. Our tracker achieves state-of-the-art performance on five challenging benchmarks, which proves great potentials in balancing accuracy and speed.