Reviewerly/exHarmony huggingface dataset:
Shared Autonomy with Learned Latent Actions

Assistive robots enable people with disabilities to conduct everyday tasks on their own.However, these tasks can be complex, containing both coarse reaching motions and fine-grained manipulation.For example, when eating, not only does one need to move to the correct food item, but they must also precisely manipulate the food in different ways (e.g., cutting, stabbing, scooping).Shared autonomy methods make robot teleoperation safer and more precise by arbitrating user inputs with robot controls.However, these works have focused mainly on the high-level task of reaching a goal from a discrete set, while largely ignoring manipulation of objects at that goal.Meanwhile, dimensionality reduction techniques for teleoperation map useful high-dimensional robot actions into an intuitive lowdimensional controller, but it is unclear if these methods can achieve the requisite precision for tasks like eating.Our insight is that-by combining intuitive embeddings from learned latent actions with robotic assistance from shared autonomy-we can enable precise assistive manipulation.In this work, we adopt learned latent actions for shared autonomy by proposing a new model structure that changes the meaning of the human's input based on the robot's confidence of the goal.We show convergence bounds on the robot's distance to the most likely goal, and develop a training procedure to learn a controller that is able to move between goals even in the presence of shared autonomy.We evaluate our method in simulations and an eating user study.See videos of our experiments here: https://youtu.be/7BouKojzVyk.
An Automatic Arabic Sign Language Recognition System based on Deep CNN: An Assistive System for the Deaf and Hard of Hearing

People with disabilities have long been ignored.With the advancement of recent technologies, so many tools and software are designed for disabled people to improve their lives.In this research, the Arabic Sign Language (ArSL) recognition system is developed using the proposed architecture of the Deep Convolutional Neural Network (CNN).The aim is to help people with hearing problems to communicate with normal people.The proposed system recognizes the signs of the Arabic alphabet based on real-time user input.The Deep CNN architectures were trained and tested using a database of more than 50000 Arabic sign images collected from random participants of different age groups.Several experiments are performed with changing CNN architectural design parameters in order to get the best recognition rates.The experimental results show that the proposed Deep CNN architecture achieves an excellent accuracy of 97.6%, which is higher than the accuracy achieved by similar other studies.
Artificial Intelligence Security in 5G Networks: Adversarial Examples for Estimating a Travel Time Task

With the rapid development of the Internet, the nextgeneration network (5G) has emerged. 5G can support a variety of new applications, such as the Internet of Things (IoT), virtual reality (VR), and the Internet of Vehicles. Most of these new applications depend on deep learning algorithms, which have made great advances in many areas of artificial intelligence (AI). However, researchers have found that AI algorithms based on deep learning pose numerous security problems. For example, deep learning is susceptible to a well-designed input sample formed by adding small perturbations to the original sample. This well-designed input with small perturbations, which are imperceptible to humans, is called an adversarial example. An adversarial example is similar to a truth example, but it can render the deep learning model invalid. In this article, we generate adversarial examples for spatiotemporal data. Based on the travel time estimation (TTE) task, we use two methods-white-box and blackbox attacks-to invalidate deep learning models. Experiment results show that the adversarial examples successfully attack the deep learning model and thus that AI security is a big challenge of 5G.
Generalizing Spatial Transformers to Projective Geometry with Applications to 2D/3D Registration

Differentiable rendering is a technique to connect 3D scenes with corresponding 2D images. Since it is differentiable, processes during image formation can be learned. Previous approaches to differentiable rendering focus on mesh-based representations of 3D scenes, which is inappropriate for medical applications where volumetric, voxelized models are used to represent anatomy. We propose a novel Projective Spatial Transformer module that generalizes spatial transformers to projective geometry, thus enabling differentiable volume rendering. We demonstrate the usefulness of this architecture on the example of 2D/3D registration between radiographs and CT scans. Specifically, we show that our transformer enables end-to-end learning of an image processing and projection model that approximates an image similarity function that is convex with respect to the pose parameters, and can thus be optimized effectively using conventional gradient descent. To the best of our knowledge, we are the first to describe the spatial transformers in the context of projective transmission imaging, including rendering and pose estimation. We hope that our developments will benefit related 3D research applications. The source code is available at https://github.com/gaocong13/Projective-Spatial-Transformers.
Comparing VR- and AR-Based Try-On Systems Using Personalized Avatars

Despite the convenience offered by e-commerce, online apparel shopping presents various product-related risks, as consumers can neither physically see nor try products on themselves. Augmented reality (AR) and virtual reality (VR) technologies have been used to improve the shopping online experience. Therefore, we propose an AR- and VR-based try-on system that provides users a novel shopping experience where they can view garments fitted onto their personalized virtual body. Recorded personalized motions are used to allow users to dynamically interact with their dressed virtual body in AR. We conducted two user studies to compare the different roles of VR- and AR-based try-ons and validate the impact of personalized motions on the virtual try-on experience. In the first user study, the mobile application with the AR- and VR-based try-on is compared to a traditional e-commerce interface. In the second user study, personalized avatars with pre-defined motion and personalized motion is compared to a personalized no-motion avatar with AR-based try-on. The result shows that AR- and VR-based try-ons can positively influence the shopping experience, compared with the traditional e-commerce interface. Overall, AR-based try-on provides a better and more realistic garment visualization than VR-based try-on. In addition, we found that personalized motions do not directly affect the user’s shopping experience.
Faster, Simpler and More Accurate Hybrid ASR Systems Using Wordpieces

In this work, we first show that on the widely used LibriSpeech benchmark, our transformer-based context-dependent connectionist temporal classification (CTC) system produces state-of-the-art results. We then show that using wordpieces as modeling units combined with CTC training, we can greatly simplify the engineering pipeline compared to conventional frame-based cross-entropy training by excluding all the GMM bootstrapping, decision tree building and force alignment steps, while still achieving very competitive word-error-rate. Additionally, using wordpieces as modeling units can significantly improve runtime efficiency since we can use larger stride without losing accuracy. We further confirm these findings on two internal VideoASR datasets: German, which is similar to English as a fusional language, and Turkish, which is an agglutinative language.
Recent Advances in Digital Multimedia Tampering Detection for Forensics Analysis

In the digital multimedia era, digital forensics is becoming an emerging area of research thanks to the large amount of image and video files generated. Ensuring the integrity of such media is of great importance in many situations. This task has become more complex, especially with the progress of symmetrical and asymmetrical network structures which make their authenticity difficult. Consequently, it is absolutely imperative to discover all possible modes of manipulation through the development of new forensics detector tools. Although many solutions have been developed, tamper-detection performance is far from reliable and it leaves this problem widely open for further investigation. In particular, many types of multimedia fraud are difficult to detect because some evidences are not exploited. For example, the symmetry and asymmetry inconsistencies related to visual feature properties are potential when applied at multiple scales and locations. We explore here this topic and propose an understandable soft taxonomy and a deep overview of the latest research concerning multimedia forgery detection. Then, an in-depth discussion and future directions for further investigation are provided. This work offers an opportunity for researchers to understand the current active field and to help them develop and evaluate their own image/video forensics approaches.
Exploring Transformers for Large-Scale Speech Recognition

While recurrent neural networks still largely define state-of-the-art speech recognition systems, the Transformer network has been proven to be a competitive alternative, especially in the offline condition. Most studies with Transformers have been constrained in a relatively small scale setting, and some forms of data argumentation approaches are usually applied to combat the data sparsity issue. In this paper, we aim at understanding the behaviors of Transformers in the large-scale speech recognition setting, where we have used around 65,000 hours of training data. We investigated various aspects on scaling up Transformers, including model initialization, warmup training as well as different Layer Normalization strategies. In the streaming condition, we compared the widely used attention mask based future context lookahead approach to the Transformer-XL network. From our experiments, we show that Transformers can achieve around 6% relative word error rate (WER) reduction compared to the BLSTM baseline in the offline fashion, while in the streaming fashion, Transformer-XL is comparable to LC-BLSTM with 800 millisecond latency constraint.
Unsupervised Video Object Segmentation with Joint Hotspot Tracking

Object tracking is a well-studied problem in computer vision while identifying salient spots of objects in a video is a less explored direction in the literature. Video eye gaze estimation methods aim to tackle a related task but salient spots in those methods are not bounded by objects and tend to produce very scattered, unstable predictions due to the noisy ground truth data. We reformulate the problem of detecting and tracking of salient object spots as a new task called object hotspot tracking. In this paper, we propose to tackle this task jointly with unsupervised video object segmentation, in real-time, with a unified framework to exploit the synergy between the two. Specifically, we propose a Weighted Correlation Siamese Network (WCS-Net) which employs a Weighted Correlation Block (WCB) for encoding the pixel-wise correspondence between a template frame and the search frame. In addition, WCB takes the initial mask/hotspot as guidance to enhance the influence of salient regions for robust tracking. Our system can operate online during inference and jointly produce the object mask and hotspot track-lets at 33 FPS. Experimental results validate the effectiveness of our network design, and show the benefits of jointly solving the hotspot tracking and object segmentation problems. In particular, our method performs favorably against state-of-the-art video eye gaze models in object hotspot tracking, and outperforms existing methods on three benchmark datasets for unsupervised video object segmentation.
How Can Blockchain Technology Accelerate Energy Efficiency Interventions? A Use Case Comparison

This paper qualitatively evaluates the application of blockchain technology for three energy efficiency use cases. To achieve the Sustainable Development Agenda, energy efficiency improvements have to double by 2030. However, the adoption of energy efficiency interventions is slow due to several market barriers. Blockchain technology is a nascent technology with the potential to address these barriers or even fundamentally change energy system designs, by enabling transparent, decentralised, and tamper-resilient systems. Nevertheless, a blockchain application comes with trade-offs and needs to be considered on a case by case basis. In this paper, we examine the benefits and constraints of a blockchain application for three different approaches to achieving energy efficiency: (i) peer-to-peer (P2P) energy trading; (ii) White Certificate Scheme (WCS); and (iii) Energy Service Companies (ESCOs). For each of these cases, we apply a decision framework to assess blockchain feasibility and outline a potential blockchain-based design. The analysis shows that blockchain functions are case dependent and that an application creates different governance and system designs due to varying case characteristics. We discuss how the identified blockchain adoption barriers can be overcome and stress the need for policy action to advance the development of pilot studies. By decentralising system governance, blockchain enables innovative designs that can accelerate the implementation of energy efficiency interventions.
Beta Embeddings for Multi-Hop Logical Reasoning in Knowledge Graphs

One of the fundamental problems in Artificial Intelligence is to perform complex multi-hop logical reasoning over the facts captured by a knowledge graph (KG). This problem is challenging, because KGs can be massive and incomplete. Recent approaches embed KG entities in a low dimensional space and then use these embeddings to find the answer entities. However, it has been an outstanding challenge of how to handle arbitrary first-order logic (FOL) queries as present methods are limited to only a subset of FOL operators. In particular, the negation operator is not supported. An additional limitation of present methods is also that they cannot naturally model uncertainty. Here, we present BetaE, a probabilistic embedding framework for answering arbitrary FOL queries over KGs. BetaE is the first method that can handle a complete set of first-order logical operations: conjunction ($\wedge$), disjunction ($\vee$), and negation ($\neg$). A key insight of BetaE is to use probabilistic distributions with bounded support, specifically the Beta distribution, and embed queries/entities as distributions, which as a consequence allows us to also faithfully model uncertainty. Logical operations are performed in the embedding space by neural operators over the probabilistic embeddings. We demonstrate the performance of BetaE on answering arbitrary FOL queries on three large, incomplete KGs. While being more general, BetaE also increases relative performance by up to 25.4% over the current state-of-the-art KG reasoning methods that can only handle conjunctive queries without negation.
The Curse of Performance Instability in Analysis Datasets: Consequences, Source, and Suggestions

We find that the performance of state-of-theart models on Natural Language Inference (NLI) and Reading Comprehension (RC) analysis/stress sets can be highly unstable.This raises three questions: (1) How will the instability affect the reliability of the conclusions drawn based on these analysis sets? (2) Where does this instability come from?(3) How should we handle this instability and what are some potential solutions?For the first question, we conduct a thorough empirical study over analysis sets and find that in addition to the unstable final performance, the instability exists all along the training curve.We also observe lower-than-expected correlations between the analysis validation set and standard validation set, questioning the effectiveness of the current model-selection routine.Next, to answer the second question, we give both theoretical explanations and empirical evidence regarding the source of the instability, demonstrating that the instability mainly comes from high inter-example correlations within analysis sets.Finally, for the third question, we discuss an initial attempt to mitigate the instability and suggest guidelines for future work such as reporting the decomposed variance for more interpretable results and fair comparison across models. 1
Exploiting Structured Knowledge in Text via Graph-Guided Representation Learning

In this work, we aim at equipping pre-trained language models with structured knowledge.We present two self-supervised tasks learning over raw text with the guidance from knowledge graphs.Building upon entity-level masked language models, our first contribution is an entity masking scheme that exploits relational knowledge underlying the text.This is fulfilled by using a linked knowledge graph to select informative entities and then masking their mentions.In addition, we use knowledge graphs to obtain distractors for the masked entities, and propose a novel distractor-suppressed ranking objective that is optimized jointly with masked language model.In contrast to existing paradigms, our approach uses knowledge graphs implicitly, only during pre-training, to inject language models with structured knowledge via learning from raw text.It is more efficient than retrieval-based methods that perform entity linking and integration during finetuning and inference, and generalizes more effectively than the methods that directly learn from concatenated graph triples.Experiments show that our proposed model achieves improved performance on five benchmarks, including question answering and knowledge base completion.
Lower Bounds and Optimal Algorithms for Personalized Federated Learning

In this work, we consider the optimization formulation of personalized federated learning recently introduced by Hanzely and Richtarik (2020) which was shown to give an alternative explanation to the workings of local {\tt SGD} methods. Our first contribution is establishing the first lower bounds for this formulation, for both the communication complexity and the local oracle complexity. Our second contribution is the design of several optimal methods matching these lower bounds in almost all regimes. These are the first provably optimal methods for personalized federated learning. Our optimal methods include an accelerated variant of {\tt FedProx}, and an accelerated variance-reduced version of {\tt FedAvg}/Local {\tt SGD}. We demonstrate the practical superiority of our methods through extensive numerical experiments.
Embedding a Blockchain Technology Pattern Into the QR Code for an Authentication Certificate

In the disruptive 4.0 era that emphasizes technological sophistication, blockchain is present as a technology that increasingly influences human life, helping humans in all aspects, including education. The role of blockchain technology in the world of education is to test the validity of diplomas, the increasing number of fake diplomas for an interest, both for work and continuing education to a higher level. The purpose of this research with the implementation of blockchain is expected to make it easier for users to verify the authenticity of a diploma. This study uses the SWOT analysis method to identify all possibilities that exist in blockchain technology. The final result of this research, the system will print a physical certificate in the form of paper in general, then the certificate will be printed a QR code. To verify numeric code on QR Code via scanning on smartphone or QR Reader. It is hoped that the blockchain technology applied to digital assets can reduce cases of forgery of diplomas and other important documents.
Progressive Network Grafting for Few-Shot Knowledge Distillation

Knowledge distillation has demonstrated encouraging performances in deep model compression. Most existing approaches, however, require massive labeled data to accomplish the knowledge transfer, making the model compression a cumbersome and costly process. In this paper, we investigate the practical few-shot knowledge distillation scenario, where we assume only a few samples without human annotations are available for each category. To this end, we introduce a principled dual-stage distillation scheme tailored for few-shot data. In the first step, we graft the student blocks one by one onto the teacher, and learn the parameters of the grafted block intertwined with those of the other teacher blocks. In the second step, the trained student blocks are progressively connected and then together grafted onto the teacher network, allowing the learned student blocks to adapt themselves to each other and eventually replace the teacher network. Experiments demonstrate that our approach, with only a few unlabeled samples, achieves gratifying results on CIFAR10,CIFAR100, and ILSVRC-2012. On CIFAR10 and CIFAR100, our performances are even on par with those of knowledge distillation schemes that utilize the full datasets. The source code is available at https://github.com/zju-vipa/NetGraft.
Semi-supervised multi-label feature selection with adaptive structure learning and manifold learning

High-dimensional multi-label data brings challenges and difficulties in multi-label learning. Therefore, feature selection as an effective dimension reduction technique is widely used in multi-label learning. Due to the high cost of collecting sufficient labels, semi-supervised multi-label feature selection has received much attention in recent years. Nevertheless, existing semi-supervised multi-label feature selection methods mainly use manifold assumption to explore the label correlations. However, due to a large amount of unlabeled data, the label correlations are inaccurate and insufficient. Therefore, we need more multi-label data structure information to guide feature selection. In this paper, we propose a unified learning framework combining adaptive global structure learning and manifold learning(SFAM). Adaptive global structure learning promises the selected features to preserve the global and sparse reconstruction structure. Manifold learning is responsible for exploring the local structure and label correlations. The combination of these two items can compensate for the negative influence brought by each other and help to select representative characteristics. The objective function of this method is not smooth and difficult to solve, so an efficient iterative algorithm is designed to make it suitable for practical applications. We evaluate the performance of SFAM on real-world data sets and compare the results with state-of-the-art supervised and semi-supervised feature selection algorithms as well as the baseline using all features. Experimental results show that SFAM has excellent performance.