Reviewerly/exHarmony huggingface dataset:
Hand gestures for emergency situations: A video dataset based on words from Indian sign language

Automatic sign language recognition provides better services to the deaf as it avoids the existing communication gap between them and the rest of the society. Hand gestures, the primary mode of sign language communication, plays a key role in improving sign language recognition. This article presents a video dataset of the hand gestures of Indian sign language (ISL) words used in emergency situations. The videos of eight ISL words have been collected from 26 individuals (including 12 males and 14 females) in the age group of 22 to 26 years with two samples from each individual in an indoor environment with normal lighting conditions. Such a video dataset is highly needed for automatic recognition of emergency situations from the sign language for the benefit of the deaf. The dataset is useful for the researchers working on vision based sign language recognition (SLR) as well as hand gesture recognition (HGR). Moreover, support vector machine based classification and deep learning based classification of the emergency gestures has been carried out and the base classification performance shows that the database can be used as a benchmarking dataset for developing novel and improved techniques for recognizing the hand gestures of emergency words in Indian sign language.
Near Real-Time Global Solar Radiation Forecasting at Multiple Time-Step Horizons Using the Long Short-Term Memory Network

This paper aims to develop the long short-term memory (LSTM) network modelling strategy based on deep learning principles, tailored for the very short-term, near-real-time global solar radiation (GSR) forecasting. To build the prescribed LSTM model, the partial autocorrelation function is applied to the high resolution, 1 min scaled solar radiation dataset that generates statistically significant lagged predictor variables describing the antecedent behaviour of GSR. The LSTM algorithm is adopted to capture the short- and the long-term dependencies within the GSR data series patterns to accurately predict the future GSR at 1, 5, 10, 15, and 30 min forecasting horizons. This objective model is benchmarked at a solar energy resource rich study site (Bac-Ninh, Vietnam) against the competing counterpart methods employing other deep learning, a statistical model, a single hidden layer and a machine learning-based model. The LSTM model generates satisfactory predictions at multiple-time step horizons, achieving a correlation coefficient exceeding 0.90, outperforming all of the counterparts. In accordance with robust statistical metrics and visual analysis of all tested data, the study ascertains the practicality of the proposed LSTM approach to generate reliable GSR forecasts. The Diebold–Mariano statistic test also shows LSTM outperforms the counterparts in most cases. The study confirms the practical utility of LSTM in renewable energy studies, and broadly in energy-monitoring devices tailored for other energy variables (e.g., hydro and wind energy).
On the Storage Cost of Private Information Retrieval

We consider the fundamental tradeoff between the storage cost and the download cost in private information retrieval (PIR) systems, without any explicit structural restrictions on the storage codes, such as maximum distance separable codes or uncoded storage. Our focus in this work is on the two extreme points: the point when the storage cost is minimal, and the point when the download cost is minimal. Two novel outer bounds are provided, which have the following implications at these two extreme points. When the messages are stored without any redundancy across the databases, the optimal PIR strategy is to download all the messages; on the other hand, for PIR capacity-achieving codes, each database can reduce the storage cost, from storing all the messages, by no more than one message on average. To better understand the second extreme point, we then focus on the two-message two-database case, and show that a stronger outer bound can be derived through a novel pseudo-message technique. This stronger outer bound suggests that a precise characterization of the storage-download tradeoff may require more sophisticated bounding techniques.
Hierarchical and Interactive Refinement Network for Edge-Preserving Salient Object Detection

Salient object detection has undergone a very rapid development with the blooming of Deep Neural Network (DNN), which is usually taken as an important preprocessing procedure in various computer vision tasks. However, the down-sampling operations, such as pooling and striding, always make the final predictions blurred at edges, which has seriously degenerated the performance of salient object detection. In this paper, we propose a simple yet effective approach, i.e., Hierarchical and Interactive Refinement Network (HIRN), to preserve the edge structures in detecting salient objects. In particular, a novel multi-stage and dual-path network structure is designed to estimate the salient edges and regions from the low-level and high-level feature maps, respectively. As a result, the predicted regions will become more accurate by enhancing the weak responses at edges, while the predicted edges will become more semantic by suppressing the false positives in background. Once the salient maps of edges and regions are obtained at the output layers, a novel edge-guided inference algorithm is introduced to further filter the resulting regions along the predicted edges. Extensive experiments on several benchmark datasets have been conducted, in which the results show that our method significantly outperforms a variety of state-of-the-art approaches.
Training Keyword Spotting Models on Non-IID Data with Federated Learning

We demonstrate that a production-quality keyword-spotting model can be trained on-device using federated learning and achieve comparable false accept and false reject rates to a centrally-trained model. To overcome the algorithmic constraints associated with fitting on-device data (which are inherently non-independent and identically distributed), we conduct thorough empirical studies of optimization algorithms and hyperparameter configurations using large-scale federated simulations. To overcome resource constraints, we replace memory intensive MTR data augmentation with SpecAugment, which reduces the false reject rate by 56%. Finally, to label examples (given the zero visibility into on-device data), we explore teacher-student training.
Semantically-Aligned Universal Tree-Structured Solver for Math Word Problems

A practical automatic textual math word problems (MWPs) solver should be able to solve various textual MWPs while most existing works only focused on one-unknown linear MWPs.Herein, we propose a simple but efficient method called Universal Expression Tree (UET) to make the first attempt to represent the equations of various MWPs uniformly.Then a semantically-aligned universal tree-structured solver (SAU-Solver) based on an encoder-decoder framework is proposed to resolve multiple types of MWPs in a unified model, benefiting from our UET representation.Our SAU-Solver generates a universal expression tree explicitly by deciding which symbol to generate according to the generated symbols' semantic meanings like human solving MWPs.Besides, our SAU-Solver also includes a novel subtree-level semanticallyaligned regularization to further enforce the semantic constraints and rationality of the generated expression tree by aligning with the contextual information.Finally, to validate the universality of our solver and extend the research boundary of MWPs, we introduce a new challenging Hybrid Math Word Problems dataset (HMWP), consisting of three types of MWPs.Experimental results on several MWPs datasets show that our model can solve universal types of MWPs and outperforms several state-of-the-art models 1 .
PRover: Proof Generation for Interpretable Reasoning over Rules

Recent work by Clark et al. (2020) shows that transformers can act as "soft theorem provers" by answering questions over explicitly provided knowledge in natural language.In our work, we take a step closer to emulating formal theorem provers, by proposing PROVER, an interpretable transformer-based model that jointly answers binary questions over rule-bases and generates the corresponding proofs.Our model learns to predict nodes and edges corresponding to proof graphs in an efficient constrained training paradigm.During inference, a valid proof, satisfying a set of global constraints is generated.We conduct experiments on synthetic, hand-authored, and human-paraphrased rule-bases to show promising results for QA and proof generation, with strong generalization performance.First, PROVER generates proofs with an accuracy of 87%, while retaining or improving performance on the QA task, compared to RuleTakers (up to 6% improvement on zero-shot evaluation).Second, when trained on questions requiring lower depths of reasoning, it generalizes significantly better to higher depths (up to 15% improvement).Third, PROVER obtains near perfect QA accuracy of 98% using only 40% of the training data.However, generating proofs for questions requiring higher depths of reasoning becomes challenging, and the accuracy drops to 65% for "depth 5", indicating significant scope for future work.
A Unified Sequence Labeling Model for Emotion Cause Pair Extraction

Emotion-cause pair extraction (ECPE) aims at extracting emotions and causes as pairs from documents, where each pair contains an emotion clause and a set of cause clauses.Existing approaches address the task by first extracting emotion and cause clauses via two binary classifiers separately, and then training another binary classifier to pair them up.However, the extracted emotion-cause pairs of different emotion types cannot be distinguished from each other through simple binary classifiers, which limits the applicability of the existing approaches.Moreover, such two-step approaches may suffer from possible cascading errors.In this paper, to address the first problem, we assign emotion type labels to emotion and cause clauses so that emotioncause pairs of different emotion types can be easily distinguished.As for the second problem, we reformulate the ECPE task as a unified sequence labeling task, which can extract multiple emotion-cause pairs in an end-to-end fashion.We propose an approach composed of a convolution neural network for encoding neighboring information and two Bidirectional Long-Short Term Memory networks for two auxiliary tasks.Experiment results demonstrate the feasibility and effectiveness of our approaches.
DeepLight: Deep Lightweight Feature Interactions for Accelerating CTR Predictions in Ad Serving

Click-through rate (CTR) prediction is a crucial task in recommender systems and online advertising. The embedding-based neural networks have been proposed to learn both explicit feature interactions through a shallow component and deep feature interactions by a deep neural network (DNN) component. These sophisticated models, however, slow down the prediction inference by at least hundreds of times. To address the issue of significantly increased serving latency and high memory usage for real-time serving in production, this paper presents DeepLight: a framework to accelerate the CTR predictions in three aspects: 1) accelerate the model inference via explicitly searching informative feature interactions in the shallow component; 2) prune redundant parameters at the inter-layer level in the DNN component; 3) prune the dense embedding vectors to make them sparse in the embedding matrix. By combining the above efforts, the proposed approach accelerates the model inference by 46X on Criteo dataset and 27X on Avazu dataset without any loss on the prediction accuracy. This paves the way for successfully deploying complicated embedding-based neural networks in real-world serving systems.
Particle Swarm Optimization for Automatically Evolving Convolutional Neural Networks for Image Classification

Designing Convolutional Neural Networks from scratch is a time-consuming process that requires specialist expertise.While automated architecture generation algorithms have been proposed, the underlying search strategies generally are computationally expensive.The existing methods also do not explore the search space efficiently, and often lead to sub-optimal solutions.In this research, we propose a novel Particle Swarm Optimization (PSO)-based model for deep architecture generation to address the above challenges.Our proposed solution incorporates three new components.Firstly, a group-based encoding strategy is devised, which enforces the candidate networks to always follow the best practices.Specifically, it ensures that the number of groups can be adjusted in accordance with the input image size.By restricting the number of groups, we can adapt the frequency of the pooling operations toward the input image size.As such, it ascertains the position and maximum frequency of the pooling operations always result in a valid network architecture without the need for additional complex governing rules.Secondly, a new velocity updating mechanism is devised, which creates new network architectures by identifying the key network configuration differences.Thirdly, a new position updating mechanism using weighted velocity strengths is devised.Both the velocity and position updating mechanisms facilitate the proposed PSO-based model to search the intermediate positions of the particles' trajectories, allowing a better trade-off between diversification and intensification to be achieved.We employ eight well-known data sets, including Convex, Rectangles, MNIST and its variants, for model evaluation.The proposed PSO-based model achieves up to 7.58% improvement in accuracy and up to 63% reduction in computational cost, in comparison with those from the current state-of-the-art methods.
MIGO-NAS: Towards Fast and Generalizable Neural Architecture Search

Neural architecture search (NAS) has achieved unprecedented performance in various computer vision tasks. However, most existing NAS methods are defected in search efficiency and model generalizability. In this paper, we propose a novel NAS framework, termed MIGO-NAS, with the aim to guarantee the efficiency and generalizability in arbitrary search spaces. On the one hand, we formulate the search space as a multivariate probabilistic distribution, which is then optimized by a novel multivariate information-geometric optimization (MIGO). By approximating the distribution with a sampling, training, and testing pipeline, MIGO guarantees the memory efficiency, training efficiency, and search flexibility. Besides, MIGO is the first time to decrease the estimation error of natural gradient in multivariate distribution. On the other hand, for a set of specific constraints, the neural architectures are generated by a novel dynamic programming network generation (DPNG), which significantly reduces the training cost under various hardware environments. Experiments validate the advantages of our approach over existing methods by establishing a superior accuracy and efficiency i.e., 2.39 test error on CIFAR-10 benchmark and 21.7 on ImageNet benchmark, with only 1.5 GPU hours and 96 GPU hours for searching, respectively. Besides, the searched architectures can be well generalize to computer vision tasks including object detection and semantic segmentation, i.e., 25×25× FLOPs compression, with 6.4 mAP gain over Pascal VOC dataset, and 29.9×29.9× FLOPs compression, with only 1.41 percent performance drop over Cityscapes dataset. The code is publicly available.
Hierarchical Reinforcement Learning With Universal Policies for Multistep Robotic Manipulation

Multistep tasks, such as block stacking or parts (dis)assembly, are complex for autonomous robotic manipulation. A robotic system for such tasks would need to hierarchically combine motion control at a lower level and symbolic planning at a higher level. Recently, reinforcement learning (RL)-based methods have been shown to handle robotic motion control with better flexibility and generalizability. However, these methods have limited capability to handle such complex tasks involving planning and control with many intermediate steps over a long time horizon. First, current RL systems cannot achieve varied outcomes by planning over intermediate steps (e.g., stacking blocks in different orders). Second, the exploration efficiency of learning multistep tasks is low, especially when rewards are sparse. To address these limitations, we develop a unified hierarchical reinforcement learning framework, named Universal Option Framework (UOF), to enable the agent to learn varied outcomes in multistep tasks. To improve learning efficiency, we train both symbolic planning and kinematic control policies in parallel, aided by two proposed techniques: 1) an auto-adjusting exploration strategy (AAES) at the low level to stabilize the parallel training, and 2) abstract demonstrations at the high level to accelerate convergence. To evaluate its performance, we performed experiments on various multistep block-stacking tasks with blocks of different shapes and combinations and with different degrees of freedom for robot control. The results demonstrate that our method can accomplish multistep manipulation tasks more efficiently and stably, and with significantly less memory consumption.
A New Fractional Chaotic System and Its Application in Image Encryption With DNA Mutation

In this paper, a color image encryption algorithm based on the fractional-order multi-scroll Chen chaotic system and DNA mutation principle is proposed. The dynamical performances of the fractional-order multi-scroll Chen chaotic system are analyzed by phase diagram, Lyapunov exponential spectrum and bifurcation diagram. Meanwhile, we realized the hardware circuit simulation of the chaotic system on the DSP platform. Based on comprehensive analysis, an image encryption algorithm is designed. In particular, a DNA algorithm with mutation effect has been introduced into the encryption algorithm. Finally, the performances of the designed encryption scheme are analyzed by key space, correlation coefficients, information entropy, histogram, differential attacks and robustness analysis. The experimental results prove that the algorithm has strong encryption capabilities and can withstand multiple decryption methods. The new color image encryption scheme proposed in this paper can realize the secure communication of digital images.
Token Labeling: Training a 85.4% Top-1 Accuracy Vision Transformer with 56M Parameters on ImageNet

This paper provides a strong baseline for vision transformers on the ImageNetclassification task. While recent vision transformers have demonstratedpromising results in ImageNet classification, their performance still lagsbehind powerful convolutional neural networks (CNNs) with approximately thesame model size. In this work, instead of describing a novel transformerarchitecture, we explore the potential of vision transformers in ImageNetclassification by developing a bag of training techniques. We show that byslightly tuning the structure of vision transformers and introducing tokenlabeling -- a new training objective, our models are able to achieve betterresults than the CNN counterparts and other transformer-based classificationmodels with similar amount of training parameters and computations. Taking avision transformer with 26M learnable parameters as an example, we can achievean 84.4% Top-1 accuracy on ImageNet. When the model size is scaled up to56M/150M, the result can be further increased to 85.4%/86.2% without extradata. We hope this study could provide researchers with useful techniques totrain powerful vision transformers. Our code and all the training details willbe made publicly available at https://github.com/zihangJiang/TokenLabeling.
RAGA: Relation-Aware Graph Attention Networks for Global Entity Alignment

Entity alignment (EA) is the task to discover entities referring to the same real-world object from different knowledge graphs (KGs), which is the most crucial step in integrating multi-source KGs. The majority of the existing embedding-based entity alignment methods embed entities and relations into a vector space based on relation triples of KGs for local alignment. As these methods insufficiently consider the multiple relations between entities, the structure information of KGs has not been fully leveraged. In this paper, we propose a novel framework based on Relation-aware Graph Attention Networks to capture the interactions between entities and relations. Our framework adopts the self-attention mechanism to spread entity information to the relations and then aggregate relation information back to entities. Furthermore, we propose a global alignment algorithm to make one-to-one entity alignments with a fine-grained similarity matrix. Experiments on three real-world cross-lingual datasets show that our framework outperforms the state-of-the-art methods.
Diverse Semantic Image Synthesis via Probability Distribution Modeling

Semantic image synthesis, translating semantic layouts to photo-realistic images, is a one-to-many mapping problem. Though impressive progress has been recently made, diverse semantic synthesis that can efficiently produce semantic-level multimodal results, still remains a challenge. In this paper, we propose a novel diverse semantic image synthesis framework from the perspective of semantic class distributions, which naturally supports diverse generation at semantic or even instance level. We achieve this by modeling class-level conditional modulation parameters as continuous probability distributions instead of discrete values, and sampling per-instance modulation parameters through instance-adaptive stochastic sampling that is consistent across the network. Moreover, we propose prior noise remapping, through linear perturbation parameters encoded from paired references, to facilitate supervised training and exemplar-based instance style control at test time. Extensive experiments on multiple datasets show that our method can achieve superior diversity and comparable quality compared to state-of-the-art methods. Code will be available at https://github.com/tzt101/INADE.git
Towards Lightweight URL-Based Phishing Detection

Nowadays, the majority of everyday computing devices, irrespective of their size and operating system, allow access to information and online services through web browsers. However, the pervasiveness of web browsing in our daily life does not come without security risks. This widespread practice of web browsing in combination with web users’ low situational awareness against cyber attacks, exposes them to a variety of threats, such as phishing, malware and profiling. Phishing attacks can compromise a target, individual or enterprise, through social interaction alone. Moreover, in the current threat landscape phishing attacks typically serve as an attack vector or initial step in a more complex campaign. To make matters worse, past work has demonstrated the inability of denylists, which are the default phishing countermeasure, to protect users from the dynamic nature of phishing URLs. In this context, our work uses supervised machine learning to block phishing attacks, based on a novel combination of features that are extracted solely from the URL. We evaluate our performance over time with a dataset which consists of active phishing attacks and compare it with Google Safe Browsing (GSB), i.e., the default security control in most popular web browsers. We find that our work outperforms GSB in all of our experiments, as well as performs well even against phishing URLs which are active one year after our model’s training.