Reviewerly/exHarmony huggingface dataset:
Color pathological image encryption algorithm using arithmetic over Galois field and coupled hyper chaotic system

Based on arithmetic operation over a Galois field and coupled hyper chaotic system, we propose a novel pathological image encryption scheme to ensure the transmission of a pathological image securely in a telemedicine system. First, the external one-time keys are sampled from colored noise prior to encryption, to serve as initial values. Second, the pixels are confused and diffused by polynomial multiplication and addition over a Galois field. Finally, the look-up tables, avalanche effect, and an appropriate number of encryption rounds are used to ensure a balance of speed, complexity, and intensity. Through simulation and security analysis we verified that the proposed scheme is faster and more resistant to common attacks.
Focal Visual-Text Attention for Memex Question Answering

Recent insights on language and vision with neural networks have been successfully applied to simple single-image visual question answering. However, to tackle real-life question answering problems on multimedia collections such as personal photos, we have to look at whole collections with sequences of photos or videos. When answering questions from a large collection, a natural problem is to identify snippets to support the answer. In this paper, we describe a novel neural network called Focal Visual-Text Attention network (FVTA) for collective reasoning in visual question answering, where both visual and text sequence information such as images and text metadata are presented. FVTA introduces an end-to-end approach that makes use of a hierarchical process to dynamically determine what media and what time to focus on in the sequential data to answer the question. FVTA can not only answer the questions well but also provides the justifications which the system results are based upon to get the answers. FVTA achieves state-of-the-art performance on the MemexQA dataset and competitive results on the MovieQA dataset.
All-in-One: Emotion, Sentiment and Intensity Prediction Using a Multi-Task Ensemble Framework

We propose a multi-task ensemble framework that jointly learns multiple related problems. The ensemble model aims to leverage the learned representations of three deep learning models (i.e., CNN, LSTM and GRU) and a hand-crafted feature representation for the predictions. Through multi-task framework, we address four problems of emotion and sentiment analysis, i.e., “emotion <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">classification</i> & <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">intensity</i> ”, “ <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">valence</i> , <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">arousal</i> & <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">dominance</i> for emotion”, “ <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">valence</i> & <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">arousal</i> for sentiment”, and “ <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3-class categorical</i> & <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">5-class ordinal classification</i> for sentiment”. The underlying problems cover two granularity (i.e., <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">coarse-grained</i> and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">fine-grained</i> ) and a diverse range of domains (i.e., <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">tweets</i> , <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Facebook posts</i> , <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">news headlines</i> , <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">blogs</i> , <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">letters</i> etc.). Experimental results suggest that the proposed multi-task framework outperforms the single-task frameworks in all experiments.
Towards Social Artificial Intelligence: Nonverbal Social Signal Prediction in a Triadic Interaction

We present a new research task and a dataset to understand human social interactions via computational methods, to ultimately endow machines with the ability to encode and decode a broad channel of social signals humans use. This research direction is essential to make a machine that genuinely communicates with humans, which we call Social Artificial Intelligence. We first formulate the "social signal prediction'' problem as a way to model the dynamics of social signals exchanged among interacting individuals in a data-driven way. We then present a new 3D motion capture dataset to explore this problem, where the broad spectrum of social signals (3D body, face, and hand motions) are captured in a triadic social interaction scenario. Baseline approaches to predict speaking status, social formation, and body gestures of interacting individuals are presented in the defined social prediction framework.
Multi-view common component discriminant analysis for cross-view classification

Cross-view classification that means to classify samples from heterogeneous views is a significant yet challenging problem in computer vision. An effective solution to this problem is the multi-view subspace learning (MvSL), which intends to find a common subspace for multi-view data. Although great progress has been made, existing methods usually fail to find a suitable subspace when multi-view data lies on nonlinear manifolds, thus leading to performance deterioration. To circumvent this drawback, we propose Multi-view Common Component Discriminant Analysis (MvCCDA) to handle view discrepancy, discriminability and nonlinearity in a joint manner. Specifically, our MvCCDA incorporates supervised information and local geometric information into the common component extraction process to learn a discriminant common subspace and to discover the nonlinear structure embedded in multi-view data. Optimization and complexity analysis of MvCCDA are also presented for completeness. Our MvCCDA is competitive with the state-of-the-art MvSL based methods on four benchmark datasets, demonstrating its superiority.
Feature Affinity-Based Pseudo Labeling for Semi-Supervised Person Re-Identification

Vision-based person re-identification aims to match a person's identity across multiple images, which is a fundamental task in multimedia content analysis and retrieval. Deep neural networks have recently manifested great potential in this task. However, a major bottleneck of existing supervised deep networks is their reliance on a large amount of annotated training data. Manual labeling for person identities in large-scale surveillance camera systems is quite challenging and incurs significant costs. Some recent studies adopt generative model outputs as training data augmentation. To more effectively use these synthetic data for an improved feature learning and re-identification performance, this paper proposes a novel feature affinity-based pseudo labeling method with two possible label encodings. To the best of our knowledge, this is the first study that employs pseudo-labeling by measuring the affinity of unlabeled samples with the underlying clusters of labeled data samples using the intermediate feature representations from deep networks. We propose training the network with the joint supervision of cross-entropy loss together with a center regularization term, which not only ensures discriminative feature representation learning but also simultaneously predicts pseudo-labels for unlabeled data. We show that both label encodings can be learned in a unified manner and help improve the overall performance. Our extensive experiments on three person re-identification datasets: Market-1501, DukeMTMC-reID, and CUHK03, demonstrate significant performance boost over the state-of-the-art person re-identification approaches.
Multi-View Support Vector Machines with the Consensus and Complementarity Information

Multi-view learning (MVL) is an active direction in machine learning that aims at exploiting the consensus and complementarity information among multiple distinct feature sets to boost the generalization performance of the counterpart algorithm. So far, two classical SVM-based MVL methods are SVM-2K and multi-view twin support vector machine (MvTSVM). They are designed only for two-view classification and cannot tackle the general multi-view classification problem. They also cannot effectively leverage the complementarity information among different feature views. In this paper, we propose two novel multi-view support vector machines with the consensus and complementarity information for MVL that not only can deal with the two-view classification problem but also the general multi-view classification problem by jointly learning multiple different views in a non-pairwise way. The disagreement among different views is regarded as a constraint or a regularization term in the objective function which plays an important role in exploring the consensus information. Combination weights for the reconstruction of each view in regularization terms are learned to explore complementarity information among different views. Finally, an efficient iteration algorithm with the classical convex quadratic programming is developed for optimization. Experimental results validate the effectiveness of our proposed methods.
G-GANISR: Gradual generative adversarial network for image super resolution

Adversarial methods have demonsterated to be signifiant at generating realistic images. However, these approaches have a challenging training process which partially attributed to the performance of discriminator. In this paper, we proposed an efficient super-resolution model based on generative adversarial network (GAN), to effectively generate reprehensive information and improve the test quality of the real-world images. To overcome the current issues, we designed the discriminator of our model based on the Least Square Loss function. The proposed network is organized by a gradual learning process from simple to advanced, which means from the small upsampling factors to the large upsampling factor that helps to improve the overall stability of the training. In particular, to control the model parameters and mitigate the training difficulties, dense residual learning strategy is adopted. Indeed, the key idea of proposed methodology is (i) fully exploit all the image details without losing information by gradually increases the task of discriminator, where the output of each layer is gradually improved in the next layer. In this way the model efficiently generates a super-resolution image even up to high scaling factors (e.g. × 8). (ii) The model is stable during the learning process, as we use least square loss instead of cross-entropy. In addition, the effects of different objective function on training stability are compared. To evaluate the model we conducted two sets of experiments, by using the proposed gradual GAN and the regular GAN to demonstrate the efficiency and stability of the proposed model for both quantitative and qualitative benchmarks.
Evaluating the impact of the augmented reality learning environment on electronics laboratory skills of engineering students

Abstract In engineering education, laboratory experience is powerful in building up the specialized aptitudes and skills of the engineering graduates by structuring and conducting practical experiments. In engineering laboratories, the student often deals with complex laboratory equipment to perform the experiment. Oscilloscope and function generator are frequently used electronic equipment in electronics engineering laboratories. Augmented reality (AR) technology has the capacity to provide a unique learning experience to the students which can ease down the difficulties faced by students while operating laboratory equipment. In this paper, an augmented reality learning environment (ARLE) is developed which could help the student in operating the laboratory equipment. The study aims to determine the impact of AR intervention on student laboratory skills, cognitive load, and learning motivation. The experiment was conducted amongst 60 first‐year undergraduate engineering students. The participants were randomly divided into two groups: Experimental group and Control group. The participants of the experimental group were given ARLE treatment while the participants from the control group were given traditional teaching treatment. The experimental results suggest that AR intervention has a significant positive impact on student laboratory skills. Also, ARLE is an effective tool in reducing the cognitive load of students while operating laboratory equipment. The student feedback indicates that the students of the experimental group operated the equipment without any fear of damaging the equipment as they were already familiarized with the equipment in ARLE. The results of this study recommend AR as an adequate technology for developing unique AR experiences for engineering education.
Benchmarking Natural Language Understanding Services for Building Conversational Agents

We have recently seen the emergence of several publicly available Natural Language Understanding (NLU) toolkits, which map user utterances to structured, but more abstract, Dialogue Act (DA) or Intent specifications, while making this process accessible to the lay developer. In this paper, we present the first wide coverage evaluation and comparison of some of the most popular NLU services, on a large, multi-domain (21 domains) dataset of 25 K user utterances that we have collected and annotated with Intent and Entity Type specifications and which will be released as part of this submission ( https://github.com/xliuhw/NLU-Evaluation-Data ). The results show that on Intent classification Watson significantly outperforms the other platforms, namely, Dialogflow, LUIS and Rasa; though these also perform well. Interestingly, on Entity Type recognition, Watson performs significantly worse due to its low Precision (At the time of producing the camera-ready version of this paper, we noticed the seemingly recent addition of a 'Contextual Entity' annotation tool to Watson, much like e.g. in Rasa. We'd threfore like to stress that this paper does not include an evaluation of this feature in Watson NLU.). Again, Dialogflow, LUIS and Rasa perform well on this task.
TF-YOLO: An Improved Incremental Network for Real-Time Object Detection

In recent years, significant advances have been gained in visual detection, and an abundance of outstanding models have been proposed. However, state-of-the-art object detection networks have some inefficiencies in detecting small targets. They commonly fail to run on portable devices or embedded systems due to their high complexity. In this workpaper, a real-time object detection model, termed as Tiny Fast You Only Look Once (TF-YOLO), is developed to implement in an embedded system. Firstly, the k-means++ algorithm is applied to cluster the dataset, which contributes to more excellent priori boxes of the targets. Secondly, inspired by the multi-scale prediction idea in the Feature Pyramid Networks (FPN) algorithm, the framework in YOLOv3 is effectively improved and optimized, by three scales to detect the earlier extracted features. In this way, the modified network is sensitive for small targets. Experimental results demonstrate that the proposed TF-YOLO method is a smaller, faster and more efficient network model increasing the performance of end-to-end training and real-time object detection for a variety of devices.
Bonnet: An Open-Source Training and Deployment Framework for Semantic Segmentation in Robotics using CNNs

The ability to interpret a scene is an important capability for a robot that is supposed to interact with its environment. The knowledge of what is in front of the robot is, for example, relevant for navigation, manipulation, or planning. Semantic segmentation labels each pixel of an image with a class label and thus provides a detailed semantic annotation of the surroundings to the robot. Convolutional neural networks (CNNs) are popular methods for addressing this type of problem. The available software for training and the integration of CNNs for real robots, however, is quite fragmented and often difficult to use for non-experts, despite the availability of several high-quality open-source frameworks for neural network implementation and training. In this paper, we propose a tool called Bonnet, which addresses this fragmentation problem by building a higher abstraction that is specific for the semantic segmentation task. It provides a modular approach to simplify the training of a semantic segmentation CNN independently of the used dataset and the intended task. Furthermore, we also address the deployment on a real robotic platform. Thus, we do not propose a new CNN approach in this paper. Instead, we provide a stable and easy-to-use tool to make this technology more approachable in the context of autonomous systems. In this sense, we aim at closing a gap between computer vision research and its use in robotics research. We provide an open-source codebase for training and deployment. The training interface is implemented in Python using TensorFlow and the deployment interface provides C++ library that can be easily integrated in an existing robotics codebase, a ROS node, and two standalone applications for label prediction in images and videos.
An Improved Butterfly Optimization Algorithm for Engineering Design Problems Using the Cross-Entropy Method

Engineering design optimization in real life is a challenging global optimization problem, and many meta-heuristic algorithms have been proposed to obtain the global best solutions. An excellent meta-heuristic algorithm has two symmetric search capabilities: local search and global search. In this paper, an improved Butterfly Optimization Algorithm (BOA) is developed by embedding the cross-entropy (CE) method into the original BOA. Based on a co-evolution technique, this new method achieves a proper balance between exploration and exploitation to enhance its global search capability, and effectively avoid it falling into a local optimum. The performance of the proposed approach was evaluated on 19 well-known benchmark test functions and three classical engineering design problems. The results of the test functions show that the proposed algorithm can provide very competitive results in terms of improved exploration, local optima avoidance, exploitation, and convergence rate. The results of the engineering problems prove that the new approach is applicable to challenging problems with constrained and unknown search spaces.
Effects of postural stability, active control, exposure duration and repeated exposures on HMD induced cybersickness

Cybersickness is common during head-mounted display (HMD) based virtual reality. This study examined whether it is possible to: (1) identify people who are more susceptible to this cybersickness; and (2) find general ways to reduce its occurrence and severity. Our participants were exposed to HMD-based virtual reality four times over two different days (using “Freedom Locomotion VR”). During these 10-min trials, participants were either free-standing or posturally restrained as they actively controlled or passively viewed their locomotion through the virtual environment. Cybersickness was found to increase steadily over time during each exposure. While this cybersickness was markedly reduced on day 2 (compared to day 1), it was not significantly altered by either the use of postural restraints or active locomotion control. However, the sick and well participants in our study were found to differ in terms of their spontaneous postural activity (before they entered virtual reality). We found that the participants who experienced stronger vection also tended to report more severe cybersickness in this study. These findings suggest that we should be able to identify people who are more susceptible to cybersickness and help them become more resistant to it (via repeated exposures to HMD-based virtual reality).
Transferable Normalization: Towards Improving Transferability of Deep Neural Networks

Deep neural networks (DNNs) excel at learning representations when trained on large-scale datasets. Pre-trained DNNs also show strong transferability when fine-tuned to other labeled datasets. However, such transferability becomes weak when the target dataset is fully unlabeled as in Unsupervised Domain Adaptation (UDA). We envision that the loss of transferability may stem from the intrinsic limitation of the architecture design of DNNs. In this paper, we delve into the components of DNN architectures and propose Transferable Normalization (TransNorm) in place of existing normalization techniques. TransNorm is an end-to-end trainable layer to make DNNs more transferable across domains. As a general method, TransNorm can be easily applied to various deep neural networks and domain adaption methods, without introducing any extra hyper-parameters or learnable parameters. Empirical results justify that TransNorm not only improves classification accuracies but also accelerates convergence for mainstream DNN-based domain adaptation methods.
A POMDP Maneuver Planner For Occlusions in Urban Scenarios

Behavior planning in urban environments must consider the various existing uncertainties in an explicit way. This work proposes a behavior planner, based on a POMDP formulation, that explicitly considers possibly occluded vehicles. The future field of view of the autonomous car is predicted over the whole planning horizon. Both, occlusions which are generated by static as well as generated by dynamic objects are hereby considered. We use Monte Carlo sampling to generate possible future episodes that are used to derive an optimized policy. The sampled episodes consider the uncertain behavior of the known traffic participants as well as the existence probability of so-called phantom vehicles in occluded areas. By representing all possible, occluded vehicle configurations by its reachable set instead of single particles, a very efficient representation is found. Therefore, we ensure to consider all possible configurations which may drive out of the occluded area in our optimized policy. We propose a generic formulation of the POMDP problem that can be applied to various scenarios for urban driving. Its performance is demonstrated by using simulation scenarios at intersections including multiple vehicles and occlusions caused by static and dynamic objects. It is shown, that the autonomous vehicle approaches occluded areas by far less conservative than a baseline strategy which considers only the current field of view (fov). This is because various, future scenarios are already considered in the policy. In fact, we show that our planner is able to drive nearly the same trajectories as an omniscient planner would.