Reviewerly/exHarmony huggingface dataset:
Developing a Vietnamese Tourism Question Answering System Using Knowledge Graph and Deep Learning

In recent years, Question Answering (QA) systems have increasingly become very popular in many sectors. This study aims to use a knowledge graph and deep learning to develop a QA system for tourism in Vietnam. First, the QA system replies to a user's question about a place in Vietnam. Then, the QA describes it in detail such as when the place was discovered, why the place's name was called like that, and so on. Finally, the system recommends some related tourist attractions to users. Meanwhile, deep learning is used to solve a simple natural language answer, and a knowledge graph is used to infer a natural language answering list related to entities in the question. The study experiments on a manual dataset collected from Vietnamese tourism websites. As a result, the QA system combining the two above approaches provides more information than other systems have done before. Besides that, the system gets 0.83 F1, 0.87 precision on the test set.
A Human-Robot Interaction System Calculating Visual Focus of Human’s Attention Level

Attention is the mental awareness of human on a particular object or a piece of information. The level of attention indicates how intense the focus is on an object or an instance. In this study, several types of human attention level have been observed. After introducing image segmentation and detection technique for facial features, eyeball movement and gaze estimation were measured. Eye movement were assessed using the video data, and a total of 10197 data instances were manually labelled for the attention level. Then Artificial Neural Network (ANN) and Recurrent Neural Network-Long Short Term Memory (LSTM) based Deep learning (DL) architectures have been proposed for analysing the data. Next, the trained DL model has been implanted into a robotic system that is capable of detecting various features; ultimately leading to the calculation of visual attention for reading, browsing, and writing purposes. This system is capable of checking the attention level of the participants and also can detect if participants are present or not. Based on a certain level of visual focus of attention (VFOA), this system interacts with the person, generates awareness and establishes verbal or visual communication with that person. The proposed ML techniques have achieved almost 99.24% validation accuracy and 99.43% test accuracy. It is also shown in the comparative study that, since the dataset volumes are limited, ANN is more suitable for attention level calculation than RNN-LSTM. We hope that the implemented robotic structure manifests the real-world implication of the proposed method.
Efficient-PrototypicalNet with self knowledge distillation for few-shot learning

The focus of recent few-shot learning research has been on the development of learning methods that can quickly adapt to unseen tasks with small amounts of data and low computational cost. In order to achieve higher performance in few-shot learning tasks, the generalizability of the method is essential to enable it generalize well from seen tasks to unseen tasks with limited number of samples. In this work, we investigate a new metric-based few-shot learning framework which transfers the knowledge from another effective classification model to produce well generalized embedding and improve the effectiveness in handling unseen tasks. The idea of our proposed Efficient-PrototypicalNet involves transfer learning, knowledge distillation, and few-shot learning. We employed a pre-trained model as a feature extractor to obtain useful features from tasks and decrease the task complexity. These features reduce the training difficulty in few-shot learning and increase the performance. Besides that, we further apply knowledge distillation to our framework and achieve extra performance improvement. The proposed Efficient-PrototypicalNet was evaluated on five benchmark datasets, i.e., Omniglot, miniImageNet, tieredImageNet, CIFAR-FS, and FC100. The proposed Efficient-PrototypicalNet achieved the state-of-the-art performance on most datasets in the 5-way K-shot image classification task, especially on the miniImageNet dataset.
Target-Aware Object Discovery and Association for Unsupervised Video Multi-Object Segmentation

This paper addresses the task of unsupervised video multi-object segmentation. Current approaches follow a two-stage paradigm: 1) detect object proposals using pre-trained Mask R-CNN, and 2) conduct generic feature matching for temporal association using re-identification techniques. However, the generic features, widely used in both stages, are not reliable for characterizing unseen objects, leading to poor generalization. To address this, we introduce a novel approach for more accurate and efficient spatio-temporal segmentation. In particular, to address instance discrimination, we propose to combine foreground region estimation and instance grouping together in one network, and additionally introduce temporal guidance for segmenting each frame, enabling more accurate object discovery. For temporal association, we complement current video object segmentation architectures with a discriminative appearance model, capable of capturing more fine-grained target-specific information. Given object proposals from the instance discrimination network, three essential strategies are adopted to achieve accurate segmentation: 1) target-specific tracking using a memory-augmented appearance model; 2) target-agnostic verification to trace possible tracklets for the proposal; 3) adaptive memory updating using the verified segments. We evaluate the proposed approach on DAVIS <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">17</inf> and YouTube-VIS, and the results demonstrate that it outperforms state-of-the-art methods both in segmentation accuracy and inference speed.
Optical image encryption and authentication using phase-only computer-generated hologram

Optical and photonics techniques such as computer-generated holograms (CGHs) are suggested to be used in security applications. In this paper, we present an approach to generate an authenticable phase-only hologram (APOH) for steganographic optical image encryption with phase optimization. The phase hologram can be calculated by the proposed integrated two-stage optimization method where Fresnel domain optimized random phase (OARP) method and modified two-step iterative Fourier transform algorithm under phase constraints are involved. The computer-generated APOH can be respectively verified and decrypted with an authentication key (A-key) and a decryption key(D-key). Numerical and experiment results prove the validity and security of the proposed approach, which offers a new method of holographic encryption for optical images.
DAEMON: Unsupervised Anomaly Detection and Interpretation for Multivariate Time Series

In many complex systems, devices are typically monitored and generating massive multivariate time series. However, due to the complex patterns and little useful labeled data, it is a great challenge to detect anomalies from these time series data. Existing methods either rely on less regularizations, or require a large number of labeled data, leading to poor accuracy in anomaly detection. To overcome those limitations, in this paper, we propose an unsupervised anomaly detection framework, called DAEMON (Adversarial Autoencoder Anomaly Detection Interpretation), which performs robustly for various datasets. The key idea is to use two discriminators to adversarially train an autoencoder to learn the normal pattern of multivariate time series, and thereafter use the reconstruction error to detect anomalies. The robustness of DAEMON is guaranteed by the regularization of hidden variables and reconstructed data using the adversarial generation method. Moreover, in order to help operators better diagnose anomalies, DAEMON provides anomaly interpretation based on the reconstruction error of the constituent univariate time series. Experiment results on four real datasets show that DAEMON can achieve an overall F1-score of 0.94, outperforming state-of-the-art methods. In addition, the anomaly interpretation accuracy of DAEMON can achieve 97%.
Blockchain applications for climate protection: A global empirical investigation

Our research consolidates the actual environment of blockchain applications that contribute in a certain way to climate protection. In view of the growing interest in climate change and the need to act on a global scale, knowledge about these applications enables investors, politicians, and citizens to drive this development forward through diverse support opportunities. This article provides an extensive overview of existing mitigation and adaptation measures based on blockchain technology. We collect data on 85 such applications and describe the empirical distributions of different attributes of these applications. In a logit regression, we analyze which application-specific and blockchain-specific characteristics determine the success of an application in the sense of an advanced operational status. We find evidence that applications of the type “energy trading” exhibit reduced chances of success, while green blockchain-based applications implementing a proof-of-stake consensus mechanism are more likely to become operational. Moreover, pursuing an initial coin offering has no significant effect on the success of an application. Our work provides the basis for a better understanding of the success factors of this new technology.
Are Cryptocurrencies a Backstop for the Stock Market in a COVID-19-Led Financial Crisis? Evidence from the NARDL Approach

The study investigates the safe haven properties and sustainability of the top five cryptocurrencies (Bitcoin, Ethereum, Dash, Monero, and Ripple) and gold for BRICS stock markets during the COVID-19 crisis period from 31 January 2020 to 17 September 2020 in comparison to the precrisis period from 1 January 2016 to 30 January 2020, in a nonlinear and asymmetric framework using Nonlinear Autoregressive Distributed Lag (NARDL) methodology. Our results show that the relationship dynamics of stock market and cryptocurrency returns both in the short and long run are changing during the COVID-19 crisis period, which justifies our study using the nonlinear and asymmetric model. As far as a sustainable safe haven is concerned, Dash and Ripple are found to be a safe haven for all the five markets before the pandemic. However, all five cryptocurrencies are found to be a safe haven for three emerging markets, such as Brazil, China, and Russia, during the financial crisis. In a comparative framework, gold is found to be a suitable safe haven only for Brazil and Russia. The results have implications for index fund managers of BRICS markets to include Dash and Ripple in their portfolio as safe haven assets to protect its value during a stock market crisis.
Permuted AdaIN: Reducing the Bias Towards Global Statistics in Image Classification

Recent work has shown that convolutional neural network classifiers overly rely on texture at the expense of shape cues. We make a similar but different distinction between shape and local image cues, on the one hand, and global image statistics, on the other. Our method, called Permuted Adaptive Instance Normalization (pAdaIN), reduces the representation of global statistics in the hidden layers of image classifiers. pAdaIN samples a random per-mutation π that rearranges the samples in a given batch. Adaptive Instance Normalization (AdaIN) is then applied between the activations of each (non-permuted) sample i and the corresponding activations of the sample π(i), thus swapping statistics between the samples of the batch. Since the global image statistics are distorted, this swapping procedure causes the network to rely on cues, such as shape or texture. By choosing the random permutation with probability p and the identity permutation otherwise, one can control the effect’s strength.With the correct choice of p, fixed apriori for all experiments and selected without considering test data, our method consistently outperforms baselines in multiple settings. In image classification, our method improves on both CIFAR100 and ImageNet using multiple architectures. In the setting of robustness, our method improves on both ImageNet-C and Cifar-100-C for multiple architectures. In the setting of domain adaptation and domain generalization, our method achieves state of the art results on the transfer learning task from GTAV to Cityscapes and on the PACS benchmark.
Accurate and Fast Image Denoising via Attention Guided Scaling

Image denoising is a classical topic yet still a challenging problem, especially for reducing noise from the texture information. Feature scaling (e.g., downscale and upscale) is a widely practice in image denoising to enlarge receptive field size and save resources. However, such a common operation would lose some visual informative details. To address those problems, we propose fast and accurate image denoising via attention guided scaling (AGS). We find that the main informative feature channel and visual primitives during the scaling should keep similar. We then propose to extract the global channel-wise attention to maintain main channel information. Moreover, we propose to collect global descriptors by considering the entire spatial feature. And we then distribute the global descriptors to local positions of the scaled feature, based on their specific needs. We further introduce AGS for adversarial training, resulting in a more powerful discriminator. Extensive experiments show the effectiveness of our proposed method, where we clearly surpass all the state-of-the-art methods on most popular synthetic and real-world denoising benchmarks quantitatively and visually. We further show that our network contributes to other high-level vision applications and improves their performances significantly.
On the Implementation of Behavior Trees in Robotics

There is a growing interest in Behavior Trees (BTs) as a tool to describe and implement robot behaviors. BTs were devised in the video game industry and their adoption in robotics resulted in the development of ad-hoc libraries to design and execute BTs that fit complex robotics software architectures. While there is broad consensus on how BTs work, some characteristics rely on the implementation choices done in the specific software library used. In this letter, we outline practical aspects in the adoption of BTs and the solutions devised by the robotics community to fully exploit the advantages of BTs in real robots. We also overview the solutions proposed in open-source libraries used in robotics, we show how BTs fit in robotic software architecture, and we present a use case example.
Teachers Do More Than Teach: Compressing Image-to-Image Models

Generative Adversarial Networks (GANs) have achieved huge success in generating high-fidelity images, however, they suffer from low efficiency due to tremendous computational cost and bulky memory usage. Recent efforts on compression GANs show noticeable progress in obtaining smaller generators by sacrificing image quality or involving a time-consuming searching process. In this work, we aim to address these issues by introducing a teacher network that provides a search space in which efficient network architectures can be found, in addition to performing knowledge distillation. First, we revisit the search space of generative models, introducing an inception-based residual block into generators. Second, to achieve target computation cost, we propose a one-step pruning algorithm that searches a student architecture from the teacher model and substantially reduces searching cost. It requires no ℓ <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> sparsity regularization and its associated hyper-parameters, simplifying the training procedure. Finally, we propose to distill knowledge through maximizing feature similarity between teacher and student via an index named Global Kernel Alignment (GKA). Our compressed networks achieve similar or even better image fidelity (FID, mIoU) than the original models with much-reduced computational cost, e.g., MACs. Code will be released at https://github.com/snap-research/CAT.
A Two-Stream Dynamic Pyramid Representation Model for Video-Based Person Re-Identification

Video-based person re-identification (Re-ID) leverages rich spatio-temporal information embedded in sequence data to further improve the retrieval accuracy comparing with single image Re-ID. However, it also brings new difficulties. <i>1)</i> Both spatial and temporal information should be considered simultaneously. <i>2)</i> Pedestrian video data often contains redundant information and <i>3)</i> suffers from data quality problems such as occlusion, background clutter. To solve the above problems, we propose a novel <b>two-stream</b> <b>Dynamic Pyramid Representation Model (DPRM)</b>. DPRM mainly consists of three sub-models, <i>i.e.</i>, Pyramidal Distribution Sampling Method (PDSM), Dynamic Pyramid Dilated Convolution (DPDC) and Pyramid Attention Pooling (PAP). PDSM is applied for more effective data pre-processing according to sequence semantic distribution. DPDC and PAP can be considered as two streams to describe the motion context and static appearance of a video sequence, respectively. By fusing the two-stream features together, we finally achieve comprehensive spatio-temporal representation. Notably, <b>dynamic pyramid</b> strategy is applied throughout the whole model. This strategy exploits multi-scale features under attention mechanism to maximally capture the most discriminative features and mitigate the impact of video data quality problems such as partial occlusion. Extensive experiments demonstrate the outperformance of DPRM. For instance, it achieves 83.0&#x0025; mAP and 89.0&#x0025; Rank-1 accuracy on MARS dataset and reaches state of the art.
Class imbalance learning using fuzzy ART and intuitionistic fuzzy twin support vector machines

The classification in imbalanced datasets is one of the main problems for machine learning techniques. Support vector machine (SVM) is biased to the majority class samples, and the minority class samples may incorrectly be considered as noise. Therefore, SVM has poor predictive accuracy for imbalanced datasets and generates inaccurate classification models. Existing class imbalance learning (CIL) techniques can make SVM less sensitive to class imbalance, but these methods suffer from issues related to noise and outliers. Moreover, despite the solid theoretical basis and good classification performance, SVM is not appropriate for the classification of large-scale datasets because the training complexity of SVM is closely related to the dataset size. Class imbalance learning (CIL) using Fuzzy adaptive resonance theory (ART) and intuitionistic fuzzy twin SVM (CIL-FART-IFTSVM), which can be applied to address the class imbalance issue in the presence of noise and outliers and large scale datasets, is proposed to overcome these substantial difficulties. In this method, we modify the distribution of the datasets using fuzzy adaptive resonance theory (Fuzzy ART) as a clustering method to overcome the imbalance problem. Then, after data reduction, IFTSVM is utilized to find excellent non-parallel hyperplanes in the generated data points. Finally, a coordinate descent system with shrinking by an active set is applied to reduce the computational complexity. Forty-five imbalanced datasets are considered to validate the performance of the proposed CIL-FART-IFTSVM method. The Friedman test and the bootstrap technique with 95% confidence intervals are applied to quantify the results statistically. The experimental results indicate that the method proposed in this paper has a better performance compared with other methods, and the training time is significantly better than that of other classifiers for large-scale datasets.
NeuralHumanFVV: Real-Time Neural Volumetric Human Performance Rendering using RGB Cameras

4D reconstruction and rendering of human activities is critical for immersive VR/AR experience. Recent advances still fail to recover fine geometry and texture results with the level of detail present in the input images from sparse multi-view RGB cameras. In this paper, we propose Neural-HumanFVV, a real-time neural human performance capture and rendering system to generate both high-quality geometry and photo-realistic texture of human activities in arbitrary novel views. We propose a neural geometry generation scheme with a hierarchical sampling strategy for real-time implicit geometry inference, as well as a novel neural blending scheme to generate high resolution (e.g., 1k) and photo-realistic texture results in the novel views. Furthermore, we adopt neural normal blending to enhance geometry details and formulate our neural geometry and texture rendering into a multi-task learning framework. Extensive experiments demonstrate the effectiveness of our approach to achieve high-quality geometry and photo-realistic free view-point reconstruction for challenging human performances.
Adaptive Image Transformer for One-Shot Object Detection

One-shot object detection tackles a challenging task that aims at identifying within a target image all object instances of the same class, implied by a query image patch. The main difficulty lies in the situation that the class label of the query patch and its respective examples are not available in the training data. Our main idea leverages the concept of language translation to boost metric-learning-based detection methods. Specifically, we emulate the language translation process to adaptively translate the feature of each object proposal to better correlate the given query feature for discriminating the class-similarity among the proposal-query pairs. To this end, we propose the Adaptive Image Transformer (AIT) module that deploys an attention-based encoder-decoder architecture to simultaneously explore intra-coder and inter-coder (i.e., each proposal-query pair) attention. The adaptive nature of our design turns out to be flexible and effective in addressing the one-shot learning scenario. With the informative attention cues, the proposed model excels in predicting the class-similarity between the target image proposals and the query image patch. Though conceptually simple, our model significantly outperforms a state-of-the-art technique, improving the unseen-class object classification from 63.8 mAP and 22.0 AP50 to 72.2 mAP and 24.3 AP50 on the PASCAL-VOC and MS-COCO benchmark datasets, respectively.