Reviewerly/exHarmony huggingface dataset:
Webly Supervised Knowledge Embedding Model for Visual Reasoning

Visual reasoning between visual image and natural language description is a long-standing challenge in computer vision. While recent approaches offer a great promise by compositionality or relational computing, most of them are oppressed by the challenge of training with datasets containing only a limited number of images with ground-truth texts. Besides, it is extremely time-consuming and difficult to build a larger dataset by annotating millions of images with text descriptions that may very likely lead to a biased model. Inspired by the majority success of webly supervised learning, we utilize readily-available web images with its noisy annotations for learning a robust representation. Our key idea is to presume on web images and corresponding tags along with fully annotated datasets in learning with knowledge embedding. We present a two-stage approach for the task that can augment knowledge through an effective embedding model with weakly supervised web data. This approach learns not only knowledge-based embeddings derived from key-value memory networks to make joint and full use of textual and visual information but also exploits the knowledge to improve the performance with knowledge-based representation learning for applying other general reasoning tasks. Experimental results on two benchmarks show that the proposed approach significantly improves performance compared with the state-of-the-art methods and guarantees the robustness of our model against visual reasoning tasks and other reasoning tasks.
Evaluation of the moral permissibility of action plans

Research in classical planning so far has been mainly concerned with generating a satisficing or an optimal plan. However, if such systems are used to make decisions that are relevant to humans, one should also consider the ethical consequences generated plans can have. Traditionally, ethical principles are formulated in an action-based manner, allowing to judge the execution of one action. We show how such a judgment can be generalized to plans. Further, we study the computational complexity of making ethical judgment about plans.
Enhancing Facial Data Diversity with Style-based Face Aging

A significant limiting factor in training fair classifiers relates to the presence of dataset bias. In particular, face datasets are typically biased in terms of attributes such as gender, age, and race. If not mitigated, bias leads to algorithms that exhibit unfair behaviour towards such groups. In this work, we address the problem of increasing the diversity of face datasets with respect to age. Concretely, we propose a novel, generative style-based architecture for data augmentation that captures fine-grained aging patterns by conditioning on multi-resolution age-discriminative representations. By evaluating on several age-annotated datasets in both single- and cross-database experiments, we show that the proposed method outperforms state-of-the-art algorithms for age transfer, especially in the case of age groups that lie in the tails of the label distribution. We further show significantly increased diversity in the augmented datasets, outperforming all compared methods according to established metrics.
Ternary MobileNets via Per-Layer Hybrid Filter Banks

MobileNets family of computer vision neural networks have fueled tremendous progress in the design and organization of resource-efficient architectures in recent years. New applications with stringent real-time requirements on highly constrained devices require further compression of MobileNets-like compute-efficient networks. Model quantization is a widely used technique to compress and accelerate neural network inference and prior works have quantized MobileNets to 4 - 6 bits, albeit with a modest to significant drop in accuracy. While quantization to sub-byte values (i.e. precision ≤ 8 bits) has been valuable, even further quantization of MobileNets to binary or ternary values is necessary to realize significant energy savings and possibly runtime speedups on specialized hardware, such as ASICs and FPGAs. Under the key observation that convolutional filters at each layer of a deep neural network may respond differently to ternary quantization, we propose a novel quantization method that generates per-layer hybrid filter banks consisting of full-precision and ternary weight filters for MobileNets. Using this proposed quantization method, we quantize a substantial portion of weight filters of MobileNets to ternary values resulting in a 27.98% savings in energy, and a 51.07% reduction in the model size, while achieving comparable accuracy and no degradation in throughput on specialized hardware in comparison to the baseline full-precision MobileNets. Finally, we demonstrate the generalizability and effectiveness of hybrid filter banks to other neural network architectures.
Omnidirectional Vision System With Laser Illumination in a Flexible Configuration and Its Calibration by One Single Snapshot

An omnidirectional vision system consisting of a fisheye camera and structured light projector has its own distinctive advantages toward other vision systems. A vision system provides a wide range of information by one single capture. Moreover, the laser feature is easy to detect and extract from the given image for further analysis. In order to obtain the reliable results, the vision system must be calibrated. There have been improvements toward calibration techniques for vision systems with laser illumination. However, these techniques are not suitable for an omnidirectional vision system in a particular configuration. Motivated by the abovementioned problem, this article presents a novel omnidirectional vision system with laser illumination in a flexible configuration and a method for its extrinsic calibration. Flexible configuration means that both camera and laser plane are no longer parallel to the floor. The calibrated vision system is evaluated experimentally by using simulated and real data to demonstrate that the proposed calibration method is more robust and provides a higher measurement accuracy.
Quantum Leap and Crash

Random numbers are essential for cryptography and scientific simulation. Generating truly random numbers for cryptography can be a slow and expensive process. Quantum physics offers a variety of promising solutions to this challenge, proposing sources of entropy that may be genuinely unpredictable, based on the inherent randomness of certain physical phenomena. These properties have been employed to design Quantum Random Number Generators (QRNGs), some of which are commercially available. In this work, we present the first published analysis of the Quantis family of QRNGs (excluding AIS-31 models), designed and manufactured by ID Quantique (IDQ). Our study also includes Comscire’s PQ32MU QRNG, and two online services: the Australian National University’s (ANU) QRNG, and the Humboldt Physik generator. Each QRNG is analysed using five batteries of statistical tests: Dieharder, National Institute of Standards and Technology (NIST) SP800-22, Ent, Tuftests and TestU01, as part of our thorough examination of their output. Our analysis highlights issues with current certification schemes, which largely rely on NIST SP800-22 and Diehard tests of randomness. We find that more recent tests of randomness identify issues in the output of QRNG, highlighting the need for mandatory post-processing even for low-security usage of random numbers sourced from QRNGs.
Object-Based Image Coding: A Learning-Driven Revisit

The Object-Based Image Coding (OBIC) that was extensively studied about two decades ago, promised a vast application perspective for both ultra-low bitrate communication and high-level semantical content understanding, but it had rarely been used due to the inefficient compact representation of object with arbitrary shape. A fundamental issue behind is how to efficiently process the arbitrary-shaped objects at a fine granularity (e.g., feature element or pixel wise). To attack this, we have proposed to apply the element-wise masking and compression by devising an object segmentation network for image layer decomposition, and parallel convolution-based neural image compression networks to process masked foreground objects and background scene separately. All components are optimized in an end-to-end learning framework to intelligently weigh their (e.g., object and background) contributions for visually pleasant reconstruction. We have conducted comprehensive experiments to evaluate the performance on PASCAL VOC dataset at a very low bitrate scenario (e.g., ≲ 0.1 bits per pixel - bpp) which have demonstrated noticeable subjective quality improvement compared with JPEG2K, HEVC-based BPG and another learned image compression method. All relevant materials are made publicly accessible at https://njuvision.github.io/ Neural-Object-Coding/.
Hindi EmotionNet

In this study, we create an emotion lexicon for the Hindi language called Hindi EmotionNet. It can assign emotional affinity to words in IndoWordNet. This lexicon contains 3,839 emotion words, with 1,246 positive and 2,399 negative words. We also introduce ambiguous (217 words) and neutral (95 words) emotions to Hindi. Positive emotion words covered nine types of positive emotions, negative emotion words covered eleven types of negative emotions, ambiguous emotion words covered seven types of ambiguous emotions, and neutral emotion words covered two neutral emotions. The proposed Hindi EmotionNet was then applied to opinion classification and emotion classification. We introduce a centrality-based approach for emotion classification that uses degree, closeness, betweenness, and page rank as centrality measures. We also created a dataset of Hindi based on screenplays, stories, and blogs in the language. We translated emotion data from SemEval 2017 into Hindi for further comparison. The proposed approach delivered promising results on opinion and emotion classification, with an accuracy of 85.78% for the former and 75.91% for the latter.
Breaking the Limits of Transportation Mode Detection: Applying Deep Learning Approach With Knowledge-Based Features

Activity recognition and transportation mode detection are the key research areas for context-aware systems. In smart environments such as cities, buildings, transportation systems etc., ambient intelligence applications watch on human activities in order to increase the quality of transportation, health, traffic and human-oriented services. Literature review shows that existing studies typically deal with recognition of motorized and non-motorized daily activities under the name of human activity recognition and transport mode detection. Only few studies worked on these problems with rich datasets in terms of the number of classes. To the best of our knowledge, for the first time, a study examines nearly the whole motorized/non-motorized transportation modes of people including still, walk, run, climbing upstairs, climbing downstairs, bicycle, motorbike, car, metro, train, high speed rail (HSR), tram and metrobus, excluding ferry and airplane. To outperform existing solutions on such a large dataset, an extended version of the well-known HTC dataset, especially consisting of similar classes, such as train, metro, tram and high speed rail, we propose a combined solution of an Long Short-Term Memory network and Healing algorithm. In our experiments, we first reveal the limits of traditional machine learning algorithms on such number of classes. In addition to that, apart from other studies exploiting deep learning approach, we then examine the potential input types for LSTM network, including raw sensor data, knowledge-based features and features obtained via Auto Encoder. Experimental results show that our proposed idea, feeding knowledge-based features into frames of LSTM network make a remarkable difference and bring out a robust, orientation-independent and generic solution for these well-known problems including activity recognition and transport mode detection. Besides, we examined the hyper-parameters of our deep learning approach and specified the effective parameter set including window size, number of frames, unit size, dropout rate and batch size. Our test results reveal that we could achieve a success rate of 95.5% and outperform the state-of-the art solutions for 12 different transportation modes with the help of our former algorithm, namely Healing, on a totally journey-independent dataset where instances for train, validation and test datasets are selected from different journeys.
Dynamic Resource Allocation Method Based on Symbiotic Organism Search Algorithm in Cloud Computing

Cloud computing is a popular and powerful paradigm for leasing IT services over the Internet. It allows sharing the resources among a large number of consumers. This requires allocating and releasing resources dynamically over time. Since resources are paid as are used, ensuring a better level of service is a challenge for cloud providers. They must provide enough resources to meet user needs with maintaining the quality of service. Therefore, QoS is a serious problem that arises during the dynamic management of heterogeneous resources. This type of problem can be resolved by minimizing execution time and resource reservation prices. It is necessary to integrate a new method well enough to improve the performance of the cloud, in this area. This article presents a dynamic resource allocation model for the cloud computing environment. In addition, we propose a meta-heuristic approach named Multi-Objectives Symbiotic Organism Search algorithm (MOSOS) for resource allocation. We specifically designed MOSOS to minimize both the makespan and cost. Simulation results revealed that MOSOS gave a better result compared to other methods. It shown significant adaptation with the dynamic change of the cloud, as well as minimizing the execution time. Hence, improving the QoS given to cloud users.
Vehicle Detection Based on Improved Yolov3 Algorithm

At present, the degree of urban traffic congestion is increasing, so it is necessary to detect and predict the road traffic situation. However, the current vehicle detection has some problems, such as poor detection effect and inaccurate classification of relatively small vehicles. To solve these problems, an improved YOLOv3 algorithm for vehicle detection is proposed. This algorithm improves the traditional YOLO algorithm. Firstly, it uses clustering analysis method to cluster the data set, and improves the network structure to increase the number of final output grids and enhance the relatively small vehicle prediction ability. Secondly, it optimizes the data set and optimizes the input image. Resolution makes it robust under different external conditions. Experiments show that the improved YOLOv3 algorithm has a higher detection accuracy and a higher detection rate than the traditional algorithm.
A Characteristic Function Approach to Deep Implicit Generative Modeling

Implicit Generative Models (IGMs) such as GANs have emerged as effective data-driven models for generating samples, particularly images. In this paper, we formulate the problem of learning an IGM as minimizing the expected distance between characteristic functions. Specifically, we minimize the distance between characteristic functions of the real and generated data distributions under a suitably-chosen weighting distribution. This distance metric, which we term as the characteristic function distance (CFD), can be (approximately) computed with linear time-complexity in the number of samples, in contrast with the quadratic-time Maximum Mean Discrepancy (MMD). By replacing the discrepancy measure in the critic of a GAN with the CFD, we obtain a model that is simple to implement and stable to train. The proposed metric enjoys desirable theoretical properties including continuity and differentiability with respect to generator parameters, and continuity in the weak topology. We further propose a variation of the CFD in which the weighting distribution parameters are also optimized during training; this obviates the need for manual tuning, and leads to an improvement in test power relative to CFD. We demonstrate experimentally that our proposed method outperforms WGAN and MMD-GAN variants on a variety of unsupervised image generation benchmarks.
Early Experience in Benchmarking Edge AI Processors with Object Detection Workloads

Nowadays, GPGPU plays an important role in data centers for Deep Learning training. However, GPU might not be suitable for many Deep Learning inference applications, especially for Edge Computing scenarios, due to its high power consumption and high cost. Thus, researchers and engineers have spent a lot of effort on designing edge-side artificial intelligence (AI) processors recently. Because of different edge-side application requirements, edge AI processors are designed with different approaches, which make these processors very diversified. This scenario makes it hard for customers to decide what kind of processors may be more beneficial for their requirements. To provide a selection guidance, this paper proposes a three-dimensional benchmarking methodology and shares the early experience of evaluating three different kinds of edge AI processors (i.e., Edge TPU, NVIDIA Xavier, and NovuTensor) with object detection workloads (i.e., Tiny-YOLO and YOLOv2 with Microsoft COCO dataset). We also characterize a GPU platform (i.e., GTX 1080 Ti) from the three dimensions of accuracy, latency, and energy efficiency. Based on our experimental observations, we find that edge AI processors are able to deliver better energy efficiency (e.g., Edge TPU has the highest energy efficiency in our experiments.), while NovuTensor and Xavier, can also provide comparable performance in latency as GPU. Further, all these edge AI processors can achieve similar accuracy as GPU. The differences among these processors and GPU are less than 3%.
Frequency Bias in Neural Networks for Input of Non-Uniform Density

Recent works have partly attributed the generalization ability of over-parameterized neural networks to frequency bias -- networks trained with gradient descent on data drawn from a uniform distribution find a low frequency fit before high frequency ones. As realistic training sets are not drawn from a uniform distribution, we here use the Neural Tangent Kernel (NTK) model to explore the effect of variable density on training dynamics. Our results, which combine analytic and empirical observations, show that when learning a pure harmonic function of frequency $\kappa$, convergence at a point $\x \in \Sphere^{d-1}$ occurs in time $O(\kappa^d/p(\x))$ where $p(\x)$ denotes the local density at $\x$. Specifically, for data in $\Sphere^1$ we analytically derive the eigenfunctions of the kernel associated with the NTK for two-layer networks. We further prove convergence results for deep, fully connected networks with respect to the spectral decomposition of the NTK. Our empirical study highlights similarities and differences between deep and shallow networks in this model.
Moiré Pattern Removal via Attentive Fractal Network

Moiré patterns are commonly seen artifacts when taking photos of screens and other objects with high-frequency textures. It's challenging to remove the moiré patterns considering its complex color and shape. In this work, we propose an Attentive Fractal Network to effectively solve this problem. First, we construct each Attentive Fractal Block with progressive feature fusion and channel-wise attention guidance. The network is then fractally stacked with the block on each of its levels. Second, to further boost the performance, we adopt a two-stage augmented refinement strategy. With these designs, our method wins the burst demoiréing track and achieves second place in single image demoireing and single image deblurring tracks inNTIRE20 Challenges. Extensive experiments demonstrate the superiority of our method for moiré pattern removal compared to existing state-of-the-art methods, and prove the effectiveness of its each component. We will publicly release our code and trained weights on https://github.com/ir1d/AFN.
SimUSR: A Simple but Strong Baseline for Unsupervised Image Super-resolution

In this paper, we tackle a fully unsupervised super-resolution problem, i.e., neither paired images nor ground truth HR images. We assume that low resolution (LR) images are relatively easy to collect compared to high resolution (HR) images. By allowing multiple LR images, we build a set of pseudo pairs by denoising and downsampling LR images and cast the original unsupervised problem into a supervised learning problem but in one level lower. Though this line of study is easy to think of and thus should have been investigated prior to any complicated unsupervised methods, surprisingly, there are currently none. Even more, we show that this simple method outperforms the state-of- the-art unsupervised method with a dramatically shorter latency at runtime, and significantly reduces the gap to the HR supervised models. We submitted our method in NTIRE 2020 super-resolution challenge and won 1st in PSNR, 2nd in SSIM, and 13th in LPIPS. This simple method should be used as the baseline to beat in the future, especially when multiple LR images are allowed during the training phase. However, even in the zero-shot condition, we argue that this method can serve as a useful baseline to see the gap between supervised and unsupervised frameworks.
A Unified Analytical model for proof-of-X schemes

Nakamoto protocol, practically solving the Byzantine Generals Problem, can support a variety of proof-based consensus engines, referred to as Proof-of-X (PoX) in permissionless Blockchains. However, there has been to date in lack of a general approach for each miner to evaluate its steady-state profit against the competitors. This paper presents a Markov model which captures explicitly the weighted resource distribution of PoX schemes in large-scale networks and unifies the analysis of different PoX schemes. The new model leads to the development of three new unified metrics for the evaluation, namely, Resource Sensitivity, System Convergence, and Resource Fairness, accounting for security, stability, and fairness, respectively. The generality and applicability of our model are validated by simulation results, revealing that among typically non-Fairness-oriented PoX schemes (such as Proof-of-Work (PoW) and Proof-of-Stake (PoS)), the strongly restricted coinage-based PoS with a Pareto-distributed resource can offer the best performance on Resource Sensitivity, while Proof-of-Publication (PoP) with normal-distributed resource performs the best on System Convergence. Our simulations also reveal the important role of carefully designed Resource Fairness parameter in balancing Resource Sensitivity and System Convergence and improving the performance compared with other non-Fairness-oriented PoX schemes.
Argument-based assessment of predictive uncertainty of data-driven environmental models

Increasing volumes of data allow environmental scientists to use machine learning to construct data-driven models of phenomena. These models can provide decision-relevant predictions, but confident decision-making requires that the involved uncertainties are understood. We argue that existing frameworks for characterizing uncertainties are not appropriate for data-driven models because of their focus on distinct locations of uncertainty. We propose a framework for uncertainty assessment that uses argument analysis to assess the justification of the assumption that the model is fit for the predictive purpose at hand. Its flexibility makes the framework applicable to data-driven models. The framework is illustrated using a case study from environmental science. We show that data-driven models can be subject to substantial second-order uncertainty, i.e., uncertainty in the assessment of the predictive uncertainty, because they are often applied to ill-understood problems. We close by discussing the implications of the predictive uncertainties of data-driven models for decision-making.