Reviewerly/exHarmony huggingface dataset:
Fine-Grained Image Analysis With Deep Learning: A Survey

Fine-grained image analysis (FGIA) is a longstanding and fundamental problem in computer vision and pattern recognition, and underpins a diverse set of real-world applications. The task of FGIA targets analyzing visual objects from subordinate categories, e.g., species of birds or models of cars. The small inter-class and large intra-class variation inherent to fine-grained image analysis makes it a challenging problem. Capitalizing on advances in deep learning, in recent years we have witnessed remarkable progress in deep learning powered FGIA. In this paper we present a systematic survey of these advances, where we attempt to re-define and broaden the field of FGIA by consolidating two fundamental fine-grained research areas - fine-grained image recognition and fine-grained image retrieval. In addition, we also review other key issues of FGIA, such as publicly available benchmark datasets and related domain-specific applications. We conclude by highlighting several research directions and open problems which need further exploration from the community.
Global genetic learning particle swarm optimization with diversity enhancement by ring topology

Genetic learning particle swarm optimization (GL-PSO) improves the performance of particle swarm optimization (PSO) by breeding superior exemplars to guide the motion of particles. However, GL-PSO adopts a global topology for exemplar generation and cannot preserve sufficient diversity to enhance exploration, and therefore, its performance on complex optimization problems is unsatisfactory. To further improve GL-PSO's performance and adaptability, two modifications are incorporated into the original GL-PSO. A ring topology is adopted in exemplar generation to enhance diversity and exploration, while a global learning component (GLC) with linearly adjusted control parameters is employed to improve the algorithm's adaptability. The resultant algorithm is referred to as global genetic learning particle swarm optimization with diversity enhancement by ring topology (GGL-PSOD). To validate the effectiveness of these two modifications, they are combined with GL-PSO separately and together and further tested experimentally. The comparison results on the CEC2017 test suite show that the adoption of the ring topology in exemplar generation can enhance the diversity and exploration capability of GL-PSO, while combining GLC alone with GL-PSO cannot achieve significant improvement. Incorporating both modifications into GL-PSO, the resultant GGL-PSOD exhibits high performance and strong adaptability on different types of CEC2017 functions. It outperforms seven representative PSO variants and five non-PSO meta-heuristics, including GL-PSO, SL-PSO, HCLPSO, EPSO, ABC, CMA-ES and CS.
Recommender system based on pairwise association rules

Recommender systems based on methods such as collaborative and content-based filtering rely on extensive user profiles and item descriptors as well as on an extensive history of user preferences. Such methods face a number of challenges; including the cold-start problem in systems characterized by irregular usage, privacy concerns, and contexts where the range of indicators representing user interests is limited. We describe a recommender algorithm that builds a model of collective preferences independently of personal user interests and does not require a complex system of ratings. The performance of the algorithm is analyzed on a large transactional data set generated by a real-world dietary intake recall system.
A novel image cipher based on 3D bit matrix and latin cubes

This paper proposes a novel image encryption algorithm based on 3D bit matrix and orthogonal Latin cubes. Unlike most of the existing image encryption algorithms, the proposed algorithm treats the original image as a 3D bit matrix, and uses Latin cubes for confusion and diffusion. The utilization of 3D bit matrix and Latin cubes makes the proposed algorithm achieve not only a desirable level of security, but also high efficiency, so the algorithm is suitable for practical application.
Deep Online Video Stabilization With Multi-Grid Warping Transformation Learning

Video stabilization techniques are essential for most hand-held captured videos due to high-frequency shakes. Several 2D-, 2.5D-, and 3D-based stabilization techniques have been presented previously, but to the best of our knowledge, no solutions based on deep neural networks had been proposed to date. The main reason for this omission is shortage in training data as well as the challenge of modeling the problem using neural networks. In this paper, we present a video stabilization technique using a convolutional neural network. Previous works usually propose an off-line algorithm that smoothes a holistic camera path based on feature matching. Instead, we focus on low-latency, real-time camera path smoothing that does not explicitly represent the camera path and does not use future frames. Our neural network model, called StabNet, learns a set of mesh-grid transformations progressively for each input frame from the previous set of stabilized camera frames and creates stable corresponding latent camera paths implicitly. To train the network, we collect a dataset of synchronized steady and unsteady video pairs via a specially designed hand-held hardware. Experimental results show that our proposed online method performs comparatively to the traditional off-line video stabilization methods without using future frames while running about 10 times faster. More importantly, our proposed StabNet is able to handle low-quality videos, such as night-scene videos, watermarked videos, blurry videos, and noisy videos, where the existing methods fail in feature extraction or matching.
Scaling description of generalization with number of parameters in deep learning

Supervised deep learning involves the training of neural networks with a large number $N$ of parameters. For large enough $N$, in the so-called over-parametrized regime, one can essentially fit the training data points. Sparsity-based arguments would suggest that the generalization error increases as $N$ grows past a certain threshold $N^{*}$. Instead, empirical studies have shown that in the over-parametrized regime, generalization error keeps decreasing with $N$. We resolve this paradox through a new framework. We rely on the so-called Neural Tangent Kernel, which connects large neural nets to kernel methods, to show that the initialization causes finite-size random fluctuations $\|f_{N}-\bar{f}_{N}\|\sim N^{-1/4}$ of the neural net output function $f_{N}$ around its expectation $\bar{f}_{N}$. These affect the generalization error $\epsilon_{N}$ for classification: under natural assumptions, it decays to a plateau value $\epsilon_{\infty}$ in a power-law fashion $\sim N^{-1/2}$. This description breaks down at a so-called jamming transition $N=N^{*}$. At this threshold, we argue that $\|f_{N}\|$ diverges. This result leads to a plausible explanation for the cusp in test error known to occur at $N^{*}$. Our results are confirmed by extensive empirical observations on the MNIST and CIFAR image datasets. Our analysis finally suggests that, given a computational envelope, the smallest generalization error is obtained using several networks of intermediate sizes, just beyond $N^{*}$, and averaging their outputs.
Achieving Super-Resolution Remote Sensing Images via the Wavelet Transform Combined With the Recursive Res-Net

Deep learning (DL) has been successfully applied to single image super-resolution (SISR), which aims at reconstructing a high-resolution (HR) image from its low-resolution (LR) counterpart. Different from most current DL-based methods, which perform reconstruction in the spatial domain, we use a scheme based in the frequency domain to reconstruct the HR image at various frequency bands. Further, we propose a method that incorporates the wavelet transform (WT) and the recursive Res-Net. The WT is applied to the LR image to divide it into various frequency components. Then, an elaborately designed network with recursive residual blocks is used to predict high-frequency components. Finally, the reconstructed image is obtained via the inverse WT. This paper has three main contributions: 1) an SISR scheme based on the frequency domain is proposed under a DL framework to fully exploit the potential to depict images at different frequency bands; 2) recursive block and residual learning in global and local manners are adopted to ease the training of the deep network, and the batch normalization layer is removed to increase the flexibility of the network, save memory, and promote speed; and 3) the low-frequency wavelet component is replaced by an LR image with more details to further improve performance. To validate the effectiveness of the proposed method, extensive experiments are performed using the NWPU-RESISC45 data set, and the results demonstrate that the proposed method outperforms several state-of-the-art methods in terms of both objective evaluation and subjective perspective.
Nearly Optimal Lattice Simulation by Product Formulas

We consider simulating an $n$-qubit Hamiltonian with nearest-neighbor interactions evolving for time $t$ on a quantum computer. We show that this simulation has gate complexity $(nt)^{1+o(1)}$ using product formulas, a straightforward approach that has been demonstrated by several experimental groups. While it is reasonable to expect this complexity---in particular, this was claimed without rigorous justification by Jordan, Lee, and Preskill---we are not aware of a straightforward proof. Our approach is based on an analysis of the local error structure of product formulas, as introduced by Descombes and Thalhammer and further simplified here. We prove error bounds for canonical product formulas, which include well-known constructions such as the Lie-Trotter-Suzuki formulas. We also develop a local error representation for time-dependent Hamiltonian simulation, and we discuss generalizations to periodic boundary conditions, constant-range interactions, and higher dimensions. Combined with a previous lower bound, our result implies that product formulas can simulate lattice Hamiltonians with nearly optimal gate complexity.
Politicians in the line of fire: Incivility and the treatment of women on social media

A seemingly inescapable feature of the digital age is that people choosing to devote their lives to politics must now be ready to face a barrage of insults and disparaging comments targeted at them through social media. This article represents an effort to document this phenomenon systematically. We implement machine learning models to predict the incivility of about 2.2 m messages addressed to Canadian politicians and US Senators on Twitter. Specifically, we test whether women in politics are more heavily targeted by online incivility, as recent media reports suggested. Our estimates indicate that roughly 15% of public messages sent to Senators can be categorized as uncivil, whereas the proportion is about four points lower in Canada. We find evidence that women are more heavily targeted by uncivil messages than men, although only among highly visible politicians.
Semi-Supervised Graph Classification: A Hierarchical Graph Perspective

Node classification and graph classification are two graph learning problems that predict the class label of a node and the class label of a graph respectively. A node of a graph usually represents a real-world entity, e.g., a user in a social network, or a protein in a protein-protein interaction network. In this work, we consider a more challenging but practically useful setting, in which a node itself is a graph instance. This leads to a hierarchical graph perspective which arises in many domains such as social network, biological network and document collection. For example, in a social network, a group of people with shared interests forms a user group, whereas a number of user groups are interconnected via interactions or common members. We study the node classification problem in the hierarchical graph where a “node” is a graph instance, e.g., a user group in the above example. As labels are usually limited in real-world data, we design two novel semi-supervised solutions named SEmi-supervised grAph cLassification via Cautious/Active Iteration (or SEAL-C/AI in short). SEAL-C/AI adopt an iterative framework that takes turns to build or update two classifiers, one working at the graph instance level and the other at the hierarchical graph level. To simplify the representation of the hierarchical graph, we propose a novel supervised, self-attentive graph embedding method called SAGE, which embeds graph instances of arbitrary size into fixed-length vectors. Through experiments on synthetic data and Tencent QQ group data, we demonstrate that SEAL-C/AI not only outperform competing methods by a significant margin in terms of accuracy/Macro-F1, but also generate meaningful interpretations of the learned representations.
Semi-Supervised Entity Alignment via Knowledge Graph Embedding with Awareness of Degree Difference

Entity alignment associates entities in different knowledge graphs if they are semantically same, and has been successfully used in the knowledge graph construction and connection. Most of the recent solutions for entity alignment are based on knowledge graph embedding, which maps knowledge entities in a low-dimension space where entities are connected with the guidance of prior aligned entity pairs. The study in this paper focuses on two important issues that limit the accuracy of current entity alignment solutions: 1) labeled data of priorly aligned entity pairs are difficult and expensive to acquire, whereas abundant of unlabeled data are not used; and 2) knowledge graph embedding is affected by entity's degree difference, which brings challenges to align high frequent and low frequent entities. We propose a semi-supervised entity alignment method (SEA) to leverage both labeled entities and the abundant unlabeled entity information for the alignment. Furthermore, we improve the knowledge graph embedding with awareness of the degree difference by performing the adversarial training. To evaluate our proposed model, we conduct extensive experiments on real-world datasets. The experimental results show that our model consistently outperforms the state-of-the-art methods with significant improvement on alignment accuracy.
A Hybrid E-Learning Recommendation Approach Based on Learners’ Influence Propagation

In e-learning recommender systems, interpersonal information between learners is very scarce, which makes it difficult to apply collaborative filtering (CF) techniques to achieve recommendations. In this study, we propose a hybrid filtering recommendation approach (SI - IFL) combining learner influence model (LIM), self-organization based (SOB) recommendation strategy, and sequential pattern mining (SPM) together for recommending learning objects (LOs) to learners. The method works as follows: (i) LIM is applied to acquire the interpersonal information by computing the influence that a learner exerts on others. LIM consists of learner similarity, knowledge credibility, and learner aggregation, meanwhile, LIM is independent of ratings. Furthermore, to address the uncertainty and fuzzy natures of learners, intuitionistic fuzzy logic (IFL) is applied to optimize the LIM. (ii) A SOB recommendation strategy is applied to recommend the optimal learner cliques for active learners by simulating the influence propagation among learners. Influence propagation means that a learner can move towards active learners, and such behaviors can stimulate the moving behaviors of his/her neighbors. This SOB recommendation approach achieves a stable structure based on distributed and bottom-up behaviors of individuals. (iii) SPM is applied to decide the final learning objects (LOs) and navigational paths based on the recommended learner cliques. The experimental results demonstrate that SI - IFL can provide personalized and diversified recommendations, and it shows promising efficiency and adaptability in e-learning scenarios.
Secure Edge of Things for Smart Healthcare Surveillance Framework

The vast development of the Internet of Things (IoT) and cloud-enabled data processing solutions provide the opportunity to build novel and fascinating smart, connected healthcare systems. Smart healthcare systems analyze the IoT-generated patient data to both enhance the quality of patient care and reduce healthcare costs. A major challenge for these systems is how the Cloud of Things can handle the data generated from billions of connected IoT devices. Edge computing infrastructure offers a promising solution by operating as a middle layer between the IoT devices and cloud computing. The Edge of Things (EoT) can offer small-scale real-time computing and storage capabilities that ensures low latency and optimal utilization of the IoT resources. However, the EoT has privacy-preservation issues, which is a significant concern for the healthcare systems that contain sensitive patient data. This paper introduces a novel EoT computing framework for secure and smart healthcare surveillance services. Fully homomorphic encryption preserves data privacy and is stored and processed within an EoT framework. A distributed approach for clustering-based techniques is developed for the proposed EoT framework with the scalability to aggregate and analyze the large-scale and heterogeneous data in the distributed EoT devices independently before it is sent to the cloud. We demonstrate the proposed framework by evaluating a case study for the patient biosignal data. Our framework rapidly accelerates the analysis response time and performance of the encrypted data processing while preserving a high level of analysis accuracy and data privacy.
Systematic risk in cryptocurrency market: Evidence from DCC-MGARCH model

This study provides a formal analysis on the structural breaks and volatility spillovers in seven largest cryptocurrencies including Bitcoin, Litecoin, Ripple, Stellar, Monero, Dash, and Bytecoin. Cumulative sum test for parameter stability, Granger Causality test, LM test for ARCH and Dynamic conditional correlation MGARCH model indicate that: (1) the structural breaks are universally present in these popular cryptocurrencies; and (2) the shifts spread from smaller cryptocurrencies (in market capitalization) to larger ones. Notably, volatility spillovers also exist with strong positive correlations among cryptocurrencies. Our findings highlight the limit of diversification benefits within the cryptocurrency market itself.
TeTrIS: Template Transformer Networks for Image Segmentation With Shape Priors

In this paper, we introduce and compare different approaches for incorporating shape prior information into neural network-based image segmentation. Specifically, we introduce the concept of template transformer networks, where a shape template is deformed to match the underlying structure of interest through an end-to-end trained spatial transformer network. This has the advantage of explicitly enforcing shape priors, and this is free of discretization artifacts by providing a soft partial volume segmentation. We also introduce a simple yet effective way of incorporating priors in the state-of-the-art pixel-wise binary classification methods such as fully convolutional networks and U-net. Here, the template shape is given as an additional input channel, incorporating this information significantly reduces false positives. We report results on synthetic data and sub-voxel segmentation of coronary lumen structures in cardiac computed tomography showing the benefit of incorporating priors in neural network-based image segmentation.
Champion-challenger analysis for credit card fraud detection: Hybrid ensemble and deep learning

Credit card fraud detection is an essential part of screening fraudulent transactions in advance of their authorization by card issuers. Although credit card frauds occur extremely infrequently, they result in huge losses as most fraudulent transactions have large values. An adequate detection of fraud allows investigators to take timely actions that can potentially prevent additional fraud or financial losses. In practice, however, investigators can only check a few alerts per day since the investigation process can be long and tedious. Thus, the primary goal of the fraud detection model is to return accurate alerts with fewer false alarms and missed frauds. Conventional fraud detection is mainly based on the hybrid ensemble of diverse machine learning models. Recently, several studies have compared deep learning and traditional machine learning models including ensemble. However, these studies used evaluation methods without considering that the real-world fraud detection system operated with the constraints: (i) the number of investigators who check the high-risk transactions from the data-driven scoring models are limited and (ii) the two types of misclassification, false alarms and missed frauds, have different costs. In this study, we conducted an in-depth comparison between the hybrid ensemble and deep learning method to determine whether or not to adopt the latter in our partner’s system that currently operates with the hybrid ensemble model. To compare the two, we introduced the champion-challenger framework and the development process of the two models. After developing the two models, we evaluated them on large transaction data sets taken from our partner, a major card issuing company in South Korea. We used various practical evaluation metrics appropriate for this domain that has severe class and cost imbalances. Moreover, we deployed these models in a real-world fraud detection system to check the post-launch performance for one month. The challenger outperformed the champion on both in off-line and post-launch tests.
Furion: Engineering High-Quality Immersive Virtual Reality on Today's Mobile Devices

Despite the growing market penetration, today's high-end virtual reality (VR) systems remain tethered, which not only limits users' VR experience but also creates a safety hazard. In this paper, we perform a systematic design study of the “elephant in the room” facing the VR industry - is it feasible to enable high-quality VR apps on untethered mobile devices such as smartphones? Our quantitative, performance-driven design study makes two contributions. First, we show that the QoE achievable for high-quality VR applications on today's mobile hardware and wireless networks via local rendering or offloading is about 10X away from the acceptable QoE, yet waiting for future mobile hardware or next-generation wireless networks (e.g., 5G) is unlikely to help, because of power limitation and the higher CPU utilization needed for processing packets under higher data rate. Second, we present Furion, a VR framework that enables high-quality, immersive mobile VR on today's mobile devices and wireless networks. Furion exploits a key insight about the VR workload that foreground interactions and background environment have contrasting predictability and rendering workload, and employs a split renderer architecture running on both the phone and the server. Supplemented with video compression, use of panoramic frames, parallel decoding on multiple cores on the phone, and view-based bitrate adaptation we demonstrate Furion can support high-quality VR apps on today's smartphones over WiFi, with under 14 ms latency and 60 FPS (the phone display refresh rate).