Reviewerly/exHarmony huggingface dataset:
Information Extraction from Text Intensive and Visually Rich Banking Documents

Document types, where visual and textual information plays an important role in their analysis and understanding, pose a new and attractive area for information extraction research. Although cheques, invoices, and receipts have been studied in some previous multi-modal studies, banking documents present an unexplored area due to the naturalness of the text they possess in addition to their visual richness. This article presents the first study which uses visual and textual information for deep-learning based information extraction on text-intensive and visually rich scanned documents which are, in this instance, unstructured banking documents, or more precisely, money transfer orders. The impact of using different neural word representations (i.e., FastText, ELMo, and BERT) on IE subtasks (namely, named entity recognition and relation extraction stages), positional features of words on document images and auxiliary learning with some other tasks are investigated. The article proposes a new relation extraction algorithm based on graph factorization to solve the complex relation extraction problem where the relations within documents are n-ary, nested, document-level, and previously indeterminate in quantity. Our experiments revealed that the use of deep learning algorithms yielded around 10 percentage points improvement on the IE sub-tasks. The inclusion of word positional features yielded around 3 percentage points of improvement in some specific information fields. Similarly, our auxiliary learning experiments yielded around 2 percentage points of improvement on some information fields associated with the specific transaction type detected by our auxiliary task. The integration of the information extraction system into a real banking environment reduced cycle times substantially. When compared to the manual workflow, document processing pipeline shortened book-to-book money transfers to 10 minutes (from 29 min.) and electronic fund transfers (EFT) to 17 minutes (from 41 min.) respectively.
ARKit and ARCore in serve to augmented reality

Many libraries are available in the development world to create augmented reality applications, their functionality differs depending on the technology used to detect and track an object, points or features in a scene. In this article, we will discover ARKit, ARCore two open source libraries that display a virtual object in the real world. Their goal is to merge digital content and information with the real world. They can interact with the components of the device (camera and screen) to detect and track characteristics of the scene in order to insert virtual content. This study implements and concretizes the different functionalities available in augmented reality to enrich the real world with additional information.
Survey of Explainable Machine Learning with Visual and Granular Methods Beyond Quasi-Explanations

This chapter surveys and analyses visual methods of explainabilityExplainability of Machine Learning (ML)Machine learning approaches with focus on moving from quasi-explanations that dominate in ML to actual domain-specific explanation supported by granular visuals. The importance of visual and granular methods to increase the interpretabilityInterpretability and validity of the ML model has grown in recent years. Visuals have an appeal to human perception, which other methods do not. ML interpretation is fundamentally a human activity, not a machine activity. Thus, visual methods are more readily interpretable. Visual granularityGranularity is a natural way for efficient ML explanation. Understanding complex causal reasoning can be beyond human abilities without “downgrading” it to human perceptual and cognitive limits. The visual exploration of multidimensional data at different levels of granularityGranularity for knowledge discovery is a long-standing research focus. While multiple efficient methods for visual representation of high-dimensional data exist, the loss of interpretable information, occlusion, and clutter continue to be a challenge, which lead to quasi-explanations. This chapter starts with the motivation and the definitions of different forms of explainabilityExplainability and how these concepts and information granularityGranularity can integrate in ML. The chapter focuses on a clear distinction between quasi-explanations and actual domain specific explanations, as well as between potentially explainable and an actually explained ML model that are critically important for the further progress of the ML explainabilityExplainability domain. We discuss foundations of interpretabilityInterpretability, overview visual interpretabilityInterpretability and present several types of methods to visualize the ML models. Next, we present methods of visual discovery of ML models, with the focus on interpretable models, based on the recently introduced concept of General Line CoordinatesGeneral line coordinates (GLC). This family of methods take the critical step of creating visual explanations that are not merely quasi-explanations but are also domain specific visual explanations while these methods themselves are domain-agnostic. The chapter includes results on theoretical limits to preserve n-D distances in lower dimensions, based on the Johnson-Lindenstrauss lemma, point-to-point and point-to-graph GLC approaches, and real-world case studies. The chapter also covers traditional visual methods for understanding multiple ML models, which include deep learningDeep learning and time series models. We illustrate that many of these methods are quasi-explanations and need further enhancement to become actual domain specific explanations. The chapter concludes with outlining open problems and current research frontiers.
Improving the robustness of recursive consequent parameters learning in evolving neuro-fuzzy systems

During the last 15 to 20 years, evolving (neuro-) fuzzy systems (E(N) FS) have enjoyed more and more attraction in the context of data stream mining and modeling processes. This is because they can be updated on the fly in a single-pass sample-wise manner and are able to perform autonomous changes of the models on structural level in order to react onto process drifts. A wide variety of evolving (neuro-) fuzzy systems approaches have been proposed in order to handle data stream mining and modeling processes by dynamically updating the rule structure and antecedents. The current denominator in the update of the consequent (output weight) parameters is the usage of the recursive (fuzzily weighted) least squares estimator (R(FW) LS), as being applied in almost all E(N) FS approaches. In this paper, we propose and examine alternative variants for consequent parameter updates, namely multi-innovation RFWLS, recursive correntropy and especially recursive weighted total least squares (RWTLS). Multi-innovation RFWLS guarantees more stability in the update whenever structural changes (i.e. changes in the antecedents) in the E(N) FS are performed. This is because rule membership degrees are actualized on (a portion of) past samples and properly integrated in each update step. Recursive correntropy addresses the problematic of outliers by down-weighing the influence of higher errors in the parameter updates. Recursive weighted total least squares also takes into account a possible noise level in the input variables (and not solely in the target variable as done in RFWLS). The approaches are compared with standard RFWLS i.) on three data stream regression problems from practical applications, which are affected by noise levels and where one embeds a known drift, and ii.) on a time-series based forecasting problem. The results based on accumulated prediction error trends over time indicate that RFWLS can be largely outperformed by the proposed alternative variants, and this with even lower sensitivity on various data noise levels. So, the proposed variants could be worth of being further considered as promising and serious alternatives.
Randomized time‐varying knapsack problems via binary beetle antennae search algorithm: Emphasis on applications in portfolio insurance

The knapsack problem is a problem in combinatorial optimization, and in many such problems, exhaustive search is not tractable. In this paper, we describe and analyze the randomized time‐varying knapsack problem (RTVKP) as a time‐varying integer linear programming (TV‐ILP) problem. In this way, we present the on‐line solution to the RTVKP combinatorial optimization problem and highlight the restrictions of static methods. In addition, the RTVKP is applied in the field of finance and converted into a portfolio insurance problem. Our methodology is confirmed by simulation tests in real‐world data sets, in order to explain being an excellent alternative to traditional approaches.
A Federated Recommender System for Online Services

Due to privacy and security constraints, directly sharing user data between parties is undesired. Such decentralized data silo issues commonly exist in recommender systems. In general, recommender systems are data-driven. The more data it uses, the better performance it obtains. The data silo issues is a severe limitation of the recommender’s performance. Federated learning is an emerging technology, which bridges the data silos and builds machine learning models without compromising user privacy and data security. We design a recommender system based on federated learning. It is known as the federated recommender system. The system implements plenty of popular algorithms to support various online recommendation services. The algorithm implementation is open-sourced. We also deploy the system on a real-world content recommendation application, achieving significant performance improvement. In this demonstration, we present the architecture of the federated recommender system and give an online demo to show its detailed working procedures and results in content recommendations.
Multi-View Spectral Clustering with High-Order Optimal Neighborhood Laplacian Matrix

Multi-view spectral clustering can effectively reveal the intrinsic cluster structure among data by performing clustering on the learned optimal embedding across views. Though demonstrating promising performance in various applications, most of existing methods usually linearly combine a group of pre-specified first-order Laplacian matrices to construct the optimal Laplacian matrix, which may result in limited representation capability and insufficient information exploitation. Also, storing and implementing complex operations on the <formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex>{$n\times n}$</tex></formula> Laplacian matrices incurs intensive storage and computation complexity. To address these issues, this paper first proposes a multi-view spectral clustering algorithm that learns a high-order optimal neighborhood Laplacian matrix, and then extends it to the late fusion version for accurate and efficient multi-view clustering. Specifically, our proposed algorithm generates the optimal Laplacian matrix by searching the neighborhood of the linear combination of both the first-order and high-order base Laplacian matrices simultaneously. By this way, the representative capacity of the learned optimal Laplacian matrix is enhanced, which is helpful to better utilize the hidden high-order connection information among data, leading to improved clustering performance. We design an efficient algorithm with proved convergence to solve the resultant optimization problem. Extensive experimental results on nine datasets demonstrate the superiority of the proposed algorithm
Blockchain-Based Knowledge Automation for CPSS-Oriented Parallel Management

Traditional organization management typically follows a top-down pyramid structure, which is widely believed to have many problems in releasing innovation potentials. In the new era of intelligent technologies, knowledge automation is required to meet the urgent demand for rapid acquisition and application of knowledge. With the rapidly deepened integration of the real world and the virtual society, cyber-physical-social system (CPSS)-oriented parallel management proves to be an effective and efficient way in solving these problems. In this article, we utilize blockchain technology and smart contracts in knowledge automation and investigate blockchain-based knowledge automation, which can be used for CPSS-oriented parallel management. We also propose a management framework based on the smart contract and discuss a case study.
An Empirical Study on Bugs Inside TensorFlow

In recent years, deep learning has become a hot research topic. Although it achieves incredible positive results in some scenarios, bugs inside deep learning software can introduce disastrous consequences, especially when the software is used in safety-critical applications. To understand the bug characteristic of deep learning software, researchers have conducted several empirical studies on deep learning bugs. Although these studies present useful findings, we notice that none of them analyze the bug characteristic inside a deep learning library like TensorFlow. We argue that some fundamental questions of bugs in deep learning libraries are still open. For example, what are the symptoms and the root causes of bugs inside TensorFlow, and where are they? As the underlying library of many deep learning projects, the answers to these questions are useful and important, since its bugs can have impacts on many deep learning projects. In this paper, we conduct the first empirical study to analyze the bugs inside a typical deep learning library, i.e., TensorFlow. Based on our results, we summarize 5 findings, and present our answers to 2 research questions. For example, we find that the symptoms and root causes of TensorFlow bugs are more like ordinary projects (e.g., Mozilla) than other machine learning libraries (e.g., Lucene). As another example, we find that most TensorFlow bugs reside in its interfaces (26.24%), learning algorithms (11.79%), and how to compile (8.02%), deploy (7.55%), and install (4.72%) TensorFlow across platforms.
Consistent Video Style Transfer via Relaxation and Regularization

In recent years, neural style transfer has attracted more and more attention, especially for image style transfer. However, temporally consistent style transfer for videos is still a challenging problem. Existing methods, either relying on a significant amount of video data with optical flows or using single-frame regularizers, fail to handle strong motions or complex variations, therefore have limited performance on real videos. In this article, we address the problem by jointly considering the intrinsic properties of stylization and temporal consistency. We first identify the cause of the conflict between style transfer and temporal consistency, and propose to reconcile this contradiction by relaxing the objective function, so as to make the stylization loss term more robust to motions. Through relaxation, style transfer is more robust to inter-frame variation without degrading the subjective effect. Then, we provide a novel formulation and understanding of temporal consistency. Based on the formulation, we analyze the drawbacks of existing training strategies and derive a new regularization. We show by experiments that the proposed regularization can better balance the spatial and temporal performance. Based on relaxation and regularization, we design a zero-shot video style transfer framework. Moreover, for better feature migration, we introduce a new module to dynamically adjust inter-channel distributions. Quantitative and qualitative results demonstrate the superiority of our method over state-of-the-art style transfer methods. Our project is publicly available at: https://daooshee.github.io/ReReVST/.
Comparative Analysis between Dijkstra and Bellman-Ford Algorithms in Shortest Path Optimization

Abstract Due to the tremendous development in the field of computer and software sciences, the theory of graphics has spread widely and quickly, even becoming one of the most important sciences that played a large role in solving many problems of many diverse applications. These applications include computer protocols, Google Maps, games and more. Many researches have discussed shortest path algorithms to solve the shortest path problem in these applications. In this study, a very popular algorithms called Dijkstra algorithm and Bellman-Ford algorithm are used to make a comparison between them on the basis of complexity and performance in terms of shortest path optimization. Our results show that Dijkstra is better than the Bellman-Ford interms of execution time and more efficient for solving the shortest path issue, but the algorithm of Dijkstra work with non-negative edge weights.
ADER: Adaptively Distilled Exemplar Replay Towards Continual Learning for Session-based Recommendation

Session-based recommendation has received growing attention recently due to the increasing privacy concern. Despite the recent success of neural session-based recommenders, they are typically developed in an offline manner using a static dataset. However, recommendation requires continual adaptation to take into account new and obsolete items and users, and requires "continual learning" in real-life applications. In this case, the recommender is updated continually and periodically with new data that arrives in each update cycle, and the updated model needs to provide recommendations for user activities before the next model update. A major challenge for continual learning with neural models is catastrophic forgetting, in which a continually trained model forgets user preference patterns it has learned before. To deal with this challenge, we propose a method called Adaptively Distilled Exemplar Replay (ADER) by periodically replaying previous training samples (i.e., exemplars) to the current model with an adaptive distillation loss. Experiments are conducted based on the state-of-the-art SASRec model using two widely used datasets to benchmark ADER with several well-known continual learning techniques. We empirically demonstrate that ADER consistently outperforms other baselines, and it even outperforms the method using all historical data at every update cycle. This result reveals that ADER is a promising solution to mitigate the catastrophic forgetting issue towards building more realistic and scalable session-based recommenders.
Scheduling Real-time Deep Learning Services as Imprecise Computations

The paper presents a real-time computing framework for intelligent real-time edge services, on behalf of local embedded devices that are themselves unable to support extensive computations. The work contributes to a new direction in realtime computing that develops scheduling algorithms for machine intelligence tasks that enable anytime prediction. We show that deep neural network workflows can be cast as imprecise computations, each with a mandatory part and (several) optional parts whose execution utility depends on input data. With our design, deep neural networks can be preempted before their completion and support anytime inference. The goal of the realtime scheduler is to maximize the average accuracy of deep neural network outputs while meeting task deadlines, thanks to opportunistic shedding of the least necessary optional parts. The work is motivated by the proliferation of increasingly ubiquitous but resource-constrained embedded devices (for applications ranging from autonomous cars to the Internet of Things) and the desire to develop services that endow them with intelligence. Experiments on recent GPU hardware and a state of the art deep neural network for machine vision illustrate that our scheme can increase the overall accuracy by 10% ~ 20% while incurring (nearly) no deadline misses.
An efficient approach for forgery detection in digital images using Hilbert–Huang​ transform

Image manipulation plays important role in fake news spreading and it may cause ethical, economic, or political problems for people and sometimes for countries. Image integrity verification becomes a very important research issue due to increasing the forged images on the Internet and social media. The objective of this paper is presenting an accurate approach for digital image forgery detection has enough capability to sense any small image tampering and robustness against image manipulation attacks. The first step in the proposed approach is converting the RGB image into YCbCr space, then, the Hilbert–Huang Transform (HHT) features extracted from the chrominance-red component Cr, then, three different classifiers; Support Vector Machines (SVM), K-Nearest Neighbors (KNN), and Artificial Neuron Networks (ANN) have been tested and compared for image classification into authentic or forged. The results are verified using Structural-Similarity (SSIM) to calculate the forgery detection accuracy. The proposed approach has been tested with seven different manipulation images datasets; CASIA-V1, CASIA-V2, MICC-F2000, MICC-F600, MICC-F220, CoMoFoD and additional dataset collected from different Internet websites and social media. Furthermore, the proposed approach has been tested against post-processing attacks such as; image compression, adding Gaussian noises or adjusting the contrast of the image. The results show that, SVM classifier has achieved the highest accuracy compared to ANN and KNN classifiers. The proposed approach has been compared with other published approaches, and the comparison proved its superiority over the previously published approaches.
Integrated Motion Planner for Real-time Aerial Videography with a Drone in a Dense Environment

This work suggests an integrated approach for a drone (or multirotor) to perform an autonomous videography task in a 3-D obstacle environment by following a moving object. The proposed system includes 1) a target motion prediction module which can be applied to dense environments and 2) a hierarchical chasing planner. Leveraging covariant optimization, the prediction module estimates the future motion of the target assuming it efforts to avoid the obstacles. The other module, chasing planner, is in a bi-level structure composed of preplanner and smooth planner. In the first phase, we exploit a graph-search method to plan a chasing corridor which incorporates safety and visibility of target. In the subsequent phase, we generate a smooth and dynamically feasible trajectory within the corridor using quadratic programming (QP). We validate our approach with multiple complex scenarios and actual experiments. The source code and the experiment video can be found in https://github.com/icsl-Jeon/traj_gen_vis and https://www.youtube.com/watch?v=_JSwXBwYRl8.