Reviewerly/exHarmony huggingface dataset:
Attend, Copy, Parse End-to-end Information Extraction from Documents

Document information extraction tasks performed by humans create data consisting of a PDF or document image input, and extracted string outputs. This end-to-end data is naturally consumed and produced when performing the task because it is valuable in and of itself. It is naturally available, at no additional cost. Unfortunately, state-of-the-art word classification methods for information extraction cannot use this data, instead requiring word-level labels which are expensive to create and consequently not available for many real life tasks. In this paper we propose the Attend, Copy, Parse architecture, a deep neural network model that can be trained directly on end-to-end data, bypassing the need for word-level labels. We evaluate the proposed architecture on a large diverse set of invoices, and outperform a state-of-the-art production system based on word classification. We believe our proposed architecture can be used on many real life information extraction tasks where word classification cannot be used due to a lack of the required word-level labels.
Market basket analysis with association rules

This study was conducted in order to make a Market Basket Analysis by using Association Rules. The data used in the study are the sales data of any supermarket received from the Vancouver Island University website. Data were analyzed in the Weka program using a data set containing 225 different products. Apriori and FP Growth, which are Association Rules algorithms, were tried in order. Since the data set is categorical, the Apriori algorithm did not yield any results. Therefore, the FP Growth algorithm was used and the top 10 rules were given according to the conviction value. The best rule accordingly; a customer who buys Milk, Sweet Relish and Pepperoni Pizza (Frozen) also gets eggs. Best rule with 21.06 Conviction and 1 (100%) confidence values are this rule. 24 customers who received these 3 products in the dataset received eggs. Similarly, also other rules were interpreted in this study. As a result, product placement in the supermarket can be made according to these rules. Thus, sales of these products will increase and supermarket revenue will increase directly.
DE-CapsNet: A Diverse Enhanced Capsule Network with Disperse Dynamic Routing

Capsule Network (CapsNet) is a methodology with good prospects in visual tasks, since it can keep a stronger relationship of spatial information than Convolutional Neural Networks (CNNs). However, the current Capsule Network do not provide performance as expected on several benchmark data sets with complex data and backgrounds. Inspired by the multiple capsules of Diverse Capsule Network (DCNet++) and the Spatial Group-wise Enhance (SGE) mechanism, we propose the Diverse Enhanced Capsule Network (DE-CapsNet), a hierarchical architecture which uses residual convolutional layers and the position-wise dot product to build diverse enhanced primary capsules with various scales of images for complex data. The architecture adopts the Sigmoid function in a dynamic routing algorithm to get a more uniform distribution of routing coefficients which obviously distinguishes the assignment probabilities between capsules. DE-CapsNet achieved state-of-the-art accuracy on Canadian Institute For Advanced Research (CIFAR-10) in the Capsule Network field and provided better performance than the ensemble of seven CapsNets on Fashion-Modified National Institue of Standards and Technology database (F-MNIST) while achieving a 50.3% reduction in the number of parameters.
Combining deep learning with token selection for patient phenotyping from electronic health records

Artificial intelligence provides the opportunity to reveal important information buried in large amounts of complex data. Electronic health records (eHRs) are a source of such big data that provide a multitude of health related clinical information about patients. However, text data from eHRs, e.g., discharge summary notes, are challenging in their analysis because these notes are free-form texts and the writing formats and styles vary considerably between different records. For this reason, in this paper we study deep learning neural networks in combination with natural language processing to analyze text data from clinical discharge summaries. We provide a detail analysis of patient phenotyping, i.e., the automatic prediction of ten patient disorders, by investigating the influence of network architectures, sample sizes and information content of tokens. Importantly, for patients suffering from Chronic Pain, the disorder that is the most difficult one to classify, we find the largest performance gain for a combined word- and sentence-level input convolutional neural network (ws-CNN). As a general result, we find that the combination of data quality and data quantity of the text data is playing a crucial role for using more complex network architectures that improve significantly beyond a word-level input CNN model. From our investigations of learning curves and token selection mechanisms, we conclude that for such a transition one requires larger sample sizes because the amount of information per sample is quite small and only carried by few tokens and token categories. Interestingly, we found that the token frequency in the eHRs follow a Zipf law and we utilized this behavior to investigate the information content of tokens by defining a token selection mechanism. The latter addresses also issues of explainable AI.
The Arm Neoverse N1 Platform: Building Blocks for the Next-Gen Cloud-to-Edge Infrastructure SoC

Recent years have seen an explosion of demand for high-performance, high-efficiency compute available at scale. This demand has skyrocketed with the move to the public cloud and 5G networking, where compute nodes must operate within strict latency constraints and power budgets. The Neoverse N1 platform is Arm's latest high end offering from a scalable portfolio of IP for high performance and energy efficient machines.
Optimizing Deep Learning Inference on Embedded Systems Through Adaptive Model Selection

Deep neural networks (DNNs) are becoming a key enabling technique for many application domains. However, on-device inference on battery-powered, resource-constrained embedding systems is often infeasible due to prohibitively long inferencing time and resource requirements of many DNNs. Offloading computation into the cloud is often unacceptable due to privacy concerns, high latency, or the lack of connectivity. Although compression algorithms often succeed in reducing inferencing times, they come at the cost of reduced accuracy. This article presents a new, alternative approach to enable efficient execution of DNNs on embedded devices. Our approach dynamically determines which DNN to use for a given input by considering the desired accuracy and inference time. It employs machine learning to develop a low-cost predictive model to quickly select a pre-trained DNN to use for a given input and the optimization constraint. We achieve this first by offline training a predictive model and then using the learned model to select a DNN model to use for new, unseen inputs. We apply our approach to two representative DNN domains: image classification and machine translation. We evaluate our approach on a Jetson TX2 embedded deep learning platform and consider a range of influential DNN models including convolutional and recurrent neural networks. For image classification, we achieve a 1.8x reduction in inference time with a 7.52% improvement in accuracy over the most capable single DNN model. For machine translation, we achieve a 1.34x reduction in inference time over the most capable single model with little impact on the quality of translation.
DGaze: CNN-Based Gaze Prediction in Dynamic Scenes

We conduct novel analyses of users' gaze behaviors in dynamic virtual scenes and, based on our analyses, we present a novel CNN-based model called DGaze for gaze prediction in HMD-based applications. We first collect 43 users' eye tracking data in 5 dynamic scenes under free-viewing conditions. Next, we perform statistical analysis of our data and observe that dynamic object positions, head rotation velocities, and salient regions are correlated with users' gaze positions. Based on our analysis, we present a CNN-based model (DGaze) that combines object position sequence, head velocity sequence, and saliency features to predict users' gaze positions. Our model can be applied to predict not only realtime gaze positions but also gaze positions in the near future and can achieve better performance than prior method. In terms of realtime prediction, DGaze achieves a 22.0% improvement over prior method in dynamic scenes and obtains an improvement of 9.5% in static scenes, based on using the angular distance as the evaluation metric. We also propose a variant of our model called DGaze_ET that can be used to predict future gaze positions with higher precision by combining accurate past gaze data gathered using an eye tracker. We further analyze our CNN architecture and verify the effectiveness of each component in our model. We apply DGaze to gaze-contingent rendering and a game, and also present the evaluation results from a user study.
Object Tracking in Satellite Videos Based on Convolutional Regression Network With Appearance and Motion Features

Object tracking is one of the most important components in numerous applications of computer vision.Remote sensing videos provided by commercial satellites make it possible to extend this topic into the earth observation domain.In satellite videos, typical moving targets like vehicles and planes only cover a small area of pixels, and they could easily be confused with surrounding complex ground scenes.Similar objects nearby in satellite videos can hardly be differed by appearance details due to the resolution constraint.Thus, tracking drift caused by distractions is also a thorny problem.Facing challenges, traditional tracking methods such as correlation filters with hand-crafted visual features achieve unsatisfactory results in satellite videos.Methods based on deep neural networks have demonstrated their superiority in various ordinary visual tracking benchmarks, but their results on satellite videos remain unexplored.In this article, deep learning technologies are applied to object tracking in satellite videos for better performance.A simple regression network is used to combine a regression model with convolutional layers and a gradient descent algorithm.The regression network fully exploits the abundant background context to learn a robust tracker.Instead of handcrafted features, both appearance features and motion features, which are extracted by pretrained deep neural networks, are used for accurate object tracking.In cases when the tracker encounters ambiguous appearance information, the motion features could provide complementary and discriminative information to improve tracking performances.Experimental results on various satellite videos show that the proposed method achieves better tracking performance than other state-of-the-arts.
Depth upsampling based on deep edge-aware learning

Depth map upsampling will unavoidably smoothen the edges leading to blurry results on the depth boundaries, especially at large upscaling factors. Given that edges represent the most important cue in addressing the task of depth upsampling, we propose a novel depth upsampling framework based on deep edge-aware learning. Unlike existing CNN-based approaches that directly predict depth values from low resolution (LR) depth input, our framework firstly learns edge information of depth boundaries from the known LR depth map and its corresponding high resolution (HR) color image as reconstruction cues. Then, two depth restoration modules, i.e., a fast depth filling strategy and a cascaded restoration network, are proposed to recover HR depth map by leveraging the predicted edge map and the HR color image. Extensive comparisons on both edge inference and depth upsampling under noisy and noiseless cases demonstrate the superiority of the proposed approaches.
Object Detection and Count of Objects in Image using Tensor Flow Object Detection API

Object Detection is widely utilized in several applications such as detecting vehicles, face detection, autonomous vehicles and pedestrians on streets. TensorFlow's Object Detection API is a powerful tool that can quickly enable anyone to build and deploy powerful image recognition software. Object detection not solely includes classifying and recognizing objects in an image however additionally localizes those objects and attracts bounding boxes around them. This paper mostly focuses on detecting harmful objects like threatening objects. To ease object detection for threatening objects, we have got Tensor flow Object Detection API to train model and we have used Faster R-CNN algorithm for implementation. The model is built on two classes of threatening Objects. The model is evaluated on test data for the two classes of detecting threatening objects.
Face Recognition using FaceNet (Survey, Performance Test, and Comparison)

Face recognition that is technology used for recognizing human faces based on certain patterns and re-detect faces in various conditions. Face recognition is currently becoming popular to be applied in various ways, especially in security systems. Various methods of face recognition have been proposed in researches and increased accuracy is the main goal in the development of face recognition methods. FaceNet is one of the new methods in face recognition technology. This method is based on a deep convolutional network and triplet loss training to carry out training data, but the training process requires complex computing and a long time. By integrating the Tensorflow learning machine and pre-trained model, the training time needed is much shorter. This research aims to conduct surveys, test performance, and compare the accuracy of the results of recognizing the face of the FaceNet method with various other methods that have been developed previously. Implementation of the FaceNet method in research using two types of pre-trained models, namely CASIA-WebFace and VGGFace2, and tested on various data sets of standard face images that have been widely used before. From the results of this research experiment, FaceNet showed excellent results and was superior to other methods. By using VGGFace2 pre-trained models, FaceNet is able to touch 100% accuracy on YALE, JAFFE, AT & T datasets, Essex faces95, Essex grimace, 99.375% for Essex faces94 dataset and the worst 77.67% for the faces96 dataset.
DeepQoE: A Multimodal Learning Framework for Video Quality of Experience (QoE) Prediction

Recently, many models have been developed to predict video Quality of Experience (QoE), yet the applicability of these models still faces significant challenges. Firstly, many models rely on features that are unique to a specific dataset and thus lack the capability to generalize. Due to the intricate interactions among these features, a unified representation that is independent of datasets with different modalities is needed. Secondly, existing models often lack the configurability to perform both classification and regression tasks. Thirdly, the sample size of the available datasets to develop these models is often very small, and the impact of limited data on the performance of QoE models has not been adequately addressed. To address these issues, in this work we develop a novel and end-to-end framework termed as DeepQoE. The proposed framework first uses a combination of deep learning techniques, such as word embedding and 3D convolutional neural network (C3D), to extract generalized features. Next, these features are combined and fed into a neural network for representation learning. A learned representation will then serve as input for classification or regression tasks. We evaluate the performance of DeepQoE with three datasets. The results show that for small datasets (e.g., WHU-MVQoE2016 and Live-Netflix Video Database), the performance of state-of-the-art machine learning algorithms is greatly improved by using the QoE representation from DeepQoE (e.g., 35.71% to 44.82%); while for the large dataset (e.g., VideoSet), our DeepQoE framework achieves significant performance improvement in comparison to the best baseline method (90.94% vs. 82.84%). In addition to the much improved performance, DeepQoE has the flexibility to fit different datasets, to learn QoE representation, and to perform both classification and regression problems. We also develop a DeepQoE based adaptive bitrate streaming (ABR) system to verify that our framework can be easily applied to multimedia communication service. The software package of the DeepQoE framework has been released to facilitate the current research on QoE.
Dataset of segmented nuclei in hematoxylin and eosin stained histopathology images of ten cancer types

Abstract The distribution and appearance of nuclei are essential markers for the diagnosis and study of cancer. Despite the importance of nuclear morphology, there is a lack of large scale, accurate, publicly accessible nucleus segmentation data. To address this, we developed an analysis pipeline that segments nuclei in whole slide tissue images from multiple cancer types with a quality control process. We have generated nucleus segmentation results in 5,060 Whole Slide Tissue images from 10 cancer types in The Cancer Genome Atlas. One key component of our work is that we carried out a multi-level quality control process (WSI-level and image patch-level), to evaluate the quality of our segmentation results. The image patch-level quality control used manual segmentation ground truth data from 1,356 sampled image patches. The datasets we publish in this work consist of roughly 5 billion quality controlled nuclei from more than 5,060 TCGA WSIs from 10 different TCGA cancer types and 1,356 manually segmented TCGA image patches from the same 10 cancer types plus additional 4 cancer types.
Analyzing ASR Pretraining for Low-Resource Speech-to-Text Translation

Previous work has shown that for low-resource source languages, automatic speech-to-text translation (AST) can be improved by pre-training an end-to-end model on automatic speech recognition (ASR) data from a high-resource language. However, it is not clear what factors - e.g., language relatedness or size of the pretraining data - yield the biggest improvements, or whether pretraining can be effectively combined with other methods such as data augmentation. Here, we experiment with pretraining on datasets of varying sizes, including languages related and unrelated to the AST source language. We find that the best predictor of final AST performance is the word error rate of the pretrained ASR model, and that differences in ASR/AST performance correlate with how phonetic information is encoded in the later RNN layers of our model. We also show that pretraining and data augmentation yield complementary benefits for AST.
A self‐adaptive synthetic over‐sampling technique for imbalanced classification

Traditionally, in supervised machine learning, (a significant) part of the available data (usually 50%-80%) is used for training and the rest—for validation. In many problems, however, the data are highly imbalanced in regard to different classes or does not have good coverage of the feasible data space which, in turn, creates problems in validation and usage phase. In this paper, we propose a technique for synthesizing feasible and likely data to help balance the classes as well as to boost the performance in terms of confusion matrix as well as overall. The idea, in a nutshell, is to synthesize data samples in close vicinity to the actual data samples specifically for the less represented (minority) classes. This has also implications to the so-called fairness of machine learning. In this paper, we propose a specific method for synthesizing data in a way to balance the classes and boost the performance, especially of the minority classes. It is generic and can be applied to different base algorithms, for example, support vector machines, k-nearest neighbour classifiers deep neural, rule-based classifiers, decision trees, and so forth. The results demonstrated that (a) a significantly more balanced (and fair) classification results can be achieved and (b) that the overall performance as well as the performance per class measured by confusion matrix can be boosted. In addition, this approach can be very valuable for the cases when the number of actual available labelled data is small which itself is one of the problems of the contemporary machine learning.
Private Information Retrieval Through Wiretap Channel II: Privacy Meets Security

We consider the problem of private information retrieval through wiretap channel II (PIR-WTC-II). In PIR-WTC-II, a user wants to retrieve a single message (file) privately out of M messages, which are stored in N replicated and non-communicating databases. An external eavesdropper observes a fraction μ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</sub> (of its choice) of the traffic exchanged between the nth database and the user. In addition to the privacy constraint, the databases should encode the returned answer strings such that the eavesdropper learns absolutely nothing about the contents of the databases. We aim at characterizing the capacity of the PIR-WTC-II under the combined privacy and security constraints. We obtain a general upper bound for the problem in the form of a max-min optimization problem, which extends the converse proof of the PIR problem under asymmetric traffic constraints. We propose an achievability scheme that satisfies the security constraint by encoding a secret key, which is generated securely at each database, into an artificial noise vector using an MDS code. The user and the databases operate at one of the corner points of the achievable scheme for the PIR under asymmetric traffic constraints such that the retrieval rate is maximized under the imposed security constraint. The upper bound and the lower bound match for the case of M = 2 and M = 3 messages, for any N, and any μ = (μ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> , · · · , μ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</sub> ).