Reviewerly/exHarmony huggingface dataset:
Cross View Fusion for 3D Human Pose Estimation

We present an approach to recover absolute 3D human poses from multi-view images by incorporating multi-view geometric priors in our model. It consists of two separate steps: (1) estimating the 2D poses in multi-view images and (2) recovering the 3D poses from the multi-view 2D poses. First, we introduce a cross-view fusion scheme into CNN to jointly estimate 2D poses for multiple views. Consequently, the 2D pose estimation for each view already benefits from other views. Second, we present a recursive Pictorial Structure Model to recover the 3D pose from the multi-view 2D poses. It gradually improves the accuracy of 3D pose with affordable computational cost. We test our method on two public datasets H36M and Total Capture. The Mean Per Joint Position Errors on the two datasets are 26mm and 29mm, which outperforms the state-of-the-arts remarkably (26mm vs 52mm, 29mm vs 35mm).
Auto-FPN: Automatic Network Architecture Adaptation for Object Detection Beyond Classification

Neural architecture search (NAS) has shown great potential in automating the manual process of designing a good CNN architecture for image classification. In this paper, we study NAS for object detection, a core computer vision task that classifies and localizes object instances in an image. Existing works focus on transferring the searched architecture from classification task (ImageNet) to the detector backbone, while the rest of the architecture of the detector remains unchanged. However, this pipeline is not task-specific or data-oriented network search which cannot guarantee optimal adaptation to any dataset. Therefore, we propose an architecture search framework named Auto-FPN specifically designed for detection beyond simply searching a classification backbone. Specifically, we propose two auto search modules for detection: Auto-fusion to search a better fusion of the multi-level features; Auto-head to search a better structure for classification and bounding-box(bbox) regression. Instead of searching for one repeatable cell structure, we relax the constraint and allow different cells. The search space of both modules covers many popular designs of detectors and allows efficient gradient-based architecture search with resource constraint (2 days for COCO on 8 GPU cards). Extensive experiments on Pascal VOC, COCO, BDD, VisualGenome and ADE demonstrate the effectiveness of the proposed method, e.g. achieving around 5% improvement than FPN in terms of mAP while requiring around 50% fewer parameters on the searched modules.
Two-Stage Cascaded U-Net: 1st Place Solution to BraTS Challenge 2019 Segmentation Task

In this paper, we devise a novel two-stage cascaded U-Net to segment the substructures of brain tumors from coarse to fine. The network is trained end-to-end on the Multimodal Brain Tumor Segmentation Challenge (BraTS) 2019 training dataset. Experimental results on the testing set demonstrate that the proposed method achieved average Dice scores of 0.83267, 0.88796 and 0.83697, as well as Hausdorff distances (95%) of 2.65056, 4.61809 and 4.13071, for the enhancing tumor, whole tumor and tumor core, respectively. The approach won the 1st place in the BraTS 2019 challenge segmentation task, with more than 70 teams participating in the challenge.
Spatio-Temporal Graph Transformer Networks for Pedestrian Trajectory Prediction

Understanding crowd motion dynamics is critical to real-world applications, e.g., surveillance systems and autonomous driving. This is challenging because it requires effectively modeling the socially aware crowd spatial interaction and complex temporal dependencies. We believe attention is the most important factor for trajectory prediction. In this paper, we present STAR, a Spatio-Temporal grAph tRansformer framework, which tackles trajectory prediction by only attention mechanisms. STAR models intra-graph crowd interaction by TGConv, a novel Transformer-based graph convolution mechanism. The inter-graph temporal dependencies are modeled by separate temporal Transformers. STAR captures complex spatio-temporal interactions by interleaving between spatial and temporal Transformers. To calibrate the temporal prediction for the long-lasting effect of disappeared pedestrians, we introduce a read-writable external memory module, consistently being updated by the temporal Transformer. We show that with only attention mechanism, STAR achieves the state-of-the-art performance on 5 commonly used real-world pedestrian prediction datasets (code available at https://github.com/Majiker/STAR).
Efficient symmetry-preserving state preparation circuits for the variational quantum eigensolver algorithm

The variational quantum eigensolver is one of the most promising approaches for performing chemistry simulations using noisy intermediate-scale quantum (NISQ) processors. The efficiency of this algorithm depends crucially on the ability to prepare multi-qubit trial states on the quantum processor that either include, or at least closely approximate, the actual energy eigenstates of the problem being simulated while avoiding states that have little overlap with them. Symmetries play a central role in determining the best trial states. Here, we present efficient state preparation circuits that respect particle number, total spin, spin projection, and time-reversal symmetries. These circuits contain the minimal number of variational parameters needed to fully span the appropriate symmetry subspace dictated by the chemistry problem while avoiding all irrelevant sectors of Hilbert space. We show how to construct these circuits for arbitrary numbers of orbitals, electrons, and spin quantum numbers, and we provide explicit decompositions and gate counts in terms of standard gate sets in each case. We test our circuits in quantum simulations of the $H_2$ and $LiH$ molecules and find that they outperform standard state preparation methods in terms of both accuracy and circuit depth.
PODNet: Pooled Outputs Distillation for Small-Tasks Incremental Learning

Lifelong learning has attracted much attention, but existing works still struggle to fight catastrophic forgetting and accumulate knowledge over long stretches of incremental learning. In this work, we propose PODNet, a model inspired by representation learning. By carefully balancing the compromise between remembering the old classes and learning new ones, PODNet fights catastrophic forgetting, even over very long runs of small incremental tasks – a setting so far unexplored by current works. PODNet innovates on existing art with an efficient spatial-based distillation-loss applied throughout the model and a representation comprising multiple proxy vectors for each class. We validate those innovations thoroughly, comparing PODNet with three state-of-the-art models on three datasets: CIFAR100, ImageNet100, and ImageNet1000. Our results showcase a significant advantage of PODNet over existing art, with accuracy gains of 12.10, 6.51, and 2.85 percentage points, respectively.
SySeVR: A Framework for Using Deep Learning to Detect Software Vulnerabilities

The detection of software vulnerabilities (or vulnerabilities for short) is an important problem that has yet to be tackled, as manifested by the many vulnerabilities reported on a daily basis. This calls for machine learning methods for vulnerability detection. Deep learning is attractive for this purpose because it alleviates the requirement to manually define features. Despite the tremendous success of deep learning in other application domains, its applicability to vulnerability detection is not systematically understood. In order to fill this void, we propose the first systematic framework for using deep learning to detect vulnerabilities in C/C++ programs with source code. The framework, dubbed Syntax-based, Semantics-based, and Vector Representations (SySeVR), focuses on obtaining program representations that can accommodate syntax and semantic information pertinent to vulnerabilities. Our experiments with 4 software products demonstrate the usefulness of the framework: we detect 15 vulnerabilities that are not reported in the National Vulnerability Database. Among these 15 vulnerabilities, 7 are unknown and have been reported to the vendors, and the other 8 have been "silently" patched by the vendors when releasing newer versions of the pertinent software products.
Deep Learning in Image Classification using Residual Network (ResNet) Variants for Detection of Colorectal Cancer

This paper investigates a deep learning method in image classification for the detection of colorectal cancer with ResNet architecture. The exceptional performance of a deep learning classification incites scholars to implement them in medical images. In this study, we trained ResNet-18 and ResNet-50 on colon glands images. The models trained to distinguish colorectal cancer into benign and malignant. We assessed our prototypes on three varieties of testing data (20%, 25%, and 40% of whole datasets). The empirical outcomes confirm that the application of ResNet-50 provides the most reliable performance for accuracy, sensitivity, and specificity value than ResNet-18 in three kinds of testing data. Upon three test assortments, we perceive the best performance value on 20% and 25% test sets with a classification accuracy of above 80%, the sensitivity of above 87%, and the specificity of above 83%. In this research, a deep learning method demonstrates the profoundly reliable and reproducible outcomes for biomedical image analysis.
One-step quantum secure direct communication

Quantum secure direct communication (QSDC) attracts much attention for it can transmit secret messages directly without sharing a key. In this article, we propose a one-step QSDC protocol, which only requires to distribute polarization-spatial-mode hyperentanglement for one round. In this QSDC protocol, the eavesdropper cannot obtain any message, so that this protocol is unconditionally secure in principle. This protocol is a two-way quantum communication and has high capacity for it can transmit two bits of secret messages with one pair of hyperentanglement. With entanglement fidelities of both polarization and spatial-mode degrees of freedom being 0.98, the maximal communication distance of this one-step QSDC can reach about 216 km. QSDC can also be used to generate the key. In this regard, the key generation rate is estimated about 2.5 times of that in the entanglement-based QKD with the communication distance of 150 km. With the help of future quantum repeaters, this QSDC protocol can provide unconditionally secure communication over arbitrarily long distance.
Fully Supervised Speaker Diarization

In this paper, we propose a fully supervised speaker diarization approach, named unbounded interleaved-state recurrent neural networks (UIS-RNN). Given extracted speaker-discriminative embeddings (a.k.a. d-vectors) from input utterances, each individual speaker is modeled by a parameter-sharing RNN, while the RNN states for different speakers interleave in the time domain. This RNN is naturally integrated with a distance-dependent Chinese restaurant process (ddCRP) to accommodate an unknown number of speakers. Our system is fully supervised and is able to learn from examples where time-stamped speaker labels are annotated. We achieved a 7.6% diarization error rate on NIST SRE 2000 CALLHOME, which is better than the state-of-the-art method using spectral clustering. Moreover, our method decodes in an online fashion while most state-of-the-art systems rely on offline clustering.
An artificial neuron implemented on an actual quantum processor

Artificial neural networks are the heart of machine learning algorithms and artificial intelligence protocols. Historically, the simplest implementation of an artificial neuron traces back to the classical Rosenblatt's `perceptron', but its long term practical applications may be hindered by the fast scaling up of computational complexity, especially relevant for the training of multilayered perceptron networks. Here we introduce a quantum information-based algorithm implementing the quantum computer version of a perceptron, which shows exponential advantage in encoding resources over alternative realizations. We experimentally test a few qubits version of this model on an actual small-scale quantum processor, which gives remarkably good answers against the expected results. We show that this quantum model of a perceptron can be used as an elementary nonlinear classifier of simple patterns, as a first step towards practical training of artificial quantum neural networks to be efficiently implemented on near-term quantum processing hardware.
Energy consumption of cryptocurrency mining: A study of electricity consumption in mining cryptocurrencies

Cryptocurrency is a relatively new combination of cryptology and currency in financial areas and is increasingly frequently used worldwide. Blockchain applications are expected to reshape the renewable energy market. However, there is a lack of studies covering the power usage of digital currencies. Therefore, this study ran experiments on mining efficiency of nine kinds of cryptocurrencies and ten algorithms. A comparison of statistical analysis of data in a benchmark and experiment results of Monero mining was conducted. Thereafter, this study provided an estimation of global electricity consumption of the Monero mining activity. The results indicated that the hashing algorithm mainly determines the mining efficiency. Data analysis and experiments and estimated Monero mining electricity consumption in the world and its carbon emission in China as a case study. In 2018, Monero mining may consume 645.62 GWh of electricity in the world after its hard fork. The Monero mining in China may consume 30.34 GWh and contribute a carbon emission of 19.12–19.42 thousand tons from April to December in 2018. Although cryptocurrency mining and blockchain technology are promising, their influence on energy conversation and sustainable development should be further studied.
Image–text sentiment analysis via deep multimodal attentive fusion

Sentiment analysis of social media data is crucial to understand people’s position, attitude, and opinion toward a certain event, which has many applications such as election prediction and product evaluation. Though great effort has been devoted to the single modality (image or text), less effort is paid to the joint analysis of multimodal data in social media. Most of the existing methods for multimodal sentiment analysis simply combine different data modalities, which results in dissatisfying performance on sentiment classification. In this paper, we propose a novel image–text sentiment analysis model, i.e., Deep Multimodal Attentive Fusion (DMAF), to exploit the discriminative features and the internal correlation between visual and semantic contents with a mixed fusion framework for sentiment analysis. Specifically, to automatically focus on discriminative regions and important words which are most related to the sentiment, two separate unimodal attention models are proposed to learn effective emotion classifiers for visual and textual modality respectively. Then, an intermediate fusion-based multimodal attention model is proposed to exploit the internal correlation between visual and textual features for joint sentiment classification. Finally, a late fusion scheme is applied to combine the three attention models for sentiment prediction. Extensive experiments are conducted to demonstrate the effectiveness of our approach on both weakly labeled and manually labeled datasets.
Web AR: A Promising Future for Mobile Augmented Reality—State of the Art, Challenges, and Insights

Mobile augmented reality (Mobile AR) is gaining increasing attention from both academia and industry. Hardware-based Mobile AR and App-based Mobile AR are the two dominant platforms for Mobile AR applications. However, hardware-based Mobile AR implementation is known to be costly and lacks flexibility, while the App-based one requires additional downloading and installation in advance and is inconvenient for cross-platform deployment. In comparison, Web-based AR (Web AR) implementation can provide a pervasive Mobile AR experience to users thanks to the many successful deployments of the Web as a lightweight and cross-platform service provisioning platform. Furthermore, the emergence of 5G mobile communication networks has the potential to enhance the communication efficiency of Mobile AR dense computing in the Web-based approach. We conjecture that Web AR will deliver an innovative technology to enrich our ways of interacting with the physical (and cyber) world around us. This paper reviews the state-of-the-art technology and existing implementations of Mobile AR, as well as enabling technologies and challenges when AR meets the Web. Furthermore, we elaborate on the different potential Web AR provisioning approaches, especially the adaptive and scalable collaborative distributed solution which adopts the osmotic computing paradigm to provide Web AR services. We conclude this paper with the discussions of open challenges and research directions under current 3G/4G networks and the future 5G networks. We hope that this paper will help researchers and developers to gain a better understanding of the state of the research and development in Web AR and at the same time stimulate more research interest and effort on delivering life-enriching Web AR experiences to the fast-growing mobile and wireless business and consumer industry of the 21st century.
Artificial intelligence in digital pathology: a roadmap to routine use in clinical practice

Abstract The use of artificial intelligence will transform clinical practice over the next decade and the early impact of this will likely be the integration of image analysis and machine learning into routine histopathology. In the UK and around the world, a digital revolution is transforming the reporting practice of diagnostic histopathology and this has sparked a proliferation of image analysis software tools. While this is an exciting development that could discover novel predictive clinical information and potentially address international pathology workforce shortages, there is a clear need for a robust and evidence‐based framework in which to develop these new tools in a collaborative manner that meets regulatory approval. With these issues in mind, the NCRI Cellular Molecular Pathology (CM‐Path) initiative and the British In Vitro Diagnostics Association (BIVDA) have set out a roadmap to help academia, industry, and clinicians develop new software tools to the point of approved clinical use. © 2019 Pathological Society of Great Britain and Ireland. Published by John Wiley &amp; Sons, Ltd.
Neural Rerendering in the Wild

We explore total scene capture --- recording, modeling, and rerendering a scene under varying appearance such as season and time of day. Starting from Internet photos of a tourist landmark, we apply traditional 3D reconstruction to register the photos and approximate the scene as a point cloud. For each photo, we render the scene points into a deep framebuffer, and train a deep neural network to learn the mapping of these initial renderings to the actual photos. This rerendering network also takes as input a latent appearance vector and a semantic mask indicating the location of transient objects like pedestrians. The model is evaluated on several datasets of publicly available images spanning a broad range of illumination conditions. We create short videos that demonstrate realistic manipulation of the image viewpoint, appearance, and semantic labels. We also compare results to prior work on scene reconstruction from Internet photos.
Classification of mammogram for early detection of breast cancer using SVM classifier and Hough transform

Breast cancer is one of the significant health problems in the world. If these abnormalities in breast cancer are detected early there is a maximum chance for recovery. For this early prediction we can go for mammography. It is one of the most effective and commonly used method for detecting and screening breast cancer. This paper presents classification of mammograms using feature extracted using Hough transform. Hough transform is a two dimensional transform. It is used to isolate feature of particular shape in an image. Miniaturized scale characterization and masses are the two most vital markers of threat, and their mechanized identification is exceptionally important for early breast cancer diagnosis. Since masses are regularly undefined from the encompassing parenchymal, computerized mass location and arrangement is significantly additionally difficult. This paper talks about the strategies for classification and feature extraction. Here, Hough transform is used to detect features of mammograms image and it is classified using SVM. The classification accuracy is more by the use of SVM classifier. This method is tested on 95 mammograms images collected and classified using SVM. From the result it shows that the proposed method is effectively classify the abnormal classes of mammograms.
Multi-Objective Evolutionary Federated Learning

Federated learning is an emerging technique used to prevent the leakage of private information. Unlike centralized learning that needs to collect data from users and store them collectively on a cloud server, federated learning makes it possible to learn a global model while the data are distributed on the users' devices. However, compared with the traditional centralized approach, the federated setting consumes considerable communication resources of the clients, which is indispensable for updating global models and prevents this technique from being widely used. In this paper, we aim to optimize the structure of the neural network models in federated learning using a multi-objective evolutionary algorithm to simultaneously minimize the communication costs and the global model test errors. A scalable method for encoding network connectivity is adapted to federated learning to enhance the efficiency in evolving deep neural networks. Experimental results on both multilayer perceptrons and convolutional neural networks indicate that the proposed optimization method is able to find optimized neural network models that can not only significantly reduce communication costs but also improve the learning performance of federated learning compared with the standard fully connected neural networks.
A Novel Sigmoid-Function-Based Adaptive Weighted Particle Swarm Optimizer

In this paper, a novel particle swarm optimization (PSO) algorithm is put forward where a sigmoid-function-based weighting strategy is developed to adaptively adjust the acceleration coefficients. The newly proposed adaptive weighting strategy takes into account both the distances from the particle to the global best position and from the particle to its personal best position, thereby having the distinguishing feature of enhancing the convergence rate. Inspired by the activation function of neural networks, the new strategy is employed to update the acceleration coefficients by using the sigmoid function. The search capability of the developed adaptive weighting PSO (AWPSO) algorithm is comprehensively evaluated via eight well-known benchmark functions including both the unimodal and multimodal cases. The experimental results demonstrate that the designed AWPSO algorithm substantially improves the convergence rate of the particle swarm optimizer and also outperforms some currently popular PSO algorithms.