Reviewerly/exHarmony huggingface dataset:
A comprehensive performance analysis of Apache Hadoop and Apache Spark for large scale data sets using HiBench

Abstract Big Data analytics for storing, processing, and analyzing large-scale datasets has become an essential tool for the industry. The advent of distributed computing frameworks such as Hadoop and Spark offers efficient solutions to analyze vast amounts of data. Due to the application programming interface (API) availability and its performance, Spark becomes very popular, even more popular than the MapReduce framework. Both these frameworks have more than 150 parameters, and the combination of these parameters has a massive impact on cluster performance. The default system parameters help the system administrator deploy their system applications without much effort, and they can measure their specific cluster performance with factory-set parameters. However, an open question remains: can new parameter selection improve cluster performance for large datasets? In this regard, this study investigates the most impacting parameters, under resource utilization, input splits, and shuffle, to compare the performance between Hadoop and Spark, using an implemented cluster in our laboratory. We used a trial-and-error approach for tuning these parameters based on a large number of experiments. In order to evaluate the frameworks of comparative analysis, we select two workloads: WordCount and TeraSort. The performance metrics are carried out based on three criteria: execution time, throughput, and speedup. Our experimental results revealed that both system performances heavily depends on input data size and correct parameter selection. The analysis of the results shows that Spark has better performance as compared to Hadoop when data sets are small, achieving up to two times speedup in WordCount workloads and up to 14 times in TeraSort workloads when default parameter values are reconfigured.
Deep learning detects genetic alterations in cancer histology generated by adversarial networks

Deep learning can detect microsatellite instability (MSI) from routine histology images in colorectal cancer (CRC). However, ethical and legal barriers impede sharing of images and genetic data, hampering development of new algorithms for detection of MSI and other biomarkers. We hypothesized that histology images synthesized by conditional generative adversarial networks (CGANs) retain information about genetic alterations. To test this, we developed a 'histology CGAN' which was trained on 256 patients (training cohort 1) and 1457 patients (training cohort 2). The CGAN synthesized 10 000 synthetic MSI and non-MSI images which contained a range of tissue types and were deemed realistic by trained observers in a blinded study. Subsequently, we trained a deep learning detector of MSI on real or synthetic images and evaluated the performance of MSI detection in a held-out set of 142 patients. When trained on real images from training cohort 1, this system achieved an area under the receiver operating curve (AUROC) of 0.742 [0.681, 0.854]. Training on the larger cohort 2 only marginally improved the AUROC to 0.757 [0.707, 0.869]. Training on purely synthetic data resulted in an AUROC of 0.743 [0.658, 0.801]. Training on both real and synthetic data further increased AUROC to 0.777 [0.715, 0.821]. We conclude that synthetic histology images retain information reflecting underlying genetic alterations in colorectal cancer. Using synthetic instead of real images to train deep learning systems yields non-inferior classifiers. This approach can be used to create large shareable data sets or to augment small data sets with rare molecular features. © 2021 The Authors. The Journal of Pathology published by John Wiley & Sons, Ltd. on behalf of The Pathological Society of Great Britain and Ireland.
Hidden Variables in Deep Learning Digital Pathology and Their Potential to Cause Batch Effects: Prediction Model Study

Background An increasing number of studies within digital pathology show the potential of artificial intelligence (AI) to diagnose cancer using histological whole slide images, which requires large and diverse data sets. While diversification may result in more generalizable AI-based systems, it can also introduce hidden variables. If neural networks are able to distinguish/learn hidden variables, these variables can introduce batch effects that compromise the accuracy of classification systems. Objective The objective of the study was to analyze the learnability of an exemplary selection of hidden variables (patient age, slide preparation date, slide origin, and scanner type) that are commonly found in whole slide image data sets in digital pathology and could create batch effects. Methods We trained four separate convolutional neural networks (CNNs) to learn four variables using a data set of digitized whole slide melanoma images from five different institutes. For robustness, each CNN training and evaluation run was repeated multiple times, and a variable was only considered learnable if the lower bound of the 95% confidence interval of its mean balanced accuracy was above 50.0%. Results A mean balanced accuracy above 50.0% was achieved for all four tasks, even when considering the lower bound of the 95% confidence interval. Performance between tasks showed wide variation, ranging from 56.1% (slide preparation date) to 100% (slide origin). Conclusions Because all of the analyzed hidden variables are learnable, they have the potential to create batch effects in dermatopathology data sets, which negatively affect AI-based classification systems. Practitioners should be aware of these and similar pitfalls when developing and evaluating such systems and address these and potentially other batch effect variables in their data sets through sufficient data set stratification.
Arbitrary-view human action recognition via novel-view action generation

Arbitrary-view human action recognition is still a big challenge due to the view changes. A possible solution is to enlarge the view range of action samples in the training set. Therefore, we propose a Two-Branch Novel-View action Generation approach based on auxiliary conditional GAN, which generates a novel-view action sample for arbitrary-view human action recognition. The generated sample enlarge the view range of action samples for training. Furthermore, to narrow the representation of actions in different views, we propose a view-domain generalization model that improves the recognition performance of arbitrary-view human action recognition. Our approach is evaluated on three large-scale RGB+D skeleton datasets including UESTC varying-view RGB+D dataset, NTU RGB+D 60, and NTU RGB+D 120 datasets, with two types of view-invariant evaluations, i.e., the cross-view, and arbitrary-view recognition. The proposed approach achieves outstanding performance in human action recognition.
Blockchain-Based Transactive Energy Framework for Connected Virtual Power Plants

Emerging technologies are helping to accelerate the ongoing energy transition. At the forefront of these new technologies is blockchain, which has the potential to disrupt energy trading markets. This article explores this potential by presenting an innovative multilevel transactive energy (TE) optimization model for the scheduling of distributed energy resources (DERs) within connected virtual power plants (VPPs). The model allows for energy transactions within a given VPP as well as between connected VPPs. A blockchain-based smart contract layer is applied on top of the TE optimization model to automate and record energy transactions. The model is formulated to adhere to the new regulations for the self-generation and self-consumption of energy in Portugal. This new set of regulations can ease barriers to entry for consumers and increase their active participation in energy markets. Results show a decrease in energy costs for consumers and increased generation of locally produced electricity. This model shows that blockchain-based smart contracts can be successfully integrated into a hierarchical energy trading model, which respects the novel energy regulation. This combination of technologies can be used to increase consumer participation, lower energy bills, and increase the penetration of locally generated electricity from renewable energy sources.
An Algorithm for Painting Large Objects Based on a Nine-Axis UR5 Robotic Manipulator

An algorithm for automatically planning trajectories designed for painting large objects is proposed in this paper to eliminate the difficulty of painting large objects and ensure their surface quality. The algorithm was divided into three phases, comprising the target point acquisition phase, the trajectory planning phase, and the UR5 robot inverse solution acquisition phase. In the target point acquisition phase, the standard triangle language (STL) file, algorithm of principal component analyses (PCA), and k-dimensional tree (k-d tree) were employed to obtain the point cloud model of the car roof to be painted. Simultaneously, the point cloud data were compressed as per the requirements of the painting process. In the trajectory planning phase, combined with the maximum operating space of the UR5 robot, the painting trajectory of the target points was converted into multiple traveling salesman problem (TSP) models, and each TSP model was created with a genetic algorithm (GA). In the last phase, in conformity with the singularities of the UR5 robot’s motion space, the painting trajectory was divided into a recommended area trajectory and a non-recommended area trajectory and created by the analytical method and sequential quadratic programming (SQP). Finally, the proposed algorithm for painting large objects was deployed in a simulation experiment. Simulation results showed that the accuracy of the algorithm could meet the requirements of painting technology, and it has promising engineering practicability.
Re-Examining Psychological Mechanisms Underlying Virtual Reality-Based Exposure for Spider Phobia

The proposed study aims at expanding results from a previous study on mechanisms of change after exposure in virtual reality (VR) and documenting the impact of adding tactile and haptic feedback. It was predicted that change in the severity of spider phobia according to the Fear of Spiders Questionnaire (FSQ) would be significantly predicted by change in dysfunctional beliefs toward spiders and self-efficacy, over and above the variance explained by a physiological measure of fear during exposure (heart rate) and presence during the immersion. Participants (N = 59) were randomly assigned to the presentation of visual stimuli only, visual plus tactile stimuli, or visual, tactile plus haptic feedback stimuli. A standard multiple regression was conducted to predict change on the FSQ using the following predictors: beliefs about spiders, beliefs about one's own behavior when facing spiders, perceived self-efficacy, disgust, presence, and heart rate. Only changes in beliefs about spiders and in perceived self-efficacy significantly predicted the reduction in fear of spiders. This result enhances our understanding of the mechanisms involved in exposure conducted in VR. Analyses of variance also show that participants reported statistically significant changes in their clinical condition, with little added value to the addition of tactile and haptic feedback. The advantages of tactile and haptic stimulation are questioned, at least in the context of only one brief exposure session and the equipment used.
Semi-Supervised Metric Learning-Based Anchor Graph Hashing for Large-Scale Image Retrieval

Hashing-based image retrieval methods have become a cutting-edge topic in the information retrieval domain due to their high efficiency and low cost. In order to perform efficient hash learning by simultaneously preserving the semantic similarity and data structures in the feature space, this paper presents the semi-supervised metric learning-based anchor graph hashing method. Our proposed approach can be divided into three parts. First, we exploit a transformation matrix to construct the anchor-based similarity graph of the training set. Second, we propose the objective function based on the triplet relationship, in which the optimal transformation matrix can be learned by using the smoothness of labels and the margin hinge loss incurred by the triplet constraint. Moreover, the stochastic gradient descent (SGD) method leverages the gradient on each triplet to update the transformation matrix. Finally, a penalty factor is designed to accelerate the execution speed of SGD. Through comparison with the retrieval results of several state-of-the-art methods on several image benchmarks, the experiments validate the feasibility and advantages of our proposed methods.
Entanglement criterion for multipartite systems based on quantum Fisher information

We propose a sufficient entanglement criterion for the multipartite systems based on quantum Fisher information. It complements the well-established positive partial transpose criterion in the sense that it is also able to detect bound entangled states. Moreover, it improves the well-known upper bound on the quantum Fisher information of separable multiqudit states introduced by Toth et al. [G. Toth and T. Vertesi, Phys. Rev. Lett. 120, 020506 (2018)] and hence detects a greater number of metrologically useful entangled states. This criterion is formulated as a simple inequality which must be fulfilled by the quantum Fisher information of separable states of multipartite systems. Violation of this inequality by a state reveals its entanglement and accordingly confirms its usefulness for quantum metrology. The efficiency of this criterion for detection of entanglement is illustrated through some examples.
Error-free secure key generation and distribution using dynamic Stokes parameters

This paper proposes and experimentally demonstrates an error-free secure key generation and distribution (SKGD) scheme in classical optical fiber link by exploiting Stokes parameters (SPs) of the state of polarization (SOP). Due to the unique birefringence distribution of the optical fiber channel, random but high-correlated SPs are shared between Alice and Bob. The dynamic SPs are also affected by the time-varying environmental factors, providing the source of randomness for the secret key extraction. As a proof of concept, key generation rate (KGR) of 213-bits/s is successfully demonstrated over 25-km standard single-mode fiber (SSMF). The error-free SKGD is realized in fiber channel using the information reconciliation (IR) technology, where Bose-Chaudhuri-Hocquenghem (BCH) codes are applied. Due to the channel uniqueness and the high-sensitivity to the initial SOP of optical signals, high-level security is provided by the proposed scheme, which is analyzed and verified against the possible fiber-tapping attacks. Moreover, the proposed SKGD scheme offers additional benefits such as simple structure, low cost, and suitablity for long-haul transmission.
Don't Blame the ELBO! A Linear VAE Perspective on Posterior Collapse

Posterior collapse in Variational Autoencoders (VAEs) with uninformative priors arises when the variational posterior distribution closely matches the prior for a subset of latent variables. This paper presents a simple and intuitive explanation for posterior collapse through the analysis of linear VAEs and their direct correspondence with Probabilistic PCA (pPCA). We explain how posterior collapse may occur in pPCA due to local maxima in the log marginal likelihood. Unexpectedly, we prove that the ELBO objective for the linear VAE does not introduce additional spurious local maxima relative to log marginal likelihood. We show further that training a linear VAE with exact variational inference recovers a uniquely identifiable global maximum corresponding to the principal component directions. Empirically, we find that our linear analysis is predictive even for high-capacity, non-linear VAEs and helps explain the relationship between the observation noise, local maxima, and posterior collapse in deep Gaussian VAEs.
Predicting Alzheimer’s Disease Conversion From Mild Cognitive Impairment Using an Extreme Learning Machine-Based Grading Method With Multimodal Data

Identifying patients with mild cognitive impairment (MCI), who are at high risk of progressing to Alzheimer’s disease (AD), is crucial for early treatment of AD. However, it is difficult to predict the cognitive states of patients. This study developed an extreme learning machine- (ELM-) based grading method to efficiently fuse multimodal data and predict MCI-to-AD conversion. First, features were extracted from magnetic resonance images (MRI) and useful features were selected using a feature selection method. Second, multiple modalities of MCI subjects, including MRI, positron emission tomography, cerebrospinal fluid biomarkers, and gene data were individually graded using the ELM method. Finally, these grading scores calculated from different modalities were fed into a classifier to discriminate subjects with progressive MCI from those with stable MCI. The proposed approach has been validated on the Alzheimer's Disease Neuroimaging Initiative (ADNI) cohort, and an accuracy of 84.7% was achieved for an AD prediction within 3 years. Experiments on predicting AD conversion from MCI within different periods showed similar results with the 3-year prediction. The experimental results demonstrate that the proposed approach benefits from the efficient fusion of four modalities, resulting in an accurate prediction of MCI-to-AD conversion.
LARa: Creating a Dataset for Human Activity Recognition in Logistics Using Semantic Attributes

Optimizations in logistics require recognition and analysis of human activities. The potential of sensor-based human activity recognition (HAR) in logistics is not yet well explored. Despite a significant increase in HAR datasets in the past twenty years, no available dataset depicts activities in logistics. This contribution presents the first freely accessible logistics-dataset. In the ’Innovationlab Hybrid Services in Logistics’ at TU Dortmund University, two picking and one packing scenarios were recreated. Fourteen subjects were recorded individually when performing warehousing activities using Optical marker-based Motion Capture (OMoCap), inertial measurement units (IMUs), and an RGB camera. A total of 758 min of recordings were labeled by 12 annotators in 474 person-h. All the given data have been labeled and categorized into 8 activity classes and 19 binary coarse-semantic descriptions, also called attributes. The dataset is deployed for solving HAR using deep networks.
Enhance Visual Recognition Under Adverse Conditions via Deep Networks

Visual recognition under adverse conditions is a very important and challenging problem of high practical value, due to the ubiquitous existence of quality distortions during image acquisition, transmission, or storage. While deep neural networks have been extensively exploited in the techniques of low-quality image restoration and high-quality image recognition tasks respectively, few studies have been done on the important problem of recognition from very low-quality images. This paper proposes a deep learning based framework for improving the performance of image and video recognition models under adverse conditions, using robust adverse pre-training or its aggressive variant. The robust adverse pre-training algorithms leverage the power of pre-training and generalizes conventional unsupervised pre-training and data augmentation methods. We further develop a transfer learning approach to cope with real-world datasets of unknown adverse conditions. The proposed framework is comprehensively evaluated on a number of image and video recognition benchmarks, and obtains significant performance improvements under various single or mixed adverse conditions. Our visualization and analysis further add to the explainability of results.
Hyperledger

Hyperledger is one of the biggest projects in the blockchain industry, which is comprised of a set of open source tools and subprojects. It includes leaders in different sectors who are aiming to build a robust, business-driven blockchain framework. It is an umbrella project, which aims to provide enterprise solutions and universal guidelines for blockchain implementation. The key projects within the Hyperledger framework are Hyperledger Fabric, Hyperledger Iroha, Hyperledger Indy, Hyperledger Sawtooth, Hyperledger Cello, Hyperledger Explorer, Hyperledger Composer. In this chapter, we have discussed the architecture, goals, and need for hyperledger projects in real-time applications.
I Can Find You! Boundary-Guided Separated Attention Network for Camouflaged Object Detection

Can you find me? By simulating how humans to discover the so-called 'perfectly'-camouflaged object, we present a novel boundary-guided separated attention network (call BSA-Net). Beyond the existing camouflaged object detection (COD) wisdom, BSA-Net utilizes two-stream separated attention modules to highlight the separator (or say the camouflaged object's boundary) between an image's background and foreground: the reverse attention stream helps erase the camouflaged object's interior to focus on the background, while the normal attention stream recovers the interior and thus pay more attention to the foreground; and both streams are followed by a boundary guider module and combined to strengthen the understanding of boundary. The core design of such separated attention is motivated by the COD procedure of humans: find the subtle difference between the foreground and background to delineate the boundary of a camouflaged object, then the boundary can help further enhance the COD accuracy. We validate on three benchmark datasets that the proposed BSA-Net is very beneficial to detect camouflaged objects with the blurred boundaries and similar colors/patterns with their backgrounds. Extensive results exhibit very clear COD improvements on our BSA-Net over sixteen SOTAs.
Understanding and correcting pathologies in the training of learned optimizers

Deep learning has shown that learned functions can dramatically outperform hand-designed functions on perceptual tasks. Analogously, this suggests that learned optimizers may similarly outperform current hand-designed optimizers, especially for specific problems. However, learned optimizers are notoriously difficult to train and have yet to demonstrate wall-clock speedups over hand-designed optimizers, and thus are rarely used in practice. Typically, learned optimizers are trained by truncated backpropagation through an unrolled optimization process resulting in gradients that are either strongly biased (for short truncations) or have exploding norm (for long truncations). In this work we propose a training scheme which overcomes both of these difficulties, by dynamically weighting two unbiased gradient estimators for a variational loss on optimizer performance, allowing us to train neural networks to perform optimization of a specific task faster than tuned first-order methods. We demonstrate these results on problems where our learned optimizer trains convolutional networks faster in wall-clock time compared to tuned first-order methods and with an improvement in test loss.
Linear space-variant optical cryptosystem via Fourier ptychography

An optical cryptosystem via Fourier ptychography with double random phase masks is proposed. The encryption process cannot be precisely simulated except by optical experiment due to vignetting, which is space variant and can act as a one-way function from the perspective of pure optics and improve the security of our system. In addition, the encryption for a high-resolution, large field-of-view, and complex-valued image is achievable. Optical experiments are presented to prove the validity and security of the proposed system. Our method will give more insight into separating optical cryptography from computer cryptography in nature.