Reviewerly/exHarmony huggingface dataset:
Error-Compensated Sparsification for Communication-Efficient Decentralized Training in Edge Environment

Communication has been considered as a major bottleneck in large-scale decentralized training systems since participating nodes iteratively exchange large amounts of intermediate data with their neighbors. Although compression techniques like sparsification can significantly reduce the communication overhead in each iteration, errors caused by compression will be accumulated, resulting in a severely degraded convergence rate. Recently, the error compensation method for sparsification has been proposed in centralized training to tolerate the accumulated compression errors. However, the analog technique and the corresponding theory about its convergence in decentralized training are still unknown. To fill in the gap, we design a method named ECSD-SGD that significantly accelerates decentralized training via error-compensated sparsification. The novelty lies in that we identify the component of the exchanging information in each iteration (i.e., the sparsified model update) and make targeted error compensation over the component. Our thorough theoretical analysis shows that ECSD-SGD supports arbitrary sparsification ratio and achieves the same convergence rate as the non-sparsified decentralized training methods. We also conduct extensive experiments on multiple deep learning models to validate our theoretical findings. Results show that ECSD-SGD outperforms all the start-of-the-art sparsified methods in terms of both the convergence speed and the final generalization accuracy.
Performance Evaluation of Population Seeding Techniques of Permutation-Coded GA Traveling Salesman Problems Based Assessment

Genetic algorithms (GAs) are a population-based meta-heuristic global optimization technique for dealing with complex problems with a very large search space. The population initialization is a crucial task in GAs because it plays a vital role in the convergence speed, problem search space exploration, and also the quality of the final optimal solution. Though the importance of deciding problem-specific population initialization in GA is widely recognized, it is hardly addressed in the literature. In this article, different population seeding techniques for permutation-coded genetic algorithms such as random, nearest neighbor (NN), gene bank (GB), sorted population (SP), and selective initialization (SI), along with three newly proposed ordered-distance-vector-based initialization techniques have been extensively studied. The ability of each population seeding technique has been examined in terms of a set of performance criteria, such as computation time, convergence rate, error rate, average convergence, convergence diversity, nearest-neighbor ratio, average distinct solutions and distribution of individuals. One of the famous combinatorial hard problems of the traveling salesman problem (TSP) is being chosen as the testbed and the experiments are performed on large-sized benchmark TSP instances obtained from standard TSPLIB. The scope of the experiments in this article is limited to the initialization phase of the GA and this restricted scope helps to assess the performance of the population seeding techniques in their intended phase alone. The experimentation analyses are carried out using statistical tools to claim the unique performance characteristic of each population seeding techniques and best performing techniques are identified based on the assessment criteria defined and the nature of the application.
SPIDERS<mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" id="d1e682" altimg="si7.svg"><mml:mo>+</mml:mo></mml:math>: A light-weight, wireless, and low-cost glasses-based wearable platform for emotion sensing and bio-signal acquisition

We present a System for Processing In-situ Bio-signal Data for Emotion Recog-nition and Sensing (SPIDERS+) – a low-cost, wireless, glasses-based platform for continuous in-situ monitoring of user’s facial expressions (apparent emotions) and real emotions. We present algorithms to provide four core functions to detect eye shape and eyebrow movements, pupillometry, zygomaticus muscle movements, and head movements, using the bio-signals acquired from three non-contact sensors (IR camera, proximity sensor, and IMU). SPIDERS+ distinguishes between different classes of apparent and real emotion states based on the aforementioned four bio-signals. We prototype advanced functionalities including facial expression detection and real emotion classification with a facial expression detector based on landmarks and optical flow that leverages changes in a user’s eyebrows and eye shapes to achieve the average true positive rate up to 83.87%, as well as a pupillometry-based real emotion classifier with higher accuracy than existing low-cost wearable platforms that use sensors requiring skin contact. In addition, SPIDERS+ also includes optional contact-based sensors such as photoplethysmography (PPG) and multi-channel electroencephalogram (EEG) to monitor other bio-signals. SPIDERS+ costs less than $20 to assemble and can continuously run for up to 9 h before recharging. We design, implement and evaluate accompanying desktop and smartphone applications to demonstrate that SPIDERS+ is a truly wireless and portable platform that can impact a wide range of applications, where the knowledge of the user’s emotional state is critical.
PCLs: Geometry-aware Neural Reconstruction of 3D Pose with Perspective Crop Layers

Local processing is an essential feature of CNNs and other neural network architectures—it is one of the reasons why they work so well on images where relevant information is, to a large extent, local. However, perspective effects stemming from the projection in a conventional camera vary for different global positions in the image. We introduce Perspective Crop Layers (PCLs)—a form of perspective crop of the region of interest based on the camera geometry— and show that accounting for the perspective consistently improves the accuracy of state-of-the-art 3D pose reconstruction methods. PCLs are modular neural network layers, which, when inserted into existing CNN and MLP architectures, deterministically remove the location-dependent perspective effects while leaving end-to-end training and the number of parameters of the underlying neural network unchanged. We demonstrate that PCL leads to improved 3D human pose reconstruction accuracy for CNN architectures that use cropping operations, such as spatial transformer networks (STN), and, somewhat surprisingly, MLPs used for 2D-to-3D key-point lifting. Our conclusion is that it is important to utilize camera calibration information when available, for classical and deep-learning-based computer vision alike. PCL offers an easy way to improve the accuracy of existing 3D reconstruction networks by making them geometry-aware. Our code is publicly available at github.com/yu-frank/PerspectiveCropLayers.
The graph-based behavior-aware recommendation for interactive news

Interactive news recommendation has been launched and attracted much attention recently. In this scenario, user’s behavior evolves from single click behavior to multiple behaviors including like, comment, share etc. However, most of the existing methods still use single click behavior as the unique criterion of judging user’s preferences. Further, although heterogeneous graphs have been applied in different areas, a proper way to construct a heterogeneous graph for interactive news data with an appropriate learning mechanism on it is still desired. To address the above concerns, we propose a graph-based behavior-aware network, which simultaneously considers six different types of behaviors as well as user’s demand on the news diversity. We have three mainsteps. First, we build an interaction behavior graph for multi-level and multi-category data. Second, we apply DeepWalk on the behavior graph to obtain entity semantics, then build a graph-based convolutional neural network called G-CNN to learn news representations, and an attention-based LSTM to learn behavior sequence representations. Third, we introduce core and coritivity features for the behavior graph, which measure the concentration degree of user’s interests. These features affect the trade-off between accuracy and diversity of our personalized recommendation system. Taking these features into account, our system finally achieves recommending news to different users at their different levels of concentration degrees.
Combining Semantic Guidance and Deep Reinforcement Learning For Generating Human Level Paintings

Generation of stroke-based non-photorealistic imagery, is an important problem in the computer vision community. As an endeavor in this direction, substantial recent research efforts have been focused on teaching machines "how to paint", in a manner similar to a human painter. However, the applicability of previous methods has been limited to datasets with little variation in position, scale and saliency of the foreground object. As a consequence, we find that these methods struggle to cover the granularity and diversity possessed by real world images. To this end, we propose a Semantic Guidance pipeline with 1) a bi-level painting procedure for learning the distinction between foreground and background brush strokes at training time. 2) We also introduce invariance to the position and scale of the foreground object through a neural alignment model, which combines object localization and spatial transformer networks in an end to end manner, to zoom into a particular semantic instance. 3) The distinguishing features of the in-focus object are then amplified by maximizing a novel guided backpropagation based focus reward. The proposed agent does not require any supervision on human stroke-data and successfully handles variations in foreground object attributes, thus, producing much higher quality canvases for the CUB-200 Birds [29] and Stanford Cars-196 [17] datasets. Finally, we demonstrate the further efficacy of our method on complex datasets with multiple foreground object instances by evaluating an extension of our method on the challenging Virtual-KITTI [2] dataset. Source code and models are available at https://github.com/1jsingh/semantic-guidance.
A Two-stage Deep Network for High Dynamic Range Image Reconstruction

Mapping a single exposure low dynamic range (LDR) image into a high dynamic range (HDR) is considered among the most strenuous image to image translation tasks due to exposure-related missing information. This study tackles the challenges of single-shot LDR to HDR mapping by proposing a novel two-stage deep network. Notably, our proposed method aims to reconstruct an HDR image with-out knowing hardware information, including camera response function (CRF) and exposure settings. Therefore, we aim to perform image enhancement task like denoising, exposure correction, etc., in the first stage. Additionally, the second stage of our deep network learns tone mapping and bit-expansion from a convex set of data samples. The qualitative and quantitative comparisons demonstrate that the proposed method can outperform the existing LDR to HDR works with a marginal difference. Apart from that, we collected an LDR image dataset incorporating different camera systems. The evaluation with our collected real-world LDR images illustrates that the proposed method can reconstruct plausible HDR images without presenting any visual artefacts. Code available : https://github.com/sharif-apu/twostageHDR_NTIRE21.
VirFace: Enhancing Face Recognition via Unlabeled Shallow Data

Recently, how to exploit unlabeled data for training face recognition models has been attracting increasing attention. However, few works consider the unlabeled shallow data <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> in real-world scenarios. The existing semi-supervised face recognition methods that focus on generating pseudo labels or minimizing softmax classification probabilities of the unlabeled data do not work very well on the unlabeled shallow data. It is still a challenge on how to effectively utilize the unlabeled shallow face data to improve the performance of face recognition. In this paper, we propose a novel face recognition method, named VirFace, to effectively exploit the unlabeled shallow data for face recognition. VirFace consists of VirClass and VirInstance. Specifically, VirClass enlarges the inter-class distance by injecting the unlabeled data as new identities, while VirInstance produces virtual instances sampled from the learned distribution of each identity to further enlarge the inter-class distance. To the best of our knowledge, we are the first to tackle the problem of unlabeled shallow face data. Extensive experiments have been conducted on both the small- and large-scale datasets, e.g. LFW and IJB-C, etc, demonstrating the superiority of the proposed method.
Augmented Reality: Focusing on Photonics in Industry 4.0

Industry 4.0 (or 4 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">th</sup> industrial revolution) facilitates horizontal and vertical digital information flow along value chains up to the end-customer and is highly relevant in a broad variety of industries. Augmented reality (AR) is a key technology in Industry 4.0, which connects the virtual and real-world environments using such digital information flows. In doing so, the technology relies upon the systems that includes hardware and software components. Particularly, optics and photonics are of much importance in the display and processing of information in these systems. However, a particular challenge is that the AR-based systems have not been adopted in the industry as much as other technologies even after several decades of their existence. Based on review of academic literature, an industrial survey and experiments conducted in the industry, this article aims to identify success factors and challenges of AR systems and metrics of photonic components that can form the basis of an AR <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">*</sup> framework for photonics-based system design for future research.
Neural Response Interpretation through the Lens of Critical Pathways

Is critical input information encoded in specific sparse pathways within the neural network? In this work, we discuss the problem of identifying these critical pathways and subsequently leverage them for interpreting the network’s response to an input. The pruning objective — selecting the smallest group of neurons for which the response remains equivalent to the original network — has been previously proposed for identifying critical pathways. We demonstrate that sparse pathways derived from pruning do not necessarily encode critical input information. To ensure sparse pathways include critical fragments of the encoded input information, we propose pathway selection via neurons’ contribution to the response. We proceed to explain how critical pathways can reveal critical input features. We prove that pathways selected via neuron contribution are locally linear (in an ℓ <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</inf> -ball), a property that we use for proposing a feature attribution method: "pathway gradient". We validate our interpretation method using mainstream evaluation experiments. The validation of pathway gradient interpretation method further confirms that selected pathways using neuron contributions correspond to critical input features. The code <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> is publicly available.
Security-enhanced optical nonlinear cryptosystem based on modified Gerchberg–Saxton iterative algorithm

For any optical encryption and decryption systems, security is the major concern. In this paper, security of time varying nonlinear term embedded double random phase encoding (DRPE) scheme is enhanced by using Modified Gerchberg Saxton Iterative Algorithm (MGSIA). The encrypted image of DRPE is given as an amplitude input to MGSIA. This makes the cryptosystem asymmetric and endures the basic attacks such as known-plaintext attack, chosen-plaintext attack, and chosen-cipher text attack. Robustness of this model has been checked with various results.
Co-Correcting: Noise-Tolerant Medical Image Classification via Mutual Label Correction

With the development of deep learning, medical image classification has been significantly improved. However, deep learning requires massive data with labels. While labeling the samples by human experts is expensive and time-consuming, collecting labels from crowd-sourcing suffers from the noises which may degenerate the accuracy of classifiers. Therefore, approaches that can effectively handle label noises are highly desired. Unfortunately, recent progress on handling label noise in deep learning has gone largely unnoticed by the medical image. To fill the gap, this paper proposes a noise-tolerant medical image classification framework named Co-Correcting, which significantly improves classification accuracy and obtains more accurate labels through dual-network mutual learning, label probability estimation, and curriculum label correcting. On two representative medical image datasets and the MNIST dataset, we test six latest Learning-with-Noisy-Labels methods and conduct comparative studies. The experiments show that Co-Correcting achieves the best accuracy and generalization under different noise ratios in various tasks. Our project can be found at: https://github.com/JiarunLiu/Co-Correcting.
Predicting lethal courses in critically ill COVID-19 patients using a machine learning model trained on patients with non-COVID-19 viral pneumonia

In a pandemic with a novel disease, disease-specific prognosis models are available only with a delay. To bridge the critical early phase, models built for similar diseases might be applied. To test the accuracy of such a knowledge transfer, we investigated how precise lethal courses in critically ill COVID-19 patients can be predicted by a model trained on critically ill non-COVID-19 viral pneumonia patients. We trained gradient boosted decision tree models on 718 (245 deceased) non-COVID-19 viral pneumonia patients to predict individual ICU mortality and applied it to 1054 (369 deceased) COVID-19 patients. Our model showed a significantly better predictive performance (AUROC 0.86 [95% CI 0.86-0.87]) than the clinical scores APACHE2 (0.63 [95% CI 0.61-0.65]), SAPS2 (0.72 [95% CI 0.71-0.74]) and SOFA (0.76 [95% CI 0.75-0.77]), the COVID-19-specific mortality prediction models of Zhou (0.76 [95% CI 0.73-0.78]) and Wang (laboratory: 0.62 [95% CI 0.59-0.65]; clinical: 0.56 [95% CI 0.55-0.58]) and the 4C COVID-19 Mortality score (0.71 [95% CI 0.70-0.72]). We conclude that lethal courses in critically ill COVID-19 patients can be predicted by a machine learning model trained on non-COVID-19 patients. Our results suggest that in a pandemic with a novel disease, prognosis models built for similar diseases can be applied, even when the diseases differ in time courses and in rates of critical and lethal courses.
Analysis of false data detection rate in generative adversarial networks using recurrent neural network

In this chapter, we aim to scale-up the operation of GAN using recurrent neural network (RNN). The GAN is designed with two deep neural network models, where the first network is called a generator that generates perceptually synthetic samples drawn through real data distribution. The first network recursively learns the network and transforms the noise vectors into samples. The second network is called a discriminator that accesses both real and synthetic samples and provides classification between these two data. The discriminator uses the same RNN to classify the generated and real data samples. The aim of using RNN in the GAN structure is to delimit the error rate using its time series prediction based on its past inputs. It further uses its neighborhood relationship between the samples to generate the target output and error generated. The errors are then propagated in the backward direction over the GAN to update the network weights to estimate the output. The RNN generator, on the other hand, reduces the probability of the RNN discriminator in identifying the false generated samples and increasing the probability of discriminator in identifying correctly the real samples. The objective function in RNN is designed in such a way that its gradient operator for the false samples is quite far from the decision boundary of RNN discriminator, thereby producing the increasing true classification rate. The experiments carried out on the real-world time series dataset show the results of accurate classification (97.01%) with increased false detection rate than benchmark GAN method.
Group article recommendation based on ER rule in Scientific Social Networks

Group Recommendation Systems (GRS) is an emerging area in both research and practice and has been successfully developed in many domains as a type of information filter to overcome the information overload problem. With the growth of Scientific Social Networks (SSNs), the need for article recommendation is emerging. Considering that researchers can be grouped according to their research interests, and article recommendation to a group of users has not been addressed in the literature, this paper aims to develop and test an inferential model to accurately recommend articles for group researchers in SSNs. In this paper, a novel approach for group article recommendation, referred to as GPRAH_ER, is proposed to improve the processes of both individual prediction and group aggregation. In the stage of individual prediction, the Probabilistic Matrix Factorization method is adopted and is further unified by using articles’ contents and group information. In the stage of group aggregation, the ER rule is introduced in the aggregation process, since it possesses the advantages of identifying group members’ impacts based on the group member’s weight and reliability. To verify the performance of the proposed method, experiments are conducted on a real dataset CiteULike. The experimental results show that the proposed GPRAH_ER method outperforms other benchmark methods, and provides a more effective recommendation of articles to researchers in SSNs.