Reviewerly/exHarmony huggingface dataset:
Ultra-dense GAN for satellite imagery super-resolution

Image super-resolution (SR) techniques improve various remote sensing applications by allowing for finer spatial details than those captured by the original acquisition sensors. Recent advances in deep learning bring a new opportunity for SR by learning the mapping from low to high resolution. The most used convolutional neural networks (CNN) based approaches are prone to excessive smoothing or blurring due to the optimization objective in mean squared error (MSE). Instead, generative adversarial network (GAN) based approaches can achieve more perceptually acceptable results. However, the preliminary design of GANs generator with simple direct- or skip-connection residual blocks compromises its SR potential. Emerging dense convolutional network (DenseNet) equipped with dense connections has shown a promising prospect in classification and super-resolution. An intuitive idea to introduce DenseNet into GAN is expected to boost SR performance. However, because convolutional kernels in the existing residual block are arranged into a one-dimensional flat structure, the formation of dense connections highly relies on skip connections (linking the current layer to all subsequent layers with a shortcut path). In order to increase connection density, the depth of the layer has to be accordingly expanded, which in turn results in training difficulties such as vanishing gradient and information propagation loss. To this end, this paper proposes an ultra-dense GAN (udGAN) for image SR, where we reform the internal layout of the residual block into a two-dimensional matrix topology. This topology can provide additional diagonal connections so that we can still accomplish enough pathways with fewer layers. In particular, the pathways are almost doubled compared to previous dense connections under the same number of layers. The achievable rich connections are flexibly adapted to the diversity of image content, thus leading to improved SR performance. Extensive experiments on public benchmark datasets and real-world satellite imagery show that our model outperforms state-of-the-art counterparts in both subjective and quantitative assessments, especially those related to perception.
Learning Robust Facial Landmark Detection via Hierarchical Structured Ensemble

Heatmap regression-based models have significantly advanced the progress of facial landmark detection. However, the lack of structural constraints always generates inaccurate heatmaps resulting in poor landmark detection performance. While hierarchical structure modeling methods have been proposed to tackle this issue, they all heavily rely on manually designed tree structures. The designed hierarchical structure is likely to be completely corrupted due to the missing or inaccurate prediction of landmarks. To the best of our knowledge, in the context of deep learning, no work before has investigated how to automatically model proper structures for facial landmarks, by discovering their inherent relations. In this paper, we propose a novel Hierarchical Structured Landmark Ensemble (HSLE) model for learning robust facial landmark detection, by using it as the structural constraints. Different from existing approaches of manually designing structures, our proposed HSLE model is constructed automatically via discovering the most robust patterns so HSLE has the ability to robustly depict both local and holistic landmark structures simultaneously. Our proposed HSLE can be readily plugged into any existing facial landmark detection baselines for further performance improvement. Extensive experimental results demonstrate our approach significantly outperforms the baseline by a large margin to achieve a state-of-the-art performance.
Compounding of Wealth in Proof-of-Stake Cryptocurrencies

Proof-of-stake (PoS) is a promising approach for designing efficient blockchains, where block proposers are randomly chosen with probability proportional to their stake. A primary concern in PoS systems is the “rich getting richer” effect, whereby wealthier nodes are more likely to get elected, and hence reap the block reward, making them even wealthier. In this paper, we introduce the notion of equitability, which quantifies how much a proposer can amplify her stake compared to her initial investment. Even with everyone following protocol (i.e., honest behavior), we show that existing methods of allocating block rewards lead to poor equitability, as does initializing systems with small stake pools and/or large rewards relative to the stake pool. We identify a geometric reward function, which we prove is maximally equitable over all choices of reward functions under honest behavior and bound the deviation for strategic actions; the proofs involve the study of optimization problems and stochastic dominances of Pólya urn processes. These results allow us to provide a systematic framework to choose the parameters of a practical incentive system for PoS cryptocurrencies.
Managing interrelated project information in AEC Knowledge Graphs

In the architecture, engineering and construction (AEC) industry stakeholders from different companies and backgrounds collaborate in realising a common goal being some physical structure. The exact goal is typically not known from the beginning, and throughout all design stages, new decisions are made - similarly to other design industries [1]. As a result, the design must adapt and subsequent consequences follow. With working methods being predominantly document-centric, highly interrelated and rapidly changing design data in a complex network of decisions, requirements and product specifications is primarily captured in static documents. In this paper, we consider a purely data-driven approach based on semantic web technologies and an earlier proposed Ontology for Property Management (OPM). The main contribution of this work consists of extensions for OPM to account for new competency questions including the description of property reliability and the reasoning logic behind derived properties. The secondary contribution is the specification of a homogeneous way to generate parametric queries for managing an OPM-compliant AEC Knowledge Graph (AEC-KG). A software library for operating an OPM-compliant AEC-KG is further presented in the form of an OPM Query Generator (OPM-QG). The library generates SPARQL 1.1 queries to query and manipulate construction project Knowledge Graphs represented using OPM. The OPM ontology aligns with latest developments in the W3C Community Group on Linked Building Data and suggests an approach to working with design data in a distributed environment using separate graphs for explicit facts and for materialised, deduced data. Finally, we evaluate the suggested approach using an open-source software artefact developed using OPM and OPM-QG, demonstrated online with an actual building Knowledge Graph. The particular design task evaluated is performing heat loss calculations for spaces of a future building using an AEC-KG described using domain- and project specific extensions of the Building Topology Ontology (BOT) in combination with OPM. With this work, we demonstrate how a typical engineering task can be accomplished and managed in an evolving design environment, thereby providing the engineers with insights to support decision making as changes occur. The application uses a strict division between the client viewer and the actual data model holding design logic, and can easily be extended to support other design tasks.
CryptoNN: Training Neural Networks over Encrypted Data

Emerging neural networks based machine learning techniques such as deep learning and its variants have shown tremendous potential in many application domains. However, they raise serious privacy concerns due to the risk of leakage of highly privacy-sensitive data when data collected from users is used to train neural network models to support predictive tasks. To tackle such serious privacy concerns, several privacy-preserving approaches have been proposed in the literature that use either secure multi-party computation (SMC) or homomorphic encryption (HE) as the underlying mechanisms. However, neither of these cryptographic approaches provides an efficient solution towards constructing a privacy-preserving machine learning model, as well as supporting both the training and inference phases. To tackle the above issue, we propose a CryptoNN framework that supports training a neural network model over encrypted data by using the emerging functional encryption scheme instead of SMC or HE. We also construct a functional encryption scheme for basic arithmetic computation to support the requirement of the proposed CryptoNN framework. We present performance evaluation and security analysis of the underlying crypto scheme and show through our experiments that CryptoNN achieves accuracy that is similar to those of the baseline neural network models on the MNIST dataset.
Technologies for Social Augmentations in User-Embodied Virtual Reality

Technologies for Virtual, Mixed, and Augmented Reality (VR, MR, and AR) allow to artificially augment social interactions and thus to go beyond what is possible in real life. Motivations for the use of social augmentations are manifold, for example, to synthesize behavior when sensory input is missing, to provide additional affordances in shared environments, or to support inclusion and training of individuals with social communication disorders. We review and categorize augmentation approaches and propose a software architecture based on four data layers. Three components further handle the status analysis, the modification, and the blending of behaviors. We present a prototype (injectX) that supports behavior tracking (body motion, eye gaze, and facial expressions from the lower face), status analysis, decision-making, augmentation, and behavior blending in immersive interactions. Along with a critical reflection, we consider further technical and ethical aspects.
Where Is My Mirror?

Mirrors are everywhere in our daily lives. Existing computer vision systems do not consider mirrors, and hence may get confused by the reflected content inside a mirror, resulting in a severe performance degradation. However, separating the real content outside a mirror from the reflected content inside it is non-trivial. The key challenge is that mirrors typically reflect contents similar to their surroundings, making it very difficult to differentiate the two. In this paper, we present a novel method to segment mirrors from an input image. To the best of our knowledge, this is the first work to address the mirror segmentation problem with a computational approach. We make the following contributions. First, we construct a large-scale mirror dataset that contains mirror images with corresponding manually annotated masks. This dataset covers a variety of daily life scenes, and will be made publicly available for future research. Second, we propose a novel network, called MirrorNet, for mirror segmentation, by modeling both semantical and low-level color/texture discontinuities between the contents inside and outside of the mirrors. Third, we conduct extensive experiments to evaluate the proposed method, and show that it outperforms the carefully chosen baselines from the state-of-the-art detection and segmentation methods.
Diagnosis of Benign and Malignant Thyroid Nodules Using Combined Conventional Ultrasound and Ultrasound Elasticity Imaging

Ultrasonography is one of the main imaging methods for diagnosing thyroid nodules. Automatic differentiation between benign and malignant nodules in ultrasound images can greatly assist inexperienced clinicians in their diagnosis. The key of problem is the effective utilization of the features of ultrasound images. In this study, we propose a method that is based on the combination of conventional ultrasound and ultrasound elasticity images based on a convolutional neural network and introduces richer feature information for the classification of benign and malignant thyroid nodules. First, the conventional network model performs pretraining on ImageNet and transfers the feature parameters to the ultrasound image domain by transfer learning so that depth features may be extracted and small samples may be processed. Then, we combine the depth features of conventional ultrasound and ultrasound elasticity images to form a hybrid feature space. Finally, the classification is completed on the hybrid feature space, and an end-to-end CNN model is implemented. The experimental results demonstrate that the accuracy of the proposed method is 0.9470, which is better than that of other single data-source methods under the same conditions.
The Catcher in the Field

Verifying the identity of voice inputs is important as voices are increasingly used for sensitive operations. Traditional methods focus on differentiating individuals via the spectrographic features of voices (e.g., voiceprint), yet cannot cope with spoofing attacks, whereby a malicious attacker synthesizes the voice with almost the same voiceprint of a victim or simply replays it. This paper proposes CaField, a text-independent speaker verification method to detect loudspeaker-based voice spoofing attacks with the goal of achieving two seemingly conflicting requirements: usability and security. The key insight of CaField is to construct "fieldprint'' with the acoustic biometrics embedded in sound fields, i.e., a physical field of acoustic energy created as the sound propagates over the air, as analogous to "voiceprint''. We find that fieldprints can be distinctive between speakers (either humans or loudspeakers), and thus we may detect the speakers being used for spoofing attacks from the authentic users. Our evaluation on a dataset of 20 people and 8 loudspeakers shows that by relying on two on-board microphones to sample sound fields while users talk to the smartphones, CaField achieves a detection accuracy of 99.16% and an equal error rate (EER) of 0.85% across multiple sessions and various voice inputs. CaField supports low audio sample rates at 8~kHz and is robust to various factors including phone displacement, user posture, recording environment, etc.
RVNet: Deep Sensor Fusion of Monocular Camera and Radar for Image-Based Obstacle Detection in Challenging Environments

Camera and radar-based obstacle detection are important research topics in environment perception for autonomous driving. Camera-based obstacle detection reports state-of-the-art accuracy, but the performance is limited in challenging environments. In challenging environments, the camera features are noisy, limiting the detection accuracy. In comparison, the radar-based obstacle detection methods using the 77 GHZ long-range radar are not affected by these challenging environments. However, the radar features are sparse with no delineation of the obstacles. The camera and radar features are complementary, and their fusion results in robust obstacle detection in varied environments. Once calibrated, the radar features can be used for localization of the image obstacles, while the camera features can be used for the delineation of the localized obstacles. We propose a novel deep learning-based sensor fusion framework, termed as the “RVNet”, for the effective fusion of the monocular camera and long-range radar for obstacle detection. The RVNet is a single shot object detection network with two input branches and two output branches. The RVNet input branches contain separate branches for the monocular camera and the radar features. The radar features are formulated using a novel feature descriptor, termed as the “sparse radar image”. For the output branches, the proposed network contains separate branches for small obstacles and big obstacles, respectively. The validation of the proposed network with state-of-the-art baseline algorithm is performed on the Nuscenes public dataset. Additionally, a detailed parameter analysis is performed with several variants of the RVNet. The experimental results show that the proposed network is better than baseline algorithms in varying environmental conditions.
Interpolated Adversarial Training

Adversarial robustness has become a central goal in deep learning, both in theory and in practice. However, successful methods to improve the adversarial robustness (such as adversarial training) greatly hurt generalization performance on the unperturbed data. This could have a major impact on how achieving adversarial robustness affects real world systems (i.e. many may opt to forego robustness if it can improve accuracy on the unperturbed data). We propose Interpolated Adversarial Training, which employs recently proposed interpolation based training methods in the framework of adversarial training. On CIFAR-10, adversarial training increases the standard test error (when there is no adversary) from 4.43% to 12.32%, whereas with our Interpolated adversarial training we retain adversarial robustness while achieving a standard test error of only 6.45%. With our technique, the relative increase in the standard error for the robust model is reduced from 178.1% to just 45.5%.
A semi-supervised Laplacian extreme learning machine and feature fusion with CNN for industrial superheat identification

The superheat degree (SD) in industrial aluminum electrolysis cell is a critical index that can maintain the energy balance, improve the current efficiency and improve production. However, the existing SD identification is mainly relying on artificial experience and the accuracy of SD is far from satisfactory. Further, artificial costs and physical equipment are expensive and time-consuming. In this paper, we propose a deep soft sensor method for SD detection. First, CNN is utilized for flame hole image feature extraction. Second, a semi-supervised extreme learning machine (ELM) that integrates Laplacian regularization is further used for SD classification. The main contributions of the paper are: (1) The proposed CNN-LapsELM utilizes the CNN for flame hole image feature extraction and then ELM for further classification, which fully takes advantage of CNN's ability for complex feature extraction, ELM's excellent generalization ability, and high computation efficiency. (2) Both the labeled and unlabeled samples are utilized for the CNN-LapsELM training process. It fully leverages the information contained in unlabeled data. At the same time, Laplacian regularization is utilized for learning the manifold structure of hole image samples, so the performance of the proposed CNN-LapsELM are improved. (3) The proposed CNN-LapsELM algorithm improves the generalization ability and robustness. The comparison result demonstrates that the CNN-LapsELM is superior to the existing SD identification and the accuracy is 87%.
DevNet: An Efficient CNN Architecture for Handwritten Devanagari Character Recognition

The writing style is a unique characteristic of a human being as it varies from one person to another. Due to such diversity in writing style, handwritten character recognition (HCR) under the purview of pattern recognition is not trivial. Conventional methods used handcrafted features that required a-priori domain knowledge, which is always not feasible. In such a case, extracting features automatically could potentially attract more interests. For this, in the literature, convolutional neural network (CNN) has been a popular approach to extract features from the image data. However, state-of-the-art works do not provide a generic CNN model for character recognition, Devanagari script, for instance. Therefore, in this work, we first study several different CNN models on publicly available handwritten Devanagari characters and numerals datasets. This means that our study is primarily focusing on comparative study by taking trainable parameters, training time and memory consumption into account. Later, we propose and design DevNet, a modified CNN architecture that produced promising results, since computational complexity and memory space are our primary concerns in design.
SPLINE-Net: Sparse Photometric Stereo Through Lighting Interpolation and Normal Estimation Networks

This paper solves the Sparse Photometric stereo through Lighting Interpolation and Normal Estimation using a generative Network (SPLINE-Net). SPLINE-Net contains a lighting interpolation network to generate dense lighting observations given a sparse set of lights as inputs followed by a normal estimation network to estimate surface normals. Both networks are jointly constrained by the proposed symmetric and asymmetric loss functions to enforce isotropic constrain and perform outlier rejection of global illumination effects. SPLINE-Net is verified to outperform existing methods for photometric stereo of general BRDFs by using only ten images of different lights instead of using nearly one hundred images.
Geodata science and geochemical mapping

Geodata science (GDS) is an interdisciplinary field in which geoscience data are mined for us to well understand the origin, evolution and future of our Earth and planet with prediction and assessment of its resources, environments, and natural hazards. The data chain of GDS involves collecting geosciences data, mining geoinformation, discovering geo-knowledge, and making spatial decisions. There are three groups of GDS methods for exploring and mining geoscience data including data statistics, data mining, and data insight and prediction. A case study on geochemical exploration data mapping was conducted to demonstrate the powerful use of GDS. The results show that GDS is a new research paradigm for exploring the spatial association of geochemical patterns, mining elemental association, and recognizing geochemical anomalies associated with mineralization via geo-computation and geo-visualization techniques in support of mineral exploration.
Fifty years of Hoare’s logic

Abstract We present a history of Hoare’s logic.
S2GAN: Share Aging Factors Across Ages and Share Aging Trends Among Individuals

Generally, we human follow the roughly common aging trends, e.g., the wrinkles only tend to be more, longer or deeper. However, the aging process of each individual is more dominated by his/her personalized factors, including the invariant factors such as identity and mole, as well as the personalized aging patterns, e.g., one may age by graying hair while another may age by receding hairline. Following this biological principle, in this work, we propose an effective and efficient method to simulate natural aging. Specifically, a personalized aging basis is established for each individual to depict his/her own aging factors. Then different ages share this basis, being derived through age-specific transforms. The age-specific transforms represent the aging trends which are shared among all individuals. The proposed method can achieve continuous face aging with favorable aging accuracy, identity preservation, and fidelity. Furthermore, befitted from the effective design, a unique model is capable of all ages and the prediction time is significantly saved.