Reviewerly/exHarmony huggingface dataset:
Multi-view 3D Morphable Face Reconstruction via Canonical Volume Fusion

Due to the capability of easy animation and editing of faces, 3D Morphable Model (3DMM) is widely used in the task of face reconstruction. Recent methods recover 3DMM coefficients by fusing the information from a set of multi-view images via end-to-end Convolutional Neural Networks (CNNs), which alleviate the inherent depth ambiguity in the single-view setting. However, most of these methods fuse global features of all views to regress the 3D morphable face, without considering the dense correspondences of multi-view images. In this paper, we propose a novel approach to reconstruct high-quality 3D morphable faces. We first use a canonical feature volume to fuse multiple view features in 3D space, which establish dense correspondences between different views. Next, to bridge the gap between CNN regression and pixel-wise optimization and further leverage the muti-view information, we propose test-time optimization to improve the regressed results with negligible additional cost. Our method achieves the state-of-the-art performance on widely-used benchmarks, demonstrating the effectiveness of our approach. Code will be released.
Second-Order Global Attention Networks for Graph Classification and Regression

Graph Neural Networks (GNNs) are powerful to learn representation of graph-structured data, which fuse both attributive and topological information. Prior researches have investigated the expressive power of GNNs by comparing it with Weisfeiler-Lehman algorithm. In spite of having achieved promising performance for the isomorphism test, existing methods assume overly restrictive requirement, which might hinder the performance on other graph-level tasks, e.g., graph classification and graph regression. In this paper, we argue the rationality of adaptively emphasizing important information. We propose a novel global attention module from two levels: channel level and node level. Specifically, we exploit second-order channel correlation to extract more discriminative representations. We validate the effectiveness of the proposed approach through extensive experiments on eight benchmark datasets. The proposed method performs better than the other state-of-the-art methods in graph classification and graph regression tasks. Notably, It achieves 2.7% improvement on DD dataset for graph classification and 7.1% absolute improvement on ZINC dataset for graph regression.
Blockchain Application Methodology for Improving Trust in the Collaborative Supply Chain

The globalization of the economy, the major disruptions due to the COVID-19 crisis, customer requirements as well as the current context of war, are straining global supply chains, which must demonstrate resilience and constantly seek to improve their performance. Performance. Among the essential approaches to achieve these objectives, collaboration between logistics partners through methods and tools that have proven themselves but also have shown limits, particularly in terms of trust. Blockchain technology, which appeared in 2008, now offers an opportunity to improve collaboration by providing answers to concerns about trust. However, its application is not generalized, indeed a large part of the companies do not apply the technology because of its shared database character because the companies have sensitive and confidential data. After an analysis of the literature to highlight the obstacles to the generalization of the use of the blockchain in the supply chain, a methodology of adoption and application of the Blockchain following eight steps is proposed with an emphasis on the smart contract.
Incomplete Cigarette Code Recognition via Unified SPA Features and Graph Space Constraints

Cigarette code is a 32-character string printed on a cigarette package, which can be used by tobacco administrations to determine the legality of distribution. Unfortunately, the recognition task for incomplete cigarette code often suffers from lowered recognition accuracy and the destruction of semantic context due to complex backgrounds and damaged characters. This paper proposes an end-to-end recognition network for incomplete cigarette code to improve recognition accuracy and estimate character landmarks. The proposed network first extracts multi-scale features using feature pyramid networks (FPN), then utilizes a spatial attention (SPA) mechanism to yield unified SPA features and integrates them into instance segmentation. This strengthens spatial representation ability and improves the recognition accuracy. A graph convolutional network (GCN) is introduced to construct graph space constraints and calculate character spatial correlations and accurately estimates missing character landmarks. Finally, we employ the Hungarian algorithm to align recognition characters with estimated landmarks and fill missing characters with ‘*’ to preserve the complete semantic context, and produce the final regularized cigarette code. The experimental results demonstrate that our proposed network reduces time consumption and improves recognition accuracy, surpassing the state-of-the-art methods.
COVID-19 MT Evaluator: A Platform for the Evaluation of Machine Translation of Public Health Information Related to COVID-19

In the wake of the COVID-19 outbreak, timely and accurate translation of public health information has become more critical than ever as it helps keep the public informed of the latest pandemic situation, personal hygiene advice and preventive measures, enabling speakers of different languages in the community to work together in the fight against the epidemic. While recent advances in machine translation (MT) featuring the use of deep artificial neural networks may enhance the cross-lingual dissemination of pandemic-related information, their overall performance in English/Chinese translation of COVID-19-related messages is largely understudied. It is against this background that in this article we present COVID-19 MT Evaluator, a platform for the evaluation of MT of public health information related to the pandemic. The system integrates major well-tested MT evaluation metrics and test datasets, with a view to facilitating the assessment of the quality of MT output and the detection and analysis of translation issues. We compare two English/Chinese MT products provided by Google Translate to illustrate the use of the evaluation platform.
ConQeng: A Middleware for Quality of Context Aware Selection, Measurement and Validation

A set of quality metrics (e.g., timeliness, completeness) together represent the Quality of Context (QoC); their values determine the usability of context to context consumers (IoT applications). Therefore, obtaining adequate ‘QoC from the context providers (context sources) represents a significant research challenge. This paper presents a framework called conQeng that addresses such a challenge through novel approaches in QoC-aware selection, QoC measurement and validation. ConQeng selects the potential context providers that deliver an adequate QoC during runtime, assesses their performance - for further selection, and transfers QoC-assured context to the context management platforms (CMPs). We have implemented conQeng in a simulated scenario involving autonomous cars, marketing service agencies as context consumers, and thermal and video cameras as context providers. The results demonstrate that it outperforms three heuristic approaches in reducing context acquisition cost and improving effectiveness and performance efficiency while obtaining adequate QoC.
Clinical Phenotyping Prediction via Auxiliary Task Selection and Adaptive Shared-Space Correction

Clinical Phenotyping is a fundamental task in clinical services, which assessments whether a patient suffers a medical condition of interest. Existing works focus on learning better patients’ representations. Recently, multi-task learning has been proposed to transfer knowledge from different tasks and achieved promising performance. However, the existing multi-task models still suffer from the serious negative transfer and slow convergence problem when multiple phenotype tasks are trained together. Meanwhile, phenotype relatedness is ignored, limiting to boost the performance of the multi-task learning for the phenotype prediction. To address these issues, we propose a private-shared multitask framework with auxiliary task selection and adaptive shared-space correction for phenotype prediction (MTL_AC). To start with, we design an auxiliary task selection method to find the most compatible phenotype task against one task by using phenotype relatedness. And then, a novel adaptive shared-space correction mechanism is proposed to address the negative transfer and slow convergence problem when two tasks are jointly trained under the private-shared multitask framework. The experimental results show that the proposed method performs better on various phenotype prediction tasks.
Visual Perception Inference on Raven’s Progressive Matrices by Semi-supervised Contrastive Learning

The current deep neural networks (DNNs) in mimicking human perception remain challenges for solving visual reasoning tasks. Human perception does not merely involve a passive observer labeling sensory signals, but also contains an active inference about object attributes and their relationships towards an intended output (e.g., an action). In this work, we propose a variational autoencoder (VAE) model to discriminate the ranking relationships between object attribute values by semi-supervised contrastive learning, dubbed as SSCL-VAE. This perception-based model solves the visual reasoning task of Raven’s Progressive Matrices (RPM) in three benchmarks (RAVEN, I-RAVEN and RAVEN-Fair), with high accuracy close to humans, as well as many end-to-end supervised models. The current work thus suggests that constructions of general cognitive abilities like human perception may empower the perceptron with DNN to solve high-level cognitive tasks such as abstract visual reasoning in a human-like manner.
FIGCI: Flow-Based Information-Geometric Causal Inference

This paper is concerned with causal discovery between two random variables X and Y with observational data. Information-Geometric Causal Inference (IGCI) is a well-established method to identify the causal direction between two variables. It assumes the cause distribution and causal mechanism are independent. However, IGCI requires the causal mechanism to be a diffeomorphism function. Fortunately, flow-based models are designed to be differentiable and have a differentiable inverse with a large capacity. We propose Flow-based IGCI (FIGCI). First, the flow-based model fits an invertible mapping between X and Y with two proposed training strategies. Second, FIGCI predicts the causal direction according to the estimated covariances between X as well as Y and the invertible mapping. Empirical studies exemplify the efficacy of FIGCI.
Study on Practical Utility of Image Dehazing Algorithms Based on Deep Learning in Computer Vision Scene Understanding

Deep learning-based methods have achieved remarkable performance for single image dehazing. However, previous studies are mostly focused on the dehazing of human-perspective images without considering the practical application in the field of computer vision. Whether the image dehazing of deep learning-based methods(IDDL) can be applied in the field of computer vision will determine the research direction of computer vision dehazing scenes. This work presents a comprehensive study that the feasibility and reliability of applying IDDL from human vision to practical computer vision scenarios. An approach for practical utility and portability evaluation of the IDDL based on human vision, including the comparison regarding the detection accuracy, model running time, and model scale of the detection method, is proposed. Our research focuses on re-evaluating the applicability of image dehazing algorithms based on a computer vision perspective, including evaluation method and evaluation index. To better verify the detection effect of the dehazing algorithm, we use the synthetic dataset method as a control experiment and make a comparison with 6 state-of-the-art dehazing methods on a real-world hazy scene. Additionally, investigate the proportion of synthetic hazy images in the training dataset used, including object detection and semantic segmentation algorithms. Extensive experiments demonstrate that the poor practical utility and portability of IDDL in computer vision are verifiable, and the IDDL solution should be extensively paid attention to concerning the computer vision domain rather than leveraging the IDDL based on human vision. Finally, We provide some methods to solve the hazy scene.
Integrative Analysis of Multi-view Histopathological Image Features for the Diagnosis of Lung Cancer

Lung cancer is one of the most widely spread cancers in the world. So far, the histopathological image remains the “gold standard” in diagnosing lung cancers, and multiple types of pathological images features have been associated with lung cancer diagnosis and progression. However, most of the existing studies only utilized single type of image features, which did not take advantages of multiple types of image features. In this paper, we propose a Block based Multi-View Graph Convolutional Network (i.e., BMVGCN), which integrates multiple types of image features from histopathological images for lung cancer diagnosis. Specifically, our method utilizes the block-based bilinear combination model to fuse different types of features. By considering the correlation among different samples, we also introduce the Graph Convolutional Network to exploit the correlations among samples that could lead to better diagnosis performance. To evaluate the effectiveness of the proposed method, we conduct the experiments for the classification of the cancer tissue and non-cancer tissue in both Lung Adenocarcinoma (i.e., LUAD) and Lung Squamous Cell Carcinoma (i.e.,LUSC), and the discrimination between LUAD and LUSC. The results show that our method can achieve superior classification performance than the comparing methods.
Following the Lecturer: Hierarchical Knowledge Concepts Prediction for Educational Videos

With an irresistible trend of intelligent learning, predicting knowledge concepts for educational videos turns out to be a fundamental and essential task, which benefits personalized recommendation, retrieval, and learning. Prior studies of videos mainly focus on relatively short human actions and object recognition, while educational videos are minutes long and have heterogeneous elements such as texts, formulas, and hand-drawn graphics that serve lecturers’ narration. Owing to the characteristics of education, most of the segmentation strategies for long-term videos do not apply well to educational videos. In addition, educational videos consist of progressive or referential sections and contain multimodal information. Thus, we propose a novel framework called Spotlight Flow Network (SFNet) to obtain hierarchical knowledge concepts for educational videos with multi-modality. Specifically, we first adopt an effective text-to-visual section segmentation strategy. Then, we model the mechanism that the viewers’ spotlight follows the lecturer and leverage the associations between sections to enhance multimodal representation. We also consider explicit inter-level constraints of the hierarchical knowledge structure and associations between sections and concepts to get better predicting performance. Extensive experimental results on real-world data demonstrate the effectiveness of SFNet.
Image Data Recoverability Against Data Collaboration and Its Countermeasure

The development machine learning and related techniques has accelerated the use of data in a variety of fields, including medicine, finance, and advertising. Because the amount of data is increasing extremely rapidly, it is often necessary to integrate data from multiple organizations for analysis. Accordingly, distributed data analysis methods that enable data analysis while protecting the privacy of the data are being prioritized. A method called data collaboration, in which data is dimensionally reduced and shared as an intermediate representation, has been proposed as a non-model-sharing learning method. In this paper, we propose an attack model based on a generative adversarial network (GAN) to evaluate the security of this new framework. We also propose a method to improve the security of data collaboration. In addition, we experimentally evaluated the risk of reconstructing the original data from the intermediate representation and determine the effect of privacy protection using an attack model.
ETH-TT: A Novel Approach for Detecting Ethereum Malicious Accounts

Ethereum, as the second generation of blockchain technology, it not only brings many advantages, but also spawns various malicious incidents. Ethereum’s anonymity makes it a hotbed of cybercrime, causing huge losses to users and severely disrupting the Ethereum ecosystem. To this end, this paper proposes a method for detecting malicious accounts in Ethereum based on ETH tracking tree (ETH-TT). Firstly, based on the transaction history replay mechanism, an ETH tracking algorithm for tracking the transaction amount of Ethereum is designed to obtain the ETH tracking tree, and extract sequence features from it. Then train the LSTM model to reduce the dimension of the sequence features to obtain the output features. Finally, detection is done by a machine learning classifier, fused with manual features from account transaction history. We uses 5576 malicious accounts and 4968 normal accounts as dataset for experiments. The results show that the ETH-TT method can achieve an F1-score of 95.4% with the cooperation of the XGBoost classifier, which is better than the detection method using only manual features.
An Optical Satellite Controller Based on Diffractive Deep Neural Network

Formulated as an optimal control problem, space relative trajectory planning is crucial for on-orbit servicing spacecraft on various missions. While a variety of deep-neural-network (DNN) methods have been proposed to solve the problem, they are energy-consuming and computationally consuming, which limits their on-board deployments. In this work, we proposed a kind of diffractive-deep-neural-network-based optical satellite controller, and transformed the solution of the optimal control problem into a light-field-based fine-grained regression task. Firstly, an electro-optic conversion module was designed to convert numerical relative state variables from electronic signals into light field as the input of a diffractive modulation (DM) module, the diffractive masks of which could be trained to implement complex light field transformation. We used another optic-electro conversion module to convert the light field at the output plane of DM module into electronic signals. Then, we trained the DM module to make the decoded electrical signals consistent with the desired optimal control commands. Therefore, when the light carrying input information and propagating through the well-trained diffractive masks, the DM module could perform diffract-based solution of optimal control problem. The simulation results substantiate the feasibility and effectiveness of our OC-Nets, which can achieve comparable performance to the latest classic DNN methods, except for a few acceptable errors. Different from classic models with much too energy consumption, once fabricated physically, the device of our optical controller can provide optimal control commands at the speed of light, with fairly little computational and energy consumption, and enable the on-board deployment on spacecraft.
CMOS-GAN: Semi-Supervised Generative Adversarial Model for Cross-Modality Face Image Synthesis

Cross-modality face image synthesis such as sketch-to-photo, NIR-to-RGB, and RGB-to-depth has wide applications in face recognition, face animation, and digital entertainment. Conventional cross-modality synthesis methods usually require paired training data, i.e., each subject has images of both modalities. However, paired data can be difficult to acquire, while unpaired data commonly exist. In this paper, we propose a novel semi-supervised cross-modality synthesis method (namely CMOS-GAN), which can leverage both paired and unpaired face images to learn a robust cross-modality synthesis model. Specifically, CMOS-GAN uses a generator of encoder-decoder architecture for new modality synthesis. We leverage pixel-wise loss, adversarial loss, classification loss, and face feature loss to exploit the information from both paired multi-modality face images and unpaired face images for model learning. In addition, since we expect the synthetic new modality can also be helpful for improving face recognition accuracy, we further use a modified triplet loss to retain the discriminative features of the subject in the synthetic modality. Experiments on three cross-modality face synthesis tasks (NIR-to-VIS, RGB-to-depth, and sketch-to-photo) show the effectiveness of the proposed approach compared with the state-of-the-art. In addition, we also collect a large-scale RGB-D dataset (VIPL-MumoFace-3K) for the RGB-to-depth synthesis task. We plan to open-source our code and VIPL-MumoFace-3K dataset to the community.
Learning Privacy-Preserving Student Networks via Discriminative-Generative Distillation

While deep models have proved successful in learning rich knowledge from massive well-annotated data, they may pose a privacy leakage risk in practical deployment. It is necessary to find an effective trade-off between high utility and strong privacy. In this work, we propose a discriminative-generative distillation approach to learn privacy-preserving deep models. Our key idea is taking models as bridge to distill knowledge from private data and then transfer it to learn a student network via two streams. First, discriminative stream trains a baseline classifier on private data and an ensemble of teachers on multiple disjoint private subsets, respectively. Then, generative stream takes the classifier as a fixed discriminator and trains a generator in a data-free manner. After that, the generator is used to generate massive synthetic data which are further applied to train a variational autoencoder (VAE). Among these synthetic data, a few of them are fed into the teacher ensemble to query labels via differentially private aggregation, while most of them are embedded to the trained VAE for reconstructing synthetic data. Finally, a semi-supervised student learning is performed to simultaneously handle two tasks: knowledge transfer from the teachers with distillation on few privately labeled synthetic data, and knowledge enhancement with tangent-normal adversarial regularization on many triples of reconstructed synthetic data. In this way, our approach can control query cost over private data and mitigate accuracy degradation in a unified manner, leading to a privacy-preserving student model. Extensive experiments and analysis clearly show the effectiveness of the proposed approach.
Deep Embedding with Class

The two flavours of DSL embedding are shallow and deep embedding. In functional languages, shallow embedding models the language constructs as functions in which the semantics are embedded. Adding semantics is therefore cumbersome while adding constructs is a breeze. Upgrading the functions to type classes lifts this limitation to a certain extent. Deeply embedded languages represent their language constructs as data and the semantics are functions on it. As a result, the language constructs are embedded in the semantics, hence adding new language constructs is laborious where adding semantics is trouble free. This paper shows that by abstracting the semantics functions in deep embedding to type classes, it is possible to easily add language constructs as well. So-called classy deep embedding results in DSLs that are extensible both in language constructs and in semantics while maintaining a concrete abstract syntax tree. Additionally, little type-level trickery or complicated boilerplate code is required to achieve this.