Reviewerly/exHarmony huggingface dataset:
Bi-Directional Center-Constrained Top-Ranking for Visible Thermal Person Re-Identification

Visible thermal person re-identification (VT-REID) is a task of matching person images captured by thermal and visible cameras, which is an extremely important issue in night-time surveillance applications. Existing cross-modality recognition works mainly focus on learning sharable feature representations to handle the cross-modality discrepancies. However, apart from the cross-modality discrepancy caused by different camera spectrums, VT-REID also suffers from large cross-modality and intra-modality variations caused by different camera environments and human poses, and so on. In this paper, we propose a dual-path network with a novel bi-directional dual-constrained top-ranking (BDTR) loss to learn discriminative feature representations. It is featured in two aspects: 1) end-to-end learning without extra metric learning step and 2) the dual-constraint simultaneously handles the cross-modality and intra-modality variations to ensure the feature discriminability. Meanwhile, a bi-directional center-constrained top-ranking (eBDTR) is proposed to incorporate the previous two constraints into a single formula, which preserves the properties to handle both cross-modality and intra-modality variations. The extensive experiments on two cross-modality re-ID datasets demonstrate the superiority of the proposed method compared to the state-of-the-arts.
Self-Supervised Learning With Geometric Constraints in Monocular Video: Connecting Flow, Depth, and Camera

We present GLNet, a self-supervised framework for learning depth, optical flow, camera pose and intrinsic parameters from monocular video - addressing the difficulty of acquiring realistic ground-truth for such tasks. We propose three contributions: 1) we design new loss functions that capture multiple geometric constraints (eg. epipolar geometry) as well as adaptive photometric loss that supports multiple moving objects, rigid and non-rigid, 2) we extend the model such that it predicts camera intrinsics, making it applicable to uncalibrated video, and 3) we propose several online refinement strategies that rely on the symmetry of our self-supervised loss in training and testing, in particular optimizing model parameters and/or the output of different tasks, leveraging their mutual interactions. The idea of jointly optimizing the system output, under all geometric and photometric constraints can be viewed as a dense generalization of classical bundle adjustment. We demonstrate the effectiveness of our method on KITTI and Cityscapes, where we outperform previous self-supervised approaches on multiple tasks. We also show good generalization for transfer learning.
CAMP: Cross-Modal Adaptive Message Passing for Text-Image Retrieval

Text-image cross-modal retrieval is a challenging task in the field of language and vision. Most previous approaches independently embed images and sentences into a joint embedding space and compare their similarities. However, previous approaches rarely explore the interactions between images and sentences before calculating similarities in the joint space. Intuitively, when matching between images and sentences, human beings would alternatively attend to regions in images and words in sentences, and select the most salient information considering the interaction between both modalities. In this paper, we propose Cross-modal Adaptive Message Passing (CAMP), which adaptively controls the information flow for message passing across modalities. Our approach not only takes comprehensive and fine-grained cross-modal interactions into account, but also properly handles negative pairs and irrelevant information with an adaptive gating scheme. Moreover, instead of conventional joint embedding approaches for text-image matching, we infer the matching score based on the fused features, and propose a hardest negative binary cross-entropy loss for training. Results on COCO and Flickr30k significantly surpass state-of-the-art methods, demonstrating the effectiveness of our approach.
Supply Chain Finance Innovation Using Blockchain

Blockchain, the underlying technology of digital currency, such as Bitcoin, has the characteristics of decentralization, stability, security, anonymity, and nontampering. Blockchain is being applied in an increasing number of fields. Supply chain finance is a financing model in which banks connect core enterprises with upstream and downstream enterprises to provide flexible financial products and services. In the traditional supply chain finance, the core enterprises with strong competitiveness and a large scale play an irreplaceable role in managing the supply chain information flow, logistics, and capital flow, which results in inequality and information asymmetry. Fraud in supply chain finance is also very serious. To solve the problems of the traditional supply chain finance, in this article, we build a new type of supply chain financial platform that uses blockchain technology to manage the whole process. This supply chain financial platform solves the problem of nontrust among the participants in the supply chain, improves the efficiency of the capital flow and information flow, reduces costs, and provides better financial services to the relevant parties in the supply chain. To protect users' privacy, we propose a new method of using homomorphic encryption in the blockchain to meet the needs of sensitive data privacy protection in supply chain financial scenarios.
Optimization for Medical Image Segmentation: Theory and Practice When Evaluating With Dice Score or Jaccard Index

In many medical imaging and classical computer vision tasks, the Dice score and Jaccard index are used to evaluate the segmentation performance. Despite the existence and great empirical success of metric-sensitive losses, i.e. relaxations of these metrics such as soft Dice, soft Jaccard and Lovász-Softmax, many researchers still use per-pixel losses, such as (weighted) cross-entropy to train CNNs for segmentation. Therefore, the target metric is in many cases not directly optimized. We investigate from a theoretical perspective, the relation within the group of metric-sensitive loss functions and question the existence of an optimal weighting scheme for weighted cross-entropy to optimize the Dice score and Jaccard index at test time. We find that the Dice score and Jaccard index approximate each other relatively and absolutely, but we find no such approximation for a weighted Hamming similarity. For the Tversky loss, the approximation gets monotonically worse when deviating from the trivial weight setting where soft Tversky equals soft Dice. We verify these results empirically in an extensive validation on six medical segmentation tasks and can confirm that metric-sensitive losses are superior to cross-entropy based loss functions in case of evaluation with Dice Score or Jaccard Index. This further holds in a multi-class setting, and across different object sizes and foreground/background ratios. These results encourage a wider adoption of metric-sensitive loss functions for medical segmentation tasks where the performance measure of interest is the Dice score or Jaccard index.
Path planning optimization of indoor mobile robot based on adaptive ant colony algorithm

In view of the shortcomings of traditional ant colony algorithm (ACO) in path planning of indoor mobile robot, such as a long time path planning, non-optimal path for the slow convergence speed, and local optimal solution characteristic of ACO, an improvement adaptive ant colony algorithm (IAACO) is proposed in this paper. In IAACO, firstly, in order to accelerate the real-time and safety of robot path planning, angle guidance factor and obstacle exclusion factor are introduced into the transfer probability of ACO; secondly, heuristic information adaptive adjustment factor and adaptive pheromone volatilization factor are introduced into the pheromone update rule of ACO, to balance the convergence and global search ability of ACO; Finally, the multi-objective performance indexes are introduced to transform the path planning problem into a multi-objective optimization problem, so as to realize the comprehensive global optimization of robot path planning. The experimental results of main parameters selection, path planning performance in different environments, diversity of the optimal solution show that IAACO can make the robot attain global optimization path, and high real-time and stability performances of path planning.
Semi-Automatic Classification Plugin: A Python tool for the download and processing of remote sensing images in QGIS

Congedo, L., (2021). Semi-Automatic Classification Plugin: A Python tool for the download and processing of remote sensing images in QGIS. Journal of Open Source Software, 6(64), 3172, https://doi.org/10.21105/joss.03172
Digital pathology and artificial intelligence in translational medicine and clinical practice

Traditional pathology approaches have played an integral role in the delivery of diagnosis, semi-quantitative or qualitative assessment of protein expression, and classification of disease. Technological advances and the increased focus on precision medicine have recently paved the way for the development of digital pathology-based approaches for quantitative pathologic assessments, namely whole slide imaging and artificial intelligence (AI)-based solutions, allowing us to explore and extract information beyond human visual perception. Within the field of immuno-oncology, the application of such methodologies in drug development and translational research have created invaluable opportunities for deciphering complex pathophysiology and the discovery of novel biomarkers and drug targets. With an increasing number of treatment options available for any given disease, practitioners face the growing challenge of selecting the most appropriate treatment for each patient. The ever-increasing utilization of AI-based approaches substantially expands our understanding of the tumor microenvironment, with digital approaches to patient stratification and selection for diagnostic assays supporting the identification of the optimal treatment regimen based on patient profiles. This review provides an overview of the opportunities and limitations around implementing AI-based methods in biomarker discovery and patient selection and discusses how advances in digital pathology and AI should be considered in the current landscape of translational medicine, touching on challenges this technology may face if adopted in clinical settings. The traditional role of pathologists in delivering accurate diagnoses or assessing biomarkers for companion diagnostics may be enhanced in precision, reproducibility, and scale by AI-powered analysis tools.
Deep Attention-Based Spatially Recursive Networks for Fine-Grained Visual Recognition

Fine-grained visual recognition is an important problem in pattern recognition applications. However, it is a challenging task due to the subtle interclass difference and large intraclass variation. Recent visual attention models are able to automatically locate critical object parts and represent them against appearance variations. However, without consideration of spatial dependencies in discriminative feature learning, these methods are underperformed in classifying fine-grained objects. In this paper, we present a deep attention-based spatially recursive model that can learn to attend to critical object parts and encode them into spatially expressive representations. Our network is technically premised on bilinear pooling, enabling local pairwise feature interactions between outputs from two different convolutional neural networks (CNNs) that correspond to distinct region detection and relevant feature extraction. Then, spatial long-short term memory (LSTMs) units are introduced to generate spatially meaningful hidden representations via the long-range dependency on all features in two dimensions. The attention model is leveraged between bilinear outcomes and spatial LSTMs for dynamic selection on varied inputs. Our model, which is composed of two-stream CNN layers, bilinear pooling, and spatial recursive encoding with attention, is end-to-end trainable to serve as the part detector and feature extractor whereby relevant features are localized, extracted, and encoded spatially for recognition purpose. We demonstrate the superiority of our method over two typical fine-grained recognition tasks: fine-grained image classification and person re-identification.
Blockchain-Based Settlement for Asset Trading

Can securities be settled on a blockchain, and, if so, what are the gains relative to existing settlement systems? The main benefit of a blockchain is faster and more flexible settlement, whereas settlement fails need to be ruled out where participants fork the chain to cancel trading losses. With a proof-of-work protocol, the blockchain needs to restrict settlement speed through block size and time in order to generate transaction fees, which finance costly mining. Despite mining being a deadweight cost, our estimates for the U.S. corporate debt market yield net gains from a blockchain in the range of 1–4 bps. Received May 31, 2017; editorial decision May 29, 2018 by Editor Itay Goldstein. Authors have furnished an Internet Appendix, which is available on the Oxford University Press Web site next to the link to the final published paper online.
Understanding the use of Virtual Reality in Marketing: A text mining-based review

The current study intends to highlight the most relevant studies in simulated realities with special attention to VR and marketing, showing how studies have evolved over time and discussing the findings. A text-mining approach using a Bayesian statistical topic model called latent Dirichlet allocation is employed to conduct a comprehensive analysis of 150 articles from 115 journals, all indexed in Web of Science. The findings reveal seven relevant topics, as well as the number of articles published over time, the authors most cited in VR papers and the leading journals in each topic. The article also provides theoretical and practical implications and suggestions for further research.
Distributed Resource Allocation in Blockchain-Based Video Streaming Systems With Mobile Edge Computing

Blockchain-based video streaming systems aim to build decentralized peer-to-peer networks with flexible monetization mechanisms for video streaming services. On these blockchain-based platforms, video transcoding, which is computationally intensive and time-consuming, is still a major challenge. Meanwhile, the block size of the underlying blockchain has significant impacts on the system performance. Therefore, this paper proposes a novel blockchain-based framework with an adaptive block size for video streaming with mobile edge computing (MEC). First, we design an incentive mechanism to facilitate collaboration among content creators, video transcoders, and consumers. In addition, we present a block size adaptation scheme for blockchain-based video streaming. Moreover, we consider two offloading modes, i.e., offloading to the nearby MEC nodes or a group of device-to-device (D2D) users, to avoid the overload of MEC nodes. Then, we formulate the issues of resource allocation, scheduling of offloading, and adaptive block size as an optimization problem. We employ a low-complexity alternating direction method of the multipliers-based algorithm to solve the problem in a distributed fashion. Simulation results are presented to show the effectiveness of the proposed scheme.
Accounting and Auditing at the Time of Blockchain Technology: A Research Agenda

Abstract Blockchain is a distributed ledger technology expected to have significant impacts on the accounting and auditing profession. This study, applicable and timely for both accounting and auditing scholars and practitioners, explores blockchain technology and its main implications for the accounting and auditing profession. The research question addressed in this study is: What are the major themes emerging from academic research and professional reports and websites debating blockchain technology in the accounting and auditing context? A literature review of academic literature and professional reports and websites is performed to identify a taxonomy of emerging themes. The study finds that the most discussed themes in scholarly works and professional sources are governance, transparency and trust issues in the blockchain ecosystem, blockchain‐enabled continuous audits, smart contract applications and the paradigmatic shift in accountants' and auditors' roles . Based on these four themes, practical implications for accountants and auditors on how to approach the blockchain development are provided. Moreover, this study offers suggestions for future research on accounting and auditing in the blockchain era.
Using Presence Questionnaires in Virtual Reality

Virtual Reality (VR) is gaining increasing importance in science, education, and entertainment. A fundamental characteristic of VR is creating presence, the experience of 'being' or 'acting', when physically situated in another place. Measuring presence is vital for VR research and development. It is typically repeatedly assessed through questionnaires completed after leaving a VR scene. Requiring participants to leave and re-enter the VR costs time and can cause disorientation. In this paper, we investigate the effect of completing presence questionnaires directly in VR. Thirty-six participants experienced two immersion levels and filled three standardized presence questionnaires in the real world or VR. We found no effect on the questionnaires' mean scores; however, we found that the variance of those measures significantly depends on the realism of the virtual scene and if the subjects had left the VR. The results indicate that, besides reducing a study's duration and reducing disorientation, completing questionnaires in VR does not change the measured presence but can increase the consistency of the variance.
Bi-Directional Cascade Network for Perceptual Edge Detection

Exploiting multi-scale representations is critical to improve edge detection for objects at different scales. To extract edges at dramatically different scales, we propose a Bi-Directional Cascade Network (BDCN) structure, where an individual layer is supervised by labeled edges at its specific scale, rather than directly applying the same supervision to all CNN outputs. Furthermore, to enrich multi-scale representations learned by BDCN, we introduce a Scale Enhancement Module (SEM) which utilizes dilated convolution to generate multi-scale features, instead of using deeper CNNs or explicitly fusing multi-scale edge maps. These new approaches encourage the learning of multi-scale representations in different layers and detect edges that are well delineated by their scales. Learning scale dedicated layers also results in compact network with a fraction of parameters. We evaluate our method on three datasets, i.e., BSDS500, NYUDv2, and Multicue, and achieve ODS Fmeasure of 0.828, 1.3% higher than current state-of-the art on BSDS500.
Knowledge Distillation via Instance Relationship Graph

The key challenge of knowledge distillation is to extract general, moderate and sufficient knowledge from a teacher network to guide a student network. In this paper, a novel Instance Relationship Graph (IRG) is proposed for knowledge distillation. It models three kinds of knowledge, including instance features, instance relationships and feature space transformation, while the latter two kinds of knowledge are neglected by previous methods. Firstly, the IRG is constructed to model the distilled knowledge of one network layer, by considering instance features and instance relationships as vertexes and edges respectively. Secondly, an IRG transformation is proposed to models the feature space transformation across layers. It is more moderate than directly mimicking the features at intermediate layers. Finally, hint loss functions are designed to force a student's IRGs to mimic the structures of a teacher's IRGs. The proposed method effectively captures the knowledge along the whole network via IRGs, and thus shows stable convergence and strong robustness to different network architectures. In addition, the proposed method shows superior performance over existing methods on datasets of various scales.
EPIC-Fusion: Audio-Visual Temporal Binding for Egocentric Action Recognition

We focus on multi-modal fusion for egocentric action recognition, and propose a novel architecture for multimodal temporal-binding, i.e. the combination of modalities within a range of temporal offsets. We train the architecture with three modalities - RGB, Flow and Audio - and combine them with mid-level fusion alongside sparse temporal sampling off used representations. In contrast with previous works, modalities are fused before temporal aggregation, with shared modality and fusion weights over time. Our proposed architecture is trained end-to-end, outperforming individual modalities as well as late-fusion of modalities. We demonstrate the importance of audio in egocentric vision, on per-class basis, for identifying actions as well as interacting objects. Our method achieves state of the art results on both the seen and unseen test sets of the largest egocentric dataset: EPIC-Kitchens, on all metrics using the public leaderboard.
Adversarial Feature Hallucination Networks for Few-Shot Learning

The recent flourish of deep learning in various tasks is largely accredited to the rich and accessible labeled data. Nonetheless, massive supervision remains a luxury for many real applications, boosting great interest in label-scarce techniques such as few-shot learning (FSL), which aims to learn concept of new classes with a few labeled samples. A natural approach to FSL is data augmentation and many recent works have proved the feasibility by proposing various data synthesis models. However, these models fail to well secure the discriminability and diversity of the synthesized data and thus often produce undesirable results. In this paper, we propose Adversarial Feature Hallucination Networks (AFHN) which is based on conditional Wasserstein Generative Adversarial networks (cWGAN) and hallucinates diverse and discriminative features conditioned on the few labeled samples. Two novel regularizers, i.e., the classification regularizer and the anti-collapse regularizer, are incorporated into AFHN to encourage discriminability and diversity of the synthesized features, respectively. Ablation study verifies the effectiveness of the proposed cWGAN based feature hallucination framework and the proposed regularizers. Comparative results on three common benchmark datasets substantiate the superiority of AFHN to existing data augmentation based FSL approaches and other state-of-the-art ones.
Dynamic Multiscale Graph Neural Networks for 3D Skeleton Based Human Motion Prediction

We propose novel dynamic multiscale graph neural networks (DMGNN) to predict 3D skeleton-based human motions. The core idea of DMGNN is to use a multiscale graph to comprehensively model the internal relations of a human body for motion feature learning. This multiscale graph is adaptive during training and dynamic across network layers. Based on this graph, we propose a multiscale graph computational unit (MGCU) to extract features at individual scales and fuse features across scales. The entire model is action-category-agnostic and follows an encoder-decoder framework. The encoder consists of a sequence of MGCUs to learn motion features. The decoder uses a proposed graph-based gate recurrent unit to generate future poses. Extensive experiments show that the proposed DMGNN outperforms state-of-the-art methods in both short and long-term predictions on the datasets of Human 3.6M and CMU Mocap. We further investigate the learned multiscale graphs for the interpretability. The codes could be downloaded from https://github.com/limaosen0/DMGNN.