Reviewerly/exHarmony huggingface dataset:
Learning Accurate Dense Correspondences and When to Trust Them

Establishing dense correspondences between a pair of images is an important and general problem. However, dense flow estimation is often inaccurate in the case of large displacements or homogeneous regions. For most applications and down-stream tasks, such as pose estimation, image manipulation, or 3D reconstruction, it is crucial to know when and where to trust the estimated matches.In this work, we aim to estimate a dense flow field relating two images, coupled with a robust pixel-wise confidence map indicating the reliability and accuracy of the prediction. We develop a flexible probabilistic approach that jointly learns the flow prediction and its uncertainty. In particular, we parametrize the predictive distribution as a constrained mixture model, ensuring better modelling of both accurate flow predictions and outliers. Moreover, we develop an architecture and training strategy tailored for robust and generalizable uncertainty prediction in the context of self-supervised training. Our approach obtains state- of-the-art results on multiple challenging geometric matching and optical flow datasets. We further validate the usefulness of our probabilistic confidence estimation for the task of pose estimation. Code and models are available at https://github.com/PruneTruong/PDCNet.
InverseForm: A Loss Function for Structured Boundary-Aware Segmentation

We present a novel boundary-aware loss term for semantic segmentation using an inverse-transformation network, which efficiently learns the degree of parametric transformations between estimated and target boundaries. This plug-in loss term complements the cross-entropy loss in capturing boundary transformations and allows consistent and significant performance improvement on segmentation backbone models without increasing their size and computational complexity. We analyze the quantitative and qualitative effects of our loss function on three indoor and outdoor segmentation benchmarks, including Cityscapes, NYU-Depth-v2, and PASCAL, integrating it into the training phase of several backbone networks in both single-task and multi-task settings. Our extensive experiments show that the proposed method consistently outperforms base-lines, and even sets the new state-of-the-art on two datasets.
MFFENet: Multiscale Feature Fusion and Enhancement Network For RGB–Thermal Urban Road Scene Parsing

Compared with traditional handcrafted features, deep learning has greatly improved the performance of scene parsing. However, it remains challenging under various environmental conditions caused by imaging limitations. Thermal imaging cameras have several advantages over cameras for the visible spectrum, such as operation in total darkness, robustness to shadow effects, insensitivity to illumination variations, and strong ability to penetrate smog and haze. These advantages of thermal imaging cameras make them ideal for the scene parsing of semantic objects in daytime and nighttime. In this paper, we propose a novel multiscale feature fusion and enhancement network (MFFENet) for accurate parsing of RGB&#x2013;thermal urban road scenes even when the quality of the available RGB data is compromised. The proposed MFFENet consists of two encoders, a feature fusion layer, and a multi-label supervision layer. We concatenate the multi-scale features with the features that contain global semantic information. Furthermore, we explore the cross-modal fusion of RGB and thermal features at multiple stages, rather than fusing them once at the low or high stage. Then, we propose a spatial attention mechanism module that provides a higher weight to (focuses more on) the foreground area, allowing MFFENet to emphasize foreground objects. Finally, multi-label supervision is introduced to optimize parameters of the proposed MFFENet. Experimental results confirm that the proposed MFFENet outperforms similar high-performing methods.
Making a Case for Federated Learning in the Internet of Vehicles and Intelligent Transportation Systems

With the incoming introduction of 5G networks and the advancement in technologies such as network function virtualization and software defined networking, new and emerging networking technologies and use cases are taking shape. One such technology is the Internet of Vehicles (IoV), which describes an interconnected system of vehicles and infrastructure. Coupled with recent developments in artificial intelligence and machine learning, IoV is transformed into an intelligent transportation system (ITS). There are, however, several operational considerations that hinder the adoption of ITSs, including scalability, high availability, and data privacy. To address these challenges, federated learning, a collaborative and distributed intelligence technique, is suggested. Through an ITS case study, the ability of a federated model deployed on roadside infrastructure throughout the network to recover from faults by leveraging group intelligence while reducing recovery time and restoring acceptable system performance is highlighted. With a multitude of use cases and benefits, federated learning is a key enabler for ITS and is poised to achieve widespread implementation in 5G and beyond networks and applications.
PySyft: A Library for Easy Federated Learning

PySyft is an open-source multi-language library enabling secure and private machine learning by wrapping and extending popular deep learning frameworks such as PyTorch in a transparent, lightweight, and user-friendly manner. Its aim is to both help popularize privacy-preserving techniques in machine learning by making them as accessible as possible via Python bindings and common tools familiar to researchers and data scientists, as well as to be extensible such that new Federated Learning (FL), Multi-Party Computation, or Differential Privacy methods can be flexibly and simply implemented and integrated. This chapter will introduce the methods available within the PySyft library and describe their implementations. We will then provide a proof-of-concept demonstration of a FL workflow using an example of how to train a convolutional neural network. Next, we review the use of PySyft in academic literature to date and discuss future use-cases and development plans. Most importantly, we introduce Duet: our tool for easier FL for scientists and data owners.
A silver lining in the COVID-19 cloud: examining customers’ value perceptions, willingness to use and pay more for robotic restaurants

The COVID-19 pandemic has severely affected the restaurant industry due to enforced closures and limitations on social gatherings, prompting restaurateurs to innovate and adapt in order to ensure the viability of their businesses. Pandemic has also induced changes in our perceptions of safety in public spaces, necessitating the adoption of social distancing and more widespread use of online platforms for purchasing and communication. While the pandemic might be a catalyst for the adoption of contactless technologies, some restaurateurs remain hesitant to invest in service robots because they are not convinced of the return on investment and the potential value service robots can deliver to their customers. Therefore, this study aims to explore customer value perceptions of service robots and their impact on customers' attitudes and behaviors toward robotic restaurants. Findings yielded by a survey of 445 potential diners in Taiwan show that customers' willingness to use and to pay more for robotic restaurants are determined by their attitudes toward robots, which are influenced by functional, conditional, epistemic, emotional, co-creation, and social values. Our survey results also reveal that the importance of conditional value is amplified by crisis-specific antecedents, namely the need for physical distancing and mysophobia. These findings have implications for restaurant pricing policies and can be considered by restaurant managers when formulating strategies aimed at sustaining their business in these challenging times.
Privacy-Preserved Federated Learning for Autonomous Driving

In recent years, the privacy issue in Vehicular Edge Computing (VEC) has gained a lot of concern. The privacy problem is even more severe in autonomous driving business than the other businesses in VEC such as ordinary navigation. Federated learning (FL), which is a privacy-preserved strategy proposed by Google, has become a hot trend to solve the privacy problem in many fields including VEC. Therefore, we introduce FL into autonomous driving to preserve vehicular privacy by keeping original data in a local vehicle and sharing the training model parameter only with the help of MEC server. Moreover, different from the common assumption of honest MEC server and honest vehicle in former studies, we take the malicious MEC servers and malicious vehicles into account. First, we consider honest-but-curious MEC server and malicious vehicles and propose a traceable identity-based privacy preserving scheme to protect the vehicular message privacy where improved Dijk-Gentry-Halevi-Vaikutanathan (DGHV) algorithm is proposed and a blockchain-based Reputation-based Incentive Autonomous Driving Mechanism (RIADM) is adopted. Further, when the case comes to the non-credibility of both parties where semi-honest MEC server and malicious vehicles are considered, we propose an anonymous identity-based privacy preserving scheme to protect the identity privacy of vehicles with Zero-Knowledge Proof (ZKP). Based on the simulation of virtual autonomous driving based on real-world road images, it is verified that our proposes scheme can reduce 73.7 % training loss of autonomous driving, increase the accuracy to around 5.55 % while keeps effective privacy of message and identity under the threat of dishonest MEC server and vehicles.
Populating 3D Scenes by Learning Human-Scene Interaction

Humans live within a 3D space and constantly interact with it to perform tasks. Such interactions involve physical contact between surfaces that is semantically meaningful. Our goal is to learn how humans interact with scenes and leverage this to enable virtual characters to do the same. To that end, we introduce a novel Human-Scene Interaction (HSI) model that encodes proximal relationships, called POSA for "Pose with prOximitieS and contActs". The representation of interaction is body-centric, which enables it to generalize to new scenes. Specifically, POSA augments the SMPL-X parametric human body model such that, for every mesh vertex, it encodes (a) the contact probability with the scene surface and (b) the corresponding semantic scene label. We learn POSA with a VAE conditioned on the SMPL-X vertices, and train on the PROX dataset, which contains SMPL-X meshes of people interacting with 3D scenes, and the corresponding scene semantics from the PROX-E dataset. We demonstrate the value of POSA with two applications. First, we automatically place 3D scans of people in scenes. We use a SMPL-X model fit to the scan as a proxy and then find its most likely placement in 3D. POSA provides an effective representation to search for "affordances" in the scene that match the likely contact relationships for that pose. We perform a perceptual study that shows significant improvement over the state of the art on this task. Second, we show that POSA’s learned representation of body-scene interaction supports monocular human pose estimation that is consistent with a 3D scene, improving on the state of the art. Our model and code are available for research purposes at https://posa.is.tue.mpg.de.
Towards Resolving the Challenge of Long-tail Distribution in UAV Images for Object Detection

Existing methods for object detection in UAV images ignored an important challenge - imbalanced class distribution in UAV images - which leads to poor performance on tail classes. We systematically investigate existing solutions to long-tail problems and unveil that re-balancing methods that are effective on natural image datasets cannot be trivially applied to UAV datasets. To this end, we rethink longtailed object detection in UAV images and propose the Dual Sampler and Head detection Network (DSHNet), which is the first work that aims to resolve long-tail distribution in UAV images. The key components in DSHNet include Class-Biased Samplers (CBS) and Bilateral Box Heads (BBH), which are developed to cope with tail classes and head classes in a dual-path manner. Without bells and whistles, DSHNet significantly boosts the performance of tail classes on different detection frameworks. Moreover, DSHNet significantly outperforms base detectors and generic approaches for long-tail problems on VisDrone and UAVDT datasets. It achieves new state-of-the-art performance when combining with image cropping methods. Code is available at https://github.com/we1pingyu/DSHNet.
Model-based 3D Hand Reconstruction via Self-Supervised Learning

Reconstructing a 3D hand from a single-view RGB image is challenging due to various hand configurations and depth ambiguity. To reliably reconstruct a 3D hand from a monocular image, most state-of-the-art methods heavily rely on 3D annotations at the training stage, but obtaining 3D annotations is expensive. To alleviate reliance on labeled training data, we propose S <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> HAND, a self-supervised 3D hand reconstruction network that can jointly estimate pose, shape, texture, and the camera viewpoint. Specifically, we obtain geometric cues from the input image through easily accessible 2D detected keypoints. To learn an accurate hand reconstruction model from these noisy geometric cues, we utilize the consistency between 2D and 3D representations and propose a set of novel losses to rationalize outputs of the neural network. For the first time, we demonstrate the feasibility of training an accurate 3D hand reconstruction network without relying on manual annotations. Our experiments show that the proposed self-supervised method achieves comparable performance with recent fully-supervised methods. The code is available at https://github.com/TerenceCYJ/S2HAND.
Stereoscopic Image Description With Trinion Fractional-Order Continuous Orthogonal Moments

Some research progress has been made on fractional-order continuous orthogonal moments (FrCOMs) in the past two years. Compared with integer-order continuous orthogonal moments (InCOMs), FrCOMs increase the number of affine invariants and effectively improve numerical stability. However, the existing types of FrCOMs are still very limited, of which all are planar image oriented. No report on stereoscopic images is available yet. To this end, in this paper, FrCOMs corresponding to various types of InCOMs are first deduced, and then, they are combined with trinion theory to construct trinion FrCOMs (TFrCOMs) applicable to stereoscopic images. Furthermore, the reconstruction performance and geometric invariance of TFrCOMs are analyzed theoretically and experimentally. Finally, an application in the stereoscopic image zero-watermarking algorithm is investigated to verify the superior performance of TFrCOMs.
Virtual reality tourism experiences: Addiction and isolation

This qualitative research note reports two neglected themes in research on virtual reality tourism experiences, i.e. its potentially addictive nature and temporary sense of isolation. Existing work on virtual reality tourism experiences has applied existing knowledge and theories and has solely tested how VR applications can positively mediate or moderate the tourist experience. This study adopted an inductive approach, analyzing contents of reviews and blogs, and consequently uncovered a temporary sense of isolation and the addictive nature of virtual reality as hidden themes within virtual reality tourism experiences. We stress the importance of further work on addiction and a sense of isolation in terms of their nature, role, and effects.
Adaptive Boundary Proposal Network for Arbitrary Shape Text Detection

Arbitrary shape text detection is a challenging task due to the high complexity and variety of scene texts. In this work, we propose a novel adaptive boundary proposal network for arbitrary shape text detection, which can learn to directly produce accurate boundary for arbitrary shape text without any post-processing. Our method mainly consists of a boundary proposal model and an innovative adaptive boundary deformation model. The boundary proposal model constructed by multi-layer dilated convolutions is adopted to produce prior information (including classification map, distance field, and direction field) and coarse boundary proposals. The adaptive boundary deformation model is an encoder-decoder network, in which the encoder mainly consists of a Graph Convolutional Network (GCN) and a Recurrent Neural Network (RNN). It aims to perform boundary deformation in an iterative way for obtaining text instance shape guided by prior information from the boundary proposal model. In this way, our method can directly and efficiently generate accurate text boundaries without complex post-processing. Extensive experiments on publicly available datasets demonstrate the state-of-the-art performance of our method. Code is available at the website: https://github.com/GXYM/TextBPN.
Decentralized exchanges: The “wild west” of cryptocurrency trading

Cryptocurrencies are traded on two types of exchanges – centralized and decentralized. Although trading in the largest cryptocurrencies primarily occurs on centralized exchanges, most newly issued tokens can only be exchanged using decentralized platforms. Volumes in these decentralized exchanges (including automated market makers) has recently increased exponentially. We examine the role of this new and unmonitored market, utilizing an extensive sample of tokens exclusively traded on decentralized platforms. We show significant differences in the listing and trading characteristics of these tokens relative to their centralized equivalents. A small selection of these tokens obtain listing on a centralized exchange during the sample period, which is accompanied by a significant increase in trading activity, consistent with market segmentation. A centralized listing results in a migration of volume away from decentralized platforms, revealing a strong preference by tokenholders for deeper and more liquid markets over the increased security and anonymity offered by decentralized exchanges.
Efficient Context-Guided Stacked Refinement Network for RGB-T Salient Object Detection

RGB-T salient object detection (SOD) aims at utilizing the complementary cues of RGB and Thermal (T) modalities to detect and segment the common objects. However, on one hand, existing methods simply fuse the features of two modalities without fully considering the characters of RGB and T. On the other hand, the high computational cost of existing methods prevents them from real-world applications (e.g., automatic driving, abnormal detection, person re-ID). To this end, we proposed an efficient encoder-decoder network named Context-guided Stacked Refinement Network (CSRNet). Specifically, we utilize a lightweight backbone and design efficient decoder parts, which greatly reduce the computational cost. To fuse RGB and T modalities, we proposed an efficient Context-guided Cross Modality Fusion (CCMF) module to filter the noise and explore the complementation of two modalities. Besides, Stacked Refinement Network (SRN) progressively refines the features from top to down via the interaction of semantic and spatial information. Extensive experiments show that our method performs favorably against state-of-the-art algorithms on RGB-T SOD task while with small model size (4.6M), few FLOPs (4.2G), and real-time speed (38 <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">fps</i> ). Our codes is available at: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/huofushuo/CSRNet</uri> .
Quantum walk and its application domains: A systematic review

Quantum random walk is the quantum counterpart of a classical random walk. The classical random walk concept has long been used as a computational framework for designing classical algorithms for complex problems. Quantum analogues of random walk provide speed-up in computational power for various algorithms such as element distinctness, spatial search, graph connectivity, etc. Quantum walks have emerged to be a universal computational model over the last decade. Quantum walk formulations applied in graph theory have shown quadratic and polynomial time in graph traversal as opposed to the exponential time taken by classical algorithms. Quantum walk models have also found use in designing quantum computers. Inspired by these facts, this article presents a substantial systematic literature review and analysis of various quantum walk formulations and their strengths and limitations w.r.t. application domains used in literature up-to-date by researchers in various fields. The analysis provided in this article may help upcoming researchers to gain new insights towards the application of quantum walk formulation in varied domains. Various performance metrics, physical implementation set-ups, coin operators, and simulators used to analyze classical and quantum walk dynamics on graphs have been described. Finally, the article discusses existing open problems and notable future directions related to quantum walk application for potential researchers.
Blockchain Technology for Governmental Supervision of Construction Work: Learning from Digital Currency Electronic Payment Systems

Blockchain technology has been explored for governmental supervision of construction work (GSCW) due to its merits of traceability, immutability, and transparency. However, its decentralized nature is seemingly incompatible with GSCW, which is a type of centralized governance per se. This research aims to find a network topology with a proper level of (de)centralization and, based on this topology, to develop a blockchain-based model for GSCW. First, a literature review is conducted to identify problems in GSCW. Then, a cross-sectoral learning is performed between GSCW and digital currency electronic payment systems. Next, a design science research method is adopted to develop a dual-layer blockchain-based GSCW model integrated with an incentive mechanism. Finally, the model is illustrated in Hyperledger Fabric and its strengths and weaknesses evaluated. It was found that the model can enable an information-sharing, tamper-proof, and privacy-preserving mechanism without affecting the current status and routines of GSCW units and project teams. The model developed in our study can serve as a valuable reference for policymakers, practitioners, and researchers to develop governance policies or blockchain applications.
AutoShape: Real-Time Shape-Aware Monocular 3D Object Detection

Existing deep learning-based approaches for monocular 3D object detection in autonomous driving often model the object as a rotated 3D cuboid while the object’s geometric shape has been ignored. In this work, we propose an approach for incorporating the shape-aware 2D/3D constraints into the 3D detection framework. Specifically, we employ the deep neural network to learn distinguished 2D keypoints in the 2D image domain and regress their corresponding 3D coordinates in the local 3D object coordinate first. Then the 2D/3D geometric constraints are built by these correspondences for each object to boost the detection performance. For generating the ground truth of 2D/3D keypoints, an automatic model-fitting approach has been proposed by fitting the deformed 3D object model and the object mask in the 2D image. The proposed framework has been verified on the public KITTI dataset and the experimental results demonstrate that by using additional geometrical constraints the detection performance has been significantly improved as compared to the baseline method. More importantly, the proposed framework achieves state-of-the-art performance with real time. Data and code will be available at https://github.com/zongdai/AutoShape