Reviewerly/exHarmony huggingface dataset:
Fine-Grained Video-Text Retrieval With Hierarchical Graph Reasoning

Cross-modal retrieval between videos and texts has attracted growing attentions due to the rapid emergence of videos on the web. The current dominant approach is to learn a joint embedding space to measure cross-modal similarities. However, simple embeddings are insufficient to represent complicated visual and textual details, such as scenes, objects, actions and their compositions. To improve fine-grained video-text retrieval, we propose a Hierarchical Graph Reasoning (HGR) model, which decomposes video-text matching into global-to-local levels. The model disentangles text into a hierarchical semantic graph including three levels of events, actions, entities, and generates hierarchical textual embeddings via attention-based graph reasoning. Different levels of texts can guide the learning of diverse and hierarchical video representations for cross-modal matching to capture both global and local details. Experimental results on three video-text datasets demonstrate the advantages of our model. Such hierarchical decomposition also enables better generalization across datasets and improves the ability to distinguish fine-grained semantic differences. Code will be released at https: //github.com/cshizhe/hgr_v2t.
Any port in a storm: Cryptocurrency safe-havens during the COVID-19 pandemic

Controlling for the polarity and subjectivity of social media data based on the development of the COVID-19 outbreak, we analyse the relationships between the largest cryptocurrencies and such time-varying realisation as to the scale of the economic shock centralised within the rapidly-escalating pandemic. We find evidence of significant growth in both returns and volumes traded, indicating that large cryptocurrencies acted as a store of value during this period of exceptional financial market stress. Further, cryptocurrency returns are found to be significantly influenced by negative sentiment relating to COVID-19. While not only providing diversification benefits for investors, results suggest that these digital assets acted as a safe-haven similar to that of precious metals during historiccrises.
Deep-Learning-Enhanced Multitarget Detection for End–Edge–Cloud Surveillance in Smart IoT

Along with the rapid development of cloud computing, IoT, and AI technologies, cloud video surveillance (CVS) has become a hotly discussed topic, especially when facing the requirement of real-time analysis in smart applications. Object detection usually plays an important role for environment monitoring and activity tracking in surveillance system. The emerging edge-cloud computing paradigm provides us an opportunity to deal with the continuously generated huge amount of surveillance data in an on-site manner across IoT systems. However, the detection performance is still far away from satisfactions due to the complex surveilling environment. In this study, we focus on the multitarget detection for real-time surveillance in smart IoT systems. A newly designed deep neural network model called A-YONet, which is constructed by combining the advantages of YOLO and MTCNN, is proposed to be deployed in an end-edge-cloud surveillance system, in order to realize the lightweight training and feature learning with limited computing sources. An intelligent detection algorithm is then developed based on a preadjusting scheme of anchor box and a multilevel feature fusion mechanism. Experiments and evaluations using two data sets, including one public data set and one homemade data set obtained in a real surveillance system, demonstrate the effectiveness of our proposed method in enhancing training efficiency and detection precision, especially for multitarget detection in smart IoT application developments.
Categorical Depth Distribution Network for Monocular 3D Object Detection

Monocular 3D object detection is a key problem for autonomous vehicles, as it provides a solution with simple configuration compared to typical multi-sensor systems. The main challenge in monocular 3D detection lies in accurately predicting object depth, which must be inferred from object and scene cues due to the lack of direct range measurement. Many methods attempt to directly estimate depth to assist in 3D detection, but show limited performance as a result of depth inaccuracy. Our proposed solution, Categorical Depth Distribution Network (CaDDN), uses a predicted categorical depth distribution for each pixel to project rich contextual feature information to the appropriate depth interval in 3D space. We then use the computationally efficient bird’s-eye-view projection and single-stage detector to produce the final output detections. We design CaDDN as a fully differentiable end-to-end approach for joint depth estimation and object detection. We validate our approach on the KITTI 3D object detection benchmark, where we rank 1 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">st</sup> among published monocular methods. We also provide the first monocular 3D detection results on the newly released Waymo Open Dataset. We provide a code release for CaDDN which is made available.
The causal relationship between Bitcoin attention and Bitcoin returns: Evidence from the Copula-based Granger causality test

This paper examines the causal relationship between Bitcoin attention (measured by the Google Trends search queries) and Bitcoin returns for the period from January 1, 2013, to December 31, 2017. For this purpose, we employ the Copula-based Granger Causality in Distribution (CGCD) test. After implementing various robustness checks, we observe that there is a bi-directional causal relationship between Bitcoin attention and Bitcoin returns with the exception of the central distributions from 40% to 80%. To put it differently, the bidirectional causality mainly exists in the left tail (poor performance) and the right tail (superior performance) of the distribution.
Particle swarm optimization based on dimensional learning strategy

In traditional particle swarm optimization (PSO) algorithm, each particle updates its velocity and position with a learning mechanism based on its personal best experience and the population best experience. The learning mechanism in traditional PSO is simple and easy to implement, but it suffers some potential problems, such as the phenomena of “oscillation” and “two steps forward, one step back”. Therefore, designing an effective learning strategy to avoid these two phenomena and to improve the search efficiency is an urgent issue for PSO research. This paper proposes a dimensional learning strategy (DLS) for discovering and integrating the promising information of the population best solution according to the personal best experience of each particle. Thereafter, a two-swarm learning PSO (TSLPSO) algorithm based on different learning strategies is proposed. One of the subpopulations constructs the learning exemplars by DLS to guide the local search of the particles, and the other subpopulation constructs the learning exemplars by the comprehensive learning strategy to guide the global search. 16 classic benchmark functions, 30 CEC2014 test functions, and 1 real-world optimization problem are used to test the proposed algorithm against with 5 typical PSO algorithms and 1 state-of-the-art differential evolution (DE) algorithm. The experimental results show that TSLPSO is statistically and significantly better than the compared algorithms for most of the test problems. Moreover, the convergence speed and convergence accuracy of TSLPSO are also significantly improved.
Coded Computation Over Heterogeneous Clusters

In large-scale distributed computing clusters, such as Amazon EC2, there are several types of “system noise” that can result in major degradation of performance: system failures, bottlenecks due to limited communication bandwidth, latency due to straggler nodes, and so on. There have been recent results that demonstrate the impact of coding for efficient utilization of computation and storage redundancy to alleviate the effect of stragglers and communication bottlenecks in homogeneous clusters. In this paper, we focus on general heterogeneous distributed computing clusters consist of a variety of computing machines with different capabilities. We propose a coding framework for speeding up distributed computing in heterogeneous clusters by trading redundancy for reducing the latency of computation. In particular, we propose heterogeneous coded matrix multiplication (HCMM) algorithm for performing distributed matrix multiplication over heterogeneous clusters that are provably asymptotically optimal for a broad class of processing time distributions. Moreover, we show that HCMM is unboundedly faster than any uncoded scheme that partitions the total workload among the workers. To demonstrate how the proposed HCMM scheme can be applied in practice, we provide results from numerical studies and Amazon EC2 experiments comparing HCMM with three benchmark load allocation schemes-uniform uncoded, load-balanced uncoded, and uniform coded. In particular, in our numerical studies, HCMM achieves speedups of up to 73%, 56%, and 42%, respectively, over the three benchmark schemes mentioned earlier. Furthermore, we carry out experiments over Amazon EC2 clusters and demonstrate how HCMM can be combined with rateless codes with nearly linear decoding complexity. In particular, we show that HCMM combined with the Luby transform codes can significantly reduce the overall execution time. HCMM is found to be up to 61%, 46%, and 36% faster than the aforementioned three benchmark schemes, respectively. Additionally, we provide a generalization to the problem of optimal load allocation in heterogeneous settings, where we take into account the monetary costs associated with distributed computing clusters. We argue that HCMM is asymptotically optimal for budget-constrained scenarios as well. In particular, we characterize the minimum possible expected cost associated with a computation task over a given cluster of machines. Furthermore, we develop a heuristic algorithm for (HCMM) load allocation for the distributed implementation of budget-limited computation tasks.
Decentralized document version control using ethereum blockchain and IPFS

In this paper, we propose a blockchain-based solution and framework for document sharing and version control to facilitate multi-user collaboration and track changes in a trusted, secure, and decentralized manner, with no involvement of a centralized trusted entity or third party. This solution is based on utilizing Ethereum smart contracts to govern and regulate the document version control functions among the creators and developers of the document and its validators. Moreover, our solution leverages the benefits of IPFS (InterPlanetary File System) to store documents on a decentralized file system. The proposed solution automates necessary interactions among multiple actors comprising developers and approvers. Smart contracts have been developed using Solidity language, and their functionalities were tested using the Remix IDE (Integrated Development Environment). The paper demonstrates that our smart contract code is free of commonly known security vulnerabilities and attacks. The code has been made publically available at Github.
DeepFL: integrating multiple fault diagnosis dimensions for deep fault localization

Learning-based fault localization has been intensively studied recently. Prior studies have shown that traditional Learning-to-Rank techniques can help precisely diagnose fault locations using various dimensions of fault-diagnosis features, such as suspiciousness values computed by various off-the-shelf fault localization techniques. However, with the increasing dimensions of features considered by advanced fault localization techniques, it can be quite challenging for the traditional Learning-to-Rank algorithms to automatically identify effective existing/latent features. In this work, we propose DeepFL, a deep learning approach to automatically learn the most effective existing/latent features for precise fault localization. Although the approach is general, in this work, we collect various suspiciousness-value-based, fault-proneness-based and textual-similarity-based features from the fault localization, defect prediction and information retrieval areas, respectively. DeepFL has been studied on 395 real bugs from the widely used Defects4J benchmark. The experimental results show DeepFL can significantly outperform state-of-the-art TraPT/FLUCCS (e.g., localizing 50+ more faults within Top-1). We also investigate the impacts of deep model configurations (e.g., loss functions and epoch settings) and features. Furthermore, DeepFL is also surprisingly effective for cross-project prediction.
Predicting the Driver's Focus of Attention: The DR(eye)VE Project

In this work we aim to predict the driver's focus of attention. The goal is to estimate what a person would pay attention to while driving, and which part of the scene around the vehicle is more critical for the task. To this end we propose a new computer vision model based on a multi-branch deep architecture that integrates three sources of information: raw video, motion and scene semantics. We also introduce DR(eye)VE, the largest dataset of driving scenes for which eye-tracking annotations are available. This dataset features more than 500,000 registered frames, matching ego-centric views (from glasses worn by drivers) and car-centric views (from roof-mounted camera), further enriched by other sensors measurements. Results highlight that several attention patterns are shared across drivers and can be reproduced to some extent. The indication of which elements in the scene are likely to capture the driver's attention may benefit several applications in the context of human-vehicle interaction and driver attention analysis.
The challenge of crafting intelligible intelligence

To trust the behavior of complex AI algorithms, especially in mission-critical settings, they must be made intelligible.
Style Transfer by Relaxed Optimal Transport and Self-Similarity

The goal of style transfer algorithms is to render the content of one image using the style of another. We propose Style Transfer by Relaxed Optimal Transport and Self-Similarity (STROTSS), a new optimization-based style transfer algorithm. We extend our method to allow user specified point-to-point or region-to-region control over visual similarity between the style image and the output. Such guidance can be used to either achieve a particular visual effect or correct errors made by unconstrained style transfer. In order to quantitatively compare our method to prior work, we conduct a large-scale user study designed to assess the style-content tradeoff across settings in style transfer algorithms. Our results indicate that for any desired level of content preservation, our method provides higher quality stylization than prior work.
Few-Shot Class-Incremental Learning

The ability to incrementally learn new classes is crucial to the development of real-world artificial intelligence systems. In this paper, we focus on a challenging but practical few-shot class-incremental learning (FSCIL) problem. FSCIL requires CNN models to incrementally learn new classes from very few labelled samples, without forgetting the previously learned ones. To address this problem, we represent the knowledge using a neural gas (NG) network, which can learn and preserve the topology of the feature manifold formed by different classes. On this basis, we propose the TOpology-Preserving knowledge InCrementer (TOPIC) framework. TOPIC mitigates the forgetting of the old classes by stabilizing NG's topology and improves the representation learning for few-shot new classes by growing and adapting NG to new training samples. Comprehensive experimental results demonstrate that our proposed method significantly outperforms other state-of-the-art class-incremental learning methods on CIFAR100, miniImageNet, and CUB200 datasets.
Data Uncertainty Learning in Face Recognition

Modeling data uncertainty is important for noisy images, but seldom explored for face recognition. The pioneer work, PFE, considers uncertainty by modeling each face image embedding as a Gaussian distribution. It is quite effective. However, it uses fixed feature (mean of the Gaussian) from an existing model. It only estimates the variance and relies on an ad-hoc and costly metric. Thus, it is not easy to use. It is unclear how uncertainty affects feature learning. This work applies data uncertainty learning to face recognition, such that the feature (mean) and uncertainty (variance) are learnt simultaneously, for the first time. Two learning methods are proposed. They are easy to use and outperform existing deterministic methods as well as PFE on challenging unconstrained scenarios. We also provide insightful analysis on how incorporating uncertainty estimation helps reducing the adverse effects of noisy samples and affects the feature learning.
Visible-Infrared Person Re-Identification via Homogeneous Augmented Tri-Modal Learning

Matching person images between the daytime visible modality and night-time infrared modality (VI-ReID) is a challenging cross-modality pedestrian retrieval problem. Existing methods usually learn the multi-modality features in raw image, ignoring the image-level discrepancy. Some methods apply GAN technique to generate the cross-modality images, but it destroys the local structure and introduces unavoidable noise. In this paper, we propose a Homogeneous Augmented Tri-Modal (HAT) learning method for VI-ReID, where an auxiliary grayscale modality is generated from their homogeneous visible images, without additional training process. It preserves the structure information of visible images and approximates the image style of infrared modality. Learning with the grayscale visible images enforces the network to mine structure relations across multiple modalities, making it robust to color variations. Specifically, we solve the tri-modal feature learning from both multi-modal classification and multi-view retrieval perspectives. For multi-modal classification, we learn a multi-modality sharing identity classifier with a parameter-sharing network, trained with a homogeneous and heterogeneous identification loss. For multi-view retrieval, we develop a weighted tri-directional ranking loss to optimize the relative distance across multiple modalities. Incorporated with two invariant regularizers, HAT simultaneously minimizes multiple modality variations. In-depth analysis demonstrates the homogeneous grayscale augmentation significantly outperforms the current state-of-the-art by a large margin.
Recent advancement in cancer detection using machine learning: Systematic survey of decades, comparisons and challenges

Cancer is a fatal illness often caused by genetic disorder aggregation and a variety of pathological changes. Cancerous cells are abnormal areas often growing in any part of human body that are life-threatening. Cancer also known as tumor must be quickly and correctly detected in the initial stage to identify what might be beneficial for its cure. Even though modality has different considerations, such as complicated history, improper diagnostics and treatement that are main causes of deaths. The aim of the research is to analyze, review, categorize and address the current developments of human body cancer detection using machine learning techniques for breast, brain, lung, liver, skin cancer leukemia. The study highlights how cancer diagnosis, cure process is assisted using machine learning with supervised, unsupervised and deep learning techniques. Several state of art techniques are categorized under the same cluster and results are compared on benchmark datasets from accuracy, sensitivity, specificity, false-positive metrics. Finally, challenges are also highlighted for possible future work.
Immersive light field video with a layered mesh representation

We present a system for capturing, reconstructing, compressing, and rendering high quality immersive light field video. We accomplish this by leveraging the recently introduced DeepView view interpolation algorithm, replacing its underlying multi-plane image (MPI) scene representation with a collection of spherical shells that are better suited for representing panoramic light field content. We further process this data to reduce the large number of shell layers to a small, fixed number of RGBA+depth layers without significant loss in visual quality. The resulting RGB, alpha, and depth channels in these layers are then compressed using conventional texture atlasing and video compression techniques. The final compressed representation is lightweight and can be rendered on mobile VR/AR platforms or in a web browser. We demonstrate light field video results using data from the 16-camera rig of [Pozo et al. 2019] as well as a new low-cost hemispherical array made from 46 synchronized action sports cameras. From this data we produce 6 degree of freedom volumetric videos with a wide 70 cm viewing baseline, 10 pixels per degree angular resolution, and a wide field of view, at 30 frames per second video frame rates. Advancing over previous work, we show that our system is able to reproduce challenging content such as view-dependent reflections, semi-transparent surfaces, and near-field objects as close as 34 cm to the surface of the camera rig.
To Err Is Human(-oid): How Do Consumers React to Robot Service Failure and Recovery?

Robots are the next wave in service technology; however, this advanced technology is not perfect. This research examines how social perceptions regarding the warmth and competence of service robots influence consumer reactions to service failures and recovery efforts by robots. We argue that humanoid (vs. nonhumanoid) service robots are more strongly associated with warmth (whereas competence does not differ). This tendency to expect greater warmth from humanoid robots has important consequences for service firms: (i) consumers are more dissatisfied due to lack of warmth following a process failure caused by a humanoid (vs. nonhumanoid; Study 1); (ii) humanoids (but not nonhumanoids) can recover a service failure by themselves via sincere apology, restoring perceptions of warmth (Study 2A); (iii) humanoids (but not nonhumanoids) can also effectively provide explanations as a recovery tactic (Study 2B); and, importantly, (iv) human intervention can be used to mitigate dissatisfaction following inadequate recovery by a nonhumanoid robot (Study 3), supporting the notion of human-robot collaboration. Taken together, this research offers theoretical implications for robot anthropomorphism and practical implications for firms employing service robots.