Reviewerly/exHarmony huggingface dataset:
RGB-D Local Implicit Function for Depth Completion of Transparent Objects

Majority of the perception methods in robotics require depth information provided by RGB-D cameras. However, standard 3D sensors fail to capture depth of transparent objects due to refraction and absorption of light. In this paper, we introduce a new approach for depth completion of transparent objects from a single RGB-D image. Key to our approach is a local implicit neural representation built on ray-voxel pairs that allows our method to generalize to unseen objects and achieve fast inference speed. Based on this representation, we present a novel frame-work that can complete missing depth given noisy RGB-D input. We further improve the depth estimation iteratively using a self-correcting refinement model. To train the whole pipeline, we build a large scale synthetic dataset with transparent objects. Experiments demonstrate that our method performs significantly better than the current state-of-the-art methods on both synthetic and real world data. In addition, our approach improves the inference speed by a factor of 20 compared to the previous best method, ClearGrasp [43]. Code will be released at https://research.nvidia.com/publication/2021-03_RGB-D-Local-Implicit.
Credit Card Fraud Detection Using Machine Learning

Credit card is the commonly used payment mode in the recent years. As the technology is developing, the number of fraud cases are also increasing and finally poses the need to develop a fraud detection algorithm to accurately find and eradicate the fraudulent activities. This research work proposes different machine learning based classification algorithms such as logistic regression, random forest, and Naive Bayes for handling the heavily imbalanced dataset. Finally, this research work will calculate the accuracy, precision, recall, f1 score, confusion matrix, and Roc-auc score.
Applications of Smart Glasses in Applied Sciences: A Systematic Review

The aim of this study is to review academic papers on the applications of smart glasses. Among 82 surveyed papers, 57 were selected through filtering. The papers were published from January 2014 to October 2020. Four research questions were set up using the systematic review method, and conclusions were drawn focusing on the research trends by year and application fields; product and operating system; sensors depending on the application purpose; and data visualization, processing, and transfer methods. It was found that the most popular commercial smart glass products are Android-based Google products. In addition, smart glasses are most often used in the healthcare field, particularly for clinical and surgical assistance or for assisting mentally or physically disabled persons. For visual data transfer, 90% of the studies conducted used a camera sensor. Smart glasses have mainly been used to visualize data based on augmented reality, in contrast with the use of mixed reality. The results of this review indicate that research related to smart glasses is steadily increasing, and technological research into the development of smart glasses is being actively conducted.
How learning effects influence knowledge contribution in online Q&amp;A community? A social cognitive perspective

Informative contributions are critical for the healthy development of online Q&A communities, which have gained increasing popularity in solving personalized open-ended problems. However, little is known about whether past contribution behaviors and corresponding community feedbacks received affect the characteristics of subsequent contributions. Drawing upon the social cognitive theory, we examine the learning effects on users' knowledge contribution behaviors. Specifically, we focus on two types of learning effects: enactive learning from one's past contribution experience and vicarious learning from observation of others' performances in a question thread. Using a dataset collected from one of the largest online Q&A communities in China, we find that the length feature of past user contributions that garner highly positive feedback, no matter through enactive or vicarious learning, would influence the informativeness of subsequent contributions in the community. These learning effects are more effective for users with higher social status. The enactive learning effect is stronger for contributors with higher social status. For the vicarious learning on higher-status contributors, the influence of high-vote long answers is stronger, but the high-vote short answers show a weaker effect. Our research provides a deeper understanding of knowledge contribution behaviors in online knowledge communities and guides for establishing a healthy knowledge contribution environment.
Self-Supervised Learning for Semi-Supervised Temporal Action Proposal

Self-supervised learning presents a remarkable performance to utilize unlabeled data for various video tasks. In this paper, we focus on applying the power of self-supervised methods to improve semi-supervised action proposal generation. Particularly, we design an effective Self-supervised Semi-supervised Temporal Action Proposal (SSTAP) framework. The SSTAP contains two crucial branches, i.e., temporal-aware semi-supervised branch and relation-aware self-supervised branch. The semi-supervised branch improves the proposal model by introducing two temporal perturbations, i.e., temporal feature shift and temporal feature flip, in the mean teacher framework. The self-supervised branch defines two pretext tasks, including masked feature reconstruction and clip-order prediction, to learn the relation of temporal clues. By this means, SSTAP can better explore unlabeled videos, and improve the discriminative abilities of learned action features. We extensively evaluate the proposed SSTAP on THUMOS14 and ActivityNet v1.3 datasets. The experimental results demonstrate that SSTAP significantly outperforms state-of-the-art semi-supervised methods and even matches fully-supervised methods. Code is available at https://github.com/wangxiang1230/SSTAP.
Blockchain for consortium: A practical paradigm in agricultural supply chain system

Trading aspect of agricultural supply chain system is sophisticated since it consists of many stages and involves various entities/agencies. Recently, blockchain technology could prove its effectiveness to solve some of the concerns in agricultural supply chain systems. Nevertheless, maximizing profit for producers (in our study farmers) is another important concern that can be addressed by consortium establishment which blockchain technology is the best solution for this purpose due to the following reasons. First, since all the nodes in the blockchain keep the verified and synchronized version of the chain, each node can verify the transactions’ transparency. Second, blockchain technology is temper-proof that means no one can change the history of the transactions. These two main features of blockchain technology can provide a suitable ground to construct a consortium among the producers. However, there are other specific requirements that a successful consortium in agricultural supply chain system should address them that motivate us to a new design of blockchain technology. More precisely, in our design we consider the problems of trustability, scalability, and share amount assignment. For trustability, we utilize cyber physical system to ensure the quantity and quality of the products. Scalability is being addressed by adopting the concept of public service platform and proposing a new consensus algorithm. And finally share amount assignment is being solved by our improved version of ant colony optimization algorithm. Experimental results and analysis prove the effectiveness and accuracy of our proposed design for blockchain technology.
Deep Polarization Imaging for 3D Shape and SVBRDF Acquisition

We present a novel method for efficient acquisition of shape and spatially varying reflectance of 3D objects using polarization cues. Unlike previous works that have exploited polarization to estimate material or object appearance under certain constraints (known shape or multiview acquisition), we lift such restrictions by coupling polarization imaging with deep learning to achieve high quality estimate of 3D object shape (surface normals and depth) and SVBRDF using single-view polarization imaging under frontal flash illumination. In addition to acquired polarization images, we provide our deep network with strong novel cues related to shape and reflectance, in the form of a normalized Stokes map and an estimate of diffuse color. We additionally describe modifications to network architecture and training loss which provide further qualitative improvements. We demonstrate our approach to achieve superior results compared to recent works employing deep learning in conjunction with flash illumination.
A global neural network learning machine: Coupled integer and fractional calculus operator with an adaptive learning scheme

Find the global optimal solution of the model is one promising research topic in computational intelligent community. Dependent on analogies to natural processes, the evolutionary swarm intelligent algorithms are widely used for solving global optimization problems which directed by the fitness values. In this paper, we propose one efficient fractional global learning machine (Fragmachine) which includes two stages (descending and ascending) to determine the optimal search path. The neural network model is used to approach the given fitness value. Specifically, for the descending stage, the integer gradient of the network output with respect the current location is employed to find the next descending point, while for the ascending stage, the fractional gradient is implemented to climb and escape from the local optimal point. We further propose one adaptive learning rate during training which relies on both the current gradient (integer or fractional) information and the fitness value. Finally, a series of numerical experiments verify the effectiveness of the proposed algorithm, Fragmachine.
Co-Attention for Conditioned Image Matching

We propose a new approach to determine correspondences between image pairs in the wild under large changes in illumination, viewpoint, context, and material. While other approaches find correspondences between pairs of images by treating the images independently, we instead condition on both images to implicitly take account of the differences between them. To achieve this, we introduce (i) a spatial attention mechanism (a co-attention module, CoAM) for conditioning the learned features on both images, and (ii) a distinctiveness score used to choose the best matches at test time. CoAM can be added to standard architectures and trained using self-supervision or supervised data, and achieves a significant performance improvement under hard conditions, e.g. large viewpoint changes. We demonstrate that models using CoAM achieve state of the art or competitive results on a wide range of tasks: local matching, camera localization, 3D reconstruction, and image stylization.
Hugging from A Distance: Building Interpersonal Relationships in Social Virtual Reality

This paper focuses on how the emerging social VR systems, as new and unique social interaction spaces that afford high-fidelity and multidimensional physical presence, may support building interpersonal relationships in a more nuanced, immersive, and embodied way. Based on 30 interviews, our investigation focuses on 1) the main reasons why people build and foster interpersonal relationships in social VR; 2) various novel activities through which users can foster relationships in social VR; and 3) the complicated influences of social VR mediated relationships on users’ online and offline social lives. We contribute to better understanding mediated interactive experiences by shedding light on the novel role of social VR in transforming how people meet, interact, and establish connections with others compared to other forms of media. We also provide potential directions to inform the design of future social VR systems to better afford healthy, fulfilling, and supportive interpersonal relationships.
Towards Accurate Text-based Image Captioning with Content Diversity Exploration

Text-based image captioning (TextCap) which aims to read and reason images with texts is crucial for a machine to understand a detailed and complex scene environment, considering that texts are omnipresent in daily life. This task, however, is very challenging because an image often contains complex texts and visual information that is hard to be described comprehensively. Existing methods attempt to extend the traditional image captioning methods to solve this task, which focus on describing the overall scene of images by one global caption. This is infeasible because the complex text and visual information cannot be described well within one caption. To resolve this difficulty, we seek to generate multiple captions that accurately describe different parts of an image in detail. To achieve this purpose, there are three key challenges: 1) it is hard to decide which parts of the texts of images to copy or paraphrase; 2) it is non-trivial to capture the complex relationship between diverse texts in an image; 3) how to generate multiple captions with diverse content is still an open problem. To conquer these, we propose a novel Anchor-Captioner method. Specifically, we first find the important tokens which are supposed to be paid more attention to and consider them as anchors. Then, for each chosen anchor, we group its rel-evant texts to construct the corresponding anchor-centred graph (ACG). Last, based on different ACGs, we conduct the multi-view caption generation to improve the content diversity of generated captions. Experimental results show that our method not only achieves SOTA performance but also generates diverse captions to describe images.
Space-Time Distillation for Video Super-Resolution

Compact video super-resolution (VSR) networks can be easily deployed on resource-limited devices, e.g., smartphones and wearable devices, but have considerable performance gaps compared with complicated VSR networks that require a large amount of computing resources. In this paper, we aim to improve the performance of compact VSR networks without changing their original architectures, through a knowledge distillation approach that transfers knowledge from a complicated VSR network to a compact one. Specifically, we propose a space-time distillation (STD) scheme to exploit both spatial and temporal knowledge in the VSR task. For space distillation, we extract spatial attention maps that hint the high-frequency video content from both networks, which are further used for transferring spatial modeling capabilities. For time distillation, we narrow the performance gap between compact models and complicated models by distilling the feature similarity of the temporal memory cells, which are encoded from the sequence of feature maps generated in the training clips using ConvLSTM. During the training process, STD can be easily incorporated into any network without changing the original network architecture. Experimental results on standard benchmarks demonstrate that, in resource-constrained situations, the proposed method notably improves the performance of existing VSR networks without increasing the inference time.
Rich features for perceptual quality assessment of UGC videos

Video quality assessment for User Generated Content (UGC) is an important topic in both industry and academia. Most existing methods only focus on one aspect of the perceptual quality assessment, such as technical quality or compression artifacts. In this paper, we create a large scale dataset to comprehensively investigate characteristics of generic UGC video quality. Besides the subjective ratings and content labels of the dataset, we also propose a DNN-based framework to thoroughly analyze importance of content, technical quality, and compression level in perceptual quality. Our model is able to provide quality scores as well as human-friendly quality indicators, to bridge the gap between low level video signals to human perceptual quality. Experimental results show that our model achieves state-of-the-art correlation with Mean Opinion Scores (MOS).
Contrastive Domain Adaptation

Recently, contrastive self-supervised learning has become a key component for learning visual representations across many computer vision tasks and benchmarks. However, contrastive learning in the context of domain adaptation remains largely underexplored. In this paper, we propose to extend contrastive learning to a new domain adaptation setting, a particular situation occurring where the similarity is learned and deployed on samples following different probability distributions without access to labels. Contrastive learning learns by comparing and contrasting positive and negative pairs of samples in an unsupervised setting without access to source and target labels. We have developed a variation of a recently proposed contrastive learning framework that helps tackle the domain adaptation problem, further identifying and removing possible negatives similar to the anchor to mitigate the effects of false negatives. Extensive experiments demonstrate that the proposed method adapts well, and improves the performance on the downstream domain adaptation task.
MaxUp: Lightweight Adversarial Training with Data Augmentation Improves Neural Network Training

We propose MaxUp, a simple and effective technique for improving the generalization performance of machine learning models, especially deep neural networks. The idea is to generate a set of augmented data with some random perturbations or transforms, and minimize the maximum, or worst case loss over the augmented data. By doing so, we implicitly introduce a smoothness or robustness regu-larization against the random perturbations, and hence improve the generation performance. For example, in the case of Gaussian perturbation, MaxUp is asymptotically equivalent to using the gradient norm of the loss as a penalty to encourage smoothness. We test MaxUp on a range of tasks, including image classification, 3D point cloud classification, and adversarial certification, on which MaxUp consistently outperforms the baseline methods, without introducing substantial computational overhead. In particular, we improve ImageNet classification from the top-1 accuracy 85.5% without extra data to 85.8%.
Confidence Score: The Forgotten Dimension of Object Detection Performance Evaluation

When deploying a model for object detection, a confidence score threshold is chosen to filter out false positives and ensure that a predicted bounding box has a certain minimum score. To achieve state-of-the-art performance on benchmark datasets, most neural networks use a rather low threshold as a high number of false positives is not penalized by standard evaluation metrics. However, in scenarios of Artificial Intelligence (AI) applications that require high confidence scores (e.g., due to legal requirements or consequences of incorrect detections are severe) or a certain level of model robustness is required, it is unclear which base model to use since they were mainly optimized for benchmark scores. In this paper, we propose a method to find the optimum performance point of a model as a basis for fairer comparison and deeper insights into the trade-offs caused by selecting a confidence score threshold.
The Translucent Patch: A Physical and Universal Attack on Object Detectors

Physical adversarial attacks against object detectors have seen increasing success in recent years. However, these attacks require direct access to the object of interest in order to apply a physical patch. Furthermore, to hide multiple objects, an adversarial patch must be applied to each object. In this paper, we propose a contactless translucent physical patch containing a carefully constructed pattern, which is placed on the camera’s lens, to fool state-of-the-art object detectors. The primary goal of our patch is to hide all instances of a selected target class. In addition, the optimization method used to construct the patch aims to ensure that the detection of other (untargeted) classes remains unharmed. Therefore, in our experiments, which are conducted on state-of-the-art object detection models used in autonomous driving, we study the effect of the patch on the detection of both the selected target class and the other classes. We show that our patch was able to prevent the detection of 42.27% of all stop sign instances while maintaining high (nearly 80%) detection of the other classes.
Spherical Confidence Learning for Face Recognition

An emerging line of research has found that spherical spaces better match the underlying geometry of facial im-ages, as evidenced by the state-of-the-art facial recognition methods which benefit empirically from spherical representations. Yet, these approaches rely on deterministic embeddings and hence suffer from the feature ambiguity dilemma, whereby ambiguous or noisy images are mapped into poorly learned regions of representation space, leading to inaccuracies. Probabilistic Face Embeddings (PFE) [17] is the first attempt to address this dilemma. However, we theoretically and empirically identify two main failures of PFE when it is applied to spherical deterministic embeddings aforementioned. To address these issues, in this paper, we propose a novel framework for face confidence learning in spherical space. Mathematically, we extend the von Mises Fisher density to its r-radius counterpart and derive a new optimization objective in closed form. Theoretically, the proposed probabilistic framework provably allows for better interpretability, leading to principled feature comparison and pooling. Extensive experimental results on multiple challenging benchmarks confirm our hypothesis and theory, and showcase the advantages of our framework over prior probabilistic methods and spherical deterministic embed-dings in various face recognition tasks.
Quantum repeaters based on concatenated bosonic and discrete-variable quantum codes

Abstract We propose an architecture of quantum-error-correction-based quantum repeaters that combines techniques used in discrete- and continuous-variable quantum information. Specifically, we propose to encode the transmitted qubits in a concatenated code consisting of two levels. On the first level we use a continuous-variable GKP code encoding the qubit in a single bosonic mode. On the second level we use a small discrete-variable code. Such an architecture has two important features. Firstly, errors on each of the two levels are corrected in repeaters of two different types. This enables for achieving performance needed in practical scenarios with a reduced cost with respect to an architecture for which all repeaters are the same. Secondly, the use of continuous-variable GKP code on the lower level generates additional analog information which enhances the error-correcting capabilities of the second-level code such that long-distance communication becomes possible with encodings consisting of only four or seven optical modes.