Reviewerly/exHarmony huggingface dataset:
Enriching ImageNet with Human Similarity Judgments and Psychological Embeddings

Advances in supervised learning approaches to object recognition flourished in part because of the availability of high-quality datasets and associated benchmarks. However, these benchmarks—such as ILSVRC—are relatively task-specific, focusing predominately on predicting class labels. We introduce a publicly-available dataset that embodies the task-general capabilities of human perception and reasoning. The Human Similarity Judgments extension to ImageNet (ImageNet-HSJ) is composed of a large set of human similarity judgments that supplements the existing ILSVRC validation set. The new dataset supports a range of task and performance metrics, including evaluation of unsupervised algorithms. We demonstrate two methods of assessment: using the similarity judgments directly and using a psychological embedding trained on the similarity judgments. This embedding space contains an order of magnitude more points (i.e., images) than previous efforts based on human judgments. We were able to scale to the full 50,000 image ILSVRC validation set through a selective sampling process that used variational Bayesian inference and model ensembles to sample aspects of the embedding space that were most uncertain. To demonstrate the utility of ImageNet-HSJ, we used the similarity ratings and the embedding space to evaluate how well several popular models conform to human similarity judgments. One finding is that the more complex models that perform better on task-specific benchmarks do not better conform to human semantic judgments. In addition to the human similarity judgments, pre-trained psychological embeddings and code for inferring variational embeddings are made publicly available. ImageNet-HSJ supports the appraisal of internal representations and the development of more humanlike models.
Deep Two-View Structure-from-Motion Revisited

Two-view structure-from-motion (SfM) is the cornerstone of 3D reconstruction and visual SLAM. Existing deep learning-based approaches formulate the problem by either recovering absolute pose scales from two consecutive frames or predicting a depth map from a single image, both of which are ill-posed problems. In contrast, we propose to revisit the problem of deep two-view SfM by leveraging the well-posedness of the classic pipeline. Our method consists of 1) an optical flow estimation network that predicts dense correspondences between two frames; 2) a normalized pose estimation module that computes relative camera poses from the 2D optical flow correspondences, and 3) a scale-invariant depth estimation network that leverages epipolar geometry to reduce the search space, refine the dense correspondences, and estimate relative depth maps. Extensive experiments show that our method outperforms all state-of-the-art two-view SfM methods by a clear margin on KITTI depth, KITTI VO, MVS, Scenes11, and SUN3D datasets in both relative pose and depth estimation.
Right for the Right Concept: Revising Neuro-Symbolic Concepts by Interacting with their Explanations

Most explanation methods in deep learning map importance estimates for a model’s prediction back to the original input space. These "visual" explanations are often insufficient, as the model’s actual concept remains elusive. Moreover, without insights into the model’s semantic concept, it is difficult —if not impossible— to intervene on the model’s behavior via its explanations, called Explanatory Interactive Learning. Consequently, we propose to intervene on a Neuro-Symbolic scene representation, which allows one to revise the model on the semantic level, e.g. "never focus on the color to make your decision". We compiled a novel confounded visual scene data set, the CLEVR-Hans data set, capturing complex compositions of different objects. The results of our experiments on CLEVR-Hans demonstrate that our semantic explanations, i.e. compositional explanations at a per-object level, can identify confounders that are not identifiable using "visual" explanations only. More importantly, feedback on this semantic level makes it possible to revise the model from focusing on these factors.
Generative Interventions for Causal Learning

We introduce a framework for learning robust visual representations that generalize to new viewpoints, backgrounds, and scene contexts. Discriminative models often learn naturally occurring spurious correlations, which cause them to fail on images outside of the training distribution. In this paper, we show that we can steer generative models to manufacture interventions on features caused by confounding factors. Experiments, visualizations, and theoretical results show this method learns robust representations more consistent with the underlying causal relationships. Our approach improves performance on multiple datasets demanding out-of-distribution generalization, and we demonstrate state-of-the-art performance generalizing from ImageNet to ObjectNet dataset.
SILedger: A Blockchain and ABE-based Access Control for Applications in SDN-IoT Networks

The Software Defined Network in Internet of Things (SDN-IoT) is enjoying growing popularity due to its flexibility, automaticity and programmability. However, there is still a lack of proper permission management on SDN-IoT applications (SIApps), especially when the SIApp’s required northbound interfaces are located in multiple heterogeneous controllers without mutual trust. Existing access control methods are usually based on centralized models, proprietary controllers, trusting conditions or manual operations. It can incur unnecessary performance degradation and poor scalability. To solve this problem, this paper proposes a SIApps’ ledger (SILedger), an open, trusted, and decentralized access control mechanism based on blockchain and attribute-based encryption (ABE). It can not only support effective authorization of SIApps in heterogeneous and untrusted SDN-IoT control domains, but also record all interactions between SIApps and resources, and thus facilitate SIApps further charging, analysis and audit. The main idea is that the SIApps are authorized using access tokens encrypted by ABE, and these tokens are seen as the currency of blockchain to be distributed. Specifically, we re-design blockchain transaction, token encryption, token initialization and token update schemes to achieve cross-domain, fine-grained and flexible SIApps’ permission management. In order to mitigate the delay and complexity problem of blockchain and ABE, we devise an access control framework that separates authorization from call process of SIApps. Finally, we perform security analysis and implement a FISCO-BCOS-based prototype for SILedger. The experimental results show that it can provide effective access control for SIApps with negligible overheads.
Reconstructing 3D Human Pose by Watching Humans in the Mirror

In this paper, we introduce the new task of reconstructing 3D human pose from a single image in which we can see the person and the person’s image through a mirror. Compared to general scenarios of 3D pose estimation from a single view, the mirror reflection provides an additional view for resolving the depth ambiguity. We develop an optimization-based approach that exploits mirror symmetry constraints for accurate 3D pose reconstruction. We also provide a method to estimate the surface normal of the mirror from vanishing points in the single image. To validate the proposed approach, we collect a large-scale dataset named Mirrored-Human, which covers a large variety of human subjects, poses and backgrounds. The experiments demonstrate that, when trained on Mirrored-Human with our reconstructed 3D poses as pseudo ground-truth, the accuracy and generalizability of existing single-view 3D pose estimators can be largely improved. The code and dataset are available at https://zju3dv.github.io/Mirrored-Human/.
Machine learning algorithms for fraud prediction in property insurance: Empirical evidence using real-world microdata

This paper evaluated fraud prediction in property insurance claims using various machine learning models based on real-world data from a major Brazilian insurance company. The models were tested recursively and average predictive results were compared controlling for false positives and false negatives. The results showed that ensemble-based methods (random forest and gradient boosting) and deep neural networks yielded the best results, exhibiting superior average performance in comparison to the other classifiers, including the commonly used logistic regression. In addition, we compiled a general profile of confirmed fraudsters from the dataset and estimated the impact of each feature in the global classification performance and for prominent cases of false positive and false negative predictions using eXplainable Artificial Intelligence methods. The findings of this study can aid risk analysts and professionals in assessing the strengths and weaknesses of each model and to build empirically effective decision rules to evaluate future insurance policies.
Socially-Aware-Clustering-Enabled Federated Learning for Edge Networks

Edge Intelligence based on federated learning (FL) can be considered to be a promising paradigm for many emerging, strict latency Internet of Things (IoT) applications. Furthermore, a rapid upsurge in the number of IoT devices is expected in the foreseeable future. Although FL enables privacy-preserving, on-device machine learning, it still exhibits a privacy leakage issue. A malicious aggregation server can infer the sensitive information of other end-devices using their local learning model updates. Furthermore, centralized FL aggregation server might stop working due to security attack or a physical damage. To address the aforementioned issues, we propose a novel concept of socially-aware-clustering-enabled dispersed FL. First, we present a novel framework for socially-aware-clustering-enabled dispersed FL. Second, we formulate a problem for minimizing the loss function of the proposed FL scheme. Third, we decompose the formulated problem into three sub-problems, such as local devices relative accuracy minimization (i.e., end-devices local accuracy maximization) sub-problem, clustering sub-problem, and resource allocation sub-problem, due to the NP-hard nature of the formulated problem. The clustering and resource allocation sub-problems are solved using low complexity schemes based on a matching theory. The end devices' relative accuracy minimization problem is solved by using a convex optimizer. Finally, numerical results are provided for validation of the proposed FL scheme. Furthermore, we show the convergence of the proposed FL scheme for image classification tasks using the MNIST dataset.
Bit Efficient Quantization for Deep Neural Networks

Quantization for deep neural networks have afforded models for edge devices that use less on-board memory and enable efficient low-power inference. In this paper, we present a comparison of model-parameter driven quantization approaches that can achieve as low as 3-bit precision without affecting accuracy. The post-training quantization approaches are data-free, and the resulting weight values are closely tied to the dataset distribution on which the model has converged to optimality. We show quantization results for a number of state-of-art deep neural networks (DNN) using large dataset like ImageNet. To better analyze quantization results, we describe the overall range and local sparsity of values afforded through various quantization schemes. We show the methods to lower bit-precision beyond quantization limits with object class clustering.
Topic analysis and development in knowledge graph research: A bibliometric review on three decades

Knowledge graph as a research topic is increasingly popular to represent structural relations between entities. Recent years have witnessed the release of various open-source and enterprise-supported knowledge graphs with dramatic growth in applying knowledge representation and reasoning into different areas like natural language processing and computer vision. This study aims to comprehensively explore the status and trends – particularly the thematic research structure – of knowledge graphs. Specifically, based on 386 research articles published from 1991 to 2020, we conducted analyses in terms of the (1) visualization of the trends of annual article and citation counts, (2) recognition of major institutions, countries/regions, and publication sources, (3) visualization of scientific collaborations of major institutions and countries/regions, and (4) detection of major research themes and their developmental tendencies. Interest in knowledge graph research has clearly increased from 1991 to 2020 and is continually expanding. China is the most prolific country in knowledge graph research. Moreover, countries/regions and institutions that have higher levels of international collaboration are more impactful. Several widely studied issues such as knowledge graph embedding, search and query based on knowledge graphs, and knowledge graphs for intangible cultural heritage are highlighted. Based on the results, we further summarize perspective directions and suggestions for researchers, practitioners, and project managers to facilitate future research on knowledge graphs.
Representation Learning via Global Temporal Alignment and Cycle-Consistency

We introduce a weakly supervised method for representation learning based on aligning temporal sequences (e.g., videos) of the same process (e.g., human action). The main idea is to use the global temporal ordering of latent correspondences across sequence pairs as a supervisory signal. In particular, we propose a loss based on scoring the optimal sequence alignment to train an embedding network. Our loss is based on a novel probabilistic path finding view of dynamic time warping (DTW) that contains the following three key features: (i) the local path routing decisions are contrastive and differentiable, (ii) pairwise distances are cast as probabilities that are contrastive as well, and (iii) our formulation naturally admits a global cycle-consistency loss that verifies correspondences. For evaluation, we consider the tasks of fine-grained action classification, few shot learning, and video synchronization. We report significant performance increases over previous methods. In addition, we report two applications of our temporal alignment framework, namely 3D pose reconstruction and fine-grained audio/visual retrieval.
Recent Advances in Blockchain and Artificial Intelligence Integration: Feasibility Analysis, Research Issues, Applications, Challenges, and Future Work

Blockchain constructs a distributed point-to-point system, which is a secure and verifiable mechanism for decentralized transaction validation and is widely used in financial economy, Internet of Things, large data, cloud computing, and edge computing. On the other hand, artificial intelligence technology is gradually promoting the intelligent development of various industries. As two promising technologies today, there is a natural advantage in the convergence between blockchain and artificial intelligence technologies. Blockchain makes artificial intelligence more autonomous and credible, and artificial intelligence can prompt blockchain toward intelligence. In this paper, we analyze the combination of blockchain and artificial intelligence from a more comprehensive and three-dimensional point of view. We first introduce the background of artificial intelligence and the concept, characteristics, and key technologies of blockchain and subsequently analyze the feasibility of combining blockchain with artificial intelligence. Next, we summarize the research work on the convergence of blockchain and artificial intelligence in home and overseas within this category. After that, we list some related application scenarios about the convergence of both technologies and also point out existing problems and challenges. Finally, we discuss the future work.
Collective Spin-Light and Light-Mediated Spin-Spin Interactions in an Optical Cavity

The interaction between an atomic ensemble and a light mode in a high-finesse optical cavity can easily reach the strong-coupling regime, where quantum effects dominate. In this regime, the interaction can be used to generate both atom-light and atom-atom entanglement. We analyze the dominant effects on the collective atomic state and the light field, and derive a unified approach that can account for atomic entanglement induced both by measurements on the light field, and by ignoring the state of the light field altogether. We present analytical expressions for the entanglement induced by the interaction, and determine the conditions that maximize the entanglement-induced gain over the standard quantum limit in quantum sensors and atomic clocks.6 MoreReceived 28 June 2021Accepted 25 March 2022Corrected 29 July 2022DOI:https://doi.org/10.1103/PRXQuantum.3.020308Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.Published by the American Physical SocietyPhysics Subject Headings (PhySH)Research AreasCavity quantum electrodynamicsEntanglement productionQuantum metrologyAtomic, Molecular & Optical
Does Bitcoin React to Trump’s Tweets?

This textual analysis with spillover effects examines whether the sentiment expressed in the US President Donald Trump’s tweets correlates to price and volume activity in the Bitcoin market. After examining 13,918 tweets from January 2017 to January 2020, we find that negative sentiment is a predictive factor for Bitcoin returns, trading volumes, realized volatility, and jumps. In addition, only negative sentiment has a Granger-causal relation with volatility. We also find that Trump’s Twitter sentiment can influence the Bitcoin market in the form of time-varying dependence. This paper also extended the COVID-19 period and found that Trump’s sentiment can be a predictive tool to the Bitcoin market during the pandemic. Our results hold robust for alternative cryptocurrencies and offer insights about this market.
Industrial Image Anomaly Localization Based on Gaussian Clustering of Pretrained Feature

Anomaly localization is valuable for improvement of complex production processing in smart manufacturing system. As the distribution of anomalies is unknowable and labeled data is few, unsupervised methods based on convolutional neural network (CNN) have been studied for anomaly localization. But there are still problems for real industrial applications, in terms of localization accuracy, computation time, and memory storage. This article proposes a novel framework called as Gaussian clustering of pretrained feature (GCPF), including the clustering and inference stage, for anomaly localization in unsupervised way. The GCPF consists of three modules which include pretrained deep feature extraction (PDFE), multiple independent multivariate Gaussian clustering (MIMGC), and multihierarchical anomaly scoring (MHAS). In the clustering stage, features of normal images are extracted by pretrained CNN at the PDFE module, and then clustered at the MIMGC module. In the inference stage, features of target images are extracted and then scored for anomaly localization at the MHAS module. The GCPF is compared with the state-of-the-art methods on MVTec dataset, achieving receiver operating characteristic curve of 96.86% over all 15 categories, and extended to NanoTWICE and DAGM datasets. The GCPF outperforms the compared methods for unsupervised anomaly localization, and significantly reserves the low computation complexity and online memory storage which are important for real industrial applications.
Review of Capacitive Touchscreen Technologies: Overview, Research Trends, and Machine Learning Approaches

Touchscreens have been studied and developed for a long time to provide user-friendly and intuitive interfaces on displays. This paper describes the touchscreen technologies in four categories of resistive, capacitive, acoustic wave, and optical methods. Then, it addresses the main studies of SNR improvement and stylus support on the capacitive touchscreens that have been widely adopted in most consumer electronics such as smartphones, tablet PCs, and notebook PCs. In addition, the machine learning approaches for capacitive touchscreens are explained in four applications of user identification/authentication, gesture detection, accuracy improvement, and input discrimination.
MDS or NMDS self-dual codes from twisted generalized Reed–Solomon codes

Self-dual maximum distance separable codes (self-dual MDS codes) and self-dual near MDS (NMDS) codes are very important in coding theory and practice. Thus, it is interesting to construct self-dual MDS or self-dual NMDS codes. In this paper, we not only give parity-check matrices of dual codes of twisted generalized Reed–Solomon codes (TGRS codes) but also present a sufficient and necessary condition of self-dual TGRS codes over $${\mathbb {F}}_q$$ with q an odd prime. Moreover, we construct several classes of self-dual MDS or self-dual NMDS codes from TGRS codes.
Fault-Tolerant Parity Readout on a Shuttling-Based Trapped-Ion Quantum Computer

Quantum error correction requires the detection of errors via reliable measurements of multiqubit correlation operators. As these operations are inherently faulty, fault-tolerant schemes for realizing quantum error correction are required. Recently, a paradigm requiring only minimal resource overhead in the form of “flag” qubits to detect and correct errors has been proposed. We experimentally demonstrate a fault-tolerant weight-4 parity-check measurement scheme, where one additional flag qubit serves to detect errors, which would otherwise proliferate into uncorrectable weight-2 errors onto the qubit register. We achieve a parity measurement fidelity of 92.3(2)%, which increases to 93.2(2)% upon conditioning to the flag readout result, which shows that the measurement scheme intercepts intrinsic errors occurring throughout the sequence. We show that the protocol is capable of reliably intercepting faults by deliberately injecting bit- and phase-flip errors. For holistic benchmarking of the parity measurement scheme, we use an entanglement witnessing scheme requiring a minimal number of measurements to verify genuine six-qubit multipartite entanglement. The demonstrated fault-tolerant parity measurement scheme constitutes the key building block in a broad class of resource-efficient flag-based quantum error correction protocols including topological color codes. Our hardware platform is based on atomic ions stored in a microchip ion trap. The qubit register is dynamically reconfigured via shuttling operations enabling effective full connectivity without operational cross talk, thereby providing key prerequisites underlying fault-tolerant circuit design. These architectural features in combination with the demonstrated approach to flag-based fault-tolerant quantum error correction open up a route toward scalable fault-tolerant quantum computing.6 MoreReceived 16 July 2021Revised 30 November 2021Accepted 24 December 2021DOI:https://doi.org/10.1103/PhysRevX.12.011032Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI.Published by the American Physical SocietyPhysics Subject Headings (PhySH)Research AreasQuantum algorithmsQuantum computationQuantum entanglementQuantum information architectures & platformsQuantum information processingQuantum Information