Reviewerly/exHarmony huggingface dataset:
Shapley variable importance cloud for interpretable machine learning

Interpretable machine learning has been focusing on explaining final models that optimize performance. The state-of-the-art Shapley additive explanations (SHAP) locally explains the variable impact on individual predictions and has recently been extended to provide global assessments across the dataset. Our work further extends "global" assessments to a set of models that are "good enough" and are practically as relevant as the final model to a prediction task. The resulting Shapley variable importance cloud consists of Shapley-based importance measures from each good model and pools information across models to provide an overall importance measure, with uncertainty explicitly quantified to support formal statistical inference. We developed visualizations to highlight the uncertainty and to illustrate its implications to practical inference. Building on a common theoretical basis, our method seamlessly complements the widely adopted SHAP assessments of a single final model to avoid biased inference, which we demonstrate in two experiments using recidivism prediction data and clinical data.
ReLive: Bridging In-Situ and Ex-Situ Visual Analytics for Analyzing Mixed Reality User Studies

The nascent field of mixed reality is seeing an ever-increasing need for user studies and field evaluation, which are particularly challenging given device heterogeneity, diversity of use, and mobile deployment. Immersive analytics tools have recently emerged to support such analysis in situ, yet the complexity of the data also warrants an ex-situ analysis using more traditional non-immersive visual analytics setups. To bridge the gap between both approaches, we introduce ReLive: a mixed-immersion visual analytics framework for exploring and analyzing mixed reality user studies. ReLive combines an in-situ virtual reality view with a complementary ex-situ desktop view. While the virtual reality view allows users to relive interactive spatial recordings replicating the original study, the synchronized desktop view provides a familiar interface for analyzing aggregated data. We validated our concepts in a two-step evaluation consisting of a design walkthrough and an empirical expert user study.
Playing Games with Ais: The Limits of GPT-3 and Similar Large Language Models

Abstract This article contributes to the debate around the abilities of large language models such as GPT-3, dealing with: firstly, evaluating how well GPT does in the Turing Test, secondly the limits of such models, especially their tendency to generate falsehoods, and thirdly the social consequences of the problems these models have with truth-telling. We start by formalising the recently proposed notion of reversible questions, which Floridi &amp; Chiriatti (2020) propose allow one to ‘identify the nature of the source of their answers’, as a probabilistic measure based on Item Response Theory from psychometrics. Following a critical assessment of the methodology which led previous scholars to dismiss GPT’s abilities, we argue against claims that GPT-3 completely lacks semantic ability. Using ideas of compression, priming, distributional semantics and semantic webs we offer our own theory of the limits of large language models like GPT-3, and argue that GPT can competently engage in various semantic tasks. The real reason GPT’s answers seem senseless being that truth-telling is not amongst them. We claim that these kinds of models cannot be forced into producing only true continuation, but rather to maximise their objective function they strategize to be plausible instead of truthful. This, we moreover claim, can hijack our intuitive capacity to evaluate the accuracy of its outputs. Finally, we show how this analysis predicts that a widespread adoption of language generators as tools for writing could result in permanent pollution of our informational ecosystem with massive amounts of very plausible but often untrue texts.
CGMDRNet: Cross-Guided Modality Difference Reduction Network for RGB-T Salient Object Detection

How to explore the interaction between the RGB and thermal modalities is the key success of the RGB-T saliency object detection (SOD). Most of the existing methods integrate multi-modality information by designing various fusion strategies. However, the modality gap between the RGB and thermal features will lead to unsatisfactory performances by simple feature concatenation. To solve this problem, we innovatively propose a cross-guided modality difference reduction network (CGMDRNet) to achieve intrinsic consistency feature fusion via reducing the modality differences. Specifically, we design a modality difference reduction (MDR) module, which is embedded in each layer of the backbone network. The module uses a cross-guided strategy to reduce the modality difference between the RGB and thermal features. Then, a cross-attention fusion (CAF) module is designed to fuse cross-modality features with small modality differences. In addition, we use a transformer-based feature enhancement (TFE) module to enhance the high-level feature representation that contributes more to performance. Finally, the high-level features guide the fusion of low-level features to obtain a saliency map with clear boundaries. Extensive experiments on three public RGB-T datasets show that the proposed CGMDRNet achieves competitive performance compared with state-of-the-art (SOTA) RGB-T SOD models.
VLP: A Survey on Vision-language Pre-training

Abstract In the past few years, the emergence of pre-training models has brought uni-modal fields such as computer vision (CV) and natural language processing (NLP) to a new era. Substantial works have shown that they are beneficial for downstream uni-modal tasks and avoid training a new model from scratch. So can such pre-trained models be applied to multi-modal tasks? Researchers have explored this problem and made significant progress. This paper surveys recent advances and new frontiers in vision-language pre-training (VLP), including image-text and video-text pre-training. To give readers a better overall grasp of VLP, we first review its recent advances in five aspects: feature extraction, model architecture, pre-training objectives, pre-training datasets, and downstream tasks. Then, we summarize the specific VLP models in detail. Finally, we discuss the new frontiers in VLP. To the best of our knowledge, this is the first survey focused on VLP. We hope that this survey can shed light on future research in the VLP field.
Denoising Aggregation of Graph Neural Networks by Using Principal Component Analysis

To avoid the overfitting phenomenon that appeared in performing graph neural networks (GNNs) on test examples, the feature encoding scheme of GNNs usually introduces the dropout procedure. However, after learning latent node representations under this scheme, Gaussian noise produced by the dropout operation is inevitably transmitted into the next neighborhood aggregation step, which necessarily hampers the unbiased aggregation ability of GNN models. To address this issue, in this article, we present a novel aggregator, denoising aggregation (DNAG), which utilizes principal component analysis (PCA) to preserve the aggregated real signals from neighboring features and simultaneously filter out the Gaussian noise. The idea is different from using PCA on traditional applications to reduce the feature dimension. We regard PCA as an aggregator to compress the neighboring node features to have better expressive denoising power. We propose new training architectures to simplify the intensive computation of PCA in DNAG. Numerical experiments show the apparent superiority of the proposed DNAG models in gaining more denoising capability and achieving the state of the art for a set of predictive tasks on several graph-structured datasets.
Self-Sovereign Identity Solution for Blockchain-Based Land Registry System: A Comparison

Providing an identity solution is essential for a reliable blockchain-based land registry system. A secure, privacy-preserving, and efficient identity solution is essential but challenging. This paper examines the current literature and provides a systematic literature review in three stages based on the three research questions (RQ) that show the assessment and interpretation process step by step. Based on the parameters and RQ specified in the research methodology section, a total of 43 primary articles have been selected from the 251 articles extracted from various scientific databases. The majority of these articles are concerned with evaluating the existing self-sovereign identity (SSI) solutions and their role in the blockchain-based land registry system to address the compliance issues in the existing SSI solutions with SSI principles and find the best possible SSI solution to address the identity problems in the land registry. The existing digital identity solutions cannot handle the requirements of the identity principle and are prone to various limitations like centralization and dependency on third parties that further augment the chance of security threats. SSI has been designed to overcome these limitations and provide a secure, reliable, and efficient identity solution that gives complete control to the users over their personal identity information (PII). This paper reviews the existing SSI solutions, evaluates them based on the SSI principles, and comes up with the best possible SSI solution for a blockchain-based land registry system. It further provides a detailed investigation of each SSI solution to present its functionalities and limitations for further improvement.
Accessible Visualization via Natural Language Descriptions: A Four-Level Model of Semantic Content

Natural language descriptions sometimes accompany visualizations to better communicate and contextualize their insights, and to improve their accessibility for readers with disabilities. However, it is difficult to evaluate the usefulness of these descriptions, and how effectively they improve access to meaningful information, because we have little understanding of the semantic content they convey, and how different readers receive this content. In response, we introduce a conceptual model for the semantic content conveyed by natural language descriptions of visualizations. Developed through a grounded theory analysis of 2,147 sentences, our model spans four levels of semantic content: enumerating visualization construction properties (e.g., marks and encodings); reporting statistical concepts and relations (e.g., extrema and correlations); identifying perceptual and cognitive phenomena (e.g., complex trends and patterns); and elucidating domain-specific insights (e.g., social and political context). To demonstrate how our model can be applied to evaluate the effectiveness of visualization descriptions, we conduct a mixed-methods evaluation with 30 blind and 90 sighted readers, and find that these reader groups differ significantly on which semantic content they rank as most useful. Together, our model and findings suggest that access to meaningful information is strongly reader-specific, and that research in automatic visualization captioning should orient toward descriptions that more richly communicate overall trends and statistics, sensitive to reader preferences. Our work further opens a space of research on natural language as a data interface coequal with visualization.
A Parametrized Comparative Analysis of Performance Between Proposed Adaptive and Personalized Tutoring System “Seis Tutor” With Existing Online Tutoring System

Face-to-face tutoring offers a learning environment that best suits the learner’s preferences (learning styles) and grasping levels (learning levels). This cognitive intelligence has been blended in our proposed intelligent tutoring system christened as “Seis Tutor”. In this paper, we have detailed the architecture of Seis Tutor system and compared it with other existing traditional tutoring systems. Further, the performance of Seis Tutor has been evaluated in terms of personalization and adaptation through a comparison with some existing tutoring systems, i.e., My Moodle, Course-Builder, and Teachable.
Networks of Ethereum Non-Fungible Tokens: A graph-based analysis of the ERC-721 ecosystem

Non-fungible tokens (NFTs) as a decentralized proof of ownership represent one of the main reasons why Ethereum is a disruptive technology. This paper presents the first systematic study of the interactions occurring in a number of NFT ecosystems. We illustrate how to retrieve transaction data available on the blockchain and structure it as a graph-based model. Thanks to this methodology, we are able to study for the first time the topological structure of NFT networks and show that their properties (degree distribution and others) are similar to those of interaction graphs in social networks. Time-dependent analysis metrics, useful to characterize market influencers and interactions between different wallets, are also introduced. Based on those, we identify across a number of NFT networks the widespread presence of both investors accumulating NFTs and individuals who make large profits.
Hybrid Transformer with Multi-level Fusion for Multimodal Knowledge Graph Completion

Multimodal Knowledge Graphs (MKGs), which organize visual-text factual knowledge, have recently been successfully applied to tasks such as information retrieval, question answering, and recommendation system. Since most MKGs are far from complete, extensive knowledge graph completion studies have been proposed focusing on the multimodal entity, relation extraction and link prediction. However, different tasks and modalities require changes to the model architecture, and not all images/objects are relevant to text input, which hinders the applicability to diverse real-world scenarios. In this paper, we propose a hybrid transformer with multi-level fusion to address those issues. Specifically, we leverage a hybrid transformer architecture with unified input-output for diverse multimodal knowledge graph completion tasks. Moreover, we propose multi-level fusion, which integrates visual and text representation via coarse-grained prefix-guided interaction and fine-grained correlation-aware fusion modules. We conduct extensive experiments to validate that our MKGformer can obtain SOTA performance on four datasets of multimodal link prediction, multimodal RE, and multimodal NER1. https://github.com/zjunlp/MKGformer.
HAM: Hybrid attention module in deep convolutional neural networks for image classification

• Proposing an attention module: Hybrid Attention Module (HAM). • HAM can be embedded into any state-of-the-art CNN architectures. • HAM improve networks performance without significantly increasing parameters. • Compared with other state-of-the-art attention modules, HAM achieve better performance on the standard datasets. • On STL-10 datasets, HAM can further reduce the negative impact of less data on the performance as networks go deeper. Recently, many researches have demonstrated that the attention mechanism has great potential in improving the performance of deep convolutional neural networks (CNNs). However, the existing methods either ignore the importance of using channel attention and spatial attention mechanisms simultaneously or bring much additional model complexity. In order to achieve a balance between performance and model complexity, we propose the Hybrid Attention Module (HAM), a really lightweight yet efficient attention module. Given an intermediate feature map as the input feature, HAM firstly produces one channel attention map and one channel refined feature through the channel submodule, and then based on the channel attention map, the spatial submodule divides the channel refined feature into two groups along the channel axis to generate a pair of spatial attention descriptors. By applying saptial attention descriptors, the spatial submodule generates the final refined feature which can adaptively emphasize the important regions. Besides, HAM is a simple and general module, it can be embedded into various mainstream deep CNN architectures seamlessly and can be trained with base CNNs in the end-to-end way. We evaluate HAM through abundant of experiments on CIFAR-10, CIFAR-100 and STL-10 datasets. The experimental results show that HAM-integrated networks achieve accuracy improvements and further reduce the negative impact of less training data on deeper networks performance than its counterparts, which proves the effectiveness of HAM.
<b>stringi</b>: Fast and Portable Character String Processing in <i>R</i>

Effective processing of character strings is required at various stages of data analysis pipelines: from data cleansing and preparation, through information extraction, to report generation. Pattern searching, string collation and sorting, normalization, transliteration, and formatting are ubiquitous in text mining, natural language processing, and bioinformatics. This paper discusses and demonstrates how and why stringi, a mature R package for fast and portable handling of string data based on ICU (International Components for Unicode), should be included in each statistician's or data scientist's repertoire to complement their numerical computing and data wrangling skills.
Towards Universal Sequence Representation Learning for Recommender Systems

In order to develop effective sequential recommenders, a series of sequence representation learning (SRL) methods are proposed to model historical user behaviors. Most existing SRL methods rely on explicit item IDs for developing the sequence models to better capture user preference. Though effective to some extent, these methods are difficult to be transferred to new recommendation scenarios, due to the limitation by explicitly modeling item IDs. To tackle this issue, we present a novel universal sequence representation learning approach, named UniSRec. The proposed approach utilizes the associated description text of items to learn transferable representations across different recommendation scenarios. For learning universal item representations, we design a lightweight item encoding architecture based on parametric whitening and mixture-of-experts enhanced adaptor. For learning universal sequence representations, we introduce two contrastive pre-training tasks by sampling multi-domain negatives. With the pre-trained universal sequence representation model, our approach can be effectively transferred to new recommendation domains or platforms in a parameter-efficient way, under either inductive or transductive settings. Extensive experiments conducted on real-world datasets demonstrate the effectiveness of the proposed approach. Especially, our approach also leads to a performance improvement in a cross-platform setting, showing the strong transferability of the proposed universal SRL method. The code and pre-trained model are available at: https://github.com/RUCAIBox/UniSRec.
A Reinforcement Learning Method for Constraint-Satisfied Services Composition

With increasing adoption and presence of Web services, service composition becomes an effective way to construct software applications. Composite services need to satisfy both the functional and the non-functional requirements. Traditional methods usually assume that the quality of service (QoS) and the behaviors of services are deterministic, and they execute the composite service after all the component services are selected. It is difficult to guarantee the satisfaction of user constraints and the successful execution of the composite service. This paper models the constraint-satisfied service composition (CSSC) problem as a Markov decision process (MDP), namely CSSC-MDP, and designs a Q-learning algorithm to solve the model. CSSC-MDP takes the uncertainty of QoS and service behavior into account, and selects a component service after the execution of previous services. Thus, CSSC-MDP can select the globally optimal service based on the constraints which need the following services to satisfy. In the case of selected service failure, CSSC-MDP can timely provide the optimal alternative service. Simulation experiments show that the proposed method can successfully solve the CSSC problem of different sizes. Comparing with three representative methods, CSSC-MDP has obvious advantages, especially in terms of the success rate of service composition.
Distributed Gauss–Newton Method for State Estimation Using Belief Propagation

We present a novel distributed Gauss-Newton method for the non-linear state estimation (SE) model based on a probabilistic inference method called belief propagation (BP). The main novelty of our work comes from applying BP sequentially over a sequence of linear approximations of the SE model, akin to what is done by the Gauss-Newton method. The resulting iterative Gauss-Newton belief propagation (GN-BP) algorithm can be interpreted as a distributed Gauss-Newton method with the same accuracy as the centralized SE, however, introducing a number of advantages of the BP framework. The paper provides extensive numerical study of the GN-BP algorithm, provides details on its convergence behavior, and gives a number of useful insights for its implementation.
Unsupervised Real-Time Control Through Variational Empowerment

Intrinsic motivation is vital for living beings. It enables skill acquisitions, triggers explorative behaviour, and hence enhances cognitive capabilities. One way of formalising the variety of behaviours induced by intrinsic motivation is empowerment, an information-theoretic measure that encodes the influence an agent exerts on its environment. Formally, empowerment is the maximum mutual information between actions and the resulting states which is prohibitively hard to compute, especially in nonlinear continuous spaces. In this work, we introduce a method for efficiently computing a lower bound on empowerment, enabling its use as an unsupervised cost function for real-time control. We demonstrate that our algorithm can reliably handle continuous dynamical systems even when system dynamics are learnt from raw data. The resulting empowerment-maximizing policies consistently drive the agents into states with high potential impact.
A Reinforcement Learning-Based Mixed Job Scheduler Scheme for Grid or IaaS Cloud

Job scheduling is a necessary prerequisite for performance optimization and resource management in the cloud computing system. Focusing on accurate scaled cloud computing environment and efficient job scheduling under Virtual Machine (VM) resource and Server Level Agreement (SLA) constraints, we introduce the architecture of cloud computing platform and optimization job scheduling scheme in this study. The system model is comprised of clearly defined separate constituent parts, including portal, job scheduler, and resources pool. By analyzing the execution process of user jobs, we designed a novel job scheduling scheme based on reinforcement learning to minimize the makespan and Average Waiting Time (AWT) under the VM resource and deadline constraints, and employ parallel multi-age parallel technologies to balance the exploration and exploitation in learning process and accelerate the convergence of Q-learning algorithm. Both simulation and real cloud platform experiment results demonstrate the efficiency of the proposed job scheduling scheme.