OpenRaiser/Intern-Atlas huggingface dataset:
Women, artificial intelligence, and key positions in collaboration networks: Towards a more equal scientific ecosystem
Venue: arXiv cs.SI 2022
Year: 2022

Scientific collaboration in almost every discipline is mainly driven by the need of sharing knowledge, expertise, and pooled resources. Science is becoming more complex which has encouraged scientists to involve more in collaborative research projects in order to better address the challenges. As a highly interdisciplinary field with a rapidly evolving scientific landscape, artificial intelligence calls for researchers with special profiles covering a diverse set of skills and expertise. Understanding gender aspects of scientific collaboration is of paramount importance, especially in a field such as artificial intelligence that has been attracting large investments. Using social network analysis, natural language processing, and machine learning and focusing on artificial intelligence publications for the period from 2000 to 2019, in this work, we comprehensively investigated the effects of several driving factors on acquiring key positions in scientific collaboration networks through a gender lens. It was found that, regardless of gender, scientific performance in terms of quantity and impact plays a crucial in possessing the "social researcher" in the network. However, subtle differences were observed between female and male researchers in acquiring the "local influencer" role.
Multi-Head Online Learning for Delayed Feedback Modeling
Venue: arXiv cs.LG 2022
Year: 2022

In online advertising, it is highly important to predict the probability and the value of a conversion (e.g., a purchase). It not only impacts user experience by showing relevant ads, but also affects ROI of advertisers and revenue of marketplaces. Unlike clicks, which often occur within minutes after impressions, conversions are expected to happen over a long period of time (e.g., 30 days for online shopping). It creates a challenge, as the true labels are only available after the long delays. Either inaccurate labels (partial conversions) are used, or models are trained on stale data (e.g., from 30 days ago). The problem is more eminent in online learning, which focuses on the live performance on the latest data. In this paper, a novel solution is presented to address this challenge using multi-head modeling. Unlike traditional methods, it directly quantizes conversions into multiple windows, such as day 1, day 2, day 3-7, and day 8-30. A sub-model is trained specifically on conversions within each window. Label freshness is maximally preserved in early models (e.g., day 1 and day 2), while late conversions are accurately utilized in models with longer delays (e.g., day 8-30). It is shown to greatly exceed the performance of known methods in online learning experiments for both conversion rate (CVR) and value per click (VPC) predictions. Lastly, as a general method for delayed feedback modeling, it can be combined with any advanced ML techniques to further improve the performance.
MAVIPER: Learning Decision Tree Policies for Interpretable Multi-Agent Reinforcement Learning
Venue: arXiv cs.LG 2022
Year: 2022

Many recent breakthroughs in multi-agent reinforcement learning (MARL) require the use of deep neural networks, which are challenging for human experts to interpret and understand. On the other hand, existing work on interpretable reinforcement learning (RL) has shown promise in extracting more interpretable decision tree-based policies from neural networks, but only in the single-agent setting. To fill this gap, we propose the first set of algorithms that extract interpretable decision-tree policies from neural networks trained with MARL. The first algorithm, IVIPER, extends VIPER, a recent method for single-agent interpretable RL, to the multi-agent setting. We demonstrate that IVIPER learns high-quality decision-tree policies for each agent. To better capture coordination between agents, we propose a novel centralized decision-tree training algorithm, MAVIPER. MAVIPER jointly grows the trees of each agent by predicting the behavior of the other agents using their anticipated trees, and uses resampling to focus on states that are critical for its interactions with other agents. We show that both algorithms generally outperform the baselines and that MAVIPER-trained agents achieve better-coordinated performance than IVIPER-trained agents on three different multi-agent particle-world environments.
A CNN with Noise Inclined Module and Denoise Framework for Hyperspectral Image Classification
Venue: arXiv cs.CV 2022
Year: 2022

Deep Neural Networks have been successfully applied in hyperspectral image classification. However, most of prior works adopt general deep architectures while ignore the intrinsic structure of the hyperspectral image, such as the physical noise generation. This would make these deep models unable to generate discriminative features and provide impressive classification performance. To leverage such intrinsic information, this work develops a novel deep learning framework with the noise inclined module and denoise framework for hyperspectral image classification. First, we model the spectral signature of hyperspectral image with the physical noise model to describe the high intraclass variance of each class and great overlapping between different classes in the image. Then, a noise inclined module is developed to capture the physical noise within each object and a denoise framework is then followed to remove such noise from the object. Finally, the CNN with noise inclined module and the denoise framework is developed to obtain discriminative features and provides good classification performance of hyperspectral image. Experiments are conducted over two commonly used real-world datasets and the experimental results show the effectiveness of the proposed method. The implementation of the proposed method and other compared methods could be accessed at https://github.com/shendu-sw/noise-physical-framework.
A Multi-domain Magneto Tunnel Junction for Racetrack Nanowire Strips
Venue: arXiv cs.ET 2022
Year: 2022

Domain-wall memory (DWM) has SRAM class access performance, low energy, high endurance, high density, and CMOS compatibility. Recently, shift reliability and processing-using-memory (PuM) proposals developed a need to count the number of parallel or anti-parallel domains in a portion of the DWM nanowire. In this paper we propose a multi-domain magneto-tunnel junction (MTJ) that can detect different resistance levels as a function of a the number of parallel or anti-parallel domains. Using detailed micromagnetic simulation with LLG, we demonstrate the multi-domain MTJ, study the benefit of its macro-size on resilience to process variation and present a macro-model for scaling the size of the multi-domain MTJ. Our results indicate scalability to seven-domains while maintaining a 16.3mV sense margin.
Molecular Absorption Effect: A Double-edged Sword of Terahertz Communications
Venue: arXiv cs.IT 2022
Year: 2022

Communications in the terahertz band (THz) (0.1--10~THz) have been regarded as a promising technology for future 6G and beyond wireless systems, to overcome the challenges of evergrowing wireless data traffic and crowded spectrum. As the frequency increases from the microwave band to the THz band, new spectrum features pose unprecedented challenges to wireless communication system design. The molecular absorption effect is one of the new THz spectrum properties, which enlarges the path loss and noise at specific frequencies. This brings in a double-edged sword for THz wireless communication systems. On one hand, from the data rate viewpoint, molecular absorption is detrimental, since it mitigates the received signal power and degrades the channel capacity. On the other hand, it is worth noticing that for wireless security and covertness, the molecular absorption effect can be utilized to safeguard THz communications among users. In this paper, the features of the molecular absorption effect and their impact on the THz system design are analyzed under various scenarios, with the ultimate goal of providing guidelines to how better exploit this unique THz phenomenon. Specifically, since the molecular absorption greatly depends on the propagation medium, different communication scenarios consisting of various media are discussed, including terrestrial, air and space, sea surface and nano-scale communications. Furthermore, two novel molecular absorption enlightened secure and covert communication schemes are presented, where the molecular absorption effect is utilized as the key and unique feature to boost security and covertness.
Deep Dense Local Feature Matching and Vehicle Removal for Indoor Visual Localization
Venue: arXiv cs.CV 2022
Year: 2022

Visual localization is an essential component of intelligent transportation systems, enabling broad applications that require understanding one's self location when other sensors are not available. It is mostly tackled by image retrieval such that the location of a query image is determined by its closest match in the previously collected images. Existing approaches focus on large scale localization where landmarks are helpful in finding the location. However, visual localization becomes challenging in small scale environments where objects are hardly recognizable. In this paper, we propose a visual localization framework that robustly finds the match for a query among the images collected from indoor parking lots. It is a challenging problem when the vehicles in the images share similar appearances and are frequently replaced such as parking lots. We propose to employ a deep dense local feature matching that resembles human perception to find correspondences and eliminating matches from vehicles automatically with a vehicle detector. The proposed solution is robust to the scenes with low textures and invariant to false matches caused by vehicles. We compare our framework with alternatives to validate our superiority on a benchmark dataset containing 267 pre-collected images and 99 query images taken from 34 sections of a parking lot. Our method achieves 86.9 percent accuracy, outperforming the alternatives.
Some equivalence relation between persistent homology and morphological dynamics
Venue: arXiv cs.CV 2022
Year: 2022

In Mathematical Morphology (MM), connected filters based on dynamics are used to filter the extrema of an image. Similarly, persistence is a concept coming from Persistent Homology (PH) and Morse Theory (MT) that represents the stability of the extrema of a Morse function. Since these two concepts seem to be closely related, in this paper we examine their relationship, and we prove that they are equal on n-D Morse functions, n $\ge$ 1. More exactly, pairing a minimum with a 1-saddle by dynamics or pairing the same 1-saddle with a minimum by persistence leads exactly to the same pairing, assuming that the critical values of the studied Morse function are unique. This result is a step further to show how much topological data analysis and mathematical morphology are related, paving the way for a more in-depth study of the relations between these two research fields.
DisinfoMeme: A Multimodal Dataset for Detecting Meme Intentionally Spreading Out Disinformation
Venue: arXiv cs.CL 2022
Year: 2022

Disinformation has become a serious problem on social media. In particular, given their short format, visual attraction, and humorous nature, memes have a significant advantage in dissemination among online communities, making them an effective vehicle for the spread of disinformation. We present DisinfoMeme to help detect disinformation memes. The dataset contains memes mined from Reddit covering three current topics: the COVID-19 pandemic, the Black Lives Matter movement, and veganism/vegetarianism. The dataset poses multiple unique challenges: limited data and label imbalance, reliance on external knowledge, multimodal reasoning, layout dependency, and noise from OCR. We test multiple widely-used unimodal and multimodal models on this dataset. The experiments show that the room for improvement is still huge for current models.
Semantic Embeddings in Semilattices
Venue: arXiv cs.DM 2022
Year: 2022

To represent anything from mathematical concepts to real-world objects, we have to resort to an encoding. Encodings, such as written language, usually assume a decoder that understands a rich shared code. A semantic embedding is a form of encoding that assumes a decoder with no knowledge, or little knowledge, beyond the basic rules of a mathematical formalism such as an algebra. Here we give a formal definition of a semantic embedding in a semilattice which can be used to resolve machine learning and classic computer science problems. Specifically, a semantic embedding of a problem is here an encoding of the problem as sentences in an algebraic theory that extends the theory of semilattices. We use the recently introduced formalism of finite atomized semilattices to study the properties of the embeddings and their finite models. For a problem embedded in a semilattice, we show that every solution has a model atomized by an irreducible subset of the non-redundant atoms of the freest model of the embedding. We give examples of semantic embeddings that can be used to find solutions for the N-Queen's completion, the Sudoku, and the Hamiltonian Path problems.
On non-detectability of non-computability and the degree of non-computability of solutions of circuit and wave equations on digital computers
Venue: arXiv cs.IT 2022
Year: 2022

It is known that there exist mathematical problems of practical relevance which cannot be computed on a Turing machine. An important example is the calculation of the first derivative of continuously differentiable functions. This paper precisely classifies the non-computability of the first derivative, and of the maximum-norm of the first derivative in the Zheng-Weihrauch hierarchy. Based on this classification, the paper investigates whether it is possible that a Turing machine detects this non-computability of the first derivative by observing the data of the problem, and whether it is possible to detect upper bounds for the peak value of the first derivative of continuously differentiable functions. So from a practical point of view, the question is whether it is possible to implement an exit-flag functionality for observing non-computability of the first derivative. This paper even studies two different types of exit-flag functionality. A strong one, where the Turing machine always has to stop, and a weak one, where the Turing machine stops if and only if the input lies within the corresponding set of interest. It will be shown that non-computability of the first derivative is not detectable by a Turing machine for two concrete examples, namely for the problem of computing the input--output behavior of simple analog circuits and for solutions of the three-dimensional wave equation. In addition, it is shown that it is even impossible to detect an upper bound for the maximum norm of the first derivative. In particular, it is shown that all three problems are not even semidecidable. Finally, we briefly discuss implications of these results for analog and quantum computing.
DFM: Dialogue Foundation Model for Universal Large-Scale Dialogue-Oriented Task Learning
Venue: arXiv cs.CL 2022
Year: 2022

Building a universal conversational agent has been a long-standing goal of the dialogue research community. Most previous works only focus on a small set of dialogue tasks. In this work, we aim to build a unified dialogue foundation model (DFM) which can be used to solve massive diverse dialogue tasks. To achieve this goal, a large-scale well-annotated dialogue dataset with rich task diversity (DialogZoo) is collected. We introduce a framework to unify all dialogue tasks and propose novel auxiliary self-supervised tasks to achieve stable training of DFM on the highly diverse large scale DialogZoo corpus. Experiments show that, compared with models of the same size, DFM can achieve state-of-the-art or competitive performance on very rich cross-domain downstream dialogue tasks. This demonstrates that DFM largely extends the ability of unified dialogue pre-trained model.
Interpretable Feature Engineering for Time Series Predictors using Attention Networks
Venue: arXiv cs.LG 2022
Year: 2022

Regression problems with time-series predictors are common in banking and many other areas of application. In this paper, we use multi-head attention networks to develop interpretable features and use them to achieve good predictive performance. The customized attention layer explicitly uses multiplicative interactions and builds feature-engineering heads that capture temporal dynamics in a parsimonious manner. Convolutional layers are used to combine multivariate time series. We also discuss methods for handling static covariates in the modeling process. Visualization and explanation tools are used to interpret the results and explain the relationship between the inputs and the extracted features. Both simulation and real dataset are used to illustrate the usefulness of the methodology. Keyword: Attention heads, Deep neural networks, Interpretable feature engineering
On the Routing Convergence Delay in the Lightning Network
Venue: arXiv cs.NI 2022
Year: 2022

Nodes in the Lightning Network synchronise routing information through a gossip protocol that makes use of a staggered broadcast mechanism. In this work, we show that the convergence delay in the network is larger than what would be expected from the protocol's specification and that payment attempt failures caused by the delay are more frequent, the larger the delay is. To this end, we measure the convergence delay incurred in the network and analyse what its primary causes are. Moreover, we further investigate and confirm our findings through a time-discrete simulation of the Lightning Network gossip protocol. We explore the use of alternative gossip protocols as well as parameter variations of the current protocol and evaluate them by the resulting bandwidth usage and convergence delay. Our research shows that there are multiple ways of lowering the convergence delay, ranging from simple parameter changes to overhauling the entire protocol.
Generic Decoding in the Cover Metric
Venue: arXiv cs.IT 2022
Year: 2022

In this paper, we study the hardness of decoding a random code endowed with the cover metric. As the cover metric lies in between the Hamming and rank metric, it presents itself as a promising candidate for code-based cryptography. We give a polynomial-time reduction from the classical Hamming-metric decoding problem, which proves the NP-hardness of the decoding problem in the cover metric. We then provide a generic decoder, following the information set decoding idea from Prange's algorithm in the Hamming metric. A study of its cost then shows that the complexity is exponential in the number of rows and columns, which is in contrast to the behaviour in the Hamming metric, where the complexity grows exponentially in the number of code symbols.
Would You Ask it that Way? Measuring and Improving Question Naturalness for Knowledge Graph Question Answering
Venue: arXiv cs.IR 2022
Year: 2022

Knowledge graph question answering (KGQA) facilitates information access by leveraging structured data without requiring formal query language expertise from the user. Instead, users can express their information needs by simply asking their questions in natural language (NL). Datasets used to train KGQA models that would provide such a service are expensive to construct, both in terms of expert and crowdsourced labor. Typically, crowdsourced labor is used to improve template-based pseudo-natural questions generated from formal queries. However, the resulting datasets often fall short of representing genuinely natural and fluent language. In the present work, we investigate ways to characterize and remedy these shortcomings. We create the IQN-KGQA test collection by sampling questions from existing KGQA datasets and evaluating them with regards to five different aspects of naturalness. Then, the questions are rewritten to improve their fluency. Finally, the performance of existing KGQA models is compared on the original and rewritten versions of the NL questions. We find that some KGQA systems fare worse when presented with more realistic formulations of NL questions. The IQN-KGQA test collection is a resource to help evaluate KGQA systems in a more realistic setting. The construction of this test collection also sheds light on the challenges of constructing large-scale KGQA datasets with genuinely NL questions.
TrustGNN: Graph Neural Network based Trust Evaluation via Learnable Propagative and Composable Nature
Venue: arXiv cs.LG 2022
Year: 2022

Trust evaluation is critical for many applications such as cyber security, social communication and recommender systems. Users and trust relationships among them can be seen as a graph. Graph neural networks (GNNs) show their powerful ability for analyzing graph-structural data. Very recently, existing work attempted to introduce the attributes and asymmetry of edges into GNNs for trust evaluation, while failed to capture some essential properties (e.g., the propagative and composable nature) of trust graphs. In this work, we propose a new GNN based trust evaluation method named TrustGNN, which integrates smartly the propagative and composable nature of trust graphs into a GNN framework for better trust evaluation. Specifically, TrustGNN designs specific propagative patterns for different propagative processes of trust, and distinguishes the contribution of different propagative processes to create new trust. Thus, TrustGNN can learn comprehensive node embeddings and predict trust relationships based on these embeddings. Experiments on some widely-used real-world datasets indicate that TrustGNN significantly outperforms the state-of-the-art methods. We further perform analytical experiments to demonstrate the effectiveness of the key designs in TrustGNN.
Automatic question generation based on sentence structure analysis using machine learning approach
Venue: arXiv cs.CL 2022
Year: 2022

Automatic question generation is one of the most challenging tasks of Natural Language Processing. It requires "bidirectional" language processing: firstly, the system has to understand the input text (Natural Language Understanding) and it then has to generate questions also in the form of text (Natural Language Generation). In this article, we introduce our framework for generating the factual questions from unstructured text in the English language. It uses a combination of traditional linguistic approaches based on sentence patterns with several machine learning methods. We firstly obtain lexical, syntactic and semantic information from an input text and we then construct a hierarchical set of patterns for each sentence. The set of features is extracted from the patterns and it is then used for automated learning of new transformation rules. Our learning process is totally data-driven because the transformation rules are obtained from a set of initial sentence-question pairs. The advantages of this approach lie in a simple expansion of new transformation rules which allows us to generate various types of questions and also in the continuous improvement of the system by reinforcement learning. The framework also includes a question evaluation module which estimates the quality of generated questions. It serves as a filter for selecting the best questions and eliminating incorrect ones or duplicates. We have performed several experiments to evaluate the correctness of generated questions and we have also compared our system with several state-of-the-art systems. Our results indicate that the quality of generated questions outperforms the state-of-the-art systems and our questions are also comparable to questions created by humans. We have also created and published an interface with all created datasets and evaluated questions, so it is possible to follow up on our work.