OpenRaiser/Intern-Atlas huggingface dataset:
Utilizing Out-Domain Datasets to Enhance Multi-Task Citation Analysis
Venue: arXiv cs.IR 2022
Year: 2022

Citations are generally analyzed using only quantitative measures while excluding qualitative aspects such as sentiment and intent. However, qualitative aspects provide deeper insights into the impact of a scientific research artifact and make it possible to focus on relevant literature free from bias associated with quantitative aspects. Therefore, it is possible to rank and categorize papers based on their sentiment and intent. For this purpose, larger citation sentiment datasets are required. However, from a time and cost perspective, curating a large citation sentiment dataset is a challenging task. Particularly, citation sentiment analysis suffers from both data scarcity and tremendous costs for dataset annotation. To overcome the bottleneck of data scarcity in the citation analysis domain we explore the impact of out-domain data during training to enhance the model performance. Our results emphasize the use of different scheduling methods based on the use case. We empirically found that a model trained using sequential data scheduling is more suitable for domain-specific usecases. Conversely, shuffled data feeding achieves better performance on a cross-domain task. Based on our findings, we propose an end-to-end trainable multi-task model that covers the sentiment and intent analysis that utilizes out-domain datasets to overcome the data scarcity.
Recognizing Concepts and Recognizing Musical Themes. A Quantum Semantic Analysis
Venue: arXiv cs.LG 2022
Year: 2022

How are abstract concepts and musical themes recognized on the basis of some previous experience? It is interesting to compare the different behaviors of human and of artificial intelligences with respect to this problem. Generally, a human mind that abstracts a concept (say, table) from a given set of known examples creates a table-Gestalt: a kind of vague and out of focus image that does not fully correspond to a particular table with well determined features. A similar situation arises in the case of musical themes. Can the construction of a gestaltic pattern, which is so natural for human minds, be taught to an intelligent machine? This problem can be successfully discussed in the framework of a quantum approach to pattern recognition and to machine learning. The basic idea is replacing classical data sets with quantum data sets, where either objects or musical themes can be formally represented as pieces of quantum information, involving the uncertainties and the ambiguities that characterize the quantum world. In this framework, the intuitive concept of Gestalt can be simulated by the mathematical concept of positive centroid of a given quantum data set. Accordingly, the crucial problem "how can we classify a new object or a new musical theme (we have listened to) on the basis of a previous experience?" can be dealt with in terms of some special quantum similarity-relations. Although recognition procedures are different for human and for artificial intelligences, there is a common method of "facing the problems" that seems to work in both cases.
DIOT: Detecting Implicit Obstacles from Trajectories
Venue: arXiv cs.DB 2022
Year: 2022

In this paper, we study a new data mining problem of obstacle detection from trajectory data. Intuitively, given two kinds of trajectories, i.e., reference and query trajectories, the obstacle is a region such that most query trajectories need to bypass this region, whereas the reference trajectories can go through as usual. We introduce a density-based definition for the obstacle based on a new normalized Dynamic Time Warping (nDTW) distance and the density functions tailored for the sub-trajectories to estimate the density variations. With this definition, we introduce a novel framework \textsf{DIOT} that utilizes the depth-first search method to detect implicit obstacles. We conduct extensive experiments over two real-life data sets. The experimental results show that \textsf{DIOT} can capture the nature of obstacles yet detect the implicit obstacles efficiently and effectively. Code is available at \url{https://github.com/1flei/obstacle}.
An Overview on Machine Translation Evaluation
Venue: arXiv cs.CL 2022
Year: 2022

Since the 1950s, machine translation (MT) has become one of the important tasks of AI and development, and has experienced several different periods and stages of development, including rule-based methods, statistical methods, and recently proposed neural network-based learning methods. Accompanying these staged leaps is the evaluation research and development of MT, especially the important role of evaluation methods in statistical translation and neural translation research. The evaluation task of MT is not only to evaluate the quality of machine translation, but also to give timely feedback to machine translation researchers on the problems existing in machine translation itself, how to improve and how to optimise. In some practical application fields, such as in the absence of reference translations, the quality estimation of machine translation plays an important role as an indicator to reveal the credibility of automatically translated target languages. This report mainly includes the following contents: a brief history of machine translation evaluation (MTE), the classification of research methods on MTE, and the the cutting-edge progress, including human evaluation, automatic evaluation, and evaluation of evaluation methods (meta-evaluation). Manual evaluation and automatic evaluation include reference-translation based and reference-translation independent participation; automatic evaluation methods include traditional n-gram string matching, models applying syntax and semantics, and deep learning models; evaluation of evaluation methods includes estimating the credibility of human evaluations, the reliability of the automatic evaluation, the reliability of the test set, etc. Advances in cutting-edge evaluation methods include task-based evaluation, using pre-trained language models based on big data, and lightweight optimisation models using distillation techniques.
Message passing all the way up
Venue: arXiv cs.LG 2022
Year: 2022

The message passing framework is the foundation of the immense success enjoyed by graph neural networks (GNNs) in recent years. In spite of its elegance, there exist many problems it provably cannot solve over given input graphs. This has led to a surge of research on going "beyond message passing", building GNNs which do not suffer from those limitations -- a term which has become ubiquitous in regular discourse. However, have those methods truly moved beyond message passing? In this position paper, I argue about the dangers of using this term -- especially when teaching graph representation learning to newcomers. I show that any function of interest we want to compute over graphs can, in all likelihood, be expressed using pairwise message passing -- just over a potentially modified graph, and argue how most practical implementations subtly do this kind of trick anyway. Hoping to initiate a productive discussion, I propose replacing "beyond message passing" with a more tame term, "augmented message passing".
Continual Auxiliary Task Learning
Venue: arXiv cs.LG 2022
Year: 2022

Learning auxiliary tasks, such as multiple predictions about the world, can provide many benefits to reinforcement learning systems. A variety of off-policy learning algorithms have been developed to learn such predictions, but as yet there is little work on how to adapt the behavior to gather useful data for those off-policy predictions. In this work, we investigate a reinforcement learning system designed to learn a collection of auxiliary tasks, with a behavior policy learning to take actions to improve those auxiliary predictions. We highlight the inherent non-stationarity in this continual auxiliary task learning problem, for both prediction learners and the behavior learner. We develop an algorithm based on successor features that facilitates tracking under non-stationary rewards, and prove the separation into learning successor features and rewards provides convergence rate improvements. We conduct an in-depth study into the resulting multi-prediction learning system.
Incorporating social norms into a configurable agent-based model of the decision to perform commuting behaviour
Venue: arXiv cs.MA 2022
Year: 2022

Interventions to increase active commuting have been recommended as a method to increase population physical activity, but evidence is mixed. Social norms related to travel behaviour may influence the uptake of active commuting interventions but are rarely considered in their design and evaluation. In this study we develop an agent-based model that incorporates social norms related to travel behaviour and demonstrate the utility of this through implementing car-free Wednesdays. A synthetic population of Waltham Forest, London, UK was generated using a microsimulation approach with data from the UK Census 2011 and UK HLS datasets. An agent-based model was created using this synthetic population which modelled how the actions of peers and neighbours, subculture, habit, weather, bicycle ownership, car ownership, environmental supportiveness, and congestion affect the decision to trave. The developed model (MOTIVATE) is a configurable agent-based model where social norms related to travel behaviour are used to provide a more realistic representation of the socio-ecological systems in which active commuting interventions may be deployed. The utility of this model is demonstrated using car-free days as a hypothetical intervention. In the control scenario, the odds of active travel were plausible at 0.091 (89% HPDI: [0.091, 0.091]). Compared to the control scenario, the odds of active travel were increased by 70.3% (89% HPDI: [70.3%, 70.3%]), in the intervention scenario, on non-car-free days; the effect is sustained to non-car-free days. The model is a useful tool for investigating the effect of how social networks and social norms influence the effectiveness of various interventions. If configured using real-world built environment data, it may be useful for investigating how social norms interact with the built environment to cause the emergence of commuting conventions.
SIPOMDPLite-Net: Lightweight, Self-Interested Learning and Planning in POSGs with Sparse Interactions
Venue: arXiv cs.MA 2022
Year: 2022

This work introduces sIPOMDPLite-net, a deep neural network (DNN) architecture for decentralized, self-interested agent control in partially observable stochastic games (POSGs) with sparse interactions between agents. The network learns to plan in contexts modeled by the interactive partially observable Markov decision process (I-POMDP) Lite framework and uses hierarchical value iteration networks to simulate the solution of nested MDPs, which I-POMDP Lite attributes to the other agent to model its behavior and predict its intention. We train sIPOMDPLite-net with expert demonstrations on small two-agent Tiger-grid tasks, for which it accurately learns the underlying I-POMDP Lite model and near-optimal policy, and the policy continues to perform well on larger grids and real-world maps. As such, sIPOMDPLite-net shows good transfer capabilities and offers a lighter learning and planning approach for individual, self-interested agents in multiagent settings.
Neural Network based Successor Representations of Space and Language
Venue: arXiv cs.AI 2022
Year: 2022

How does the mind organize thoughts? The hippocampal-entorhinal complex is thought to support domain-general representation and processing of structural knowledge of arbitrary state, feature and concept spaces. In particular, it enables the formation of cognitive maps, and navigation on these maps, thereby broadly contributing to cognition. It has been proposed that the concept of multi-scale successor representations provides an explanation of the underlying computations performed by place and grid cells. Here, we present a neural network based approach to learn such representations, and its application to different scenarios: a spatial exploration task based on supervised learning, a spatial navigation task based on reinforcement learning, and a non-spatial task where linguistic constructions have to be inferred by observing sample sentences. In all scenarios, the neural network correctly learns and approximates the underlying structure by building successor representations. Furthermore, the resulting neural firing patterns are strikingly similar to experimentally observed place and grid cell firing patterns. We conclude that cognitive maps and neural network-based successor representations of structured knowledge provide a promising way to overcome some of the short comings of deep learning towards artificial general intelligence.
Real-time Over-the-air Adversarial Perturbations for Digital Communications using Deep Neural Networks
Venue: arXiv cs.CR 2022
Year: 2022

Deep neural networks (DNNs) are increasingly being used in a variety of traditional radiofrequency (RF) problems. Previous work has shown that while DNN classifiers are typically more accurate than traditional signal processing algorithms, they are vulnerable to intentionally crafted adversarial perturbations which can deceive the DNN classifiers and significantly reduce their accuracy. Such intentional adversarial perturbations can be used by RF communications systems to avoid reactive-jammers and interception systems which rely on DNN classifiers to identify their target modulation scheme. While previous research on RF adversarial perturbations has established the theoretical feasibility of such attacks using simulation studies, critical questions concerning real-world implementation and viability remain unanswered. This work attempts to bridge this gap by defining class-specific and sample-independent adversarial perturbations which are shown to be effective yet computationally feasible in real-time and time-invariant. We demonstrate the effectiveness of these attacks over-the-air across a physical channel using software-defined radios (SDRs). Finally, we demonstrate that these adversarial perturbations can be emitted from a source other than the communications device, making these attacks practical for devices that cannot manipulate their transmitted signals at the physical layer.
Unravelling Token Ecosystem of EOSIO Blockchain
Venue: arXiv cs.CR 2022
Year: 2022

Being the largest Initial Coin Offering project, EOSIO has attracted great interest in cryptocurrency markets. Despite its popularity and prosperity (e.g., 26,311,585,008 token transactions occurred from June 8, 2018 to Aug. 5, 2020), there is almost no work investigating the EOSIO token ecosystem. To fill this gap, we are the first to conduct a systematic investigation on the EOSIO token ecosystem by conducting a comprehensive graph analysis on the entire on-chain EOSIO data (nearly 135 million blocks). We construct token creator graphs, token-contract creator graphs, token holder graphs, and token transfer graphs to characterize token creators, holders, and transfer activities. Through graph analysis, we have obtained many insightful findings and observed some abnormal trading patterns. Moreover, we propose a fake-token detection algorithm to identify tokens generated by fake users or fake transactions and analyze their corresponding manipulation behaviors. Evaluation results also demonstrate the effectiveness of our algorithm.
FUNQUE: Fusion of Unified Quality Evaluators
Venue: arXiv cs.CV 2022
Year: 2022

Fusion-based quality assessment has emerged as a powerful method for developing high-performance quality models from quality models that individually achieve lower performances. A prominent example of such an algorithm is VMAF, which has been widely adopted as an industry standard for video quality prediction along with SSIM. In addition to advancing the state-of-the-art, it is imperative to alleviate the computational burden presented by the use of a heterogeneous set of quality models. In this paper, we unify "atom" quality models by computing them on a common transform domain that accounts for the Human Visual System, and we propose FUNQUE, a quality model that fuses unified quality evaluators. We demonstrate that in comparison to the state-of-the-art, FUNQUE offers significant improvements in both correlation against subjective scores and efficiency, due to computation sharing.
Performance Modeling of Metric-Based Serverless Computing Platforms
Venue: arXiv cs.DC 2022
Year: 2022

Analytical performance models are very effective in ensuring the quality of service and cost of service deployment remain desirable under different conditions and workloads. While various analytical performance models have been proposed for previous paradigms in cloud computing, serverless computing lacks such models that can provide developers with performance guarantees. Besides, most serverless computing platforms still require developers' input to specify the configuration for their deployment that could affect both the performance and cost of their deployment, without providing them with any direct and immediate feedback. In previous studies, we built such performance models for steady-state and transient analysis of scale-per-request serverless computing platforms (e.g., AWS Lambda, Azure Functions, Google Cloud Functions) that could give developers immediate feedback about the quality of service and cost of their deployments. In this work, we aim to develop analytical performance models for the latest trend in serverless computing platforms that use concurrency value and the rate of requests per second for autoscaling decisions. Examples of such serverless computing platforms are Knative and Google Cloud Run (a managed Knative service by Google). The proposed performance model can help developers and providers predict the performance and cost of deployments with different configurations which could help them tune the configuration toward the best outcome. We validate the applicability and accuracy of the proposed performance model by extensive real-world experimentation on Knative and show that our performance model is able to accurately predict the steady-state characteristics of a given workload with minimal amount of data collection.
ML-Aided Collision Recovery for UHF-RFID Systems
Venue: arXiv cs.NI 2022
Year: 2022

We propose a collision recovery algorithm with the aid of machine learning (ML-aided) for passive Ultra High Frequency (UHF) Radio Frequency Identification (RFID) systems. The proposed method aims at recovering the tags under collision to improve the system performance. We first estimate the number of tags from the collided signal by utilizing machine learning tools and show that the number of colliding tags can be estimated with high accuracy. Second, we employ a simple yet effective deep learning model to find the experienced channel coefficients. The proposed method allows the reader to separate each tag's signal from the received one by applying maximum likelihood decoding. We perform simulations to illustrate that the use of deep learning is highly beneficial and demonstrate that the proposed approach boosts the throughput performance of the standard framed slotted ALOHA (FSA) protocol from 0.368 to 1.837, where the receiver is equipped with a single antenna and capable of decoding up to 4 tags.
An End-to-End Cascaded Image Deraining and Object Detection Neural Network
Venue: arXiv cs.CV 2022
Year: 2022

While the deep learning-based image deraining methods have made great progress in recent years, there are two major shortcomings in their application in real-world situations. Firstly, the gap between the low-level vision task represented by rain removal and the high-level vision task represented by object detection is significant, and the low-level vision task can hardly contribute to the high-level vision task. Secondly, the quality of the deraining dataset needs to be improved. In fact, the rain lines in many baselines have a large gap with the real rain lines, and the resolution of the deraining dataset images is generally not ideally. Meanwhile, there are few common datasets for both the low-level vision task and the high-level vision task. In this paper, we explore the combination of the low-level vision task with the high-level vision task. Specifically, we propose an end-to-end object detection network for reducing the impact of rainfall, which consists of two cascaded networks, an improved image deraining network and an object detection network, respectively. We also design the components of the loss function to accommodate the characteristics of the different sub-networks. We then propose a dataset based on the KITTI dataset for rainfall removal and object detection, on which our network surpasses the state-of-the-art with a significant improvement in metrics. Besides, our proposed network is measured on driving videos collected by self-driving vehicles and shows positive results for rain removal and object detection.
Neural Generalised AutoRegressive Conditional Heteroskedasticity
Venue: arXiv cs.LG 2022
Year: 2022

We propose Neural GARCH, a class of methods to model conditional heteroskedasticity in financial time series. Neural GARCH is a neural network adaptation of the GARCH 1,1 model in the univariate case, and the diagonal BEKK 1,1 model in the multivariate case. We allow the coefficients of a GARCH model to be time varying in order to reflect the constantly changing dynamics of financial markets. The time varying coefficients are parameterised by a recurrent neural network that is trained with stochastic gradient variational Bayes. We propose two variants of our model, one with normal innovations and the other with Students t innovations. We test our models on a wide range of univariate and multivariate financial time series, and we find that the Neural Students t model consistently outperforms the others.
Efficient CDF Approximations for Normalizing Flows
Venue: arXiv cs.LG 2022
Year: 2022

Normalizing flows model a complex target distribution in terms of a bijective transform operating on a simple base distribution. As such, they enable tractable computation of a number of important statistical quantities, particularly likelihoods and samples. Despite these appealing properties, the computation of more complex inference tasks, such as the cumulative distribution function (CDF) over a complex region (e.g., a polytope) remains challenging. Traditional CDF approximations using Monte-Carlo techniques are unbiased but have unbounded variance and low sample efficiency. Instead, we build upon the diffeomorphic properties of normalizing flows and leverage the divergence theorem to estimate the CDF over a closed region in target space in terms of the flux across its \emph{boundary}, as induced by the normalizing flow. We describe both deterministic and stochastic instances of this estimator: while the deterministic variant iteratively improves the estimate by strategically subdividing the boundary, the stochastic variant provides unbiased estimates. Our experiments on popular flow architectures and UCI benchmark datasets show a marked improvement in sample efficiency as compared to traditional estimators.