OpenRaiser/Intern-Atlas huggingface dataset:
zbMATH Open: API Solutions and Research Challenges
Venue: arXiv cs.DL 2021
Year: 2021

We present zbMATH Open, the most comprehensive collection of reviews and bibliographic metadata of scholarly literature in mathematics. Besides our website https://zbMATH.org which is openly accessible since the beginning of this year, we provide API endpoints to offer our data. The API improves interoperability with others, i.e., digital libraries, and allows using our data for research purposes. In this article, we (1) illustrate the current and future overview of the services offered by zbMATH; (2) present the initial version of the zbMATH links API; (3) analyze potentials and limitations of the links API based on the example of the NIST Digital Library of Mathematical Functions; (4) and finally, present the zbMATH Open dataset as a research resource and discuss connected open research problems.
Self-Adaptive Swarm System (SASS)
Venue: arXiv cs.MA 2021
Year: 2021

Distributed artificial intelligence (DAI) studies artificial intelligence entities working together to reason, plan, solve problems, organize behaviors and strategies, make collective decisions and learn. This Ph.D. research proposes a principled Multi-Agent Systems (MAS) cooperation framework -- Self-Adaptive Swarm System (SASS) -- to bridge the fourth level automation gap between perception, communication, planning, execution, decision-making, and learning.
On the Evolution of Neuron Communities in a Deep Learning Architecture
Venue: arXiv cs.LG 2021
Year: 2021

Deep learning techniques are increasingly being adopted for classification tasks over the past decade, yet explaining how deep learning architectures can achieve state-of-the-art performance is still an elusive goal. While all the training information is embedded deeply in a trained model, we still do not understand much about its performance by only analyzing the model. This paper examines the neuron activation patterns of deep learning-based classification models and explores whether the models' performances can be explained through neurons' activation behavior. We propose two approaches: one that models neurons' activation behavior as a graph and examines whether the neurons form meaningful communities, and the other examines the predictability of neurons' behavior using entropy. Our comprehensive experimental study reveals that both the community quality and entropy can provide new insights into the deep learning models' performances, thus paves a novel way of explaining deep learning models directly from the neurons' activation pattern.
Pricing Ordered Items
Venue: arXiv cs.GT 2021
Year: 2021

We study the revenue guarantees and approximability of item pricing. Recent work shows that with $n$ heterogeneous items, item-pricing guarantees an $O(\log n)$ approximation to the optimal revenue achievable by any (buy-many) mechanism, even when buyers have arbitrarily combinatorial valuations. However, finding good item prices is challenging -- it is known that even under unit-demand valuations, it is NP-hard to find item prices that approximate the revenue of the optimal item pricing better than $O(\sqrt{n})$. Our work provides a more fine-grained analysis of the revenue guarantees and computational complexity in terms of the number of item ``categories'' which may be significantly fewer than $n$. We assume the items are partitioned in $k$ categories so that items within a category are totally-ordered and a buyer's value for a bundle depends only on the best item contained from every category. We show that item-pricing guarantees an $O(\log k)$ approximation to the optimal (buy-many) revenue and provide a PTAS for computing the optimal item-pricing when $k$ is constant. We also provide a matching lower bound showing that the problem is (strongly) NP-hard even when $k=1$. Our results naturally extend to the case where items are only partially ordered, in which case the revenue guarantees and computational complexity depend on the width of the partial ordering, i.e. the largest set for which no two items are comparable.
Learning subtree pattern importance for Weisfeiler-Lehmanbased graph kernels
Venue: arXiv cs.LG 2021
Year: 2021

Graph is an usual representation of relational data, which are ubiquitous in manydomains such as molecules, biological and social networks. A popular approach to learningwith graph structured data is to make use of graph kernels, which measure the similaritybetween graphs and are plugged into a kernel machine such as a support vector machine.Weisfeiler-Lehman (WL) based graph kernels, which employ WL labeling scheme to extract subtree patterns and perform node embedding, are demonstrated to achieve great performance while being efficiently computable. However, one of the main drawbacks of ageneral kernel is the decoupling of kernel construction and learning process. For moleculargraphs, usual kernels such as WL subtree, based on substructures of the molecules, consider all available substructures having the same importance, which might not be suitable inpractice. In this paper, we propose a method to learn the weights of subtree patterns in the framework of WWL kernels, the state of the art method for graph classification task [14]. To overcome the computational issue on large scale data sets, we present an efficient learning algorithm and also derive a generalization gap bound to show its convergence. Finally, through experiments on synthetic and real-world data sets, we demonstrate the effectiveness of our proposed method for learning the weights of subtree patterns.
Blockchain for IoT Access Control: Recent Trends and Future Research Directions
Venue: arXiv cs.DC 2021
Year: 2021

With the rapid development of wireless sensor networks, smart devices, and traditional information and communication technologies, there is tremendous growth in the use of Internet of Things (IoT) applications and services in our everyday life. IoT systems deal with high volumes of data. This data can be particularly sensitive, as it may include health, financial, location, and other highly personal information. Fine-grained security management in IoT demands effective access control. Several proposals discuss access control for the IoT, however, a limited focus is given to the emerging blockchain-based solutions for IoT access control. In this paper, we review the recent trends and critical needs for blockchain-based solutions for IoT access control. We identify several important aspects of blockchain, including decentralised control, secure storage and sharing information in a trustless manner, for IoT access control including their benefits and limitations. Finally, we note some future research directions on how to converge blockchain in IoT access control efficiently and effectively.
Making Better Use of Bilingual Information for Cross-Lingual AMR Parsing
Venue: arXiv cs.CL 2021
Year: 2021

Abstract Meaning Representation (AMR) is a rooted, labeled, acyclic graph representing the semantics of natural language. As previous works show, although AMR is designed for English at first, it can also represent semantics in other languages. However, they find that concepts in their predicted AMR graphs are less specific. We argue that the misprediction of concepts is due to the high relevance between English tokens and AMR concepts. In this work, we introduce bilingual input, namely the translated texts as well as non-English texts, in order to enable the model to predict more accurate concepts. Besides, we also introduce an auxiliary task, requiring the decoder to predict the English sequences at the same time. The auxiliary task can help the decoder understand what exactly the corresponding English tokens are. Our proposed cross-lingual AMR parser surpasses previous state-of-the-art parser by 10.6 points on Smatch F1 score. The ablation study also demonstrates the efficacy of our proposed modules.
ClipGen: A Deep Generative Model for Clipart Vectorization and Synthesis
Venue: arXiv cs.GR 2021
Year: 2021

This paper presents a novel deep learning-based approach for automatically vectorizing and synthesizing the clipart of man-made objects. Given a raster clipart image and its corresponding object category (e.g., airplanes), the proposed method sequentially generates new layers, each of which is composed of a new closed path filled with a single color. The final result is obtained by compositing all layers together into a vector clipart image that falls into the target category. The proposed approach is based on an iterative generative model that (i) decides whether to continue synthesizing a new layer and (ii) determines the geometry and appearance of the new layer. We formulated a joint loss function for training our generative model, including the shape similarity, symmetry, and local curve smoothness losses, as well as vector graphics rendering accuracy loss for synthesizing clipart recognizable by humans. We also introduced a collection of man-made object clipart, ClipNet, which is composed of closed-path layers, and two designed preprocessing tasks to clean up and enrich the original raw clipart. To validate the proposed approach, we conducted several experiments and demonstrated its ability to vectorize and synthesize various clipart categories. We envision that our generative model can facilitate efficient and intuitive clipart designs for novice users and graphic designers.
Self-supervision of Feature Transformation for Further Improving Supervised Learning
Venue: arXiv cs.CV 2021
Year: 2021

Self-supervised learning, which benefits from automatically constructing labels through pre-designed pretext task, has recently been applied for strengthen supervised learning. Since previous self-supervised pretext tasks are based on input, they may incur huge additional training overhead. In this paper we find that features in CNNs can be also used for self-supervision. Thus we creatively design the \emph{feature-based pretext task} which requires only a small amount of additional training overhead. In our task we discard different particular regions of features, and then train the model to distinguish these different features. In order to fully apply our feature-based pretext task in supervised learning, we also propose a novel learning framework containing multi-classifiers for further improvement. Original labels will be expanded to joint labels via self-supervision of feature transformations. With more semantic information provided by our self-supervised tasks, this approach can train CNNs more effectively. Extensive experiments on various supervised learning tasks demonstrate the accuracy improvement and wide applicability of our method.
Attacking Adversarial Attacks as A Defense
Venue: arXiv cs.LG 2021
Year: 2021

It is well known that adversarial attacks can fool deep neural networks with imperceptible perturbations. Although adversarial training significantly improves model robustness, failure cases of defense still broadly exist. In this work, we find that the adversarial attacks can also be vulnerable to small perturbations. Namely, on adversarially-trained models, perturbing adversarial examples with a small random noise may invalidate their misled predictions. After carefully examining state-of-the-art attacks of various kinds, we find that all these attacks have this deficiency to different extents. Enlightened by this finding, we propose to counter attacks by crafting more effective defensive perturbations. Our defensive perturbations leverage the advantage that adversarial training endows the ground-truth class with smaller local Lipschitzness. By simultaneously attacking all the classes, the misled predictions with larger Lipschitzness can be flipped into correct ones. We verify our defensive perturbation with both empirical experiments and theoretical analyses on a linear model. On CIFAR10, it boosts the state-of-the-art model from 66.16% to 72.66% against the four attacks of AutoAttack, including 71.76% to 83.30% against the Square attack. On ImageNet, the top-1 robust accuracy of FastAT is improved from 33.18% to 38.54% under the 100-step PGD attack.
Eight Reasons Why Cybersecurity on Novel Generations of Brain-Computer Interfaces Must Be Prioritized
Venue: arXiv cs.CR 2021
Year: 2021

This article presents eight neural cyberattacks affecting spontaneous neural activity, inspired by well-known cyberattacks from the computer science domain: Neural Flooding, Neural Jamming, Neural Scanning, Neural Selective Forwarding, Neural Spoofing, Neural Sybil, Neural Sinkhole and Neural Nonce. These cyberattacks are based on the exploitation of vulnerabilities existing in the new generation of Brain-Computer Interfaces. After presenting their formal definitions, the cyberattacks have been implemented over a neuronal simulation. To evaluate the impact of each cyberattack, they have been implemented in a Convolutional Neural Network (CNN) simulating a portion of a mouse's visual cortex. This implementation is based on existing literature indicating the similarities that CNNs have with neuronal structures from the visual cortex. Some conclusions are also provided, indicating that Neural Nonce and Neural Jamming are the most impactful cyberattacks for short-term effects, while Neural Scanning and Neural Nonce are the most damaging for long-term effects.
Reachability Problems for Transmission Graphs
Venue: arXiv cs.CG 2021
Year: 2021

Let $P$ be a set of $n$ points in the plane where each point $p$ of $P$ is associated with a radius $r_p>0$.The transmission graph $G=(P,E)$ of $P$ is defined as the directed graph such that $E$ contains an edge from $p$ to $q$ if and only if $|pq|\leq r_p$ for any two points $p$ and $q$ in $P$, where $|pq|$ denotes the Euclidean distance between $p$ and $q$. In this paper, we present a data structure of size $O(n^{5/3})$ such that for any two points in $P$, we can check in $O(n^{2/3})$ time if there is a path in $G$ between the two points. This is the first data structure for answering reachability queries whose performance depends only on $n$ but not on the number of edges.
A Compact Model for Scalable MTJ Simulation
Venue: arXiv cs.ET 2021
Year: 2021

This paper presents a physics-based modeling framework for the analysis and transient simulation of circuits containing Spin-Transfer Torque (STT) Magnetic Tunnel Junction (MTJ) devices. The framework provides the tools to analyze the stochastic behavior of MTJs and to generate Verilog-A compact models for their simulation in large VLSI designs, addressing the need for an industry-ready model accounting for real-world reliability and scalability requirements. Device dynamics are described by the Landau-Lifshitz-Gilbert-Slonczewsky (s-LLGS ) stochastic magnetization considering Voltage-Controlled Magnetic Anisotropy (VCMA) and the non-negligible statistical effects caused by thermal noise. Model behavior is validated against the OOMMF magnetic simulator and its performance is characterized on a 1-Mb 28 nm Magnetoresistive-RAM (MRAM) memory product.
Satellite- and Cache-assisted UAV: A Joint Cache Placement, Resource Allocation, and Trajectory Optimization for 6G Aerial Networks
Venue: arXiv cs.IT 2021
Year: 2021

This paper considers LEO satellite- and cache-assisted UAV communications for content delivery in terrestrial networks, which shows great potential for next-generation systems to provide ubiquitous connectivity and high capacity. Specifically, caching is provided by the UAV to reduce backhaul congestion, and the LEO satellite supports the UAV's backhaul link. In this context, we aim to maximize the minimum achievable throughput per ground user (GU) by jointly optimizing cache placement, the UAV's resource allocation, and trajectory while cache capacity and flight time are limited. The formulated problem is challenging to solve directly due to its non-convexity and combinatorial nature. To find a solution, the problem is decomposed into three sub-problems: (1) cache placement optimization with fixed UAV resources and trajectory, followed by (2) the UAV resources optimization with fixed cache placement vector and trajectory, and finally, (3) we optimize the UAV trajectory with fixed cache placement and UAV resources. Based on the solutions of sub-problems, an efficient alternating algorithm is proposed utilizing the block coordinate descent (BCD) and successive convex approximation (SCA) methods. Simulation results show that the max-min throughput and total achievable throughput enhancement can be achieved by applying our proposed algorithm instead of other benchmark schemes.
RLupus: Cooperation through emergent communication in The Werewolf social deduction game
Venue: arXiv cs.MA 2021
Year: 2021

This paper focuses on the emergence of communication to support cooperation in environments modeled as social deduction games (SDG), that are games where players communicate freely to deduce each others' hidden intentions. We first state the problem by giving a general formalization of SDG and a possible solution framework based on reinforcement learning. Next, we focus on a specific SDG, known as The Werewolf, and study if and how various forms of communication influence the outcome of the game. Experimental results show that introducing a communication signal greatly increases the winning chances of a class of players. We also study the effect of the signal's length and range on the overall performance showing a non-linear relationship.
IChannels: Exploiting Current Management Mechanisms to Create Covert Channels in Modern Processors
Venue: arXiv cs.DC 2021
Year: 2021

To operate efficiently across a wide range of workloads with varying power requirements, a modern processor applies different current management mechanisms, which briefly throttle instruction execution while they adjust voltage and frequency to accommodate for power-hungry instructions (PHIs) in the instruction stream. Doing so 1) reduces the power consumption of non-PHI instructions in typical workloads and 2) optimizes system voltage regulators' cost and area for the common use case while limiting current consumption when executing PHIs. However, these mechanisms may compromise a system's confidentiality guarantees. In particular, we observe that multilevel side-effects of throttling mechanisms, due to PHI-related current management mechanisms, can be detected by two different software contexts (i.e., sender and receiver) running on 1) the same hardware thread, 2) co-located Simultaneous Multi-Threading (SMT) threads, and 3) different physical cores. Based on these new observations on current management mechanisms, we develop a new set of covert channels, IChannels, and demonstrate them in real modern Intel processors (which span more than 70% of the entire client and server processor market). Our analysis shows that IChannels provides more than 24x the channel capacity of state-of-the-art power management covert channels. We propose practical and effective mitigations to each covert channel in IChannels by leveraging the insights we gain through a rigorous characterization of real systems.
HEAP -- The autonomous walking excavator
Venue: arXiv cs.RO 2021
Year: 2021

The demand and the potential for automation in the construction sector is unmatched, particularly for increasing environmental sustainability, improving worker safety and reducing labor shortages. We have developed an autonomous walking excavator - based one of the most versatile machines found on construction sites - as one way to begin fulfilling this potential. This article describes the process of converting an off-the-shelf construction machine into an autonomous robotic system. First we outline the necessary sensing equipment for full autonomy and the novel actuation of the legs, and compare three different complementary actuation principles for the excavator's arm. Second, we solve the state estimation problem for a general wheeled-legged robot. Beside kinematic measurements, it includes GNSS-RTK, to absolutely reference the machine on a construction site. Third, we developed individual controllers for driving, chassis balancing and arm motions allowing for fully autonomous operation. Lastly, we highlight the machine's potential in four different real-world applications, e.g. autonomous trench digging, autonomous assembly of dry stone walls, autonomous forestry work and semi-autonomous teleoperation. On top, we also share some development insights and possible future research directions.
Global Context Enhanced Graph Neural Networks for Session-based Recommendation
Venue: arXiv cs.IR 2021
Year: 2021

Session-based recommendation (SBR) is a challenging task, which aims at recommending items based on anonymous behavior sequences. Almost all the existing solutions for SBR model user preference only based on the current session without exploiting the other sessions, which may contain both relevant and irrelevant item-transitions to the current session. This paper proposes a novel approach, called Global Context Enhanced Graph Neural Networks (GCE-GNN) to exploit item transitions over all sessions in a more subtle manner for better inferring the user preference of the current session. Specifically, GCE-GNN learns two levels of item embeddings from session graph and global graph, respectively: (i) Session graph, which is to learn the session-level item embedding by modeling pairwise item-transitions within the current session; and (ii) Global graph, which is to learn the global-level item embedding by modeling pairwise item-transitions over all sessions. In GCE-GNN, we propose a novel global-level item representation learning layer, which employs a session-aware attention mechanism to recursively incorporate the neighbors' embeddings of each node on the global graph. We also design a session-level item representation learning layer, which employs a GNN on the session graph to learn session-level item embeddings within the current session. Moreover, GCE-GNN aggregates the learnt item representations in the two levels with a soft attention mechanism. Experiments on three benchmark datasets demonstrate that GCE-GNN outperforms the state-of-the-art methods consistently.
Pretrained Encoders are All You Need
Venue: arXiv cs.LG 2021
Year: 2021

Data-efficiency and generalization are key challenges in deep learning and deep reinforcement learning as many models are trained on large-scale, domain-specific, and expensive-to-label datasets. Self-supervised models trained on large-scale uncurated datasets have shown successful transfer to diverse settings. We investigate using pretrained image representations and spatio-temporal attention for state representation learning in Atari. We also explore fine-tuning pretrained representations with self-supervised techniques, i.e., contrastive predictive coding, spatio-temporal contrastive learning, and augmentations. Our results show that pretrained representations are at par with state-of-the-art self-supervised methods trained on domain-specific data. Pretrained representations, thus, yield data and compute-efficient state representations. https://github.com/PAL-ML/PEARL_v1