OpenRaiser/Intern-Atlas huggingface dataset:
Relation Classification with Entity Type Restriction
Venue: arXiv cs.CL 2021
Year: 2021

Relation classification aims to predict a relation between two entities in a sentence. The existing methods regard all relations as the candidate relations for the two entities in a sentence. These methods neglect the restrictions on candidate relations by entity types, which leads to some inappropriate relations being candidate relations. In this paper, we propose a novel paradigm, RElation Classification with ENtity Type restriction (RECENT), which exploits entity types to restrict candidate relations. Specially, the mutual restrictions of relations and entity types are formalized and introduced into relation classification. Besides, the proposed paradigm, RECENT, is model-agnostic. Based on two representative models GCN and SpanBERT respectively, RECENT_GCN and RECENT_SpanBERT are trained in RECENT. Experimental results on a standard dataset indicate that RECENT improves the performance of GCN and SpanBERT by 6.9 and 4.4 F1 points, respectively. Especially, RECENT_SpanBERT achieves a new state-of-the-art on TACRED.
Progressively Normalized Self-Attention Network for Video Polyp Segmentation
Venue: arXiv cs.CV 2021
Year: 2021

Existing video polyp segmentation (VPS) models typically employ convolutional neural networks (CNNs) to extract features. However, due to their limited receptive fields, CNNs can not fully exploit the global temporal and spatial information in successive video frames, resulting in false-positive segmentation results. In this paper, we propose the novel PNS-Net (Progressively Normalized Self-attention Network), which can efficiently learn representations from polyp videos with real-time speed (~140fps) on a single RTX 2080 GPU and no post-processing. Our PNS-Net is based solely on a basic normalized self-attention block, equipping with recurrence and CNNs entirely. Experiments on challenging VPS datasets demonstrate that the proposed PNS-Net achieves state-of-the-art performance. We also conduct extensive experiments to study the effectiveness of the channel split, soft-attention, and progressive learning strategy. We find that our PNS-Net works well under different settings, making it a promising solution to the VPS task.
Fast Game Content Adaptation Through Bayesian-based Player Modelling
Venue: arXiv cs.AI 2021
Year: 2021

In games, as well as many user-facing systems, adapting content to users' preferences and experience is an important challenge. This paper explores a novel method to realize this goal in the context of dynamic difficulty adjustment (DDA). Here the aim is to constantly adapt the content of a game to the skill level of the player, keeping them engaged by avoiding states that are either too difficult or too easy. Current systems for DDA rely on expensive data mining, or on hand-crafted rules designed for particular domains, and usually adapts to keep players in the flow, leaving no room for the designer to present content that is purposefully easy or difficult. This paper presents Fast Bayesian Content Adaption (FBCA), a system for DDA that is agnostic to the domain and that can target particular difficulties. We deploy this framework in two different domains: the puzzle game Sudoku, and a simple Roguelike game. By modifying the acquisition function's optimization, we are reliably able to present a content with a bespoke difficulty for players with different skill levels in less than five iterations for Sudoku and fifteen iterations for the simple Roguelike. Our method significantly outperforms simpler DDA heuristics with the added benefit of maintaining a model of the user. These results point towards a promising alternative for content adaption in a variety of different domains.
Study of a Hybrid Photovoltaic-Wind Smart Microgrid using Data Science Approach
Venue: arXiv cs.CY 2021
Year: 2021

In this paper, a smart microgrid implemented in Paracas, Ica, Peru, composed of 6kWp PV + 6kW Wind and that provides electricity to a rural community of 40 families, was studied using a data science approach. Real data of solar irradiance, wind speed, energy demand, and voltage of the battery bank from 2 periods of operation were studied to find patterns, seasonality, and existing correlations between the analyzed data. Among the main results are the periodicity of renewable resources and demand, the weekly behavior of electricity demand and how it has progressively increased from an average of 0.7kW in 2019 to 1.2kW in 2021, and how power outages are repeated at certain hours in the morning when resources are low or there is a failure in the battery bank. These analyzed data will be used to improve sizing techniques and provide recommendations for energy management to optimize the performance of smart microgrids.
ACAE-REMIND for Online Continual Learning with Compressed Feature Replay
Venue: arXiv cs.CV 2021
Year: 2021

Online continual learning aims to learn from a non-IID stream of data from a number of different tasks, where the learner is only allowed to consider data once. Methods are typically allowed to use a limited buffer to store some of the images in the stream. Recently, it was found that feature replay, where an intermediate layer representation of the image is stored (or generated) leads to superior results than image replay, while requiring less memory. Quantized exemplars can further reduce the memory usage. However, a drawback of these methods is that they use a fixed (or very intransigent) backbone network. This significantly limits the learning of representations that can discriminate between all tasks. To address this problem, we propose an auxiliary classifier auto-encoder (ACAE) module for feature replay at intermediate layers with high compression rates. The reduced memory footprint per image allows us to save more exemplars for replay. In our experiments, we conduct task-agnostic evaluation under online continual learning setting and get state-of-the-art performance on ImageNet-Subset, CIFAR100 and CIFAR10 dataset.
On the Robustness of Domain Constraints
Venue: arXiv cs.CR 2021
Year: 2021

Machine learning is vulnerable to adversarial examples-inputs designed to cause models to perform poorly. However, it is unclear if adversarial examples represent realistic inputs in the modeled domains. Diverse domains such as networks and phishing have domain constraints-complex relationships between features that an adversary must satisfy for an attack to be realized (in addition to any adversary-specific goals). In this paper, we explore how domain constraints limit adversarial capabilities and how adversaries can adapt their strategies to create realistic (constraint-compliant) examples. In this, we develop techniques to learn domain constraints from data, and show how the learned constraints can be integrated into the adversarial crafting process. We evaluate the efficacy of our approach in network intrusion and phishing datasets and find: (1) up to 82% of adversarial examples produced by state-of-the-art crafting algorithms violate domain constraints, (2) domain constraints are robust to adversarial examples; enforcing constraints yields an increase in model accuracy by up to 34%. We observe not only that adversaries must alter inputs to satisfy domain constraints, but that these constraints make the generation of valid adversarial examples far more challenging.
LEA: A Learned Encoding Advisor for Column Stores
Venue: arXiv cs.DB 2021
Year: 2021

Data warehouses organize data in a columnar format to enable faster scans and better compression. Modern systems offer a variety of column encodings that can reduce storage footprint and improve query performance. Selecting a good encoding scheme for a particular column is an optimization problem that depends on the data, the query workload, and the underlying hardware. We introduce Learned Encoding Advisor (LEA), a learned approach to column encoding selection. LEA is trained on synthetic datasets with various distributions on the target system. Once trained, LEA uses sample data and statistics (such as cardinality) from the user's database to predict the optimal column encodings. LEA can optimize for encoded size, query performance, or a combination of the two. Compared to the heuristic-based encoding advisor of a commercial column store on TPC-H, LEA achieves 19% lower query latency while using 26% less space.
Effective Attention Sheds Light On Interpretability
Venue: arXiv cs.CL 2021
Year: 2021

An attention matrix of a transformer self-attention sublayer can provably be decomposed into two components and only one of them (effective attention) contributes to the model output. This leads us to ask whether visualizing effective attention gives different conclusions than interpretation of standard attention. Using a subset of the GLUE tasks and BERT, we carry out an analysis to compare the two attention matrices, and show that their interpretations differ. Effective attention is less associated with the features related to the language modeling pretraining such as the separator token, and it has more potential to illustrate linguistic features captured by the model for solving the end-task. Given the found differences, we recommend using effective attention for studying a transformer's behavior since it is more pertinent to the model output by design.
LNGate: Powering IoT with Next Generation Lightning Micro-payments using Threshold Cryptography
Venue: arXiv cs.CR 2021
Year: 2021

Bitcoin has emerged as a revolutionary payment system with its decentralized ledger concept however it has significant problems such as high transaction fees and long confirmation times. Lightning Network (LN), which was introduced much later, solves most of these problems with an innovative concept called off-chain payments. With this advancement, Bitcoin has become an attractive venue to perform micro-payments which can also be adopted in many IoT applications (e.g. toll payments). Nevertheless, it is not feasible to host LN and Bitcoin on IoT devices due to the storage, memory, and processing requirements. Therefore, in this paper, we propose an efficient and secure protocol that enables an IoT device to use LN through an untrusted gateway node. The gateway hosts LN and Bitcoin nodes and can open & close LN channels, send LN payments on behalf of the IoT device. This delegation approach is powered by a (2,2)-threshold scheme that requires the IoT device and the LN gateway to jointly perform all LN operations which in turn secures both parties' funds. Specifically, we propose to thresholdize LN's Bitcoin public and private keys as well as its commitment points. With these and several other protocol level changes, IoT device is protected against revoked state broadcast, collusion, and ransom attacks. We implemented the proposed protocol by changing LN's source code and thoroughly evaluated its performance using a Raspberry Pi. Our evaluation results show that computational and communication delays associated with the protocol are negligible. To the best of our knowledge, this is the first work that implemented threshold cryptography in LN.
Multiple Meta-model Quantifying for Medical Visual Question Answering
Venue: arXiv cs.CV 2021
Year: 2021

Transfer learning is an important step to extract meaningful features and overcome the data limitation in the medical Visual Question Answering (VQA) task. However, most of the existing medical VQA methods rely on external data for transfer learning, while the meta-data within the dataset is not fully utilized. In this paper, we present a new multiple meta-model quantifying method that effectively learns meta-annotation and leverages meaningful features to the medical VQA task. Our proposed method is designed to increase meta-data by auto-annotation, deal with noisy labels, and output meta-models which provide robust features for medical VQA tasks. Extensively experimental results on two public medical VQA datasets show that our approach achieves superior accuracy in comparison with other state-of-the-art methods, while does not require external data to train meta-models.
Assessing the Learning Behavioral Intention of Commuters in Mobility Practices
Venue: arXiv cs.HC 2021
Year: 2021

Learning behavior mechanism is widely anticipated in managed settings through the formal syllabus. However, heading for learning stimulus whilst daily mobility practices through urban transit is the novel feature in learning sciences. Theory of planned behavior (TPB), technology acceptance model (TAM), and service quality of transit are conceptualized to assess the learning behavioral intention (LBI) of commuters in Greater Kuala Lumpur. An online survey was conducted to understand the LBI of 117 travelers who use the technology to engage in the informal learning process during daily commuting. The results explored that all the model variables i.e., perceived ease of use, perceived usefulness, service quality, and subjective norms are significant predictors of LBI. The perceived usefulness of learning during traveling and transit service quality has a vibrant impact on LBI. The research will support the informal learning mechanism from commuters point of view. The study is a novel contribution to transport and learning literature that will open the new prospect of research in urban mobility and its connotation with personal learning and development.
Active Visual SLAM with Independently Rotating Camera
Venue: arXiv cs.RO 2021
Year: 2021

In active Visual-SLAM (V-SLAM), a robot relies on the information retrieved by its cameras to control its own movements for autonomous mapping of the environment. Cameras are usually statically linked to the robot's body, limiting the extra degrees of freedom for visual information acquisition. In this work, we overcome the aforementioned problem by introducing and leveraging an independently rotating camera on the robot base. This enables us to continuously control the heading of the camera, obtaining the desired optimal orientation for active V-SLAM, without rotating the robot itself. However, this additional degree of freedom introduces additional estimation uncertainties, which need to be accounted for. We do this by extending our robot's state estimate to include the camera state and jointly estimate the uncertainties. We develop our method based on a state-of-the-art active V-SLAM approach for omnidirectional robots and evaluate it through rigorous simulation and real robot experiments. We obtain more accurate maps, with lower energy consumption, while maintaining the benefits of the active approach with respect to the baseline. We also demonstrate how our method easily generalizes to other non-omnidirectional robotic platforms, which was a limitation of the previous approach. Code and implementation details are provided as open-source.
Efficient Transfer Learning via Joint Adaptation of Network Architecture and Weight
Venue: arXiv cs.CV 2021
Year: 2021

Transfer learning can boost the performance on the targettask by leveraging the knowledge of the source domain. Recent worksin neural architecture search (NAS), especially one-shot NAS, can aidtransfer learning by establishing sufficient network search space. How-ever, existing NAS methods tend to approximate huge search spaces byexplicitly building giant super-networks with multiple sub-paths, anddiscard super-network weights after a child structure is found. Both thecharacteristics of existing approaches causes repetitive network trainingon source tasks in transfer learning. To remedy the above issues, we re-duce the super-network size by randomly dropping connection betweennetwork blocks while embedding a larger search space. Moreover, wereuse super-network weights to avoid redundant training by proposinga novel framework consisting of two modules, the neural architecturesearch module for architecture transfer and the neural weight searchmodule for weight transfer. These two modules conduct search on thetarget task based on a reduced super-networks, so we only need to trainonce on the source task. We experiment our framework on both MS-COCO and CUB-200 for the object detection and fine-grained imageclassification tasks, and show promising improvements with onlyO(CN)super-network complexity.
Performability of Network Service Chains: Stochastic Modeling and Assessment of Softwarized IP Multimedia Subsystem
Venue: arXiv cs.NI 2021
Year: 2021

Service provisioning mechanisms implemented across 5G infrastructures take broadly into use the network service chain concept. Typically, it is coupled with Network Function Virtualization (NFV) paradigm, and consists in defining a pre-determined path traversed by a set of softwarized network nodes to provide specific services. A well known chain-like framework is the IP Multimedia Subsystem (IMS), a key infrastructure of 5G networks, that we characterize both by a performance and an availability perspective. Precisely, supported by a designed from scratch testbed realized through Clearwater platform, we perform a stochastic assessment of a softwarized IMS (softIMS) architecture where two main stages stand out: i) a performance analysis, where, exploiting the queueing network decomposition method, we formalize an optimization problem of resource allocation by modeling each softIMS node as an M/G/c system; ii) an availability assessment, where, adopting the Stochastic Reward Net methodology, we are able to characterize the behavior of softIMS in terms of failure/repair events, and to derive a set of optimal configurations satisfying a given availability requirement (e.g. five nines) while minimizing deployment costs. Two routines dubbed OptCNT and OptSearchChain have been devised to govern the performance and availability analyses, respectively.
Music Generation using Three-layered LSTM
Venue: arXiv cs.SD 2021
Year: 2021

This paper explores the idea of utilising Long Short-Term Memory neural networks (LSTMNN) for the generation of musical sequences in ABC notation. The proposed approach takes ABC notations from the Nottingham dataset and encodes it to be fed as input for the neural networks. The primary objective is to input the neural networks with an arbitrary note, let the network process and augment a sequence based on the note until a good piece of music is produced. Multiple calibrations have been done to amend the parameters of the network for optimal generation. The output is assessed on the basis of rhythm, harmony, and grammar accuracy.
Essay-BR: a Brazilian Corpus of Essays
Venue: arXiv cs.CL 2021
Year: 2021

Automatic Essay Scoring (AES) is defined as the computer technology that evaluates and scores the written essays, aiming to provide computational models to grade essays either automatically or with minimal human involvement. While there are several AES studies in a variety of languages, few of them are focused on the Portuguese language. The main reason is the lack of a corpus with manually graded essays. In order to bridge this gap, we create a large corpus with several essays written by Brazilian high school students on an online platform. All of the essays are argumentative and were scored across five competencies by experts. Moreover, we conducted an experiment on the created corpus and showed challenges posed by the Portuguese language. Our corpus is publicly available at https://github.com/rafaelanchieta/essay.
Enhancing Security of TAS/MRC Based Mixed RF-UOWC System with Induced Underwater Turbulence Effect
Venue: arXiv cs.IT 2021
Year: 2021

Post commercial deployment of fifth-generation (5G) technologies, the consideration of sixth-generation (6G) networks is drawing remarkable attention from research communities. Researchers suggest that similar to 5G, 6G technology must be human-centric where high secrecy together with high data rate will be the key features. These challenges can be easily overcome utilizing PHY security techniques over high-frequency free-space or underwater optical wireless communication (UOWC) technologies. But in long-distance communication, turbulence components drastically affect the optical signals, leading to the invention of the combination of radio-frequency (RF) links with optical links. This work deals with the secrecy performance analysis of a mixed RF-UOWC system where an eavesdropper tries to intercept RF communications. RF and optical links undergo $η-μ$ and mixture exponential generalized Gamma distributions, respectively. To keep pace with the high data rate of optical technologies, we exploit the antenna selection scheme at the source and maximal ratio combining diversity at the relay and eavesdropper, while the eavesdropper is unaware of the antenna selection scheme. We derive closed-form expressions of average secrecy capacity, secrecy outage probability, and strictly positive secrecy capacity to demonstrate the impacts of the system parameters on the secrecy behavior. Finally, the expressions are corroborated via Monte-Carlo simulations.
Automatic View Selection in Graph Databases
Venue: arXiv cs.DB 2021
Year: 2021

Recently, several works have studied the problem of view selection in graph databases. However, existing methods cannot fully exploit the graph properties of views, e.g., supergraph views and common subgraph views, which leads to a low view utility and duplicate view content. To address the problem, we propose an end-to-end graph view selection tool, G-View, which can judiciously generate a view set from a query workload by exploring the graph properties of candidate views and considering their efficacy. Specifically, given a graph query set and a space budget, G-View translates each query to a candidate view pattern and checks the query containment via a filtering-and-verification framework. G-View then selects the views using a graph gene algorithm (GGA), which relies on a three-phase framework that explores graph view transformations to reduce the view space and optimize the view benefit. Finally, G-View generates the extended graph views that persist all the edge-induced subgraphs to answer the subgraph and supergraph queries simultaneously. Extensive experiments on real-life and synthetic datasets demonstrated G-View achieved averagely 21x and 2x query performance speedup over two view-based methods while having 2x and 5x smaller space overhead, respectively. Moreover, the proposed selection algorithm, GGA, outperformed other selection methods in both effectiveness and efficiency.
A Privacy-Preserving Approach to Extraction of Personal Information through Automatic Annotation and Federated Learning
Venue: arXiv cs.CL 2021
Year: 2021

We curated WikiPII, an automatically labeled dataset composed of Wikipedia biography pages, annotated for personal information extraction. Although automatic annotation can lead to a high degree of label noise, it is an inexpensive process and can generate large volumes of annotated documents. We trained a BERT-based NER model with WikiPII and showed that with an adequately large training dataset, the model can significantly decrease the cost of manual information extraction, despite the high level of label noise. In a similar approach, organizations can leverage text mining techniques to create customized annotated datasets from their historical data without sharing the raw data for human annotation. Also, we explore collaborative training of NER models through federated learning when the annotation is noisy. Our results suggest that depending on the level of trust to the ML operator and the volume of the available data, distributed training can be an effective way of training a personal information identifier in a privacy-preserved manner. Research material is available at https://github.com/ratmcu/wikipiifed.