OpenRaiser/Intern-Atlas huggingface dataset:
Slower is Better: Revisiting the Forgetting Mechanism in LSTM for Slower Information Decay
Venue: arXiv cs.LG 2021
Year: 2021

Sequential information contains short- to long-range dependencies; however, learning long-timescale information has been a challenge for recurrent neural networks. Despite improvements in long short-term memory networks (LSTMs), the forgetting mechanism results in the exponential decay of information, limiting their capacity to capture long-timescale information. Here, we propose a power law forget gate, which instead learns to forget information along a slower power law decay function. Specifically, the new gate learns to control the power law decay factor, p, allowing the network to adjust the information decay rate according to task demands. Our experiments show that an LSTM with power law forget gates (pLSTM) can effectively capture long-range dependencies beyond hundreds of elements on image classification, language modeling, and categorization tasks, improving performance over the vanilla LSTM. We also inspected the revised forget gate by varying the initialization of p, setting p to a fixed value, and ablating cells in the pLSTM network. The results show that the information decay can be controlled by the learnable decay factor p, which allows pLSTM to achieve its superior performance. Altogether, we found that LSTM with the proposed forget gate can learn long-term dependencies, outperforming other recurrent networks in multiple domains; such gating mechanism can be integrated into other architectures for improving the learning of long timescale information in recurrent neural networks.
Neural Trajectory Fields for Dynamic Novel View Synthesis
Venue: arXiv cs.CV 2021
Year: 2021

Recent approaches to render photorealistic views from a limited set of photographs have pushed the boundaries of our interactions with pictures of static scenes. The ability to recreate moments, that is, time-varying sequences, is perhaps an even more interesting scenario, but it remains largely unsolved. We introduce DCT-NeRF, a coordinatebased neural representation for dynamic scenes. DCTNeRF learns smooth and stable trajectories over the input sequence for each point in space. This allows us to enforce consistency between any two frames in the sequence, which results in high quality reconstruction, particularly in dynamic regions.
Model Predictive Control with Environment Adaptation for Legged Locomotion
Venue: arXiv cs.RO 2021
Year: 2021

Re-planning in legged locomotion is crucial to track the desired user velocity while adapting to the terrain and rejecting external disturbances. In this work, we propose and test in experiments a real-time Nonlinear Model Predictive Control (NMPC) tailored to a legged robot for achieving dynamic locomotion on a variety of terrains. We introduce a mobility-based criterion to define an NMPC cost that enhances the locomotion of quadruped robots while maximizing leg mobility and improves adaptation to the terrain features. Our NMPC is based on the real-time iteration scheme that allows us to re-plan online at $25\,\mathrm{Hz}$ with a prediction horizon of $2$ seconds. We use the single rigid body dynamic model defined in the center of mass frame in order to increase the computational efficiency. In simulations, the NMPC is tested to traverse a set of pallets of different sizes, to walk into a V-shaped chimney,and to locomote over rough terrain. In real experiments, we demonstrate the effectiveness of our NMPC with the mobility feature that allowed IIT's $87\, \mathrm{kg}$ quadruped robot HyQ to achieve an omni-directional walk on flat terrain, to traverse a static pallet, and to adapt to a repositioned pallet during a walk.
Designing Multimodal Datasets for NLP Challenges
Venue: arXiv cs.CL 2021
Year: 2021

In this paper, we argue that the design and development of multimodal datasets for natural language processing (NLP) challenges should be enhanced in two significant respects: to more broadly represent commonsense semantic inferences; and to better reflect the dynamics of actions and events, through a substantive alignment of textual and visual information. We identify challenges and tasks that are reflective of linguistic and cognitive competencies that humans have when speaking and reasoning, rather than merely the performance of systems on isolated tasks. We introduce the distinction between challenge-based tasks and competence-based performance, and describe a diagnostic dataset, Recipe-to-Video Questions (R2VQ), designed for testing competence-based comprehension over a multimodal recipe collection (http://r2vq.org/). The corpus contains detailed annotation supporting such inferencing tasks and facilitating a rich set of question families that we use to evaluate NLP systems.
Communication-Efficient LDPC Code Design for Data Availability Oracle in Side Blockchains
Venue: arXiv cs.IT 2021
Year: 2021

A popular method of improving the throughput of blockchain systems is by running smaller side blockchains that push the hashes of their blocks onto a trusted blockchain. Side blockchains are vulnerable to stalling attacks where a side blockchain node pushes the hash of a block to the trusted blockchain but makes the block unavailable to other side blockchain nodes. Recently, Sheng et al. proposed a data availability oracle based on LDPC codes and a data dispersal protocol as a solution to the above problem. While showing improvements, the codes and dispersal protocol were designed disjointly which may not be optimal in terms of the communication cost associated with the oracle. In this paper, we provide a tailored dispersal protocol and specialized LDPC code construction based on the Progressive Edge Growth (PEG) algorithm, called the dispersal-efficient PEG (DE-PEG) algorithm, aimed to reduce the communication cost associated with the new dispersal protocol. Our new code construction reduces the communication cost and, additionally, is less restrictive in terms of system design.
A Hybrid Wired/Wireless Deterministic Network for Smart Grid
Venue: arXiv cs.NI 2021
Year: 2021

With the rapid growth of time-critical applications in smart grid, robotics, autonomous vehicles, and industrial automation, demand for high reliability, low latency and strictly bounded jitter is sharply increasing. High-precision time synchronization communications, such as Time Triggered Ethernet (TTE), have been successfully developed for wired networks. However, the high cost of deploying additional equipment and extra wiring limits the scalability of these networks. Therefore, in this paper, a hybrid wired/wireless high-precision time synchronization network based on a combination of high-speed TTE and 5G Ultra-Reliable and Low-Latency Communications (URLLC) is proposed. The main motivation is to comply with the low latency, low jitter, and high reliability requirements of time critical applications, such as smart grid synchrophasor communications. Therefore, in the proposed hybrid network architecture, a high-speed TTE is considered as the main bus (i.e., backbone network), whereas a Precision Time Protocol (PTP) aided 5G-URLLC-based wireless access is used as a sub-network. The main challenge is to achieve interoperability between the PTP aided URLLC and the TTE, while ensuring high precision timing and synchronization. The simulation results demonstrate the impact of the PTP-aided URLLC in maintaining network reliability, latency, and jitter in full coordination with the TTE-network.
SAFIN: Arbitrary Style Transfer With Self-Attentive Factorized Instance Normalization
Venue: arXiv cs.CV 2021
Year: 2021

Artistic style transfer aims to transfer the style characteristics of one image onto another image while retaining its content. Existing approaches commonly leverage various normalization techniques, although these face limitations in adequately transferring diverse textures to different spatial locations. Self-Attention-based approaches have tackled this issue with partial success but suffer from unwanted artifacts. Motivated by these observations, this paper aims to combine the best of both worlds: self-attention and normalization. That yields a new plug-and-play module that we name Self-Attentive Factorized Instance Normalization (SAFIN). SAFIN is essentially a spatially adaptive normalization module whose parameters are inferred through attention on the content and style image. We demonstrate that plugging SAFIN into the base network of another state-of-the-art method results in enhanced stylization. We also develop a novel base network composed of Wavelet Transform for multi-scale style transfer, which when combined with SAFIN, produces visually appealing results with lesser unwanted textures.
Adaptive Warm-Start MCTS in AlphaZero-like Deep Reinforcement Learning
Venue: arXiv cs.AI 2021
Year: 2021

AlphaZero has achieved impressive performance in deep reinforcement learning by utilizing an architecture that combines search and training of a neural network in self-play. Many researchers are looking for ways to reproduce and improve results for other games/tasks. However, the architecture is designed to learn from scratch, tabula rasa, accepting a cold-start problem in self-play. Recently, a warm-start enhancement method for Monte Carlo Tree Search was proposed to improve the self-play starting phase. It employs a fixed parameter $I^\prime$ to control the warm-start length. Improved performance was reported in small board games. In this paper we present results with an adaptive switch method. Experiments show that our approach works better than the fixed $I^\prime$, especially for "deep," tactical, games (Othello and Connect Four). We conjecture that the adaptive value for $I^\prime$ is also influenced by the size of the game, and that on average $I^\prime$ will increase with game size. We conclude that AlphaZero-like deep reinforcement learning benefits from adaptive rollout based warm-start, as Rapid Action Value Estimate did for rollout-based reinforcement learning 15 years ago.
Adaptive Test-Time Augmentation for Low-Power CPU
Venue: arXiv cs.CV 2021
Year: 2021

Convolutional Neural Networks (ConvNets) are trained offline using the few available data and may therefore suffer from substantial accuracy loss when ported on the field, where unseen input patterns received under unpredictable external conditions can mislead the model. Test-Time Augmentation (TTA) techniques aim to alleviate such common side effect at inference-time, first running multiple feed-forward passes on a set of altered versions of the same input sample, and then computing the main outcome through a consensus of the aggregated predictions. Unfortunately, the implementation of TTA on embedded CPUs introduces latency penalties that limit its adoption on edge applications. To tackle this issue, we propose AdapTTA, an adaptive implementation of TTA that controls the number of feed-forward passes dynamically, depending on the complexity of the input. Experimental results on state-of-the-art ConvNets for image classification deployed on a commercial ARM Cortex-A CPU demonstrate AdapTTA reaches remarkable latency savings, from 1.49X to 2.21X, and hence a higher frame rate compared to static TTA, still preserving the same accuracy gain.
VSR: A Unified Framework for Document Layout Analysis combining Vision, Semantics and Relations
Venue: arXiv cs.CV 2021
Year: 2021

Document layout analysis is crucial for understanding document structures. On this task, vision and semantics of documents, and relations between layout components contribute to the understanding process. Though many works have been proposed to exploit the above information, they show unsatisfactory results. NLP-based methods model layout analysis as a sequence labeling task and show insufficient capabilities in layout modeling. CV-based methods model layout analysis as a detection or segmentation task, but bear limitations of inefficient modality fusion and lack of relation modeling between layout components. To address the above limitations, we propose a unified framework VSR for document layout analysis, combining vision, semantics and relations. VSR supports both NLP-based and CV-based methods. Specifically, we first introduce vision through document image and semantics through text embedding maps. Then, modality-specific visual and semantic features are extracted using a two-stream network, which are adaptively fused to make full use of complementary information. Finally, given component candidates, a relation module based on graph neural network is incorported to model relations between components and output final results. On three popular benchmarks, VSR outperforms previous models by large margins. Code will be released soon.
LANA: Towards Personalized Deep Knowledge Tracing Through Distinguishable Interactive Sequences
Venue: arXiv cs.CY 2021
Year: 2021

In educational applications, Knowledge Tracing (KT), the problem of accurately predicting students' responses to future questions by summarizing their knowledge states, has been widely studied for decades as it is considered a fundamental task towards adaptive online learning. Among all the proposed KT methods, Deep Knowledge Tracing (DKT) and its variants are by far the most effective ones due to the high flexibility of the neural network. However, DKT often ignores the inherent differences between students (e.g. memory skills, reasoning skills, ...), averaging the performances of all students, leading to the lack of personalization, and therefore was considered insufficient for adaptive learning. To alleviate this problem, in this paper, we proposed Leveled Attentive KNowledge TrAcing (LANA), which firstly uses a novel student-related features extractor (SRFE) to distill students' unique inherent properties from their respective interactive sequences. Secondly, the pivot module was utilized to dynamically reconstruct the decoder of the neural network on attention of the extracted features, successfully distinguishing the performance between students over time. Moreover, inspired by Item Response Theory (IRT), the interpretable Rasch model was used to cluster students by their ability levels, and thereby utilizing leveled learning to assign different encoders to different groups of students. With pivot module reconstructed the decoder for individual students and leveled learning specialized encoders for groups, personalized DKT was achieved. Extensive experiments conducted on two real-world large-scale datasets demonstrated that our proposed LANA improves the AUC score by at least 1.00% (i.e. EdNet 1.46% and RAIEd2020 1.00%), substantially surpassing the other State-Of-The-Art KT methods.
A Pragmatic Approach to Regulating Artificial Intelligence: A Technology Regulator's Perspective
Venue: arXiv cs.CY 2021
Year: 2021

Artificial Intelligence (AI) and the regulation thereof is a topic that is increasingly being discussed within various fora. Various proposals have been made in literature for defining regulatory bodies and/or related regulation. In this paper, we present a pragmatic approach for providing a technology assurance regulatory framework. To the best knowledge of the authors this work presents the first national AI technology assurance legal and regulatory framework that has been implemented by a national authority empowered through law to do so. In aim of both providing assurances where required and not stifling innovation yet supporting it, herein it is proposed that such regulation should not be mandated for all AI-based systems and that rather it should primarily provide a voluntary framework and only be mandated in sectors and activities where required and as deemed necessary by other authorities for regulated and critical areas.
Intelligence and Unambitiousness Using Algorithmic Information Theory
Venue: arXiv cs.AI 2021
Year: 2021

Algorithmic Information Theory has inspired intractable constructions of general intelligence (AGI), and undiscovered tractable approximations are likely feasible. Reinforcement Learning (RL), the dominant paradigm by which an agent might learn to solve arbitrary solvable problems, gives an agent a dangerous incentive: to gain arbitrary "power" in order to intervene in the provision of their own reward. We review the arguments that generally intelligent algorithmic-information-theoretic reinforcement learners such as Hutter's (2005) AIXI would seek arbitrary power, including over us. Then, using an information-theoretic exploration schedule, and a setup inspired by causal influence theory, we present a variant of AIXI which learns to not seek arbitrary power; we call it "unambitious". We show that our agent learns to accrue reward at least as well as a human mentor, while relying on that mentor with diminishing probability. And given a formal assumption that we probe empirically, we show that eventually, the agent's world-model incorporates the following true fact: intervening in the "outside world" will have no effect on reward acquisition; hence, it has no incentive to shape the outside world.
A Scalable Algorithm for Anomaly Detection via Learning-Based Controlled Sensing
Venue: arXiv cs.LG 2021
Year: 2021

We address the problem of sequentially selecting and observing processes from a given set to find the anomalies among them. The decision-maker observes one process at a time and obtains a noisy binary indicator of whether or not the corresponding process is anomalous. In this setting, we develop an anomaly detection algorithm that chooses the process to be observed at a given time instant, decides when to stop taking observations, and makes a decision regarding the anomalous processes. The objective of the detection algorithm is to arrive at a decision with an accuracy exceeding a desired value while minimizing the delay in decision making. Our algorithm relies on a Markov decision process defined using the marginal probability of each process being normal or anomalous, conditioned on the observations. We implement the detection algorithm using the deep actor-critic reinforcement learning framework. Unlike prior work on this topic that has exponential complexity in the number of processes, our algorithm has computational and memory requirements that are both polynomial in the number of processes. We demonstrate the efficacy of our algorithm using numerical experiments by comparing it with the state-of-the-art methods.
On the Monitorability of Session Types, in Theory and Practice (Extended Version)
Venue: arXiv cs.PL 2021
Year: 2021

In concurrent and distributed systems, software components are expected to communicate according to predetermined protocols and APIs - and if a component does not observe them, the system's reliability is compromised. Furthermore, isolating and fixing protocol/API errors can be very difficult. Many methods have been proposed to check the correctness of communicating systems, ranging from compile-time to run-time verification; among such methods, session types have been applied for both static type-checking, and run-time monitoring. This work takes a fresh look at the run-time verification of communicating systems using session types, in theory and in practice. On the theoretical side, we develop a novel formal model of session-monitored processes; with it, we formulate and prove new results on the monitorability of session types, connecting their run-time and static verification - in terms of soundness (i.e., whether monitors only flag ill-typed processes) and completeness (i.e., whether all ill-typed processes can be flagged by a monitor). On the practical side, we show that our monitoring theory is indeed realisable: building upon our formal model, we develop a Scala toolkit for the automatic generation of session monitors. Our executable monitors can be used to instrument black-box processes written in any programming language; we assess the viability of our approach with a series of benchmarks.
A one-armed CNN for exoplanet detection from light curves
Venue: arXiv cs.CV 2021
Year: 2021

We propose Genesis, a one-armed simplified Convolutional Neural Network (CNN)for exoplanet detection, and compare it to the more complex, two-armed CNN called Astronet. Furthermore, we examine how Monte Carlo cross-validation affects the estimation of the exoplanet detection performance. Finally, we increase the input resolution twofold to assess its effect on performance. The experiments reveal that (i)the reduced complexity of Genesis, i.e., a more than 95% reduction in the number of free parameters, incurs a small performance cost of about 0.5% compared to Astronet, (ii) Monte Carlo cross-validation provides a more realistic performance estimate that is almost 0.7% below the original estimate, and (iii) the twofold increase in input resolution decreases the average performance by about 0.5%. We conclude by arguing that further exploration of shallower CNN architectures may be beneficial in order to improve the generalizability of CNN-based exoplanet detection across surveys.
Comprehensive Study of Security and Privacy of Emerging Non-Volatile Memories
Venue: arXiv cs.CR 2021
Year: 2021

At the end of Silicon roadmap, keeping the leakage power in tolerable limit and bridging the bandwidth gap between processor and memory have become some of the biggest challenges. Several promising Non-Volatile Memories (NVMs) such as, Spin-Transfer Torque RAM (STTRAM), Magnetic RAM (MRAM), Phase Change Memory (PCM), Resistive RAM (RRAM) and Ferroelectric RAM (FeRAM) are being investigated to address the above issues since they offer high density and consumes zero leakage power. On one hand, the desirable properties of emerging NVMs make them suitable candidates for several applications including replacement of conventional memories. On the other hand, their unique characteristics such as, high and asymmetric read/write current and persistence bring new threats to data security and privacy. Some of these memories are already deployed in full systems and as discrete chips and are believed to become ubiquitous in future computing devices. Therefore, it is of utmost important to investigate their security and privacy issues. Note that these NVMs can be considered for cache, main memory or storage application. They are also suitable to implement in-memory computation which increases system throughput and eliminates Von-Neumann Bottleneck. Compute-capable NVMs impose new security and privacy challenges that are fundamentally different than their storage counterpart. This work identifies NVM vulnerabilities, attack vectors originating from device level all the way to circuits and systems considering both storage and compute applications. We also summarize the circuit/system level countermeasures to make the NVMs robust against security and privacy issues.
SyntheticFur dataset for neural rendering
Venue: arXiv cs.LG 2021
Year: 2021

We introduce a new dataset called SyntheticFur built specifically for machine learning training. The dataset consists of ray traced synthetic fur renders with corresponding rasterized input buffers and simulation data files. We procedurally generated approximately 140,000 images and 15 simulations with Houdini. The images consist of fur groomed with different skin primitives and move with various motions in a predefined set of lighting environments. We also demonstrated how the dataset could be used with neural rendering to significantly improve fur graphics using inexpensive input buffers by training a conditional generative adversarial network with perceptual loss. We hope the availability of such high fidelity fur renders will encourage new advances with neural rendering for a variety of applications.