OpenRaiser/Intern-Atlas huggingface dataset:
One Pixel, One Interaction, One Game: An Experiment in Minimalist Game Design
Venue: arXiv cs.HC 2022
Year: 2022

Minimalist game design was introduced a decade ago as a general design principle with a list of key properties for minimalist games: basic controls, simple but aesthetically pleasing visuals, interesting player choices with vast possibility spaces, and sounds that resonate with the design. In this paper, we present an experiment we did to explore minimalism in games using a bottom-up approach. We invited a small group of professional game designers and a larger group of game design students to participate in a seminal experiment on minimalism in game design. We started from the most basic game elements: one pixel and one key which provide the least amount of information we can display and reasonably the most elementary action players can perform. We designed a game that starts with a black pixel and asks players to press a key when the pixel turns white. This minimal game, almost a Skinner box, captures the essential elements of the mechanics of games like "The Impossible Game," which asks players to do nothing more than press a key at the right moment. We presented this game concept to the professional game designers and challenged them to create other games with the least amount of player interaction and displayed information. We did not specify any constraints (as usually done in other contexts) and left them free to express their view of minimalistic game design. We repeated the experiment with 100+ students attending a master-level course on video game design and development at our institution. We then analyzed the creations of the two groups, discussing the idea of minimalistic design that emerges from the submitted game concepts.
Learning Sequential Descriptors for Sequence-based Visual Place Recognition
Venue: arXiv cs.CV 2022
Year: 2022

In robotics, Visual Place Recognition is a continuous process that receives as input a video stream to produce a hypothesis of the robot's current position within a map of known places. This task requires robust, scalable, and efficient techniques for real applications. This work proposes a detailed taxonomy of techniques using sequential descriptors, highlighting different mechanism to fuse the information from the individual images. This categorization is supported by a complete benchmark of experimental results that provides evidence on the strengths and weaknesses of these different architectural choices. In comparison to existing sequential descriptors methods, we further investigate the viability of Transformers instead of CNN backbones, and we propose a new ad-hoc sequence-level aggregator called SeqVLAD, which outperforms prior state of the art on different datasets. The code is available at https://github.com/vandal-vpr/vg-transformers.
Constrained Training of Neural Networks via Theorem Proving
Venue: arXiv cs.AI 2022
Year: 2022

We introduce a theorem proving approach to the specification and generation of temporal logical constraints for training neural networks. We formalise a deep embedding of linear temporal logic over finite traces (LTL$_f$) and an associated evaluation function characterising its semantics within the higher-order logic of the Isabelle theorem prover. We then proceed to formalise a loss function $\mathcal{L}$ that we formally prove to be sound, and differentiable to a function $d\mathcal{L}$. We subsequently use Isabelle's automatic code generation mechanism to produce OCaml versions of LTL$_f$, $\mathcal{L}$ and $d\mathcal{L}$ that we integrate with PyTorch via OCaml bindings for Python. We show that, when used for training in an existing deep learning framework for dynamic movement, our approach produces expected results for common movement specification patterns such as obstacle avoidance and patrolling. The distinctive benefit of our approach is the fully rigorous method for constrained training, eliminating many of the risks inherent to ad-hoc implementations of logical aspects directly in an "unsafe" programming language such as Python.
Towards Semantic Communication Protocols: A Probabilistic Logic Perspective
Venue: arXiv cs.IT 2022
Year: 2022

Classical medium access control (MAC) protocols are interpretable, yet their task-agnostic control signaling messages (CMs) are ill-suited for emerging mission-critical applications. By contrast, neural network (NN) based protocol models (NPMs) learn to generate task-specific CMs, but their rationale and impact lack interpretability. To fill this void, in this article we propose, for the first time, a semantic protocol model (SPM) constructed by transforming an NPM into an interpretable symbolic graph written in the probabilistic logic programming language (ProbLog). This transformation is viable by extracting and merging common CMs and their connections while treating the NPM as a CM generator. By extensive simulations, we corroborate that the SPM tightly approximates its original NPM while occupying only 0.02% memory. By leveraging its interpretability and memory-efficiency, we demonstrate several SPM-enabled applications such as SPM reconfiguration for collision-avoidance, as well as comparing different SPMs via semantic entropy calculation and storing multiple SPMs to cope with non-stationary environments.
High Performance Simulation for Scalable Multi-Agent Reinforcement Learning
Venue: arXiv cs.MA 2022
Year: 2022

Multi-agent reinforcement learning experiments and open-source training environments are typically limited in scale, supporting tens or sometimes up to hundreds of interacting agents. In this paper we demonstrate the use of Vogue, a high performance agent based model (ABM) framework. Vogue serves as a multi-agent training environment, supporting thousands to tens of thousands of interacting agents while maintaining high training throughput by running both the environment and reinforcement learning (RL) agents on the GPU. High performance multi-agent environments at this scale have the potential to enable the learning of robust and flexible policies for use in ABMs and simulations of complex systems. We demonstrate training performance with two newly developed, large scale multi-agent training environments. Moreover, we show that these environments can train shared RL policies on time-scales of minutes and hours.
Optimal Pattern synthesis of linear antenna array using Ant Hill Colonization Optimization algorithm(AHCOA)
Venue: arXiv cs.NE 2022
Year: 2022

The aim of this paper is to introduce AHCOA to the electromagnetic and antenna community. AHCOA is a new nature inspired meta heuristic algorithm inspired by how there is a hierarchy and departments in the ant hill colonization. It has high probabilistic potential in solving not only unconstrained but also constrained optimization problems. In this paper the AHCOA is applied to linear antenna array for better pattern synthesis in the following ways : By uniform excitation considering equal spacing of the antenna elements with respect to the uniform array. AHCOA is used in obtaining an array pattern to achieve minimum side lobe levels. The results are compared to other state of the art nature based algorithms such as ant lion optimizer, which show a considerable improvement in AHCOA.
A Framework Based on Generational and Environmental Response Strategies for Dynamic Multi-objective Optimization
Venue: arXiv cs.NE 2022
Year: 2022

Due to the dynamics and uncertainty of the dynamic multi-objective optimization problems (DMOPs), it is difficult for algorithms to find a satisfactory solution set before the next environmental change, especially for some complex environments. One reason may be that the information in the environmental static stage can not be used well in the traditional framework. In this paper, a novel framework based on generational and environmental response strategies (FGERS) is proposed, in which response strategies are run both in the environmental change stage and the environmental static stage to obtain population evolution information of those both stages. Unlike in the traditional framework, response strategies are only run in the environmental change stage. For simplicity, the feed-forward center point strategy was chosen to be the response strategy in the novel dynamic framework (FGERS-CPS). FGERS-CPS is not only to predict change trend of the optimum solution set in the environmental change stage, but to predict the evolution trend of the population after several generations in the environmental static stage. Together with the feed-forward center point strategy, a simple memory strategy and adaptive diversity maintenance strategy were used to form the complete FGERS-CPS. On 13 DMOPs with various characteristics, FGERS-CPS was compared with four classical response strategies in the traditional framework. Experimental results show that FGERS-CPS is effective for DMOPs.
Adaptive Self-supervision Algorithms for Physics-informed Neural Networks
Venue: arXiv cs.LG 2022
Year: 2022

Physics-informed neural networks (PINNs) incorporate physical knowledge from the problem domain as a soft constraint on the loss function, but recent work has shown that this can lead to optimization difficulties. Here, we study the impact of the location of the collocation points on the trainability of these models. We find that the vanilla PINN performance can be significantly boosted by adapting the location of the collocation points as training proceeds. Specifically, we propose a novel adaptive collocation scheme which progressively allocates more collocation points (without increasing their number) to areas where the model is making higher errors (based on the gradient of the loss function in the domain). This, coupled with a judicious restarting of the training during any optimization stalls (by simply resampling the collocation points in order to adjust the loss landscape) leads to better estimates for the prediction error. We present results for several problems, including a 2D Poisson and diffusion-advection system with different forcing functions. We find that training vanilla PINNs for these problems can result in up to 70% prediction error in the solution, especially in the regime of low collocation points. In contrast, our adaptive schemes can achieve up to an order of magnitude smaller error, with similar computational complexity as the baseline. Furthermore, we find that the adaptive methods consistently perform on-par or slightly better than vanilla PINN method, even for large collocation point regimes. The code for all the experiments has been open sourced.
An Integrated Framework for DevSecOps Adoption
Venue: arXiv cs.SE 2022
Year: 2022

Introduction of DevOps into the software development life cycle represents a cultural shift in the IT culture, amalgamating development and operations to improve delivery speed in a rapid and maintainable manner. At the same time, security threats and breaches are expected to grow as more enterprises move to new agile frameworks for rapid product delivery. Meanwhile, DevSecOps is a mindset change that revolutionizes software development by embedding security at each step of the software cycle, leading to resilient software. This paper discusses a framework organization can use to embed DevSecOps swiftly and efficiently into the general IT culture.
Out of Distribution Detection via Neural Network Anchoring
Venue: arXiv cs.LG 2022
Year: 2022

Our goal in this paper is to exploit heteroscedastic temperature scaling as a calibration strategy for out of distribution (OOD) detection. Heteroscedasticity here refers to the fact that the optimal temperature parameter for each sample can be different, as opposed to conventional approaches that use the same value for the entire distribution. To enable this, we propose a new training strategy called anchoring that can estimate appropriate temperature values for each sample, leading to state-of-the-art OOD detection performance across several benchmarks. Using NTK theory, we show that this temperature function estimate is closely linked to the epistemic uncertainty of the classifier, which explains its behavior. In contrast to some of the best-performing OOD detection approaches, our method does not require exposure to additional outlier datasets, custom calibration objectives, or model ensembling. Through empirical studies with different OOD detection settings -- far OOD, near OOD, and semantically coherent OOD - we establish a highly effective OOD detection approach. Code to reproduce our results is available at github.com/LLNL/AMP
Cross-Attention Transformer for Video Interpolation
Venue: arXiv cs.CV 2022
Year: 2022

We propose TAIN (Transformers and Attention for video INterpolation), a residual neural network for video interpolation, which aims to interpolate an intermediate frame given two consecutive image frames around it. We first present a novel vision transformer module, named Cross Similarity (CS), to globally aggregate input image features with similar appearance as those of the predicted interpolated frame. These CS features are then used to refine the interpolated prediction. To account for occlusions in the CS features, we propose an Image Attention (IA) module to allow the network to focus on CS features from one frame over those of the other. TAIN outperforms existing methods that do not require flow estimation and performs comparably to flow-based methods while being computationally efficient in terms of inference time on Vimeo90k, UCF101, and SNU-FILM benchmarks.
A Survey of Task-Based Machine Learning Content Extraction Services for VIDINT
Venue: arXiv cs.ET 2022
Year: 2022

This paper provides a comparison of current video content extraction tools with a focus on comparing commercial task-based machine learning services. Video intelligence (VIDINT) data has become a critical intelligence source in the past decade. The need for AI-based analytics and automation tools to extract and structure content from video has quickly become a priority for organizations needing to search, analyze and exploit video at scale. With rapid growth in machine learning technology, the maturity of machine transcription, machine translation, topic tagging, and object recognition tasks are improving at an exponential rate, breaking performance records in speed and accuracy as new applications evolve. Each section of this paper reviews and compares products, software resources and video analytics capabilities based on tasks relevant to extracting information from video with machine learning techniques.
A Study on Self-Supervised Object Detection Pretraining
Venue: arXiv cs.CV 2022
Year: 2022

In this work, we study different approaches to self-supervised pretraining of object detection models. We first design a general framework to learn a spatially consistent dense representation from an image, by randomly sampling and projecting boxes to each augmented view and maximizing the similarity between corresponding box features. We study existing design choices in the literature, such as box generation, feature extraction strategies, and using multiple views inspired by its success on instance-level image representation learning techniques. Our results suggest that the method is robust to different choices of hyperparameters, and using multiple views is not as effective as shown for instance-level image representation learning. We also design two auxiliary tasks to predict boxes in one view from their features in the other view, by (1) predicting boxes from the sampled set by using a contrastive loss, and (2) predicting box coordinates using a transformer, which potentially benefits downstream object detection tasks. We found that these tasks do not lead to better object detection performance when finetuning the pretrained model on labeled data.
Online algorithms for finding distinct substrings with length and multiple prefix and suffix conditions
Venue: arXiv cs.DS 2022
Year: 2022

Let two static sequences of strings $P$ and $S$, representing prefix and suffix conditions respectively, be given as input for preprocessing. For the query, let two positive integers $k_1$ and $k_2$ be given, as well as a string $T$ given in an online manner, such that $T_i$ represents the length-$i$ prefix of $T$ for $1 \leq i \leq |T|$. In this paper we are interested in computing the set $\mathit{ans_i}$ of distinct substrings $w$ of $T_i$ such that $k_1 \leq |w| \leq k_2$, and $w$ contains some $p \in P$ as a prefix and some $s \in S$ as a suffix. More specifically, the counting problem is to output $|\mathit{ans_i}|$, whereas the reporting problem is to output all elements of $\mathit{ans_i}$, for each iteration $i$. Let $σ$ denote the alphabet size, and for a sequence of strings $A$, $\Vert A\Vert=\sum_{u\in A}|u|$. Then, we show that after $O((\Vert P\Vert +\Vert S\Vert)\logσ)$-time preprocessing, the solutions for the counting and reporting problems for each iteration up to $i$ can be output in $O(|T_i| \logσ)$ and $O(|T_i| \logσ+ |\mathit{ans_i}|)$ total time. The preprocessing time can be reduced to $O(\Vert P\Vert +\Vert S\Vert)$ for integer alphabets of size polynomial with regard to $\Vert P\Vert +\Vert S\Vert$. Our algorithms have possible applications to network traffic classification.
Improved Binary Forward Exploration: Learning Rate Scheduling Method for Stochastic Optimization
Venue: arXiv cs.LG 2022
Year: 2022

A new gradient-based optimization approach by automatically scheduling the learning rate has been proposed recently, which is called Binary Forward Exploration (BFE). The Adaptive version of BFE has also been discussed thereafter. In this paper, the improved algorithms based on them will be investigated, in order to optimize the efficiency and robustness of the new methodology. This improved approach provides a new perspective to scheduling the update of learning rate and will be compared with the stochastic gradient descent, aka SGD algorithm with momentum or Nesterov momentum and the most successful adaptive learning rate algorithm e.g. Adam. The goal of this method does not aim to beat others but provide a different viewpoint to optimize the gradient descent process. This approach combines the advantages of the first-order and second-order optimizations in the aspects of speed and efficiency.
SiaTrans: Siamese Transformer Network for RGB-D Salient Object Detection with Depth Image Classification
Venue: arXiv cs.CV 2022
Year: 2022

RGB-D SOD uses depth information to handle challenging scenes and obtain high-quality saliency maps. Existing state-of-the-art RGB-D saliency detection methods overwhelmingly rely on the strategy of directly fusing depth information. Although these methods improve the accuracy of saliency prediction through various cross-modality fusion strategies, misinformation provided by some poor-quality depth images can affect the saliency prediction result. To address this issue, a novel RGB-D salient object detection model (SiaTrans) is proposed in this paper, which allows training on depth image quality classification at the same time as training on SOD. In light of the common information between RGB and depth images on salient objects, SiaTrans uses a Siamese transformer network with shared weight parameters as the encoder and extracts RGB and depth features concatenated on the batch dimension, saving space resources without compromising performance. SiaTrans uses the Class token in the backbone network (T2T-ViT) to classify the quality of depth images without preventing the token sequence from going on with the saliency detection task. Transformer-based cross-modality fusion module (CMF) can effectively fuse RGB and depth information. And in the testing process, CMF can choose to fuse cross-modality information or enhance RGB information according to the quality classification signal of the depth image. The greatest benefit of our designed CMF and decoder is that they maintain the consistency of RGB and RGB-D information decoding: SiaTrans decodes RGB-D or RGB information under the same model parameters according to the classification signal during testing. Comprehensive experiments on nine RGB-D SOD benchmark datasets show that SiaTrans has the best overall performance and the least computation compared with recent state-of-the-art methods.
Complexity of Public Goods Games on Graphs
Venue: arXiv cs.GT 2022
Year: 2022

We study the computational complexity of "public goods games on networks". In this model, each vertex in a graph is an agent that needs to take a binary decision of whether to "produce a good" or not. Each agent's utility depends on the number of its neighbors in the graph that produce the good, as well as on its own action. This dependence can be captured by a "pattern" $T:{\rm I\!N}\rightarrow\{0,1\}$ that describes an agent's best response to every possible number of neighbors that produce the good. Answering a question of [Papadimitriou and Peng, 2021], we prove that for some simple pattern $T$ the problem of determining whether a non-trivial pure Nash equilibrium exists is NP-complete. We extend our result to a wide class of such $T$, but also find a new polynomial time algorithm for some specific simple pattern $T$. We leave open the goal of characterizing the complexity for all patterns.
Scheduling Versus Contention for Massive Random Access in Massive MIMO Systems
Venue: arXiv cs.IT 2022
Year: 2022

Massive machine-type communications protocols have typically been designed under the assumption that coordination between users requires significant communication overhead and is thus impractical. Recent progress in efficient activity detection and collision-free scheduling, however, indicates that the cost of coordination can be much less than the naive scheme for scheduling. This work considers a scenario in which a massive number of devices with sporadic traffic seek to access a massive multiple-input multiple-output (MIMO) base-station (BS) and explores an approach in which device activity detection is followed by a single common feedback broadcast message, which is used both to schedule the active users to different transmission slots and to assign orthogonal pilots to the users for channel estimation. The proposed coordinated communication scheme is compared to two prevalent contention-based schemes: coded pilot access, which is based on the principle of coded slotted ALOHA, and an approximate message passing scheme for joint user activity detection and channel estimation. Numerical results indicate that scheduled massive access provides significant gains in the number of successful transmissions per slot and in sum rate, due to the reduced interference, at only a small cost of feedback.
Attention and Self-Attention in Random Forests
Venue: arXiv cs.LG 2022
Year: 2022

New models of random forests jointly using the attention and self-attention mechanisms are proposed for solving the regression problem. The models can be regarded as extensions of the attention-based random forest whose idea stems from applying a combination of the Nadaraya-Watson kernel regression and the Huber's contamination model to random forests. The self-attention aims to capture dependencies of the tree predictions and to remove noise or anomalous predictions in the random forest. The self-attention module is trained jointly with the attention module for computing weights. It is shown that the training process of attention weights is reduced to solving a single quadratic or linear optimization problem. Three modifications of the general approach are proposed and compared. A specific multi-head self-attention for the random forest is also considered. Heads of the self-attention are obtained by changing its tuning parameters including the kernel parameters and the contamination parameter of models. Numerical experiments with various datasets illustrate the proposed models and show that the supplement of the self-attention improves the model performance for many datasets.