OpenRaiser/Intern-Atlas huggingface dataset:
CIRA Guide to Custom Loss Functions for Neural Networks in Environmental Sciences -- Version 1
Venue: arXiv cs.LG 2021
Year: 2021

Neural networks are increasingly used in environmental science applications. Furthermore, neural network models are trained by minimizing a loss function, and it is crucial to choose the loss function very carefully for environmental science applications, as it determines what exactly is being optimized. Standard loss functions do not cover all the needs of the environmental sciences, which makes it important for scientists to be able to develop their own custom loss functions so that they can implement many of the classic performance measures already developed in environmental science, including measures developed for spatial model verification. However, there are very few resources available that cover the basics of custom loss function development comprehensively, and to the best of our knowledge none that focus on the needs of environmental scientists. This document seeks to fill this gap by providing a guide on how to write custom loss functions targeted toward environmental science applications. Topics include the basics of writing custom loss functions, common pitfalls, functions to use in loss functions, examples such as fractions skill score as loss function, how to incorporate physical constraints, discrete and soft discretization, and concepts such as focal, robust, and adaptive loss. While examples are currently provided in this guide for Python with Keras and the TensorFlow backend, the basic concepts also apply to other environments, such as Python with PyTorch. Similarly, while the sample loss functions provided here are from meteorology, these are just examples of how to create custom loss functions. Other fields in the environmental sciences have very similar needs for custom loss functions, e.g., for evaluating spatial forecasts effectively, and the concepts discussed here can be applied there as well. All code samples are provided in a GitHub repository.
Sensory Modality Mapping for Game Adaptation and Design
Venue: arXiv cs.HC 2021
Year: 2021

In this paper we examine methods for taking game-related information provided in one sensory modality and transforming it to another sensor modality in order to more effectively accommodate sensory-constrained players. We then consider methods for the adaptation and design of games for which gameplay interactions are constrained to a subset of sensory modalities in ways that preserve a common level of novelty-of-experience for players with different sensory capabilities. It is hoped that improved shared experiences can promote interactions among a more diverse spectrum of players.
Adapting the Function Approximation Architecture in Online Reinforcement Learning
Venue: arXiv cs.LG 2021
Year: 2021

The performance of a reinforcement learning (RL) system depends on the computational architecture used to approximate a value function. Deep learning methods provide both optimization techniques and architectures for approximating nonlinear functions from noisy, high-dimensional observations. However, prevailing optimization techniques are not designed for strictly-incremental online updates. Nor are standard architectures designed for observations with an a priori unknown structure: for example, light sensors randomly dispersed in space. This paper proposes an online RL prediction algorithm with an adaptive architecture that efficiently finds useful nonlinear features. The algorithm is evaluated in a spatial domain with high-dimensional, stochastic observations. The algorithm outperforms non-adaptive baseline architectures and approaches the performance of an architecture given side-channel information. These results are a step towards scalable RL algorithms for more general problems, where the observation structure is not available.
Intentional Forgetting
Venue: arXiv cs.CR 2021
Year: 2021

Many damaging cybersecurity attacks are enabled when an attacker can access residual sensitive information (e.g. cryptographic keys, personal identifiers) left behind from earlier computation. Attackers can sometimes use residual information to take control of a system, impersonate a user, or manipulate data. Current approaches to addressing access to residual sensitive information aim to patch individual software or hardware vulnerabilities. While such patching approaches are necessary to mitigate sometimes serious security vulnerabilities in the near term, they cannot address the underlying issue: explicit requirements for adequately eliminating residual information and explicit representations of the erasure capabilities of systems are necessary to ensure that sensitive information is handled as expected. This position paper introduces the concept of intentional forgetting and the capabilities that are needed to achieve it. Intentional forgetting enables software and hardware system designers at every level of abstraction to clearly specify and rigorously reason about the forgetting capabilities required of and provided by a system. We identify related work that may help to illuminate challenges or contribute to solutions and consider conceptual and engineering tradeoffs in implementations of forgetting capabilities. We discuss approaches to modeling intentional forgetting and then modeling the strength of a system's forgetting capability by its resistance to disclosing information to different types of detectors. Research is needed in a variety of domains to advance the theory, specification techniques, system foundations, implementation tools, and methodologies for effective, practical forgetting. We highlight research challenges in several domains and encourage cross-disciplinary collaboration to one day create a robust theory and practice of intentional forgetting.
Degree Tables for Secure Distributed Matrix Multiplication
Venue: arXiv cs.IT 2021
Year: 2021

We consider the problem of secure distributed matrix multiplication (SDMM) in which a user wishes to compute the product of two matrices with the assistance of honest but curious servers. We construct polynomial codes for SDMM by studying a recently introduced combinatorial tool called the degree table. For a fixed partitioning, minimizing the total communication cost of a polynomial code for SDMM is equivalent to minimizing $N$, the number of distinct elements in the corresponding degree table. We propose new constructions of degree tables with a low number of distinct elements. These new constructions lead to a general family of polynomial codes for SDMM, which we call $\mathsf{GASP}_{r}$ (Gap Additive Secure Polynomial codes) parametrized by an integer $r$. $\mathsf{GASP}_{r}$ outperforms all previously known polynomial codes for SDMM under an outer product partitioning. We also present lower bounds on $N$ and prove the optimality or asymptotic optimality of our constructions for certain regimes. Moreover, we formulate the construction of optimal degree tables as an integer linear program and use it to prove the optimality of $\mathsf{GASP}_{r}$ for all the system parameters that we were able to test.
Many Agent Reinforcement Learning Under Partial Observability
Venue: arXiv cs.LG 2021
Year: 2021

Recent renewed interest in multi-agent reinforcement learning (MARL) has generated an impressive array of techniques that leverage deep reinforcement learning, primarily actor-critic architectures, and can be applied to a limited range of settings in terms of observability and communication. However, a continuing limitation of much of this work is the curse of dimensionality when it comes to representations based on joint actions, which grow exponentially with the number of agents. In this paper, we squarely focus on this challenge of scalability. We apply the key insight of action anonymity, which leads to permutation invariance of joint actions, to two recently presented deep MARL algorithms, MADDPG and IA2C, and compare these instantiations to another recent technique that leverages action anonymity, viz., mean-field MARL. We show that our instantiations can learn the optimal behavior in a broader class of agent networks than the mean-field method, using a recently introduced pragmatic domain.
On Effects of Compression with Hyperdimensional Computing in Distributed Randomized Neural Networks
Venue: arXiv cs.LG 2021
Year: 2021

A change of the prevalent supervised learning techniques is foreseeable in the near future: from the complex, computational expensive algorithms to more flexible and elementary training ones. The strong revitalization of randomized algorithms can be framed in this prospect steering. We recently proposed a model for distributed classification based on randomized neural networks and hyperdimensional computing, which takes into account cost of information exchange between agents using compression. The use of compression is important as it addresses the issues related to the communication bottleneck, however, the original approach is rigid in the way the compression is used. Therefore, in this work, we propose a more flexible approach to compression and compare it to conventional compression algorithms, dimensionality reduction, and quantization techniques.
Heuristic Stopping Rules For Technology-Assisted Review
Venue: arXiv cs.IR 2021
Year: 2021

Technology-assisted review (TAR) refers to human-in-the-loop active learning workflows for finding relevant documents in large collections. These workflows often must meet a target for the proportion of relevant documents found (i.e. recall) while also holding down costs. A variety of heuristic stopping rules have been suggested for striking this tradeoff in particular settings, but none have been tested against a range of recall targets and tasks. We propose two new heuristic stopping rules, Quant and QuantCI based on model-based estimation techniques from survey research. We compare them against a range of proposed heuristics and find they are accurate at hitting a range of recall targets while substantially reducing review costs.
Approximation Algorithms for Two-Bar Charts Packing Problem
Venue: arXiv cs.DM 2021
Year: 2021

In the Two-Bar Charts Packing Problem (2-BCPP), it is required to pack the bar charts (BCs) consisting of two bars into the horizontal unit-height strip of minimal length. The bars may move vertically within the strip, but it is forbidden to change the order and separate the chart's bars. Recently, for this new problem, which is a generalization of the Bin Packing Problem (BPP), Strip Packing Problem (SPP), and 2-Dimensional Vector Packing Problem (2-DVPP), several approximation algorithms with guaranteed estimates were proposed. However, after a preliminary analysis of the solutions constructed by approximation algorithms, we discerned that the guaranteed estimates are inaccurate. This fact inspired us to conduct a numerical experiment in which the approximate solutions are compared to each other and with the optimal ones. To construct the optimal solutions or lower bounds for optimum, we use the Boolean Linear Programming (BLP) formulation of 2-BCPP proposed earlier and apply the CPLEX package. We also use a database of instances for BPP with known optimal solutions to construct the instances for the 2-BCPP with known minimal packing length. The results of the simulation make up the main content of this paper.
Graph-based Joint Pandemic Concern and Relation Extraction on Twitter
Venue: arXiv cs.CL 2021
Year: 2021

Public concern detection provides potential guidance to the authorities for crisis management before or during a pandemic outbreak. Detecting people's concerns and attention from online social media platforms has been widely acknowledged as an effective approach to relieve public panic and prevent a social crisis. However, detecting concerns in time from massive information in social media turns out to be a big challenge, especially when sufficient manually labeled data is in the absence of public health emergencies, e.g., COVID-19. In this paper, we propose a novel end-to-end deep learning model to identify people's concerns and the corresponding relations based on Graph Convolutional Network and Bi-directional Long Short Term Memory integrated with Concern Graph. Except for the sequential features from BERT embeddings, the regional features of tweets can be extracted by the Concern Graph module, which not only benefits the concern detection but also enables our model to be high noise-tolerant. Thus, our model can address the issue of insufficient manually labeled data. We conduct extensive experiments to evaluate the proposed model by using both manually labeled tweets and automatically labeled tweets. The experimental results show that our model can outperform the state-of-art models on real-world datasets.
Data Enforced: An Exploratory Impact Analysis of Automated Speed Enforcement in the District of Columbia
Venue: arXiv cs.CY 2021
Year: 2021

In 2015, the District of Columbia framed a Vision Zero mission and action plan, with a target of achieving zero traffic fatalities by 2024. This study examines the impacts of Automated Speed Enforcement (ASE) and its role in achieving the goals of Vision Zero. Independent datasets containing detailed information about traffic crashes, ASE camera locations, and citation records, and driving speeds across the District's streets were collected, combined, and analyzed to identify patterns and trends in crashes, speed limit violations, and speeding behavior before and after the ASE camera installation. The results of this exploratory analysis confirm the safety benefits of ASE systems in Washington, D.C. The study also provides a blueprint for the different means of evaluating the short-term impact of ASE systems using different data sources which can aid practitioners in better evaluating existing systems and support the decision-making process regarding future installations.
Does gamification affect flow experience? A systematic literature review
Venue: arXiv cs.HC 2021
Year: 2021

In recent years, studies in different areas have used gamification to improve users' flow experience. However, due to the high variety of the conducted studies and the lack of secondary studies (e.g., systematic literature reviews) in this field, it is difficult to get the state-of-the-art of this research domain. To address this problem, we conducted a systematic literature review to identify i) which gamification design methods have been used in the studies about gamification and Flow Theory, ii) which gamification elements have been used in these studies, iii) which methods have been used to evaluate the users' flow experience in gamified settings, and iv) how gamification affects users' flow experience. The main results show that there is growing interest to this field, as the number of publications is increasing. The most significant interest is in the area of gamification in education. However, there is no unanimity regarding the preferred method of the study or the effects of gamification on users' experience. Our results highlight the importance of conducting new experimental studies investigating how gamification affects the users' flow experience in different gamified settings, applications and domains.
Multi-Granularity Network with Modal Attention for Dense Affective Understanding
Venue: arXiv cs.CV 2021
Year: 2021

Video affective understanding, which aims to predict the evoked expressions by the video content, is desired for video creation and recommendation. In the recent EEV challenge, a dense affective understanding task is proposed and requires frame-level affective prediction. In this paper, we propose a multi-granularity network with modal attention (MGN-MA), which employs multi-granularity features for better description of the target frame. Specifically, the multi-granularity features could be divided into frame-level, clips-level and video-level features, which corresponds to visual-salient content, semantic-context and video theme information. Then the modal attention fusion module is designed to fuse the multi-granularity features and emphasize more affection-relevant modals. Finally, the fused feature is fed into a Mixtures Of Experts (MOE) classifier to predict the expressions. Further employing model-ensemble post-processing, the proposed method achieves the correlation score of 0.02292 in the EEV challenge.
How COVID-19 Has Changed Crowdfunding: Evidence From GoFundMe
Venue: arXiv cs.CY 2021
Year: 2021

While the long-term effects of COVID-19 are yet to be determined, its immediate impact on crowdfunding is nonetheless significant. This study takes a computational approach to more deeply comprehend this change. Using a unique data set of all the campaigns published over the past two years on GoFundMe, we explore the factors that have led to the successful funding of a crowdfunding project. In particular, we study a corpus of crowdfunded projects, analyzing cover images and other variables commonly present on crowdfunding sites. Furthermore, we construct a classifier and a regression model to assess the significance of features based on XGBoost. In addition, we employ counterfactual analysis to investigate the causality between features and the success of crowdfunding. More importantly, sentiment analysis and the paired sample t-test are performed to examine the differences in crowdfunding campaigns before and after the COVID-19 outbreak that started in March 2020. First, we note that there is significant racial disparity in crowdfunding success. Second, we find that sad emotion expressed through the campaign's description became significant after the COVID-19 outbreak. Considering all these factors, our findings shed light on the impact of COVID-19 on crowdfunding campaigns.
Recurrent Stacking of Layers in Neural Networks: An Application to Neural Machine Translation
Venue: arXiv cs.CL 2021
Year: 2021

In deep neural network modeling, the most common practice is to stack a number of recurrent, convolutional, or feed-forward layers in order to obtain high-quality continuous space representations which in turn improves the quality of the network's prediction. Conventionally, each layer in the stack has its own parameters which leads to a significant increase in the number of model parameters. In this paper, we propose to share parameters across all layers thereby leading to a recurrently stacked neural network model. We report on an extensive case study on neural machine translation (NMT), where we apply our proposed method to an encoder-decoder based neural network model, i.e., the Transformer model, and experiment with three Japanese--English translation datasets. We empirically demonstrate that the translation quality of a model that recurrently stacks a single layer 6 times, despite having significantly fewer parameters, approaches that of a model that stacks 6 layers where each layer has different parameters. We also explore the limits of recurrent stacking where we train extremely deep NMT models. This paper also examines the utility of our recurrently stacked model as a student model through transfer learning via leveraging pre-trained parameters and knowledge distillation, and shows that it compensates for the performance drops in translation quality that the direct training of recurrently stacked model brings. We also show how transfer learning helps in faster decoding on top of the already reduced number of parameters due to recurrent stacking. Finally, we analyze the effects of recurrently stacked layers by visualizing the attentions of models that use recurrently stacked layers and models that do not.
Energy Prioritized Caching for Cellular D2D Networks
Venue: arXiv cs.NI 2021
Year: 2021

Multimedia content transmission heavily taxes network resources and puts a significant burden on wireless systems in terms of capacity and energy consumption. In this context, device-to-device (D2D) paradigm has a utilitarian value to alleviate the network burden by utilizing short-range transmissions with less energy cost. For the realization of proper D2D networking, caching in devices is essential. Additionally, caching needs to operate efficiently with the aim of realizing network-wide energy improvements. To this end, we study multimedia caching in cellular D2D networks and propose an energy prioritized D2D caching (EPDC) algorithm in this work. We also investigate the optimal caching policy in that system setting. The impact of device capacity and D2D transmission range on energy consumption is studied by focusing on different operation modes such as D2D and base station transmissions. According to the simulation results, EPDC algorithm has substantial energy-efficiency gains over the commonly utilized Least Recently Used (LRU) algorithm and content-based caching algorithms PDC and SXO. For larger D2D transmission ranges, this improvement becomes more evident.
Shape Prior Non-Uniform Sampling Guided Real-time Stereo 3D Object Detection
Venue: arXiv cs.CV 2021
Year: 2021

Pseudo-LiDAR based 3D object detectors have gained popularity due to their high accuracy. However, these methods need dense depth supervision and suffer from inferior speed. To solve these two issues, a recently introduced RTS3D builds an efficient 4D Feature-Consistency Embedding (FCE) space for the intermediate representation of object without depth supervision. FCE space splits the entire object region into 3D uniform grid latent space for feature sampling point generation, which ignores the importance of different object regions. However, we argue that, compared with the inner region, the outer region plays a more important role for accurate 3D detection. To encode more information from the outer region, we propose a shape prior non-uniform sampling strategy that performs dense sampling in outer region and sparse sampling in inner region. As a result, more points are sampled from the outer region and more useful features are extracted for 3D detection. Further, to enhance the feature discrimination of each sampling point, we propose a high-level semantic enhanced FCE module to exploit more contextual information and suppress noise better. Experiments on the KITTI dataset are performed to show the effectiveness of the proposed method. Compared with the baseline RTS3D, our proposed method has 2.57% improvement on AP3d almost without extra network parameters. Moreover, our proposed method outperforms the state-of-the-art methods without extra supervision at a real-time speed.
Training or Architecture? How to Incorporate Invariance in Neural Networks
Venue: arXiv cs.CV 2021
Year: 2021

Many applications require the robustness, or ideally the invariance, of a neural network to certain transformations of input data. Most commonly, this requirement is addressed by either augmenting the training data, using adversarial training, or defining network architectures that include the desired invariance automatically. Unfortunately, the latter often relies on the ability to enlist all possible transformations, which make such approaches largely infeasible for infinite sets of transformations, such as arbitrary rotations or scaling. In this work, we propose a method for provably invariant network architectures with respect to group actions by choosing one element from a (possibly continuous) orbit based on a fixed criterion. In a nutshell, we intend to 'undo' any possible transformation before feeding the data into the actual network. We analyze properties of such approaches, extend them to equivariant networks, and demonstrate their advantages in terms of robustness as well as computational efficiency in several numerical examples. In particular, we investigate the robustness with respect to rotations of images (which can possibly hold up to discretization artifacts only) as well as the provable rotational and scaling invariance of 3D point cloud classification.