pwc-archive/papers-with-abstracts huggingface dataset:
SiamEvent: Event-based Object Tracking via Edge-aware Similarity Learning with Siamese Networks
Authors: Yujeong Chae, Lin Wang, Kuk-Jin Yoon
Date: 2021-09-28 00:00:00
Tasks: Motion Compensation, Object, Object Tracking

Event cameras are novel sensors that perceive the per-pixel intensity changes and output asynchronous event streams, showing lots of advantages over traditional cameras, such as high dynamic range (HDR) and no motion blur. It has been shown that events alone can be used for object tracking by motion compensation or prediction. However, existing methods assume that the target always moves and is the stand-alone object. Moreover, they fail to track the stopped non-independent moving objects on fixed scenes. In this paper, we propose a novel event-based object tracking framework, called SiamEvent, using Siamese networks via edge-aware similarity learning. Importantly, to find the part having the most similar edge structure of target, we propose to correlate the embedded events at two timestamps to compute the target edge similarity. The Siamese network enables tracking arbitrary target edge by finding the part with the highest similarity score. This extends the possibility of event-based object tracking applied not only for the independent stand-alone moving objects, but also for various settings of the camera and scenes. In addition, target edge initialization and edge detector are also proposed to prevent SiamEvent from the drifting problem. Lastly, we built an open dataset including various synthetic and real scenes to train and evaluate SiamEvent. Extensive experiments demonstrate that SiamEvent achieves up to 15% tracking performance enhancement than the baselines on the real-world scenes and more robust tracking performance in the challenging HDR and motion blur conditions.
Emergent Neural Network Mechanisms for Generalization to Objects in Novel Orientations
Authors: Avi Cooper, Xavier Boix, Daniel Harari, Spandan Madan, Hanspeter Pfister, Tomotake Sasaki, Pawan Sinha
Date: 2021-09-28 00:00:00

The capability of Deep Neural Networks (DNNs) to recognize objects in orientations outside the distribution of the training data is not well understood. We present evidence that DNNs are capable of generalizing to objects in novel orientations by disseminating orientation-invariance obtained from familiar objects seen from many viewpoints. This capability strengthens when training the DNN with an increasing number of familiar objects, but only in orientations that involve 2D rotations of familiar orientations. We show that this dissemination is achieved via neurons tuned to common features between familiar and unfamiliar objects. These results implicate brain-like neural mechanisms for generalization.
An Adaptive Deep Learning Framework for Day-ahead Forecasting of Photovoltaic Power Generation
Authors: Xing Luo, Dongxiao Zhang
Date: 2021-09-28 00:00:00
Tasks: General Knowledge

Accurate forecasts of photovoltaic power generation (PVPG) are essential to optimize operations between energy supply and demand. Recently, the propagation of sensors and smart meters has produced an enormous volume of data, which supports the development of data based PVPG forecasting. Although emerging deep learning (DL) models, such as the long short-term memory (LSTM) model, based on historical data, have provided effective solutions for PVPG forecasting with great successes, these models utilize offline learning. As a result, DL models cannot take advantage of the opportunity to learn from newly-arrived data, and are unable to handle concept drift caused by installing extra PV units and unforeseen PV unit failures. Consequently, to improve day-ahead PVPG forecasting accuracy, as well as eliminate the impacts of concept drift, this paper proposes an adaptive LSTM (AD-LSTM) model, which is a DL framework that can not only acquire general knowledge from historical data, but also dynamically learn specific knowledge from newly-arrived data. A two-phase adaptive learning strategy (TP-ALS) is integrated into AD-LSTM, and a sliding window (SDWIN) algorithm is proposed, to detect concept drift in PV systems. Multiple datasets from PV systems are utilized to assess the feasibility and effectiveness of the proposed approaches. The developed AD-LSTM model demonstrates greater forecasting capability than the offline LSTM model, particularly in the presence of concept drift. Additionally, the proposed AD-LSTM model also achieves superior performance in terms of day-ahead PVPG forecasting compared to other traditional machine learning models and statistical models in the literature.
DynG2G: An Efficient Stochastic Graph Embedding Method for Temporal Graphs
Authors: Mengjia Xu, Apoorva Vikram Singh, George Em Karniadakis
Date: 2021-09-28 00:00:00
Tasks: Diversity, Dynamic graph embedding, Graph Embedding, Triplet, Uncertainty Quantification

Dynamic graph embedding has gained great attention recently due to its capability of learning low dimensional graph representations for complex temporal graphs with high accuracy. However, recent advances mostly focus on learning node embeddings as deterministic "vectors" for static graphs yet disregarding the key graph temporal dynamics and the evolving uncertainties associated with node embedding in the latent space. In this work, we propose an efficient stochastic dynamic graph embedding method (DynG2G) that applies an inductive feed-forward encoder trained with node triplet-based contrastive loss. Every node per timestamp is encoded as a time-dependent probabilistic multivariate Gaussian distribution in the latent space, hence we can quantify the node embedding uncertainty on-the-fly. We adopted eight different benchmarks that represent diversity in size (from 96 nodes to 87,626 and from 13,398 edges to 4,870,863) and diversity in dynamics. We demonstrate via extensive experiments on these eight dynamic graph benchmarks that DynG2G achieves new state-of-the-art performance in capturing the underlying temporal node embeddings. We also demonstrate that DynG2G can predict the evolving node embedding uncertainty, which plays a crucial role in quantifying the intrinsic dimensionality of the dynamical system over time. We obtain a universal relation of the optimal embedding dimension, $L_o$, versus the effective dimensionality of uncertainty, $D_u$, and we infer that $L_o=D_u$ for all cases. This implies that the uncertainty quantification approach we employ in the DynG2G correctly captures the intrinsic dimensionality of the dynamics of such evolving graphs despite the diverse nature and composition of the graphs at each timestamp. Moreover, this $L_0 - D_u$ correlation provides a clear path to select adaptively the optimum embedding size at each timestamp by setting $L \ge D_u$.
SYGMA: System for Generalizable Modular Question Answering OverKnowledge Bases
Authors: Sumit Neelam, Udit Sharma, Hima Karanam, Shajith Ikbal, Pavan Kapanipathi, Ibrahim Abdelaziz, Nandana Mihindukulasooriya, Young-suk Lee, Santosh Srivastava, Cezar Pendus, Saswati Dana, Dinesh Garg, Achille Fokoue, G P Shrivatsa Bhargav, Dinesh Khandelwal, Srinivas Ravishankar, Sairam Gurajada, Maria Chang, Rosario Uceda-Sosa, Salim Roukos, Alexander Gray, Guilherme LimaRyan Riegel, Francois Luus, L Venkata Subramaniam
Date: 2021-09-28 00:00:00
Tasks: Knowledge Base Question Answering, Question Answering

Knowledge Base Question Answering (KBQA) tasks that in-volve complex reasoning are emerging as an important re-search direction. However, most KBQA systems struggle withgeneralizability, particularly on two dimensions: (a) acrossmultiple reasoning types where both datasets and systems haveprimarily focused on multi-hop reasoning, and (b) across mul-tiple knowledge bases, where KBQA approaches are specif-ically tuned to a single knowledge base. In this paper, wepresent SYGMA, a modular approach facilitating general-izability across multiple knowledge bases and multiple rea-soning types. Specifically, SYGMA contains three high levelmodules: 1) KB-agnostic question understanding module thatis common across KBs 2) Rules to support additional reason-ing types and 3) KB-specific question mapping and answeringmodule to address the KB-specific aspects of the answer ex-traction. We demonstrate effectiveness of our system by evalu-ating on datasets belonging to two distinct knowledge bases,DBpedia and Wikidata. In addition, to demonstrate extensi-bility to additional reasoning types we evaluate on multi-hopreasoning datasets and a new Temporal KBQA benchmarkdataset on Wikidata, namedTempQA-WD1, introduced in thispaper. We show that our generalizable approach has bettercompetetive performance on multiple datasets on DBpediaand Wikidata that requires both multi-hop and temporal rea-soning
Conditional Cross-Design Synthesis Estimators for Generalizability in Medicaid
Authors: Irina Degtiar, Tim Layton, Jacob Wallace, Sherri Rose
Date: 2021-09-27 00:00:00
Tasks: Causal Inference, Design Synthesis

While much of the causal inference literature has focused on addressing internal validity biases, both internal and external validity are necessary for unbiased estimates in a target population of interest. However, few generalizability approaches exist for estimating causal quantities in a target population when the target population is not well-represented by a randomized study but is reflected when additionally incorporating observational data. To generalize to a target population represented by a union of these data, we propose a class of novel conditional cross-design synthesis estimators that combine randomized and observational data, while addressing their respective biases. The estimators include outcome regression, propensity weighting, and double robust approaches. All use the covariate overlap between the randomized and observational data to remove potential unmeasured confounding bias. We apply these methods to estimate the causal effect of managed care plans on health care spending among Medicaid beneficiaries in New York City.
Evaluation of Deep Neural Network Domain Adaptation Techniques for Image Recognition
Authors: Alan Preciado-Grijalva, Venkata Santosh Sai Ramireddy Muthireddy
Date: 2021-09-28 00:00:00
Tasks: Domain Adaptation, image-classification, Image Classification, Unsupervised Domain Adaptation

It has been well proved that deep networks are efficient at extracting features from a given (source) labeled dataset. However, it is not always the case that they can generalize well to other (target) datasets which very often have a different underlying distribution. In this report, we evaluate four different domain adaptation techniques for image classification tasks: DeepCORAL, DeepDomainConfusion, CDAN and CDAN+E. These techniques are unsupervised given that the target dataset dopes not carry any labels during training phase. We evaluate model performance on the office-31 dataset. A link to the github repository of this report can be found here: https://github.com/agrija9/Deep-Unsupervised-Domain-Adaptation.
The Role of Lookahead and Approximate Policy Evaluation in Reinforcement Learning with Linear Value Function Approximation
Authors: Anna Winnicki, Joseph Lubars, Michael Livesay, R. Srikant
Date: 2021-09-28 00:00:00

Function approximation is widely used in reinforcement learning to handle the computational difficulties associated with very large state spaces. However, function approximation introduces errors which may lead to instabilities when using approximate dynamic programming techniques to obtain the optimal policy. Therefore, techniques such as lookahead for policy improvement and m-step rollout for policy evaluation are used in practice to improve the performance of approximate dynamic programming with function approximation. We quantitatively characterize, for the first time, the impact of lookahead and m-step rollout on the performance of approximate dynamic programming (DP) with function approximation: (i) without a sufficient combination of lookahead and m-step rollout, approximate DP may not converge, (ii) both lookahead and m-step rollout improve the convergence rate of approximate DP, and (iii) lookahead helps mitigate the effect of function approximation and the discount factor on the asymptotic performance of the algorithm. Our results are presented for two approximate DP methods: one which uses least-squares regression to perform function approximation and another which performs several steps of gradient descent of the least-squares objective in each iteration.
Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasets
Authors: Frederik Ebert, Yanlai Yang, Karl Schmeckpeper, Bernadette Bucher, Georgios Georgakis, Kostas Daniilidis, Chelsea Finn, Sergey Levine
Date: 2021-09-27 00:00:00
Tasks: Domain Generalization

Robot learning holds the promise of learning policies that generalize broadly. However, such generalization requires sufficiently diverse datasets of the task of interest, which can be prohibitively expensive to collect. In other fields, such as computer vision, it is common to utilize shared, reusable datasets, such as ImageNet, to overcome this challenge, but this has proven difficult in robotics. In this paper, we ask: what would it take to enable practical data reuse in robotics for end-to-end skill learning? We hypothesize that the key is to use datasets with multiple tasks and multiple domains, such that a new user that wants to train their robot to perform a new task in a new domain can include this dataset in their training process and benefit from cross-task and cross-domain generalization. To evaluate this hypothesis, we collect a large multi-domain and multi-task dataset, with 7,200 demonstrations constituting 71 tasks across 10 environments, and empirically study how this data can improve the learning of new tasks in new environments. We find that jointly training with the proposed dataset and 50 demonstrations of a never-before-seen task in a new domain on average leads to a 2x improvement in success rate compared to using target domain data alone. We also find that data for only a few tasks in a new domain can bridge the domain gap and make it possible for a robot to perform a variety of prior tasks that were only seen in other domains. These results suggest that reusing diverse multi-task and multi-domain datasets, including our open-source dataset, may pave the way for broader robot generalization, eliminating the need to re-collect data for each new robot learning project.
The Tensor Brain: A Unified Theory of Perception, Memory and Semantic Decoding
Authors: Volker Tresp, Sahand Sharifzadeh, Hang Li, Dario Konopatzki, Yunpu Ma
Date: 2021-09-27 00:00:00
Tasks: Decision Making, Self-Supervised Learning

We present a unified computational theory of an agent's perception and memory. In our model, perception, episodic memory, and semantic memory are realized by different operational modes of the oscillating interactions between a symbolic index layer and a subsymbolic representation layer. The two layers form a bilayer tensor network (BTN). Although memory appears to be about the past, its main purpose is to support the agent in the present and the future. Recent episodic memory provides the agent with a sense of the here and now. Remote episodic memory retrieves relevant past experiences to provide information about possible future scenarios. This aids the agent in decision-making. "Future" episodic memory, based on expected future events, guides planning and action. Semantic memory retrieves specific information, which is not delivered by current perception, and defines priors for future observations. We argue that it is important for the agent to encode individual entities, not just classes and attributes. We demonstrate that a form of self-supervised learning can acquire new concepts and refine existing ones. We test our model on a standard benchmark data set, which we expanded to contain richer representations for attributes, classes, and individuals. Our key hypothesis is that obtaining a better understanding of perception and memory is a crucial prerequisite to comprehending human-level intelligence.
Model-Free Reinforcement Learning for Optimal Control of MarkovDecision Processes Under Signal Temporal Logic Specifications
Authors: Krishna C. Kalagarla, Rahul Jain, Pierluigi Nuzzo
Date: 2021-09-27 00:00:00
Tasks: Motion Planning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We present a model-free reinforcement learning algorithm to find an optimal policy for a finite-horizon Markov decision process while guaranteeing a desired lower bound on the probability of satisfying a signal temporal logic (STL) specification. We propose a method to effectively augment the MDP state space to capture the required state history and express the STL objective as a reachability objective. The planning problem can then be formulated as a finite-horizon constrained Markov decision process (CMDP). For a general finite horizon CMDP problem with unknown transition probability, we develop a reinforcement learning scheme that can leverage any model-free RL algorithm to provide an approximately optimal policy out of the general space of non-stationary randomized policies. We illustrate the effectiveness of our approach in the context of robotic motion planning for complex missions under uncertainty and performance objectives.
Trustworthy AI and Robotics and the Implications for the AEC Industry: A Systematic Literature Review and Future Potentials
Authors: Newsha Emaminejad, Reza Akhavian
Date: 2021-09-27 00:00:00
Tasks: Systematic Literature Review

Human-technology interaction deals with trust as an inevitable requirement for user acceptance. As the applications of artificial intelligence (AI) and robotics emerge and with their ever-growing socio-economic influence in various fields of research and practice, there is an imminent need to study trust in such systems. With the opaque work mechanism of AI-based systems and the prospect of intelligent robots as workers' companions, context-specific interdisciplinary studies on trust are key in increasing their adoption. Through a thorough systematic literature review on (1) trust in AI and robotics (AIR) and (2) AIR applications in the architecture, engineering, and construction (AEC) industry, this study identifies common trust dimensions in the literature and uses them to organize the paper. Furthermore, the connections of the identified dimensions to the existing and potential AEC applications are determined and discussed. Finally, major future directions on trustworthy AI and robotics in AEC research and practice are outlined.
A taxonomy of strategic human interactions in traffic conflicts
Authors: Atrisha Sarkar, Kate Larson, Krzysztof Czarnecki
Date: 2021-09-27 00:00:00
Tasks: Autonomous Vehicles, Navigate

In order to enable autonomous vehicles (AV) to navigate busy traffic situations, in recent years there has been a focus on game-theoretic models for strategic behavior planning in AVs. However, a lack of common taxonomy impedes a broader understanding of the strategies the models generate as well as the development of safety specification to identity what strategies are safe for an AV to execute. Based on common patterns of interaction in traffic conflicts, we develop a taxonomy for strategic interactions along the dimensions of agents' initial response to right-of-way rules and subsequent response to other agents' behavior. Furthermore, we demonstrate a process of automatic mapping of strategies generated by a strategic planner to the categories in the taxonomy, and based on vehicle-vehicle and vehicle-pedestrian interaction simulation, we evaluate two popular solution concepts used in strategic planning in AVs, QLk and Subgame perfect $\epsilon$-Nash Equilibrium, with respect to those categories.
IGAN: Inferent and Generative Adversarial Networks
Authors: Dr. Luc Vignaud
Date: 2021-09-27 00:00:00
Tasks: Articles, Translation

I present IGAN (Inferent Generative Adversarial Networks), a neural architecture that learns both a generative and an inference model on a complex high dimensional data distribution, i.e. a bidirectional mapping between data samples and a simpler low-dimensional latent space. It extends the traditional GAN framework with inference by rewriting the adversarial strategy in both the image and the latent space with an entangled game between data-latent encoded posteriors and priors. It brings a measurable stability and convergence to the classical GAN scheme, while keeping its generative quality and remaining simple and frugal in order to run on a lab PC. IGAN fosters the encoded latents to span the full prior space: this enables the exploitation of an enlarged and self-organised latent space in an unsupervised manner. An analysis of previously published articles sets the theoretical ground for the proposed algorithm. A qualitative demonstration of potential applications like self-supervision or multi-modal data translation is given on common image datasets including SAR and optical imagery.
Urban Driver: Learning to Drive from Real-world Demonstrations Using Policy Gradients
Authors: Oliver Scheel, Luca Bergamini, Maciej Wołczyk, Błażej Osiński, Peter Ondruska
Date: 2021-09-27 00:00:00

In this work we are the first to present an offline policy gradient method for learning imitative policies for complex urban driving from a large corpus of real-world demonstrations. This is achieved by building a differentiable data-driven simulator on top of perception outputs and high-fidelity HD maps of the area. It allows us to synthesize new driving experiences from existing demonstrations using mid-level representations. Using this simulator we then train a policy network in closed-loop employing policy gradients. We train our proposed method on 100 hours of expert demonstrations on urban roads and show that it learns complex driving policies that generalize well and can perform a variety of driving maneuvers. We demonstrate this in simulation as well as deploy our model to self-driving vehicles in the real-world. Our method outperforms previously demonstrated state-of-the-art for urban driving scenarios -- all this without the need for complex state perturbations or collecting additional on-policy data during training. We make code and data publicly available.
GANG-MAM: GAN based enGine for Modifying Android Malware
Authors: Renjith G, Sonia Laudanna, Aji S, Corrado Aaron Visaggio, Vinod P
Date: 2021-09-27 00:00:00

Malware detectors based on machine learning are vulnerable to adversarial attacks. Generative Adversarial Networks (GAN) are architectures based on Neural Networks that could produce successful adversarial samples. The interest towards this technology is quickly growing. In this paper, we propose a system that produces a feature vector for making an Android malware strongly evasive and then modify the malicious program accordingly. Such a system could have a twofold contribution: it could be used to generate datasets to validate systems for detecting GAN-based malware and to enlarge the training and testing dataset for making more robust malware classifiers.