Visual Explanation by High-Level Abduction: On Answer-Set Programming Driven Reasoning About Moving Objects

We propose a hybrid architecture for systematically computing robust visual explanation(s) encompassing hypothesis formation, belief revision, and default reasoning with video data. The architecture consists of two tightly integrated synergistic components: (1) (functional) answer set programming based abductive reasoning with SPACE-TIME TRACKLETS as native entities; and (2) a visual processing pipeline for detection based object tracking and motion analysis. We present the formal framework, its general implementation as a (declarative) method in answer set programming, and an example application and evaluation based on two diverse video datasets: the MOTChallenge benchmark developed by the vision community, and a recently developed Movie Dataset. Introduction A range of empirical research areas such as cognitive psychology and visual perception articulate human visual sensemaking as an inherently abductive (reasoning) process (Moriarty 1996;Magnani 2015) involving tight linkages between low-level sub-symbolic processes on the one hand, and highlevel object and event-based segmentation and inference involving concepts and relations on the other. In spite of the state of the art in artificial intelligence and computer vision, and most recent advances in neural visual processing, generalised explainable visual perception with conceptual categories in the context of dynamic visuo-spatial imagery remains an exceptionally challenging problem presenting many research opportunities at the interface of Logic, Language, and Computer Vision. Explainable Visual Perception We define explainable visual perception from a human-centred, and commonsense reasoning viewpoint. In this paper, it denotes the ability to declaratively: VXP1: hypothesise spatio-temporal belief (states) and events; events may be both primitive or temporally-ordered aggregates; from a more foundational viewpoint, what is alluded to here is a robust mechanism for counterfactual reasoning. Copyright c © 2018, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. VXP2: revise spatio-temporal beliefs, e.g., by nonmonotonically updating conflicting knowledge, to fix inherently incompatible configurations in space-time defying geometric constraints and commonsense laws of naive physics, e.g., pertaining to physical (un)realisability, spatio-temporal continuity. VXP3: make default assumptions, e.g., about spatiotemporal property persistence concerning occupancy or position of objects; identity of tracked objects in space-time. Explanatory reasoning in general is one of the hallmarks of general human reasoning ability; robust explainable visual perception particularly stands out as a foundational functional capability within the human visuo-spatial perception faculty. In this respect, the following considerations — establishing the scope of this paper— are important wrt. VXP1−3 : • our notion of explainability is driven by the ability to support commonsense, semantic question-answering over dynamic visuo-spatial imagery within a declarative KR setting; • the features alluded to in VXP1−3 are not exhaustive; we focus on those aspects that we deem most essential for the particular case of movement tracking. A Hybrid Architecture for Visual Explanation This paper is driven by the development of a visual explanation component within a large-scale computational vision & perception system targeted at a range of cognitive interaction technologies and autonomous systems where dynamic visuo-spatial imagery is inherent. The key contribution is a hybrid visual explanation method based on the integration of high-level abductive reasoning within Answer Set Programming (ASP) ((Brewka, Eiter, and Truszczyński 2011)) on the one hand, and lowlevel visual processing for object tracking on the other. The core focus of the paper is on the theory, implementation, and applied evaluation of the visual explanation method. We particularly emphasise the closely-knit nature of two key subcomponents representing abductive explanation (Σabd) and low-level motion tracking (Σtrk) modules respectively: Σabd. ASP-based abductive reasoning with abstract visuospatial concepts —such as OBJECTS, EVENTS, SPACE-TIME TRACKLETS— as native objects within ASP The Thirty-Second AAAI Conference on Artificial Intelligence (AAAI-18)