The Landscape of AI Safety and Beneficence Research: Input for Brainstorming at Beneficial AI 2017∗

ing patterns to establish semantics requires a dynamic mix of different reasoning abilities [452]. See section Unsupervised Model Learning which has many overlaps with this. 4.4. 2.3 Seeded Common Sense A core of common sense concepts, relationships, and dynamics can be used as as a pretraining or seeding to bootstrap a system that will learn additional ontology and knowledge around what is seeded [97]. When the core of the relevant contextual landscape can be specified via structured knowledge like this, one can effectively improve the system’s contextual awareness by providing a higher quantity and broader scope of usable world-knowledge than one might expect the system to typically need. Abstracting patterns, which has similarities to unsupervised model learning, requires this kind of dynamic mix of reasoning abilities [452]. 4.4. 2.4 Metareasoning Metareasoning, reasoning about reasoning, is an important common sense faculty that includes metalevel control of introspection and of computational activities and modelling the self [109, 103]. It helps to contextualize and modulate common sense reasoning and selection of appropriate algorithms [269]. 4.4. 2.5 Lengthening Horizons Planning and lookahead time horizons significantly influence both the effectiveness of and the stability of agents. A potential technique to do so for online-planning markov decision processes is to do state abstraction [34]. See section Follow-on Analysis. 4.4. 3 Concept Geometry Conceptual spaces, be they vector spaces, semantic networks, more implicit subsymbolic structures, or symbolic-subsymbolic hybrid representations, are how concepts are defined, grounded, and related to each other [161]. Any ontology, including implicit ones in subsymbolic architectures, is biased by the faculties with which it was constructed and intended [411, 161]. See section Value Structuring as that addresses the geometry of human values. See section Perceptual Distance Agnostic World Models as that addresses proximal versus distal spaces. 4.4. 3.1 Implicit Human Concepts Beyond the realism of concepts, a possible desideratum for enhanced human compatibility is similarity as compared with how humans conceptualize the world. Concepts can be defined with respect to groundings to human-like primitives such as senses, maintaining human-like concept relationships and spaces [412, 445]. Hierarchical structured probabilistic models are one plausible avenue for human-level concept learning [248]. One should also aim to understand theoretically and practically how learned representations of high-level human concepts could be expected to generalize, or fail to do so, in radically new contexts [439]. 4.4. 3.2 Conservative Concepts How can a classifier be trained to develop useful concepts that exclude quite atypical examples and edge cases? [437] Including negative examples to try to get a better sense of the classification boundaries is insufficient [171, 394]. Novel ways of analyzing cluster spaces may be necessary. See section Generative Adversarial Networks as actor-critic arrangements can refine the understanding of a concept, though more research needs to be done to scale the generalizations by the critic. See section Goal Stability as