TASK DEFINITION: In this task, you are given a part of an article. Your task is to generate headline (title) for this text. Preferred headlines are under fifteen words.
PROBLEM: Reactive (memoryless) policies are sufficient in completely observable Markov decision pro­ cesses (MDPs), but some kind of memory is usually necessary for optimal control of a par­ tially observable MDP. Policies with finite mem­ ory can be represented as finite-state automata. In this paper, we extend Baird and Moore's YAPS algorithm to the problem of learning gen­ eral finite-state automata. Because it performs stochastic gradient descent, this algorithm can be shown to converge to a locally optimal finite­ state controller. We provide the details of the algorithm and then consider the question of un­ der what conditions stochastic gradient descent will outperform exact gradient descent. We con­ clude with empirical results comparing the per­ formance of stochastic and exact gradient de­ scent, and showing the ability of our algorithm to extract the useful information contained in the sequence of past observations to compensate for the lack of observability at each time-step.

SOLUTION: Learning Finite-State Controllers for Partially Observable Environments

PROBLEM: The belief bias effect is a phenomenon which occurs when we think that we judge an argument based on our reasoning, but are actually influenced by our beliefs and prior knowledge. Evans, Barston and Pollard carried out a psychological syllogistic reasoning task to prove this effect. Participants were asked whether they would accept or reject a given syllogism. We discuss one specific case which is commonly assumed to be believable but which is actually not logically valid. By introducing abnormalities, abduction and background knowledge, we adequately model this case under the weak completion semantics. Our formalization reveals new questions about possible extensions in abductive reasoning. For instance, observations and their explanations might include some relevant prior abductive contextual information concerning some side-effect or leading to a contestable or refutable side-effect. A weaker notion indicates the support of some relevant consequences by a prior abductive context. Yet another definition describes jointly supported relevant consequences, which captures the idea of two observations containing mutually supportive side-effects. Though motivated with and exemplified by the running psychology application, the various new general abductive context definitions are introduced here and given a declarative semantics for the first time, and have a much wider scope of application. Inspection points, a concept introduced by Pereira and Pinto, allows us to express these definitions syntactically and intertwine them into an operational semantics.

SOLUTION: Contextual Abductive Reasoning with Side-Effects

PROBLEM: The production of color language is essential for grounded language generation. Color descriptions have many challenging properties: they can be vague, compositionally complex, and denotationally rich. We present an effective approach to generating color descriptions using recurrent neural networks and a Fouriertransformed color representation. Our model outperforms previous work on a conditional language modeling task over a large corpus of naturalistic color descriptions. In addition, probing the model’s output reveals that it can accurately produce not only basic color terms but also descriptors with non-convex denotations (“greenish”), bare modifiers (“bright”, “dull”), and compositional phrases (“faded teal”) not seen in training.

SOLUTION:
Learning to Generate Compositional Color Descriptions