Title: Enforcing constraints on outputs with unconstrained inference
Authors: Jay Yoon Lee, Michael L. Wick, Jean-Baptiste Tristan
Keywords: Natural language processing, Structured prediction, Deep learning
TL;DR: An inference method for enforcing hard constraints on the outputs of neural networks without combinatorial search, with applications in NLP and structured prediction.
Abstract:
  Increasingly, practitioners apply neural networks to complex
  problems in natural language processing (NLP), such as syntactic
  parsing, that have rich output structures. Many such applications
  require deterministic constraints on the output values; for example,
  requiring that the sequential outputs encode a valid tree. While
  hidden units might capture such properties, the network is not
  always able to learn them from the training data alone, and
  practitioners must then resort to post-processing. In this paper, we
  present an inference method for neural networks that enforces
  deterministic constraints on outputs without performing
  post-processing or expensive discrete search over the feasible
  space. Instead, for each input, we nudge the continuous weights
  until the network's unconstrained inference procedure generates an
  output that satisfies the constraints. We find that our method
  reduces the number of violating outputs by up to 81\%, while
  improving accuracy.
Venue: Submitted to ICLR 2017