  When topic models are used for discovery of topics in text collections, a
question that arises naturally is how well the model-induced topics correspond
to topics of interest to the analyst. We investigate an approach to topic model
evaluation based on measuring topic coverage, and propose measures of coverage
based on matching between model topics and reference topics. We demonstrate the
benefits of the approach by evaluating, in a series of experiments, different
types of topic models on two distinct text domains. The experiments include
evaluation of model quality, analysis of coverage of distinct topic categories,
and the relation between coverage and other topic model evaluation methods. The
contributions of the paper include the measures of coverage and the
recommendations for the use of topic models for topic discovery.
