Mid-implementation of using pos models.  Will break if pos_vocabulary_sizes has values.