Wouldn't it be more obvious to handle the duplicates here?

    pairs = enumerate(vocabulary)
    vocabulary = {}
    for i, t in pairs:
        if vocabulary.setdefault(t, i) != i:
            raise ValueError("Duplicate term in vocabulary: %r" % t)

That would save a bit of code.