kenrunio/transcriber-punctuation-onnx

FullStop multilingual punctuation, ONNX

oliverguhr/fullstop-punctuation-multilang-large (English, German, French, Italian), converted to ONNX so it runs with ONNX Runtime and the tokenizers library alone, without PyTorch or Transformers. It is used by Transcriber to put full stops, commas and capitals back into run-on stretches of speech-to-text output.

For each word the model predicts the punctuation that follows it.

The files

• model.onnx + model.onnx.data (2.2 GB together; the second holds the weights, keep both in one folder)
• tokenizer.json: the original tokenizer, for the tokenizers library
• labels.json: class id to label: 0 (none), ., ,, ?, -, :

Inputs ids and mask (int64, shape (batch, tokens)); output logits (float32, shape (batch, tokens, 6)). Take the label of each word's first token:

import json, numpy as np, onnxruntime as ort
from tokenizers import Tokenizer
tok = Tokenizer.from_file("tokenizer.json"); tok.enable_truncation(512)
labels = {int(k): v for k, v in json.load(open("labels.json")).items()}
sess = ort.InferenceSession("model.onnx")
words = "so today we will look at the results and then we will take your questions".split()
enc = tok.encode(words, is_pretokenized=True)
ids = np.array([enc.ids], dtype=np.int64)
pred = sess.run(None, {"ids": ids, "mask": np.ones_like(ids)})[0].argmax(-1)[0]
first = {}
for pos, w in enumerate(enc.word_ids):
    if w is not None and w not in first:
        first[w] = labels[int(pred[pos])]

How it was made, and checked

Exported with torch.onnx.export(..., dynamo=True, opset_version=18, external_data=True) by tools/export_models.py. On the 10,768 words of a 90-minute transcript the ONNX model gave the same label as the PyTorch original for every word.

Licence and credit

MIT, as the original (LICENSE). This is a format conversion of the original weights; the weights themselves are unchanged. The model is by Oliver Guhr, trained on Europarl; see the original model card and FullStop.