openai

/

whisper-large-v3

like 2.61k

I have also tried to use the pipeline chunk batching as described in https://huggingface.co/docs/transformers/en/main_classes/pipelines#pipeline-chunk-batching, but it simply just runs once with all the transcript in a single final chunk,

this is the code i tried

# ... imports and model loading
pipe = pipeline(
    "automatic-speech-recognition",
    model=model,
    tokenizer=processor.tokenizer,
    feature_extractor=processor.feature_extractor,
    torch_dtype=torch_dtype,
    max_new_tokens=128,
    chunk_length_s=30,
    batch_size=16,
    return_timestamps=True,
    device=device,
    generate_kwargs={"language": "en"}
)

dataset = load_dataset("distil-whisper/librispeech_long", "clean", split="validation")
sample = dataset[0]["audio"]

all_model_outputs = []
for preprocessed in pipe.preprocess(sample):
    model_outputs = pipe.forward(preprocessed)
    print('processed chunk: ', model_outputs)
    all_model_outputs.append(model_outputs)
outputs = pipe.postprocess(all_model_outputs)
print('complete output: ', outputs)

this is the output (most points on the tensor omitted):

processed chunk:  {'is_last': True, 'tokens': tensor([[50365, .....,  0, 50598]])}
complete output: {'text': " Mr. Quilter is the apostle of the middle classes, and we are glad to welcome his gospel. Nor is Mr. Quilter's manner less interesting than his matter. He tells us that at this festive season of the year, with Christmas and roast beef looming before us, similes drawn from eating and its results occur most readily to the mind. He has grave doubts whether Sir Frederick Leighton's work is really Greek after all, and can discover in it but little of rocky Ithaca. Linnell's pictures are a sort of Upguards and Adam paintings, and Mason's exquisite idylls are as national as a jingo poem. Mr. Burkett Foster's landscapes smile at one much in the same way that Mr. Carker

Have you found any solution?