generation/streamers
generation/streamers.TextStreamer
Simple text streamer that prints the token(s) to stdout as soon as entire words are formed.
Kind: static class of generation/streamers
new TextStreamer(tokenizer, options)
| Param | Type | Default | Description |
|---|---|---|---|
| tokenizer | * | ||
| options | Object | ||
| [options.skip_prompt] | boolean | false | Whether to skip the prompt tokens |
| [options.callback_function] | function | | Function to call when a piece of text is ready to display |
| [options.token_callback_function] | function | | Function to call when a new token is generated |
| [options.decode_kwargs] | Object | {} | Additional keyword arguments to pass to the tokenizer's decode method |
textStreamer.put(value)
Receives tokens, decodes them, and prints them to stdout as soon as they form entire words.
Kind: instance method of TextStreamer
| Param | Type |
|---|---|
| value | Array.<Array<bigint>> |
textStreamer.end()
Flushes any remaining cache and prints a newline to stdout.
Kind: instance method of TextStreamer
textStreamer.on_finalized_text(text, stream_end)
Prints the new text to stdout. If the stream is ending, also prints a newline.
Kind: instance method of TextStreamer
| Param | Type |
|---|---|
| text | string |
| stream_end | boolean |
generation/streamers.WhisperTextStreamer
Utility class to handle streaming of tokens generated by whisper speech-to-text models. Callback functions are invoked when each of the following events occur:
- A new chunk starts (on_chunk_start)
- A new token is generated (callback_function)
- A chunk ends (on_chunk_end)
- The stream is finalized (on_finalize)
Kind: static class of generation/streamers
new WhisperTextStreamer(tokenizer, options)
| Param | Type | Default | Description |
|---|---|---|---|
| tokenizer | * | ||
| options | Object | ||
| [options.skip_prompt] | boolean | false | Whether to skip the prompt tokens |
| [options.callback_function] | function | | Function to call when a piece of text is ready to display |
| [options.token_callback_function] | function | | Function to call when a new token is generated |
| [options.on_chunk_start] | function | | Function to call when a new chunk starts |
| [options.on_chunk_end] | function | | Function to call when a chunk ends |
| [options.on_finalize] | function | | Function to call when the stream is finalized |
| [options.time_precision] | number | 0.02 | Precision of the timestamps |
| [options.skip_special_tokens] | boolean | true | Whether to skip special tokens when decoding |
| [options.decode_kwargs] | Object | {} | Additional keyword arguments to pass to the tokenizer's decode method |
whisperTextStreamer.put(value)
Kind: instance method of WhisperTextStreamer
| Param | Type |
|---|---|
| value | Array.<Array<bigint>> |
< > Update on GitHub