Interface: TextGenerationParameters
Additional inference parameters
Additional inference parameters for Text Generation
Indexable
▪ [property: string]: unknown
Properties
best _ of
• Optional best_of: number
The number of sampling queries to run. Only the best one (in terms of total logprob) will be returned.
Defined in
inference/src/tasks/nlp/textGeneration.ts:34
decoder _ input _ details
• Optional decoder_input_details: boolean
Whether or not to output decoder input details
Defined in
inference/src/tasks/nlp/textGeneration.ts:38
details
• Optional details: boolean
Whether or not to output details
Defined in
inference/src/tasks/nlp/textGeneration.ts:42
do _ sample
• Optional do_sample: boolean
Whether to use logits sampling instead of greedy decoding when generating new tokens.
Defined in
inference/src/tasks/nlp/textGeneration.ts:46
max _ new _ tokens
• Optional max_new_tokens: number
The maximum number of tokens to generate.
Defined in
inference/src/tasks/nlp/textGeneration.ts:50
repetition _ penalty
• Optional repetition_penalty: number
The parameter for repetition penalty. A value of 1.0 means no penalty. See this paper for more details.
Defined in
inference/src/tasks/nlp/textGeneration.ts:55
return _ full _ text
• Optional return_full_text: boolean
Whether to prepend the prompt to the generated text.
Defined in
inference/src/tasks/nlp/textGeneration.ts:59
seed
• Optional seed: number
The random sampling seed.
Defined in
inference/src/tasks/nlp/textGeneration.ts:63
stop _ sequences
• Optional stop_sequences: string[]
Stop generating tokens if a member of stop_sequences is generated.
Defined in
inference/src/tasks/nlp/textGeneration.ts:67
temperature
• Optional temperature: number
The value used to modulate the logits distribution.
Defined in
inference/src/tasks/nlp/textGeneration.ts:71
top _ k
• Optional top_k: number
The number of highest probability vocabulary tokens to keep for top-k-filtering.
Defined in
inference/src/tasks/nlp/textGeneration.ts:75
top _ p
• Optional top_p: number
If set to < 1, only the smallest set of most probable tokens with probabilities that add
up to top_p or higher are kept for generation.
Defined in
inference/src/tasks/nlp/textGeneration.ts:80
truncate
• Optional truncate: number
Truncate input tokens to the given size.
Defined in
inference/src/tasks/nlp/textGeneration.ts:84
typical _ p
• Optional typical_p: number
Typical Decoding mass. See Typical Decoding for Natural Language Generation for more information
Defined in
inference/src/tasks/nlp/textGeneration.ts:89
watermark
• Optional watermark: boolean
Watermarking with A Watermark for Large Language Models