Felladrin/onnx-zephyr-smol_llama-100m-dpo-full

INT8 ONNX version of amazingvince/zephyr-smol_llama-100m-dpo-full to use with Transformers.js.

Usage

import { pipeline } from '@xenova/transformers';

const prompt = `<|system|>
You are a helpful assistant who answers the user's questions.
<|user|>
Question: What is a dog?
<|assistant|>
Answer: A dog is a four-legged, domesticated animal that is a member of the class Mammalia, which includes all mammals. Dogs are known for their loyalty, playfulness, and ability to be trained for various tasks. They are also used for hunting, herding, and as service animals.
<|user|>
Question: What is the capital of France?
<|assistant|>
Answer: The capital of France is Paris. Paris is located in the north-central region of France and is known for its famous landmarks, such as the Eiffel Tower, the Louvre Museum, and Notre-Dame Cathedral.
<|user|>
Question: What is the color of an apple?
<|assistant|>`;
const generator = await pipeline('text-generation', 'Felladrin/onnx-zephyr-smol_llama-100m-dpo-full', { quantized: false });
const output = await generator(prompt, { max_new_tokens: 19 });
console.log(output[0].generated_text); // "Answer: The color of an apple is a combination of red, yellow, and orange."