Felladrin's picture
Update README.md
471bad6
metadata
license: bigscience-bloom-rail-1.0
datasets:
  - ehartford/wizard_vicuna_70k_unfiltered
  - shahules786/orca-chat
  - timdettmers/openassistant-guanaco
  - laion/OIG
language:
  - fr
  - en
library_name: transformers.js
base_model: cmarkea/bloomz-560m-sft-chat

INT8 ONNX version of cmarkea/bloomz-560m-sft-chat to use with Transformers.js.

Example usage

Pipeline API

import { pipeline } from '@xenova/transformers';

const generator = await pipeline('text-generation', 'Felladrin/onnx-bloomz-560m-sft-chat');
const output =  await generator('</s>What is the first world war?<s>', { add_special_tokens: true, max_new_tokens: 128, repetition_penalty: 1.2});
console.log(output); // The first world war was a conflict between the United States and the Soviet Union. The conflict began in World War II and lasted until the end of World War III.

Auto Classes

import { AutoModelForCausalLM, AutoTokenizer } from '@xenova/transformers';

const model_path = 'Felladrin/onnx-bloomz-560m-sft-chat';
const model = await AutoModelForCausalLM.from_pretrained(model_path);
const tokenizer = await AutoTokenizer.from_pretrained(model_path);

const prompt = '</s>What is the first world war?<s>';
const { input_ids } = tokenizer(prompt);
const tokens = await model.generate(input_ids, { max_new_tokens: 128, repetition_penalty: 1.2});
console.log(tokenizer.decode(tokens[0], { skip_special_tokens: true }));
// The first world war was a conflict between the United States and the Soviet Union. The conflict began in World War II and lasted until the end of World War III.