metadata

license: bsd-3-clause

starcoder-toolbench

starcoder-toolbench is a 15 billion parameter model used for api based action generation. It is instruction tuned from starcoder on api based action generation datasets.

Model Details

Model Description

Developed by: SambaNova Systems
Model type: Language Model
Language(s): English
License:
Finetuned from model: starcoder

Basic Information

Paper: [Link]
Github: [Link]

Licensing

TBD

Uses

Click to expand

Direct Use

This model is intended for commercial and research use.

Out-of-Scope Use

starcoder-toolbench should NOT be used for purpose other than API based action generation.

Recommendations

Users should be made aware of the risks, biases, limitations, and restrictions of the model, which are listed down at the bottom of the page.

How to Get Started with the Model

Click to expand

Loading in model with Huggingface

from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("sambanovasystems/starcoder-toolbench")
model = AutoModelForCausalLM.from_pretrained("sambanovasystems/starcoder-toolbench", device_map="auto", torch_dtype="auto")

Training Details

Click to expand

Training Data

Fenglu to add

Training Procedure

We trained starcoder-toolbench on 4 80GB A100 gpu's. We started from starcoder. We finetuned it on XXX dataset. All of the code used to prepare the datasets and the scripts to run training and inference are open-sourced and freely available at [githublink here](dummy link)

Prompting Style Used For Training

Hyperparameters

Hardware: A100 GPU
Optimizer: AdamW
Grad accumulation: 1
Epochs: 8
Global Batch size: 16
Batch tokens: 16 * 2048 = 32,768 tokens
Learning Rate: 1e-5
Learning Rate Scheduler: Fixed LR
Weight decay: 0.1

Acknowledgment

Cite starcoder-toolbench

@software{bloomchat,
  title = {{BLOOMChat: a New Open Multilingual Chat LLM}},
  author = {SambaNova Systems, Together Computer},
  url = {https://huggingface.co/sambanovasystems/BLOOMChat-176B-v1}
  month = {5},
  year = {2023},
  version = {1.0},
}