Package: tokencost
TokenCostClientside token counting + price estimation for LLM apps and AI agents. Tokencost helps calculate the USD cost of using major Large Language Model (LLMs) APIs by calculating the estimated cost of prompts and completions.Building AI agents? Check outAgentOpsFeaturesLLM Price TrackingMajor LLM providers frequently add new models and update pricing. This repo helps track the latest price changesToken countingAccurately count prompt tokens before sending OpenAI requestsEasy integrationGet the cost of a prompt or completion with a single functionExample usage:fromtokencostimportcalculate_prompt_cost,calculate_completion_costmodel="gpt-3.5-turbo"prompt=[{"role":"user","content":"Hello world"}]completion="How may I assist you today?"prompt_cost=calculate_prompt_cost(prompt,model)completion_cost=calculate_completion_cost(completion,model)print(f"{prompt_cost}+{completion_cost}={prompt_cost+completion_cost}")# 0.0000135 + 0.000014 = 0.0000275InstallationRecommended:PyPI:pipinstalltokencostUsageCost estimatesCalculating the cost of prompts and completions from OpenAI requestsfromopenaiimportOpenAIclient=OpenAI()model="gpt-3.5-turbo"prompt=[{"role":"user","content":"Say this is a test"}]chat_completion=client.chat.completions.create(messages=prompt,model=model)completion=chat_completion.choices[0].message.content# "This is a test."prompt_cost=calculate_prompt_cost(prompt,model)completion_cost=calculate_completion_cost(completion,model)print(f"{prompt_cost}+{completion_cost}={prompt_cost+completion_cost}")# 0.0000180 + 0.000010 = 0.0000280Calculating cost using string prompts instead of messages:fromtokencostimportcalculate_prompt_costprompt_string="Hello world"response="How may I assist you today?"model="gpt-3.5-turbo"prompt_cost=calculate_prompt_cost(prompt_string,model)print(f"Cost: ${prompt_cost}")# Cost: $3e-06Counting tokensfromtokencostimportcount_message_tokens,count_string_tokensmessage_prompt=[{"role":"user","content":"Hello world"}]# Counting tokens in prompts formatted as message listsprint(count_message_tokens(message_prompt,model="gpt-3.5-turbo"))# 9# Alternatively, counting tokens in string promptsprint(count_string_tokens(prompt="Hello world",model="gpt-3.5-turbo"))# 2Cost tableUnits denominated in USD. All prices can be located inmodel_prices.json.Prices last updated Jan 30, 2024 from:https://openai.com/pricingandhttps://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.jsonModel NamePrompt Cost (USD)Completion Cost (USD)Max Prompt Tokensgpt-4$0.00003000$0.000060008192gpt-4-0314$0.00003000$0.000060008192gpt-4-0613$0.00003000$0.000060008192gpt-4-32k$0.00006000$0.0001200032768gpt-4-32k-0314$0.00006000$0.0001200032768gpt-4-32k-0613$0.00006000$0.0001200032768gpt-4-1106-preview$0.00001000$0.00003000128000gpt-4-0125-preview$0.00001000$0.00003000128000gpt-4-vision-preview$0.00001000$0.00003000128000gpt-3.5-turbo$0.00000150$0.000002004097gpt-3.5-turbo-0301$0.00000150$0.000002004097gpt-3.5-turbo-0613$0.00000150$0.000002004097gpt-3.5-turbo-1106$0.00000050$0.0000015016385gpt-3.5-turbo-0125$0.00000050$0.0000015016385gpt-3.5-turbo-16k$0.00000300$0.0000040016385gpt-3.5-turbo-16k-0613$0.00000300$0.0000040016385text-embedding-ada-002$0.00000010$0.000000008191text-embedding-3-small$0.00000002$0.000000008191text-embedding-3-large$0.00000013$0.000000008191azure/gpt-4-1106-preview$0.00001000$0.00003000128000azure/gpt-4-0613$0.00003000$0.000060008192azure/gpt-4-32k-0613$0.00006000$0.0001200032768azure/gpt-4-32k$0.00006000$0.0001200032768azure/gpt-4$0.00003000$0.000060008192azure/gpt-35-turbo-16k-0613$0.00000300$0.0000040016385azure/gpt-35-turbo-1106$0.00000150$0.0000020016384azure/gpt-35-turbo-16k$0.00000300$0.0000040016385azure/gpt-35-turbo$0.00000150$0.000002004097azure/text-embedding-ada-002$0.00000010$0.000000008191text-davinci-003$0.00000200$0.000002004097text-curie-001$0.00000200$0.000002002049text-babbage-001$0.00000040$0.000000402049text-ada-001$0.00000040$0.000000402049babbage-002$0.00000040$0.0000004016384davinci-002$0.00000200$0.0000020016384gpt-3.5-turbo-instruct$0.00000150$0.000002008192claude-instant-1$0.00000160$0.00000550100000mistral/mistral-tiny$0.00000010$0.000000408192mistral/mistral-small$0.00000060$0.000001908192mistral/mistral-medium$0.00000270$0.000008208192claude-instant-1.2$0.00000010$0.00000050100000claude-2$0.00000800$0.00002400100000claude-2.1$0.00000800$0.00002400200000text-bison$0.00000010$0.000000108192text-bison@001$0.00000010$0.000000108192text-unicorn$0.00001000$0.000028008192text-unicorn@001$0.00001000$0.000028008192chat-bison$0.00000010$0.000000104096chat-bison@001$0.00000010$0.000000104096chat-bison@002$0.00000010$0.000000104096chat-bison-32k$0.00000010$0.0000001032000code-bison$0.00000010$0.000000106144code-bison@001$0.00000010$0.000000106144code-gecko@001$0.00000010$0.000000102048code-gecko@002$0.00000010$0.000000102048code-gecko$0.00000010$0.000000102048codechat-bison$0.00000010$0.000000106144codechat-bison@001$0.00000010$0.000000106144codechat-bison-32k$0.00000010$0.0000001032000gemini-pro$0.00000020$0.0000005030720gemini-pro-vision$0.00000020$0.0000005030720palm/chat-bison$0.00000010$0.000000104096palm/chat-bison-001$0.00000010$0.000000104096palm/text-bison$0.00000010$0.000000108196palm/text-bison-001$0.00000010$0.000000108196palm/text-bison-safety-off$0.00000010$0.000000108196palm/text-bison-safety-recitation-off$0.00000010$0.000000108196command-nightly$0.00001500$0.000015004096command$0.00001500$0.000015004096command-light$0.00001500$0.000015004096command-medium-beta$0.00001500$0.000015004096command-xlarge-beta$0.00001500$0.000015004096openrouter/openai/gpt-3.5-turbo$0.00000150$0.000002004095openrouter/openai/gpt-3.5-turbo-16k$0.00000300$0.0000040016383openrouter/openai/gpt-4$0.00003000$0.000060008192openrouter/anthropic/claude-instant-v1$0.00000160$0.00000550100000openrouter/anthropic/claude-2$0.00001100$0.00003260100000openrouter/google/palm-2-chat-bison$0.00000050$0.000000508000openrouter/google/palm-2-codechat-bison$0.00000050$0.000000508000openrouter/meta-llama/llama-2-13b-chat$0.00000020$0.000000204096openrouter/meta-llama/llama-2-70b-chat$0.00000150$0.000001504096openrouter/meta-llama/codellama-34b-instruct$0.00000050$0.000000508096openrouter/nousresearch/nous-hermes-llama2-13b$0.00000020$0.000000204096openrouter/mancer/weaver$0.00000560$0.000005608000openrouter/gryphe/mythomax-l2-13b$0.00000180$0.000001808192openrouter/jondurbin/airoboros-l2-70b-2.1$0.00001380$0.000013804096openrouter/undi95/remm-slerp-l2-13b$0.00000180$0.000001806144openrouter/pygmalionai/mythalion-13b$0.00000180$0.000001804096openrouter/mistralai/mistral-7b-instruct$0.00000000$0.000000004096j2-ultra$0.00001500$0.000015008192j2-mid$0.00001000$0.000010008192j2-light$0.00000300$0.000003008192dolphin$0.00002000$0.000020004096chatdolphin$0.00002000$0.000020004096luminous-base$0.00003000$0.000033002048luminous-base-control$0.00003740$0.000041202048luminous-extended$0.00004500$0.000049402048luminous-extended-control$0.00005620$0.000061802048luminous-supreme$0.00017500$0.000192502048luminous-supreme-control$0.00021870$0.000240602048ai21.j2-mid-v1$0.00001250$0.000012508191ai21.j2-ultra-v1$0.00001880$0.000018808191amazon.titan-text-lite-v1$0.00000030$0.000000408000amazon.titan-text-express-v1$0.00000130$0.000001708000anthropic.claude-v1$0.00000800$0.00002400100000bedrock/us-east-1/anthropic.claude-v1$0.00000800$0.00002400100000bedrock/us-west-2/anthropic.claude-v1$0.00000800$0.00002400100000bedrock/ap-northeast-1/anthropic.claude-v1$0.00000800$0.00002400100000bedrock/eu-central-1/anthropic.claude-v1$0.00000800$0.00002400100000anthropic.claude-v2$0.00000800$0.00002400100000bedrock/us-east-1/anthropic.claude-v2$0.00000800$0.00002400100000bedrock/us-west-2/anthropic.claude-v2$0.00000800$0.00002400100000bedrock/ap-northeast-1/anthropic.claude-v2$0.00000800$0.00002400100000bedrock/eu-central-1/anthropic.claude-v2$0.00000800$0.00002400100000anthropic.claude-v2:1$0.00000800$0.00002400200000bedrock/us-east-1/anthropic.claude-v2:1$0.00000800$0.00002400100000bedrock/us-west-2/anthropic.claude-v2:1$0.00000800$0.00002400100000bedrock/ap-northeast-1/anthropic.claude-v2:1$0.00000800$0.00002400100000bedrock/eu-central-1/anthropic.claude-v2:1$0.00000800$0.00002400100000anthropic.claude-instant-v1$0.00000160$0.00000550100000bedrock/us-east-1/anthropic.claude-instant-v1$0.00000080$0.00000240100000bedrock/us-west-2/anthropic.claude-instant-v1$0.00000080$0.00000240100000bedrock/ap-northeast-1/anthropic.claude-instant-v1$0.00000220$0.00000750100000bedrock/eu-central-1/anthropic.claude-instant-v1$0.00000240$0.00000830100000cohere.command-text-v14$0.00000150$0.000002004096cohere.command-light-text-v14$0.00000030$0.000000604000cohere.embed-english-v3$0.00000010$0.00000000512cohere.embed-multilingual-v3$0.00000010$0.00000000512meta.llama2-13b-chat-v1$0.00000070$0.000001004096meta.llama2-70b-chat-v1$0.00000190$0.000002504096sagemaker/meta-textgeneration-llama-2-7b$0.00000000$0.000000004096sagemaker/meta-textgeneration-llama-2-7b-f$0.00000000$0.000000004096sagemaker/meta-textgeneration-llama-2-13b$0.00000000$0.000000004096sagemaker/meta-textgeneration-llama-2-13b-f$0.00000000$0.000000004096sagemaker/meta-textgeneration-llama-2-70b$0.00000000$0.000000004096sagemaker/meta-textgeneration-llama-2-70b-b-f$0.00000000$0.000000004096together-ai-7.1b-20b$0.00000040$0.000000401000ollama/llama2$0.00000000$0.000000004096ollama/llama2:13b$0.00000000$0.000000004096ollama/llama2:70b$0.00000000$0.000000004096ollama/llama2-uncensored$0.00000000$0.000000004096ollama/mistral$0.00000000$0.000000008192ollama/codellama$0.00000000$0.000000004096ollama/orca-mini$0.00000000$0.000000004096ollama/vicuna$0.00000000$0.000000002048deepinfra/meta-llama/Llama-2-70b-chat-hf$0.00000070$0.000000904096deepinfra/codellama/CodeLlama-34b-Instruct-hf$0.00000060$0.000000604096deepinfra/meta-llama/Llama-2-13b-chat-hf$0.00000030$0.000000304096deepinfra/meta-llama/Llama-2-7b-chat-hf$0.00000020$0.000000204096deepinfra/mistralai/Mistral-7B-Instruct-v0.1$0.00000020$0.000000204096deepinfra/jondurbin/airoboros-l2-70b-gpt4-1.4.1$0.00000070$0.000000904096perplexity/pplx-7b-chat$0.00000000$0.000000008192perplexity/pplx-70b-chat$0.00000000$0.000000004096perplexity/pplx-7b-online$0.00000000$0.000500004096perplexity/pplx-70b-online$0.00000000$0.000500004096perplexity/llama-2-13b-chat$0.00000000$0.000000004096perplexity/llama-2-70b-chat$0.00000000$0.000000004096perplexity/mistral-7b-instruct$0.00000000$0.000000004096perplexity/replit-code-v1.5-3b$0.00000000$0.000000004096anyscale/mistralai/Mistral-7B-Instruct-v0.1$0.00000010$0.0000001016384anyscale/HuggingFaceH4/zephyr-7b-beta$0.00000010$0.0000001016384anyscale/meta-llama/Llama-2-7b-chat-hf$0.00000010$0.000000104096anyscale/meta-llama/Llama-2-13b-chat-hf$0.00000020$0.000000204096anyscale/meta-llama/Llama-2-70b-chat-hf$0.00000100$0.000001004096anyscale/codellama/CodeLlama-34b-Instruct-hf$0.00000100$0.0000010016384cloudflare/@cf/meta/llama-2-7b-chat-fp16$0.00000190$0.000001903072cloudflare/@cf/meta/llama-2-7b-chat-int8$0.00000190$0.000001902048cloudflare/@cf/mistral/mistral-7b-instruct-v0.1$0.00000190$0.000001908192cloudflare/@hf/thebloke/codellama-7b-instruct-awq$0.00000190$0.000001904096voyage/voyage-01$0.00000010$0.000000004096voyage/voyage-lite-01$0.00000010$0.000000004096Callback handlersYou may also calculate token costs in LLM wrapper/framework libraries using callbacks.LlamaIndexpipinstall`'tokencost[llama-index]'`To use the base callback handler, you may import it:fromtokencost.callbacks.llama_indeximportBaseCallbackHandlerand pass to your framework callback handler.Langchain(Coming Soon)Running locallyInstallation viaGitHub:gitclonegit@github.com:AgentOps-AI/tokencost.gitcdtokencost
pipinstall-e.Running testsInstallpytestif you don't have it alreadypipinstallpytestRun thetests/folder while in the parent directorypytesttestsThis repo also supportstox, simply runpython -m tox.ContributingContributions to TokenCost are welcome! Feel free to create anissuefor any bug reports, complaints, or feature suggestions.LicenseTokenCost is released under the MIT License.