Package: tokmon
⚡️NEW:tokmon beam- a self-hosted web UI to monitor API usage data, running fromlocalhost.______   ______    __  __    __    __    ______    __   __    
/\__  _\ /\  __ \  /\ \/ /   /\ "-./  \  /\  __ \  /\ "-.\ \   
\/_/\ \/ \ \ \/\ \ \ \  _"-. \ \ \-./\ \ \ \ \/\ \ \ \ \-.  \  
   \ \_\  \ \_____\ \ \_\ \_\ \ \_\ \ \_\ \ \_____\ \ \_\\"\_\ 
    \/_/   \/_____/  \/_/\/_/  \/_/  \/_/  \/_____/  \/_/ \/_/ 
                                                                   

CLI utility to monitor your OpenAI API token usagetokmon🔤🧐 - CLI to monitor OpenAI API usagetokmon(TokenMonitor) lets you keep track of your program's OpenAI API token usage.You can usetokmonjust like you would use thetimeutility, but instead of execution time you get OpenAI usage and cost stats.Installationpip install tokmonUsageNote:  tokmon works forgpt-*models (gpt-3.5-turbo,gpt-4, etc.). If you need support for other models (e.g.davinci) seetokmon#6.$tokmon/path/to/your/<yourprogram>[arg1][arg2]...tokmoncostreport:================================================================================Monitoredinvocation:python3./tests/python_example.py--promptsay'hello, tokmon!'Models:['gpt-3.5-turbo-0301']TotalUsage:{'total_prompt_tokens':26,'total_completion_tokens':12,'total_tokens':38}Pricing:{'gpt-3.5-turbo-0301':{'prompt_cost':0.002,'completion_cost':0.002,'per_tokens':1000}}TotalCost:$0.000076================================================================================WritingcostsummarytoJSONfile:./tokmon_usage_summary_1682039505.jsonAfter your program finishes running (or youctrl^Cout it),tokmonwill print a summary that looks like the above.tokmonalso generates a detailed report and saves it as aJSON file.You can use the--beam <url>flag to stream token usage data to a server. Seetokmon --beamfor more information.DemoTry it out# Install tokmonpipinstalltokmon# Clone this repo and `cd` into itgitclonehttps://github.com/yagil/tokmon.git&&cdtokmon/# export your OpenAI API key. This will be used in the test program (source in ./tests/python_example.py)exportOPENAI_API_KEY="YOUR_OPENAI_API_KEY"# Run tokmon (save the JSON summary to the current folder)# The test uses the openai python package. Install it if you don't have it: `pip install openai`tokmon--json_out=.python3./tests/python_example.py--prompt"say 'hello, tokmon!'"tokmonalso supports this:If your program uses multiple OpenAI models in the same invocation, their respective usages will be reflected in the report.You can run multiple instances oftokmonsimultaneously. Each invocation will generate a separate usage report.Pass a--json_out /your/path/report.jsonto get a detailed breakdown + conversation history in JSON format.Usetokmonwith your application or scriptWarningThis is a debugging tool. It is not intended to be used in any consequential setting. Use your best judgement!Prependtokmonto your normal program invocation like so:$tokmon/path/to/your/<yourprogram>[arg1][arg2]...Run and use your program just like you would normally (arguments and all). Interactive usage is supported as well.PythonSimply addtokmonto the beginning of your regular invocation$tokmonpython/path/to/your/script.pyThis will work for scripts and long running programs like Django / Flask / FastAPI servers.NodeFor scripts:$tokmonnode/path/to/your/script.jsnpm runEdit yourpackage.json's "scripts" entry to includetokmon.{"scripts":{"dev":"tokmon next dev",...}}Full usage and cost summary (JSON){"total_cost":0.0019199999999999998,"total_usage":{"total_prompt_tokens":18,"total_completion_tokens":23,"total_tokens":41},"pricing_data":"{'gpt-4-0314': {'prompt_cost': 0.03, 'completion_cost': 0.06, 'per_tokens': 1000}}","models":["gpt-4-0314"],"raw_data":[{"model":"gpt-4-0314","usage":{"prompt_tokens":18,"completion_tokens":23,"total_tokens":41},"cost":0.0019199999999999998,"messages":[{"role":"system","content":"You're a helpful assistant."},{"role":"user","content":"hello"},{"role":"assistant","content":"Hello! How can I help you today? If you have any questions or need assistance, feel free to ask."}]}]}How it workstokmonuses themitmproxy libraryto intercept HTTP requests and responses between your program and the OpenAI API.
It then processes the request and response data to calculate token usage and cost based ontokmon/openai-pricing.json.tokmonworks for programs inpython/node(using OpenAI's clients), orcurl(run directly, and not i.e. in a bash script).
SeeGolang instructionsfor instructions on how to usetokmonwith Golang programs.if youmanually installmitmproxy's CA certificate, it should work for all executables (note: haven't tested this.)In most cases,tokmonrelies on the'usage'field inOpenAI's API responsesfor token counts. For streaming requests, however,tokmonuses OpenAI'stiktoken librarydirectly to count the tokens. As of writing OpenAI's API does not return usage data for streaming requests (reference.)openai-pricing.jsonThe pricing data was extracted from OpenAI's website with the help of ChatGPT.tokmonis usingtokmon/openai-pricing.jsonfrom its package.{"last_updated":"2023-04-12","data_sources":["https://openai.com/pricing","https://platform.openai.com/docs/models/model-endpoint-compatibility"],"gpt-4":{"prompt_cost":0.03,"completion_cost":0.06,"per_tokens":1000},"gpt-4-0314":{"prompt_cost":0.03,"completion_cost":0.06,"per_tokens":1000},"gpt-4-32k":{"prompt_cost":0.06,"completion_cost":0.12,"per_tokens":1000},"gpt-4-32k-0314":{"prompt_cost":0.06,"completion_cost":0.12,"per_tokens":1000},"gpt-3.5-turbo":{"prompt_cost":0.002,"completion_cost":0.002,"per_tokens":1000},"gpt-3.5-turbo-0301":{"prompt_cost":0.002,"completion_cost":0.002,"per_tokens":1000},"text-davinci-003":{"cost":0.02,"per_tokens":1000},"text-curie-001":{"cost":0.002,"per_tokens":1000},"text-babbage-001":{"cost":0.0005,"per_tokens":1000},"text-ada-001":{"cost":0.0004,"per_tokens":1000},"text-embedding-ada-002":{"cost":0.0004,"per_tokens":1000}}You can override the default pricing with:tokmon --pricing /path/to/your/custom-openai-pricing.json ...This pricing JSON is incomplete (missing DALL-E, etc.), it may be incorrect, and it may go out of date.For best results, make sure to check that you have the latest pricing.MiscGolangHacky workaround for Golang programs. Add this to your program:// Import these packagesimport("os""crypto/tls""crypto/x509""io/ioutil""net/http")// Place this code somewhere in your go program *before* you make any calls to OpenAI's API.certFile:=os.Getenv("TOKMON_SSL_CERT_FILE")// This env variable will be set by tokmoncaCert,err:=ioutil.ReadFile(certFile)iferr==nil{caCertPool,_:=x509.SystemCertPool()caCertPool.AppendCertsFromPEM(caCert)http.DefaultTransport.(*http.Transport).TLSClientConfig=&tls.Config{RootCAs:caCertPool,}}Current LimitationsEvent streaming:tokmonbuffers Server-Sent Events (SSE) until thedata: [DONE]chunk is received. If the monitored program leverages event streaming, its behavior will be modified.Issue:tokmon#4Only chat models are supported (gpt-3.5-turbo,gpt-4and variants)Issue:tokmon#6ContributingIf you'd like to contribute to the project, please follow these steps:Fork the repository.Create a new branch for your changes.Make your changes and test them.Submit a pull request with a clear description of your changes and any relevant information.Warningtokmoncomes without any warranty or guarantee whatsoever.tokmonwas tested on macOS only. It might not work on other platforms.This tool may not work as intended, have unknown side effects, may output incorrect information, or not work at all.The pricing data inopenai-pricing.jsonmay go out of date.