Repository: BerriAI/litellm
Pull request: [FIX] Update token fields in schema.prisma to use BigInt for improved…
… handling of large values across LiteLLM models.

## Title
Integer overflow in PostgreSQL spend-tracking query causes inaccurate totals (#10379)

<!-- e.g. "Implement user authentication feature" -->

## Relevant issues
Fixes #10379 


## Pre-Submission checklist

**Please complete all items before asking a LiteLLM maintainer to review your PR**

- [ ] I have Added testing in the [`tests/litellm/`](https://github.com/BerriAI/litellm/tree/main/tests/litellm) directory, **Adding at least 1 test is a hard requirement** - [see details](https://docs.litellm.ai/docs/extras/contributing_code)
- [x] I have added a screenshot of my new test passing locally 
- [x] My PR passes all unit tests on [`make test-unit`](https://docs.litellm.ai/docs/extras/contributing_code)
- [x] My PR's scope is as isolated as possible, it only solves 1 specific problem

![image](https://github.com/user-attachments/assets/035f4f7e-87e1-421d-b8a1-a3a84d6a9ab8)


## Type

<!-- Select the type of Pull Request -->
<!-- Keep only the necessary ones -->

🐛 Bug Fix

## Changes

Update `LiteLLM_DailyUserSpend`, `LiteLLM_DailyTeamSpend` and `LiteLLM_DailyTagSpend`  model fields from `Int` to `BigInt`





Problem statement:
[Bug]: PostgreSQL Integer Overflow Error in Spend Tracking System
### What happened?

Issue Description
When running LiteLLM with high usage, I encountered a PostgreSQL error indicating integer overflow in the spend tracking system. This occurs during the execution of the db_update_spend_transaction_handler function.



### Relevant log output

```shell
ConnectorError(ConnectorError { user_facing_error: None, kind: QueryError(PostgresError { code: "22003", message: "integer out of range", severity: "ERROR", detail: None, column: None, hint: None }), transient: false })
```

### Are you a ML Ops Team?

No

### What LiteLLM version are you on ?

main-v1.67.4-stable

### Twitter / LinkedIn details

_No response_

Hints from issue discussion:
What is your avg daly usage @vovolie, it is more then 32^2-1?
Yes, our usage is well beyond 32^2-1, with completion_tokens exceeding 10B.
@ishaan-jaff, @krrishdholakia  Should i try to fix this.
I think moving from int to bigint would fix this.



Facing the same issue...

@krrishdholakia @ishaan-jaff 

tokens also 10B+
yes @husnain7766 can you file a PR. Happy to review 
@husnain7766 just wondering if you have been able to make any progress?
Confirming is the fix here that all ints in the DB need to be moved to bigint ? 
> Confirming is the fix here that all ints in the DB need to be moved to bigint ? 

Yes that's all. We get out of ranger error from the db.
@ma-armenta are you using the redis transaction buffer ? 

https://docs.litellm.ai/docs/proxy/db_deadlocks#setup-on-litellm-config 
Hey - trying to understand this better - which table / column is this impacting? 

Is it spendlogs or the `daily_` tables ?

Commit message: [FIX] Update token fields in schema.prisma to use BigInt for improved handling of large values across LiteLLM models.

Patch to review:
diff --git a/litellm/proxy/schema.prisma b/litellm/proxy/schema.prisma
index 3c20b9a3cda1..f5d2db0c4a1e 100644
--- a/litellm/proxy/schema.prisma
+++ b/litellm/proxy/schema.prisma
@@ -238,9 +238,9 @@ model LiteLLM_SpendLogs {
   call_type           String
   api_key             String  @default ("") // Hashed API Token. Not the actual Virtual Key. Equivalent to 'token' column in LiteLLM_VerificationToken
   spend               Float    @default(0.0)
-  total_tokens        Int     @default(0)
-  prompt_tokens       Int     @default(0)
-  completion_tokens   Int     @default(0)
+  total_tokens        BigInt     @default(0)
+  prompt_tokens       BigInt     @default(0)
+  completion_tokens   BigInt     @default(0)
   startTime           DateTime // Assuming start_time is a DateTime field
   endTime             DateTime // Assuming end_time is a DateTime field
   completionStartTime DateTime? // Assuming completionStartTime is a DateTime field
@@ -361,14 +361,14 @@ model LiteLLM_DailyUserSpend {
   model               String   
   model_group         String?  
   custom_llm_provider String?  
-  prompt_tokens       Int      @default(0)
-  completion_tokens   Int      @default(0)
-  cache_read_input_tokens     Int      @default(0)
-  cache_creation_input_tokens Int      @default(0)
+  prompt_tokens       BigInt      @default(0)
+  completion_tokens   BigInt      @default(0)
+  cache_read_input_tokens     BigInt      @default(0)
+  cache_creation_input_tokens BigInt      @default(0)
   spend               Float    @default(0.0)
-  api_requests        Int      @default(0)
-  successful_requests Int      @default(0)
-  failed_requests     Int      @default(0)
+  api_requests        BigInt      @default(0)
+  successful_requests BigInt      @default(0)
+  failed_requests     BigInt      @default(0)
   created_at          DateTime @default(now())
   updated_at          DateTime @updatedAt
 
@@ -388,14 +388,14 @@ model LiteLLM_DailyTeamSpend {
   model               String   
   model_group         String?  
   custom_llm_provider String?  
-  prompt_tokens       Int      @default(0)
-  completion_tokens   Int      @default(0)
-  cache_read_input_tokens     Int      @default(0)
-  cache_creation_input_tokens Int      @default(0)
+  prompt_tokens       BigInt      @default(0)
+  completion_tokens   BigInt      @default(0)
+  cache_read_input_tokens     BigInt      @default(0)
+  cache_creation_input_tokens BigInt      @default(0)
   spend               Float    @default(0.0)
-  api_requests        Int      @default(0)
-  successful_requests Int      @default(0)
-  failed_requests     Int      @default(0)
+  api_requests        BigInt      @default(0)
+  successful_requests BigInt      @default(0)
+  failed_requests     BigInt      @default(0)
   created_at          DateTime @default(now())
   updated_at          DateTime @updatedAt
 
@@ -415,14 +415,14 @@ model LiteLLM_DailyTagSpend {
   model               String   
   model_group         String?  
   custom_llm_provider String?  
-  prompt_tokens       Int      @default(0)
-  completion_tokens   Int      @default(0)
-  cache_read_input_tokens     Int      @default(0)
-  cache_creation_input_tokens Int      @default(0)
+  prompt_tokens       BigInt      @default(0)
+  completion_tokens   BigInt      @default(0)
+  cache_read_input_tokens     BigInt      @default(0)
+  cache_creation_input_tokens BigInt      @default(0)
   spend               Float    @default(0.0)
-  api_requests        Int      @default(0)
-  successful_requests Int      @default(0)
-  failed_requests     Int      @default(0)
+  api_requests        BigInt      @default(0)
+  successful_requests BigInt      @default(0)
+  failed_requests     BigInt      @default(0)
   created_at          DateTime @default(now())
   updated_at          DateTime @updatedAt
 
diff --git a/schema.prisma b/schema.prisma
index 3c20b9a3cda1..f5d2db0c4a1e 100644
--- a/schema.prisma
+++ b/schema.prisma
@@ -238,9 +238,9 @@ model LiteLLM_SpendLogs {
   call_type           String
   api_key             String  @default ("") // Hashed API Token. Not the actual Virtual Key. Equivalent to 'token' column in LiteLLM_VerificationToken
   spend               Float    @default(0.0)
-  total_tokens        Int     @default(0)
-  prompt_tokens       Int     @default(0)
-  completion_tokens   Int     @default(0)
+  total_tokens        BigInt     @default(0)
+  prompt_tokens       BigInt     @default(0)
+  completion_tokens   BigInt     @default(0)
   startTime           DateTime // Assuming start_time is a DateTime field
   endTime             DateTime // Assuming end_time is a DateTime field
   completionStartTime DateTime? // Assuming completionStartTime is a DateTime field
@@ -361,14 +361,14 @@ model LiteLLM_DailyUserSpend {
   model               String   
   model_group         String?  
   custom_llm_provider String?  
-  prompt_tokens       Int      @default(0)
-  completion_tokens   Int      @default(0)
-  cache_read_input_tokens     Int      @default(0)
-  cache_creation_input_tokens Int      @default(0)
+  prompt_tokens       BigInt      @default(0)
+  completion_tokens   BigInt      @default(0)
+  cache_read_input_tokens     BigInt      @default(0)
+  cache_creation_input_tokens BigInt      @default(0)
   spend               Float    @default(0.0)
-  api_requests        Int      @default(0)
-  successful_requests Int      @default(0)
-  failed_requests     Int      @default(0)
+  api_requests        BigInt      @default(0)
+  successful_requests BigInt      @default(0)
+  failed_requests     BigInt      @default(0)
   created_at          DateTime @default(now())
   updated_at          DateTime @updatedAt
 
@@ -388,14 +388,14 @@ model LiteLLM_DailyTeamSpend {
   model               String   
   model_group         String?  
   custom_llm_provider String?  
-  prompt_tokens       Int      @default(0)
-  completion_tokens   Int      @default(0)
-  cache_read_input_tokens     Int      @default(0)
-  cache_creation_input_tokens Int      @default(0)
+  prompt_tokens       BigInt      @default(0)
+  completion_tokens   BigInt      @default(0)
+  cache_read_input_tokens     BigInt      @default(0)
+  cache_creation_input_tokens BigInt      @default(0)
   spend               Float    @default(0.0)
-  api_requests        Int      @default(0)
-  successful_requests Int      @default(0)
-  failed_requests     Int      @default(0)
+  api_requests        BigInt      @default(0)
+  successful_requests BigInt      @default(0)
+  failed_requests     BigInt      @default(0)
   created_at          DateTime @default(now())
   updated_at          DateTime @updatedAt
 
@@ -415,14 +415,14 @@ model LiteLLM_DailyTagSpend {
   model               String   
   model_group         String?  
   custom_llm_provider String?  
-  prompt_tokens       Int      @default(0)
-  completion_tokens   Int      @default(0)
-  cache_read_input_tokens     Int      @default(0)
-  cache_creation_input_tokens Int      @default(0)
+  prompt_tokens       BigInt      @default(0)
+  completion_tokens   BigInt      @default(0)
+  cache_read_input_tokens     BigInt      @default(0)
+  cache_creation_input_tokens BigInt      @default(0)
   spend               Float    @default(0.0)
-  api_requests        Int      @default(0)
-  successful_requests Int      @default(0)
-  failed_requests     Int      @default(0)
+  api_requests        BigInt      @default(0)
+  successful_requests BigInt      @default(0)
+  failed_requests     BigInt      @default(0)
   created_at          DateTime @default(now())
   updated_at          DateTime @updatedAt
 


Code review comments:
diff_hunk: @@ -238,9 +238,9 @@ model LiteLLM_SpendLogs {
   call_type           String
   api_key             String  @default ("") // Hashed API Token. Not the actual Virtual Key. Equivalent to 'token' column in LiteLLM_VerificationToken
   spend               Float    @default(0.0)
-  total_tokens        Int     @default(0)
-  prompt_tokens       Int     @default(0)
-  completion_tokens   Int     @default(0)
+  total_tokens        BigInt     @default(0)
path: litellm/proxy/schema.prisma
text: @user1:
please don't edit SpendLogs, this is not causing the warning you're seeing and spendLogs is 100M+ Logs large, this can cause isssue for users. 

@author:
Hmm make sense. Will fix this.

@author:
@user2-jaff dose this commit resolves it?