Repository: tensorzero/tensorzero (language: rust)

Issue:
"Raw Data History" component causes massive performance degradation for long-running SFT jobs
The component got so full during a Llama 3 70B fine-tune that it froze my tab.


Discussion / hints:
Hey, I think the issue is mainly because we keep adding to the history array without any limit, each new update adds more data that needs to be stored and rendered. maybe we can just keep the last 10-50 items to fix this? should be a one line fix, I can take this up

but if we really need to see the full history, we could even store the whole history in browser IndexedDB and only render what's needed. 
I think it's fine to truncate—we don't need entire history in this raw form and if we need some specific number we'll parse it.

On Sat, Jan 25, 2025 at 11:59 AM, Chetan < ***@***.*** > wrote:

> 
> 
> 
> 
> 
> 
> 
> Hey, I think the issue is mainly because we keep adding to the history
> array without any limit, each new update adds more data that needs to be
> stored and rendered. maybe we can just keep the last 10-50 items to fix
> this? should be a one line fix, I can take this up
> 
> 
> 
> but if we really need to see the full history, we could even store the
> whole history in browser IndexedDB and only render what's needed.
> 
> 
> 
> —
> Reply to this email directly, view it on GitHub (
> https://github.com/tensorzero/tensorzero/issues/839#issuecomment-2614030605
> ) , or unsubscribe (
> https://github.com/notifications/unsubscribe-auth/ACGQ6BDNFKDAWN5GDRWY5PD2MO7IXAVCNFSM6AAAAABV2TTJMWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMMJUGAZTANRQGU
> ).
> You are receiving this because you were assigned. Message ID: <tensorzero/tensorzero/issues/839/2614030605
> @ github. com>
> 
> 
>
> I think it's fine to truncate—we don't need entire history in this raw form and if we need some specific number we'll parse it.
> […](#)

Should I raise a PR, or are you working on this?
I haven't started it so a PR would be appreciated!!

On Sat, Jan 25, 2025 at 12:24 PM, Chetan < ***@***.*** > wrote:

> 
> 
> 
> 
> 
> 
>> 
>> 
>> I think it's fine to truncate—we don't need entire history in this raw
>> form and if we need some specific number we'll parse it.
>> … ( # )
>> 
>> 
> 
> 
> 
> Should I raise a PR, or are you working on this?
> 
> 
> 
> —
> Reply to this email directly, view it on GitHub (
> https://github.com/tensorzero/tensorzero/issues/839#issuecomment-2614037599
> ) , or unsubscribe (
> https://github.com/notifications/unsubscribe-auth/ACGQ6BEAL4HEHIFVSZJAIKT2MPCDLAVCNFSM6AAAAABV2TTJMWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMMJUGAZTONJZHE
> ).
> You are receiving this because you were assigned. Message ID: <tensorzero/tensorzero/issues/839/2614037599
> @ github. com>
> 
> 
>

Fix patch:
diff --git a/ui/app/routes/optimization/supervised-fine-tuning/RawDataAccordion.tsx b/ui/app/routes/optimization/supervised-fine-tuning/RawDataAccordion.tsx
index cf5eb83db..09ad2b555 100644
--- a/ui/app/routes/optimization/supervised-fine-tuning/RawDataAccordion.tsx
+++ b/ui/app/routes/optimization/supervised-fine-tuning/RawDataAccordion.tsx
@@ -16,6 +16,8 @@ type RawDataWithTimestamp = RawData & {
   timestamp: Date;
 };
 
+const MAX_HISTORY_ITEMS = 20;
+
 export function RawDataAccordion({ rawData }: RawDataAccordionProps) {
   const [history, setHistory] = useState<RawDataWithTimestamp[]>([]);
   // Add state for accordion value
@@ -35,7 +37,11 @@ export function RawDataAccordion({ rawData }: RawDataAccordionProps) {
   // Add new raw data to history when it changes
   useEffect(() => {
     if (rawData) {
-      setHistory((prev) => [...prev, { ...rawData, timestamp: new Date() }]);
+      setHistory((prev) =>
+        [...prev, { ...rawData, timestamp: new Date() }].slice(
+          -MAX_HISTORY_ITEMS,
+        ),
+      );
     }
   }, [rawData]);
 


Test patch: