Repository: CheckerNetwork/spark-dashboard
Pull request #28: feat: stats table for client RSR scores
Author: login: NikolasHaimerl
Created: 2025-02-25T12:29:34Z | Merged: 2025-03-10T11:50:14Z

**CHANGELOG**

1. Add data fetcher for spark clients rsr summary
2. Add stats table for client RSR scores similar to miner stats [table](https://github.com/CheckerNetwork/spark-dashboard/blob/main/src/index.md?plain=1#L329)
3. Per Client RSR visualization similar to miner RSR [visualization](https://github.com/CheckerNetwork/spark-dashboard/blob/main/src/provider/%5Bprovider%5D.md).

The visualization for the RSR scores for clients closely resembles that of the miner RSR scores. 
The two major features added in this PR is the stats table for clients and the per client stats visualization which appears once you click on one of the client ids in the stats table for clients. 

Closes the two subtasks of https://github.com/CheckerNetwork/roadmap/issues/193


Diff:
diff --git a/observablehq.config.js b/observablehq.config.js
index 06fbc25..e12a71d 100644
--- a/observablehq.config.js
+++ b/observablehq.config.js
@@ -1,14 +1,22 @@
 import { jsonFetcher } from './src/data/json-fetcher.js'
 import { getDateXDaysAgo } from './src/utils/date-utils.js'
 
-const start = '2024-04-07'
+const startProviders = '2024-04-07'
+const startClients = '2025-02-25'
 const end = getDateXDaysAgo(1)
 
-const result = await jsonFetcher(
-  `https://stats.filspark.com/miners/retrieval-success-rate/summary?from=${start}&to=${end}`,
+const providersSummary = await jsonFetcher(
+  `https://stats.filspark.com/miners/retrieval-success-rate/summary?from=${startProviders}&to=${end}`,
+)
+const clientsSummary = await jsonFetcher(
+  `https://stats.filspark.com/clients/retrieval-success-rate/summary?from=${startClients}&to=${end}`,
+)
+const providerPaths = providersSummary.map(
+  (provider) => `/provider/${provider.miner_id}`,
+)
+const clientPaths = clientsSummary.map(
+  (client) => `/client/${client.client_id}`,
 )
-const providerPaths = result.map((provider) => `/provider/${provider.miner_id}`)
-
 // See https://observablehq.com/framework/config for documentation.
 export default {
   // The app’s title; used in the sidebar and webpage titles.
@@ -45,5 +53,5 @@ export default {
   // linkify: true, // convert URLs in Markdown to links
   // typographer: false, // smart quotes and other typographic improvements
   // cleanUrls: true, // drop .html from URLs
-  dynamicPaths: providerPaths,
+  dynamicPaths: [...providerPaths, ...clientPaths],
 }
diff --git a/src/client/[client].md b/src/client/[client].md
new file mode 100644
index 0000000..ff8860c
--- /dev/null
+++ b/src/client/[client].md
@@ -0,0 +1,86 @@
+---
+toc: false
+title: Storage Client Summary
+---
+
+```js
+import { LineGraph } from '../components/line-graph.js'
+import { getDateXDaysAgo } from '../utils/date-utils.js'
+
+const data = FileAttachment(
+  `../data/${observable.params.client}-spark-client-rsr-summary.json`,
+).json()
+```
+
+<div class="hero">
+  <body><a href="/"><img src="../media/spark-logomark-blue-with-bbox.png" alt="Spark Logo" width="300" /></a><body>
+    <h2>Dashboard Beta</h2>
+    <body><a href="https://filspark.com/dashboard" target="_blank" rel="noopener noreferrer">(Click here for Legacy Spark Grafana Dashboard)</a><body>
+</div>
+
+<h4>Storage Client Spark RSR Summary</h4>
+<body>This section shows the storage client Spark Retrieval Success Rate Score summary. You can adjust the date range. Records start on the 25th February 2025.</body>
+
+```js
+const startDate = getDateXDaysAgo(180)
+const minStartDate = '2025-02-25'
+const start = view(
+  Inputs.date({
+    label: 'Start',
+    value:
+      new Date(startDate) >= new Date(minStartDate) ? startDate : minStartDate,
+  }),
+)
+const end = view(Inputs.date({ label: 'End', value: getDateXDaysAgo(1) }))
+```
+
+<h3>Stats for ${observable.params.client}</h3>
+
+<div class="grid grid-cols" style="grid-auto-rows: 500px;">
+  <div class="card">${
+    resize((width) => LineGraph(data, {width, title: "Retrieval Success Rate", start, end }))
+  }</div>
+</div>
+
+<style>
+
+.hero {
+  display: flex;
+  flex-direction: column;
+  align-items: center;
+  font-family: var(--sans-serif);
+  margin: 4rem 0 8rem;
+  text-wrap: balance;
+  text-align: center;
+}
+
+.hero h1 {
+  margin: 1rem 0;
+  padding: 1rem 0;
+  max-width: none;
+  font-size: 14vw;
+  font-weight: 900;
+  line-height: 1;
+  background: linear-gradient(30deg, var(--theme-foreground-focus), currentColor);
+  -webkit-background-clip: text;
+  -webkit-text-fill-color: transparent;
+  background-clip: text;
+}
+
+.hero h2 {
+  margin: 0;
+  max-width: 34em;
+  font-size: 20px;
+  font-style: initial;
+  font-weight: 500;
+  line-height: 1.5;
+  color: var(--theme-foreground-muted);
+}
+
+@media (min-width: 640px) {
+  .hero h1 {
+    font-size: 90px;
+  }
+}
+
+</style>
diff --git a/src/data/[client]-spark-client-rsr-summary.json.js b/src/data/[client]-spark-client-rsr-summary.json.js
new file mode 100644
index 0000000..90c29ab
--- /dev/null
+++ b/src/data/[client]-spark-client-rsr-summary.json.js
@@ -0,0 +1,20 @@
+import pRetry from 'p-retry'
+import { jsonFetcher } from './json-fetcher.js'
+import { getDateXDaysAgo } from '../utils/date-utils.js'
+import { parseArgs } from 'node:util'
+
+const {
+  values: { client },
+} = parseArgs({
+  options: { client: { type: 'string' } },
+})
+const start = '2025-02-25'
+const end = getDateXDaysAgo(1)
+const summary = await pRetry(
+  () =>
+    jsonFetcher(
+      `https://stats.filspark.com/client/${client}/retrieval-success-rate/summary?from=${start}&to=${end}`,
+    ),
+  { retries: 3 },
+)
+process.stdout.write(JSON.stringify(summary))
diff --git a/src/data/spark-clients-rsr.json.js b/src/data/spark-clients-rsr.json.js
new file mode 100644
index 0000000..9ded8dc
--- /dev/null
+++ b/src/data/spark-clients-rsr.json.js
@@ -0,0 +1,10 @@
+import { jsonFetcher } from './json-fetcher.js'
+import { getDateXDaysAgo } from '../utils/date-utils.js'
+
+const from = getDateXDaysAgo(31)
+const to = getDateXDaysAgo(1)
+
+const output = await jsonFetcher(
+  `https://stats.filspark.com/clients/retrieval-success-rate/summary?from=${from}&to=${to}`,
+)
+process.stdout.write(JSON.stringify(output))
diff --git a/src/data/spark-miner-rsr-summaries.json.js b/src/data/spark-miner-rsr-summaries.json.js
index 565f270..6c74098 100644
--- a/src/data/spark-miner-rsr-summaries.json.js
+++ b/src/data/spark-miner-rsr-summaries.json.js
@@ -3,6 +3,8 @@ import { getDateXDaysAgo } from '../utils/date-utils.js'
 
 const summaries = {}
 
+// Fetch the last 100 days of data
+// We want this number to be small enough so that the resulting json file that is generated is not too large (<25MB)
 for (let i = 100; i >= 1; i--) {
   const dayString = getDateXDaysAgo(i)
   summaries[dayString] = await jsonFetcher(
diff --git a/src/index.md b/src/index.md
index 5e925c3..045f2ac 100644
--- a/src/index.md
+++ b/src/index.md
@@ -24,6 +24,7 @@ const SparkMinerRsrSummaries = FileAttachment(
 const SparkRetrievalTimes = FileAttachment(
   './data/spark-retrieval-timings.json',
 ).json()
+const SparkClientRates = FileAttachment('./data/spark-clients-rsr.json').json()
 ```
 
 ```js
@@ -49,6 +50,13 @@ const tidySparkMinerRates = SparkMinerRates.sort(
     success_rate_http: `${(record.success_rate_http * 100).toFixed(2)}%`,
   }
 })
+const tidySparkClientRates = SparkClientRates.sort(
+  (recordA, recordB) => recordB.success_rate - recordA.success_rate,
+).map((record) => ({
+  ...record,
+  success_rate: `${(record.success_rate * 100).toFixed(2)}%`,
+  success_rate_http: `${(record.success_rate_http * 100).toFixed(2)}%`,
+}))
 ```
 
 <div class="hero">
@@ -317,7 +325,7 @@ ${JSON.stringify(
 <body>The following table shows the Spark RSR and TTFB values calculated in aggregate for each Filecoin Storage Provider over the past 30 days. Click on a miner id to view stats about this storage provider.</body>
 
 ```js
-const search = view(
+const searchMinerStats = view(
   Inputs.search(tidySparkMinerRates, {
     placeholder: 'Search Storage Providers…',
   }),
@@ -325,7 +333,24 @@ const search = view(
 ```
 
 <div class="card" style="padding: 0;">
-  ${Inputs.table(search, {rows: 16, format: {miner_id: id => htl.html`<a href=./provider/${id} target=_blank>${id}</a>`}})}
+  ${Inputs.table(searchMinerStats, {rows: 16, format: {miner_id: id => htl.html`<a href=./provider/${id} target=_blank>${id}</a>`}})}
+</div>
+
+<div class="divider"></div>
+
+<h4>Spark Client RSR Table</h4>
+<body>The following table shows the Spark RSR values calculated in aggregate for each Filecoin Storage Client over the past 30 days. Click on a client id to view stats about this storage client.</body>
+
+```js
+const searchClientStats = view(
+  Inputs.search(tidySparkClientRates, {
+    placeholder: 'Search Storage Clients...',
+  }),
+)
+```
+
+<div class="card" style="padding: 0;">
+  ${Inputs.table(searchClientStats, {rows: 16, format: {client_id: id => htl.html`<a href=./client/${id} target=_blank>${id}</a>`}})}
 </div>
 
 <style>


Reviews:
user: login: pyropy
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-03T13:49:26Z
user: login: bajtos
body: The changes in `src/index.md` look good to me 👍🏻 

However, the per-client page does not load correctly in the preview, PTAL.

I am not an expert on this codebase, I'll defer the final review & approval to @pyropy.
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-03T13:52:09Z
user: login: bajtos
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-03T13:54:06Z
user: login: NikolasHaimerl
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-03T15:08:09Z
user: login: NikolasHaimerl
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-03T15:09:28Z
user: login: pyropy
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-03T17:20:50Z
user: login: NikolasHaimerl
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-04T09:16:44Z
user: login: pyropy
body: Great job Nikolas and kudos to you for solving the summary issue! 👏🏻 There is just few small improvements left and I think we'll be ready to merge.
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-05T14:51:54Z
user: login: NikolasHaimerl
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-06T08:10:19Z
user: login: pyropy
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-06T08:32:18Z
user: login: pyropy
body: Awesome job Nikolas :shipit: 
state: APPROVED
author_association: MEMBER
submitted_at: 2025-03-10T11:31:34Z

Review comments:
diff_hunk: @@ -0,0 +1,78 @@
+---
+toc: false
+title: Storage Client Summary
+---
+
+```js
+import { LineGraph } from '../components/line-graph.js'
+import { getDateXDaysAgo } from '../utils/date-utils.js'
+
+const data = FileAttachment(
path: src/client/[client].md
user: login: pyropy
body: Trying to load client page using the preview URL doesn't really work for me. Have you maybe forgot to commit changes you've done to `observablehq.config.js`? For the (storage) provider page we had to extend `observablehq.config.js` to load list of all storage providers and build page for each of them during the build phase. 

<img width="1440" alt="Screenshot 2025-03-03 at 14 43 55" src="https://github.com/user-attachments/assets/14a28298-825a-454e-a255-b0e6dce36f5d" />

created_at: 2025-03-03T13:45:46Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,78 @@
+---
+toc: false
+title: Storage Client Summary
+---
+
+```js
+import { LineGraph } from '../components/line-graph.js'
+import { getDateXDaysAgo } from '../utils/date-utils.js'
+
+const data = FileAttachment(
path: src/client/[client].md
user: login: pyropy
body: I have tried both Firefox and Safari and neither work (screenshot above is from Brave).
created_at: 2025-03-03T13:46:45Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,78 @@
+---
+toc: false
+title: Storage Client Summary
+---
+
+```js
+import { LineGraph } from '../components/line-graph.js'
+import { getDateXDaysAgo } from '../utils/date-utils.js'
+
+const data = FileAttachment(
+  `../data/${observable.params.client}-spark-client-rsr-summary.json`,
+).json()
+```
+
+<div class="hero">
+  <body><a href="/"><img src="../media/spark-logomark-blue-with-bbox.png" alt="Spark Logo" width="300" /></a><body>
+    <h2>Dashboard Beta</h2>
+    <body><a href="https://filspark.com/dashboard" target="_blank" rel="noopener noreferrer">(Click here for Legacy Spark Grafana Dashboard)</a><body>
+</div>
+
+<h4>Storage Client Spark RSR Summary</h4>
+<body>This section shows the storage client Spark Retrieval Success Rate Score summary. You can adjust the date range. Records start on the 25th February 2025.</body>
+
+```js
+const start = view(Inputs.date({ label: 'Start', value: getDateXDaysAgo(180) }))
+const end = view(Inputs.date({ label: 'End', value: getDateXDaysAgo(1) }))
+```
+
+<h3>Stats for ${observable.params.client}</h3>
+
+<div class="grid grid-cols" style="grid-auto-rows: 500px;">
+  <div class="card">${
+    resize((width) => LineGraph(data, {width, title: "Retrieval Success Rate", start, end }))
+  }</div>
+</div>
path: src/client/[client].md
user: login: bajtos
body: Can you please confirm this will show success rates for both "HTTP only" and "HTTP or Graphsync", using the same colours as the per-miner chart?

See e.g. https://nhaimerl-per-client-stats-ta.spark-dashboard.pages.dev/provider/f03252730
created_at: 2025-03-03T13:54:06Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,78 @@
+---
+toc: false
+title: Storage Client Summary
+---
+
+```js
+import { LineGraph } from '../components/line-graph.js'
+import { getDateXDaysAgo } from '../utils/date-utils.js'
+
+const data = FileAttachment(
+  `../data/${observable.params.client}-spark-client-rsr-summary.json`,
+).json()
+```
+
+<div class="hero">
+  <body><a href="/"><img src="../media/spark-logomark-blue-with-bbox.png" alt="Spark Logo" width="300" /></a><body>
+    <h2>Dashboard Beta</h2>
+    <body><a href="https://filspark.com/dashboard" target="_blank" rel="noopener noreferrer">(Click here for Legacy Spark Grafana Dashboard)</a><body>
+</div>
+
+<h4>Storage Client Spark RSR Summary</h4>
+<body>This section shows the storage client Spark Retrieval Success Rate Score summary. You can adjust the date range. Records start on the 25th February 2025.</body>
+
+```js
+const start = view(Inputs.date({ label: 'Start', value: getDateXDaysAgo(180) }))
+const end = view(Inputs.date({ label: 'End', value: getDateXDaysAgo(1) }))
+```
+
+<h3>Stats for ${observable.params.client}</h3>
+
+<div class="grid grid-cols" style="grid-auto-rows: 500px;">
+  <div class="card">${
+    resize((width) => LineGraph(data, {width, title: "Retrieval Success Rate", start, end }))
+  }</div>
+</div>
path: src/client/[client].md
user: login: NikolasHaimerl
body: Yes, I inserted a screenshot from the rsr per client page [here](https://github.com/CheckerNetwork/spark-dashboard/pull/28#issuecomment-2694210018). There you can see that it is the same color as the per miner chart. The function that is called is the same `LineGraph(data, {...`, so I would expect both plots to have the same layout and format. 
created_at: 2025-03-03T15:08:09Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,78 @@
+---
+toc: false
+title: Storage Client Summary
+---
+
+```js
+import { LineGraph } from '../components/line-graph.js'
+import { getDateXDaysAgo } from '../utils/date-utils.js'
+
+const data = FileAttachment(
path: src/client/[client].md
user: login: NikolasHaimerl
body: Hm interesting, when running it locally with `npm run dev` the graph load just fine. 

> Have you maybe forgot to commit changes you've done to observablehq.config.js? For the (storage) provider page we had to extend observablehq.config.js to load list of all storage providers and build page for each of them during the build phase.

Could you explain what you meant by that? 
created_at: 2025-03-03T15:09:27Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,78 @@
+---
+toc: false
+title: Storage Client Summary
+---
+
+```js
+import { LineGraph } from '../components/line-graph.js'
+import { getDateXDaysAgo } from '../utils/date-utils.js'
+
+const data = FileAttachment(
path: src/client/[client].md
user: login: pyropy
body: > Hm interesting, when running it locally with `npm run dev` the graph load just fine.

Observable can be a bit deceiving, something that works in development may not be always work in the production mode. I had the same issues in #9 

I've also tested  (storage) provider paths by removing `dynamicPaths` propery from `observablehq.config.js` and running first in development mode and then building the page (a.k.a. production mode). In development mode it worked perfectly, but when I tried to access the built version I got this as a response: 

![image](https://github.com/user-attachments/assets/b6b7c918-076d-4869-8a0d-b28dd339e7ba)

> Could you explain what you meant by that?

Sadly Observable lacks good documentation on this; If you want to have a dynamic page (like [client].md) you need to define list of dynamic paths in [observablehq.config.js](https://github.com/CheckerNetwork/spark-dashboard/blob/main/observablehq.config.js). During the build Observable will build each page in alist defined under the `dynamicPath` property. We have already done this for [storage providers](https://github.com/CheckerNetwork/spark-dashboard/blob/fbb8ef7997d4d9d33f254f872007de85dddd2b09/observablehq.config.js#L10).

So generally speaking `observablehq.config.js` should look something like this:


```javascript
import { jsonFetcher } from './src/data/json-fetcher.js'
import { getDateXDaysAgo } from './src/utils/date-utils.js'

const start = '2024-04-07'
const end = getDateXDaysAgo(1)

const providerSummary = await jsonFetcher(
  `https://stats.filspark.com/miners/retrieval-success-rate/summary?from=${start}&to=${end}`,
)
const clientsSummary= await jsonFetcher(
  `https://stats.filspark.com/clients/retrieval-success-rate/summary?from=${start}&to=${end}`,
)

const providerPaths = providerSummary .map((provider) => `/provider/${provider.miner_id}`)
const clientPaths = clientsSummary.map((client) => `/client/${client.client_id}`)


// See https://observablehq.com/framework/config for documentation.
export default {
  // omitted for clarity 
  dynamicPaths: [...providerPaths, ...clientPaths],
}
```

created_at: 2025-03-03T17:20:50Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,78 @@
+---
+toc: false
+title: Storage Client Summary
+---
+
+```js
+import { LineGraph } from '../components/line-graph.js'
+import { getDateXDaysAgo } from '../utils/date-utils.js'
+
+const data = FileAttachment(
path: src/client/[client].md
user: login: NikolasHaimerl
body: Thanks for your help! 
Seems like the build is failing. Both `observable build` and `observable preview` work locally and I can see the client builds being generated. Maybe once I get access to cloudflare I can debug this further. 
created_at: 2025-03-04T09:16:43Z
author_association: CONTRIBUTOR
diff_hunk: @@ -318,15 +326,32 @@ ${JSON.stringify(
 <body>The following table shows the Spark RSR and TTFB values calculated in aggregate for each Filecoin Storage Provider over the past 30 days. Click on a miner id to view stats about this storage provider.</body>
 
 ```js
-const search = view(
+const searchMinerStats = view(
   Inputs.search(tidySparkMinerRates, {
     placeholder: 'Search Storage Providers…',
   }),
 )
 ```
 
 <div class="card" style="padding: 0;">
-  ${Inputs.table(search, {rows: 16, format: {miner_id: id => htl.html`<a href=./provider/${id} target=_blank>${id}</a>`}})}
+  ${Inputs.table(searchMinerStats, {rows: 16, format: {miner_id: id => htl.html`<a href=./provider/${id} target=_blank>${id}</a>`}})}
+</div>
+
+<div class="divider"></div>
+
+<h4>Spark Client RSR Table</h4>
+<body>The following table shows the Spark RSR values calculated in aggregate for each Filecoin Storage Client over the past 30 days. Click on a client id to view stats about this storage provider.</body>
path: src/index.md
user: login: pyropy
body: ```suggestion
<body>The following table shows the Spark RSR values calculated in aggregate for each Filecoin Storage Client over the past 30 days. Click on a client id to view stats about this storage client.</body>
```
created_at: 2025-03-05T14:46:18Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,78 @@
+---
+toc: false
+title: Storage Client Summary
+---
+
+```js
+import { LineGraph } from '../components/line-graph.js'
+import { getDateXDaysAgo } from '../utils/date-utils.js'
+
+const data = FileAttachment(
+  `../data/${observable.params.client}-spark-client-rsr-summary.json`,
+).json()
+```
+
+<div class="hero">
+  <body><a href="/"><img src="../media/spark-logomark-blue-with-bbox.png" alt="Spark Logo" width="300" /></a><body>
+    <h2>Dashboard Beta</h2>
+    <body><a href="https://filspark.com/dashboard" target="_blank" rel="noopener noreferrer">(Click here for Legacy Spark Grafana Dashboard)</a><body>
+</div>
+
+<h4>Storage Client Spark RSR Summary</h4>
+<body>This section shows the storage client Spark Retrieval Success Rate Score summary. You can adjust the date range. Records start on the 25th February 2025.</body>
+
+```js
+const start = view(Inputs.date({ label: 'Start', value: getDateXDaysAgo(180) }))
path: src/client/[client].md
user: login: pyropy
body: ```suggestion
const start = view(Inputs.date({ label: 'Start', value: '2025-02-25') }))
```

I propose that by default we set start value to `25th February` as noted by paragraph above: "Records start on the 25th February 2025."
created_at: 2025-03-05T14:48:24Z
author_association: MEMBER
diff_hunk: @@ -3,7 +3,7 @@ import { getDateXDaysAgo } from '../utils/date-utils.js'
 
 const summaries = {}
 
-for (let i = 201; i >= 1; i--) {
+for (let i = 100; i >= 1; i--) {
path: src/data/spark-miner-rsr-summaries.json.js
user: login: pyropy
body: Great job on solving this 👏🏻 
created_at: 2025-03-05T14:48:48Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,78 @@
+---
+toc: false
+title: Storage Client Summary
+---
+
+```js
+import { LineGraph } from '../components/line-graph.js'
+import { getDateXDaysAgo } from '../utils/date-utils.js'
+
+const data = FileAttachment(
+  `../data/${observable.params.client}-spark-client-rsr-summary.json`,
+).json()
+```
+
+<div class="hero">
+  <body><a href="/"><img src="../media/spark-logomark-blue-with-bbox.png" alt="Spark Logo" width="300" /></a><body>
+    <h2>Dashboard Beta</h2>
+    <body><a href="https://filspark.com/dashboard" target="_blank" rel="noopener noreferrer">(Click here for Legacy Spark Grafana Dashboard)</a><body>
+</div>
+
+<h4>Storage Client Spark RSR Summary</h4>
+<body>This section shows the storage client Spark Retrieval Success Rate Score summary. You can adjust the date range. Records start on the 25th February 2025.</body>
+
+```js
+const start = view(Inputs.date({ label: 'Start', value: getDateXDaysAgo(180) }))
path: src/client/[client].md
user: login: NikolasHaimerl
body: Good point let's do somethig like this:
```suggestion
const startDate = getDateXDaysAgo(180)
const minStartDate = '2025-02-25'
const start = view(Inputs.date({ label: 'Start', value: new Date(startDate) >= new Date(minStartDate) ? startDate : minStartDate }))
```
created_at: 2025-03-06T08:10:19Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,78 @@
+---
+toc: false
+title: Storage Client Summary
+---
+
+```js
+import { LineGraph } from '../components/line-graph.js'
+import { getDateXDaysAgo } from '../utils/date-utils.js'
+
+const data = FileAttachment(
+  `../data/${observable.params.client}-spark-client-rsr-summary.json`,
+).json()
+```
+
+<div class="hero">
+  <body><a href="/"><img src="../media/spark-logomark-blue-with-bbox.png" alt="Spark Logo" width="300" /></a><body>
+    <h2>Dashboard Beta</h2>
+    <body><a href="https://filspark.com/dashboard" target="_blank" rel="noopener noreferrer">(Click here for Legacy Spark Grafana Dashboard)</a><body>
+</div>
+
+<h4>Storage Client Spark RSR Summary</h4>
+<body>This section shows the storage client Spark Retrieval Success Rate Score summary. You can adjust the date range. Records start on the 25th February 2025.</body>
+
+```js
+const start = view(Inputs.date({ label: 'Start', value: getDateXDaysAgo(180) }))
path: src/client/[client].md
user: login: pyropy
body: @NikolasHaimerl that looks great! 
created_at: 2025-03-06T08:32:18Z
author_association: MEMBER

PR comments:
user: login: cloudflare-workers-and-pages[bot]
created_at: 2025-02-25T12:30:38Z
body: ## Deploying spark-dashboard with &nbsp;<a href="https://pages.dev"><img alt="Cloudflare Pages" src="https://user-images.githubusercontent.com/23264/106598434-9e719e00-654f-11eb-9e59-6167043cfa01.png" width="16"></a> &nbsp;Cloudflare Pages

<table><tr><td><strong>Latest commit:</strong> </td><td>
<code>87b7675</code>
</td></tr>
<tr><td><strong>Status:</strong></td><td>&nbsp;✅&nbsp; Deploy successful!</td></tr>
<tr><td><strong>Preview URL:</strong></td><td>
<a href='https://ecd9516b.spark-dashboard.pages.dev'>https://ecd9516b.spark-dashboard.pages.dev</a>
</td></tr>
<tr><td><strong>Branch Preview URL:</strong></td><td>
<a href='https://nhaimerl-per-client-stats-ta.spark-dashboard.pages.dev'>https://nhaimerl-per-client-stats-ta.spark-dashboard.pages.dev</a>
</td></tr>
</table>

[View logs](https://dash.cloudflare.com/37573110e38849a343d93b727953188f/pages/view/spark-dashboard/ecd9516b-ed2b-4944-8dea-46c10c2faf4f)

author_association: NONE
user: login: NikolasHaimerl
created_at: 2025-02-25T12:34:59Z
body: Waiting for opening PR for review until the spark API is ready to provide data. See: https://stats.filspark.com/clients/retrieval-success-rate/summary?from=2025-02-25&to=2025-02-25
author_association: CONTRIBUTOR
user: login: NikolasHaimerl
created_at: 2025-03-03T12:28:53Z
body: > Waiting for opening PR for review until the spark API is ready to provide data. See: https://stats.filspark.com/clients/retrieval-success-rate/summary?from=2025-02-25&to=2025-02-25

The data per client fills up slowly and asymmetrically per client. 
An example of how the per client scores look like once there is enough data for the graph to form is given here.

![image](https://github.com/user-attachments/assets/ecdf9a6d-9729-4e35-893e-0e2db560239c)

author_association: CONTRIBUTOR
user: login: bajtos
created_at: 2025-03-03T13:47:54Z
body: @NikolasHaimerl The per-client page does not load the styles, could you please take a look?

https://nhaimerl-per-client-stats-ta.spark-dashboard.pages.dev/client/f03319412

<img width="1316" alt="Screenshot 2025-03-03 at 14 47 35" src="https://github.com/user-attachments/assets/045d224b-2b13-4322-9597-e42cb70845a2" />
author_association: MEMBER
user: login: NikolasHaimerl
created_at: 2025-03-04T12:37:31Z
body: @bajtos @pyropy it seems that even when changing the date range which is fetched for the miner summary, we still get a file that is larger than 25MiB. 

> 12:55:40.907 | ✘ [ERROR] Error: Pages only supports files up to 25 MiB in size
-- | --
12:55:40.907 |  
12:55:40.907 | _file/data/spark-miner-rsr-summaries.bbc3288d.json is 27.3 MiB in size


I did not see an option to increase the size for files in cloudflare do you perhaps know more? 
author_association: CONTRIBUTOR
user: login: juliangruber
created_at: 2025-03-05T10:35:19Z
body: > @bajtos @pyropy it seems that even when changing the date range which is fetched for the miner summary, we still get a file that is larger than 25MiB.
> 
> > 12:55:40.907 | ✘ [ERROR] Error: Pages only supports files up to 25 MiB in size
> > -- | --
> > 12:55:40.907 |  
> > 12:55:40.907 | _file/data/spark-miner-rsr-summaries.bbc3288d.json is 27.3 MiB in size
> 
> I did not see an option to increase the size for files in cloudflare do you perhaps know more?

Question is also whether we want to send >25MB over the wire for the dashboard. Mobile users will have a bad time
author_association: MEMBER
user: login: pyropy
created_at: 2025-03-05T11:14:49Z
body: > > @bajtos @pyropy it seems that even when changing the date range which is fetched for the miner summary, we still get a file that is larger than 25MiB.
> > > 12:55:40.907
> > > ✘ [ERROR] Error: Pages only supports files up to 25 MiB in size
> > > 
> > > 
> > > 
> > > 
> > > 12:55:40.907
> > >  
> > > 
> > > 
> > > 12:55:40.907
> > > _file/data/spark-miner-rsr-summaries.bbc3288d.json is 27.3 MiB in size
> > 
> > 
> > I did not see an option to increase the size for files in cloudflare do you perhaps know more?
> 
> Question is also whether we want to send >25MB over the wire for the dashboard. Mobile users will have a bad time

That's a fair question. I wonder if paginating the table would solve the issue? I found some example on [paginating the table in Observable](https://observablehq.com/@tmcw/tables/2), maybe we could work something out based on that?

Otherwise I'm afraid we'd have to remove the table and add some global search for both storage providers and clients.
author_association: MEMBER
user: login: juliangruber
created_at: 2025-03-05T11:21:21Z
body: @pyropy can you tell how https://filecoin-project.github.io/filecoin-storage-providers-market/ / https://github.com/filecoin-project/filecoin-storage-providers-market solved this issue? This project serves as good inspiration for us
author_association: MEMBER
user: login: juliangruber
created_at: 2025-03-05T12:36:57Z
body: Let's take a step back on the 25mb issue. Does it even make sense to visualize 25mb worth of data in a table?
author_association: MEMBER
user: login: NikolasHaimerl
created_at: 2025-03-05T12:56:16Z
body: @pyropy I think I found the issue, the [summary](https://github.com/CheckerNetwork/spark-dashboard/blob/fbb8ef7997d4d9d33f254f872007de85dddd2b09/src/data/spark-miner-rsr-summaries.json.js#L6) which goes back 201 days has become too large. I decreased it in eca1f49 to the past 100 days and now it seems to be just fine :) 
author_association: CONTRIBUTOR
user: login: NikolasHaimerl
created_at: 2025-03-06T11:21:48Z
body: > Let's take a step back on the 25mb issue. Does it even make sense to visualize 25mb worth of data in a table?

We found the issue and reduced the number of days that we fetch the miner rsr summary for, see [here](https://github.com/CheckerNetwork/spark-dashboard/pull/28#issuecomment-2700856778). 
author_association: CONTRIBUTOR
user: login: bajtos
created_at: 2025-03-06T15:38:21Z
body: @NikolasHaimerl can you please resolve the merge conflicts?

<img width="837" alt="Screenshot 2025-03-06 at 16 37 53" src="https://github.com/user-attachments/assets/a7ca2ab0-845a-48fd-a00a-84993d7c56e5" />

author_association: MEMBER