Repository: pyth-network/pyth-crosschain
Pull request #2619: feat(metrics): add price update delay metric and update method
Author: login: cctdaniel
Created: 2025-04-25T06:18:21Z | Merged: 2025-04-29T02:30:27Z

## Summary

Added a new gauge metric `pyth_price_update_delay` to track the delay between source and target timestamps relative to each price feed's configured threshold. The metric is calculated as:
```
sourceLatestPricePublishTime - targetLatestPricePublishTime - priceConfigTimeDifference
```

## Rationale

- Each price feed has its own `timeDifference` configuration, making static thresholds suboptimal
- Enables dynamic alerting based on per-feed configuration rather than hardcoded values
- Handles market hours gracefully - during off-hours, `sourceLatestPrice` won't update, keeping the delay constant
- Allows setting unified alerts (e.g. `delay > 2min`) while respecting individual feed configurations
- Simplifies monitoring by normalizing delays against their expected thresholds

## How has this been tested?

- [x] Current tests cover my changes
- [ ] Added new tests
- [x] Manually tested the code

Steps to verify:
1. Deployed to testnet and confirmed metric appears in Prometheus
2. Verified calculation is correct by comparing:
   - `sourceLatestPrice.publishTime - targetLatestPrice.publishTime` against raw timestamps
   - Final gauge value accounts for `priceConfigTimeDifference`
3. Checked behavior during market hours and off-hours
4. Confirmed alerts can be set using simple thresholds that work across all feeds

The changes are non-invasive and purely additive to the metrics system, with no impact on core price pushing functionality.



Diff:
diff --git a/apps/price_pusher/package.json b/apps/price_pusher/package.json
index 19b3740a3f..de10c99b1d 100644
--- a/apps/price_pusher/package.json
+++ b/apps/price_pusher/package.json
@@ -1,6 +1,6 @@
 {
   "name": "@pythnetwork/price-pusher",
-  "version": "9.3.0",
+  "version": "9.3.1",
   "description": "Pyth Price Pusher",
   "homepage": "https://pyth.network",
   "main": "lib/index.js",
diff --git a/apps/price_pusher/src/controller.ts b/apps/price_pusher/src/controller.ts
index 787c19b036..ffedd414f0 100644
--- a/apps/price_pusher/src/controller.ts
+++ b/apps/price_pusher/src/controller.ts
@@ -53,12 +53,13 @@ export class Controller {
         const sourceLatestPrice =
           this.sourcePriceListener.getLatestPriceInfo(priceId);
 
-        // Update metrics for the last published time if available
-        if (this.metrics && targetLatestPrice) {
-          this.metrics.updateLastPublishedTime(
+        if (this.metrics && targetLatestPrice && sourceLatestPrice) {
+          this.metrics.updateTimestamps(
             priceId,
             alias,
-            targetLatestPrice,
+            targetLatestPrice.publishTime,
+            sourceLatestPrice.publishTime,
+            priceConfig.timeDifference,
           );
         }
 
diff --git a/apps/price_pusher/src/metrics.ts b/apps/price_pusher/src/metrics.ts
index 0090f717dd..700c6e979b 100644
--- a/apps/price_pusher/src/metrics.ts
+++ b/apps/price_pusher/src/metrics.ts
@@ -1,6 +1,5 @@
 import { Registry, Counter, Gauge } from "prom-client";
 import express from "express";
-import { PriceInfo } from "./interface";
 import { Logger } from "pino";
 import { UpdateCondition } from "./price-config";
 
@@ -14,6 +13,8 @@ export class PricePusherMetrics {
   public lastPublishedTime: Gauge<string>;
   public priceUpdateAttempts: Counter<string>;
   public priceFeedsTotal: Gauge<string>;
+  public sourceTimestamp: Gauge<string>;
+  public configuredTimeDifference: Gauge<string>;
   // Wallet metrics
   public walletBalance: Gauge<string>;
 
@@ -46,6 +47,20 @@ export class PricePusherMetrics {
       registers: [this.registry],
     });
 
+    this.sourceTimestamp = new Gauge({
+      name: "pyth_source_timestamp",
+      help: "Latest source chain price publish timestamp",
+      labelNames: ["price_id", "alias"],
+      registers: [this.registry],
+    });
+
+    this.configuredTimeDifference = new Gauge({
+      name: "pyth_configured_time_difference",
+      help: "Configured time difference threshold between source and target chains",
+      labelNames: ["price_id", "alias"],
+      registers: [this.registry],
+    });
+
     // Wallet balance metric
     this.walletBalance = new Gauge({
       name: "pyth_wallet_balance",
@@ -68,18 +83,6 @@ export class PricePusherMetrics {
     });
   }
 
-  // Update the last published time for a price feed
-  public updateLastPublishedTime(
-    priceId: string,
-    alias: string,
-    priceInfo: PriceInfo,
-  ): void {
-    this.lastPublishedTime.set(
-      { price_id: priceId, alias },
-      priceInfo.publishTime,
-    );
-  }
-
   // Record a successful price update
   public recordPriceUpdate(
     priceId: string,
@@ -133,6 +136,28 @@ export class PricePusherMetrics {
     this.priceFeedsTotal.set(count);
   }
 
+  // Update source, target and configured time difference timestamps
+  public updateTimestamps(
+    priceId: string,
+    alias: string,
+    targetLatestPricePublishTime: number,
+    sourceLatestPricePublishTime: number,
+    priceConfigTimeDifference: number,
+  ): void {
+    this.sourceTimestamp.set(
+      { price_id: priceId, alias },
+      sourceLatestPricePublishTime,
+    );
+    this.lastPublishedTime.set(
+      { price_id: priceId, alias },
+      targetLatestPricePublishTime,
+    );
+    this.configuredTimeDifference.set(
+      { price_id: priceId, alias },
+      priceConfigTimeDifference,
+    );
+  }
+
   // Update wallet balance
   public updateWalletBalance(
     walletAddress: string,


Reviews:
user: login: ali-behjati
body: Sorry for a lot of change of minds here, i was wondering whether we can expose the raw data behind `delay` here and calculate the `delay` in our metrics platform instead. In this way we can expose the details of the delay in the dashboards as well.
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-04-28T06:53:59Z
user: login: cctdaniel
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-28T09:13:36Z
user: login: ali-behjati
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-04-28T09:36:23Z
user: login: ali-behjati
state: APPROVED
author_association: COLLABORATOR
submitted_at: 2025-04-28T09:36:31Z
user: login: cctdaniel
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-28T09:47:14Z

Review comments:
diff_hunk: @@ -14,6 +14,7 @@ export class PricePusherMetrics {
   public lastPublishedTime: Gauge<string>;
   public priceUpdateAttempts: Counter<string>;
   public priceFeedsTotal: Gauge<string>;
+  public priceUpdateDelay: Gauge<string>;
path: apps/price_pusher/src/metrics.ts
user: login: ali-behjati
body: Out of curiosity, why are these string :?
created_at: 2025-04-28T06:52:25Z
author_association: COLLABORATOR
diff_hunk: @@ -14,6 +14,7 @@ export class PricePusherMetrics {
   public lastPublishedTime: Gauge<string>;
   public priceUpdateAttempts: Counter<string>;
   public priceFeedsTotal: Gauge<string>;
+  public priceUpdateDelay: Gauge<string>;
path: apps/price_pusher/src/metrics.ts
user: login: cctdaniel
body: these are for the label names
created_at: 2025-04-28T09:13:35Z
author_association: CONTRIBUTOR
diff_hunk: @@ -47,9 +49,23 @@ export class PricePusherMetrics {
       registers: [this.registry],
     });
 
-    this.priceUpdateDelay = new Gauge({
-      name: "pyth_price_update_delay",
-      help: "Delay between source and target timestamps relative to configured threshold (positive means over threshold)",
+    this.sourceTimestamp = new Gauge({
+      name: "pyth_source_timestamp",
+      help: "Latest source chain price publish timestamp",
+      labelNames: ["price_id", "alias"],
+      registers: [this.registry],
+    });
+
+    this.targetTimestamp = new Gauge({
path: apps/price_pusher/src/metrics.ts
user: login: ali-behjati
body: what's the difference between this and `lastPublishedTime` ?
created_at: 2025-04-28T09:36:23Z
author_association: COLLABORATOR
diff_hunk: @@ -47,9 +49,23 @@ export class PricePusherMetrics {
       registers: [this.registry],
     });
 
-    this.priceUpdateDelay = new Gauge({
-      name: "pyth_price_update_delay",
-      help: "Delay between source and target timestamps relative to configured threshold (positive means over threshold)",
+    this.sourceTimestamp = new Gauge({
+      name: "pyth_source_timestamp",
+      help: "Latest source chain price publish timestamp",
+      labelNames: ["price_id", "alias"],
+      registers: [this.registry],
+    });
+
+    this.targetTimestamp = new Gauge({
path: apps/price_pusher/src/metrics.ts
user: login: cctdaniel
body: yeah its redundant, i will remove `targetTimestamp` and use `lastPublishedTime` instead
created_at: 2025-04-28T09:47:13Z
author_association: CONTRIBUTOR

PR comments:
user: login: vercel[bot]
created_at: 2025-04-25T06:18:26Z
body: [vc]: #/U8B3TYAcOtObJRoaNP/6wHVS6qXG9SOQTcQlKux95w=:eyJpc01vbm9yZXBvIjp0cnVlLCJ0eXBlIjoiZ2l0aHViIiwicHJvamVjdHMiOlt7Im5hbWUiOiJwcm9wb3NhbHMiLCJyb290RGlyZWN0b3J5IjoiZ292ZXJuYW5jZS94Y19hZG1pbi9wYWNrYWdlcy94Y19hZG1pbl9mcm9udGVuZCIsImluc3BlY3RvclVybCI6Imh0dHBzOi8vdmVyY2VsLmNvbS9weXRoLW5ldHdvcmsvcHJvcG9zYWxzLzhMZDNMUXRtamVvU0drbXM0QVZNMnNTRFVQdUYiLCJwcmV2aWV3VXJsIjoicHJvcG9zYWxzLWdpdC1wcmljZS11cGRhdGUtZGVsYXktcHl0aC1uZXR3b3JrLnZlcmNlbC5hcHAiLCJuZXh0Q29tbWl0U3RhdHVzIjoiSUdOT1JFRCIsImxpdmVGZWVkYmFjayI6eyJyZXNvbHZlZCI6MCwidW5yZXNvbHZlZCI6MCwidG90YWwiOjAsImxpbmsiOiIifX0seyJuYW1lIjoiYXBpLXJlZmVyZW5jZSIsInJvb3REaXJlY3RvcnkiOiJhcHBzL2FwaS1yZWZlcmVuY2UiLCJpbnNwZWN0b3JVcmwiOiJodHRwczovL3ZlcmNlbC5jb20vcHl0aC1uZXR3b3JrL2FwaS1yZWZlcmVuY2UvTlBXcmFLZVR4d0o3V2ZNRmpMM1VVOUN5UXVOWCIsInByZXZpZXdVcmwiOiJhcGktcmVmZXJlbmNlLWdpdC1wcmljZS11cGRhdGUtZGVsYXktcHl0aC1uZXR3b3JrLnZlcmNlbC5hcHAiLCJuZXh0Q29tbWl0U3RhdHVzIjoiSUdOT1JFRCIsImxpdmVGZWVkYmFjayI6eyJyZXNvbHZlZCI6MCwidW5yZXNvbHZlZCI6MCwidG90YWwiOjAsImxpbmsiOiIifX0seyJuYW1lIjoic3Rha2luZyIsInJvb3REaXJlY3RvcnkiOiJhcHBzL3N0YWtpbmciLCJpbnNwZWN0b3JVcmwiOiJodHRwczovL3ZlcmNlbC5jb20vcHl0aC1uZXR3b3JrL3N0YWtpbmcvNjVEQzFTNVdOamVnUml6b2hTN0h4eTJXOHA3YiIsInByZXZpZXdVcmwiOiJzdGFraW5nLWdpdC1wcmljZS11cGRhdGUtZGVsYXktcHl0aC1uZXR3b3JrLnZlcmNlbC5hcHAiLCJuZXh0Q29tbWl0U3RhdHVzIjoiREVQTE9ZRUQiLCJsaXZlRmVlZGJhY2siOnsicmVzb2x2ZWQiOjAsInVucmVzb2x2ZWQiOjAsInRvdGFsIjowLCJsaW5rIjoic3Rha2luZy1naXQtcHJpY2UtdXBkYXRlLWRlbGF5LXB5dGgtbmV0d29yay52ZXJjZWwuYXBwIn19LHsibmFtZSI6ImNvbXBvbmVudC1saWJyYXJ5Iiwicm9vdERpcmVjdG9yeSI6InBhY2thZ2VzL2NvbXBvbmVudC1saWJyYXJ5IiwiaW5zcGVjdG9yVXJsIjoiaHR0cHM6Ly92ZXJjZWwuY29tL3B5dGgtbmV0d29yay9jb21wb25lbnQtbGlicmFyeS80cnFwRkJNZFVLSmdtcGlCWlBlaXljcFpITmRGIiwicHJldmlld1VybCI6ImNvbXBvbmVudC1saWJyYXJ5LWdpdC1wcmljZS11cGRhdGUtZGVsYXktcHl0aC1uZXR3b3JrLnZlcmNlbC5hcHAiLCJuZXh0Q29tbWl0U3RhdHVzIjoiSUdOT1JFRCIsImxpdmVGZWVkYmFjayI6eyJyZXNvbHZlZCI6MCwidW5yZXNvbHZlZCI6MCwidG90YWwiOjAsImxpbmsiOiIifX0seyJuYW1lIjoiaW5zaWdodHMiLCJyb290RGlyZWN0b3J5IjoiYXBwcy9pbnNpZ2h0cyIsImluc3BlY3RvclVybCI6Imh0dHBzOi8vdmVyY2VsLmNvbS9weXRoLW5ldHdvcmsvaW5zaWdodHMvNkZMQVVtSm1MdjRyVXpCVTZ5b3IzZzk4endTeCIsInByZXZpZXdVcmwiOiJpbnNpZ2h0cy1naXQtcHJpY2UtdXBkYXRlLWRlbGF5LXB5dGgtbmV0d29yay52ZXJjZWwuYXBwIiwibmV4dENvbW1pdFN0YXR1cyI6IkRFUExPWUVEIiwibGl2ZUZlZWRiYWNrIjp7InJlc29sdmVkIjowLCJ1bnJlc29sdmVkIjowLCJ0b3RhbCI6MCwibGluayI6Imluc2lnaHRzLWdpdC1wcmljZS11cGRhdGUtZGVsYXktcHl0aC1uZXR3b3JrLnZlcmNlbC5hcHAifX0seyJuYW1lIjoiZW50cm9weS1kZWJ1Z2dlciIsInJvb3REaXJlY3RvcnkiOiJhcHBzL2VudHJvcHktZGVidWdnZXIiLCJpbnNwZWN0b3JVcmwiOiJodHRwczovL3ZlcmNlbC5jb20vcHl0aC1uZXR3b3JrL2VudHJvcHktZGVidWdnZXIvMkNUNzVUTGJ4dW9qSnpEYWFMbXZuYUxBcWRlcCIsInByZXZpZXdVcmwiOiJlbnRyb3B5LWRlYnVnZ2VyLWdpdC1wcmljZS11cGRhdGUtZGVsYXktcHl0aC1uZXR3b3JrLnZlcmNlbC5hcHAiLCJuZXh0Q29tbWl0U3RhdHVzIjoiREVQTE9ZRUQiLCJsaXZlRmVlZGJhY2siOnsicmVzb2x2ZWQiOjAsInVucmVzb2x2ZWQiOjAsInRvdGFsIjowLCJsaW5rIjoiZW50cm9weS1kZWJ1Z2dlci1naXQtcHJpY2UtdXBkYXRlLWRlbGF5LXB5dGgtbmV0d29yay52ZXJjZWwuYXBwIn19XX0=
**The latest updates on your projects**. Learn more about [Vercel for Git ↗︎](https://vercel.link/github-learn-more)

| Name | Status | Preview | Comments | Updated (UTC) |
| :--- | :----- | :------ | :------- | :------ |
| **entropy-debugger** | ✅ Ready ([Inspect](https://vercel.com/pyth-network/entropy-debugger/2CT75TLbxuojJzDaaLmvnaLAqdep)) | [Visit Preview](https://entropy-debugger-git-price-update-delay-pyth-network.vercel.app) | 💬 [**Add feedback**](https://vercel.live/open-feedback/entropy-debugger-git-price-update-delay-pyth-network.vercel.app?via=pr-comment-feedback-link) | Apr 28, 2025 9:59am |
| **insights** | ✅ Ready ([Inspect](https://vercel.com/pyth-network/insights/6FLAUmJmLv4rUzBU6yor3g98zwSx)) | [Visit Preview](https://insights-git-price-update-delay-pyth-network.vercel.app) | 💬 [**Add feedback**](https://vercel.live/open-feedback/insights-git-price-update-delay-pyth-network.vercel.app?via=pr-comment-feedback-link) | Apr 28, 2025 9:59am |
| **staking** | ✅ Ready ([Inspect](https://vercel.com/pyth-network/staking/65DC1S5WNjegRizohS7Hxy2W8p7b)) | [Visit Preview](https://staking-git-price-update-delay-pyth-network.vercel.app) | 💬 [**Add feedback**](https://vercel.live/open-feedback/staking-git-price-update-delay-pyth-network.vercel.app?via=pr-comment-feedback-link) | Apr 28, 2025 9:59am |

<details><summary>3 Skipped Deployments</summary>

| Name | Status | Preview | Comments | Updated (UTC) |
| :--- | :----- | :------ | :------- | :------ |
| **api-reference** | ⬜️ Ignored ([Inspect](https://vercel.com/pyth-network/api-reference/NPWraKeTxwJ7WfMFjL3UU9CyQuNX)) | [Visit Preview](https://api-reference-git-price-update-delay-pyth-network.vercel.app) |  | Apr 28, 2025 9:59am |
| **component-library** | ⬜️ Ignored ([Inspect](https://vercel.com/pyth-network/component-library/4rqpFBMdUKJgmpiBZPeiycpZHNdF)) | [Visit Preview](https://component-library-git-price-update-delay-pyth-network.vercel.app) |  | Apr 28, 2025 9:59am |
| **proposals** | ⬜️ Ignored ([Inspect](https://vercel.com/pyth-network/proposals/8Ld3LQtmjeoSGkms4AVM2sSDUPuF)) | [Visit Preview](https://proposals-git-price-update-delay-pyth-network.vercel.app) |  | Apr 28, 2025 9:59am |
</details>







author_association: NONE
user: login: ali-behjati
created_at: 2025-04-28T09:36:41Z
body: Ah please bump the version too.
author_association: COLLABORATOR