Repository: lightninglabs/taproot-assets (go)

Issue:
[bug]: Mint Not Showing as Confirmed Despite 6+ confirmations
### Background

We've had 2 reports of mint transactions having 6+ on-chain confirmations, but not being recognized as confirmed by the tapd daemon.  

### Your environment

Both are on the latest, non-rc, version of litd (v14.1) running in integrated mode with btcd backend. One instance occurred in testnet and one instance occurred in mainnet. 

### Steps to reproduce

Unknown at this stage. 

### Expected behavior

Once a mint transaction is confirmed on-chain, it should show up as confirmed within the tapd daemon for `tapcli assets list`

### Actual behavior

*Instance 1: Testnet* 
 Minted some assets on testnet. The onchain transactions have many confirmations, however when I do tapcli assets list it shows them both as unconfirmed. Why is that?
```
    "assets":  [],
    "unconfirmed_transfers":  "0",
    "unconfirmed_mints":  "2" 
```

I minted an asset with `--new_grouped_asset` , finalized it and this worked. It created an onchain transaction.
I think immediately tried to mint another asset by specifying the group key created in this first mint. It failed because I had no available UTXOs for minting which makes sense. 
This batch switched to 
```
BATCH_STATE_FROZEN  error messsage:
[tapcli] unable to finalize batch: rpc error: code = Unknown desc = unable to finalize batch: unable to finalize minting batch: unable to fund minting PSBT for batch: 0225a910dd5be980022b7ec39bd8118581d052985f4b562dc6311d0cdf5b3f93fb unable to fund psbt: unable to fund psbt: rpc error: code = Unknown desc = error selecting coins: not enough witness outputs to create funding transaction, need 0.00001000 BTC only have 0 BTC available
```

I waited for the first mint to get confirmed and then once it was confirmed I minted a second asset with the group key no problem. This worked and published just fine. So at this point we have 2 Batches with `BATCH_STATE_BROADCAST` and one with `BATCH_STATE_FROZEN`.

Even after many onchain confirmations, the 2 successful batches never moved to `BATCH_STATE_FINALIZED`. I only had 2 unconfirmed mints. I restarted the node to try to get it to kick in. After a restart, that Frozen batch actually got broadcasted (which was unexpected) and confirmed. That batch now shows as `BATCH_STATE_FINALIZED` , but the two successful ones are still at `BATCH_STATE_BROADCAST` . I don’t understand why.
Additionally I see this in my logs:
```
[WRN] UNIV: Cannot push proof via logged server push: cannot push proof to remote server(testnet.universe.lightning.finance:10029): cannot push proof to remote server(testnet.universe.lightning.finance:10029): rpc error: code = Unknown desc = unable to verify proof: unable to verify proof (issuance-f538d95ef3dc5e750275e7ea3f2c1af3342d024ba4b9ed230365ce0ed14620e5 (asset_id=0000000000000000000000000000000000000000000000000000000000000000, group_key=3127efed588df7cac5e5d8df2e6a1f1b88a07f742ed4bffb2bb923f3bf922c2e, proof_type=issuance), outpoint=60723e7fe5d2e4be3b12f737343aecc087b8babcba0a74937e6c562f6fbf360b:0, scriptKey=027d21392ac9c0421517355bc119ead8c9f072be156334777d9cde79f7e77af8b1): group key not known: 023127efed588df7cac5e5d8df2e6a1f1b88a07f742ed4bffb2bb923f3bf922c2e: group verifier: no matching asset group: sql: no rows in result set: group key not known
```

Tried again with a brand new node and only 1 batch. It is still stuck at `BATCH_STATE_BROADCAST` even with many confirmations. Here are logs for the second attempt (second node): 

```
2025-05-08 10:54:41	
2025-05-08 15:54:41.382 [INF] UNIV: Attempting to sync universe: host=testnet.universe.lightning.finance:10029, sync_type=full, ids=([]universe.Identifier) <nil>
2025-05-08 10:54:41	
2025-05-08 15:54:41.382 [INF] UNIV: Syncing Universe state with server=testnet.universe.lightning.finance:10029
2025-05-08 10:54:41	
2025-05-08 15:54:41.382 [INF] UNIV: Synchronizing with 1 federation members
2025-05-08 10:54:28	
2025-05-08 15:54:28.389 [INF] LNWL: Marking unconfirmed transaction ac6cf8ada16e0d6c7eabef767713a1f033609a6a6c47da89b0ebf2f18e239021 mined in block 4325420
2025-05-08 10:54:27	
2025-05-08 15:54:27.078 [INF] GRPH: Block 00000000009f47287a9cad804392b1b6d0717944167c4607d16e051d65810068 (height=4325420) closed 0 channels
2025-05-08 10:54:27	
2025-05-08 15:54:27.070 [INF] GRPH: Pruning channel graph using block 00000000009f47287a9cad804392b1b6d0717944167c4607d16e051d65810068 (height=4325420)
2025-05-08 10:54:27	
2025-05-08 15:54:27.028 [ERR] GRDN: unable to advance state machine: unable to construct minting proofs: invalid proof file generated: failed to validate proof block header: unable to retrieve block hash: rpc error: code = Unknown desc = -1: Block number out of range
2025-05-08 10:54:27	
2025-05-08 15:54:27.028 [ERR] RPCS: [/chainrpc.ChainKit/GetBlockHash]: -1: Block number out of range
2025-05-08 10:54:27	
2025-05-08 15:54:27.021 [INF] GRDN: BatchCaretaker(037ffbe2ee1316dd27f1873ec3426d7748046e1427f1b32a47925bd4188251043b), advancing from state=BatchStateConfirmed to state=BatchStateFinalized
2025-05-08 10:54:27	
2025-05-08 15:54:27.021 [INF] GRDN: MintingBatch(037ffbe2ee1316dd27f1873ec3426d7748046e1427f1b32a47925bd4188251043b): confirmed at block(hash=00000000009f47287a9cad804392b1b6d0717944167c4607d16e051d65810068, height=4325420)
2025-05-08 10:54:27	
2025-05-08 15:54:27.008 [INF] GRDN: New block at height 4325420
2025-05-08 10:54:27	
2025-05-08 15:54:27.008 [INF] RPCS: Received new block notification: height=4325420
2025-05-08 10:54:27	
2025-05-08 15:54:27.008 [INF] UTXN: Attempting to graduate height=4325420: num_kids=0, num_babies=0
2025-05-08 10:54:27	
2025-05-08 15:54:27.008 [INF] LNWL: Web API returning 1012 sat/kvb for min relay feerate
2025-05-08 10:54:27	
2025-05-08 15:54:27.007 [INF] NTFN: New block: height=4325420, sha=00000000009f47287a9cad804392b1b6d0717944167c4607d16e051d65810068
2025-05-08 10:54:13	
2025-05-08 15:54:13.717 [INF] GRPH: Processed channels=0 updates=0 nodes=6 in last 2m30.002032922s
2025-05-08 10:50:43	
2025-05-08 15:50:43.715 [INF] GRPH: Processed channels=0 updates=0 nodes=2 in last 59.999667454s
2025-05-08 10:49:43	
2025-05-08 15:49:43.715 [INF] GRPH: Processed channels=0 updates=2 nodes=2 in last 1m0.000801406s
2025-05-08 10:49:38	
2025-05-08 15:49:38.732 [INF] NTFN: New confirmation subscription: conf_id=2, txid=ac6cf8ada16e0d6c7eabef767713a1f033609a6a6c47da89b0ebf2f18e239021, num_confs=1 height_hint=4325419
2025-05-08 10:49:38	
2025-05-08 15:49:38.731 [INF] GRDN: BatchCaretaker(037ffbe2ee1316dd27f1873ec3426d7748046e1427f1b32a47925bd4188251043b): transition states: BatchStateBroadcast -> BatchStateBroadcast
2025-05-08 10:49:38	
2025-05-08 15:49:38.731 [INF] NTFN: New confirmation subscription: conf_id=1, txid=ac6cf8ada16e0d6c7eabef767713a1f033609a6a6c47da89b0ebf2f18e239021, num_confs=6 height_hint=4325419
2025-05-08 10:49:38	
2025-05-08 15:49:38.730 [INF] LNWL: ac6cf8ada16e0d6c7eabef767713a1f033609a6a6c47da89b0ebf2f18e239021: tx already in mempool
2025-05-08 10:49:38	
2025-05-08 15:49:38.720 [WRN] BTWL: Transaction ac6cf8ada16e0d6c7eabef767713a1f033609a6a6c47da89b0ebf2f18e239021 not accepted by mempool: already have transaction in mempool ac6cf8ada16e0d6c7eabef767713a1f033609a6a6c47da89b0ebf2f18e239021
2025-05-08 10:49:38	
2025-05-08 15:49:38.719 [INF] LNWL: Inserting unconfirmed transaction ac6cf8ada16e0d6c7eabef767713a1f033609a6a6c47da89b0ebf2f18e239021
2025-05-08 10:49:38	
2025-05-08 15:49:38.716 [INF] GRDN: BatchCaretaker(037ffbe2ee1316dd27f1873ec3426d7748046e1427f1b32a47925bd4188251043b): extracted finalized GenesisTx
2025-05-08 10:49:38	
2025-05-08 15:49:38.716 [INF] GRDN: BatchCaretaker(037ffbe2ee1316dd27f1873ec3426d7748046e1427f1b32a47925bd4188251043b): transition states: BatchStateCommitted -> BatchStateBroadcast
2025-05-08 10:49:38	
2025-05-08 15:49:38.714 [INF] LNWL: Imported address tb1pdtj6jqm50p0pv9vstgw8kd25zum3xgk7m259hk3w5jr5q27de8vq285gdv
2025-05-08 10:49:38	
2025-05-08 15:49:38.707 [INF] GRDN: BatchCaretaker(037ffbe2ee1316dd27f1873ec3426d7748046e1427f1b32a47925bd4188251043b): GenesisPacket finalized (absolute_fee_sats: 770)
2025-05-08 10:49:38	
2025-05-08 15:49:38.703 [INF] GRDN: BatchCaretaker(037ffbe2ee1316dd27f1873ec3426d7748046e1427f1b32a47925bd4188251043b): finalizing GenesisPacket
2025-05-08 10:49:38	
2025-05-08 15:49:38.703 [INF] GRDN: BatchCaretaker(037ffbe2ee1316dd27f1873ec3426d7748046e1427f1b32a47925bd4188251043b): transition states: BatchStateFrozen -> BatchStateCommitted
2025-05-08 10:49:38	
2025-05-08 15:49:38.697 [INF] GRDN: BatchCaretaker(037ffbe2ee1316dd27f1873ec3426d7748046e1427f1b32a47925bd4188251043b): committing sprouts to disk
2025-05-08 10:49:38	
2025-05-08 15:49:38.696 [INF] GRDN: BatchCaretaker(037ffbe2ee1316dd27f1873ec3426d7748046e1427f1b32a47925bd4188251043b): mapping 1 seedlings to asset sprouts, with genesis_point=bd47d017443255d4ca53d9f6f5a2923ef3a1ca03e2521ce31bc9e721db80e747:1
2025-05-08 10:49:38	
2025-05-08 15:49:38.696 [INF] GRDN: BatchCaretaker(037ffbe2ee1316dd27f1873ec3426d7748046e1427f1b32a47925bd4188251043b), advancing from state=BatchStateFrozen to state=BatchStateBroadcast
2025-05-08 10:49:38	
2025-05-08 15:49:38.681 [INF] GRDN: Funded GenesisPacket for batch: 037ffbe2ee1316dd27f1873ec3426d7748046e1427f1b32a47925bd4188251043b
2025-05-08 10:49:38	
2025-05-08 15:49:38.666 [INF] LNWL: Web API returning 1012 sat/kvb for min relay feerate
2025-05-08 10:49:38	
2025-05-08 15:49:38.666 [WRN] LNWL: Web API does not have a fee rate for target=6, using the fee rate for target=2 instead
2025-05-08 10:49:38	
2025-05-08 15:49:38.666 [INF] GRDN: estimated fee rate for batch: 037ffbe2ee1316dd27f1873ec3426d7748046e1427f1b32a47925bd4188251043b, 4236 sat/kvb
2025-05-08 10:49:38	
2025-05-08 15:49:38.665 [INF] LNWL: Web API returning 1012 sat/kvb for min relay feerate
2025-05-08 10:49:38	
2025-05-08 15:49:38.665 [WRN] LNWL: Web API does not have a fee rate for target=6, using the fee rate for target=2 instead
2025-05-08 10:49:38	
2025-05-08 15:49:38.665 [INF] GRDN: creating skeleton PSBT for batch: 037ffbe2ee1316dd27f1873ec3426d7748046e1427f1b32a47925bd4188251043b
2025-05-08 10:49:38	
2025-05-08 15:49:38.665 [INF] GRDN: Attempting to fund batch: 037ffbe2ee1316dd27f1873ec3426d7748046e1427f1b32a47925bd4188251043b
2025-05-08 10:49:38	
2025-05-08 15:49:38.661 [INF] GRDN: Freezing MintingBatch(key=037ffbe2ee1316dd27f1873ec3426d7748046e1427f1b32a47925bd4188251043b, num_assets=1)
2025-05-08 10:49:38	
2025-05-08 15:49:38.661 [INF] GRDN: Finalizing batch 037ffbe2ee1316dd27f1873ec3426d7748046e1427f1b32a47925bd4188251043b
2025-05-08 10:49:38	
2025-05-08 15:49:38.661 [INF] LNWL: Web API returning 1012 sat/kvb for min relay feerate
2025-05-08 10:49:38	
2025-05-08 15:49:38.661 [WRN] LNWL: Web API does not have a fee rate for target=6, using the fee rate for target=2 instead
2025-05-08 10:49:38	
2025-05-08 15:49:38.656 [INF] LITD: Handling gRPC request: /mintrpc.Mint/FinalizeBatch
2025-05-08 10:49:22	
2025-05-08 15:49:22.866 [INF] GRDN: Request for new seedling: AssetSeedling(name=x45d, type=Normal, amt=50000, version=0) received
2025-05-08 10:49:22	
2025-05-08 15:49:22.859 [INF] GRDN: Adding AssetSeedling(name=x45d, type=Normal, amt=50000, version=0) received to new MintingBatch
2025-05-08 10:49:22	
2025-05-08 15:49:22.848 [INF] GRDN: Creating new MintingBatch
2025-05-08 10:49:22	
2025-05-08 15:49:22.830 [INF] RPCS: [MintAsset]: version=0, type=Normal, name=x45d, amt=50000, enable_emission=true
```

*Instance 2: Mainnet* 

`tapcli assets list` is showing:
  
```  
"unconfirmed_transfers":  "2",
    "unconfirmed_mints":  "1"
```
however the mint I performed >6 confirmations. Also in that output I can see the genesis info of that new asset, however the  chain_anchor.block_height has a value of 0 seemingly indicating it does not recognize being confirmed. When I do tapcli universe roots  this new asset does not show up. Restarting the node did not seem to kick it into place.

It appears logs run into this same `block number out of range` log

```
2025-05-20 10:49:45	
2025-05-20 15:49:45.462 [INF] LNWL: Marking unconfirmed transaction ec3ca4e9ad6bba0265eaf7aea1d8347d7f1817c088a372b27879bf15b1417f20 mined in block 897579
2025-05-20 10:49:45	
2025-05-20 15:49:45.461 [ERR] SWPR: Failed to handle bump event: transaction rejected: output already spent
2025-05-20 10:49:45	
2025-05-20 15:49:45.461 [ERR] SWPR: Fee bump attempt failed for tx=fdf4b555cc0769fc74d93c9b64cfe4e3f0a0efd52fc0e2a173bc2889e4e3c881: transaction rejected: output already spent
2025-05-20 10:49:45	
2025-05-20 15:49:45.461 [ERR] SWPR: Removing monitor record=5, tx=fdf4b555cc0769fc74d93c9b64cfe4e3f0a0efd52fc0e2a173bc2889e4e3c881, due to err: transaction rejected: output already spent
2025-05-20 10:49:45	
2025-05-20 15:49:45.461 [ERR] SWPR: Failed to publish tx fdf4b555cc0769fc74d93c9b64cfe4e3f0a0efd52fc0e2a173bc2889e4e3c881: transaction rejected: output already spent
2025-05-20 10:49:45	
2025-05-20 15:49:45.461 [INF] LNWL: Removed invalid transaction: fdf4b555cc0769fc74d93c9b64cfe4e3f0a0efd52fc0e2a173bc2889e4e3c881
2025-05-20 10:49:45	
2025-05-20 15:49:45.459 [INF] LNWL: fdf4b555cc0769fc74d93c9b64cfe4e3f0a0efd52fc0e2a173bc2889e4e3c881: broadcast failed because of: missing inputs
2025-05-20 10:49:45	
2025-05-20 15:49:45.406 [INF] LNWL: Marking unconfirmed transaction 3f7b30bbea3d75457f59417f430a129c1ad9835ada78ee8ce91a290b5d33bf16 mined in block 897579
2025-05-20 10:49:45	
2025-05-20 15:49:45.383 [WRN] BTWL: Transaction fdf4b555cc0769fc74d93c9b64cfe4e3f0a0efd52fc0e2a173bc2889e4e3c881 not accepted by mempool: already have transaction in mempool fdf4b555cc0769fc74d93c9b64cfe4e3f0a0efd52fc0e2a173bc2889e4e3c881
2025-05-20 10:49:45	
2025-05-20 15:49:45.382 [INF] LNWL: Inserting unconfirmed transaction fdf4b555cc0769fc74d93c9b64cfe4e3f0a0efd52fc0e2a173bc2889e4e3c881
2025-05-20 10:49:45	
2025-05-20 15:49:45.380 [INF] TCHN: Sweep request had no vPkts, exiting
2025-05-20 10:49:45	
2025-05-20 15:49:45.380 [INF] TCHN: Register broadcast of sweep_tx=(*wire.MsgTx)(0x4005f8bec0)({
2025-05-20 10:49:44	
2025-05-20 15:49:44.203 [ERR] GRDN: unable to advance state machine: unable to construct minting proofs: invalid proof file generated: failed to validate proof block header: unable to retrieve block hash: rpc error: code = Unknown desc = -1: Block number out of range
2025-05-20 10:49:44	
2025-05-20 15:49:44.203 [ERR] RPCS: [/chainrpc.ChainKit/GetBlockHash]: -1: Block number out of range
2025-05-20 10:49:44	
2025-05-20 15:49:44.196 [INF] GRDN: BatchCaretaker(022b49045d127a90e18e462f840448326668e294cc597a6ec6645e3d2a7a74c1d0), advancing from state=BatchStateConfirmed to state=BatchStateFinalized
2025-05-20 10:49:44	
2025-05-20 15:49:44.196 [INF] GRDN: MintingBatch(022b49045d127a90e18e462f840448326668e294cc597a6ec6645e3d2a7a74c1d0): confirmed at block(hash=00000000000000000001c38308acdd75b35cfcb41101967d9b776245630fe06c, height=897579)
2025-05-20 10:49:44	
2025-05-20 15:49:44.144 [INF] GRDN: New block at height 897579
2025-05-20 10:49:44	
2025-05-20 15:49:44.144 [INF] RPCS: Received new block notification: height=897579
2025-05-20 10:49:44	
2025-05-20 15:49:44.144 [INF] UTXN: Attempting to graduate height=897579: num_kids=0, num_babies=0
2025-05-20 10:49:44	
2025-05-20 15:49:44.144 [INF] LNWL: Web API returning 1012 sat/kvb for min relay feerate
2025-05-20 10:49:44	
2025-05-20 15:49:44.143 [INF] NTFN: New block: height=897579, sha=00000000000000000001c38308acdd75b35cfcb41101967d9b776245630fe06c
```

I went ahead and re-minted, but did not fee bump the mint this time, and it worked. 








Pull request description:
 chain_bridge+fn: add retry logic for RPC calls to improve reliability in load-balanced environments
This PR fixes #1562 by implementing automatic retry logic with exponential backoff for all block and header fetching operations in the chain bridge.

Along the way, we introduce a generic `RetryFuncN` function in the `fn` package that wraps any fallible operation with configurable retry logic featuring:
  - Exponential backoff to avoid overwhelming the backend
  - Context cancellation support for clean shutdown
  - Configurable retry parameters (count, delays, backoff multiplier)

  The retry wrapper is then applied to all key RPC operations in `chain_bridge.go`:
  - `GetBlock` - Ensures we can always fetch full block data
  - `GetBlockHeader` - Ensures header retrieval succeeds
  - `GetBlockHash` - Ensures height-to-hash lookups work
  - `CurrentHeight` - Ensures chain tip queries succeed
  - `GetBlockTimestamp` - Ensures timestamp lookups work
  - `PublishTransaction` - Improves transaction broadcast reliability
  - `EstimateFee` - Ensures fee estimation succeeds

  With these changes, temporary inconsistencies or failures in load-balanced LND setups are automatically retried, ensuring that block and header fetching operations eventually succeed as long as at least one backend node is functional.

Patch:
diff --git a/chain_bridge.go b/chain_bridge.go
index 33268d4c..1f8836cc 100644
--- a/chain_bridge.go
+++ b/chain_bridge.go
@@ -11,6 +11,7 @@ import (
 	"github.com/lightninglabs/lndclient"
 	"github.com/lightninglabs/neutrino/cache/lru"
 	"github.com/lightninglabs/taproot-assets/asset"
+	"github.com/lightninglabs/taproot-assets/fn"
 	"github.com/lightninglabs/taproot-assets/proof"
 	"github.com/lightninglabs/taproot-assets/rfq"
 	"github.com/lightninglabs/taproot-assets/tapchannel"
@@ -49,6 +50,7 @@ type LndRpcChainBridge struct {
 	lnd *lndclient.LndServices
 
 	blockTimestampCache *lru.Cache[uint32, cacheableTimestamp]
+	retryConfig         fn.RetryConfig
 
 	assetStore *tapdb.AssetStore
 }
@@ -63,7 +65,8 @@ func NewLndRpcChainBridge(lnd *lndclient.LndServices,
 		blockTimestampCache: lru.NewCache[uint32, cacheableTimestamp](
 			maxNumBlocksInCache,
 		),
-		assetStore: assetStore,
+		retryConfig: fn.DefaultRetryConfig(),
+		assetStore:  assetStore,
 	}
 }
 
@@ -112,25 +115,35 @@ func (l *LndRpcChainBridge) RegisterBlockEpochNtfn(
 func (l *LndRpcChainBridge) GetBlock(ctx context.Context,
 	hash chainhash.Hash) (*wire.MsgBlock, error) {
 
-	block, err := l.lnd.ChainKit.GetBlock(ctx, hash)
-	if err != nil {
-		return nil, fmt.Errorf("unable to retrieve block: %w", err)
-	}
-
-	return block, nil
+	return fn.RetryFuncN(
+		ctx, l.retryConfig, func() (*wire.MsgBlock, error) {
+			block, err := l.lnd.ChainKit.GetBlock(ctx, hash)
+			if err != nil {
+				return nil, fmt.Errorf(
+					"unable to retrieve block: %w", err,
+				)
+			}
+			return block, nil
+		},
+	)
 }
 
 // GetBlockHeader returns a block header given its hash.
 func (l *LndRpcChainBridge) GetBlockHeader(ctx context.Context,
 	hash chainhash.Hash) (*wire.BlockHeader, error) {
 
-	header, err := l.lnd.ChainKit.GetBlockHeader(ctx, hash)
-	if err != nil {
-		return nil, fmt.Errorf("unable to retrieve block header: %w",
-			err)
-	}
-
-	return header, nil
+	return fn.RetryFuncN(
+		ctx, l.retryConfig, func() (*wire.BlockHeader, error) {
+			header, err := l.lnd.ChainKit.GetBlockHeader(ctx, hash)
+			if err != nil {
+				return nil, fmt.Errorf(
+					"unable to retrieve block "+
+						"header: %w", err,
+				)
+			}
+			return header, nil
+		},
+	)
 }
 
 // GetBlockHash returns the hash of the block in the best blockchain at the
@@ -138,13 +151,20 @@ func (l *LndRpcChainBridge) GetBlockHeader(ctx context.Context,
 func (l *LndRpcChainBridge) GetBlockHash(ctx context.Context,
 	blockHeight int64) (chainhash.Hash, error) {
 
-	blockHash, err := l.lnd.ChainKit.GetBlockHash(ctx, blockHeight)
-	if err != nil {
-		return chainhash.Hash{}, fmt.Errorf("unable to retrieve "+
-			"block hash: %w", err)
-	}
-
-	return blockHash, nil
+	return fn.RetryFuncN(
+		ctx, l.retryConfig, func() (chainhash.Hash, error) {
+			blockHash, err := l.lnd.ChainKit.GetBlockHash(
+				ctx, blockHeight,
+			)
+			if err != nil {
+				return chainhash.Hash{}, fmt.Errorf(
+					"unable to retrieve block hash: %w",
+					err,
+				)
+			}
+			return blockHash, nil
+		},
+	)
 }
 
 // VerifyBlock returns an error if a block (with given header and height) is not
@@ -184,12 +204,17 @@ func (l *LndRpcChainBridge) VerifyBlock(ctx context.Context,
 
 // CurrentHeight return the current height of the main chain.
 func (l *LndRpcChainBridge) CurrentHeight(ctx context.Context) (uint32, error) {
-	_, bestHeight, err := l.lnd.ChainKit.GetBestBlock(ctx)
-	if err != nil {
-		return 0, fmt.Errorf("unable to grab block height: %w", err)
-	}
-
-	return uint32(bestHeight), nil
+	return fn.RetryFuncN(
+		ctx, l.retryConfig, func() (uint32, error) {
+			_, bestHeight, err := l.lnd.ChainKit.GetBestBlock(ctx)
+			if err != nil {
+				return 0, fmt.Errorf(
+					"unable to grab block height: %w", err,
+				)
+			}
+			return uint32(bestHeight), nil
+		},
+	)
 }
 
 // GetBlockTimestamp returns the timestamp of the block at the given height.
@@ -207,7 +232,11 @@ func (l *LndRpcChainBridge) GetBlockTimestamp(ctx context.Context,
 		return int64(cacheTS)
 	}
 
-	hash, err := l.lnd.ChainKit.GetBlockHash(ctx, int64(height))
+	hash, err := fn.RetryFuncN(
+		ctx, l.retryConfig, func() (chainhash.Hash, error) {
+			return l.lnd.ChainKit.GetBlockHash(ctx, int64(height))
+		},
+	)
 	if err != nil {
 		return 0
 	}
@@ -229,14 +258,27 @@ func (l *LndRpcChainBridge) GetBlockTimestamp(ctx context.Context,
 func (l *LndRpcChainBridge) PublishTransaction(ctx context.Context,
 	tx *wire.MsgTx, label string) error {
 
-	return l.lnd.WalletKit.PublishTransaction(ctx, tx, label)
+	_, err := fn.RetryFuncN(
+		ctx, l.retryConfig, func() (struct{}, error) {
+			return struct{}{}, l.lnd.WalletKit.PublishTransaction(
+				ctx, tx, label,
+			)
+		},
+	)
+	return err
 }
 
 // EstimateFee returns a fee estimate for the confirmation target.
 func (l *LndRpcChainBridge) EstimateFee(ctx context.Context,
 	confTarget uint32) (chainfee.SatPerKWeight, error) {
 
-	return l.lnd.WalletKit.EstimateFeeRate(ctx, int32(confTarget))
+	return fn.RetryFuncN(
+		ctx, l.retryConfig, func() (chainfee.SatPerKWeight, error) {
+			return l.lnd.WalletKit.EstimateFeeRate(
+				ctx, int32(confTarget),
+			)
+		},
+	)
 }
 
 // GenFileChainLookup generates a chain lookup interface for the given
diff --git a/fn/retry.go b/fn/retry.go
new file mode 100644
index 00000000..b7639959
--- /dev/null
+++ b/fn/retry.go
@@ -0,0 +1,86 @@
+package fn
+
+import (
+	"context"
+	"time"
+)
+
+// RetryConfig defines the parameters for exponential backoff retry behavior.
+type RetryConfig struct {
+	// MaxRetries specifies how many times to retry after the initial
+	// attempt fails.
+	MaxRetries int
+
+	// InitialBackoff sets the delay before the first retry attempt.
+	InitialBackoff time.Duration
+
+	// BackoffMultiplier determines the exponential growth rate of the
+	// backoff duration between successive retries.
+	BackoffMultiplier float64
+
+	// MaxBackoff caps the delay between retries to prevent excessive
+	// wait times.
+	MaxBackoff time.Duration
+}
+
+// DefaultRetryConfig provides sensible defaults for retrying RPC calls in
+// load-balanced environments where transient failures are expected.
+func DefaultRetryConfig() RetryConfig {
+	return RetryConfig{
+		MaxRetries:        10,
+		InitialBackoff:    100 * time.Millisecond,
+		BackoffMultiplier: 2.0,
+		MaxBackoff:        5 * time.Second,
+	}
+}
+
+// RetryFuncN executes the provided function with exponential backoff retry
+// logic. This is particularly useful for RPC calls in load-balanced
+// environments where nodes may temporarily return inconsistent results. The
+// function respects context cancellation and returns immediately if the context
+// is cancelled.
+func RetryFuncN[T any](ctx context.Context,
+	config RetryConfig, fn func() (T, error)) (T, error) {
+
+	var (
+		result T
+		err    error
+	)
+
+	backoff := config.InitialBackoff
+
+	// We'll retry the function up to MaxRetries times, backing off each
+	// time until it succeeds.
+	for attempt := 0; attempt <= config.MaxRetries; attempt++ {
+		result, err = fn()
+		if err == nil {
+			return result, nil
+		}
+
+		if attempt == config.MaxRetries {
+			return result, err
+		}
+
+		// Cap the backoff at the configured maximum to prevent
+		// excessive delays.
+		if backoff > config.MaxBackoff {
+			backoff = config.MaxBackoff
+		}
+
+		// Wait for the backoff duration or until the context is
+		// cancelled, whichever comes first.
+		select {
+		case <-ctx.Done():
+			return result, ctx.Err()
+
+		case <-time.After(backoff):
+			// Apply the multiplier to implement exponential
+			// backoff.
+			backoff = time.Duration(
+				float64(backoff) * config.BackoffMultiplier,
+			)
+		}
+	}
+
+	return result, err
+}


Test patch:
diff --git a/fn/retry_test.go b/fn/retry_test.go
new file mode 100644
index 00000000..cac521d6
--- /dev/null
+++ b/fn/retry_test.go
@@ -0,0 +1,288 @@
+package fn
+
+import (
+	"context"
+	"errors"
+	"sync/atomic"
+	"testing"
+	"time"
+
+	"github.com/stretchr/testify/require"
+	"pgregory.net/rapid"
+)
+
+// TestRetryFuncNSuccessReturnsImmediately verifies that a successful function
+// returns immediately without any retries.
+func TestRetryFuncNSuccessReturnsImmediately(t *testing.T) {
+	t.Parallel()
+
+	rapid.Check(t, func(t *rapid.T) {
+		// Generate a random retry config with reasonable bounds.
+		config := RetryConfig{
+			MaxRetries: rapid.IntRange(1, 10).Draw(t, "maxRetries"),
+			InitialBackoff: time.Duration(
+				rapid.IntRange(1, 100).Draw(
+					t, "initialBackoffMs",
+				),
+			) * time.Millisecond,
+			BackoffMultiplier: rapid.Float64Range(
+				1.1, 3.0,
+			).Draw(t, "backoffMultiplier"),
+			MaxBackoff: time.Duration(
+				rapid.IntRange(100, 5000).Draw(
+					t, "maxBackoffMs",
+				),
+			) * time.Millisecond,
+		}
+
+		// Generate a random value to return.
+		expectedValue := rapid.Int().Draw(t, "expectedValue")
+
+		// Track how many times the function is called.
+		var callCount atomic.Int32
+
+		ctx := context.Background()
+		start := time.Now()
+
+		result, err := RetryFuncN(ctx, config, func() (int, error) {
+			callCount.Add(1)
+			return expectedValue, nil
+		})
+
+		elapsed := time.Since(start)
+
+		// The function should only be called once.
+		require.Equal(t, int32(1), callCount.Load())
+
+		// No error should be returned.
+		require.NoError(t, err)
+
+		// The correct value should be returned.
+		require.Equal(t, expectedValue, result)
+
+		// The function should return almost immediately (allowing for
+		// some execution overhead).
+		require.Less(t, elapsed, 10*time.Millisecond)
+	})
+}
+
+// TestRetryFuncNRetriesExactlyMaxRetries verifies that a function that always
+// fails is retried exactly MaxRetries times.
+func TestRetryFuncNRetriesExactlyMaxRetries(t *testing.T) {
+	t.Parallel()
+
+	rapid.Check(t, func(t *rapid.T) {
+		// Generate a random retry config.
+		maxRetries := rapid.IntRange(0, 5).Draw(t, "maxRetries")
+		config := RetryConfig{
+			MaxRetries: maxRetries,
+			InitialBackoff: time.Duration(
+				rapid.IntRange(1, 10).Draw(
+					t, "initialBackoffMs",
+				),
+			) * time.Millisecond,
+			BackoffMultiplier: rapid.Float64Range(
+				1.1, 2.0,
+			).Draw(t, "backoffMultiplier"),
+			MaxBackoff: time.Duration(
+				rapid.IntRange(50, 100).Draw(t, "maxBackoffMs"),
+			) * time.Millisecond,
+		}
+
+		// Track how many times the function is called.
+		var callCount atomic.Int32
+
+		// Create a consistent error for all attempts.
+		expectedErr := errors.New("persistent failure")
+
+		ctx := context.Background()
+
+		_, err := RetryFuncN(ctx, config, func() (int, error) {
+			callCount.Add(1)
+			return 0, expectedErr
+		})
+
+		// The function should be called exactly MaxRetries + 1 times
+		// (initial attempt + retries).
+		require.Equal(t, int32(maxRetries+1), callCount.Load())
+
+		// The final error should be returned.
+		require.Equal(t, expectedErr, err)
+	})
+}
+
+// TestRetryFuncNBackoffIncreases verifies that the backoff duration increases
+// exponentially between retries.
+func TestRetryFuncNBackoffIncreases(t *testing.T) {
+	t.Parallel()
+
+	rapid.Check(t, func(t *rapid.T) {
+		// Generate retry config with at least 2 retries to observe
+		// backoff behavior.
+		config := RetryConfig{
+			MaxRetries: rapid.IntRange(2, 3).Draw(
+				t, "maxRetries",
+			),
+			InitialBackoff: time.Duration(
+				// We use a slightly larger initial backoff
+				// range here to avoid flakes on slow CI
+				// machines where scheduling overhead can be
+				// significant for very short sleep durations.
+				// Reduced range for faster test execution.
+				rapid.IntRange(10, 25).Draw(
+					t, "initialBackoffMs",
+				),
+			) * time.Millisecond,
+			BackoffMultiplier: rapid.Float64Range(
+				1.5, 2.5,
+			).Draw(t, "backoffMultiplier"),
+			MaxBackoff: time.Duration(
+				rapid.IntRange(50, 150).Draw(
+					t, "maxBackoffMs",
+				),
+			) * time.Millisecond,
+		}
+
+		// Track call times to measure backoff.
+		var callTimes []time.Time
+
+		ctx := context.Background()
+
+		_, err := RetryFuncN(ctx, config, func() (int, error) {
+			callTimes = append(callTimes, time.Now())
+			return 0, errors.New("fail")
+		})
+
+		require.Error(t, err)
+		require.Len(t, callTimes, config.MaxRetries+1)
+
+		expectedBackoff := config.InitialBackoff
+		for i := 1; i < len(callTimes); i++ {
+			actualBackoff := callTimes[i].Sub(callTimes[i-1])
+
+			// To avoid flakes, we use an asymmetric check for the
+			// backoff duration. We expect the actual backoff to be
+			// close to the expected backoff, but we allow for a
+			// generous upper bound to account for scheduling delays
+			// on busy systems. // The actual backoff should be
+			// reasonably close to the expected backoff. We allow it
+			// to be slightly shorter due to timer precision.
+			lowerBound := time.Duration(
+				float64(expectedBackoff) * 0.8,
+			)
+			require.GreaterOrEqual(t, actualBackoff, lowerBound)
+
+			// The actual backoff can be longer due to scheduling
+			// delays. We allow a generous upper bound.
+			upperBound := time.Duration(
+				float64(expectedBackoff)*1.5,
+			) + 100*time.Millisecond
+			require.LessOrEqual(t, actualBackoff, upperBound)
+
+			// Calculate the next expected backoff, capping at
+			// MaxBackoff.
+			expectedBackoff = time.Duration(
+				float64(expectedBackoff) *
+					config.BackoffMultiplier,
+			)
+			if expectedBackoff > config.MaxBackoff {
+				expectedBackoff = config.MaxBackoff
+			}
+		}
+	})
+}
+
+// TestRetryFuncNContextCancellation verifies that context cancellation stops
+// the retry loop immediately.
+func TestRetryFuncNContextCancellation(t *testing.T) {
+	t.Parallel()
+
+	rapid.Check(t, func(t *rapid.T) {
+		// Generate a retry config with shorter timeouts for faster test
+		// execution.
+		config := RetryConfig{
+			MaxRetries: rapid.IntRange(2, 5).Draw(t, "maxRetries"),
+			InitialBackoff: time.Duration(
+				rapid.IntRange(10, 50).Draw(
+					t, "initialBackoffMs",
+				),
+			) * time.Millisecond,
+			BackoffMultiplier: 1.5,
+			MaxBackoff:        100 * time.Millisecond,
+		}
+
+		// Track how many times the function is called.
+		var callCount atomic.Int32
+
+		// Cancel the context after the first attempt to ensure we
+		// cancel during a backoff wait.
+		ctx, cancel := context.WithCancel(context.Background())
+
+		// Schedule cancellation after a short delay.
+		go func() {
+			time.Sleep(5 * time.Millisecond)
+			cancel()
+		}()
+
+		_, err := RetryFuncN(ctx, config, func() (int, error) {
+			callCount.Add(1)
+			return 0, errors.New("fail")
+		})
+
+		// The error should be the context cancellation error.
+		require.Equal(t, context.Canceled, err)
+
+		// The function should have been called at least once but not
+		// more than MaxRetries+1 times.
+		calls := callCount.Load()
+		require.GreaterOrEqual(t, calls, int32(1))
+		require.LessOrEqual(t, calls, int32(config.MaxRetries+1))
+	})
+}
+
+// TestRetryFuncNEventualSuccess verifies that if a function succeeds after some
+// failures, the correct result is returned.
+func TestRetryFuncNEventualSuccess(t *testing.T) {
+	t.Parallel()
+
+	rapid.Check(t, func(t *rapid.T) {
+		config := RetryConfig{
+			MaxRetries: rapid.IntRange(3, 10).Draw(t, "maxRetries"),
+			InitialBackoff: time.Duration(
+				rapid.IntRange(1, 10).Draw(
+					t, "initialBackoffMs",
+				),
+			) * time.Millisecond,
+			BackoffMultiplier: 2.0,
+			MaxBackoff:        50 * time.Millisecond,
+		}
+
+		// Determine after how many attempts the function should
+		// succeed.
+		succeedAfter := rapid.IntRange(
+			1, config.MaxRetries+1,
+		).Draw(t, "succeedAfter")
+
+		expectedValue := rapid.Int().Draw(t, "expectedValue")
+
+		// Track how many times the function is called.
+		var callCount atomic.Int32
+
+		ctx := context.Background()
+
+		result, err := RetryFuncN(ctx, config, func() (int, error) {
+			count := callCount.Add(1)
+			if int(count) >= succeedAfter {
+				return expectedValue, nil
+			}
+			return 0, errors.New("temporary failure")
+		})
+
+		// The function should succeed.
+		require.NoError(t, err)
+		require.Equal(t, expectedValue, result)
+
+		// The function should be called exactly succeedAfter times.
+		require.Equal(t, int32(succeedAfter), callCount.Load())
+	})
+}