Repository: celestiaorg/celestia-core (go)

Issue:
core `/header` and `/block` rpc endpoint returns a malformed response
### Summary of Bug

When we make a request for blocks before `6,621,759` against our node running `v4.0.2-mocha` we receive the following:

```
`/header` response: {"jsonrpc":"2.0","id":-1,"result":{"header":null}}
`/block` response: {"jsonrpc":"2.0","id":-1,"result":{"block_id":{"hash":"","parts":{"total":0,"hash":""}},"block":null}}
```

According to the JSON RPC spec, a response should be either successful, in which case it has a "result" property, or an error, in which case it has an "error" property -- this looks to be an error, but bundled up in a JSON RPC success response.

We should be getting an `error` as our node is pruning blocks (`min-retain-blocks = 1`) and does not have `6,621,759`. Instead it looks like we're getting an empty `success` response.

When we query our mainnet node that is running `v3.8.1`, we do get an error for a missing block. We have the same `min-retain-blocks = 1` setting on the mainnet node.

```
{"jsonrpc":"2.0","id":-1,"error":{"code":-32603,"message":"Internal error","data":"height 60 is not available, lowest height is 5873259"}}
```

So we believe it is a regression in `v4.x.x`

### Version

4.0.2-mocha

### Steps to Reproduce

1. run a `/header` query for a known missing block
```
curl --location 'https://<v4.0.2-mocha consensus node ip>:<port>/header?height=<missing block height>'
```

Pull request:
fix: return proper JSON-RPC errors for missing blocks in /header and /block endpoints
## Summary

This PR fixes malformed JSON-RPC responses returned by the `/header` and `/block` RPC endpoints when requested blocks are not available (e.g., pruned blocks). Instead of returning malformed success responses with null values, these endpoints now return proper JSON-RPC error responses.

## Problem

When requesting blocks that don't exist or have been pruned, the RPC endpoints were returning malformed JSON-RPC success responses:

```json
// /header response for missing block
{"jsonrpc":"2.0","id":-1,"result":{"header":null}}

// /block response for missing block  
{"jsonrpc":"2.0","id":-1,"result":{"block_id":{"hash":"","parts":{"total":0,"hash":""}},"block":null}}
```

According to the JSON-RPC specification, responses should either be successful (with a "result" property) or an error (with an "error" property). The above responses are malformed because they return success responses with meaningless null/empty values.

## Solution

Modified four RPC endpoint functions to return proper errors instead of empty result structures when blocks are not available:

- `Header()` - returns error when `blockMeta == nil`
- `HeaderByHash()` - returns error when `blockMeta == nil` 
- `Block()` - returns error when `blockMeta == nil`
- `BlockByHash()` - returns error when `block == nil`

## After Fix

Now these endpoints return proper JSON-RPC error responses:

```json
{"jsonrpc":"2.0","id":-1,"error":{"code":-32603,"message":"height X is not available"}}
```

## Changes Made

- **4 lines changed** in `rpc/core/blocks.go` - minimal surgical fix
- Added comprehensive tests in `rpc/core/missing_block_test.go` to validate the behavior
- Error messages are consistent with existing patterns in the codebase

## Testing

- All existing RPC tests continue to pass
- New tests validate that missing blocks return proper errors
- Verified that the JSON-RPC layer properly wraps the errors in standard error responses

This is a regression fix that restores the proper v3.x behavior where missing blocks return JSON-RPC errors instead of malformed success responses.

Fixes #2043.

<!-- START COPILOT CODING AGENT TIPS -->
---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more [Copilot coding agent tips](https://gh.io/copilot-coding-agent-tips) in the docs.

Patch:
diff --git a/abci/types/types.pb.go b/abci/types/types.pb.go
index febd49e11..5bfb30b46 100644
--- a/abci/types/types.pb.go
+++ b/abci/types/types.pb.go
@@ -1215,7 +1215,7 @@ type RequestProcessProposal struct {
 	SquareSize      uint64 `protobuf:"varint,9,opt,name=square_size,json=squareSize,proto3" json:"square_size,omitempty"`
 	DataRootHash    []byte `protobuf:"bytes,10,opt,name=data_root_hash,json=dataRootHash,proto3" json:"data_root_hash,omitempty"`
 	// header is the header of the block that is being processed
-	// this is needed for v3 to sync with nova as the header is stored in state
+	// this is needed for v3 to sync with multiplexer as the header is stored in state
 	Header *types1.Header `protobuf:"bytes,20,opt,name=header,proto3" json:"header,omitempty"`
 }
 
@@ -1517,7 +1517,7 @@ type RequestFinalizeBlock struct {
 	// proposer_address is the address of the public key of the original proposer of the block.
 	ProposerAddress []byte `protobuf:"bytes,8,opt,name=proposer_address,json=proposerAddress,proto3" json:"proposer_address,omitempty"`
 	// header is the header of the block that is being finalized
-	// this is needed for v3 to sync with nova as the header is stored in state
+	// this is needed for v3 to sync with multiplexer as the header is stored in state
 	Header *types1.Header `protobuf:"bytes,20,opt,name=header,proto3" json:"header,omitempty"`
 }
 
diff --git a/blocksync/pool.go b/blocksync/pool.go
index 7b754cb0c..3ad91e8d8 100644
--- a/blocksync/pool.go
+++ b/blocksync/pool.go
@@ -336,6 +336,13 @@ func (pool *BlockPool) AddBlock(peerID p2p.ID, block *types.Block, extCommit *ty
 	}
 
 	if !requester.setBlock(block, extCommit, peerID) {
+		// Check if this peer was recently banned. If so, this is likely a race condition
+		// where the block arrived after the peer was banned and reset from the requester.
+		// This is not an error, just a timing issue.
+		if pool.isPeerBanned(peerID) {
+			pool.Logger.Debug("Ignoring block from recently banned peer", "peer", peerID, "height", block.Height)
+			return nil
+		}
 		err := fmt.Errorf("requested block #%d from %v, not %s", block.Height, requester.requestedFrom(), peerID)
 		pool.sendError(err, peerID)
 		return err
diff --git a/consensus/propagation/commitment.go b/consensus/propagation/commitment.go
index c6fd3e3f1..52a5b469b 100644
--- a/consensus/propagation/commitment.go
+++ b/consensus/propagation/commitment.go
@@ -150,7 +150,7 @@ func (blockProp *Reactor) handleCompactBlock(cb *proptypes.CompactBlock, peer p2
 	err := blockProp.validateCompactBlock(cb)
 	if !proposer && err != nil {
 		blockProp.DeleteRound(cb.Proposal.Height, cb.Proposal.Round)
-		blockProp.Logger.Info("failed to validate proposal. ignoring", "err", err, "height", cb.Proposal.Height, "round", cb.Proposal.Round)
+		blockProp.Logger.Debug("failed to validate proposal. ignoring", "err", err, "height", cb.Proposal.Height, "round", cb.Proposal.Round)
 		return
 	}
 
@@ -170,7 +170,7 @@ func (blockProp *Reactor) handleCompactBlock(cb *proptypes.CompactBlock, peer p2
 		case blockProp.proposalChan <- cb.Proposal:
 		}
 		// check if we have any transactions that are in the compact block
-		go blockProp.recoverPartsFromMempool(cb)
+		blockProp.recoverPartsFromMempool(cb)
 	}
 
 	blockProp.broadcastCompactBlock(cb, peer)
@@ -229,7 +229,6 @@ func (blockProp *Reactor) recoverPartsFromMempool(cb *proptypes.CompactBlock) {
 		return
 	}
 
-	originalParts := partSet.Original()
 	recoveredCount := 0
 	haves := proptypes.HaveParts{
 		Height: cb.Proposal.Height,
@@ -237,12 +236,12 @@ func (blockProp *Reactor) recoverPartsFromMempool(cb *proptypes.CompactBlock) {
 		Parts:  make([]proptypes.PartMetaData, 0),
 	}
 	for _, p := range parts {
-		if originalParts.HasPart(int(p.Index)) {
+		if partSet.HasPart(int(p.Index)) {
 			continue
 		}
 		p.Proof = *proofs[p.Index]
 
-		added, err := originalParts.AddPart(p)
+		added, err := partSet.AddOriginalPart(p)
 		if err != nil {
 			blockProp.Logger.Error("failed to add locally recovered part", "err", err)
 			continue
diff --git a/consensus/propagation/reactor.go b/consensus/propagation/reactor.go
index 458d3b82a..a793d60aa 100644
--- a/consensus/propagation/reactor.go
+++ b/consensus/propagation/reactor.go
@@ -243,6 +243,7 @@ func (blockProp *Reactor) ReceiveEnvelope(e p2p.Envelope) {
 		case *proptypes.HaveParts:
 			blockProp.handleHaves(e.Src.ID(), msg)
 		case *proptypes.RecoveryPart:
+			schema.WriteReceivedPart(blockProp.traceClient, msg.Height, msg.Round, int(msg.Index))
 			blockProp.handleRecoveryPart(e.Src.ID(), msg)
 		default:
 			blockProp.Logger.Error(fmt.Sprintf("Unknown message type %v", reflect.TypeOf(msg)))
diff --git a/consensus/propagation/types/combined_partset.go b/consensus/propagation/types/combined_partset.go
index b6c000845..eaa68949b 100644
--- a/consensus/propagation/types/combined_partset.go
+++ b/consensus/propagation/types/combined_partset.go
@@ -133,7 +133,7 @@ func (cps *CombinedPartSet) Decode() error {
 	return nil
 }
 
-// AddPart adds a part to the combined part set. It assumes that the parts being
+// AddPart adds a recovery part to the combined part set. It assumes that the parts being
 // added have already been verified.
 func (cps *CombinedPartSet) AddPart(part *RecoveryPart, proof merkle.Proof) (bool, error) {
 	p := &types.Part{
@@ -162,6 +162,17 @@ func (cps *CombinedPartSet) AddPart(part *RecoveryPart, proof merkle.Proof) (boo
 	return added, err
 }
 
+// AddOriginalPart adds an original part to the combined partset.
+func (cps *CombinedPartSet) AddOriginalPart(part *types.Part) (bool, error) {
+	cps.mtx.Lock()
+	defer cps.mtx.Unlock()
+	added, err := cps.original.AddPart(part)
+	if added {
+		cps.totalMap.SetIndex(int(part.Index), true)
+	}
+	return added, err
+}
+
 func (cps *CombinedPartSet) HasPart(index int) bool {
 	return cps.totalMap.GetIndex(index)
 }
diff --git a/consensus/state.go b/consensus/state.go
index 979017975..2ce1a0cd3 100644
--- a/consensus/state.go
+++ b/consensus/state.go
@@ -1421,8 +1421,6 @@ func (cs *State) defaultDoPrevote(height int64, round int32) {
 		return
 	}
 
-	schema.WriteABCI(cs.traceClient, schema.ProcessProposalStart, height, round)
-
 	// Validate proposal block, from consensus' perspective
 	err := cs.blockExec.ValidateBlock(cs.state, cs.ProposalBlock)
 	if err != nil {
@@ -1443,14 +1441,15 @@ func (cs *State) defaultDoPrevote(height int64, round int32) {
 		Please see `PrepareProosal`-`ProcessProposal` coherence and determinism properties
 		in the ABCI++ specification.
 	*/
-	schema.WriteABCI(cs.traceClient, schema.ProcessProposalEnd, height, round)
 
+	schema.WriteABCI(cs.traceClient, schema.ProcessProposalStart, height, round)
 	isAppValid, err := cs.blockExec.ProcessProposal(cs.ProposalBlock, cs.state)
 	if err != nil {
 		panic(fmt.Sprintf(
 			"state machine returned an error (%v) when calling ProcessProposal", err,
 		))
 	}
+	schema.WriteABCI(cs.traceClient, schema.ProcessProposalEnd, height, round)
 	cs.metrics.MarkProposalProcessed(isAppValid)
 
 	// Vote nil if the Application rejected the block
diff --git a/go.mod b/go.mod
index 0bd5d6ca2..56ca6c4f4 100644
--- a/go.mod
+++ b/go.mod
@@ -11,7 +11,7 @@ require (
 	github.com/aws/aws-sdk-go-v2 v1.36.5
 	github.com/aws/aws-sdk-go-v2/config v1.29.17
 	github.com/aws/aws-sdk-go-v2/credentials v1.17.70
-	github.com/aws/aws-sdk-go-v2/service/s3 v1.82.0
+	github.com/aws/aws-sdk-go-v2/service/s3 v1.83.0
 	github.com/btcsuite/btcd/btcutil v1.1.6
 	github.com/celestiaorg/go-square/v2 v2.3.0
 	github.com/celestiaorg/nmt v0.24.0
diff --git a/go.sum b/go.sum
index 093e1cc1a..7014fc69a 100644
--- a/go.sum
+++ b/go.sum
@@ -54,8 +54,8 @@ github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.12.17 h1:t0E6FzRE
 github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.12.17/go.mod h1:ygpklyoaypuyDvOM5ujWGrYWpAK3h7ugnmKCU/76Ys4=
 github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.18.17 h1:qcLWgdhq45sDM9na4cvXax9dyLitn8EYBRl8Ak4XtG4=
 github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.18.17/go.mod h1:M+jkjBFZ2J6DJrjMv2+vkBbuht6kxJYtJiwoVgX4p4U=
-github.com/aws/aws-sdk-go-v2/service/s3 v1.82.0 h1:JubM8CGDDFaAOmBrd8CRYNr49ZNgEAiLwGwgNMdS0nw=
-github.com/aws/aws-sdk-go-v2/service/s3 v1.82.0/go.mod h1:kUklwasNoCn5YpyAqC/97r6dzTA1SRKJfKq16SXeoDU=
+github.com/aws/aws-sdk-go-v2/service/s3 v1.83.0 h1:5Y75q0RPQoAbieyOuGLhjV9P3txvYgXv2lg0UwJOfmE=
+github.com/aws/aws-sdk-go-v2/service/s3 v1.83.0/go.mod h1:kUklwasNoCn5YpyAqC/97r6dzTA1SRKJfKq16SXeoDU=
 github.com/aws/aws-sdk-go-v2/service/sso v1.25.5 h1:AIRJ3lfb2w/1/8wOOSqYb9fUKGwQbtysJ2H1MofRUPg=
 github.com/aws/aws-sdk-go-v2/service/sso v1.25.5/go.mod h1:b7SiVprpU+iGazDUqvRSLf5XmCdn+JtT1on7uNL6Ipc=
 github.com/aws/aws-sdk-go-v2/service/ssooidc v1.30.3 h1:BpOxT3yhLwSJ77qIY3DoHAQjZsc4HEGfMCE4NGy3uFg=
diff --git a/libs/trace/schema/recovery.go b/libs/trace/schema/recovery.go
new file mode 100644
index 000000000..38f829e6e
--- /dev/null
+++ b/libs/trace/schema/recovery.go
@@ -0,0 +1,34 @@
+package schema
+
+import (
+	"github.com/cometbft/cometbft/libs/trace"
+)
+
+// RecoveryTables returns the list of tables that are used for recovery
+// tracing.
+func RecoveryTables() []string {
+	return []string{
+		ReceivedPartTable,
+	}
+}
+
+const (
+	// ReceivedPartTable tracks all the parts received by propagation reactor
+	ReceivedPartTable = "recovery_received_part"
+)
+
+// ReceivedPart describes schema for the "recovery_received_part" table.
+type ReceivedPart struct {
+	Height int64 `json:"height"`
+	Round  int32 `json:"round"`
+	Index  int   `json:"index"`
+}
+
+// Table returns the table name for the ReceivedPart struct.
+func (ReceivedPart) Table() string {
+	return ReceivedPartTable
+}
+
+func WriteReceivedPart(client trace.Tracer, height int64, round int32, index int) {
+	client.Write(ReceivedPart{Height: height, Round: round, Index: index})
+}
diff --git a/libs/trace/schema/schema.go b/libs/trace/schema/schema.go
index 9f2377e0e..7cba3becf 100644
--- a/libs/trace/schema/schema.go
+++ b/libs/trace/schema/schema.go
@@ -16,6 +16,7 @@ func AllTables() []string {
 	tables = append(tables, ConsensusTables()...)
 	tables = append(tables, P2PTables()...)
 	tables = append(tables, ABCITable)
+	tables = append(tables, RecoveryTables()...)
 	return tables
 }
 
diff --git a/node/node.go b/node/node.go
index e6a9564b7..272901d26 100644
--- a/node/node.go
+++ b/node/node.go
@@ -418,6 +418,7 @@ func NewNodeWithContext(ctx context.Context,
 		blockStore,
 		sm.BlockExecutorWithMetrics(smMetrics),
 		sm.BlockExecutorWithRootDir(config.RootDir),
+		sm.BlockExecutorWithTracer(tracer),
 	)
 
 	offlineStateSyncHeight := int64(0)
diff --git a/p2p/conn/connection.go b/p2p/conn/connection.go
index 5aa89b896..9f06d0525 100644
--- a/p2p/conn/connection.go
+++ b/p2p/conn/connection.go
@@ -627,7 +627,7 @@ FOR_LOOP:
 
 			if c.IsRunning() {
 				if err == io.EOF {
-					c.Logger.Info("Connection is closed @ recvRoutine (likely by the other side)", "conn", c)
+					c.Logger.Debug("Connection is closed @ recvRoutine (likely by the other side)", "conn", c)
 				} else {
 					c.Logger.Debug("Connection failed @ recvRoutine (reading byte)", "conn", c, "err", err)
 				}
diff --git a/p2p/pex/addrbook.go b/p2p/pex/addrbook.go
index 867df974f..397ee7987 100644
--- a/p2p/pex/addrbook.go
+++ b/p2p/pex/addrbook.go
@@ -473,7 +473,7 @@ func (a *addrBook) addToNewBucket(ka *knownAddress, bucketIdx int) error {
 
 	// Enforce max addresses.
 	if len(bucket) > newBucketSize {
-		a.Logger.Info("new bucket is full, expiring new")
+		a.Logger.Debug("new bucket is full, expiring new")
 		a.expireNew(bucketIdx)
 	}
 
diff --git a/p2p/pex/file.go b/p2p/pex/file.go
index 38eec9636..9f2cc6352 100644
--- a/p2p/pex/file.go
+++ b/p2p/pex/file.go
@@ -19,7 +19,7 @@ func (a *addrBook) saveToFile(filePath string) {
 	a.mtx.Lock()
 	defer a.mtx.Unlock()
 
-	a.Logger.Info("Saving AddrBook to file", "size", a.size())
+	a.Logger.Debug("Saving AddrBook to file", "size", a.size())
 
 	addrs := make([]*knownAddress, 0, len(a.addrLookup))
 	for _, ka := range a.addrLookup {
diff --git a/p2p/pex/pex_reactor.go b/p2p/pex/pex_reactor.go
index b66368b62..6b18ed940 100644
--- a/p2p/pex/pex_reactor.go
+++ b/p2p/pex/pex_reactor.go
@@ -98,9 +98,12 @@ type Reactor struct {
 }
 
 func (r *Reactor) minReceiveRequestInterval() time.Duration {
-	// NOTE: must be around ensurePeersPeriod, otherwise we'll request
+	// NOTE: must be less than ensurePeersPeriod, otherwise we'll request
 	// peers too quickly from others and they'll think we're bad!
-	return r.ensurePeersPeriod
+	// According to the spec, the minimum accepted interval should be
+	// ensurePeersPeriod / 3 to allow for timing variations while still
+	// preventing abuse.
+	return r.ensurePeersPeriod / 3
 }
 
 // ReactorConfig holds reactor specific configuration data.
diff --git a/p2p/switch.go b/p2p/switch.go
index f30943b67..40613a6fd 100644
--- a/p2p/switch.go
+++ b/p2p/switch.go
@@ -739,7 +739,7 @@ func (sw *Switch) acceptRoutine() {
 			// Ignore connection if we already have enough peers.
 			_, in, _ := sw.NumPeers()
 			if in >= sw.config.MaxNumInboundPeers {
-				sw.Logger.Info(
+				sw.Logger.Debug(
 					"Ignoring inbound connection: already have enough inbound peers",
 					"address", p.SocketAddr(),
 					"have", in,
diff --git a/proto/tendermint/abci/types.proto b/proto/tendermint/abci/types.proto
index 65a2bca62..613e91686 100644
--- a/proto/tendermint/abci/types.proto
+++ b/proto/tendermint/abci/types.proto
@@ -159,7 +159,7 @@ message RequestProcessProposal {
   bytes  data_root_hash   = 10;
 
   // header is the header of the block that is being processed
-  // this is needed for v3 to sync with nova as the header is stored in state
+  // this is needed for v3 to sync with multiplexer as the header is stored in state
   tendermint.types.Header header = 20;
 }
 
@@ -202,7 +202,7 @@ message RequestFinalizeBlock {
   bytes proposer_address = 8;
 
   // header is the header of the block that is being finalized
-  // this is needed for v3 to sync with nova as the header is stored in state
+  // this is needed for v3 to sync with multiplexer as the header is stored in state
   tendermint.types.Header header = 20;
 }
 
diff --git a/rpc/core/blocks.go b/rpc/core/blocks.go
index dd0d29935..2bee4229a 100644
--- a/rpc/core/blocks.go
+++ b/rpc/core/blocks.go
@@ -105,7 +105,7 @@ func (env *Environment) Header(_ *rpctypes.Context, heightPtr *int64) (*ctypes.R
 
 	blockMeta := env.BlockStore.LoadBlockMeta(height)
 	if blockMeta == nil {
-		return &ctypes.ResultHeader{}, nil
+		return nil, fmt.Errorf("height %d is not available", height)
 	}
 
 	return &ctypes.ResultHeader{Header: &blockMeta.Header}, nil
@@ -120,7 +120,7 @@ func (env *Environment) HeaderByHash(_ *rpctypes.Context, hash bytes.HexBytes) (
 
 	blockMeta := env.BlockStore.LoadBlockMetaByHash(hash)
 	if blockMeta == nil {
-		return &ctypes.ResultHeader{}, nil
+		return nil, fmt.Errorf("block with hash %X is not available", hash)
 	}
 
 	return &ctypes.ResultHeader{Header: &blockMeta.Header}, nil
@@ -138,7 +138,7 @@ func (env *Environment) Block(_ *rpctypes.Context, heightPtr *int64) (*ctypes.Re
 	block := env.BlockStore.LoadBlock(height)
 	blockMeta := env.BlockStore.LoadBlockMeta(height)
 	if blockMeta == nil {
-		return &ctypes.ResultBlock{BlockID: types.BlockID{}, Block: block}, nil
+		return nil, fmt.Errorf("height %d is not available", height)
 	}
 	return &ctypes.ResultBlock{BlockID: blockMeta.BlockID, Block: block}, nil
 }
@@ -148,7 +148,7 @@ func (env *Environment) Block(_ *rpctypes.Context, heightPtr *int64) (*ctypes.Re
 func (env *Environment) BlockByHash(_ *rpctypes.Context, hash []byte) (*ctypes.ResultBlock, error) {
 	block := env.BlockStore.LoadBlockByHash(hash)
 	if block == nil {
-		return &ctypes.ResultBlock{BlockID: types.BlockID{}, Block: nil}, nil
+		return nil, fmt.Errorf("block with hash %X is not available", hash)
 	}
 	// If block is not nil, then blockMeta can't be nil.
 	blockMeta := env.BlockStore.LoadBlockMeta(block.Height)
diff --git a/state/execution.go b/state/execution.go
index dfd3d1398..46378c361 100644
--- a/state/execution.go
+++ b/state/execution.go
@@ -7,6 +7,9 @@ import (
 	"path/filepath"
 	"time"
 
+	"github.com/cometbft/cometbft/libs/trace"
+	"github.com/cometbft/cometbft/libs/trace/schema"
+
 	abci "github.com/cometbft/cometbft/abci/types"
 	cryptoenc "github.com/cometbft/cometbft/crypto/encoding"
 	"github.com/cometbft/cometbft/libs/fail"
@@ -47,6 +50,9 @@ type BlockExecutor struct {
 
 	// root directory for debug file saving
 	rootDir string
+
+	// tracer optional tracer
+	tracer trace.Tracer
 }
 
 type BlockExecutorOption func(executor *BlockExecutor)
@@ -63,6 +69,12 @@ func BlockExecutorWithRootDir(rootDir string) BlockExecutorOption {
 	}
 }
 
+func BlockExecutorWithTracer(tracer trace.Tracer) BlockExecutorOption {
+	return func(blockExec *BlockExecutor) {
+		blockExec.tracer = tracer
+	}
+}
+
 // NewBlockExecutor returns a new BlockExecutor with a NopEventBus.
 // Call SetEventBus to provide one.
 func NewBlockExecutor(
@@ -83,6 +95,7 @@ func NewBlockExecutor(
 		logger:     logger,
 		metrics:    NopMetrics(),
 		blockStore: blockStore,
+		tracer:     trace.NoOpTracer(),
 	}
 
 	for _, option := range options {
@@ -164,7 +177,9 @@ func (blockExec *BlockExecutor) CreateProposalBlock(
 			}
 		}()
 
+		schema.WriteABCI(blockExec.tracer, schema.PrepareProposalStart, block.Height, -1)
 		rpp, err = blockExec.proxyApp.PrepareProposal(ctx, req)
+		schema.WriteABCI(blockExec.tracer, schema.PrepareProposalEnd, block.Height, -1)
 	}()
 	if err != nil {
 		// For non-panic errors, also save the failed proposal block
@@ -231,7 +246,7 @@ func (blockExec *BlockExecutor) ProcessProposal(
 		ProposerAddress:    block.ProposerAddress,
 		NextValidatorsHash: block.NextValidatorsHash,
 
-		// needed for v3 to sync with nova as the header is stored in state
+		// needed for v3 to sync with multiplexer as the header is stored in state
 		Header: pbHeader,
 	})
 	if err != nil {
@@ -304,7 +319,7 @@ func (blockExec *BlockExecutor) applyBlock(state State, blockID types.BlockID, b
 		Misbehavior:        block.Evidence.Evidence.ToABCI(),
 		Txs:                txs,
 
-		// needed for v3 to sync with nova as the header is stored in state
+		// needed for v3 to sync with multiplexer as the header is stored in state
 		Header: pbHeader,
 	})
 	endTime := time.Now().UnixNano()
@@ -854,7 +869,7 @@ func ExecCommitBlock(
 		Misbehavior:        block.Evidence.Evidence.ToABCI(),
 		Txs:                block.Txs.ToSliceOfBytes(),
 
-		// needed for v3 to sync with nova as the header is stored in state
+		// needed for v3 to sync with multiplexer as the header is stored in state
 		Header: pbHeader,
 	})
 	if err != nil {


Test patch:
diff --git a/p2p/pex/pex_reactor_test.go b/p2p/pex/pex_reactor_test.go
index 41948a059..44197cc5d 100644
--- a/p2p/pex/pex_reactor_test.go
+++ b/p2p/pex/pex_reactor_test.go
@@ -171,6 +171,92 @@ func TestPEXReactorRequestMessageAbuse(t *testing.T) {
 	assert.True(t, book.IsBanned(peerAddr))
 }
 
+func TestPEXReactorRequestIntervalValidation(t *testing.T) {
+	r, book := createReactor(&ReactorConfig{})
+	defer teardownReactor(book)
+
+	// Set a shorter ensurePeersPeriod to avoid long test times
+	// Using 3 seconds instead of default 30 seconds
+	testEnsurePeersPeriod := 3 * time.Second
+	r.SetEnsurePeersPeriod(testEnsurePeersPeriod)
+
+	sw := createSwitchAndAddReactors(r)
+	sw.SetAddrBook(book)
+
+	peer := mock.NewPeer(nil)
+	peerAddr := peer.SocketAddr()
+	p2p.AddPeerToSwitchPeerSet(sw, peer)
+	assert.True(t, sw.Peers().Has(peer.ID()))
+	err := book.AddAddress(peerAddr, peerAddr)
+	require.NoError(t, err)
+	require.True(t, book.HasAddress(peerAddr))
+
+	id := string(peer.ID())
+
+	// With ensurePeersPeriod of 3s, minReceiveRequestInterval should be 1s (3s / 3)
+	expectedMinInterval := testEnsurePeersPeriod / 3
+	assert.Equal(t, expectedMinInterval, r.minReceiveRequestInterval())
+
+	// First request should always be accepted (creates the entry)
+	r.Receive(p2p.Envelope{ChannelID: PexChannel, Src: peer, Message: &tmp2p.PexRequest{}})
+	assert.True(t, r.lastReceivedRequests.Has(id))
+	assert.True(t, sw.Peers().Has(peer.ID()))
+
+	// Second request should also be accepted (sets the lastReceived time)
+	r.Receive(p2p.Envelope{ChannelID: PexChannel, Src: peer, Message: &tmp2p.PexRequest{}})
+	assert.True(t, r.lastReceivedRequests.Has(id))
+	assert.True(t, sw.Peers().Has(peer.ID()))
+
+	// Third request sent too soon (within minInterval) should be rejected
+	// Sleep for half the minimum interval to ensure we're too early
+	time.Sleep(expectedMinInterval / 2)
+	r.Receive(p2p.Envelope{ChannelID: PexChannel, Src: peer, Message: &tmp2p.PexRequest{}})
+	assert.False(t, r.lastReceivedRequests.Has(id), "Peer should be disconnected for sending request too soon")
+	assert.False(t, sw.Peers().Has(peer.ID()), "Peer should be removed from switch")
+	assert.True(t, book.IsBanned(peerAddr), "Peer should be banned")
+}
+
+func TestPEXReactorRequestIntervalAccepted(t *testing.T) {
+	r, book := createReactor(&ReactorConfig{})
+	defer teardownReactor(book)
+
+	// Set a shorter ensurePeersPeriod to avoid long test times
+	// Using 3 seconds instead of default 30 seconds
+	testEnsurePeersPeriod := 3 * time.Second
+	r.SetEnsurePeersPeriod(testEnsurePeersPeriod)
+
+	sw := createSwitchAndAddReactors(r)
+	sw.SetAddrBook(book)
+
+	peer := mock.NewPeer(nil)
+	peerAddr := peer.SocketAddr()
+	p2p.AddPeerToSwitchPeerSet(sw, peer)
+	assert.True(t, sw.Peers().Has(peer.ID()))
+	err := book.AddAddress(peerAddr, peerAddr)
+	require.NoError(t, err)
+	require.True(t, book.HasAddress(peerAddr))
+
+	id := string(peer.ID())
+
+	// With ensurePeersPeriod of 3s, minReceiveRequestInterval should be 1s (3s / 3)
+	expectedMinInterval := testEnsurePeersPeriod / 3
+	assert.Equal(t, expectedMinInterval, r.minReceiveRequestInterval())
+
+	// First and second requests for initialization
+	r.Receive(p2p.Envelope{ChannelID: PexChannel, Src: peer, Message: &tmp2p.PexRequest{}})
+	r.Receive(p2p.Envelope{ChannelID: PexChannel, Src: peer, Message: &tmp2p.PexRequest{}})
+	assert.True(t, sw.Peers().Has(peer.ID()))
+
+	// Wait for the minimum interval to pass
+	time.Sleep(expectedMinInterval + 10*time.Millisecond)
+
+	// Third request sent after appropriate interval should be accepted
+	r.Receive(p2p.Envelope{ChannelID: PexChannel, Src: peer, Message: &tmp2p.PexRequest{}})
+	assert.True(t, r.lastReceivedRequests.Has(id), "Peer should remain connected after sending request at proper interval")
+	assert.True(t, sw.Peers().Has(peer.ID()), "Peer should remain in switch")
+	assert.False(t, book.IsBanned(peerAddr), "Peer should not be banned")
+}
+
 func TestPEXReactorAddrsMessageAbuse(t *testing.T) {
 	r, book := createReactor(&ReactorConfig{})
 	defer teardownReactor(book)
diff --git a/rpc/core/blocks_test.go b/rpc/core/blocks_test.go
index 9db43415d..7eb61da98 100644
--- a/rpc/core/blocks_test.go
+++ b/rpc/core/blocks_test.go
@@ -127,6 +127,72 @@ func TestBlockResults(t *testing.T) {
 	}
 }
 
+func TestMissingBlocks(t *testing.T) {
+	env := &Environment{}
+	env.StateStore = sm.NewStore(dbm.NewMemDB(), sm.StoreOptions{
+		DiscardABCIResponses: false,
+	})
+
+	mockstore := &mocks.BlockStore{}
+	mockstore.On("Height").Return(int64(100))
+	mockstore.On("Base").Return(int64(10))
+	// Simulate missing/pruned blocks by returning nil
+	mockstore.On("LoadBlockMeta", int64(50)).Return(nil)
+	mockstore.On("LoadBlock", int64(50)).Return(nil)
+	mockstore.On("LoadBlockMetaByHash", []byte("missing")).Return(nil)
+	mockstore.On("LoadBlockByHash", []byte("missing")).Return(nil)
+	env.BlockStore = mockstore
+
+	testCases := []struct {
+		name       string
+		testFunc   func() error
+		wantErrMsg string
+	}{
+		{
+			name: "Header missing block",
+			testFunc: func() error {
+				height := int64(50)
+				_, err := env.Header(&rpctypes.Context{}, &height)
+				return err
+			},
+			wantErrMsg: "not available",
+		},
+		{
+			name: "HeaderByHash missing block",
+			testFunc: func() error {
+				_, err := env.HeaderByHash(&rpctypes.Context{}, []byte("missing"))
+				return err
+			},
+			wantErrMsg: "not available",
+		},
+		{
+			name: "Block missing block",
+			testFunc: func() error {
+				height := int64(50)
+				_, err := env.Block(&rpctypes.Context{}, &height)
+				return err
+			},
+			wantErrMsg: "not available",
+		},
+		{
+			name: "BlockByHash missing block",
+			testFunc: func() error {
+				_, err := env.BlockByHash(&rpctypes.Context{}, []byte("missing"))
+				return err
+			},
+			wantErrMsg: "not available",
+		},
+	}
+
+	for _, tc := range testCases {
+		t.Run(tc.name, func(t *testing.T) {
+			err := tc.testFunc()
+			require.Error(t, err)
+			assert.Contains(t, err.Error(), tc.wantErrMsg)
+		})
+	}
+}
+
 func TestEncodeDataRootTuple(t *testing.T) {
 	height := uint64(2)
 	dataRoot, err := hex.DecodeString("82dc1607d84557d3579ce602a45f5872e821c36dbda7ec926dfa17ebc8d5c013")
diff --git a/rpc/jsonrpc/jsonrpc_test.go b/rpc/jsonrpc/jsonrpc_test.go
index a9af35ad4..3f55811cf 100644
--- a/rpc/jsonrpc/jsonrpc_test.go
+++ b/rpc/jsonrpc/jsonrpc_test.go
@@ -20,6 +20,7 @@ import (
 
 	cmtbytes "github.com/cometbft/cometbft/libs/bytes"
 	"github.com/cometbft/cometbft/libs/log"
+	cmtnet "github.com/cometbft/cometbft/libs/net"
 	cmtrand "github.com/cometbft/cometbft/libs/rand"
 
 	client "github.com/cometbft/cometbft/rpc/jsonrpc/client"
@@ -29,8 +30,6 @@ import (
 
 // Client and Server should work over tcp or unix sockets
 const (
-	tcpAddr = "tcp://127.0.0.1:47768"
-
 	unixSocket = "/tmp/rpc_test.sock"
 	unixAddr   = "unix://" + unixSocket
 
@@ -39,6 +38,10 @@ const (
 	testVal = "acbd"
 )
 
+var (
+	tcpAddr string
+)
+
 var ctx = context.Background()
 
 type ResultEcho struct {
@@ -120,10 +123,17 @@ var colorFn = func(keyvals ...interface{}) term.FgBgColor {
 
 // launch unix and tcp servers
 func setup() {
+	// Get a free port for TCP server
+	port, err := cmtnet.GetFreePort()
+	if err != nil {
+		panic(err)
+	}
+	tcpAddr = fmt.Sprintf("tcp://127.0.0.1:%d", port)
+
 	logger := log.NewTMLoggerWithColorFn(log.NewSyncWriter(os.Stdout), colorFn)
 
 	cmd := exec.Command("rm", "-f", unixSocket)
-	err := cmd.Start()
+	err = cmd.Start()
 	if err != nil {
 		panic(err)
 	}