Repository: commonwarexyz/monorepo
Pull request #799: Remove leftover mentions of protoc/protobuf
Author: login: BrendanChou
Labels: name: documentation
description: Improvements or additions to documentation
name: code hygiene
Created: 2025-04-22T17:27:10Z | Merged: 2025-04-22T22:53:34Z

Fixes #795 

Linked issue #795: [codec] Remove Dangling Protobuf References
* https://github.com/commonwarexyz/monorepo/blob/93a6a12e0d065e4d34fa40fdbc195b2e4fc8f039/p2p/src/authenticated/mod.rs#L15-L82
* https://github.com/commonwarexyz/monorepo/blob/4c291da233780a6241c99ee59d982ac99419b958/p2p/src/authenticated/network.rs#L21-L33
* https://github.com/commonwarexyz/monorepo/blob/93a6a12e0d065e4d34fa40fdbc195b2e4fc8f039/p2p/src/authenticated/config.rs#L38
* https://github.com/commonwarexyz/monorepo/blob/93a6a12e0d065e4d34fa40fdbc195b2e4fc8f039/.github/actions/setup/action.yml#L28-L46
* https://github.com/commonwarexyz/monorepo/blob/93a6a12e0d065e4d34fa40fdbc195b2e4fc8f039/.github/actions/setup/action.yml#L6-L9

Diff:
diff --git a/.github/actions/setup/action.yml b/.github/actions/setup/action.yml
index b8d2cc7b21..f83c9ffa49 100644
--- a/.github/actions/setup/action.yml
+++ b/.github/actions/setup/action.yml
@@ -3,10 +3,6 @@ inputs:
   rust-version:
     description: 'The version of Rust to use in cache keys'
     required: true
-  protoc-version:
-    description: 'The version of protoc to use in cache keys'
-    default: '28.2'
-    required: false
 runs:
   using: 'composite'
   steps:
@@ -25,22 +21,3 @@ runs:
       with:
         path: target
         key: ${{ runner.os }}-cargo-target-${{ hashFiles('**/Cargo.lock') }}-${{ inputs.rust-version }}
-    - name: Create protoc directory
-      run: mkdir -p ~/protoc
-      shell: bash
-    - name: Cache protoc
-      id: cache-protoc
-      uses: actions/cache@v4
-      with:
-        path: ~/protoc
-        key: ${{ runner.os }}-protoc-${{ inputs.protoc-version }}
-    - name: Install protoc
-      if: steps.cache-protoc.outputs.cache-hit != 'true'
-      run: |
-        curl -L -o protoc.zip https://github.com/protocolbuffers/protobuf/releases/download/v${{ inputs.protoc-version }}/protoc-${{ inputs.protoc-version }}-linux-x86_64.zip
-        unzip protoc.zip -d ~/protoc
-        rm protoc.zip
-      shell: bash
-    - name: Add protoc to PATH
-      run: echo "$HOME/protoc/bin" >> $GITHUB_PATH
-      shell: bash
diff --git a/p2p/src/authenticated/config.rs b/p2p/src/authenticated/config.rs
index e04b178a89..945fa99ee9 100644
--- a/p2p/src/authenticated/config.rs
+++ b/p2p/src/authenticated/config.rs
@@ -35,7 +35,7 @@ pub struct Config<C: Scheme> {
     /// Maximum size allowed for messages over any connection.
     ///
     /// The actual size of the network message will be higher due to overhead from the protocol;
-    /// this includes additional metadata, protobuf encoding of the bytes, and cryptographic signatures.
+    /// this may include additional metadata, data from the codec, and/or cryptographic signatures.
     pub max_message_size: usize,
 
     /// Message backlog allowed for internal actors.
diff --git a/p2p/src/authenticated/mod.rs b/p2p/src/authenticated/mod.rs
index 38764b3c31..2d631de2f6 100644
--- a/p2p/src/authenticated/mod.rs
+++ b/p2p/src/authenticated/mod.rs
@@ -7,84 +7,93 @@
 //!
 //! # Features
 //!
-//! * Configurable Cryptography Scheme for Peer Identities (BLS, ed25519, etc.)
-//! * Automatic Peer Discovery Using Bit Vectors (Used as Ping/Pongs)
-//! * Multiplexing With Configurable Rate Limiting Per Channel and Send Prioritization
-//! * Optional Message Compression (using `zstd`)
+//! - Configurable Cryptography Scheme for Peer Identities (BLS, ed25519, etc.)
+//! - Automatic Peer Discovery Using Bit Vectors (Also Used as Ping Messages)
+//! - Multiplexing With Configurable Rate Limiting Per Channel and Send Prioritization
+//! - Optional Message Compression (using `zstd`)
 //!
 //! # Design
 //!
 //! ## Discovery
 //!
-//! Peer discovery relies heavily on the assumption that all peers are known and synchronized at each index (a user-provided tuple of
-//! `(u64, Vec<PublicKey>)`). Using this assumption, we can construct a sorted bit vector that represents our knowledge
-//! of peer IPs (where 1 == we know, 0 == we don't know). This means we can represent our knowledge of 1000 peers in only 125 bytes!
-//!
-//! _If peers at a given index are not synchronized, peers may signal their knowledge of peer IPs that another peer may
-//! incorrectly respond to (associating a given index with a different peer) or fail to respond to (if the bit vector representation
-//! of the set is smaller/larger than expected). It is up to the application to ensure sets are synchronized._
-//!
-//! Because this representation is so efficient/small, peers send bit vectors to each other periodically as a "ping" to keep
-//! the connection alive. Because it may be useful to be connected to multiple indexes of peers at a given time (i.e. to
-//! perform a DKG with a new set of peers), it is possible to configure this crate to maintain connections to multiple
-//! indexes (and pings are a random index we are trying to connect to).
-//!
-//! ```protobuf
-//! message BitVec {
-//!     uint64 index = 1;
-//!     bytes bits = 2;
-//! }
-//! ```
-//!
-//! Upon receiving a bit vector, a peer will select a random collection of peers (under a configured max) that it knows about that the
-//! sender does not. If the sender knows about all peers that we know about, the receiver does nothing (and relies on its bit vector
-//! to serve as a pong to keep the connection alive).
-//!
-//! ```protobuf
-//! message Peers {
-//!     repeated Peer peers = 1;
-//! }
-//! ```
-//!
-//! If a peer learns about an updated address for a peer, it will update the record it has stored (for itself and for future gossip).
-//! This record is created during instantiation and is sent immediately after a connection is established (right after the handshake).
-//! This means that a peer that learned about an outdated record for a peer will update it immediately upon being dialed.
-//!
-//! ```protobuf
-//! message Peer {
-//!     bytes socket = 1;
-//!     uint64 timestamp = 2;
-//!     Signature signature = 3;
-//! }
-//! ```
-//!
-//! To get all of this started, a peer must first be bootstrapped with a list of known peers/addresses. The peer will dial these
-//! other peers, send its own record, send a bit vector (with all 0's except its own position in the sorted list), and then
-//! wait for the other peer to respond with some set of unknown peers. Different peers do not need to agree on who this list of
-//! bootstrapping peers is (this list is configurable). Knowledge of bootstrappers and connections to them are never dropped,
-//! even if the bootstrapper is not in any known peer set.
-//!
-//! _If a peer is not in any registered peer set (to its knowledge) but is dialed by a peer that is, it will accept the connection.
-//! This allows peers that have a more up-to-date version of the peer set to connect, exchange application-level information, and for
-//! the said peer to potentially learn of an updated peer set (of which it is part)._
+//! Peer discovery operates under the assumption that all peers are aware of and synchronized on
+//! the composition of peer sets at specific, user-provided indices (`u64`). Each index maps to a
+//! list of authorized `PublicKey`s (`(u64, Vec<PublicKey>)`). Based on this shared knowledge, each
+//! peer can construct a sorted bit vector message (`BitVec`) representing its knowledge of the
+//! dialable addresses [`SocketAddr`](std::net::SocketAddr) for the peers in that set.
+//! The `BitVec` message contains:
+//! - `index`: The `u64` index the bit vector applies to.
+//! - `bits`: The bit vector itself, where a '1' signifies knowledge of the corresponding
+//!   peer's address in the sorted list for that index.
+//!
+//! _Warning: If peers are not synchronized on the peer set composition at a given index,
+//! discovery messages can be misinterpreted. A peer might associate a bit vector index with the
+//! wrong peer or fail to parse the vector if its length doesn't match the expected set size. The
+//! application layer is responsible for ensuring peer set synchronization._
+//!
+//! Due to their small size, these `BitVec` messages are exchanged periodically (configured by
+//! `gossip_bit_vec_frequency` in the [`Config`]) between connected peers. This serves as both a
+//! peer discovery mechanism and a keep-alive "ping" message to maintain the underlying
+//! connection, especially during periods of low application-level traffic. The protocol supports
+//! tracking multiple peer sets concurrently (up to `tracked_peer_sets`), each identified by its
+//! `index`. This is useful, for instance, during transitions like distributed key generation
+//! (DKG) where connections to both old and new peer sets are needed simultaneously.
+//!
+//! Upon receiving a `BitVec` message, a peer compares it against its own knowledge for the same
+//! index. If the receiving peer knows addresses that the sender marked as '0' (unknown), it
+//! selects a random subset of these known `PeerInfo` structures (up to `peer_gossip_max_count`)
+//! and sends them back in a `Payload::Peers` message. Each `PeerInfo` structure verifies a peer's
+//! address claim and contains:
+//! - `socket`: The [`SocketAddr`](std::net::SocketAddr) of the peer.
+//! - `timestamp`: A `u64` timestamp indicating when the address was attested.
+//! - `public_key`: The peer's public key.
+//! - `signature`: The peer's cryptographic signature over the `socket` and `timestamp`.
+//!
+//! If the receiver doesn't know any addresses the sender is unaware of, it sends no
+//! `Payload::Peers` response; the received `BitVec` implicitly acts as a "pong".
+//!
+//! If a peer receives a `PeerInfo` message (either directly or through gossip) containing a more
+//! recent timestamp for a known peer's address, it updates its local `Record`. This updated
+//! `PeerInfo` is also used in future gossip messages. Each peer generates its own signed
+//! `PeerInfo` upon startup and sends it immediately after establishing a connection (following
+//! the cryptographic handshake). This ensures that if a peer connects using an outdated address
+//! record, it will be corrected promptly by the peer being dialed.
+//!
+//! To initiate the discovery process, a peer needs a list of `bootstrappers` (defined in
+//! [`Config`]) - known peer public keys and their corresponding socket addresses. The peer
+//! attempts to dial these bootstrappers, performs the handshake, sends its own `PeerInfo`, and
+//! then sends a `BitVec` for the relevant peer set(s) (initially only knowing its own address,
+//! marked as '1'). It then waits for responses, learning about other peers through the
+//! `Payload::Peers` messages received. Bootstrapper information is persisted, and connections to
+//! them are maintained even if they aren't part of any currently tracked peer sets. Different
+//! peers can have different bootstrapper lists.
+//!
+//! _Note: If a peer (listener) receives a connection request from another peer (dialer) that
+//! belongs to a registered peer set, the listener will accept the connection, even if the
+//! listener itself hasn't yet learned about that specific peer set (or has an older version). The
+//! core requirement is that the listener recognizes the *dialer's public key* as belonging to
+//! *some* authorized set it tracks (see `actors::tracker::Actor`). This mechanism allows peers
+//! with more up-to-date peer set information to connect and propagate that information, enabling
+//! the listener to potentially learn about newer sets it is part of._
 //!
 //! ## Messages
 //!
-//! Messages are sent using the Data message type. This message type is used to send arbitrary bytes to a given channel.
-//! The message must be smaller (in bytes) than the configured maximum message size. If the message is larger, an error will be returned.
-//! It is possible for a sender to prioritize messages over others.
+//! Application-level data is exchanged using the `Payload::Data` message type, which wraps an
+//! internal `Data` structure. This structure contains:
+//! - `channel`: A `u32` identifier used to route the message to the correct application handler.
+//! - `message`: The arbitrary application payload as `Bytes`.
 //!
-//! ```protobuf
-//! message Data {
-//!     uint32 channel = 1;
-//!     bytes message = 2;
-//! }
-//! ```
+//! The size of the `message` bytes (after potential compression) must not exceed the configured
+//! `max_message_size`. If it does, the sending operation will fail with
+//! [`Error::MessageTooLarge`]. Messages can be sent with `priority`, allowing certain
+//! communications to potentially bypass lower-priority messages waiting in send queues across all
+//! channels. Each registered channel ([`Sender`], [`Receiver`]) handles its own message queuing,
+//! rate limiting, and optional `zstd` compression/decompression.
 //!
 //! # Example
 //!
 //! ```rust
-//! use commonware_p2p::authenticated::{self, Network};
+//! use commonware_p2p::{authenticated::{self, Network}, Sender, Recipients};
 //! use commonware_cryptography::{Ed25519, Signer, Verifier};
 //! use commonware_runtime::{tokio::{self, Executor}, Spawner, Runner, Metrics};
 //! use governor::Quota;
@@ -126,7 +135,7 @@
 //!     signer.clone(),
 //!     application_namespace,
 //!     SocketAddr::new(IpAddr::V4(Ipv4Addr::UNSPECIFIED), 3000),
-//!     SocketAddr::new(IpAddr::V4(Ipv4Addr::UNSPECIFIED), 3000),
+//!     SocketAddr::new(IpAddr::V4(Ipv4Addr::LOCALHOST), 3000), // Use a specific dialable addr
 //!     bootstrappers,
 //!     MAX_MESSAGE_SIZE,
 //! );
@@ -140,12 +149,12 @@
 //!     //
 //!     // In production, this would be updated as new peer sets are created (like when
 //!     // the composition of a validator set changes).
-//!     oracle.register(0, vec![signer.public_key(), peer1, peer2, peer3]);
+//!     oracle.register(0, vec![signer.public_key(), peer1, peer2, peer3]).await;
 //!
 //!     // Register some channel
 //!     const MAX_MESSAGE_BACKLOG: usize = 128;
 //!     const COMPRESSION_LEVEL: Option<i32> = Some(3);
-//!     let (sender, receiver) = network.register(
+//!     let (mut sender, receiver) = network.register(
 //!         0,
 //!         Quota::per_second(NonZeroU32::new(1).unwrap()),
 //!         MAX_MESSAGE_BACKLOG,
@@ -155,7 +164,8 @@
 //!     // Run network
 //!     let network_handler = network.start();
 //!
-//!     // ... Use sender and receiver ...
+//!     // Example: Use sender
+//!     let _ = sender.send(Recipients::All, bytes::Bytes::from_static(b"hello"), false).await;
 //!
 //!     // Shutdown network
 //!     network_handler.abort();
diff --git a/p2p/src/authenticated/network.rs b/p2p/src/authenticated/network.rs
index 81c09e6ebf..5e4af44629 100644
--- a/p2p/src/authenticated/network.rs
+++ b/p2p/src/authenticated/network.rs
@@ -4,6 +4,7 @@ use super::{
     actors::{dialer, listener, router, spawner, tracker},
     channels::{self, Channels},
     config::Config,
+    types,
 };
 use crate::Channel;
 use commonware_cryptography::Scheme;
@@ -18,20 +19,6 @@ use rand::{CryptoRng, Rng};
 use std::marker::PhantomData;
 use tracing::{debug, info, warn};
 
-// The maximum overhead of encoding a `message: Bytes` into a protobuf `message Message`
-// Should be at most 18 bytes for messages under 4GB, but we add a bit of padding.
-//
-// The byte overhead is calculated as follows:
-// - 1  Data field number
-// - 5* Data length varint
-// - 1  Channel field number
-// - 5  Channel value varint
-// - 1  Message field number
-// - 5* Message length varint
-//
-// (*) assumes that the length is no more than 4GB
-const PROTOBUF_OVERHEAD: usize = 64;
-
 /// Unique suffix for all messages signed by the tracker.
 const TRACKER_SUFFIX: &[u8] = b"_TRACKER";
 
@@ -182,7 +169,7 @@ impl<
         let stream_cfg = public_key::Config {
             crypto: self.cfg.crypto,
             namespace: union(&self.cfg.namespace, STREAM_SUFFIX),
-            max_message_size: self.cfg.max_message_size + PROTOBUF_OVERHEAD,
+            max_message_size: self.cfg.max_message_size + types::MAX_PAYLOAD_DATA_OVERHEAD,
             synchrony_bound: self.cfg.synchrony_bound,
             max_handshake_age: self.cfg.max_handshake_age,
             handshake_timeout: self.cfg.handshake_timeout,
diff --git a/p2p/src/authenticated/types.rs b/p2p/src/authenticated/types.rs
index 04a2cec6c4..d460511e5d 100644
--- a/p2p/src/authenticated/types.rs
+++ b/p2p/src/authenticated/types.rs
@@ -6,6 +6,17 @@ use commonware_cryptography::Verifier;
 use commonware_utils::BitVec as UtilsBitVec;
 use std::net::SocketAddr;
 
+/// The maximum overhead (in bytes) when encoding a `message` into a [`Payload::Data`].
+///
+/// The byte overhead is calculated as the sum of the following:
+/// - 1: Payload enum value
+/// - 5: Channel varint
+/// - 5: Message length varint (lengths longer than 32 bits are forbidden by the codec)
+pub const MAX_PAYLOAD_DATA_OVERHEAD: usize = 1 + 5 + 5;
+
+/// Configuration when deserializing messages.
+///
+/// This is used to limit the size of the messages received from peers.
 #[derive(Clone)]
 pub struct Config {
     /// The maximum number of peers that can be sent in a `Peers` message.
@@ -15,7 +26,7 @@ pub struct Config {
     pub max_bitvec: usize,
 }
 
-// Payload is the only allowed message format that can be sent between peers.
+/// Payload is the only allowed message format that can be sent between peers.
 #[derive(Clone, Debug, PartialEq)]
 pub enum Payload<C: Verifier> {
     /// Bit vector that represents the peers a peer knows about.
@@ -182,7 +193,7 @@ impl<C: Verifier> Read for PeerInfo<C> {
     }
 }
 
-// Data is an arbitrary message sent between peers.
+/// Data is an arbitrary message sent between peers.
 #[derive(Clone, Debug, PartialEq)]
 pub struct Data {
     /// A unique identifier for the channel the message is sent on.
@@ -339,4 +350,18 @@ mod tests {
         let result = Payload::<Secp256r1>::decode_cfg(&invalid_payload[..], &cfg);
         assert!(result.is_err());
     }
+
+    #[test]
+    fn test_max_payload_data_overhead() {
+        let message = Bytes::from(vec![0; 1 << 29]);
+        let message_len = message.len();
+        let payload = Payload::<Secp256r1>::Data(Data {
+            channel: u32::MAX,
+            message,
+        });
+        assert_eq!(
+            payload.encode_size(),
+            message_len + MAX_PAYLOAD_DATA_OVERHEAD
+        );
+    }
 }


Reviews:
user: login: BrendanChou
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-04-22T17:28:06Z
user: login: patrick-ogrady
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-22T17:51:16Z
user: login: patrick-ogrady
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-22T17:52:03Z
user: login: patrick-ogrady
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-22T17:53:11Z
user: login: patrick-ogrady
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-22T20:18:02Z
user: login: patrick-ogrady
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-22T20:19:43Z
user: login: patrick-ogrady
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-22T22:45:46Z
user: login: patrick-ogrady
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-04-22T22:46:00Z

Review comments:
diff_hunk: @@ -182,7 +168,7 @@ impl<
         let stream_cfg = public_key::Config {
             crypto: self.cfg.crypto,
             namespace: union(&self.cfg.namespace, STREAM_SUFFIX),
-            max_message_size: self.cfg.max_message_size + PROTOBUF_OVERHEAD,
path: p2p/src/authenticated/network.rs
user: login: BrendanChou
body: still investigating whether this is safe to do or wether we need to add padding for stuff like the channel, etc
created_at: 2025-04-22T17:28:06Z
author_association: COLLABORATOR
diff_hunk: @@ -25,39 +25,18 @@
 //! of the set is smaller/larger than expected). It is up to the application to ensure sets are synchronized._
 //!
 //! Because this representation is so efficient/small, peers send bit vectors to each other periodically as a "ping" to keep
-//! the connection alive. Because it may be useful to be connected to multiple indexes of peers at a given time (i.e. to
-//! perform a DKG with a new set of peers), it is possible to configure this crate to maintain connections to multiple
-//! indexes (and pings are a random index we are trying to connect to).
-//!
-//! ```protobuf
-//! message BitVec {
-//!     uint64 index = 1;
-//!     bytes bits = 2;
-//! }
-//! ```
+//! the connection alive. It may be useful to be connected to multiple indexes of peers at a given time; for example to
+//! perform a DKG with a new set of peers while still be connected to an old set of peers. Thus, we allow maintaining connections
+//! to multiple indexes. The index is included in the message as a `u64`.
 //!
 //! Upon receiving a bit vector, a peer will select a random collection of peers (under a configured max) that it knows about that the
 //! sender does not. If the sender knows about all peers that we know about, the receiver does nothing (and relies on its bit vector
 //! to serve as a pong to keep the connection alive).
 //!
-//! ```protobuf
-//! message Peers {
-//!     repeated Peer peers = 1;
-//! }
-//! ```
-//!
 //! If a peer learns about an updated address for a peer, it will update the record it has stored (for itself and for future gossip).
 //! This record is created during instantiation and is sent immediately after a connection is established (right after the handshake).
 //! This means that a peer that learned about an outdated record for a peer will update it immediately upon being dialed.
 //!
-//! ```protobuf
path: p2p/src/authenticated/mod.rs
user: login: patrick-ogrady
body: I think we should retain some list/diagram of what goes in each of these messages. A number of folks have reached out to tell me how useful these are to understanding what is going on.
created_at: 2025-04-22T17:51:16Z
author_association: CONTRIBUTOR
diff_hunk: @@ -25,39 +25,18 @@
 //! of the set is smaller/larger than expected). It is up to the application to ensure sets are synchronized._
 //!
 //! Because this representation is so efficient/small, peers send bit vectors to each other periodically as a "ping" to keep
-//! the connection alive. Because it may be useful to be connected to multiple indexes of peers at a given time (i.e. to
-//! perform a DKG with a new set of peers), it is possible to configure this crate to maintain connections to multiple
-//! indexes (and pings are a random index we are trying to connect to).
-//!
-//! ```protobuf
-//! message BitVec {
-//!     uint64 index = 1;
-//!     bytes bits = 2;
-//! }
-//! ```
+//! the connection alive. It may be useful to be connected to multiple indexes of peers at a given time; for example to
+//! perform a DKG with a new set of peers while still be connected to an old set of peers. Thus, we allow maintaining connections
+//! to multiple indexes. The index is included in the message as a `u64`.
 //!
 //! Upon receiving a bit vector, a peer will select a random collection of peers (under a configured max) that it knows about that the
 //! sender does not. If the sender knows about all peers that we know about, the receiver does nothing (and relies on its bit vector
 //! to serve as a pong to keep the connection alive).
 //!
-//! ```protobuf
-//! message Peers {
-//!     repeated Peer peers = 1;
-//! }
-//! ```
-//!
 //! If a peer learns about an updated address for a peer, it will update the record it has stored (for itself and for future gossip).
 //! This record is created during instantiation and is sent immediately after a connection is established (right after the handshake).
 //! This means that a peer that learned about an outdated record for a peer will update it immediately upon being dialed.
 //!
-//! ```protobuf
path: p2p/src/authenticated/mod.rs
user: login: patrick-ogrady
body: Whatever we do should probably be standardized with what is in `stream::public_key`:

https://github.com/commonwarexyz/monorepo/blob/d62e3270a3970b2c1c0ab5863e0e04e844873b23/stream/src/public_key/mod.rs#L28-L30
created_at: 2025-04-22T17:52:02Z
author_association: CONTRIBUTOR
diff_hunk: @@ -182,7 +168,7 @@ impl<
         let stream_cfg = public_key::Config {
             crypto: self.cfg.crypto,
             namespace: union(&self.cfg.namespace, STREAM_SUFFIX),
-            max_message_size: self.cfg.max_message_size + PROTOBUF_OVERHEAD,
path: p2p/src/authenticated/network.rs
user: login: patrick-ogrady
body: AFAIK this was just to account for size above what we could "see". If there is a way to add a test for this by creating messages at the limit (and one past), that would be phenomenal.
created_at: 2025-04-22T17:53:11Z
author_association: CONTRIBUTOR
diff_hunk: @@ -18,19 +18,14 @@ use rand::{CryptoRng, Rng};
 use std::marker::PhantomData;
 use tracing::{debug, info, warn};
 
-// The maximum overhead of encoding a `message: Bytes` into a protobuf `message Message`
-// Should be at most 18 bytes for messages under 4GB, but we add a bit of padding.
-//
-// The byte overhead is calculated as follows:
-// - 1  Data field number
-// - 5* Data length varint
-// - 1  Channel field number
-// - 5  Channel value varint
-// - 1  Message field number
-// - 5* Message length varint
-//
-// (*) assumes that the length is no more than 4GB
-const PROTOBUF_OVERHEAD: usize = 64;
+/// The maximum overhead of encoding a `message: Bytes` into a full
+/// [Payload](super::types::Payload), in bytes.
+///
+/// The byte overhead is calculated as the sum of the following:
+/// - 1: Payload enum value
+/// - 5: Channel varint
+/// - 5: Message length varint (assumes length is no more than 4GB)
+const MAX_PAYLOAD_OVERHEAD: usize = 11;
path: p2p/src/authenticated/network.rs
user: login: patrick-ogrady
body: Can we pull this from the `types` or `wire` file instead of maintaining it here?

I think that would also provide a great opportunity to assert that an example messages has less than this overhead over the raw payload.
created_at: 2025-04-22T20:18:02Z
author_association: CONTRIBUTOR
diff_hunk: @@ -126,7 +135,7 @@
 //!     signer.clone(),
 //!     application_namespace,
 //!     SocketAddr::new(IpAddr::V4(Ipv4Addr::UNSPECIFIED), 3000),
-//!     SocketAddr::new(IpAddr::V4(Ipv4Addr::UNSPECIFIED), 3000),
+//!     SocketAddr::new(IpAddr::V4(Ipv4Addr::LOCALHOST), 3000), // Use a specific dialable addr
path: p2p/src/authenticated/mod.rs
user: login: patrick-ogrady
body: 🙏 
created_at: 2025-04-22T20:19:43Z
author_association: CONTRIBUTOR
diff_hunk: @@ -339,4 +350,18 @@ mod tests {
         let result = Payload::<Secp256r1>::decode_cfg(&invalid_payload[..], &cfg);
         assert!(result.is_err());
     }
+
+    #[test]
+    fn test_max_payload_data_overhead() {
path: p2p/src/authenticated/types.rs
user: login: patrick-ogrady
body: ❤️ 
created_at: 2025-04-22T22:45:46Z
author_association: CONTRIBUTOR

PR comments:
user: login: codecov[bot]
created_at: 2025-04-22T22:53:50Z
body: ## [Codecov](https://app.codecov.io/gh/commonwarexyz/monorepo/pull/799?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=commonwarexyz) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 89.42%. Comparing base [(`4c291da`)](https://app.codecov.io/gh/commonwarexyz/monorepo/commit/4c291da233780a6241c99ee59d982ac99419b958?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=commonwarexyz) to head [(`e708077`)](https://app.codecov.io/gh/commonwarexyz/monorepo/commit/e7080779bb83b58e01fea24ed6f17a3ac3f07935?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=commonwarexyz).
> Report is 5 commits behind head on main.

```diff
@@            Coverage Diff             @@
##             main     #799      +/-   ##
==========================================
+ Coverage   89.40%   89.42%   +0.02%     
==========================================
  Files         166      166              
  Lines       41136    41314     +178     
==========================================
+ Hits        36776    36947     +171     
- Misses       4360     4367       +7     
```

| [Files with missing lines](https://app.codecov.io/gh/commonwarexyz/monorepo/pull/799?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=commonwarexyz) | Coverage Δ | |
|---|---|---|
| [p2p/src/authenticated/config.rs](https://app.codecov.io/gh/commonwarexyz/monorepo/pull/799?src=pr&el=tree&filepath=p2p%2Fsrc%2Fauthenticated%2Fconfig.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=commonwarexyz#diff-cDJwL3NyYy9hdXRoZW50aWNhdGVkL2NvbmZpZy5ycw==) | `31.95% <ø> (ø)` | |
| [p2p/src/authenticated/mod.rs](https://app.codecov.io/gh/commonwarexyz/monorepo/pull/799?src=pr&el=tree&filepath=p2p%2Fsrc%2Fauthenticated%2Fmod.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=commonwarexyz#diff-cDJwL3NyYy9hdXRoZW50aWNhdGVkL21vZC5ycw==) | `100.00% <ø> (ø)` | |
| [p2p/src/authenticated/network.rs](https://app.codecov.io/gh/commonwarexyz/monorepo/pull/799?src=pr&el=tree&filepath=p2p%2Fsrc%2Fauthenticated%2Fnetwork.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=commonwarexyz#diff-cDJwL3NyYy9hdXRoZW50aWNhdGVkL25ldHdvcmsucnM=) | `80.85% <100.00%> (ø)` | |
| [p2p/src/authenticated/types.rs](https://app.codecov.io/gh/commonwarexyz/monorepo/pull/799?src=pr&el=tree&filepath=p2p%2Fsrc%2Fauthenticated%2Ftypes.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=commonwarexyz#diff-cDJwL3NyYy9hdXRoZW50aWNhdGVkL3R5cGVzLnJz) | `98.46% <100.00%> (+0.09%)` | :arrow_up: |

... and [19 files with indirect coverage changes](https://app.codecov.io/gh/commonwarexyz/monorepo/pull/799/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=commonwarexyz)

------

[Continue to review full report in Codecov by Sentry](https://app.codecov.io/gh/commonwarexyz/monorepo/pull/799?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=commonwarexyz).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=commonwarexyz)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://app.codecov.io/gh/commonwarexyz/monorepo/pull/799?dropdown=coverage&src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=commonwarexyz). Last update [4c291da...e708077](https://app.codecov.io/gh/commonwarexyz/monorepo/pull/799?dropdown=coverage&src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=commonwarexyz). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=commonwarexyz).

<details><summary> :rocket: New features to boost your workflow: </summary>

- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
author_association: NONE