Repository: waku-org/nwaku
Pull request #3346: feat: introduce `preset` option
Author: login: fryorcraken
Created: 2025-03-29T04:50:41Z | Merged: 2025-03-31T22:28:19Z

# Description

Introduce `--preset` configuration to explicitly override a set of config value based on the target Waku network (e.g. TWN, Status, Railgun).
Adding more preset will be done in separate PR.
Overwriting config from cluster-id will be deprecated as a second step.

# Changes

- [x] introduce `--preset` config option
- [x] introduced tests for config matters

# How to test

1. Run `wakunode2` with `--preset=twn` and confirms it can connect to The Waku Network
2. Run `wakunode2` with `--cluster-id=1` and confirms it can **still** connect to The Waku Network (to be deprecated)
3. Run `wakunode2` with Status network parameters and cofirm it can still connect to Status network

## Notes

A redo of #2859 but with a smaller scope.
I added a number of TODO that I will address in follow-up PRs.


Diff:
diff --git a/tests/all_tests_waku.nim b/tests/all_tests_waku.nim
index 39ac57cafd..3e847ae86b 100644
--- a/tests/all_tests_waku.nim
+++ b/tests/all_tests_waku.nim
@@ -30,7 +30,7 @@ import
 const os* {.strdefine.} = ""
 when os == "Linux" and
     # GitHub only supports container actions on Linux
-    # and we need to start a postgress database in a docker container
+    # and we need to start a postgres database in a docker container
     defined(postgres):
   import
     ./waku_archive/test_driver_postgres_query,
@@ -106,3 +106,4 @@ import
 import ./waku_rln_relay/test_all
 
 # Node Factory
+import ./factory/test_config
diff --git a/tests/factory/test_config.nim b/tests/factory/test_config.nim
new file mode 100644
index 0000000000..4a1f6d78fd
--- /dev/null
+++ b/tests/factory/test_config.nim
@@ -0,0 +1,157 @@
+{.used.}
+
+import
+  std/options,
+  testutils/unittests,
+  chronos,
+  libp2p/crypto/[crypto, secp],
+  libp2p/multiaddress,
+  nimcrypto/utils,
+  secp256k1,
+  confutils
+import
+  ../../waku/factory/external_config,
+  ../../waku/factory/internal_config,
+  ../../waku/factory/networks_config,
+  ../../waku/common/logging
+
+suite "Waku config - apply preset":
+  test "Default preset is TWN":
+    ## Setup
+    let expectedConf = ClusterConf.TheWakuNetworkConf()
+
+    ## Given
+    let preConfig = WakuNodeConf(cmd: noCommand, preset: "default")
+
+    ## When
+    let res = applyPresetConfiguration(preConfig)
+    assert res.isOk(), $res.error
+
+    ## Then
+    let conf = res.get()
+    assert conf.maxMessageSize == expectedConf.maxMessageSize
+    assert conf.clusterId == expectedConf.clusterId
+    assert conf.rlnRelay == expectedConf.rlnRelay
+    assert conf.rlnRelayEthContractAddress == expectedConf.rlnRelayEthContractAddress
+    assert conf.rlnRelayDynamic == expectedConf.rlnRelayDynamic
+    assert conf.rlnRelayChainId == expectedConf.rlnRelayChainId
+    assert conf.rlnRelayBandwidthThreshold == expectedConf.rlnRelayBandwidthThreshold
+    assert conf.rlnEpochSizeSec == expectedConf.rlnEpochSizeSec
+    assert conf.rlnRelayUserMessageLimit == expectedConf.rlnRelayUserMessageLimit
+    assert conf.numShardsInNetwork == expectedConf.numShardsInNetwork
+    assert conf.discv5BootstrapNodes == expectedConf.discv5BootstrapNodes
+
+  test "Subscribes to all valid shards in default network":
+    ## Setup
+    let expectedConf = ClusterConf.TheWakuNetworkConf()
+
+    ## Given
+    let shards: seq[uint16] = @[0, 1, 2, 3, 4, 5, 6, 7]
+    let preConfig = WakuNodeConf(cmd: noCommand, preset: "default", shards: shards)
+
+    ## When
+    let res = applyPresetConfiguration(preConfig)
+    assert res.isOk(), $res.error
+
+    ## Then
+    let conf = res.get()
+    assert conf.shards.len == expectedConf.numShardsInNetwork.int
+
+  test "Subscribes to some valid shards in default network":
+    ## Setup
+    let expectedConf = ClusterConf.TheWakuNetworkConf()
+
+    ## Given
+    let shards: seq[uint16] = @[0, 4, 7]
+    let preConfig = WakuNodeConf(cmd: noCommand, preset: "default", shards: shards)
+
+    ## When
+    let resConf = applyPresetConfiguration(preConfig)
+    let res = validateShards(resConf.get())
+    assert res.isOk(), $res.error
+
+    ## Then
+    let conf = resConf.get()
+    assert conf.shards.len() == shards.len()
+    for index, shard in shards:
+      assert shard in conf.shards
+
+  test "Subscribes to invalid shards in default network":
+    ## Setup
+
+    ## Given
+    let shards: seq[uint16] = @[0, 4, 7, 10]
+    let preConfig = WakuNodeConf(cmd: noCommand, preset: "default", shards: shards)
+    let postConfig = applyPresetConfiguration(preConfig)
+
+    ## When
+    let res = validateShards(postConfig.get())
+
+    ## Then
+    assert res.isErr(), "Invalid shard was accepted"
+
+suite "Waku config - node key":
+  test "Passed node key is used":
+    ## Setup
+    let nodeKeyStr =
+      "0011223344556677889900aabbccddeeff0011223344556677889900aabbccddeeff"
+    let nodekey = block:
+      let key = SkPrivateKey.init(utils.fromHex(nodeKeyStr)).tryGet()
+      crypto.PrivateKey(scheme: Secp256k1, skkey: key)
+
+    ## Given
+    let config = WakuNodeConf.load(version = "", cmdLine = @["--nodekey=" & nodeKeyStr])
+
+    ## When
+    let res = getNodeKey(config)
+    assert res.isOk(), $res.error
+
+    ## Then
+    let resKey = res.get()
+    assert utils.toHex(resKey.getRawBytes().get()) ==
+      utils.toHex(nodekey.getRawBytes().get())
+
+suite "Waku config - Shards":
+  test "Shards are valid":
+    ## Setup
+
+    ## Given
+    let shards: seq[uint16] = @[0, 2, 4]
+    let numShardsInNetwork = 5.uint32
+    let config = WakuNodeConf(
+      cmd: noCommand, shards: shards, numShardsInNetwork: numShardsInNetwork
+    )
+
+    ## When
+    let res = validateShards(config)
+
+    ## Then
+    assert res.isOk(), $res.error
+
+  test "Shards are not in range":
+    ## Setup
+
+    ## Given
+    let shards: seq[uint16] = @[0, 2, 5]
+    let numShardsInNetwork = 5.uint32
+    let config = WakuNodeConf(
+      cmd: noCommand, shards: shards, numShardsInNetwork: numShardsInNetwork
+    )
+
+    ## When
+    let res = validateShards(config)
+
+    ## Then
+    assert res.isErr(), "Invalid shard was accepted"
+
+  test "Shard is passed without num shards":
+    ## Setup
+
+    ## Given
+    let config = WakuNodeConf.load(version = "", cmdLine = @["--shard=32"])
+
+    ## When
+    let res = validateShards(config)
+
+    ## Then
+    assert res.isOk(), $res.error
diff --git a/waku/factory/external_config.nim b/waku/factory/external_config.nim
index 9bc073426b..100d1b6446 100644
--- a/waku/factory/external_config.nim
+++ b/waku/factory/external_config.nim
@@ -76,7 +76,7 @@ type WakuNodeConf* = object
   .}: EthRpcUrl
 
   rlnRelayEthContractAddress* {.
-    desc: "Address of membership contract on an Ethereum testnet",
+    desc: "Address of membership contract on an Ethereum testnet.",
     defaultValue: "",
     name: "rln-relay-eth-contract-address"
   .}: string
@@ -100,6 +100,7 @@ type WakuNodeConf* = object
     name: "rln-relay-eth-private-key"
   .}: string
 
+  # TODO: Remove "Default is" when it's already visible on the CLI
   rlnRelayUserMessageLimit* {.
     desc:
       "Set a user message limit for the rln membership registration. Must be a positive integer. Default is 1.",
@@ -145,6 +146,13 @@ type WakuNodeConf* = object
     .}: seq[ProtectedShard]
 
     ## General node config
+    preset* {.
+      desc:
+        "Network preset to use. 'twn' is The RLN-protected Waku Network (cluster 1).",
+      defaultValue: "",
+      name: "preset"
+    .}: string
+
     clusterId* {.
       desc:
         "Cluster id that the node is running in. Node in a different cluster id is disconnected.",
@@ -276,7 +284,7 @@ hence would have reachability issues.""",
     .}: bool
 
     rlnRelay* {.
-      desc: "Enable spam protection through rln-relay: true|false",
+      desc: "Enable spam protection through rln-relay: true|false.",
       defaultValue: false,
       name: "rln-relay"
     .}: bool
@@ -287,7 +295,7 @@ hence would have reachability issues.""",
     .}: Option[uint]
 
     rlnRelayDynamic* {.
-      desc: "Enable  waku-rln-relay with on-chain dynamic group management: true|false",
+      desc: "Enable  waku-rln-relay with on-chain dynamic group management: true|false.",
       defaultValue: false,
       name: "rln-relay-dynamic"
     .}: bool
@@ -311,7 +319,8 @@ hence would have reachability issues.""",
     .}: string
 
     rlnRelayBandwidthThreshold* {.
-      desc: "Message rate in bytes/sec after which verification of proofs should happen",
+      desc:
+        "Message rate in bytes/sec after which verification of proofs should happen.",
       defaultValue: 0, # to maintain backwards compatibility
       name: "rln-relay-bandwidth-threshold"
     .}: int
@@ -327,6 +336,7 @@ hence would have reachability issues.""",
       name: "keep-alive"
     .}: bool
 
+    # TODO: This is trying to do too much, this should only be used for autosharding, which itself should be configurable
     # If numShardsInNetwork is not set, we use the number of shards configured as numShardsInNetwork
     numShardsInNetwork* {.
       desc: "Number of shards in the network",
@@ -590,7 +600,7 @@ with the drawback of consuming some more bandwidth.""",
 
     ## Discovery v5 config
     discv5Discovery* {.
-      desc: "Enable discovering nodes via Node Discovery v5",
+      desc: "Enable discovering nodes via Node Discovery v5.",
       defaultValue: false,
       name: "discv5-discovery"
     .}: bool
diff --git a/waku/factory/internal_config.nim b/waku/factory/internal_config.nim
index 064a03acd9..08f11f1c5f 100644
--- a/waku/factory/internal_config.nim
+++ b/waku/factory/internal_config.nim
@@ -4,7 +4,7 @@ import
   libp2p/crypto/crypto,
   libp2p/multiaddress,
   libp2p/nameresolving/dnsresolver,
-  std/[options, sequtils, net],
+  std/[options, sequtils, strutils, net],
   results
 import
   ./external_config,
@@ -12,7 +12,8 @@ import
   ../node/config,
   ../waku_enr/capabilities,
   ../waku_enr,
-  ../waku_core
+  ../waku_core,
+  ./networks_config
 
 proc enrConfiguration*(
     conf: WakuNodeConf, netConfig: NetConfig, key: crypto.PrivateKey
@@ -157,3 +158,71 @@ proc networkConfiguration*(conf: WakuNodeConf, clientId: string): NetConfigResul
   )
 
   return netConfigRes
+
+proc applyPresetConfiguration*(srcConf: WakuNodeConf): Result[WakuNodeConf, string] =
+  var resConf = srcConf
+
+  if resConf.clusterId == 1:
+    warn(
+      "TWN - The Waku Network configuration will not be applied when `--cluster-id=1` is passed in future releases. Use `--preset=twn` instead."
+    )
+    resConf.preset = "twn"
+
+  case toLowerAscii(resConf.preset)
+  of "twn":
+    let twnClusterConf = ClusterConf.TheWakuNetworkConf()
+
+    # Override configuration
+    resConf.maxMessageSize = twnClusterConf.maxMessageSize
+    resConf.clusterId = twnClusterConf.clusterId
+    resConf.rlnRelay = twnClusterConf.rlnRelay
+    resConf.rlnRelayEthContractAddress = twnClusterConf.rlnRelayEthContractAddress
+    resConf.rlnRelayChainId = twnClusterConf.rlnRelayChainId
+    resConf.rlnRelayDynamic = twnClusterConf.rlnRelayDynamic
+    resConf.rlnRelayBandwidthThreshold = twnClusterConf.rlnRelayBandwidthThreshold
+    resConf.discv5Discovery = twnClusterConf.discv5Discovery
+    resConf.discv5BootstrapNodes =
+      resConf.discv5BootstrapNodes & twnClusterConf.discv5BootstrapNodes
+    resConf.rlnEpochSizeSec = twnClusterConf.rlnEpochSizeSec
+    resConf.rlnRelayUserMessageLimit = twnClusterConf.rlnRelayUserMessageLimit
+    resConf.numShardsInNetwork = twnClusterConf.numShardsInNetwork
+
+    if resConf.relay:
+      resConf.rlnRelay = twnClusterConf.rlnRelay
+  else:
+    discard
+
+  return ok(resConf)
+
+# TODO: numShardsInNetwork should be mandatory with autosharding, and unneeded otherwise
+proc getNumShardsInNetwork*(conf: WakuNodeConf): uint32 =
+  if conf.numShardsInNetwork != 0:
+    return conf.numShardsInNetwork
+  # If conf.numShardsInNetwork is not set, use 1024 - the maximum possible as per the static sharding spec
+  # https://github.com/waku-org/specs/blob/master/standards/core/relay-sharding.md#static-sharding
+  return uint32(MaxShardIndex + 1)
+
+proc validateShards*(conf: WakuNodeConf): Result[void, string] =
+  let numShardsInNetwork = getNumShardsInNetwork(conf)
+
+  for shard in conf.shards:
+    if shard >= numShardsInNetwork:
+      let msg =
+        "validateShards invalid shard: " & $shard & " when numShardsInNetwork: " &
+        $numShardsInNetwork # fmt doesn't work
+      error "validateShards failed", error = msg
+      return err(msg)
+
+  return ok()
+
+proc getNodeKey*(
+    conf: WakuNodeConf, rng: ref HmacDrbgContext = crypto.newRng()
+): Result[PrivateKey, string] =
+  if conf.nodekey.isSome():
+    return ok(conf.nodekey.get())
+
+  warn "missing node key, generating new set"
+  let key = crypto.PrivateKey.random(Secp256k1, rng[]).valueOr:
+    error "Failed to generate key", error = error
+    return err("Failed to generate key: " & $error)
+  return ok(key)
diff --git a/waku/factory/networks_config.nim b/waku/factory/networks_config.nim
index 41678f5904..8dfefbf349 100644
--- a/waku/factory/networks_config.nim
+++ b/waku/factory/networks_config.nim
@@ -30,6 +30,7 @@ proc TheWakuNetworkConf*(T: type ClusterConf): ClusterConf =
     rlnRelayUserMessageLimit: 100,
     numShardsInNetwork: 8,
     discv5Discovery: true,
+      # TODO: Why is this part of the conf? eg an edge node would not have this
     discv5BootstrapNodes:
       @[
         "enr:-QESuED0qW1BCmF-oH_ARGPr97Nv767bl_43uoy70vrbah3EaCAdK3Q0iRQ6wkSTTpdrg_dU_NC2ydO8leSlRpBX4pxiAYJpZIJ2NIJpcIRA4VDAim11bHRpYWRkcnO4XAArNiZub2RlLTAxLmRvLWFtczMud2FrdS5zYW5kYm94LnN0YXR1cy5pbQZ2XwAtNiZub2RlLTAxLmRvLWFtczMud2FrdS5zYW5kYm94LnN0YXR1cy5pbQYfQN4DgnJzkwABCAAAAAEAAgADAAQABQAGAAeJc2VjcDI1NmsxoQOTd-h5owwj-cx7xrmbvQKU8CV3Fomfdvcv1MBc-67T5oN0Y3CCdl-DdWRwgiMohXdha3UyDw",
diff --git a/waku/factory/node_factory.nim b/waku/factory/node_factory.nim
index d2d6b1d992..3142ff7668 100644
--- a/waku/factory/node_factory.nim
+++ b/waku/factory/node_factory.nim
@@ -137,13 +137,6 @@ proc initNode(
 
 ## Mount protocols
 
-proc getNumShardsInNetwork*(conf: WakuNodeConf): uint32 =
-  if conf.numShardsInNetwork != 0:
-    return conf.numShardsInNetwork
-  # If conf.numShardsInNetwork is not set, use 1024 - the maximum possible as per the static sharding spec
-  # https://github.com/waku-org/specs/blob/master/standards/core/relay-sharding.md#static-sharding
-  return uint32(MaxShardIndex + 1)
-
 proc getAutoshards*(
     node: WakuNode, contentTopics: seq[string]
 ): Result[seq[RelayShard], string] =
@@ -265,6 +258,7 @@ proc setupProtocols(
 
   if conf.numShardsInNetwork == 0:
     warn "Number of shards in network not configured, setting it to",
+      # TODO: If not configured, it mounts 1024 shards! Make it a mandatory configuration instead
       numShardsInNetwork = $numShardsInNetwork
 
   node.mountSharding(conf.clusterId, numShardsInNetwork).isOkOr:
diff --git a/waku/factory/waku.nim b/waku/factory/waku.nim
index 5462531768..854df8ddef 100644
--- a/waku/factory/waku.nim
+++ b/waku/factory/waku.nim
@@ -99,19 +99,6 @@ proc logConfig(conf: WakuNodeConf) =
 func version*(waku: Waku): string =
   waku.version
 
-proc validateShards(conf: WakuNodeConf): Result[void, string] =
-  let numShardsInNetwork = getNumShardsInNetwork(conf)
-
-  for shard in conf.shards:
-    if shard >= numShardsInNetwork:
-      let msg =
-        "validateShards invalid shard: " & $shard & " when numShardsInNetwork: " &
-        $numShardsInNetwork # fmt doesn't work
-      error "validateShards failed", error = msg
-      return err(msg)
-
-  return ok()
-
 proc setupSwitchServices(
     waku: Waku, conf: WakuNodeConf, circuitRelay: Relay, rng: ref HmacDrbgContext
 ) =
@@ -214,46 +201,28 @@ proc new*(
         shards.add(shard)
       confCopy.shards = shards
 
-  case confCopy.clusterId
-
-  # cluster-id=1 (aka The Waku Network)
-  of 1:
-    let twnClusterConf = ClusterConf.TheWakuNetworkConf()
-
-    # Override configuration
-    confCopy.maxMessageSize = twnClusterConf.maxMessageSize
-    confCopy.clusterId = twnClusterConf.clusterId
-    confCopy.rlnRelayEthContractAddress = twnClusterConf.rlnRelayEthContractAddress
-    confCopy.rlnRelayChainId = twnClusterConf.rlnRelayChainId
-    confCopy.rlnRelayDynamic = twnClusterConf.rlnRelayDynamic
-    confCopy.rlnRelayBandwidthThreshold = twnClusterConf.rlnRelayBandwidthThreshold
-    confCopy.discv5Discovery = twnClusterConf.discv5Discovery
-    confCopy.discv5BootstrapNodes =
-      confCopy.discv5BootstrapNodes & twnClusterConf.discv5BootstrapNodes
-    confCopy.rlnEpochSizeSec = twnClusterConf.rlnEpochSizeSec
-    confCopy.rlnRelayUserMessageLimit = twnClusterConf.rlnRelayUserMessageLimit
-    confCopy.numShardsInNetwork = twnClusterConf.numShardsInNetwork
-
-    # Only set rlnRelay to true if relay is configured
-    if confCopy.relay:
-      confCopy.rlnRelay = twnClusterConf.rlnRelay
-  else:
-    discard
+  # Why can't I replace this block with a concise `.valueOr`?
+  confCopy = block:
+    let res = applyPresetConfiguration(confCopy)
+    if res.isErr():
+      error "Failed to complete the config", error = res.error
+      return err("Failed to complete the config:" & $res.error)
+    res.get()
 
-  info "Running nwaku node", version = git_version
   logConfig(confCopy)
 
+  info "Running nwaku node", version = git_version
+
   let validateShardsRes = validateShards(confCopy)
   if validateShardsRes.isErr():
     error "Failed validating shards", error = $validateShardsRes.error
     return err("Failed validating shards: " & $validateShardsRes.error)
 
-  if not confCopy.nodekey.isSome():
-    let keyRes = crypto.PrivateKey.random(Secp256k1, rng[])
-    if keyRes.isErr():
-      error "Failed to generate key", error = $keyRes.error
-      return err("Failed to generate key: " & $keyRes.error)
-    confCopy.nodekey = some(keyRes.get())
+  let keyRes = getNodeKey(confCopy, rng)
+  if keyRes.isErr():
+    error "Failed to generate key", error = $keyRes.error
+    return err("Failed to generate key: " & $keyRes.error)
+  confCopy.nodeKey = some(keyRes.get())
 
   var relay = newCircuitRelay(confCopy.isRelayClient)
 
@@ -284,6 +253,7 @@ proc new*(
 
   var waku = Waku(
     version: git_version,
+    # TODO: WakuNodeConf is re-used for too many context, `conf` here should be a dedicated subtype
     conf: confCopy,
     rng: rng,
     key: confCopy.nodekey.get(),


Reviews:
user: login: Ivansete-status
body: Thanks for it! Some comments so far.

In general, I would avoid adding an additional parameter.

I think we need to make sure the CLI passed properly override the implicitly set of parameters.

For example, when passing `rlnRelayEthContractAddress` CLI should override the implicit value that happens when clusterId==1.

state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-31T08:02:02Z
user: login: gabrielmer
body: Thanks so much! Really like the idea of having presets!

Added a couple comments/questions :))
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-31T08:20:44Z
user: login: fryorcraken
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-31T10:57:18Z
user: login: fryorcraken
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-31T10:57:38Z
user: login: fryorcraken
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-31T10:59:01Z
user: login: fryorcraken
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-31T11:06:26Z
user: login: fryorcraken
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-31T11:06:31Z
user: login: Ivansete-status
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-31T11:40:40Z
user: login: gabrielmer
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-31T11:41:04Z
user: login: gabrielmer
body: LGTM, thanks so much! 😍 
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-03-31T11:44:33Z
user: login: Ivansete-status
state: APPROVED
author_association: COLLABORATOR
submitted_at: 2025-03-31T12:19:39Z
user: login: Ivansete-status
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-31T12:26:52Z
user: login: fryorcraken
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-31T12:40:29Z
user: login: Ivansete-status
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-31T13:57:25Z
user: login: fryorcraken
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-31T22:28:55Z

Review comments:
diff_hunk: @@ -145,6 +146,12 @@ type WakuNodeConf* = object
     .}: seq[ProtectedShard]
 
     ## General node config
+    preset* {.
+      desc: "Network preset to use." & "Must be one of 'default', ''",
+      defaultValue: "",
+      name: "preset"
path: waku/factory/external_config.nim
user: login: Ivansete-status
body: I think we need to explain what "default" or "" means
```suggestion
    preset* {.
      desc: "Network preset to use. Must be one of 'default', ''",
      defaultValue: "",
      name: "preset"
```
created_at: 2025-03-31T07:32:28Z
author_association: COLLABORATOR
diff_hunk: @@ -265,6 +258,7 @@ proc setupProtocols(
 
   if conf.numShardsInNetwork == 0:
     warn "Number of shards in network not configured, setting it to",
+      # TODO: If not configured, it mounts 1024 shards! Make it a mandatory configuration instead
path: waku/factory/node_factory.nim
user: login: Ivansete-status
body: This is what already happens afaik. For example, by setting `cluster-id=42` and `shard=0` I see the following log at node start: `INF 2025-03-31 09:43:58.663+02:00 mounting sharding                          topics="waku node" tid=413683 file=waku_node.nim:218 clusterId=42 shardCount=1024
`
created_at: 2025-03-31T07:45:19Z
author_association: COLLABORATOR
diff_hunk: @@ -203,75 +190,57 @@ proc new*(
 
     if shardsOpt.isSome():
       let relayShards = shardsOpt.get()
-      if relayShards.clusterId != confCopy.clusterId:
+      if relayShards.clusterId != srcConf.clusterId:
         error "clusterId of the pubsub topic should match the node's cluster. e.g. --pubsub-topic=/waku/2/rs/22/1 and --cluster-id=22",
-          nodeCluster = confCopy.clusterId, pubsubCluster = relayShards.clusterId
+          nodeCluster = srcConf.clusterId, pubsubCluster = relayShards.clusterId
         return err(
           "clusterId of the pubsub topic should match the node's cluster. e.g. --pubsub-topic=/waku/2/rs/22/1 and --cluster-id=22"
         )
 
       for shard in relayShards.shardIds:
         shards.add(shard)
-      confCopy.shards = shards
-
-  case confCopy.clusterId
-
-  # cluster-id=1 (aka The Waku Network)
-  of 1:
-    let twnClusterConf = ClusterConf.TheWakuNetworkConf()
-
-    # Override configuration
-    confCopy.maxMessageSize = twnClusterConf.maxMessageSize
-    confCopy.clusterId = twnClusterConf.clusterId
-    confCopy.rlnRelayEthContractAddress = twnClusterConf.rlnRelayEthContractAddress
-    confCopy.rlnRelayChainId = twnClusterConf.rlnRelayChainId
-    confCopy.rlnRelayDynamic = twnClusterConf.rlnRelayDynamic
-    confCopy.rlnRelayBandwidthThreshold = twnClusterConf.rlnRelayBandwidthThreshold
-    confCopy.discv5Discovery = twnClusterConf.discv5Discovery
-    confCopy.discv5BootstrapNodes =
-      confCopy.discv5BootstrapNodes & twnClusterConf.discv5BootstrapNodes
-    confCopy.rlnEpochSizeSec = twnClusterConf.rlnEpochSizeSec
-    confCopy.rlnRelayUserMessageLimit = twnClusterConf.rlnRelayUserMessageLimit
-    confCopy.numShardsInNetwork = twnClusterConf.numShardsInNetwork
-
-    # Only set rlnRelay to true if relay is configured
-    if confCopy.relay:
-      confCopy.rlnRelay = twnClusterConf.rlnRelay
-  else:
-    discard
+      srcConf.shards = shards
+
+  # Why can't I replace this block with a concise `.valueOr`?
path: waku/factory/waku.nim
user: login: Ivansete-status
body: There might be imported modules collision that prevents from properly using `valueOr`.
```suggestion
```
created_at: 2025-03-31T07:47:56Z
author_association: COLLABORATOR
diff_hunk: @@ -265,6 +258,7 @@ proc setupProtocols(
 
   if conf.numShardsInNetwork == 0:
     warn "Number of shards in network not configured, setting it to",
+      # TODO: If not configured, it mounts 1024 shards! Make it a mandatory configuration instead
path: waku/factory/node_factory.nim
user: login: gabrielmer
body: I think we didn't want to make it a mandatory configuration and chose a default value so it works out of the box for everyone and if someone needs a different value they can change it. But I guess if someone doesn't want to use a preset and has to set up everything manually, then it's ok to make them set up the number of shards in the network.

We got to the 1024 default here: https://github.com/waku-org/nwaku/pull/2997#discussion_r1741720714
created_at: 2025-03-31T08:03:23Z
author_association: CONTRIBUTOR
diff_hunk: @@ -482,18 +476,11 @@ proc startNode*(
   return ok()
 
 proc setupNode*(
-    conf: WakuNodeConf, rng: ref HmacDrbgContext = crypto.newRng(), relay: Relay
+    conf: WakuNodeConf,
+    key: crypto.PrivateKey,
path: waku/factory/node_factory.nim
user: login: gabrielmer
body: why adding an additional argument for the key if it should be included in `conf`?
created_at: 2025-03-31T08:08:17Z
author_association: CONTRIBUTOR
diff_hunk: @@ -184,16 +171,16 @@ proc setupAppCallbacks(
   return ok()
 
 proc new*(
-    T: type Waku, confCopy: var WakuNodeConf, appCallbacks: AppCallbacks = nil
+    T: type Waku, srcConf: var WakuNodeConf, appCallbacks: AppCallbacks = nil
path: waku/factory/waku.nim
user: login: gabrielmer
body: I think that we should use `finalConf` as an argument and not `srcConf`. Notice we have `var WakuNodeConf` so we can modify the passed variable

We do it because if a user sets `conf`, passes it to `new()` and `new()` overrides configurations, whenever the user passes `conf` to other functions it won't be updated with the actual configurations the node is using and then there's bugs that come from that.

So we figured that whenever we create a node and pass a `conf` variable, if we override values we should do it in the passed variable too, so whenever the same variable is used for subsequent function calls it contains the right values
created_at: 2025-03-31T08:17:59Z
author_association: CONTRIBUTOR
diff_hunk: @@ -265,6 +258,7 @@ proc setupProtocols(
 
   if conf.numShardsInNetwork == 0:
     warn "Number of shards in network not configured, setting it to",
+      # TODO: If not configured, it mounts 1024 shards! Make it a mandatory configuration instead
path: waku/factory/node_factory.nim
user: login: fryorcraken
body: I will do a follow-up PR to clean this TODO.

IMO, the `numShardsInNetwork` should be `0` by default and it should mean that autosharding is NOT enabled.

then, to enable autosharding, this value must be set, and would be the way to enable it.

So that there is no need to come up with some "default" value in case it's not set when we want autosharding.


created_at: 2025-03-31T10:57:18Z
author_association: COLLABORATOR
diff_hunk: @@ -203,75 +190,57 @@ proc new*(
 
     if shardsOpt.isSome():
       let relayShards = shardsOpt.get()
-      if relayShards.clusterId != confCopy.clusterId:
+      if relayShards.clusterId != srcConf.clusterId:
         error "clusterId of the pubsub topic should match the node's cluster. e.g. --pubsub-topic=/waku/2/rs/22/1 and --cluster-id=22",
-          nodeCluster = confCopy.clusterId, pubsubCluster = relayShards.clusterId
+          nodeCluster = srcConf.clusterId, pubsubCluster = relayShards.clusterId
         return err(
           "clusterId of the pubsub topic should match the node's cluster. e.g. --pubsub-topic=/waku/2/rs/22/1 and --cluster-id=22"
         )
 
       for shard in relayShards.shardIds:
         shards.add(shard)
-      confCopy.shards = shards
-
-  case confCopy.clusterId
-
-  # cluster-id=1 (aka The Waku Network)
-  of 1:
-    let twnClusterConf = ClusterConf.TheWakuNetworkConf()
-
-    # Override configuration
-    confCopy.maxMessageSize = twnClusterConf.maxMessageSize
-    confCopy.clusterId = twnClusterConf.clusterId
-    confCopy.rlnRelayEthContractAddress = twnClusterConf.rlnRelayEthContractAddress
-    confCopy.rlnRelayChainId = twnClusterConf.rlnRelayChainId
-    confCopy.rlnRelayDynamic = twnClusterConf.rlnRelayDynamic
-    confCopy.rlnRelayBandwidthThreshold = twnClusterConf.rlnRelayBandwidthThreshold
-    confCopy.discv5Discovery = twnClusterConf.discv5Discovery
-    confCopy.discv5BootstrapNodes =
-      confCopy.discv5BootstrapNodes & twnClusterConf.discv5BootstrapNodes
-    confCopy.rlnEpochSizeSec = twnClusterConf.rlnEpochSizeSec
-    confCopy.rlnRelayUserMessageLimit = twnClusterConf.rlnRelayUserMessageLimit
-    confCopy.numShardsInNetwork = twnClusterConf.numShardsInNetwork
-
-    # Only set rlnRelay to true if relay is configured
-    if confCopy.relay:
-      confCopy.rlnRelay = twnClusterConf.rlnRelay
-  else:
-    discard
+      srcConf.shards = shards
+
+  # Why can't I replace this block with a concise `.valueOr`?
path: waku/factory/waku.nim
user: login: fryorcraken
body: I'll remove that unless someone know how to fix it
created_at: 2025-03-31T10:57:38Z
author_association: COLLABORATOR
diff_hunk: @@ -265,6 +258,7 @@ proc setupProtocols(
 
   if conf.numShardsInNetwork == 0:
     warn "Number of shards in network not configured, setting it to",
+      # TODO: If not configured, it mounts 1024 shards! Make it a mandatory configuration instead
path: waku/factory/node_factory.nim
user: login: fryorcraken
body: See here: https://github.com/waku-org/nwaku/pull/3346#discussion_r2020826347

IMO:

- if set -> autosharding is enabled
- if not set (eg `0` or 1) -> not auto-sharding.

Working "out of the box" -> preset or network with 1 shard (you don't need more than one shard when you start)
created_at: 2025-03-31T10:59:01Z
author_association: COLLABORATOR
diff_hunk: @@ -184,16 +171,16 @@ proc setupAppCallbacks(
   return ok()
 
 proc new*(
-    T: type Waku, confCopy: var WakuNodeConf, appCallbacks: AppCallbacks = nil
+    T: type Waku, srcConf: var WakuNodeConf, appCallbacks: AppCallbacks = nil
path: waku/factory/waku.nim
user: login: fryorcraken
body: The real issue here is that `WakuNodeConf` is used in too many context. It is not single-purpose.

It is used to grab value from CLI Args. Meaning a bunch of those are `option`s.
It is used to pass the final config to setup a waku node, after those which can't be option have been overridden by `new`.

And then it's used in a bunch of other places as you mentioned.

A better solution would be to have 2 different types for CLI and for actual node configuration. Where the CLI may have `option` but the one for actual node configuration does not -> all values are set.

This way, we can avoid duplicate code like key generation and we can also avoid the bug you just mentioned -> because all those function who need the "final" config, will get an object that does not have `option` that may be `none`.

I'll tackle your comment in this PR to not introduce bugs, and then apply the solution above in a new PR.

created_at: 2025-03-31T11:06:26Z
author_association: COLLABORATOR
diff_hunk: @@ -482,18 +476,11 @@ proc startNode*(
   return ok()
 
 proc setupNode*(
-    conf: WakuNodeConf, rng: ref HmacDrbgContext = crypto.newRng(), relay: Relay
+    conf: WakuNodeConf,
+    key: crypto.PrivateKey,
path: waku/factory/node_factory.nim
user: login: fryorcraken
body: See https://github.com/waku-org/nwaku/pull/3346#discussion_r2020590111
created_at: 2025-03-31T11:06:31Z
author_association: COLLABORATOR
diff_hunk: @@ -145,6 +146,13 @@ type WakuNodeConf* = object
     .}: seq[ProtectedShard]
 
     ## General node config
+    preset* {.
+      desc:
+        "Network preset to use. 'default' is The RLN-protected Waku Network (cluster 1).",
path: waku/factory/external_config.nim
user: login: Ivansete-status
body: Instead of `default` I think it'd better to use `twn` ( more explicit .)
created_at: 2025-03-31T11:32:20Z
author_association: COLLABORATOR
diff_hunk: @@ -184,16 +171,16 @@ proc setupAppCallbacks(
   return ok()
 
 proc new*(
-    T: type Waku, confCopy: var WakuNodeConf, appCallbacks: AppCallbacks = nil
+    T: type Waku, srcConf: var WakuNodeConf, appCallbacks: AppCallbacks = nil
path: waku/factory/waku.nim
user: login: gabrielmer
body: Yes I agree that our usage of the configuration is a bit messy and we use the same object for everything.

The way I look at it is that we get the initial input from the user and then we progressively process, set and override whatever we need to in the process of actually setting up the node.

If we can just in one place do all the changes from CLI input to actual values to be configured, it would definitely be much cleaner. Not sure if all the transformation logic fits in a single place or some parts of the logic have to happen during different steps of the node configuration, we would need to attempt and see, but definitely worth trying :)
created_at: 2025-03-31T11:41:04Z
author_association: CONTRIBUTOR
diff_hunk: @@ -157,3 +158,71 @@ proc networkConfiguration*(conf: WakuNodeConf, clientId: string): NetConfigResul
   )
 
   return netConfigRes
+
+proc applyPresetConfiguration*(srcConf: WakuNodeConf): Result[WakuNodeConf, string] =
+  var resConf = srcConf
+
+  if resConf.clusterId == 1:
+    warn(
+      "Default (The Waku Network) configuration will not be applied when `--cluster-id=1` is passed in future releases. Use `--preset=sandbox` instead."
+    )
+    resConf.preset = "default"
+
+  case resConf.preset
path: waku/factory/internal_config.nim
user: login: Ivansete-status
body: We need to make sure is case insensitive.
```suggestion
  case resConf.preset.toLower()
```
created_at: 2025-03-31T12:26:52Z
author_association: COLLABORATOR
diff_hunk: @@ -184,16 +171,16 @@ proc setupAppCallbacks(
   return ok()
 
 proc new*(
-    T: type Waku, confCopy: var WakuNodeConf, appCallbacks: AppCallbacks = nil
+    T: type Waku, srcConf: var WakuNodeConf, appCallbacks: AppCallbacks = nil
path: waku/factory/waku.nim
user: login: fryorcraken
body: > If we can just in one place do all the changes from CLI input to actual values to be configured, it would definitely be much cleaner.

yes

> Not sure if all the transformation logic fits in a single place or some parts of the logic have to happen during different steps of the node configuration, we would need to attempt and see, but definitely worth trying :)

yes
created_at: 2025-03-31T12:40:29Z
author_association: COLLABORATOR
diff_hunk: @@ -145,6 +146,13 @@ type WakuNodeConf* = object
     .}: seq[ProtectedShard]
 
     ## General node config
+    preset* {.
+      desc:
+        "Network preset to use. 'twn' is The RLN-protected Waku Network (cluster 1).",
+      defaultValue: "",
path: waku/factory/external_config.nim
user: login: Ivansete-status
body: Sorry, I missed this detail earlier on.
Do we want `twn` to be config by default or is something we should explicitly configure?

```suggestion
      defaultValue: "twn",
```
created_at: 2025-03-31T13:57:24Z
author_association: COLLABORATOR
diff_hunk: @@ -145,6 +146,13 @@ type WakuNodeConf* = object
     .}: seq[ProtectedShard]
 
     ## General node config
+    preset* {.
+      desc:
+        "Network preset to use. 'twn' is The RLN-protected Waku Network (cluster 1).",
+      defaultValue: "",
path: waku/factory/external_config.nim
user: login: fryorcraken
body: Explicitly configure for now as there are challenges around RLN and need for web3 RPC endpoint.
created_at: 2025-03-31T22:28:55Z
author_association: COLLABORATOR

PR comments:
user: login: github-actions[bot]
created_at: 2025-03-31T01:27:38Z
body: You can find the image built from this PR at

```
quay.io/wakuorg/nwaku-pr:3346
```

Built from 9130388dee6a87ee5af73707c8d1737aa643cd36
<!-- thollander/actions-comment-pull-request "execution-rln-v" -->
author_association: NONE
user: login: fryorcraken
created_at: 2025-03-31T08:01:31Z
body: Mac test fail is RLN related so I don't think it's me :)


```
  [FAILED ] (  95.9s) clearNullifierLog: should clear epochs > MaxEpochGap[0m
```
author_association: COLLABORATOR
user: login: fryorcraken
created_at: 2025-03-31T12:12:09Z
body: > In general, I would avoid adding an additional parameter.

I agree, already too many parameters, many not user friendly at all.

The intent is to define more presets once this is introduced.

I can do a follow PR to remove some parameters.
author_association: COLLABORATOR
user: login: fryorcraken
created_at: 2025-03-31T12:16:53Z
body: > For example, when passing rlnRelayEthContractAddress CLI should override the implicit value that happens when clusterId==1.

The actual problem is that when you pass cluster id 1, there is a surprising side effect of other parameters being overridden.

Having a dedicate option for presets remove the need for the side effect.
It simplifies code and UX.

author_association: COLLABORATOR