Repository: calimero-network/core
Pull request #1236: feat(e2e) : improve comment generation
Author: login: cy4n1d3-p1x3l
Labels: name: external
Created: 2025-04-26T18:56:10Z | Merged: 2025-05-29T05:26:25Z

# [e2e tests] Improve generation of comments and support for multiple applications

## Description

The e2e tests now run for multiple applications. Fixes #1166 

## Test plan

Testing done locally.

## Documentation update

Not needed as of now.


Linked issue #1166: Improve generation of comments from e2e tests
Currently, every e2e scenario report is written in separate comment, we would need to combine them in one comment for better overview.

Linked issue comments: user: login: chefsale
created_at: 2025-04-21T03:50:35Z
body: Is this still relevant, if yes not sure what has to be done?
author_association: MEMBER
user: login: fbozic
created_at: 2025-04-25T09:25:52Z
body: > Is this still relevant, if yes not sure what has to be done?

Still relevant. We should add back support for testing multiple applications per protocol and combine that into an aggregated report. 

Example of current report, notice that we have duplicate references to the protocol. 
- `Scenario` should be renamed to `Protocol`, value stays the same
- `Protocol/Step` should be renamed to `application/step`, and it should have the name of the application for a value
![Image](https://github.com/user-attachments/assets/44e48f88-c045-4cc3-9fa5-f37e0879ec7a)
author_association: CONTRIBUTOR
user: login: chefsale
created_at: 2025-04-25T09:30:45Z
body: Can we make it a bounty then? What is the estimated complaxity of this issue? Is it just a matter of reformatting and re-enabling multiple apps or there is a bigger scope behind it?
author_association: MEMBER
user: login: fbozic
created_at: 2025-04-25T09:45:09Z
body: > Can we make it a bounty then? What is the estimated complaxity of this issue? Is it just a matter of reformatting and re-enabling multiple apps or there is a bigger scope behind it?

Yeah, we can make it bounty. 
Definition of done:
- [ ] Extend e2e-tests driver and config to support multiple applications per protocol
  - Current config layout is `config/scenarios/<protocol>/test.json`
  - Desired config layout is `config/protocols/<protocol>/<application-test>.json`
  - Drop protocol field from test config ([example](https://github.com/calimero-network/core/blob/24f0cc6bfea9476314bf57784a503cee05d1d0a5/e2e-tests/config/scenarios/ethereum/test.json#L2)) since it is the same as the name of the directory
  - Run all application tests on a single set of nodes (do not shut down nodes between application tests)
- [ ] Update e2e-tests workflow, mainly [`Submit Report` job](https://github.com/calimero-network/core/blob/24f0cc6bfea9476314bf57784a503cee05d1d0a5/.github/workflows/e2e-tests.yml#L203)
  - Might not be needed, depending on the implementation above, check `e2e-tests combine` command for more info 

Simplest way of running e2e-tests locally is with `cargo run -p e2e-tests -- --input-dir ./e2e-tests/config --output-dir ./e2e-tests/corpus --merod-binary ./target/debug/merod --meroctl-binary ./target/debug/meroctl --protocols near`. Near doesn't require any additional tooling, while other protocols do (icp might be second simplest to run).
author_association: CONTRIBUTOR
user: login: cy4n1d3-p1x3l
created_at: 2025-04-25T18:49:08Z
body: @antonpaisov I would like to work upon it. 
author_association: CONTRIBUTOR

Diff:
diff --git a/.github/workflows/e2e-tests.yml b/.github/workflows/e2e-tests.yml
index a84930cfc3..86acf030cf 100644
--- a/.github/workflows/e2e-tests.yml
+++ b/.github/workflows/e2e-tests.yml
@@ -184,7 +184,7 @@ jobs:
             --output-dir ./e2e-tests/corpus \
             --merod-binary ./target/debug/merod \
             --meroctl-binary ./target/debug/meroctl \
-            --scenarios ${{ matrix.protocol }}
+            --protocols ${{ matrix.protocol }}
 
       - name: Upload Test Artifacts
         if: ${{ !cancelled() }}
diff --git a/e2e-tests/config/config.json b/e2e-tests/config/config.json
index 22820ceda2..490499bcf7 100644
--- a/e2e-tests/config/config.json
+++ b/e2e-tests/config/config.json
@@ -47,4 +47,4 @@
       }
     }
   ]
-}
+}
\ No newline at end of file
diff --git a/e2e-tests/config/scenarios/ethereum/test.json b/e2e-tests/config/protocols/ethereum/demo-blockchain-integrations.json
similarity index 88%
rename from e2e-tests/config/scenarios/ethereum/test.json
rename to e2e-tests/config/protocols/ethereum/demo-blockchain-integrations.json
index e968ca7309..561f79d735 100644
--- a/e2e-tests/config/scenarios/ethereum/test.json
+++ b/e2e-tests/config/protocols/ethereum/demo-blockchain-integrations.json
@@ -1,5 +1,4 @@
 {
-  "protocol": "ethereum",
   "steps": [
     {
       "applicationInstall": {
@@ -61,7 +60,7 @@
           "limit": 10
         },
         "description": [
-            "get current proposals and extract the ID of the first one"
+          "get current proposals and extract the ID of the first one"
         ]
       }
     },
@@ -71,11 +70,11 @@
         "argsJson": {
           "proposal_id": "${proposal_id}",
           "message": {
-              "id": "msg1",
-              "author": "inviter",
-              "text": "This is a test message for the proposal",
-              "created_at": "2023-11-24T12:34:56Z",
-              "proposal_id": "${proposal_id}"
+            "id": "msg1",
+            "author": "inviter",
+            "text": "This is a test message for the proposal",
+            "created_at": "2023-11-24T12:34:56Z",
+            "proposal_id": "${proposal_id}"
           }
         },
         "expectedResultJson": null,
@@ -86,7 +85,9 @@
       "wait": {
         "for": "broadcast",
         "durationMs": 5000,
-        "description": ["wait exactly 5s for the broadcast to propagate"]
+        "description": [
+          "wait exactly 5s for the broadcast to propagate"
+        ]
       }
     },
     {
@@ -127,9 +128,11 @@
       "verifyExternalState": {
         "contractId": "0x9fE46736679d2D9a65F0992F2272dE9f3c7fa6e0",
         "methodName": "getValue(string)",
-        "args": ["someKey"],
+        "args": [
+          "someKey"
+        ],
         "expectedValue": "someValue"
       }
     }
   ]
-}
+}
\ No newline at end of file
diff --git a/e2e-tests/config/protocols/ethereum/kv-store-test.json b/e2e-tests/config/protocols/ethereum/kv-store-test.json
new file mode 100644
index 0000000000..f28fb18a75
--- /dev/null
+++ b/e2e-tests/config/protocols/ethereum/kv-store-test.json
@@ -0,0 +1,108 @@
+{
+  "steps": [
+    {
+      "applicationInstall": {
+        "application": {
+          "localFile": "./apps/kv-store/res/kv_store.wasm"
+        },
+        "target": "allMembers"
+      }
+    },
+    {
+      "contextCreate": null
+    },
+    {
+      "contextCreateAlias": {
+        "aliasName": "my_context"
+      }
+    },
+    {
+      "call": {
+        "methodName": "set",
+        "argsJson": {
+          "key": "foo",
+          "value": "bar"
+        },
+        "expectedResultJson": null,
+        "target": "inviter"
+      }
+    },
+    {
+      "call": {
+        "methodName": "get",
+        "argsJson": {
+          "key": "foo"
+        },
+        "expectedResultJson": "bar",
+        "target": "inviter"
+      }
+    },
+    {
+      "contextInviteJoin": null
+    },
+    {
+      "wait": {
+        "for": "consensus",
+        "durationMs": 5000,
+        "description": [
+          "assuming it takes 5s to propagate we should",
+          "only need to wait 5 * ceil(log2(nodes)) seconds"
+        ]
+      }
+    },
+    {
+      "call": {
+        "methodName": "get",
+        "argsJson": {
+          "key": "foo"
+        },
+        "expectedResultJson": "bar",
+        "target": "allMembers",
+        "retries": 20,
+        "intervalMs": 5000,
+        "description": [
+          "if we don't reach consensus in the ideal case",
+          "wait 5 seconds for nodes that have not yet synced",
+          "in the worst case, wait 20 * 5s for nodes that uselessly",
+          "keep syncing with themselves without having the state"
+        ]
+      }
+    },
+    {
+      "call": {
+        "methodName": "set",
+        "argsJson": {
+          "key": "foo",
+          "value": "baz"
+        },
+        "expectedResultJson": null,
+        "target": "inviter"
+      }
+    },
+    {
+      "wait": {
+        "for": "broadcast",
+        "durationMs": 5000,
+        "description": [
+          "wait exactly 5s for the broadcast to propagate"
+        ]
+      }
+    },
+    {
+      "call": {
+        "methodName": "get",
+        "argsJson": {
+          "key": "foo"
+        },
+        "expectedResultJson": "baz",
+        "target": "allMembers",
+        "retries": 5,
+        "intervalMs": 5000,
+        "description": [
+          "if a node still hasn't received the broadcast",
+          "try 5 more times in 5 seconds, but no more"
+        ]
+      }
+    }
+  ]
+}
\ No newline at end of file
diff --git a/e2e-tests/config/scenarios/icp/test.json b/e2e-tests/config/protocols/icp/demo-blockchain-integrations.json
similarity index 88%
rename from e2e-tests/config/scenarios/icp/test.json
rename to e2e-tests/config/protocols/icp/demo-blockchain-integrations.json
index 749d8dd2f8..46563691ed 100644
--- a/e2e-tests/config/scenarios/icp/test.json
+++ b/e2e-tests/config/protocols/icp/demo-blockchain-integrations.json
@@ -1,5 +1,4 @@
 {
-  "protocol": "icp",
   "steps": [
     {
       "applicationInstall": {
@@ -61,7 +60,7 @@
           "limit": 10
         },
         "description": [
-            "get current proposals and extract the ID of the first one"
+          "get current proposals and extract the ID of the first one"
         ]
       }
     },
@@ -71,11 +70,11 @@
         "argsJson": {
           "proposal_id": "${proposal_id}",
           "message": {
-              "id": "msg1",
-              "author": "inviter",
-              "text": "This is a test message for the proposal",
-              "created_at": "2023-11-24T12:34:56Z",
-              "proposal_id": "${proposal_id}"
+            "id": "msg1",
+            "author": "inviter",
+            "text": "This is a test message for the proposal",
+            "created_at": "2023-11-24T12:34:56Z",
+            "proposal_id": "${proposal_id}"
           }
         },
         "expectedResultJson": null,
@@ -86,7 +85,9 @@
       "wait": {
         "for": "broadcast",
         "durationMs": 5000,
-        "description": ["wait exactly 5s for the broadcast to propagate"]
+        "description": [
+          "wait exactly 5s for the broadcast to propagate"
+        ]
       }
     },
     {
@@ -132,4 +133,4 @@
       }
     }
   ]
-}
+}
\ No newline at end of file
diff --git a/e2e-tests/config/protocols/icp/kv-store-test.json b/e2e-tests/config/protocols/icp/kv-store-test.json
new file mode 100644
index 0000000000..f28fb18a75
--- /dev/null
+++ b/e2e-tests/config/protocols/icp/kv-store-test.json
@@ -0,0 +1,108 @@
+{
+  "steps": [
+    {
+      "applicationInstall": {
+        "application": {
+          "localFile": "./apps/kv-store/res/kv_store.wasm"
+        },
+        "target": "allMembers"
+      }
+    },
+    {
+      "contextCreate": null
+    },
+    {
+      "contextCreateAlias": {
+        "aliasName": "my_context"
+      }
+    },
+    {
+      "call": {
+        "methodName": "set",
+        "argsJson": {
+          "key": "foo",
+          "value": "bar"
+        },
+        "expectedResultJson": null,
+        "target": "inviter"
+      }
+    },
+    {
+      "call": {
+        "methodName": "get",
+        "argsJson": {
+          "key": "foo"
+        },
+        "expectedResultJson": "bar",
+        "target": "inviter"
+      }
+    },
+    {
+      "contextInviteJoin": null
+    },
+    {
+      "wait": {
+        "for": "consensus",
+        "durationMs": 5000,
+        "description": [
+          "assuming it takes 5s to propagate we should",
+          "only need to wait 5 * ceil(log2(nodes)) seconds"
+        ]
+      }
+    },
+    {
+      "call": {
+        "methodName": "get",
+        "argsJson": {
+          "key": "foo"
+        },
+        "expectedResultJson": "bar",
+        "target": "allMembers",
+        "retries": 20,
+        "intervalMs": 5000,
+        "description": [
+          "if we don't reach consensus in the ideal case",
+          "wait 5 seconds for nodes that have not yet synced",
+          "in the worst case, wait 20 * 5s for nodes that uselessly",
+          "keep syncing with themselves without having the state"
+        ]
+      }
+    },
+    {
+      "call": {
+        "methodName": "set",
+        "argsJson": {
+          "key": "foo",
+          "value": "baz"
+        },
+        "expectedResultJson": null,
+        "target": "inviter"
+      }
+    },
+    {
+      "wait": {
+        "for": "broadcast",
+        "durationMs": 5000,
+        "description": [
+          "wait exactly 5s for the broadcast to propagate"
+        ]
+      }
+    },
+    {
+      "call": {
+        "methodName": "get",
+        "argsJson": {
+          "key": "foo"
+        },
+        "expectedResultJson": "baz",
+        "target": "allMembers",
+        "retries": 5,
+        "intervalMs": 5000,
+        "description": [
+          "if a node still hasn't received the broadcast",
+          "try 5 more times in 5 seconds, but no more"
+        ]
+      }
+    }
+  ]
+}
\ No newline at end of file
diff --git a/e2e-tests/config/scenarios/kv-store/test.json b/e2e-tests/config/protocols/kv-store/test.json
similarity index 77%
rename from e2e-tests/config/scenarios/kv-store/test.json
rename to e2e-tests/config/protocols/kv-store/test.json
index 0137ea41d9..b2ea47a12c 100644
--- a/e2e-tests/config/scenarios/kv-store/test.json
+++ b/e2e-tests/config/protocols/kv-store/test.json
@@ -1,9 +1,10 @@
 {
-  "protocol": "ethereum",
   "steps": [
     {
       "applicationInstall": {
-        "application": { "localFile": "./apps/kv-store/res/kv_store.wasm" },
+        "application": {
+          "localFile": "./apps/kv-store/res/kv_store.wasm"
+        },
         "target": "allMembers"
       }
     },
@@ -18,7 +19,10 @@
     {
       "call": {
         "methodName": "set",
-        "argsJson": { "key": "foo", "value": "bar" },
+        "argsJson": {
+          "key": "foo",
+          "value": "bar"
+        },
         "expectedResultJson": null,
         "target": "inviter"
       }
@@ -26,7 +30,9 @@
     {
       "call": {
         "methodName": "get",
-        "argsJson": { "key": "foo" },
+        "argsJson": {
+          "key": "foo"
+        },
         "expectedResultJson": "bar",
         "target": "inviter"
       }
@@ -47,7 +53,9 @@
     {
       "call": {
         "methodName": "get",
-        "argsJson": { "key": "foo" },
+        "argsJson": {
+          "key": "foo"
+        },
         "expectedResultJson": "bar",
         "target": "allMembers",
         "retries": 20,
@@ -63,7 +71,10 @@
     {
       "call": {
         "methodName": "set",
-        "argsJson": { "key": "foo", "value": "baz" },
+        "argsJson": {
+          "key": "foo",
+          "value": "baz"
+        },
         "expectedResultJson": null,
         "target": "inviter"
       }
@@ -72,13 +83,17 @@
       "wait": {
         "for": "broadcast",
         "durationMs": 5000,
-        "description": ["wait exactly 5s for the broadcast to propagate"]
+        "description": [
+          "wait exactly 5s for the broadcast to propagate"
+        ]
       }
     },
     {
       "call": {
         "methodName": "get",
-        "argsJson": { "key": "foo" },
+        "argsJson": {
+          "key": "foo"
+        },
         "expectedResultJson": "baz",
         "target": "allMembers",
         "retries": 5,
diff --git a/e2e-tests/config/scenarios/near/test.json b/e2e-tests/config/protocols/near/demo-blockchain-integrations.json
similarity index 88%
rename from e2e-tests/config/scenarios/near/test.json
rename to e2e-tests/config/protocols/near/demo-blockchain-integrations.json
index f1f5c94884..eeab68336c 100644
--- a/e2e-tests/config/scenarios/near/test.json
+++ b/e2e-tests/config/protocols/near/demo-blockchain-integrations.json
@@ -1,5 +1,4 @@
 {
-  "protocol": "near",
   "steps": [
     {
       "applicationInstall": {
@@ -61,7 +60,7 @@
           "limit": 10
         },
         "description": [
-            "get current proposals and extract the ID of the first one"
+          "get current proposals and extract the ID of the first one"
         ]
       }
     },
@@ -71,11 +70,11 @@
         "argsJson": {
           "proposal_id": "${proposal_id}",
           "message": {
-              "id": "msg1",
-              "author": "inviter",
-              "text": "This is a test message for the proposal",
-              "created_at": "2023-11-24T12:34:56Z",
-              "proposal_id": "${proposal_id}"
+            "id": "msg1",
+            "author": "inviter",
+            "text": "This is a test message for the proposal",
+            "created_at": "2023-11-24T12:34:56Z",
+            "proposal_id": "${proposal_id}"
           }
         },
         "expectedResultJson": null,
@@ -86,7 +85,9 @@
       "wait": {
         "for": "broadcast",
         "durationMs": 5000,
-        "description": ["wait exactly 5s for the broadcast to propagate"]
+        "description": [
+          "wait exactly 5s for the broadcast to propagate"
+        ]
       }
     },
     {
@@ -131,4 +132,4 @@
       }
     }
   ]
-}
+}
\ No newline at end of file
diff --git a/e2e-tests/config/protocols/near/kv-store-test.json b/e2e-tests/config/protocols/near/kv-store-test.json
new file mode 100644
index 0000000000..f28fb18a75
--- /dev/null
+++ b/e2e-tests/config/protocols/near/kv-store-test.json
@@ -0,0 +1,108 @@
+{
+  "steps": [
+    {
+      "applicationInstall": {
+        "application": {
+          "localFile": "./apps/kv-store/res/kv_store.wasm"
+        },
+        "target": "allMembers"
+      }
+    },
+    {
+      "contextCreate": null
+    },
+    {
+      "contextCreateAlias": {
+        "aliasName": "my_context"
+      }
+    },
+    {
+      "call": {
+        "methodName": "set",
+        "argsJson": {
+          "key": "foo",
+          "value": "bar"
+        },
+        "expectedResultJson": null,
+        "target": "inviter"
+      }
+    },
+    {
+      "call": {
+        "methodName": "get",
+        "argsJson": {
+          "key": "foo"
+        },
+        "expectedResultJson": "bar",
+        "target": "inviter"
+      }
+    },
+    {
+      "contextInviteJoin": null
+    },
+    {
+      "wait": {
+        "for": "consensus",
+        "durationMs": 5000,
+        "description": [
+          "assuming it takes 5s to propagate we should",
+          "only need to wait 5 * ceil(log2(nodes)) seconds"
+        ]
+      }
+    },
+    {
+      "call": {
+        "methodName": "get",
+        "argsJson": {
+          "key": "foo"
+        },
+        "expectedResultJson": "bar",
+        "target": "allMembers",
+        "retries": 20,
+        "intervalMs": 5000,
+        "description": [
+          "if we don't reach consensus in the ideal case",
+          "wait 5 seconds for nodes that have not yet synced",
+          "in the worst case, wait 20 * 5s for nodes that uselessly",
+          "keep syncing with themselves without having the state"
+        ]
+      }
+    },
+    {
+      "call": {
+        "methodName": "set",
+        "argsJson": {
+          "key": "foo",
+          "value": "baz"
+        },
+        "expectedResultJson": null,
+        "target": "inviter"
+      }
+    },
+    {
+      "wait": {
+        "for": "broadcast",
+        "durationMs": 5000,
+        "description": [
+          "wait exactly 5s for the broadcast to propagate"
+        ]
+      }
+    },
+    {
+      "call": {
+        "methodName": "get",
+        "argsJson": {
+          "key": "foo"
+        },
+        "expectedResultJson": "baz",
+        "target": "allMembers",
+        "retries": 5,
+        "intervalMs": 5000,
+        "description": [
+          "if a node still hasn't received the broadcast",
+          "try 5 more times in 5 seconds, but no more"
+        ]
+      }
+    }
+  ]
+}
\ No newline at end of file
diff --git a/e2e-tests/config/scenarios/stellar/test.json b/e2e-tests/config/protocols/stellar/demo-blockchain-integrations.json
similarity index 89%
rename from e2e-tests/config/scenarios/stellar/test.json
rename to e2e-tests/config/protocols/stellar/demo-blockchain-integrations.json
index e06ac17600..573dc81da4 100644
--- a/e2e-tests/config/scenarios/stellar/test.json
+++ b/e2e-tests/config/protocols/stellar/demo-blockchain-integrations.json
@@ -1,9 +1,8 @@
 {
-  "protocol": "stellar",
   "steps": [
     {
       "applicationInstall": {
-        "application": { 
+        "application": {
           "url": "https://github.com/calimero-network/demo-blockchain-integrations/releases/download/latest/blockchain.wasm.gz"
         },
         "target": "allMembers"
@@ -61,7 +60,7 @@
           "limit": 10
         },
         "description": [
-            "get current proposals and extract the ID of the first one"
+          "get current proposals and extract the ID of the first one"
         ]
       }
     },
@@ -87,7 +86,9 @@
       "verifyExternalState": {
         "contractId": "CDT6O4UMQBAY6PZTGRCBXDYHTFSL55NE4CZ3COCGPE5UEHRAODIZKDEJ",
         "methodName": "get_value",
-        "args": ["someKey"],
+        "args": [
+          "someKey"
+        ],
         "expectedValue": "someValue"
       }
     },
@@ -127,7 +128,7 @@
           "limit": 10
         },
         "description": [
-            "get current proposals and extract the ID of the first one"
+          "get current proposals and extract the ID of the first one"
         ]
       }
     },
@@ -137,11 +138,11 @@
         "argsJson": {
           "proposal_id": "${proposal_id}",
           "message": {
-              "id": "msg1",
-              "author": "inviter",
-              "text": "This is a test message for the proposal",
-              "created_at": "2023-11-24T12:34:56Z",
-              "proposal_id": "${proposal_id}"
+            "id": "msg1",
+            "author": "inviter",
+            "text": "This is a test message for the proposal",
+            "created_at": "2023-11-24T12:34:56Z",
+            "proposal_id": "${proposal_id}"
           }
         },
         "expectedResultJson": null,
@@ -152,7 +153,9 @@
       "wait": {
         "for": "broadcast",
         "durationMs": 5000,
-        "description": ["wait exactly 5s for the broadcast to propagate"]
+        "description": [
+          "wait exactly 5s for the broadcast to propagate"
+        ]
       }
     },
     {
@@ -179,4 +182,4 @@
       }
     }
   ]
-}
+}
\ No newline at end of file
diff --git a/e2e-tests/config/protocols/stellar/kv-store-test.json b/e2e-tests/config/protocols/stellar/kv-store-test.json
new file mode 100644
index 0000000000..f28fb18a75
--- /dev/null
+++ b/e2e-tests/config/protocols/stellar/kv-store-test.json
@@ -0,0 +1,108 @@
+{
+  "steps": [
+    {
+      "applicationInstall": {
+        "application": {
+          "localFile": "./apps/kv-store/res/kv_store.wasm"
+        },
+        "target": "allMembers"
+      }
+    },
+    {
+      "contextCreate": null
+    },
+    {
+      "contextCreateAlias": {
+        "aliasName": "my_context"
+      }
+    },
+    {
+      "call": {
+        "methodName": "set",
+        "argsJson": {
+          "key": "foo",
+          "value": "bar"
+        },
+        "expectedResultJson": null,
+        "target": "inviter"
+      }
+    },
+    {
+      "call": {
+        "methodName": "get",
+        "argsJson": {
+          "key": "foo"
+        },
+        "expectedResultJson": "bar",
+        "target": "inviter"
+      }
+    },
+    {
+      "contextInviteJoin": null
+    },
+    {
+      "wait": {
+        "for": "consensus",
+        "durationMs": 5000,
+        "description": [
+          "assuming it takes 5s to propagate we should",
+          "only need to wait 5 * ceil(log2(nodes)) seconds"
+        ]
+      }
+    },
+    {
+      "call": {
+        "methodName": "get",
+        "argsJson": {
+          "key": "foo"
+        },
+        "expectedResultJson": "bar",
+        "target": "allMembers",
+        "retries": 20,
+        "intervalMs": 5000,
+        "description": [
+          "if we don't reach consensus in the ideal case",
+          "wait 5 seconds for nodes that have not yet synced",
+          "in the worst case, wait 20 * 5s for nodes that uselessly",
+          "keep syncing with themselves without having the state"
+        ]
+      }
+    },
+    {
+      "call": {
+        "methodName": "set",
+        "argsJson": {
+          "key": "foo",
+          "value": "baz"
+        },
+        "expectedResultJson": null,
+        "target": "inviter"
+      }
+    },
+    {
+      "wait": {
+        "for": "broadcast",
+        "durationMs": 5000,
+        "description": [
+          "wait exactly 5s for the broadcast to propagate"
+        ]
+      }
+    },
+    {
+      "call": {
+        "methodName": "get",
+        "argsJson": {
+          "key": "foo"
+        },
+        "expectedResultJson": "baz",
+        "target": "allMembers",
+        "retries": 5,
+        "intervalMs": 5000,
+        "description": [
+          "if a node still hasn't received the broadcast",
+          "try 5 more times in 5 seconds, but no more"
+        ]
+      }
+    }
+  ]
+}
\ No newline at end of file
diff --git a/e2e-tests/src/driver.rs b/e2e-tests/src/driver.rs
index f20dbb39de..7e62ba6fed 100644
--- a/e2e-tests/src/driver.rs
+++ b/e2e-tests/src/driver.rs
@@ -24,7 +24,7 @@ use crate::protocol::near::NearSandboxEnvironment;
 use crate::protocol::stellar::StellarSandboxEnvironment;
 use crate::protocol::ProtocolSandboxEnvironment;
 use crate::steps::TestScenario;
-use crate::TestEnvironment;
+use crate::{Protocol, TestEnvironment};
 
 pub struct TestContext<'a> {
     pub inviter: String,
@@ -34,7 +34,7 @@ pub struct TestContext<'a> {
     pub context_id: Option<String>,
     pub inviter_public_key: Option<String>,
     pub invitees_public_keys: HashMap<String, String>,
-    pub protocol_name: &'a str,
+    pub protocol_name: &'a Protocol,
     pub protocol: &'a ProtocolSandboxEnvironment,
     pub output_writer: OutputWriter,
     pub context_alias: Option<String>,
@@ -52,7 +52,7 @@ impl<'a> TestContext<'a> {
         invitees: Vec<String>,
         meroctl: &'a Meroctl,
         output_writer: OutputWriter,
-        protocol_name: &'a str,
+        protocol_name: &'a Protocol,
         proposal_id: Option<String>,
         protocol: &'a ProtocolSandboxEnvironment,
     ) -> Self {
@@ -95,133 +95,124 @@ impl Driver {
         self.environment.init().await?;
 
         let mut report = TestRunReport::new();
-        let mut initialized_protocols: HashMap<String, ProtocolSandboxEnvironment> = HashMap::new();
-
-        // Run scenarios directory by directory
-        let scenarios_dir = self.environment.input_dir.join("scenarios");
-        let mut entries = read_dir(scenarios_dir).await?;
-
-        while let Some(entry) = entries.next_entry().await? {
-            let path = entry.path();
-            if path.is_dir() {
-                let test_file_path = path.join("test.json");
-                if test_file_path.exists() {
-                    let test_content = read(&test_file_path).await?;
-                    let test_json: serde_json::Value = from_slice(&test_content)?;
-
-                    if let Some(protocol_name) = test_json.get("protocol").and_then(|p| p.as_str())
-                    {
-                        // Skip if this isn't one of the requested scenarios/protocols
-                        if !self
-                            .environment
-                            .scenarios
-                            .iter()
-                            .any(|s| s.to_string().to_lowercase() == protocol_name)
-                        {
-                            continue;
-                        }
+        let mut initialized_protocols: HashMap<Protocol, ProtocolSandboxEnvironment> =
+            HashMap::new();
 
-                        // Initialize protocol if not already done
-                        if !initialized_protocols.contains_key(protocol_name) {
-                            // Find and initialize the protocol sandbox
-                            for protocol_sandbox in &self.config.protocol_sandboxes {
-                                let config_protocol_name = match protocol_sandbox {
-                                    ProtocolSandboxConfig::Stellar(_) => "stellar",
-                                    ProtocolSandboxConfig::Near(_) => "near",
-                                    ProtocolSandboxConfig::Icp(_) => "icp",
-                                    ProtocolSandboxConfig::Ethereum(_) => "ethereum",
-                                };
-
-                                if config_protocol_name == protocol_name {
-                                    let sandbox_env = match protocol_sandbox {
-                                        ProtocolSandboxConfig::Stellar(config) => {
-                                            ProtocolSandboxEnvironment::Stellar(
-                                                StellarSandboxEnvironment::init(config.clone())?,
-                                            )
-                                        }
-                                        ProtocolSandboxConfig::Near(config) => {
-                                            ProtocolSandboxEnvironment::Near(
-                                                NearSandboxEnvironment::init(config.clone())
-                                                    .await?,
-                                            )
-                                        }
-                                        ProtocolSandboxConfig::Icp(config) => {
-                                            ProtocolSandboxEnvironment::Icp(
-                                                IcpSandboxEnvironment::init(config.clone())?,
-                                            )
-                                        }
-                                        ProtocolSandboxConfig::Ethereum(config) => {
-                                            ProtocolSandboxEnvironment::Ethereum(
-                                                EthereumSandboxEnvironment::init(config.clone())?,
-                                            )
-                                        }
-                                    };
-                                    if initialized_protocols
-                                        .insert(protocol_name.to_owned(), sandbox_env)
-                                        .is_some()
-                                    {
-                                        self.environment.output_writer.write_str(&format!(
-                                            "Warning: Overwriting existing protocol {}",
-                                            protocol_name
-                                        ));
-                                    }
-                                    break;
-                                }
-                            }
-                        }
+        let protocols_dir = self.environment.input_dir.join("protocols");
 
-                        // If we have the protocol initialized, run the scenario
-                        if let Some(sandbox) = initialized_protocols.get(protocol_name) {
-                            let mero = self.setup_mero(&vec![sandbox.clone()]).await?;
-
-                            let Some((inviter, invitees)) = self.pick_inviter_node(&mero.ds) else {
-                                bail!("Not enough nodes to run the test")
-                            };
-
-                            self.environment
-                                .output_writer
-                                .write_str(&format!("Picked inviter: {inviter}"));
-                            self.environment
-                                .output_writer
-                                .write_str(&format!("Picked invitees: {invitees:?}"));
-
-                            let mut ctx = TestContext::new(
-                                inviter,
-                                invitees,
-                                &mero.ctl,
-                                self.environment.output_writer,
-                                protocol_name,
-                                None,
-                                sandbox,
-                            );
-
-                            // Parse the scenario from already loaded test_content
-                            let scenario: TestScenario = from_slice(&test_content)?;
-
-                            self.environment
-                                .output_writer
-                                .write_header(&format!("Running protocol {}", sandbox.name()), 1);
-
-                            report = self
-                                .run_scenarios(
-                                    &mut ctx,
-                                    report,
-                                    protocol_name,
-                                    scenario,
-                                    &test_file_path,
-                                )
-                                .await?;
-
-                            self.environment
-                                .output_writer
-                                .write_header(&format!("Finished protocol {}", sandbox.name()), 1);
-
-                            // Stop mero after running scenarios
-                            self.stop_merods(&mero.ds).await;
-                        }
+        for protocol in &self.environment.protocols {
+            if initialized_protocols.contains_key(protocol) {
+                continue;
+            }
+
+            for sandbox_cfg in &self.config.protocol_sandboxes {
+                let config_protocol = match sandbox_cfg {
+                    ProtocolSandboxConfig::Stellar(_) => Protocol::Stellar,
+                    ProtocolSandboxConfig::Near(_) => Protocol::Near,
+                    ProtocolSandboxConfig::Icp(_) => Protocol::Icp,
+                    ProtocolSandboxConfig::Ethereum(_) => Protocol::Ethereum,
+                };
+
+                if &config_protocol != protocol {
+                    continue;
+                }
+
+                let sandbox_env = match sandbox_cfg {
+                    ProtocolSandboxConfig::Stellar(config) => ProtocolSandboxEnvironment::Stellar(
+                        StellarSandboxEnvironment::init(config.clone())?,
+                    ),
+                    ProtocolSandboxConfig::Near(config) => ProtocolSandboxEnvironment::Near(
+                        NearSandboxEnvironment::init(config.clone()).await?,
+                    ),
+                    ProtocolSandboxConfig::Icp(config) => ProtocolSandboxEnvironment::Icp(
+                        IcpSandboxEnvironment::init(config.clone())?,
+                    ),
+                    ProtocolSandboxConfig::Ethereum(config) => {
+                        ProtocolSandboxEnvironment::Ethereum(EthereumSandboxEnvironment::init(
+                            config.clone(),
+                        )?)
                     }
+                };
+
+                initialized_protocols.insert(*protocol, sandbox_env);
+                break;
+            }
+        }
+
+        for protocol_name in &self.environment.protocols {
+            let protocol_path = protocols_dir.join(protocol_name.as_str());
+
+            if !protocol_path.is_dir() {
+                self.environment.output_writer.write_str(&format!(
+                    "No directory for protocol: {}",
+                    protocol_name.as_str()
+                ));
+                continue;
+            }
+
+            let Some(sandbox) = initialized_protocols.get(&protocol_name) else {
+                bail!(
+                    "Sandbox not initialized for protocol: {}",
+                    protocol_name.as_str()
+                );
+            };
+
+            let mero = self.setup_mero(&sandbox.clone()).await?;
+
+            let Some((inviter, invitees)) = self.pick_inviter_node(&mero.ds) else {
+                bail!(
+                    "Not enough nodes to run test for protocol {}",
+                    protocol_name.as_str()
+                )
+            };
+
+            self.environment
+                .output_writer
+                .write_str(&format!("Picked inviter: {inviter}"));
+            self.environment
+                .output_writer
+                .write_str(&format!("Picked invitees: {invitees:?}"));
+
+            let mut applications = read_dir(&protocol_path).await?;
+            while let Some(app) = applications.next_entry().await? {
+                if !app.file_type().await?.is_file() {
+                    continue;
+                }
+                let mut ctx = TestContext::new(
+                    inviter.clone(),
+                    invitees.clone(),
+                    &mero.ctl,
+                    self.environment.output_writer,
+                    protocol_name,
+                    None,
+                    sandbox,
+                );
+                let test_file_path = app.path();
+
+                let Some(app_name) = test_file_path.file_stem().and_then(|s| s.to_str()) else {
+                    bail!("No application name found");
+                };
+
+                if !test_file_path.is_file() {
+                    continue;
                 }
+                let test_content = read(&test_file_path).await?;
+                let scenario: TestScenario = from_slice(&test_content)?;
+
+                self.environment
+                    .output_writer
+                    .write_header(&format!("Running protocol {}", sandbox.name()), 1);
+
+                report = self
+                    .run_scenarios(&mut ctx, report, app_name, scenario, &test_file_path)
+                    .await?;
+
+                self.environment
+                    .output_writer
+                    .write_header(&format!("Finished protocol {}", sandbox.name()), 1);
             }
+
+            self.stop_merods(&mero.ds).await;
         }
 
         if let Err(e) = report.result() {
@@ -241,10 +232,7 @@ impl Driver {
         report.result()
     }
 
-    async fn setup_mero(
-        &self,
-        sandbox_environments: &Vec<ProtocolSandboxEnvironment>,
-    ) -> EyreResult<Mero> {
+    async fn setup_mero(&self, sandbox: &ProtocolSandboxEnvironment) -> EyreResult<Mero> {
         self.environment
             .output_writer
             .write_header("Starting merod nodes", 2);
@@ -265,11 +253,7 @@ impl Driver {
                     self.environment.test_id
                 )];
 
-                let mut node_args = vec![];
-                for sandbox in sandbox_environments {
-                    node_args = sandbox.node_args(&node_name).await?;
-                }
-
+                let node_args = sandbox.node_args(&node_name).await?;
                 let config_args = config_args.iter().chain(node_args.iter());
 
                 let merod = Merod::new(
@@ -342,20 +326,20 @@ impl Driver {
         &self,
         ctx: &mut TestContext<'_>,
         mut report: TestRunReport,
-        scenario_name: &str,
+        app_name: &str,
         scenario: TestScenario,
         file_path: &PathBuf,
     ) -> EyreResult<TestRunReport> {
         let scenario_report = self
-            .run_scenario(ctx, scenario_name, scenario, file_path)
+            .run_scenario(ctx, app_name, scenario, file_path)
             .await?;
 
         drop(
             report
                 .scenario_matrix
-                .entry(scenario_report.scenario_name.clone())
+                .entry(ctx.protocol_name.as_str().to_owned())
                 .or_default()
-                .insert(ctx.protocol_name.to_owned(), scenario_report),
+                .insert(app_name.to_owned(), scenario_report),
         );
 
         Ok(report)
@@ -364,7 +348,7 @@ impl Driver {
     async fn run_scenario(
         &self,
         ctx: &mut TestContext<'_>,
-        scenario_name: &str,
+        app_name: &str,
         scenario: TestScenario,
         file_path: &PathBuf,
     ) -> EyreResult<TestScenarioReport> {
@@ -379,7 +363,7 @@ impl Driver {
             .output_writer
             .write_str(&format!("Steps count: {}", scenario.steps.len()));
 
-        let mut report = TestScenarioReport::new(scenario_name.to_owned());
+        let mut report = TestScenarioReport::new(app_name.to_owned());
 
         let mut scenario_failed = false;
         for (i, step) in scenario.steps.iter().enumerate() {
@@ -515,54 +499,46 @@ impl TestRunReport {
 
     fn to_markdown(&self) -> EyreResult<String> {
         let mut markdown = String::new();
-
         writeln!(&mut markdown, "## E2E tests report")?;
 
-        for (scenario, protocols) in &self.scenario_matrix {
-            writeln!(&mut markdown, "### Scenario: {scenario}")?;
+        for (protocol, applications) in &self.scenario_matrix {
+            writeln!(&mut markdown, "### Protocol: {protocol}")?;
 
-            // Collecting all step names
-            let mut step_names = vec![];
-            for report in protocols.values() {
+            for (app_name, report) in applications {
+                let mut step_names = vec![];
                 for step in &report.steps {
                     if !step_names.contains(&step.step_name) {
                         step_names.push(step.step_name.clone());
                     }
                 }
-            }
-
-            // Write table header
-            write!(&mut markdown, "| Protocol/Step |")?;
-            for step_name in &step_names {
-                write!(&mut markdown, " {step_name} |")?;
-            }
-            writeln!(&mut markdown)?;
 
-            // Write table header separator
-            write!(&mut markdown, "| :--- |")?;
-            for _ in &step_names {
-                write!(&mut markdown, " :---: |")?;
-            }
-            writeln!(&mut markdown)?;
-
-            // Write table rows
-            for (protocol, report) in protocols {
-                write!(&mut markdown, "| {protocol} |")?;
+                // Write table header
+                write!(&mut markdown, "| Application/Step |")?;
+                for step_name in &step_names {
+                    write!(&mut markdown, " {step_name} |")?;
+                }
+                writeln!(&mut markdown)?;
+                // Write table header separator
+                write!(&mut markdown, "| :--- |")?;
+                for _ in &step_names {
+                    write!(&mut markdown, " :---: |")?;
+                }
+                writeln!(&mut markdown)?;
+                write!(&mut markdown, "| {app_name} |")?;
+                // Results row
                 for step_name in &step_names {
                     let result = report.steps.iter().find_map(|step| {
                         (&step.step_name == step_name).then_some(step.result.as_ref())
                     });
-
                     let result = match result {
                         None => "-",
                         Some(None) => ":fast_forward:",
                         Some(Some(Ok(_))) => ":white_check_mark:",
                         Some(Some(Err(_))) => ":x:",
                     };
-
                     write!(&mut markdown, " {result} |")?;
                 }
-                writeln!(&mut markdown)?;
+                writeln!(&mut markdown, "\n")?;
             }
             writeln!(&mut markdown)?;
         }
diff --git a/e2e-tests/src/main.rs b/e2e-tests/src/main.rs
index bf35c99472..ea6d487182 100644
--- a/e2e-tests/src/main.rs
+++ b/e2e-tests/src/main.rs
@@ -61,7 +61,7 @@ pub enum Commands {
 
 #[derive(Debug, Args)]
 pub struct RootArgs {
-    /// Directory containing the test configuration and test scenarios.
+    /// Directory containing the test configuration and test protocols.
     /// In root directory, there should be a `config.json` file. This file
     /// contains the configuration for the test run. Refer to the `Config`
     /// struct for more information.
@@ -90,26 +90,26 @@ pub struct RootArgs {
     #[arg(env = "E2E_OUTPUT_FORMAT", hide_env_values = true)]
     pub output_format: OutputFormat,
 
-    /// Scenarios to run
-    #[arg(long, value_name = "SCENARIO", value_enum, num_args = 1.., value_delimiter = ',')]
-    pub scenarios: Vec<Scenario>,
+    /// Protocols to run
+    #[arg(long, value_name = "PROTOCOL", value_enum, num_args = 1.., value_delimiter = ',')]
+    pub protocols: Vec<Protocol>,
 }
 
-#[derive(Debug, Clone, clap::ValueEnum, Copy)]
-pub enum Scenario {
+#[derive(Debug, Hash, Clone, clap::ValueEnum, Copy, PartialEq, Eq)]
+pub enum Protocol {
     Ethereum,
     Near,
     Stellar,
     Icp,
 }
 
-impl std::fmt::Display for Scenario {
-    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
+impl Protocol {
+    fn as_str(&self) -> &'static str {
         match self {
-            Scenario::Ethereum => write!(f, "ethereum"),
-            Scenario::Near => write!(f, "near"),
-            Scenario::Stellar => write!(f, "stellar"),
-            Scenario::Icp => write!(f, "icp"),
+            Protocol::Ethereum => "ethereum",
+            Protocol::Near => "near",
+            Protocol::Stellar => "stellar",
+            Protocol::Icp => "icp",
         }
     }
 }
@@ -125,7 +125,7 @@ pub struct TestEnvironment {
     pub logs_dir: Utf8PathBuf,
     pub icp_dir: Utf8PathBuf,
     pub output_writer: OutputWriter,
-    pub scenarios: Vec<Scenario>,
+    pub protocols: Vec<Protocol>,
 }
 
 impl From<RootArgs> for TestEnvironment {
@@ -142,7 +142,7 @@ impl From<RootArgs> for TestEnvironment {
             logs_dir: val.output_dir.join("logs"),
             icp_dir: val.output_dir.join("icp"),
             output_writer: OutputWriter::new(val.output_format),
-            scenarios: val.scenarios,
+            protocols: val.protocols,
         }
     }
 }
diff --git a/e2e-tests/src/steps/context_create.rs b/e2e-tests/src/steps/context_create.rs
index 7319a2a298..ab84187bd3 100644
--- a/e2e-tests/src/steps/context_create.rs
+++ b/e2e-tests/src/steps/context_create.rs
@@ -19,7 +19,7 @@ impl Test for ContextCreateStep {
 
         let (context_id, member_public_key) = ctx
             .meroctl
-            .context_create(&ctx.inviter, application_id, &ctx.protocol_name)
+            .context_create(&ctx.inviter, application_id, ctx.protocol_name.as_str())
             .await?;
 
         ctx.context_id = Some(context_id);
diff --git a/e2e-tests/src/steps/verify_external_state.rs b/e2e-tests/src/steps/verify_external_state.rs
index 739cd42411..f63b4d6eb4 100644
--- a/e2e-tests/src/steps/verify_external_state.rs
+++ b/e2e-tests/src/steps/verify_external_state.rs
@@ -50,7 +50,8 @@ impl Test for VerifyExternalStateStep {
     async fn run_assert(&self, ctx: &mut TestContext<'_>) -> EyreResult<()> {
         ctx.output_writer.write_str(&format!(
             "Verifying external contract state in {} protocol for contract: {}",
-            ctx.protocol_name, self.contract_id
+            ctx.protocol_name.as_str(),
+            self.contract_id
         ));
 
         let max_retries = self.retries.unwrap_or(1);


Reviews:
user: login: fbozic
body: We are missing ✅ or ❌ for the first step. Can you rename `test` to `demo-blockchain-integrations` as well?
<img width="290" alt="image" src="https://github.com/user-attachments/assets/aea5d756-b89e-44d0-b041-e3ace8ef6bf7" />

I think the simple way to implement this is to:
- iterate over each protocol from arguments
  - init protocol sandboxes
- iterate over each protocol from arguments
  - init merod and meroctl (you can now get a reference to initialized protocol sandbox required for init)
  - iterate over each application
    - run application test
  - teardown merod and meroctl

state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-30T09:51:53Z
user: login: cy4n1d3-p1x3l
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-02T22:02:22Z
user: login: fbozic
body: Looks better, but I think you have a bug in the code and the tests are failing because of that. 
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-06T16:33:04Z
user: login: cy4n1d3-p1x3l
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-06T16:58:03Z
user: login: miraclx
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-13T19:40:29Z
user: login: cy4n1d3-p1x3l
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-16T21:04:48Z
user: login: miraclx
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-17T00:47:04Z
user: login: cy4n1d3-p1x3l
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-18T06:35:45Z
user: login: cy4n1d3-p1x3l
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-20T18:31:55Z
user: login: cy4n1d3-p1x3l
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-20T18:35:37Z
user: login: cy4n1d3-p1x3l
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-20T18:36:03Z
user: login: miraclx
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-22T22:25:23Z
user: login: miraclx
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-22T22:31:32Z
user: login: miraclx
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-22T22:35:42Z
user: login: miraclx
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-27T04:05:28Z
user: login: miraclx
body: the rest looks okay to me, I've pinged the other guys to pitch in about the comments in https://github.com/calimero-network/core/pull/1236#discussion_r2087487563 & https://github.com/calimero-network/core/pull/1236#discussion_r2103486645
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-28T11:29:14Z
user: login: miraclx
body: So we discussed the issues mentioned above and they seem to be the same on `master`, which, while incorrect is out of scope for this PR.

Thanks for driving this to completion. 🎉 
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-05-28T14:30:28Z
user: login: fbozic
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-05-28T15:17:58Z

Review comments:
diff_hunk: @@ -72,9 +73,11 @@ impl NearSandboxEnvironment {
     }
 
     pub async fn node_args(&self, node_name: &str) -> EyreResult<Vec<String>> {
+        let timestamp = Utc::now().timestamp();
+        let unique_node_name = format!("{}-{}", node_name, timestamp);
path: e2e-tests/src/protocol/near.rs
user: login: fbozic
body: I don't understand this change. Is it leftover from testing? One protocol sandbox should be enough for multiple apps that are being tested.
created_at: 2025-04-30T09:36:16Z
author_association: CONTRIBUTOR
diff_hunk: @@ -72,9 +73,11 @@ impl NearSandboxEnvironment {
     }
 
     pub async fn node_args(&self, node_name: &str) -> EyreResult<Vec<String>> {
+        let timestamp = Utc::now().timestamp();
+        let unique_node_name = format!("{}-{}", node_name, timestamp);
path: e2e-tests/src/protocol/near.rs
user: login: cy4n1d3-p1x3l
body: yeah it should be
initially testing was giving me errors
created_at: 2025-05-02T22:02:22Z
author_association: CONTRIBUTOR
diff_hunk: @@ -97,131 +97,113 @@ impl Driver {
         let mut report = TestRunReport::new();
         let mut initialized_protocols: HashMap<String, ProtocolSandboxEnvironment> = HashMap::new();
 
-        // Run scenarios directory by directory
-        let scenarios_dir = self.environment.input_dir.join("scenarios");
-        let mut entries = read_dir(scenarios_dir).await?;
-
-        while let Some(entry) = entries.next_entry().await? {
-            let path = entry.path();
-            if path.is_dir() {
-                let test_file_path = path.join("test.json");
+        let scenarios_dir = self.environment.input_dir.join("protocols");
path: e2e-tests/src/driver.rs
user: login: fbozic
body: `protocols_dir` would probably be better name for variable
created_at: 2025-05-06T16:18:00Z
author_association: CONTRIBUTOR
diff_hunk: @@ -97,131 +97,113 @@ impl Driver {
         let mut report = TestRunReport::new();
         let mut initialized_protocols: HashMap<String, ProtocolSandboxEnvironment> = HashMap::new();
 
-        // Run scenarios directory by directory
-        let scenarios_dir = self.environment.input_dir.join("scenarios");
-        let mut entries = read_dir(scenarios_dir).await?;
-
-        while let Some(entry) = entries.next_entry().await? {
-            let path = entry.path();
-            if path.is_dir() {
-                let test_file_path = path.join("test.json");
+        let scenarios_dir = self.environment.input_dir.join("protocols");
+
+        for protocol_name in &self.environment.scenarios {
path: e2e-tests/src/driver.rs
user: login: fbozic
body: I think you are not checking anymore if `--protocols` (`self.environment.protocols`) includes a protocol for which we are setting up the sandbox.   
Check this comment: https://github.com/calimero-network/core/pull/1236#pullrequestreview-2806469272, the simplest way should be to iterate over `self.environment.protocols`.
created_at: 2025-05-06T16:32:14Z
author_association: CONTRIBUTOR
diff_hunk: @@ -97,131 +97,113 @@ impl Driver {
         let mut report = TestRunReport::new();
         let mut initialized_protocols: HashMap<String, ProtocolSandboxEnvironment> = HashMap::new();
 
-        // Run scenarios directory by directory
-        let scenarios_dir = self.environment.input_dir.join("scenarios");
-        let mut entries = read_dir(scenarios_dir).await?;
-
-        while let Some(entry) = entries.next_entry().await? {
-            let path = entry.path();
-            if path.is_dir() {
-                let test_file_path = path.join("test.json");
+        let scenarios_dir = self.environment.input_dir.join("protocols");
+
+        for protocol_name in &self.environment.scenarios {
path: e2e-tests/src/driver.rs
user: login: cy4n1d3-p1x3l
body: i did not yet rename scenario to protocol in many places. Is the confusion arising because of that?
Because i've implemented exactly like you commented.
The tests ran fine once which generated the report we can see in the comments but after i pushed another commit the tests failed.
created_at: 2025-05-06T16:58:03Z
author_association: CONTRIBUTOR
diff_hunk: @@ -97,131 +97,113 @@ impl Driver {
         let mut report = TestRunReport::new();
         let mut initialized_protocols: HashMap<String, ProtocolSandboxEnvironment> = HashMap::new();
 
-        // Run scenarios directory by directory
-        let scenarios_dir = self.environment.input_dir.join("scenarios");
-        let mut entries = read_dir(scenarios_dir).await?;
-
-        while let Some(entry) = entries.next_entry().await? {
-            let path = entry.path();
-            if path.is_dir() {
-                let test_file_path = path.join("test.json");
+        let protocols_dir = self.environment.input_dir.join("protocols");
+
+        for protocol_name in &self.environment.protocols {
+            let protocol_name = protocol_name.to_string().to_lowercase();
path: e2e-tests/src/driver.rs
user: login: miraclx
body: impl `Eq, PartialEq` instead of string comparison
created_at: 2025-05-13T18:19:27Z
author_association: CONTRIBUTOR
diff_hunk: @@ -97,131 +97,113 @@ impl Driver {
         let mut report = TestRunReport::new();
         let mut initialized_protocols: HashMap<String, ProtocolSandboxEnvironment> = HashMap::new();
 
-        // Run scenarios directory by directory
-        let scenarios_dir = self.environment.input_dir.join("scenarios");
-        let mut entries = read_dir(scenarios_dir).await?;
-
-        while let Some(entry) = entries.next_entry().await? {
-            let path = entry.path();
-            if path.is_dir() {
-                let test_file_path = path.join("test.json");
+        let protocols_dir = self.environment.input_dir.join("protocols");
+
+        for protocol_name in &self.environment.protocols {
+            let protocol_name = protocol_name.to_string().to_lowercase();
+
+            if initialized_protocols.contains_key(&protocol_name) {
+                continue;
+            }
+
+            for sandbox_cfg in &self.config.protocol_sandboxes {
+                let config_protocol_name = match sandbox_cfg {
+                    ProtocolSandboxConfig::Stellar(_) => "stellar",
+                    ProtocolSandboxConfig::Near(_) => "near",
+                    ProtocolSandboxConfig::Icp(_) => "icp",
+                    ProtocolSandboxConfig::Ethereum(_) => "ethereum",
+                };
+
+                if config_protocol_name == protocol_name {
+                    let sandbox_env = match sandbox_cfg {
+                        ProtocolSandboxConfig::Stellar(config) => {
+                            ProtocolSandboxEnvironment::Stellar(StellarSandboxEnvironment::init(
path: e2e-tests/src/driver.rs
user: login: miraclx
body: here, there's no guarantee we don't call `init` twice for the same protocol


created_at: 2025-05-13T18:41:40Z
author_association: CONTRIBUTOR
diff_hunk: @@ -97,131 +97,113 @@ impl Driver {
         let mut report = TestRunReport::new();
         let mut initialized_protocols: HashMap<String, ProtocolSandboxEnvironment> = HashMap::new();
 
-        // Run scenarios directory by directory
-        let scenarios_dir = self.environment.input_dir.join("scenarios");
-        let mut entries = read_dir(scenarios_dir).await?;
-
-        while let Some(entry) = entries.next_entry().await? {
-            let path = entry.path();
-            if path.is_dir() {
-                let test_file_path = path.join("test.json");
+        let protocols_dir = self.environment.input_dir.join("protocols");
+
+        for protocol_name in &self.environment.protocols {
+            let protocol_name = protocol_name.to_string().to_lowercase();
+
+            if initialized_protocols.contains_key(&protocol_name) {
+                continue;
+            }
+
+            for sandbox_cfg in &self.config.protocol_sandboxes {
+                let config_protocol_name = match sandbox_cfg {
+                    ProtocolSandboxConfig::Stellar(_) => "stellar",
+                    ProtocolSandboxConfig::Near(_) => "near",
+                    ProtocolSandboxConfig::Icp(_) => "icp",
+                    ProtocolSandboxConfig::Ethereum(_) => "ethereum",
+                };
+
+                if config_protocol_name == protocol_name {
+                    let sandbox_env = match sandbox_cfg {
+                        ProtocolSandboxConfig::Stellar(config) => {
+                            ProtocolSandboxEnvironment::Stellar(StellarSandboxEnvironment::init(
+                                config.clone(),
+                            )?)
+                        }
+                        ProtocolSandboxConfig::Near(config) => ProtocolSandboxEnvironment::Near(
+                            NearSandboxEnvironment::init(config.clone()).await?,
+                        ),
+                        ProtocolSandboxConfig::Icp(config) => ProtocolSandboxEnvironment::Icp(
+                            IcpSandboxEnvironment::init(config.clone())?,
+                        ),
+                        ProtocolSandboxConfig::Ethereum(config) => {
+                            ProtocolSandboxEnvironment::Ethereum(EthereumSandboxEnvironment::init(
+                                config.clone(),
+                            )?)
+                        }
+                    };
+
+                    initialized_protocols.insert(protocol_name.clone(), sandbox_env);
+                    break;
+                }
+            }
+        }
+
+        for protocol_name in &self.environment.protocols {
path: e2e-tests/src/driver.rs
user: login: miraclx
body: why iterate over the same list twice?
created_at: 2025-05-13T19:01:15Z
author_association: CONTRIBUTOR
diff_hunk: @@ -97,131 +97,113 @@ impl Driver {
         let mut report = TestRunReport::new();
         let mut initialized_protocols: HashMap<String, ProtocolSandboxEnvironment> = HashMap::new();
 
-        // Run scenarios directory by directory
-        let scenarios_dir = self.environment.input_dir.join("scenarios");
-        let mut entries = read_dir(scenarios_dir).await?;
-
-        while let Some(entry) = entries.next_entry().await? {
-            let path = entry.path();
-            if path.is_dir() {
-                let test_file_path = path.join("test.json");
+        let protocols_dir = self.environment.input_dir.join("protocols");
+
+        for protocol_name in &self.environment.protocols {
+            let protocol_name = protocol_name.to_string().to_lowercase();
+
+            if initialized_protocols.contains_key(&protocol_name) {
+                continue;
+            }
+
+            for sandbox_cfg in &self.config.protocol_sandboxes {
+                let config_protocol_name = match sandbox_cfg {
+                    ProtocolSandboxConfig::Stellar(_) => "stellar",
+                    ProtocolSandboxConfig::Near(_) => "near",
+                    ProtocolSandboxConfig::Icp(_) => "icp",
+                    ProtocolSandboxConfig::Ethereum(_) => "ethereum",
+                };
+
+                if config_protocol_name == protocol_name {
+                    let sandbox_env = match sandbox_cfg {
+                        ProtocolSandboxConfig::Stellar(config) => {
+                            ProtocolSandboxEnvironment::Stellar(StellarSandboxEnvironment::init(
+                                config.clone(),
+                            )?)
+                        }
+                        ProtocolSandboxConfig::Near(config) => ProtocolSandboxEnvironment::Near(
+                            NearSandboxEnvironment::init(config.clone()).await?,
+                        ),
+                        ProtocolSandboxConfig::Icp(config) => ProtocolSandboxEnvironment::Icp(
+                            IcpSandboxEnvironment::init(config.clone())?,
+                        ),
+                        ProtocolSandboxConfig::Ethereum(config) => {
+                            ProtocolSandboxEnvironment::Ethereum(EthereumSandboxEnvironment::init(
+                                config.clone(),
+                            )?)
+                        }
+                    };
+
+                    initialized_protocols.insert(protocol_name.clone(), sandbox_env);
+                    break;
+                }
+            }
+        }
+
+        for protocol_name in &self.environment.protocols {
+            let protocol_name = protocol_name.to_string().to_lowercase();
+            let protocol_path = protocols_dir.join(&protocol_name);
+
+            if !protocol_path.exists() || !protocol_path.is_dir() {
path: e2e-tests/src/driver.rs
user: login: miraclx
body: [`Utf8Path::is_dir`](https://docs.rs/camino/latest/camino/struct.Utf8PathBuf.html#method.is_dir) will only be true if it exists and is a dir

```suggestion
            if !protocol_path.is_dir() {
```
created_at: 2025-05-13T19:03:43Z
author_association: CONTRIBUTOR
diff_hunk: @@ -92,26 +92,26 @@ pub struct RootArgs {
     #[arg(env = "E2E_OUTPUT_FORMAT", hide_env_values = true)]
     pub output_format: OutputFormat,
 
-    /// Scenarios to run
-    #[arg(long, value_name = "SCENARIO", value_enum, num_args = 1.., value_delimiter = ',')]
-    pub scenarios: Vec<Scenario>,
+    /// Protocols to run
+    #[arg(long, value_name = "PROTOCOL", value_enum, num_args = 1.., value_delimiter = ',')]
+    pub protocols: Vec<Protocol>,
 }
 
 #[derive(Debug, Clone, clap::ValueEnum, Copy)]
-pub enum Scenario {
+pub enum Protocol {
     Ethereum,
     Near,
     Stellar,
     Icp,
 }
 
-impl std::fmt::Display for Scenario {
+impl std::fmt::Display for Protocol {
     fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
         match self {
-            Scenario::Ethereum => write!(f, "ethereum"),
-            Scenario::Near => write!(f, "near"),
-            Scenario::Stellar => write!(f, "stellar"),
-            Scenario::Icp => write!(f, "icp"),
+            Protocol::Ethereum => write!(f, "ethereum"),
+            Protocol::Near => write!(f, "near"),
+            Protocol::Stellar => write!(f, "stellar"),
+            Protocol::Icp => write!(f, "icp"),
         }
     }
 }
path: e2e-tests/src/main.rs
user: login: miraclx
body: first, introduce a `Protocol::as_str` method which you can then call from here (if it's absolutely necessary, I'm not convinced it is)
created_at: 2025-05-13T19:12:17Z
author_association: CONTRIBUTOR
diff_hunk: @@ -515,54 +497,46 @@ impl TestRunReport {
 
     fn to_markdown(&self) -> EyreResult<String> {
         let mut markdown = String::new();
-
         writeln!(&mut markdown, "## E2E tests report")?;
 
-        for (scenario, protocols) in &self.scenario_matrix {
-            writeln!(&mut markdown, "### Scenario: {scenario}")?;
+        for (protocol, applications) in &self.scenario_matrix {
+            writeln!(&mut markdown, "## Protocol: {protocol}")?;
 
-            // Collecting all step names
-            let mut step_names = vec![];
-            for report in protocols.values() {
+            for (app_name, report) in applications {
+                let mut step_names = vec![];
                 for step in &report.steps {
                     if !step_names.contains(&step.step_name) {
                         step_names.push(step.step_name.clone());
                     }
                 }
-            }
-
-            // Write table header
-            write!(&mut markdown, "| Protocol/Step |")?;
-            for step_name in &step_names {
-                write!(&mut markdown, " {step_name} |")?;
-            }
-            writeln!(&mut markdown)?;
 
-            // Write table header separator
-            write!(&mut markdown, "| :--- |")?;
-            for _ in &step_names {
-                write!(&mut markdown, " :---: |")?;
-            }
-            writeln!(&mut markdown)?;
-
-            // Write table rows
-            for (protocol, report) in protocols {
-                write!(&mut markdown, "| {protocol} |")?;
+                // Write table header
+                write!(&mut markdown, "| Application/Step |")?;
+                for step_name in &step_names {
+                    write!(&mut markdown, " {step_name} |")?;
+                }
+                writeln!(&mut markdown)?;
+                // Write table header separator
+                write!(&mut markdown, "| :--- |")?;
+                for _ in &step_names {
+                    write!(&mut markdown, " :---: |")?;
+                }
+                writeln!(&mut markdown)?;
+                write!(&mut markdown, "| {app_name} |")?;
+                // Results row
                 for step_name in &step_names {
                     let result = report.steps.iter().find_map(|step| {
                         (&step.step_name == step_name).then_some(step.result.as_ref())
                     });
-
                     let result = match result {
                         None => "-",
                         Some(None) => ":fast_forward:",
                         Some(Some(Ok(_))) => ":white_check_mark:",
                         Some(Some(Err(_))) => ":x:",
                     };
-
                     write!(&mut markdown, " {result} |")?;
                 }
-                writeln!(&mut markdown)?;
+                writeln!(&mut markdown, "\n")?;
path: e2e-tests/src/driver.rs
user: login: miraclx
body: this is `writeln!`, so unless you want two newlines, it's not necessary
created_at: 2025-05-13T19:14:47Z
author_association: CONTRIBUTOR
diff_hunk: @@ -97,131 +97,113 @@ impl Driver {
         let mut report = TestRunReport::new();
         let mut initialized_protocols: HashMap<String, ProtocolSandboxEnvironment> = HashMap::new();
 
-        // Run scenarios directory by directory
-        let scenarios_dir = self.environment.input_dir.join("scenarios");
-        let mut entries = read_dir(scenarios_dir).await?;
-
-        while let Some(entry) = entries.next_entry().await? {
-            let path = entry.path();
-            if path.is_dir() {
-                let test_file_path = path.join("test.json");
+        let protocols_dir = self.environment.input_dir.join("protocols");
+
+        for protocol_name in &self.environment.protocols {
+            let protocol_name = protocol_name.to_string().to_lowercase();
+
+            if initialized_protocols.contains_key(&protocol_name) {
+                continue;
+            }
+
+            for sandbox_cfg in &self.config.protocol_sandboxes {
+                let config_protocol_name = match sandbox_cfg {
+                    ProtocolSandboxConfig::Stellar(_) => "stellar",
+                    ProtocolSandboxConfig::Near(_) => "near",
+                    ProtocolSandboxConfig::Icp(_) => "icp",
+                    ProtocolSandboxConfig::Ethereum(_) => "ethereum",
+                };
+
+                if config_protocol_name == protocol_name {
+                    let sandbox_env = match sandbox_cfg {
+                        ProtocolSandboxConfig::Stellar(config) => {
+                            ProtocolSandboxEnvironment::Stellar(StellarSandboxEnvironment::init(
+                                config.clone(),
+                            )?)
+                        }
+                        ProtocolSandboxConfig::Near(config) => ProtocolSandboxEnvironment::Near(
+                            NearSandboxEnvironment::init(config.clone()).await?,
+                        ),
+                        ProtocolSandboxConfig::Icp(config) => ProtocolSandboxEnvironment::Icp(
+                            IcpSandboxEnvironment::init(config.clone())?,
+                        ),
+                        ProtocolSandboxConfig::Ethereum(config) => {
+                            ProtocolSandboxEnvironment::Ethereum(EthereumSandboxEnvironment::init(
+                                config.clone(),
+                            )?)
+                        }
+                    };
+
+                    initialized_protocols.insert(protocol_name.clone(), sandbox_env);
+                    break;
+                }
+            }
+        }
+
+        for protocol_name in &self.environment.protocols {
+            let protocol_name = protocol_name.to_string().to_lowercase();
+            let protocol_path = protocols_dir.join(&protocol_name);
+
+            if !protocol_path.exists() || !protocol_path.is_dir() {
+                self.environment
+                    .output_writer
+                    .write_str(&format!("No directory for protocol: {}", protocol_name));
+                continue;
+            }
+
+            let Some(sandbox) = initialized_protocols.get(&protocol_name) else {
+                bail!("Sandbox not initialized for protocol: {}", protocol_name);
+            };
+
+            let mero = self.setup_mero(&vec![sandbox.clone()]).await?;
path: e2e-tests/src/driver.rs
user: login: miraclx
body: 1. you can change `setup_mero` to accept a `&[_]` instead of a `&Vec<_>`

    ```suggestion
                let mero = self.setup_mero(&[sandbox.clone()]).await?;
    ```

    and finally you won't even need to clone if you took in a `&[impl AsRef<ProtocolSandboxEnvironment>]`

2. won't this mean we'll only ever have nodes with 1 protocol configuration? even though we can specify `--protocols near,ethereum`, it'll only setup `node1` with `near`, and `node2` with `ethereum`, forcing `node1` to use the relayer when it needs to do any ethereum operations and vice-versa, correct me if I'm wrong

cc: @fbozic 
created_at: 2025-05-13T19:25:07Z
author_association: CONTRIBUTOR
diff_hunk: @@ -97,131 +97,113 @@ impl Driver {
         let mut report = TestRunReport::new();
         let mut initialized_protocols: HashMap<String, ProtocolSandboxEnvironment> = HashMap::new();
 
-        // Run scenarios directory by directory
-        let scenarios_dir = self.environment.input_dir.join("scenarios");
-        let mut entries = read_dir(scenarios_dir).await?;
-
-        while let Some(entry) = entries.next_entry().await? {
-            let path = entry.path();
-            if path.is_dir() {
-                let test_file_path = path.join("test.json");
+        let protocols_dir = self.environment.input_dir.join("protocols");
+
+        for protocol_name in &self.environment.protocols {
+            let protocol_name = protocol_name.to_string().to_lowercase();
+
+            if initialized_protocols.contains_key(&protocol_name) {
+                continue;
+            }
+
+            for sandbox_cfg in &self.config.protocol_sandboxes {
+                let config_protocol_name = match sandbox_cfg {
+                    ProtocolSandboxConfig::Stellar(_) => "stellar",
+                    ProtocolSandboxConfig::Near(_) => "near",
+                    ProtocolSandboxConfig::Icp(_) => "icp",
+                    ProtocolSandboxConfig::Ethereum(_) => "ethereum",
+                };
+
+                if config_protocol_name == protocol_name {
+                    let sandbox_env = match sandbox_cfg {
+                        ProtocolSandboxConfig::Stellar(config) => {
+                            ProtocolSandboxEnvironment::Stellar(StellarSandboxEnvironment::init(
+                                config.clone(),
+                            )?)
+                        }
+                        ProtocolSandboxConfig::Near(config) => ProtocolSandboxEnvironment::Near(
+                            NearSandboxEnvironment::init(config.clone()).await?,
+                        ),
+                        ProtocolSandboxConfig::Icp(config) => ProtocolSandboxEnvironment::Icp(
+                            IcpSandboxEnvironment::init(config.clone())?,
+                        ),
+                        ProtocolSandboxConfig::Ethereum(config) => {
+                            ProtocolSandboxEnvironment::Ethereum(EthereumSandboxEnvironment::init(
+                                config.clone(),
+                            )?)
+                        }
+                    };
+
+                    initialized_protocols.insert(protocol_name.clone(), sandbox_env);
+                    break;
+                }
+            }
+        }
+
+        for protocol_name in &self.environment.protocols {
+            let protocol_name = protocol_name.to_string().to_lowercase();
+            let protocol_path = protocols_dir.join(&protocol_name);
+
+            if !protocol_path.exists() || !protocol_path.is_dir() {
+                self.environment
+                    .output_writer
+                    .write_str(&format!("No directory for protocol: {}", protocol_name));
+                continue;
+            }
+
+            let Some(sandbox) = initialized_protocols.get(&protocol_name) else {
+                bail!("Sandbox not initialized for protocol: {}", protocol_name);
+            };
+
+            let mero = self.setup_mero(&vec![sandbox.clone()]).await?;
+
+            let Some((inviter, invitees)) = self.pick_inviter_node(&mero.ds) else {
+                bail!("Not enough nodes to run test for protocol {protocol_name}")
+            };
+
+            self.environment
+                .output_writer
+                .write_str(&format!("Picked inviter: {inviter}"));
+            self.environment
+                .output_writer
+                .write_str(&format!("Picked invitees: {invitees:?}"));
path: e2e-tests/src/driver.rs
user: login: miraclx
body: we'll also be printing these N times for every protocol
created_at: 2025-05-13T19:30:25Z
author_association: CONTRIBUTOR
diff_hunk: @@ -97,131 +97,113 @@ impl Driver {
         let mut report = TestRunReport::new();
         let mut initialized_protocols: HashMap<String, ProtocolSandboxEnvironment> = HashMap::new();
 
-        // Run scenarios directory by directory
-        let scenarios_dir = self.environment.input_dir.join("scenarios");
-        let mut entries = read_dir(scenarios_dir).await?;
-
-        while let Some(entry) = entries.next_entry().await? {
-            let path = entry.path();
-            if path.is_dir() {
-                let test_file_path = path.join("test.json");
+        let protocols_dir = self.environment.input_dir.join("protocols");
+
+        for protocol_name in &self.environment.protocols {
+            let protocol_name = protocol_name.to_string().to_lowercase();
+
+            if initialized_protocols.contains_key(&protocol_name) {
+                continue;
+            }
+
+            for sandbox_cfg in &self.config.protocol_sandboxes {
+                let config_protocol_name = match sandbox_cfg {
+                    ProtocolSandboxConfig::Stellar(_) => "stellar",
+                    ProtocolSandboxConfig::Near(_) => "near",
+                    ProtocolSandboxConfig::Icp(_) => "icp",
+                    ProtocolSandboxConfig::Ethereum(_) => "ethereum",
+                };
+
+                if config_protocol_name == protocol_name {
+                    let sandbox_env = match sandbox_cfg {
+                        ProtocolSandboxConfig::Stellar(config) => {
+                            ProtocolSandboxEnvironment::Stellar(StellarSandboxEnvironment::init(
+                                config.clone(),
+                            )?)
+                        }
+                        ProtocolSandboxConfig::Near(config) => ProtocolSandboxEnvironment::Near(
+                            NearSandboxEnvironment::init(config.clone()).await?,
+                        ),
+                        ProtocolSandboxConfig::Icp(config) => ProtocolSandboxEnvironment::Icp(
+                            IcpSandboxEnvironment::init(config.clone())?,
+                        ),
+                        ProtocolSandboxConfig::Ethereum(config) => {
+                            ProtocolSandboxEnvironment::Ethereum(EthereumSandboxEnvironment::init(
+                                config.clone(),
+                            )?)
+                        }
+                    };
+
+                    initialized_protocols.insert(protocol_name.clone(), sandbox_env);
+                    break;
+                }
+            }
+        }
+
+        for protocol_name in &self.environment.protocols {
+            let protocol_name = protocol_name.to_string().to_lowercase();
+            let protocol_path = protocols_dir.join(&protocol_name);
+
+            if !protocol_path.exists() || !protocol_path.is_dir() {
+                self.environment
+                    .output_writer
+                    .write_str(&format!("No directory for protocol: {}", protocol_name));
+                continue;
+            }
+
+            let Some(sandbox) = initialized_protocols.get(&protocol_name) else {
+                bail!("Sandbox not initialized for protocol: {}", protocol_name);
+            };
+
+            let mero = self.setup_mero(&vec![sandbox.clone()]).await?;
+
+            let Some((inviter, invitees)) = self.pick_inviter_node(&mero.ds) else {
+                bail!("Not enough nodes to run test for protocol {protocol_name}")
+            };
+
+            self.environment
+                .output_writer
+                .write_str(&format!("Picked inviter: {inviter}"));
+            self.environment
+                .output_writer
+                .write_str(&format!("Picked invitees: {invitees:?}"));
+
+            let mut applications = read_dir(&protocol_path).await?;
+            while let Some(app) = applications.next_entry().await? {
path: e2e-tests/src/driver.rs
user: login: miraclx
body: there's no checks to make sure the contents of the protocol path is a file
created_at: 2025-05-13T19:34:58Z
author_association: CONTRIBUTOR
diff_hunk: @@ -97,131 +97,113 @@ impl Driver {
         let mut report = TestRunReport::new();
         let mut initialized_protocols: HashMap<String, ProtocolSandboxEnvironment> = HashMap::new();
 
-        // Run scenarios directory by directory
-        let scenarios_dir = self.environment.input_dir.join("scenarios");
-        let mut entries = read_dir(scenarios_dir).await?;
-
-        while let Some(entry) = entries.next_entry().await? {
-            let path = entry.path();
-            if path.is_dir() {
-                let test_file_path = path.join("test.json");
+        let protocols_dir = self.environment.input_dir.join("protocols");
+
+        for protocol_name in &self.environment.protocols {
+            let protocol_name = protocol_name.to_string().to_lowercase();
+
+            if initialized_protocols.contains_key(&protocol_name) {
+                continue;
+            }
+
+            for sandbox_cfg in &self.config.protocol_sandboxes {
+                let config_protocol_name = match sandbox_cfg {
+                    ProtocolSandboxConfig::Stellar(_) => "stellar",
+                    ProtocolSandboxConfig::Near(_) => "near",
+                    ProtocolSandboxConfig::Icp(_) => "icp",
+                    ProtocolSandboxConfig::Ethereum(_) => "ethereum",
+                };
+
+                if config_protocol_name == protocol_name {
+                    let sandbox_env = match sandbox_cfg {
+                        ProtocolSandboxConfig::Stellar(config) => {
+                            ProtocolSandboxEnvironment::Stellar(StellarSandboxEnvironment::init(
+                                config.clone(),
+                            )?)
+                        }
+                        ProtocolSandboxConfig::Near(config) => ProtocolSandboxEnvironment::Near(
+                            NearSandboxEnvironment::init(config.clone()).await?,
+                        ),
+                        ProtocolSandboxConfig::Icp(config) => ProtocolSandboxEnvironment::Icp(
+                            IcpSandboxEnvironment::init(config.clone())?,
+                        ),
+                        ProtocolSandboxConfig::Ethereum(config) => {
+                            ProtocolSandboxEnvironment::Ethereum(EthereumSandboxEnvironment::init(
+                                config.clone(),
+                            )?)
+                        }
+                    };
+
+                    initialized_protocols.insert(protocol_name.clone(), sandbox_env);
+                    break;
+                }
+            }
+        }
+
+        for protocol_name in &self.environment.protocols {
+            let protocol_name = protocol_name.to_string().to_lowercase();
+            let protocol_path = protocols_dir.join(&protocol_name);
+
+            if !protocol_path.exists() || !protocol_path.is_dir() {
+                self.environment
+                    .output_writer
+                    .write_str(&format!("No directory for protocol: {}", protocol_name));
+                continue;
+            }
+
+            let Some(sandbox) = initialized_protocols.get(&protocol_name) else {
+                bail!("Sandbox not initialized for protocol: {}", protocol_name);
+            };
+
+            let mero = self.setup_mero(&vec![sandbox.clone()]).await?;
+
+            let Some((inviter, invitees)) = self.pick_inviter_node(&mero.ds) else {
+                bail!("Not enough nodes to run test for protocol {protocol_name}")
+            };
+
+            self.environment
+                .output_writer
+                .write_str(&format!("Picked inviter: {inviter}"));
+            self.environment
+                .output_writer
+                .write_str(&format!("Picked invitees: {invitees:?}"));
+
+            let mut applications = read_dir(&protocol_path).await?;
+            while let Some(app) = applications.next_entry().await? {
+                let mut ctx = TestContext::new(
+                    inviter.clone(),
+                    invitees.clone(),
+                    &mero.ctl,
+                    self.environment.output_writer,
+                    &protocol_name,
+                    None,
+                    sandbox,
+                );
+                let test_file_path = app.path();
+                let app_name = test_file_path
+                    .file_stem()
+                    .and_then(|s| s.to_str())
+                    .unwrap_or_default();
path: e2e-tests/src/driver.rs
user: login: miraclx
body: `default` (which is an empty string) is not a good default here

`None` is only returned when there wasn't a file name, we can either ignore such cases or throw an error, but defaulting isn't it

https://doc.rust-lang.org/nightly/std/path/struct.PathBuf.html#method.file_stem:~:text=The%20stem%20is%3A-,None%2C%20if%20there%20is%20no%20file%20name%3B,-The%20entire%20file


created_at: 2025-05-13T19:39:37Z
author_association: CONTRIBUTOR
diff_hunk: @@ -97,131 +97,113 @@ impl Driver {
         let mut report = TestRunReport::new();
         let mut initialized_protocols: HashMap<String, ProtocolSandboxEnvironment> = HashMap::new();
 
-        // Run scenarios directory by directory
-        let scenarios_dir = self.environment.input_dir.join("scenarios");
-        let mut entries = read_dir(scenarios_dir).await?;
-
-        while let Some(entry) = entries.next_entry().await? {
-            let path = entry.path();
-            if path.is_dir() {
-                let test_file_path = path.join("test.json");
+        let protocols_dir = self.environment.input_dir.join("protocols");
+
+        for protocol_name in &self.environment.protocols {
+            let protocol_name = protocol_name.to_string().to_lowercase();
+
+            if initialized_protocols.contains_key(&protocol_name) {
+                continue;
+            }
+
+            for sandbox_cfg in &self.config.protocol_sandboxes {
+                let config_protocol_name = match sandbox_cfg {
+                    ProtocolSandboxConfig::Stellar(_) => "stellar",
+                    ProtocolSandboxConfig::Near(_) => "near",
+                    ProtocolSandboxConfig::Icp(_) => "icp",
+                    ProtocolSandboxConfig::Ethereum(_) => "ethereum",
+                };
+
+                if config_protocol_name == protocol_name {
+                    let sandbox_env = match sandbox_cfg {
+                        ProtocolSandboxConfig::Stellar(config) => {
+                            ProtocolSandboxEnvironment::Stellar(StellarSandboxEnvironment::init(
path: e2e-tests/src/driver.rs
user: login: cy4n1d3-p1x3l
body: there is a initialized_protocol list for that purpose
created_at: 2025-05-16T21:04:48Z
author_association: CONTRIBUTOR
diff_hunk: @@ -97,131 +97,113 @@ impl Driver {
         let mut report = TestRunReport::new();
         let mut initialized_protocols: HashMap<String, ProtocolSandboxEnvironment> = HashMap::new();
 
-        // Run scenarios directory by directory
-        let scenarios_dir = self.environment.input_dir.join("scenarios");
-        let mut entries = read_dir(scenarios_dir).await?;
-
-        while let Some(entry) = entries.next_entry().await? {
-            let path = entry.path();
-            if path.is_dir() {
-                let test_file_path = path.join("test.json");
+        let protocols_dir = self.environment.input_dir.join("protocols");
+
+        for protocol_name in &self.environment.protocols {
+            let protocol_name = protocol_name.to_string().to_lowercase();
+
+            if initialized_protocols.contains_key(&protocol_name) {
+                continue;
+            }
+
+            for sandbox_cfg in &self.config.protocol_sandboxes {
+                let config_protocol_name = match sandbox_cfg {
+                    ProtocolSandboxConfig::Stellar(_) => "stellar",
+                    ProtocolSandboxConfig::Near(_) => "near",
+                    ProtocolSandboxConfig::Icp(_) => "icp",
+                    ProtocolSandboxConfig::Ethereum(_) => "ethereum",
+                };
+
+                if config_protocol_name == protocol_name {
+                    let sandbox_env = match sandbox_cfg {
+                        ProtocolSandboxConfig::Stellar(config) => {
+                            ProtocolSandboxEnvironment::Stellar(StellarSandboxEnvironment::init(
path: e2e-tests/src/driver.rs
user: login: miraclx
body: ah that's a bit hard to read, but I see most of this logic was pretty much here before with some improvements, and that's fair

can you use the bail-early approach you did here:

```rust
            if initialized_protocols.contains_key(&protocol_name) {
                continue;
            }
```

to this too:

```rust
                if config_protocol_name != protocol_name {
                    continue;
                }
                
                let sandbox_env = match __;
```
created_at: 2025-05-17T00:46:18Z
author_association: CONTRIBUTOR
diff_hunk: @@ -515,54 +497,46 @@ impl TestRunReport {
 
     fn to_markdown(&self) -> EyreResult<String> {
         let mut markdown = String::new();
-
         writeln!(&mut markdown, "## E2E tests report")?;
 
-        for (scenario, protocols) in &self.scenario_matrix {
-            writeln!(&mut markdown, "### Scenario: {scenario}")?;
+        for (protocol, applications) in &self.scenario_matrix {
+            writeln!(&mut markdown, "## Protocol: {protocol}")?;
 
-            // Collecting all step names
-            let mut step_names = vec![];
-            for report in protocols.values() {
+            for (app_name, report) in applications {
+                let mut step_names = vec![];
                 for step in &report.steps {
                     if !step_names.contains(&step.step_name) {
                         step_names.push(step.step_name.clone());
                     }
                 }
-            }
-
-            // Write table header
-            write!(&mut markdown, "| Protocol/Step |")?;
-            for step_name in &step_names {
-                write!(&mut markdown, " {step_name} |")?;
-            }
-            writeln!(&mut markdown)?;
 
-            // Write table header separator
-            write!(&mut markdown, "| :--- |")?;
-            for _ in &step_names {
-                write!(&mut markdown, " :---: |")?;
-            }
-            writeln!(&mut markdown)?;
-
-            // Write table rows
-            for (protocol, report) in protocols {
-                write!(&mut markdown, "| {protocol} |")?;
+                // Write table header
+                write!(&mut markdown, "| Application/Step |")?;
+                for step_name in &step_names {
+                    write!(&mut markdown, " {step_name} |")?;
+                }
+                writeln!(&mut markdown)?;
+                // Write table header separator
+                write!(&mut markdown, "| :--- |")?;
+                for _ in &step_names {
+                    write!(&mut markdown, " :---: |")?;
+                }
+                writeln!(&mut markdown)?;
+                write!(&mut markdown, "| {app_name} |")?;
+                // Results row
                 for step_name in &step_names {
                     let result = report.steps.iter().find_map(|step| {
                         (&step.step_name == step_name).then_some(step.result.as_ref())
                     });
-
                     let result = match result {
                         None => "-",
                         Some(None) => ":fast_forward:",
                         Some(Some(Ok(_))) => ":white_check_mark:",
                         Some(Some(Err(_))) => ":x:",
                     };
-
                     write!(&mut markdown, " {result} |")?;
                 }
-                writeln!(&mut markdown)?;
+                writeln!(&mut markdown, "\n")?;
path: e2e-tests/src/driver.rs
user: login: cy4n1d3-p1x3l
body: yeah added purposefully
the md for different applications wont display correctly otherwise
created_at: 2025-05-18T06:35:45Z
author_association: CONTRIBUTOR
diff_hunk: @@ -97,131 +97,113 @@ impl Driver {
         let mut report = TestRunReport::new();
         let mut initialized_protocols: HashMap<String, ProtocolSandboxEnvironment> = HashMap::new();
 
-        // Run scenarios directory by directory
-        let scenarios_dir = self.environment.input_dir.join("scenarios");
-        let mut entries = read_dir(scenarios_dir).await?;
-
-        while let Some(entry) = entries.next_entry().await? {
-            let path = entry.path();
-            if path.is_dir() {
-                let test_file_path = path.join("test.json");
+        let protocols_dir = self.environment.input_dir.join("protocols");
+
+        for protocol_name in &self.environment.protocols {
+            let protocol_name = protocol_name.to_string().to_lowercase();
+
+            if initialized_protocols.contains_key(&protocol_name) {
+                continue;
+            }
+
+            for sandbox_cfg in &self.config.protocol_sandboxes {
+                let config_protocol_name = match sandbox_cfg {
+                    ProtocolSandboxConfig::Stellar(_) => "stellar",
+                    ProtocolSandboxConfig::Near(_) => "near",
+                    ProtocolSandboxConfig::Icp(_) => "icp",
+                    ProtocolSandboxConfig::Ethereum(_) => "ethereum",
+                };
+
+                if config_protocol_name == protocol_name {
+                    let sandbox_env = match sandbox_cfg {
+                        ProtocolSandboxConfig::Stellar(config) => {
+                            ProtocolSandboxEnvironment::Stellar(StellarSandboxEnvironment::init(
+                                config.clone(),
+                            )?)
+                        }
+                        ProtocolSandboxConfig::Near(config) => ProtocolSandboxEnvironment::Near(
+                            NearSandboxEnvironment::init(config.clone()).await?,
+                        ),
+                        ProtocolSandboxConfig::Icp(config) => ProtocolSandboxEnvironment::Icp(
+                            IcpSandboxEnvironment::init(config.clone())?,
+                        ),
+                        ProtocolSandboxConfig::Ethereum(config) => {
+                            ProtocolSandboxEnvironment::Ethereum(EthereumSandboxEnvironment::init(
+                                config.clone(),
+                            )?)
+                        }
+                    };
+
+                    initialized_protocols.insert(protocol_name.clone(), sandbox_env);
+                    break;
+                }
+            }
+        }
+
+        for protocol_name in &self.environment.protocols {
path: e2e-tests/src/driver.rs
user: login: cy4n1d3-p1x3l
body: there can be two ways to implement
1. Initialize the required sandboxes all first and then run the required tests on the apps.
2. Iterate over the protocols only once and initialize the sanbox and run required tests.

the 1st one is implemented as suggested by @fbozic and fits with the earlier implementation.

What is the better way? cc @miraclx @fbozic 
created_at: 2025-05-20T18:31:55Z
author_association: CONTRIBUTOR
diff_hunk: @@ -97,131 +97,113 @@ impl Driver {
         let mut report = TestRunReport::new();
         let mut initialized_protocols: HashMap<String, ProtocolSandboxEnvironment> = HashMap::new();
 
-        // Run scenarios directory by directory
-        let scenarios_dir = self.environment.input_dir.join("scenarios");
-        let mut entries = read_dir(scenarios_dir).await?;
-
-        while let Some(entry) = entries.next_entry().await? {
-            let path = entry.path();
-            if path.is_dir() {
-                let test_file_path = path.join("test.json");
+        let protocols_dir = self.environment.input_dir.join("protocols");
+
+        for protocol_name in &self.environment.protocols {
+            let protocol_name = protocol_name.to_string().to_lowercase();
+
+            if initialized_protocols.contains_key(&protocol_name) {
+                continue;
+            }
+
+            for sandbox_cfg in &self.config.protocol_sandboxes {
+                let config_protocol_name = match sandbox_cfg {
+                    ProtocolSandboxConfig::Stellar(_) => "stellar",
+                    ProtocolSandboxConfig::Near(_) => "near",
+                    ProtocolSandboxConfig::Icp(_) => "icp",
+                    ProtocolSandboxConfig::Ethereum(_) => "ethereum",
+                };
+
+                if config_protocol_name == protocol_name {
+                    let sandbox_env = match sandbox_cfg {
+                        ProtocolSandboxConfig::Stellar(config) => {
+                            ProtocolSandboxEnvironment::Stellar(StellarSandboxEnvironment::init(
+                                config.clone(),
+                            )?)
+                        }
+                        ProtocolSandboxConfig::Near(config) => ProtocolSandboxEnvironment::Near(
+                            NearSandboxEnvironment::init(config.clone()).await?,
+                        ),
+                        ProtocolSandboxConfig::Icp(config) => ProtocolSandboxEnvironment::Icp(
+                            IcpSandboxEnvironment::init(config.clone())?,
+                        ),
+                        ProtocolSandboxConfig::Ethereum(config) => {
+                            ProtocolSandboxEnvironment::Ethereum(EthereumSandboxEnvironment::init(
+                                config.clone(),
+                            )?)
+                        }
+                    };
+
+                    initialized_protocols.insert(protocol_name.clone(), sandbox_env);
+                    break;
+                }
+            }
+        }
+
+        for protocol_name in &self.environment.protocols {
+            let protocol_name = protocol_name.to_string().to_lowercase();
+            let protocol_path = protocols_dir.join(&protocol_name);
+
+            if !protocol_path.exists() || !protocol_path.is_dir() {
+                self.environment
+                    .output_writer
+                    .write_str(&format!("No directory for protocol: {}", protocol_name));
+                continue;
+            }
+
+            let Some(sandbox) = initialized_protocols.get(&protocol_name) else {
+                bail!("Sandbox not initialized for protocol: {}", protocol_name);
+            };
+
+            let mero = self.setup_mero(&vec![sandbox.clone()]).await?;
+
+            let Some((inviter, invitees)) = self.pick_inviter_node(&mero.ds) else {
+                bail!("Not enough nodes to run test for protocol {protocol_name}")
+            };
+
+            self.environment
+                .output_writer
+                .write_str(&format!("Picked inviter: {inviter}"));
+            self.environment
+                .output_writer
+                .write_str(&format!("Picked invitees: {invitees:?}"));
path: e2e-tests/src/driver.rs
user: login: cy4n1d3-p1x3l
body: if we tend to run all the apps on a single set of nodes then it would be printed only once per protocol.
created_at: 2025-05-20T18:35:37Z
author_association: CONTRIBUTOR
diff_hunk: @@ -97,131 +97,113 @@ impl Driver {
         let mut report = TestRunReport::new();
         let mut initialized_protocols: HashMap<String, ProtocolSandboxEnvironment> = HashMap::new();
 
-        // Run scenarios directory by directory
-        let scenarios_dir = self.environment.input_dir.join("scenarios");
-        let mut entries = read_dir(scenarios_dir).await?;
-
-        while let Some(entry) = entries.next_entry().await? {
-            let path = entry.path();
-            if path.is_dir() {
-                let test_file_path = path.join("test.json");
+        let protocols_dir = self.environment.input_dir.join("protocols");
+
+        for protocol_name in &self.environment.protocols {
+            let protocol_name = protocol_name.to_string().to_lowercase();
+
+            if initialized_protocols.contains_key(&protocol_name) {
+                continue;
+            }
+
+            for sandbox_cfg in &self.config.protocol_sandboxes {
+                let config_protocol_name = match sandbox_cfg {
+                    ProtocolSandboxConfig::Stellar(_) => "stellar",
+                    ProtocolSandboxConfig::Near(_) => "near",
+                    ProtocolSandboxConfig::Icp(_) => "icp",
+                    ProtocolSandboxConfig::Ethereum(_) => "ethereum",
+                };
+
+                if config_protocol_name == protocol_name {
+                    let sandbox_env = match sandbox_cfg {
+                        ProtocolSandboxConfig::Stellar(config) => {
+                            ProtocolSandboxEnvironment::Stellar(StellarSandboxEnvironment::init(
+                                config.clone(),
+                            )?)
+                        }
+                        ProtocolSandboxConfig::Near(config) => ProtocolSandboxEnvironment::Near(
+                            NearSandboxEnvironment::init(config.clone()).await?,
+                        ),
+                        ProtocolSandboxConfig::Icp(config) => ProtocolSandboxEnvironment::Icp(
+                            IcpSandboxEnvironment::init(config.clone())?,
+                        ),
+                        ProtocolSandboxConfig::Ethereum(config) => {
+                            ProtocolSandboxEnvironment::Ethereum(EthereumSandboxEnvironment::init(
+                                config.clone(),
+                            )?)
+                        }
+                    };
+
+                    initialized_protocols.insert(protocol_name.clone(), sandbox_env);
+                    break;
+                }
+            }
+        }
+
+        for protocol_name in &self.environment.protocols {
+            let protocol_name = protocol_name.to_string().to_lowercase();
+            let protocol_path = protocols_dir.join(&protocol_name);
+
+            if !protocol_path.exists() || !protocol_path.is_dir() {
+                self.environment
+                    .output_writer
+                    .write_str(&format!("No directory for protocol: {}", protocol_name));
+                continue;
+            }
+
+            let Some(sandbox) = initialized_protocols.get(&protocol_name) else {
+                bail!("Sandbox not initialized for protocol: {}", protocol_name);
+            };
+
+            let mero = self.setup_mero(&vec![sandbox.clone()]).await?;
+
+            let Some((inviter, invitees)) = self.pick_inviter_node(&mero.ds) else {
+                bail!("Not enough nodes to run test for protocol {protocol_name}")
+            };
+
+            self.environment
+                .output_writer
+                .write_str(&format!("Picked inviter: {inviter}"));
+            self.environment
+                .output_writer
+                .write_str(&format!("Picked invitees: {invitees:?}"));
+
+            let mut applications = read_dir(&protocol_path).await?;
+            while let Some(app) = applications.next_entry().await? {
+                let mut ctx = TestContext::new(
+                    inviter.clone(),
+                    invitees.clone(),
+                    &mero.ctl,
+                    self.environment.output_writer,
+                    &protocol_name,
+                    None,
+                    sandbox,
+                );
+                let test_file_path = app.path();
+                let app_name = test_file_path
+                    .file_stem()
+                    .and_then(|s| s.to_str())
+                    .unwrap_or_default();
path: e2e-tests/src/driver.rs
user: login: cy4n1d3-p1x3l
body: returning an error looks good.
created_at: 2025-05-20T18:36:03Z
author_association: CONTRIBUTOR
diff_hunk: @@ -41,6 +41,14 @@ pub struct TestContext<'a> {
     pub proposal_id: Option<String>,
 }
 
+#[derive(Debug, Hash, Clone, clap::ValueEnum, Copy, PartialEq, Eq)]
+pub enum Protocol {
+    Ethereum,
+    Near,
+    Stellar,
+    Icp,
+}
path: e2e-tests/src/driver.rs
user: login: miraclx
body: ~nit: I find it best to keep things that are exposed to the CLI all in one place, as it's easy to forget the `clap::ValueEnum` flag

so this should still be in `main.rs`, and if you have reasons to not have it there, it should at the very least be in `protocol.rs`
created_at: 2025-05-22T22:00:42Z
author_association: CONTRIBUTOR
diff_hunk: @@ -97,131 +97,113 @@ impl Driver {
         let mut report = TestRunReport::new();
         let mut initialized_protocols: HashMap<String, ProtocolSandboxEnvironment> = HashMap::new();
 
-        // Run scenarios directory by directory
-        let scenarios_dir = self.environment.input_dir.join("scenarios");
-        let mut entries = read_dir(scenarios_dir).await?;
-
-        while let Some(entry) = entries.next_entry().await? {
-            let path = entry.path();
-            if path.is_dir() {
-                let test_file_path = path.join("test.json");
+        let protocols_dir = self.environment.input_dir.join("protocols");
+
+        for protocol_name in &self.environment.protocols {
+            let protocol_name = protocol_name.to_string().to_lowercase();
+
+            if initialized_protocols.contains_key(&protocol_name) {
+                continue;
+            }
+
+            for sandbox_cfg in &self.config.protocol_sandboxes {
+                let config_protocol_name = match sandbox_cfg {
+                    ProtocolSandboxConfig::Stellar(_) => "stellar",
+                    ProtocolSandboxConfig::Near(_) => "near",
+                    ProtocolSandboxConfig::Icp(_) => "icp",
+                    ProtocolSandboxConfig::Ethereum(_) => "ethereum",
+                };
+
+                if config_protocol_name == protocol_name {
+                    let sandbox_env = match sandbox_cfg {
+                        ProtocolSandboxConfig::Stellar(config) => {
+                            ProtocolSandboxEnvironment::Stellar(StellarSandboxEnvironment::init(
+                                config.clone(),
+                            )?)
+                        }
+                        ProtocolSandboxConfig::Near(config) => ProtocolSandboxEnvironment::Near(
+                            NearSandboxEnvironment::init(config.clone()).await?,
+                        ),
+                        ProtocolSandboxConfig::Icp(config) => ProtocolSandboxEnvironment::Icp(
+                            IcpSandboxEnvironment::init(config.clone())?,
+                        ),
+                        ProtocolSandboxConfig::Ethereum(config) => {
+                            ProtocolSandboxEnvironment::Ethereum(EthereumSandboxEnvironment::init(
+                                config.clone(),
+                            )?)
+                        }
+                    };
+
+                    initialized_protocols.insert(protocol_name.clone(), sandbox_env);
+                    break;
+                }
+            }
+        }
+
+        for protocol_name in &self.environment.protocols {
path: e2e-tests/src/driver.rs
user: login: miraclx
body: it's alright as you have it, sandboxes will be small anyway, so it shouldn't matter much
created_at: 2025-05-22T22:03:17Z
author_association: CONTRIBUTOR
diff_hunk: @@ -92,26 +92,26 @@ pub struct RootArgs {
     #[arg(env = "E2E_OUTPUT_FORMAT", hide_env_values = true)]
     pub output_format: OutputFormat,
 
-    /// Scenarios to run
-    #[arg(long, value_name = "SCENARIO", value_enum, num_args = 1.., value_delimiter = ',')]
-    pub scenarios: Vec<Scenario>,
+    /// Protocols to run
+    #[arg(long, value_name = "PROTOCOL", value_enum, num_args = 1.., value_delimiter = ',')]
+    pub protocols: Vec<Protocol>,
 }
 
 #[derive(Debug, Clone, clap::ValueEnum, Copy)]
-pub enum Scenario {
+pub enum Protocol {
     Ethereum,
     Near,
     Stellar,
     Icp,
 }
 
-impl std::fmt::Display for Scenario {
+impl std::fmt::Display for Protocol {
     fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
         match self {
-            Scenario::Ethereum => write!(f, "ethereum"),
-            Scenario::Near => write!(f, "near"),
-            Scenario::Stellar => write!(f, "stellar"),
-            Scenario::Icp => write!(f, "icp"),
+            Protocol::Ethereum => write!(f, "ethereum"),
+            Protocol::Near => write!(f, "near"),
+            Protocol::Stellar => write!(f, "stellar"),
+            Protocol::Icp => write!(f, "icp"),
         }
     }
 }
path: e2e-tests/src/main.rs
user: login: miraclx
body: this is unresolved. drop `Protocol: fmt::Display` which returns a `String` for `Protocol::as_str()` which returns `&'static str`
created_at: 2025-05-22T22:08:58Z
author_association: CONTRIBUTOR
diff_hunk: @@ -95,56 +103,54 @@ impl Driver {
         self.environment.init().await?;
 
         let mut report = TestRunReport::new();
-        let mut initialized_protocols: HashMap<String, ProtocolSandboxEnvironment> = HashMap::new();
+        let mut initialized_protocols: HashMap<Protocol, ProtocolSandboxEnvironment> =
+            HashMap::new();
 
         let protocols_dir = self.environment.input_dir.join("protocols");
 
-        for protocol_name in &self.environment.protocols {
-            let protocol_name = protocol_name.to_string().to_lowercase();
-
-            if initialized_protocols.contains_key(&protocol_name) {
+        for protocol in &self.environment.protocols {
+            if initialized_protocols.contains_key(protocol) {
                 continue;
             }
 
             for sandbox_cfg in &self.config.protocol_sandboxes {
-                let config_protocol_name = match sandbox_cfg {
-                    ProtocolSandboxConfig::Stellar(_) => "stellar",
-                    ProtocolSandboxConfig::Near(_) => "near",
-                    ProtocolSandboxConfig::Icp(_) => "icp",
-                    ProtocolSandboxConfig::Ethereum(_) => "ethereum",
+                let config_protocol = match sandbox_cfg {
+                    ProtocolSandboxConfig::Stellar(_) => Protocol::Stellar,
+                    ProtocolSandboxConfig::Near(_) => Protocol::Near,
+                    ProtocolSandboxConfig::Icp(_) => Protocol::Icp,
+                    ProtocolSandboxConfig::Ethereum(_) => Protocol::Ethereum,
                 };
 
-                if config_protocol_name == protocol_name {
-                    let sandbox_env = match sandbox_cfg {
-                        ProtocolSandboxConfig::Stellar(config) => {
-                            ProtocolSandboxEnvironment::Stellar(StellarSandboxEnvironment::init(
-                                config.clone(),
-                            )?)
-                        }
-                        ProtocolSandboxConfig::Near(config) => ProtocolSandboxEnvironment::Near(
-                            NearSandboxEnvironment::init(config.clone()).await?,
-                        ),
-                        ProtocolSandboxConfig::Icp(config) => ProtocolSandboxEnvironment::Icp(
-                            IcpSandboxEnvironment::init(config.clone())?,
-                        ),
-                        ProtocolSandboxConfig::Ethereum(config) => {
-                            ProtocolSandboxEnvironment::Ethereum(EthereumSandboxEnvironment::init(
-                                config.clone(),
-                            )?)
-                        }
-                    };
-
-                    initialized_protocols.insert(protocol_name.clone(), sandbox_env);
-                    break;
+                if &config_protocol != protocol {
+                    continue;
                 }
+
+                let sandbox_env = match sandbox_cfg {
+                    ProtocolSandboxConfig::Stellar(config) => ProtocolSandboxEnvironment::Stellar(
+                        StellarSandboxEnvironment::init(config.clone())?,
+                    ),
+                    ProtocolSandboxConfig::Near(config) => ProtocolSandboxEnvironment::Near(
+                        NearSandboxEnvironment::init(config.clone()).await?,
+                    ),
+                    ProtocolSandboxConfig::Icp(config) => ProtocolSandboxEnvironment::Icp(
+                        IcpSandboxEnvironment::init(config.clone())?,
+                    ),
+                    ProtocolSandboxConfig::Ethereum(config) => {
+                        ProtocolSandboxEnvironment::Ethereum(EthereumSandboxEnvironment::init(
+                            config.clone(),
+                        )?)
+                    }
+                };
+
+                initialized_protocols.insert(*protocol, sandbox_env);
+                break;
             }
         }
 
         for protocol_name in &self.environment.protocols {
-            let protocol_name = protocol_name.to_string().to_lowercase();
-            let protocol_path = protocols_dir.join(&protocol_name);
+            let protocol_path = protocols_dir.join(&protocol_name.to_string().to_lowercase());
path: e2e-tests/src/driver.rs
user: login: miraclx
body: then this can become:

```suggestion
            let protocol_path = protocols_dir.join(protocol_name.as_str());
```
created_at: 2025-05-22T22:09:27Z
author_association: CONTRIBUTOR
diff_hunk: @@ -170,6 +176,10 @@ impl Driver {
 
             let mut applications = read_dir(&protocol_path).await?;
             while let Some(app) = applications.next_entry().await? {
+                let protocol_name = protocol_name.to_string().to_lowercase();
path: e2e-tests/src/driver.rs
user: login: miraclx
body: you shouldn't need to make a string here, patch `TestContext` to take `Protocol` instead of `&str`
created_at: 2025-05-22T22:10:21Z
author_association: CONTRIBUTOR
diff_hunk: @@ -97,131 +97,113 @@ impl Driver {
         let mut report = TestRunReport::new();
         let mut initialized_protocols: HashMap<String, ProtocolSandboxEnvironment> = HashMap::new();
 
-        // Run scenarios directory by directory
-        let scenarios_dir = self.environment.input_dir.join("scenarios");
-        let mut entries = read_dir(scenarios_dir).await?;
-
-        while let Some(entry) = entries.next_entry().await? {
-            let path = entry.path();
-            if path.is_dir() {
-                let test_file_path = path.join("test.json");
+        let protocols_dir = self.environment.input_dir.join("protocols");
+
+        for protocol_name in &self.environment.protocols {
+            let protocol_name = protocol_name.to_string().to_lowercase();
+
+            if initialized_protocols.contains_key(&protocol_name) {
+                continue;
+            }
+
+            for sandbox_cfg in &self.config.protocol_sandboxes {
+                let config_protocol_name = match sandbox_cfg {
+                    ProtocolSandboxConfig::Stellar(_) => "stellar",
+                    ProtocolSandboxConfig::Near(_) => "near",
+                    ProtocolSandboxConfig::Icp(_) => "icp",
+                    ProtocolSandboxConfig::Ethereum(_) => "ethereum",
+                };
+
+                if config_protocol_name == protocol_name {
+                    let sandbox_env = match sandbox_cfg {
+                        ProtocolSandboxConfig::Stellar(config) => {
+                            ProtocolSandboxEnvironment::Stellar(StellarSandboxEnvironment::init(
+                                config.clone(),
+                            )?)
+                        }
+                        ProtocolSandboxConfig::Near(config) => ProtocolSandboxEnvironment::Near(
+                            NearSandboxEnvironment::init(config.clone()).await?,
+                        ),
+                        ProtocolSandboxConfig::Icp(config) => ProtocolSandboxEnvironment::Icp(
+                            IcpSandboxEnvironment::init(config.clone())?,
+                        ),
+                        ProtocolSandboxConfig::Ethereum(config) => {
+                            ProtocolSandboxEnvironment::Ethereum(EthereumSandboxEnvironment::init(
+                                config.clone(),
+                            )?)
+                        }
+                    };
+
+                    initialized_protocols.insert(protocol_name.clone(), sandbox_env);
+                    break;
+                }
+            }
+        }
+
+        for protocol_name in &self.environment.protocols {
+            let protocol_name = protocol_name.to_string().to_lowercase();
+            let protocol_path = protocols_dir.join(&protocol_name);
+
+            if !protocol_path.exists() || !protocol_path.is_dir() {
+                self.environment
+                    .output_writer
+                    .write_str(&format!("No directory for protocol: {}", protocol_name));
+                continue;
+            }
+
+            let Some(sandbox) = initialized_protocols.get(&protocol_name) else {
+                bail!("Sandbox not initialized for protocol: {}", protocol_name);
+            };
+
+            let mero = self.setup_mero(&vec![sandbox.clone()]).await?;
path: e2e-tests/src/driver.rs
user: login: miraclx
body: this is unresolved
created_at: 2025-05-22T22:24:40Z
author_association: CONTRIBUTOR
diff_hunk: @@ -515,54 +511,46 @@ impl TestRunReport {
 
     fn to_markdown(&self) -> EyreResult<String> {
         let mut markdown = String::new();
-
         writeln!(&mut markdown, "## E2E tests report")?;
 
-        for (scenario, protocols) in &self.scenario_matrix {
-            writeln!(&mut markdown, "### Scenario: {scenario}")?;
+        for (protocol, applications) in &self.scenario_matrix {
+            writeln!(&mut markdown, "## Protocol: {protocol}")?;
path: e2e-tests/src/driver.rs
user: login: miraclx
body: ```suggestion
            writeln!(&mut markdown, "### Protocol: {protocol}")?;
```
created_at: 2025-05-22T22:26:20Z
author_association: CONTRIBUTOR
diff_hunk: @@ -95,133 +103,121 @@ impl Driver {
         self.environment.init().await?;
 
         let mut report = TestRunReport::new();
-        let mut initialized_protocols: HashMap<String, ProtocolSandboxEnvironment> = HashMap::new();
+        let mut initialized_protocols: HashMap<Protocol, ProtocolSandboxEnvironment> =
+            HashMap::new();
+
+        let protocols_dir = self.environment.input_dir.join("protocols");
+
+        for protocol in &self.environment.protocols {
+            if initialized_protocols.contains_key(protocol) {
+                continue;
+            }
+
+            for sandbox_cfg in &self.config.protocol_sandboxes {
+                let config_protocol = match sandbox_cfg {
+                    ProtocolSandboxConfig::Stellar(_) => Protocol::Stellar,
+                    ProtocolSandboxConfig::Near(_) => Protocol::Near,
+                    ProtocolSandboxConfig::Icp(_) => Protocol::Icp,
+                    ProtocolSandboxConfig::Ethereum(_) => Protocol::Ethereum,
+                };
+
+                if &config_protocol != protocol {
+                    continue;
+                }
+
+                let sandbox_env = match sandbox_cfg {
+                    ProtocolSandboxConfig::Stellar(config) => ProtocolSandboxEnvironment::Stellar(
+                        StellarSandboxEnvironment::init(config.clone())?,
+                    ),
+                    ProtocolSandboxConfig::Near(config) => ProtocolSandboxEnvironment::Near(
+                        NearSandboxEnvironment::init(config.clone()).await?,
+                    ),
+                    ProtocolSandboxConfig::Icp(config) => ProtocolSandboxEnvironment::Icp(
+                        IcpSandboxEnvironment::init(config.clone())?,
+                    ),
+                    ProtocolSandboxConfig::Ethereum(config) => {
+                        ProtocolSandboxEnvironment::Ethereum(EthereumSandboxEnvironment::init(
+                            config.clone(),
+                        )?)
+                    }
+                };
+
+                initialized_protocols.insert(*protocol, sandbox_env);
+                break;
+            }
+        }
+
+        for protocol_name in &self.environment.protocols {
+            let protocol_path = protocols_dir.join(&protocol_name.to_string().to_lowercase());
+
+            if !protocol_path.is_dir() {
+                self.environment
+                    .output_writer
+                    .write_str(&format!("No directory for protocol: {}", protocol_name));
+                continue;
+            }
+
+            let Some(sandbox) = initialized_protocols.get(&protocol_name) else {
+                bail!("Sandbox not initialized for protocol: {}", protocol_name);
+            };
+
+            let mero = self.setup_mero(&vec![sandbox.clone()]).await?;
+
+            let Some((inviter, invitees)) = self.pick_inviter_node(&mero.ds) else {
+                bail!("Not enough nodes to run test for protocol {protocol_name}")
+            };
+
+            self.environment
+                .output_writer
+                .write_str(&format!("Picked inviter: {inviter}"));
+            self.environment
+                .output_writer
+                .write_str(&format!("Picked invitees: {invitees:?}"));
 
-        // Run scenarios directory by directory
-        let scenarios_dir = self.environment.input_dir.join("scenarios");
-        let mut entries = read_dir(scenarios_dir).await?;
+            let mut applications = read_dir(&protocol_path).await?;
+            while let Some(app) = applications.next_entry().await? {
+                let protocol_name = protocol_name.to_string().to_lowercase();
+                if app.file_type().await?.is_dir() {
+                    continue;
+                }
+                let mut ctx = TestContext::new(
+                    inviter.clone(),
+                    invitees.clone(),
+                    &mero.ctl,
+                    self.environment.output_writer,
+                    &protocol_name,
+                    None,
+                    sandbox,
+                );
+                let test_file_path = app.path();
+                // let app_name = test_file_path
+                //     .file_stem()
+                //     .and_then(|s| s.to_str())
+                //     .unwrap_or_default();
path: e2e-tests/src/driver.rs
user: login: miraclx
body: ^?
created_at: 2025-05-22T22:29:33Z
author_association: CONTRIBUTOR
diff_hunk: @@ -97,131 +97,113 @@ impl Driver {
         let mut report = TestRunReport::new();
         let mut initialized_protocols: HashMap<String, ProtocolSandboxEnvironment> = HashMap::new();
 
-        // Run scenarios directory by directory
-        let scenarios_dir = self.environment.input_dir.join("scenarios");
-        let mut entries = read_dir(scenarios_dir).await?;
-
-        while let Some(entry) = entries.next_entry().await? {
-            let path = entry.path();
-            if path.is_dir() {
-                let test_file_path = path.join("test.json");
+        let protocols_dir = self.environment.input_dir.join("protocols");
+
+        for protocol_name in &self.environment.protocols {
+            let protocol_name = protocol_name.to_string().to_lowercase();
+
+            if initialized_protocols.contains_key(&protocol_name) {
+                continue;
+            }
+
+            for sandbox_cfg in &self.config.protocol_sandboxes {
+                let config_protocol_name = match sandbox_cfg {
+                    ProtocolSandboxConfig::Stellar(_) => "stellar",
+                    ProtocolSandboxConfig::Near(_) => "near",
+                    ProtocolSandboxConfig::Icp(_) => "icp",
+                    ProtocolSandboxConfig::Ethereum(_) => "ethereum",
+                };
+
+                if config_protocol_name == protocol_name {
+                    let sandbox_env = match sandbox_cfg {
+                        ProtocolSandboxConfig::Stellar(config) => {
+                            ProtocolSandboxEnvironment::Stellar(StellarSandboxEnvironment::init(
+                                config.clone(),
+                            )?)
+                        }
+                        ProtocolSandboxConfig::Near(config) => ProtocolSandboxEnvironment::Near(
+                            NearSandboxEnvironment::init(config.clone()).await?,
+                        ),
+                        ProtocolSandboxConfig::Icp(config) => ProtocolSandboxEnvironment::Icp(
+                            IcpSandboxEnvironment::init(config.clone())?,
+                        ),
+                        ProtocolSandboxConfig::Ethereum(config) => {
+                            ProtocolSandboxEnvironment::Ethereum(EthereumSandboxEnvironment::init(
+                                config.clone(),
+                            )?)
+                        }
+                    };
+
+                    initialized_protocols.insert(protocol_name.clone(), sandbox_env);
+                    break;
+                }
+            }
+        }
+
+        for protocol_name in &self.environment.protocols {
+            let protocol_name = protocol_name.to_string().to_lowercase();
+            let protocol_path = protocols_dir.join(&protocol_name);
+
+            if !protocol_path.exists() || !protocol_path.is_dir() {
+                self.environment
+                    .output_writer
+                    .write_str(&format!("No directory for protocol: {}", protocol_name));
+                continue;
+            }
+
+            let Some(sandbox) = initialized_protocols.get(&protocol_name) else {
+                bail!("Sandbox not initialized for protocol: {}", protocol_name);
+            };
+
+            let mero = self.setup_mero(&vec![sandbox.clone()]).await?;
+
+            let Some((inviter, invitees)) = self.pick_inviter_node(&mero.ds) else {
+                bail!("Not enough nodes to run test for protocol {protocol_name}")
+            };
+
+            self.environment
+                .output_writer
+                .write_str(&format!("Picked inviter: {inviter}"));
+            self.environment
+                .output_writer
+                .write_str(&format!("Picked invitees: {invitees:?}"));
path: e2e-tests/src/driver.rs
user: login: miraclx
body: but why are we changing the inviter / invitee sets on every protocol, but not on every application run?

and why at all, why not keep it the same throughout the run? imo there's no reason to change it
created_at: 2025-05-22T22:31:26Z
author_association: CONTRIBUTOR
diff_hunk: @@ -95,133 +103,121 @@ impl Driver {
         self.environment.init().await?;
 
         let mut report = TestRunReport::new();
-        let mut initialized_protocols: HashMap<String, ProtocolSandboxEnvironment> = HashMap::new();
+        let mut initialized_protocols: HashMap<Protocol, ProtocolSandboxEnvironment> =
+            HashMap::new();
+
+        let protocols_dir = self.environment.input_dir.join("protocols");
+
+        for protocol in &self.environment.protocols {
+            if initialized_protocols.contains_key(protocol) {
+                continue;
+            }
+
+            for sandbox_cfg in &self.config.protocol_sandboxes {
+                let config_protocol = match sandbox_cfg {
+                    ProtocolSandboxConfig::Stellar(_) => Protocol::Stellar,
+                    ProtocolSandboxConfig::Near(_) => Protocol::Near,
+                    ProtocolSandboxConfig::Icp(_) => Protocol::Icp,
+                    ProtocolSandboxConfig::Ethereum(_) => Protocol::Ethereum,
+                };
+
+                if &config_protocol != protocol {
+                    continue;
+                }
+
+                let sandbox_env = match sandbox_cfg {
+                    ProtocolSandboxConfig::Stellar(config) => ProtocolSandboxEnvironment::Stellar(
+                        StellarSandboxEnvironment::init(config.clone())?,
+                    ),
+                    ProtocolSandboxConfig::Near(config) => ProtocolSandboxEnvironment::Near(
+                        NearSandboxEnvironment::init(config.clone()).await?,
+                    ),
+                    ProtocolSandboxConfig::Icp(config) => ProtocolSandboxEnvironment::Icp(
+                        IcpSandboxEnvironment::init(config.clone())?,
+                    ),
+                    ProtocolSandboxConfig::Ethereum(config) => {
+                        ProtocolSandboxEnvironment::Ethereum(EthereumSandboxEnvironment::init(
+                            config.clone(),
+                        )?)
+                    }
+                };
+
+                initialized_protocols.insert(*protocol, sandbox_env);
+                break;
+            }
+        }
+
+        for protocol_name in &self.environment.protocols {
+            let protocol_path = protocols_dir.join(&protocol_name.to_string().to_lowercase());
+
+            if !protocol_path.is_dir() {
+                self.environment
+                    .output_writer
+                    .write_str(&format!("No directory for protocol: {}", protocol_name));
+                continue;
+            }
+
+            let Some(sandbox) = initialized_protocols.get(&protocol_name) else {
+                bail!("Sandbox not initialized for protocol: {}", protocol_name);
+            };
+
+            let mero = self.setup_mero(&vec![sandbox.clone()]).await?;
+
+            let Some((inviter, invitees)) = self.pick_inviter_node(&mero.ds) else {
+                bail!("Not enough nodes to run test for protocol {protocol_name}")
+            };
+
+            self.environment
+                .output_writer
+                .write_str(&format!("Picked inviter: {inviter}"));
+            self.environment
+                .output_writer
+                .write_str(&format!("Picked invitees: {invitees:?}"));
 
-        // Run scenarios directory by directory
-        let scenarios_dir = self.environment.input_dir.join("scenarios");
-        let mut entries = read_dir(scenarios_dir).await?;
+            let mut applications = read_dir(&protocol_path).await?;
+            while let Some(app) = applications.next_entry().await? {
+                let protocol_name = protocol_name.to_string().to_lowercase();
+                if app.file_type().await?.is_dir() {
+                    continue;
+                }
path: e2e-tests/src/driver.rs
user: login: miraclx
body: why not [`is_file`](https://doc.rust-lang.org/nightly/std/fs/struct.FileType.html#method.is_file)?
created_at: 2025-05-22T22:33:43Z
author_association: CONTRIBUTOR
diff_hunk: @@ -95,133 +103,121 @@ impl Driver {
         self.environment.init().await?;
 
         let mut report = TestRunReport::new();
-        let mut initialized_protocols: HashMap<String, ProtocolSandboxEnvironment> = HashMap::new();
+        let mut initialized_protocols: HashMap<Protocol, ProtocolSandboxEnvironment> =
+            HashMap::new();
+
+        let protocols_dir = self.environment.input_dir.join("protocols");
+
+        for protocol in &self.environment.protocols {
+            if initialized_protocols.contains_key(protocol) {
+                continue;
+            }
+
+            for sandbox_cfg in &self.config.protocol_sandboxes {
+                let config_protocol = match sandbox_cfg {
+                    ProtocolSandboxConfig::Stellar(_) => Protocol::Stellar,
+                    ProtocolSandboxConfig::Near(_) => Protocol::Near,
+                    ProtocolSandboxConfig::Icp(_) => Protocol::Icp,
+                    ProtocolSandboxConfig::Ethereum(_) => Protocol::Ethereum,
+                };
+
+                if &config_protocol != protocol {
+                    continue;
+                }
+
+                let sandbox_env = match sandbox_cfg {
+                    ProtocolSandboxConfig::Stellar(config) => ProtocolSandboxEnvironment::Stellar(
+                        StellarSandboxEnvironment::init(config.clone())?,
+                    ),
+                    ProtocolSandboxConfig::Near(config) => ProtocolSandboxEnvironment::Near(
+                        NearSandboxEnvironment::init(config.clone()).await?,
+                    ),
+                    ProtocolSandboxConfig::Icp(config) => ProtocolSandboxEnvironment::Icp(
+                        IcpSandboxEnvironment::init(config.clone())?,
+                    ),
+                    ProtocolSandboxConfig::Ethereum(config) => {
+                        ProtocolSandboxEnvironment::Ethereum(EthereumSandboxEnvironment::init(
+                            config.clone(),
+                        )?)
+                    }
+                };
+
+                initialized_protocols.insert(*protocol, sandbox_env);
+                break;
+            }
+        }
+
+        for protocol_name in &self.environment.protocols {
+            let protocol_path = protocols_dir.join(&protocol_name.to_string().to_lowercase());
+
+            if !protocol_path.is_dir() {
+                self.environment
+                    .output_writer
+                    .write_str(&format!("No directory for protocol: {}", protocol_name));
+                continue;
+            }
+
+            let Some(sandbox) = initialized_protocols.get(&protocol_name) else {
+                bail!("Sandbox not initialized for protocol: {}", protocol_name);
+            };
+
+            let mero = self.setup_mero(&vec![sandbox.clone()]).await?;
+
+            let Some((inviter, invitees)) = self.pick_inviter_node(&mero.ds) else {
+                bail!("Not enough nodes to run test for protocol {protocol_name}")
+            };
+
+            self.environment
+                .output_writer
+                .write_str(&format!("Picked inviter: {inviter}"));
+            self.environment
+                .output_writer
+                .write_str(&format!("Picked invitees: {invitees:?}"));
 
-        // Run scenarios directory by directory
-        let scenarios_dir = self.environment.input_dir.join("scenarios");
-        let mut entries = read_dir(scenarios_dir).await?;
+            let mut applications = read_dir(&protocol_path).await?;
+            while let Some(app) = applications.next_entry().await? {
+                let protocol_name = protocol_name.to_string().to_lowercase();
+                if app.file_type().await?.is_dir() {
+                    continue;
+                }
+                let mut ctx = TestContext::new(
+                    inviter.clone(),
+                    invitees.clone(),
+                    &mero.ctl,
+                    self.environment.output_writer,
+                    &protocol_name,
+                    None,
+                    sandbox,
+                );
+                let test_file_path = app.path();
+                // let app_name = test_file_path
+                //     .file_stem()
+                //     .and_then(|s| s.to_str())
+                //     .unwrap_or_default();
+
+                let Some(app_name) = test_file_path.file_stem().and_then(|s| s.to_str()) else {
+                    bail!("No application name found");
+                };
 
-        while let Some(entry) = entries.next_entry().await? {
-            let path = entry.path();
-            if path.is_dir() {
-                let test_file_path = path.join("test.json");
                 if test_file_path.exists() {
                     let test_content = read(&test_file_path).await?;
path: e2e-tests/src/driver.rs
user: login: miraclx
body: `is_file`?

also, write it as

```rust
if !test_file_path.is_file() {
    continue;
}

let test_content = __;
```
created_at: 2025-05-22T22:34:46Z
author_association: CONTRIBUTOR
diff_hunk: @@ -349,7 +337,7 @@ impl Driver {
         drop(
             report
                 .scenario_matrix
-                .entry(ctx.protocol_name.to_owned())
+                .entry((*ctx.protocol_name.as_str()).to_string())
path: e2e-tests/src/driver.rs
user: login: miraclx
body: why the deref?

```suggestion
                .entry(ctx.protocol_name.as_str().to_string())
```
created_at: 2025-05-27T04:04:10Z
author_association: CONTRIBUTOR

PR comments:
user: login: cy4n1d3-p1x3l
created_at: 2025-04-26T18:58:38Z
body: Workflows not updated yet. Waiting for a review @fbozic @chefsale @miraclx.
author_association: CONTRIBUTOR
user: login: cy4n1d3-p1x3l
created_at: 2025-05-23T18:15:20Z
body: > but why are we changing the inviter / invitee sets on every protocol, but not on every application run?
> 
> and why at all, why not keep it the same throughout the run? imo there's no reason to change it

not sure about this
rest all changes are done
author_association: CONTRIBUTOR
user: login: cy4n1d3-p1x3l
created_at: 2025-05-28T14:32:31Z
body: > So we discussed the issues mentioned above and they seem to be the same on `master`, which, while incorrect is out of scope for this PR.
> 
> Thanks for driving this to completion. 🎉

Thanks for the approval.
Though i'm curious to know about the solution to those comments.
author_association: CONTRIBUTOR
user: login: miraclx
created_at: 2025-05-28T14:41:34Z
body: > Though i'm curious to know about the solution to those comments.

In this form:

1. Identify all protocols for all scenarios ahead of time
2. Spin up `merod` with all protocols configured
3. Pick inviter & invitees
4. Run all scenarios using the preconfigured nodes
author_association: CONTRIBUTOR
user: login: github-actions[bot]
created_at: 2025-05-28T14:51:08Z
body: ## E2E tests report
### Protocol: ethereum
| Application/Step | 0. app install (AllMembers) | 1. ctx create | 2. ctx invite-join | 3. wait (Consensus) | 4. call (create_new_proposal, Inviter) | 5. get proposals | 6. call (send_proposal_messages, Inviter) | 7. wait (Broadcast) | 8. call (get_proposal_messages, Invitees) | 9. call (approve_proposal, Invitees) | 10. verify external state |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| demo-blockchain-integrations | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: |

| Application/Step | 0. app install (AllMembers) | 1. ctx create | 2. alias create | 3. call (set, Inviter) | 4. call (get, Inviter) | 5. ctx invite-join | 6. wait (Consensus) | 7. call (get, AllMembers) | 8. call (set, Inviter) | 9. wait (Broadcast) | 10. call (get, AllMembers) |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| kv-store-test | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: |


### Protocol: icp
| Application/Step | 0. app install (AllMembers) | 1. ctx create | 2. ctx invite-join | 3. wait (Consensus) | 4. call (create_new_proposal, Inviter) | 5. get proposals | 6. call (send_proposal_messages, Inviter) | 7. wait (Broadcast) | 8. call (get_proposal_messages, Invitees) | 9. call (approve_proposal, Invitees) | 10. verify external state |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| demo-blockchain-integrations | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: |

| Application/Step | 0. app install (AllMembers) | 1. ctx create | 2. alias create | 3. call (set, Inviter) | 4. call (get, Inviter) | 5. ctx invite-join | 6. wait (Consensus) | 7. call (get, AllMembers) | 8. call (set, Inviter) | 9. wait (Broadcast) | 10. call (get, AllMembers) |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| kv-store-test | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: |


### Protocol: near
| Application/Step | 0. app install (AllMembers) | 1. ctx create | 2. ctx invite-join | 3. wait (Consensus) | 4. call (create_new_proposal, Inviter) | 5. get proposals | 6. call (send_proposal_messages, Inviter) | 7. wait (Broadcast) | 8. call (get_proposal_messages, Invitees) | 9. call (approve_proposal, Invitees) | 10. verify external state |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| demo-blockchain-integrations | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: |

| Application/Step | 0. app install (AllMembers) | 1. ctx create | 2. alias create | 3. call (set, Inviter) | 4. call (get, Inviter) | 5. ctx invite-join | 6. wait (Consensus) | 7. call (get, AllMembers) | 8. call (set, Inviter) | 9. wait (Broadcast) | 10. call (get, AllMembers) |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| kv-store-test | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: |


### Protocol: stellar
| Application/Step | 0. app install (AllMembers) | 1. ctx create | 2. ctx invite-join | 3. wait (Consensus) | 4. call (create_new_proposal, Inviter) | 5. get proposals | 6. call (approve_proposal, Invitees) | 7. verify external state | 8. call (create_new_proposal, Inviter) | 9. get proposals | 10. call (send_proposal_messages, Inviter) | 11. wait (Broadcast) | 12. call (get_proposal_messages, Invitees) |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| demo-blockchain-integrations | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: |

| Application/Step | 0. app install (AllMembers) | 1. ctx create | 2. alias create | 3. call (set, Inviter) | 4. call (get, Inviter) | 5. ctx invite-join | 6. wait (Consensus) | 7. call (get, AllMembers) | 8. call (set, Inviter) | 9. wait (Broadcast) | 10. call (get, AllMembers) |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| kv-store-test | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: |
<!-- thollander/actions-comment-pull-request "e2e-tests-report" -->
author_association: NONE
user: login: cy4n1d3-p1x3l
created_at: 2025-05-28T15:27:02Z
body: > > Though i'm curious to know about the solution to those comments.
> 
> In this form:
> 
>     1. Identify all protocols for all scenarios ahead of time
> 
>     2. Spin up `merod` with all protocols configured
> 
>     3. Pick inviter & invitees
> 
>     4. Run all scenarios using the preconfigured nodes

Ahh i see

author_association: CONTRIBUTOR