Repository: celestiaorg/celestia-app

Issue:
v4.0.2-mocha error about path to genesis
### Summary of Bug

After upgrading from `3.8.1-mocha` to `v4.0.2-mocha `I've had an error about genesis path
```
ERR failed to start api server err="couldn't read GenesisDoc file: open /home/celestia/.celestia-app/config/genesis.json: no such file or directory"
Error: couldn't read GenesisDoc file: open /home/celestia/.celestia-app/config/genesis.json: no such file or directory
2025/06/05 05:53:42 Process finished with error: exit status 1
```
Our home folder for celestia-appd not default. Command for start celestia-appd` celestia-appd --home <$path_to_celestia_folder> start`

Looks like the `v4.0.2-mocha` version of celestia-appd ignore genesis path in config.toml.


### Version

v4.0.2

### Steps to Reproduce

Start celestia-appd using a different path than the default one to home folder.
`celestia-appd --home <$path_to_celestia_folder> start`

Discussion:
@mindstyle85: we have observed one other having this issue and somehow it was resolved when he replaced the genesis file@rootulp: I couldn't reproduce with the single-node.sh script. 

```
$ mkdir ~/foo
$ ./scripts/single-node.sh ~/foo
celestia-app version: 4.0.2-arabica-9-g984786703
celestia-app home: /Users/rootulp/foo
celestia-app genesis file: /Users/rootulp/foo/config/genesis.json

Do you want to delete existing /Users/rootulp/foo and start a new local testnet? [y/n]
y
Deleting /Users/rootulp/foo...
Initializing validator and node config files...
Adding a new key to the keyring...
Adding genesis account...
Creating a genesis tx...
Collecting genesis txs...
Tracing is set up with the ability to pull traced data from the node on the address http://127.0.0.1:26661
Starting celestia-app...
11:39AM INF P2P Node ID ID=f14ee398f82411f8650bf37701f7d466a755d5ee file=/Users/rootulp/foo/config/node_key.json module=p2p
11:39AM INF Adding persistent peers addrs=[] module=p2p
11:39AM INF Adding unconditional peer ids ids=[] module=p2p
11:39AM INF Add our address to book addr=f14ee398f82411f8650bf37701f7d466a755d5ee@0.0.0.0:26656 book=/Users/rootulp/foo/config/addrbook.json module=p2p
11:39AM INF service start impl="P2P Switch" module=p2p msg="Starting P2P Switch service"
11:39AM INF service start book=/Users/rootulp/foo/config/addrbook.json impl=AddrBook module=p2p msg="Starting AddrBook service"
11:39AM INF Saving AddrBook to file book=/Users/rootulp/foo/config/addrbook.json module=p2p size=0
11:39AM INF finalized block block_app_hash=1584AC60131C24648EE758320B1E9FE8010A1A2B25C51408212C1A0A60A4DF35 height=1 module=state num_txs_res=0 num_val_updates=0
11:39AM INF executed block app_hash=1584AC60131C24648EE758320B1E9FE8010A1A2B25C51408212C1A0A60A4DF35 height=1 module=state
11:39AM INF committed state block_app_hash=E3B0C44298FC1C149AFBF4C8996FB92427AE41E4649B934CA495991B7852B855 height=1 module=state
11:39AM INF finalized block block_app_hash=C4A30A5511342D04F69722F349F2AADE3796CF7CAAE0DBD7DC0B4459C02797B6 height=2 module=state num_txs_res=0 num_val_updates=0
11:39AM INF executed block app_hash=C4A30A5511342D04F69722F349F2AADE3796CF7CAAE0DBD7DC0B4459C02797B6 height=2 module=state
11:39AM INF committed state block_app_hash=1584AC60131C24648EE758320B1E9FE8010A1A2B25C51408212C1A0A60A4DF35 height=2 module=state
```

Can you please help us understand how to reproduce the issue?
1. What value are you using for `--home`?
2.  Can you please copy + paste the exact command you use to start the node?
3. Can you please run `ls /home/celestia/.celestia-app/config/`?
4. Can you please run `ls <$path_to_celestia_folder>`?@antony-everstake: We started celestia-app using symlink.
`ln -s <$path_to_celestia_folder>  /home/celestia`@antony-everstake: @rootulp you used to `4.0.2-arabica-9-g984786703` but I wrote about `4.0.2-mocha ` version.@rootulp: > We started celestia-app using symlink.
ln -s <$path_to_celestia_folder>  /home/celestia

Can you please elaborate? Does it work if you symlink or it still doesn't work? 

Update: it works after symlink.

>@rootulp you used to 4.0.2-arabica-9-g984786703 but I wrote about 4.0.2-mocha  version.

Good point. They're functionally equivalent. `v4.0.2-mocha` also works with a custom home folder.

```
$ ./scripts/single-node.sh ~/foo
celestia-app version: 4.0.2-mocha
celestia-app home: /Users/rootulp/foo
celestia-app genesis file: /Users/rootulp/foo/config/genesis.json

Do you want to delete existing /Users/rootulp/foo and start a new local testnet? [y/n]
y
Deleting /Users/rootulp/foo...
Initializing validator and node config files...
Adding a new key to the keyring...
Adding genesis account...
Creating a genesis tx...
Collecting genesis txs...
Tracing is set up with the ability to pull traced data from the node on the address http://127.0.0.1:26661
Starting celestia-app...
11:52AM INF P2P Node ID ID=b00590a4b7e111cfafe4b939ad9cdf5e7be17cd3 file=/Users/rootulp/foo/config/node_key.json module=p2p
11:52AM INF Adding persistent peers addrs=[] module=p2p
11:52AM INF Adding unconditional peer ids ids=[] module=p2p
11:52AM INF Add our address to book addr=b00590a4b7e111cfafe4b939ad9cdf5e7be17cd3@0.0.0.0:26656 book=/Users/rootulp/foo/config/addrbook.json module=p2p
11:52AM INF service start impl="P2P Switch" module=p2p msg="Starting P2P Switch service"
11:52AM INF service start book=/Users/rootulp/foo/config/addrbook.json impl=AddrBook module=p2p msg="Starting AddrBook service"
11:52AM INF Saving AddrBook to file book=/Users/rootulp/foo/config/addrbook.json module=p2p size=0
11:52AM INF finalized block block_app_hash=E962DD3046AFCF64FDAD6306A9EBD18C4D178E062018B09319F84F21DB3865F8 height=1 module=state num_txs_res=0 num_val_updates=0
11:52AM INF executed block app_hash=E962DD3046AFCF64FDAD6306A9EBD18C4D178E062018B09319F84F21DB3865F8 height=1 module=state
11:52AM INF committed state block_app_hash=E3B0C44298FC1C149AFBF4C8996FB92427AE41E4649B934CA495991B7852B855 height=1 module=state
11:52AM INF finalized block block_app_hash=00615266AFA64A462DC38168FA1D5BA7568D70E044DE29877A80BAACAC13C49C height=2 module=state num_txs_res=0 num_val_updates=0
11:52AM INF executed block app_hash=00615266AFA64A462DC38168FA1D5BA7568D70E044DE29877A80BAACAC13C49C height=2 module=state
11:52AM INF committed state block_app_hash=E962DD3046AFCF64FDAD6306A9EBD18C4D178E062018B09319F84F21DB3865F8 height=2 module=state
```

@rootulp: Spent an hour on a call with Anton and we couldn't reproduce on my machine but could reproduce on his. Some things unique to his setup:
- They're on Linux
- They use `--home /mnt`
- They used an older config file, not a newly generated one

I'll continue trying to reproduce and will talk with him tomorrow if I can't reproduce.@rootulp: This only reproduces on Anton's machine with the ordering:

```
celestia-appd --home ~/foo start
```

It does not reproduce with the ordering:

```
celestia-appd start --home ~/foo 
```@rootulp: I'm on Ubuntu 6.11.0-9-generic
Anton is on Ubuntu 6.8.0-45-generic

Mine
```
lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 24.10
Release:	24.10
Codename:	oracular
```

Anton's
```
No LSB modules are available.
Distributor ID:    Ubuntu
Description:    Ubuntu 24.04.1 LTS
Release:    24.04
Codename:    noble
```
@rootulp: Neither of us are using `echo $CELESTIA_APP_HOME`@rootulp: ## v3.x

We used a Viper flag to override the home directory https://github.com/celestiaorg/celestia-app/blob/0a7154538e6b47ca806e9f4b3fd9ff1581213099/cmd/celestia-appd/cmd/start.go#L161


## v4.x

We use this Cosmos SDK helper: https://github.com/celestiaorg/celestia-app/blob/984786703466aca8d6004ac8f871b1ad18791888/app/init.go#L30

```go
	// get the home directory from the flag
	args := os.Args
	for i := 0; i < len(args); i++ {
		if args[i] == "--home" && i+1 < len(args) {
			return filepath.Clean(args[i+1]), nil
		} else if strings.HasPrefix(args[i], "--home=") {
			return filepath.Clean(args[i][7:]), nil
		}
	}
```@rootulp: Next step to try and reproduce:
- [x] Create an Ubuntu 24.04.1 LTS machine 
- [x] Install Go go version go1.24.1 linux/amd64
- [x] Install celestia-app v4.0.2-mocha

I reproduced the error!

```
root@rootulp-scaleway:~/celestia-app# ~/go/bin/celestia-appd --home /foo init mocha-4
root@rootulp-scaleway:~/celestia-app# ~/go/bin/celestia-appd --home /foo download-genesis mocha-4
root@rootulp-scaleway:~/celestia-app# ~/go/bin/celestia-appd --home /foo start
6:19PM INF initializing multiplexer app_version=1 chain_id=mocha-4 module=server
6:19PM INF initialized remote app client address=127.0.0.1:36658 module=server
6:19PM INF starting comet node module=server
6:19PM INF No default value exists for the v2 upgrade height when the chainID is
6:19PM INF starting ABCI without Tendermint
6:19PM ERR failed to start api server err="couldn't read GenesisDoc file: open /root/.celestia-app/config/genesis.json: no such file or directory"
Error: couldn't read GenesisDoc file: open /root/.celestia-app/config/genesis.json: no such file or directory
2025/06/05 18:19:26 Process finished with error: exit status 1
```@rootulp: Found the problem. Another bug in multiplexer.

```
--> Installing celestia-appd with multiplexer support
root@rootulp-scaleway:~/celestia-app# ~/go/bin/celestia-appd --home /foo start
args: [/root/go/bin/celestia-appd --home /foo start]
Found home flag: /foo
NodeHome: /foo
6:29PM INF initializing multiplexer app_version=1 chain_id=mocha-4 module=server
programArgs: [/root/go/bin/celestia-appd --home /foo start]
after stripping programArgs: [/foo start]
```@mindstyle85: wow nice catch!@rootulp: Thanks @antony-everstake for pairing and helping me reproduce the error.@antony-everstake: Glad I could help. Thanks @rootulp for the good communication and quick response.

Solution patch:
diff --git a/docs/release-notes/release-notes.md b/docs/release-notes/release-notes.md
index 08cf642d..b3144298 100644
--- a/docs/release-notes/release-notes.md
+++ b/docs/release-notes/release-notes.md
@@ -10,6 +10,8 @@ This guide provides notes for major version releases. These notes may be helpful
 
 Celestia-app v4.0.0 introduces support for a [multiplexer](https://github.com/celestiaorg/celestia-app/tree/e5d5ac6732c55150ea3573e17bec162fe836e0c6/multiplexer) that makes it easier for node operators to run a consensus node that can sync from genesis. The multiplexer contains an embedded celestia-app v3.x.x binary that will be used to sync the node from genesis. After the chain advances to app version v4, the multiplexer will stop routing requests to the embedded celestia-app v3.x.x binary and will instead route requests to the the v4.x.x state machine. Binaries that are installed from source (via `make install`) will include support for the multiplexer. To install Celestia without the multiplexer, you can use the `make install-standalone` target. Note that the standalone binary will only be able to run on networks that have already upgraded to app version v4.
 
+`make install` currently downloads a v3.x binary with goleveldb support. If you use pebbledb, you will need to build the v3.x binary from source (with the `pebbledb` build tag) and include it in the app's embedded binary directory (by default: `~/.celestia-app/bin/`).
+
 #### `rpc.grpc_laddr`
 
 The `rpc.grpc_laddr` config option is now required when running the celestia-app binary with the multiplexer. This option can be set via CLI flag `--rpc.grpc_laddr tcp://127.0.0.1:9098` or in the `config.toml`:
@@ -29,14 +31,17 @@ Celestia-app v4.0.0 includes significant state machine changes due to major depe
 #### New Messages (Added Modules)
 
 **`x/circuit` Circuit Breaker Module** ([cosmos-sdk docs](https://docs.cosmos.network/v0.50/build/modules/circuit)):
+
 - `MsgAuthorizeCircuitBreaker` - Grant circuit breaker permissions
 - `MsgTripCircuitBreaker` - Disable message execution
 - `MsgResetCircuitBreaker` - Re-enable message execution
 
 **`x/consensus` Consensus Parameters Module** ([cosmos-sdk docs](https://docs.cosmos.network/v0.50/build/modules/consensus)):
+
 - `MsgUpdateParams` - Update consensus parameters via governance (replaces CometBFT consensus param updates)
 
 **Hyperlane**:
+
 - `hyperlane/core` - Cross-chain messaging infrastructure
 - `hyperlane/warp` - Token bridging and routing
 
@@ -45,14 +50,17 @@ Celestia-app v4.0.0 includes significant state machine changes due to major depe
 **`x/crisis` Module** - Removed in Cosmos SDK v0.50.x
 
 **`x/capability` Module** - Removed in Cosmos SDK v0.50.x:
+
 - IBC capability management integrated directly into IBC v8 modules
 
 **`x/paramfilter` Module** - Celestia-specific module removed:
+
 - Parameter filtering functionality replaced by circuit breaker
 
 #### Changed Messages and Logic
 
 **Parameter Management Migration**:
+
 - **Generic parameter proposals** deprecated in favor of module-specific `MsgUpdateParams` messages
 - **Consensus parameters** moved from CometBFT to dedicated `x/consensus` module
 - **All modules** now use module-specific parameter update messages instead of legacy `x/params` proposals
@@ -62,11 +70,13 @@ Celestia-app v4.0.0 includes significant state machine changes due to major depe
 ### Library Consumers (v4.0.0)
 
 **Import Changes**:
+
 - Add: `cosmossdk.io/x/circuit`, `cosmossdk.io/x/consensus`
 - Remove: `x/capability`, `x/crisis`, `x/paramfilter`
 - Update: `github.com/cosmos/ibc-go/v8` (from v6)
 
 **API Breaking Changes** ([cosmos-sdk migration guide](https://docs.cosmos.network/v0.50/build/migrations/upgrading)):
+
 - Module keepers now accept `context.Context` instead of `sdk.Context`
 - `BeginBlock`/`EndBlock` signatures changed
 - Parameter updates require module-specific `MsgUpdateParams` messages
diff --git a/go.mod b/go.mod
index f12994b3..46aae179 100644
--- a/go.mod
+++ b/go.mod
@@ -16,7 +16,7 @@ require (
 	cosmossdk.io/x/feegrant v0.1.1
 	cosmossdk.io/x/tx v0.13.8
 	cosmossdk.io/x/upgrade v0.1.4
-	github.com/bcp-innovations/hyperlane-cosmos v1.0.0
+	github.com/bcp-innovations/hyperlane-cosmos v1.0.1
 	github.com/celestiaorg/go-square/v2 v2.2.0
 	github.com/celestiaorg/knuu v0.16.3
 	github.com/celestiaorg/nmt v0.24.0
diff --git a/go.sum b/go.sum
index 2aac6d11..2d8f067b 100644
--- a/go.sum
+++ b/go.sum
@@ -322,8 +322,8 @@ github.com/aws/aws-sdk-go-v2/service/sts v1.33.14 h1:TzeR06UCMUq+KA3bDkujxK1GVGy
 github.com/aws/aws-sdk-go-v2/service/sts v1.33.14/go.mod h1:dspXf/oYWGWo6DEvj98wpaTeqt5+DMidZD0A9BYTizc=
 github.com/aws/smithy-go v1.22.2 h1:6D9hW43xKFrRx/tXXfAlIZc4JI+yQe6snnWcQyxSyLQ=
 github.com/aws/smithy-go v1.22.2/go.mod h1:irrKGvNn1InZwb2d7fkIRNucdfwR8R+Ts3wxYa/cJHg=
-github.com/bcp-innovations/hyperlane-cosmos v1.0.0 h1:h1AP5gTCCWABlIPxy5pXXuSvMmDgCie1U/iRxNa+TT0=
-github.com/bcp-innovations/hyperlane-cosmos v1.0.0/go.mod h1:3yfa0io5Ii6GmhHWsWl2LEAOEHsqWuMgw2R02+LPogw=
+github.com/bcp-innovations/hyperlane-cosmos v1.0.1 h1:gT8OqyJ866Q6AHOlIXKxSdLjd0p8crKG9XXERIWoh4c=
+github.com/bcp-innovations/hyperlane-cosmos v1.0.1/go.mod h1:3yfa0io5Ii6GmhHWsWl2LEAOEHsqWuMgw2R02+LPogw=
 github.com/benbjohnson/clock v1.1.0/go.mod h1:J11/hYXuz8f4ySSvYwY0FKfm+ezbsZBKZxNJlLklBHA=
 github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q=
 github.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+CedLV8=
diff --git a/multiplexer/abci/multiplexer.go b/multiplexer/abci/multiplexer.go
index 70d7329d..e7a93220 100644
--- a/multiplexer/abci/multiplexer.go
+++ b/multiplexer/abci/multiplexer.go
@@ -226,10 +226,7 @@ func (m *Multiplexer) startApp() error {
 	}
 
 	if currentVersion.Appd.Pid() == appd.AppdStopped {
-		programArgs := os.Args
-		if len(os.Args) > 2 {
-			programArgs = os.Args[2:] // remove 'appd start' args
-		}
+		programArgs := removeStart(os.Args)
 
 		// start an embedded app.
 		m.logger.Debug("starting embedded app", "app_version", currentVersion.AppVersion, "args", currentVersion.GetStartArgs(programArgs))
@@ -248,6 +245,21 @@ func (m *Multiplexer) startApp() error {
 	return m.initRemoteGrpcConn()
 }
 
+// removeStart removes the first argument (the binary name) and the start argument from args.
+func removeStart(args []string) []string {
+	if len(args) == 0 {
+		return args
+	}
+	result := []string{}
+	args = args[1:] // remove the first argument (the binary name)
+	for _, arg := range args {
+		if arg != "start" {
+			result = append(result, arg)
+		}
+	}
+	return result
+}
+
 // initRemoteGrpcConn initializes a gRPC connection to the remote application client and configures transport credentials.
 func (m *Multiplexer) initRemoteGrpcConn() error {
 	// prepare remote app client



Test patch:
diff --git a/multiplexer/abci/multiplexer_test.go b/multiplexer/abci/multiplexer_test.go
index 3e788244..f4e97d03 100644
--- a/multiplexer/abci/multiplexer_test.go
+++ b/multiplexer/abci/multiplexer_test.go
@@ -12,3 +12,43 @@ func TestOpenTraceWriter(t *testing.T) {
 		require.NoError(t, err)
 	})
 }
+
+func Test_getProgramArgs(t *testing.T) {
+	type testCase struct {
+		name  string
+		input []string
+		want  []string
+	}
+	tests := []testCase{
+		{
+			name:  "should return empty slice if no args are provided",
+			input: []string{},
+			want:  []string{},
+		},
+		{
+			name:  "should return empty slice if just celestia-appd is provided",
+			input: []string{"celestia-appd"},
+			want:  []string{},
+		},
+		{
+			name:  "should remove celestia-appd and start from input",
+			input: []string{"celestia-appd", "start", "--home", "foo"},
+			want:  []string{"--home", "foo"},
+		},
+		{
+			name:  "should preserve extra additional args",
+			input: []string{"celestia-appd", "start", "--home", "foo", "--grpc.enable", "--api.enable"},
+			want:  []string{"--home", "foo", "--grpc.enable", "--api.enable"},
+		},
+		{
+			// Reproduces https://github.com/celestiaorg/celestia-app/issues/4926
+			name:  "should preserve --home if included before start",
+			input: []string{"celestia-appd", "--home", "foo", "start"},
+			want:  []string{"--home", "foo"},
+		},
+	}
+	for _, test := range tests {
+		got := removeStart(test.input)
+		require.Equal(t, test.want, got)
+	}
+}
diff --git a/test/docker-e2e/go.mod b/test/docker-e2e/go.mod
index ddeedac2..1f9db4d3 100644
--- a/test/docker-e2e/go.mod
+++ b/test/docker-e2e/go.mod
@@ -67,7 +67,7 @@ require (
 	github.com/aws/aws-sdk-go-v2/service/ssooidc v1.28.14 // indirect
 	github.com/aws/aws-sdk-go-v2/service/sts v1.33.14 // indirect
 	github.com/aws/smithy-go v1.22.2 // indirect
-	github.com/bcp-innovations/hyperlane-cosmos v1.0.0 // indirect
+	github.com/bcp-innovations/hyperlane-cosmos v1.0.1 // indirect
 	github.com/beorn7/perks v1.0.1 // indirect
 	github.com/bgentry/go-netrc v0.0.0-20140422174119-9fd32a8b3d3d // indirect
 	github.com/bgentry/speakeasy v0.2.0 // indirect
diff --git a/test/docker-e2e/go.sum b/test/docker-e2e/go.sum
index 4e941c4f..3d2b2d69 100644
--- a/test/docker-e2e/go.sum
+++ b/test/docker-e2e/go.sum
@@ -318,8 +318,8 @@ github.com/aws/aws-sdk-go-v2/service/sts v1.33.14 h1:TzeR06UCMUq+KA3bDkujxK1GVGy
 github.com/aws/aws-sdk-go-v2/service/sts v1.33.14/go.mod h1:dspXf/oYWGWo6DEvj98wpaTeqt5+DMidZD0A9BYTizc=
 github.com/aws/smithy-go v1.22.2 h1:6D9hW43xKFrRx/tXXfAlIZc4JI+yQe6snnWcQyxSyLQ=
 github.com/aws/smithy-go v1.22.2/go.mod h1:irrKGvNn1InZwb2d7fkIRNucdfwR8R+Ts3wxYa/cJHg=
-github.com/bcp-innovations/hyperlane-cosmos v1.0.0 h1:h1AP5gTCCWABlIPxy5pXXuSvMmDgCie1U/iRxNa+TT0=
-github.com/bcp-innovations/hyperlane-cosmos v1.0.0/go.mod h1:3yfa0io5Ii6GmhHWsWl2LEAOEHsqWuMgw2R02+LPogw=
+github.com/bcp-innovations/hyperlane-cosmos v1.0.1 h1:gT8OqyJ866Q6AHOlIXKxSdLjd0p8crKG9XXERIWoh4c=
+github.com/bcp-innovations/hyperlane-cosmos v1.0.1/go.mod h1:3yfa0io5Ii6GmhHWsWl2LEAOEHsqWuMgw2R02+LPogw=
 github.com/benbjohnson/clock v1.1.0/go.mod h1:J11/hYXuz8f4ySSvYwY0FKfm+ezbsZBKZxNJlLklBHA=
 github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q=
 github.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+CedLV8=