GHSA-jg6f-48ff-5xrw: IBC-Go has Non-deterministic JSON Unmarshalling of IBC Acknowledgement
Name: ASA-2025-004: Non-deterministic JSON Unmarshalling of IBC Acknowledgement can result in a chain halt
Component: IBC-Go
Criticality: Critical (Considerable Impact; Almost Certain Likelihood per [ACMv1.2](https://github.com/interchainio/security/blob/main/resources/CLASSIFICATION_MATRIX.md))
Affected versions: IBC-Go >= v7; Earlier IBC-Go versions may also be affected.
Affected users: Validators, Full nodes, IBC Middleware authorsDescriptionAn issue was discovered in IBC-Go's deserialization of acknowledgements that results in non-deterministic behavior which can halt a chain. Any user that can open an IBC channel can introduce this state to the chainPatchesThe new IBC-Go releases below address this issue:[v7.9.2](https://github.com/cosmos/ibc-go/releases/tag/v7.9.2)[v8.6.1](https://github.com/cosmos/ibc-go/releases/tag/v8.6.1)WorkaroundsTo prevent this state from being introduced to a chain, it is possible to permission Channel Opening as a workaround.Notes on Re-ReleaseIs this state breaking? Probably not but it depends on your transfer middlewaresThis patch is not state breaking unless you depend on transfer middlewares that deserialize and serialize acknowledgement packets before passing them to the transfer handler.  As far as we can tell, these middlewares are rare. For example, packet-forward-middleware and ibc-hooks, do not serialize ack packets in this way and therefore aren't broken by this patch. So if these are the only transfer middlewares you depend on, you can safely apply this patch in a rolling manner (and we've already cut new versions of these for you).What to do if you do depend on ack-serializing middlewareIn the unlikely case that you depend on middlewares that serialize ack packets and you do not update them when you apply this patch, all transfers that are handled by the middleware will fail (or experience other unexpected behavior) if the serialization approach differs from the transfer app's. If you have such dependencies and do not update them, validators who apply the patch in a rolling manner will halt when they upgrade, and transfers processed by the middleware will just fail once everyone has upgraded.To update these middlewares and avoid failing transfers or a chain halt, you will simply need to change the serialization approach in the middleware to use ibc-go's codec: `transfertypes.ModuleCdc.[Must]MarshalJSON`, rather than whatever you're doing today.  For example:When you do make a change to the serialization approach, this will make the patch state breaking and you will need a coordinated upgrade. So for absolute clarity: chains with these ack-serializing middlewares must do coordinated upgradesWhy we retracted the earlier patch in favor of this approachWe retracted the releases of ibc-go we cut earlier today because these broke all transfer middlewares that deserialized then re-serialized receive packets differently than the transfer app. It turned out that this was a common pattern (unlike serializing/deserializing ack packets), so widely used middlewares, including packet-forward-middleware, broke unexpectedly.In the new set of patches, we removed this constraint on how middlewares serialize receive packets, preventing this breakage. Only the serialization requirement on acknowledgement packets remains. This is convenient because this is the only constraint we had to add to fix the vulnerability, and middlewares that deserialize and serialize ack packets are much less common than ones that do so for receive packets. The constraint on receive packets was added for defense in depth.Testing we have done to gain more confidence in this releaseIn addition to testing ibc-go, we also did the following:Tested pfm v7 and v8 after bumping dependenciesTested ibc-hooks v7 and v8 after bumping dependenciesRan a patched node on mainnet on the cosmos hub and triggered failing and successful transactions that used PFMRan a patched node on osmosis and triggered failing and successful transactions that used ibc-hooks
This is a more thorough process than before, so we have higher confidence.TimelineFebruary 18, 2025, 4:54am PST: Issue reported to the Cosmos Bug Bounty programFebruary 18, 2025, 6:56am PST: Issue triaged by Amulet on-call, and distributed to Core teamFebruary 18, 2025, 8:15am PST: Core team completes validation of issueFebruary 25, 2025, 8:00am PST / 17:00 CET: Pre-notification deliveredFebruary 27, 2025, 8:00am PST / 17:00 CET: Patch made availableFebruary 27, 2025, 1:00pm PST: Patch re-release made availableThis issue was reported to the Cosmos Bug Bounty Program by swelf19 on HackerOne on February 18, 2025. If you believe you have found a bug in the Interchain Stack or would like to contribute to the program by reporting a bug, please see https://hackerone.com/cosmos.If you have questions about Interchain security efforts, please reach out to our official communication channel at [security@interchain.io](mailto:security@interchain.io). For more information about the Interchain Foundation’s engagement with Amulet, and to sign up for security notification emails, please see https://github.com/interchainio/security.A Github Security Advisory for this issue is available in the IBC-Go [repository](https://github.com/cosmos/ibc-go/security/advisories/GHSA-jg6f-48ff-5xrw).
CVSS v4.0: 9.3