Repository: neo-project/neo
Pull request #3802: Ensure the interop types are serializable before commit
Author: login: shargon
Labels: name: Bug
description: Used to tag confirmed bugs
name: Critical
description: Issues (bugs) that need to be fixed ASAP
name: Ready to Merge
Created: 2025-03-05T08:55:33Z | Merged: 2025-03-20T10:23:05Z

# Description

Make sure the manifest is serializable before committing it.
Before this commit, a possible error will be thrown during the commit.

## Type of change

<!-- Please delete options that are not relevant. -->

- [ ] Optimization (the change is only an optimization)
- [ ] Style (the change is only a code style for better maintenance or standard purpose)
- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
- [ ] This change requires a documentation update

# How Has This Been Tested?

<!-- Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration -->

- [ ] Test A
- [ ] Test B

**Test Configuration**:


# Checklist:

- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged and published in downstream modules



Diff:
diff --git a/src/Neo/SmartContract/Native/ContractManagement.cs b/src/Neo/SmartContract/Native/ContractManagement.cs
index 7cdfdad91a..5a45b2ca15 100644
--- a/src/Neo/SmartContract/Native/ContractManagement.cs
+++ b/src/Neo/SmartContract/Native/ContractManagement.cs
@@ -93,7 +93,7 @@ internal override async ContractTask OnPersistAsync(ApplicationEngine engine)
                     else
                     {
                         // Parse old contract
-                        var oldContract = state.GetInteroperable<ContractState>(false);
+                        using var sealInterop = state.GetInteroperable(out ContractState oldContract, false);
                         // Increase the update counter
                         oldContract.UpdateCounter++;
                         // Modify nef and manifest
@@ -247,7 +247,7 @@ private async ContractTask<ContractState> Deploy(ApplicationEngine engine, byte[
 
             if (!contract.Manifest.IsValid(engine.Limits, hash)) throw new InvalidOperationException($"Invalid Manifest: {hash}");
 
-            engine.SnapshotCache.Add(key, new StorageItem(contract));
+            engine.SnapshotCache.Add(key, StorageItem.CreateSealed(contract));
             engine.SnapshotCache.Add(CreateStorageKey(Prefix_ContractHash, contract.Id), new StorageItem(hash.ToArray()));
 
             await OnDeployAsync(engine, contract, data, false);
@@ -268,7 +268,10 @@ private ContractTask Update(ApplicationEngine engine, byte[] nefFile, byte[] man
 
             engine.AddFee(engine.StoragePrice * ((nefFile?.Length ?? 0) + (manifest?.Length ?? 0)));
 
-            var contract = engine.SnapshotCache.GetAndChange(CreateStorageKey(Prefix_Contract, engine.CallingScriptHash))?.GetInteroperable<ContractState>(false);
+            var contractState = engine.SnapshotCache.GetAndChange(CreateStorageKey(Prefix_Contract, engine.CallingScriptHash))
+                ?? throw new InvalidOperationException($"Updating Contract Does Not Exist: {engine.CallingScriptHash}");
+
+            using var sealInterop = contractState.GetInteroperable(out ContractState contract, false);
             if (contract is null) throw new InvalidOperationException($"Updating Contract Does Not Exist: {engine.CallingScriptHash}");
             if (contract.UpdateCounter == ushort.MaxValue) throw new InvalidOperationException($"The contract reached the maximum number of updates.");
 
diff --git a/src/Neo/SmartContract/Native/LedgerContract.cs b/src/Neo/SmartContract/Native/LedgerContract.cs
index 9c0e1fe492..664cd53bc4 100644
--- a/src/Neo/SmartContract/Native/LedgerContract.cs
+++ b/src/Neo/SmartContract/Native/LedgerContract.cs
@@ -48,7 +48,7 @@ internal override ContractTask OnPersistAsync(ApplicationEngine engine)
                 State = VMState.NONE
             }).ToArray();
             engine.SnapshotCache.Add(CreateStorageKey(Prefix_BlockHash, engine.PersistingBlock.Index), new StorageItem(engine.PersistingBlock.Hash.ToArray()));
-            engine.SnapshotCache.Add(CreateStorageKey(Prefix_Block, engine.PersistingBlock.Hash), new StorageItem(Trim(engine.PersistingBlock).ToArray()));
+            engine.SnapshotCache.Add(CreateStorageKey(Prefix_Block, engine.PersistingBlock.Hash), new StorageItem(TrimmedBlock.Create(engine.PersistingBlock).ToArray()));
             foreach (TransactionState tx in transactions)
             {
                 // It's possible that there are previously saved malicious conflict records for this transaction.
@@ -373,14 +373,5 @@ private Transaction GetTransactionFromBlock(ApplicationEngine engine, byte[] blo
                 throw new ArgumentOutOfRangeException(nameof(txIndex));
             return GetTransaction(engine.SnapshotCache, block.Hashes[txIndex]);
         }
-
-        private static TrimmedBlock Trim(Block block)
-        {
-            return new TrimmedBlock
-            {
-                Header = block.Header,
-                Hashes = block.Transactions.Select(p => p.Hash).ToArray()
-            };
-        }
     }
 }
diff --git a/src/Neo/SmartContract/Native/OracleContract.cs b/src/Neo/SmartContract/Native/OracleContract.cs
index ac9c7a96e6..0e609a5c3b 100644
--- a/src/Neo/SmartContract/Native/OracleContract.cs
+++ b/src/Neo/SmartContract/Native/OracleContract.cs
@@ -163,13 +163,14 @@ internal override async ContractTask PostPersistAsync(ApplicationEngine engine)
 
                 //Remove the request from storage
                 StorageKey key = CreateStorageKey(Prefix_Request, response.Id);
-                OracleRequest request = engine.SnapshotCache.TryGet(key)?.GetInteroperable<OracleRequest>();
+                // Don't need to seal because it's read-only
+                var request = engine.SnapshotCache.TryGet(key)?.GetInteroperable<OracleRequest>();
                 if (request == null) continue;
                 engine.SnapshotCache.Delete(key);
 
                 //Remove the id from IdList
                 key = CreateStorageKey(Prefix_IdList, GetUrlHash(request.Url));
-                IdList list = engine.SnapshotCache.GetAndChange(key).GetInteroperable<IdList>();
+                using var sealInterop = engine.SnapshotCache.GetAndChange(key).GetInteroperable(out IdList list);
                 if (!list.Remove(response.Id)) throw new InvalidOperationException();
                 if (list.Count == 0) engine.SnapshotCache.Delete(key);
 
@@ -215,7 +216,7 @@ private async ContractTask Request(ApplicationEngine engine, string url, string
             //Put the request to storage
             if (ContractManagement.GetContract(engine.SnapshotCache, engine.CallingScriptHash) is null)
                 throw new InvalidOperationException();
-            engine.SnapshotCache.Add(CreateStorageKey(Prefix_Request, id), new StorageItem(new OracleRequest
+            var request = new OracleRequest
             {
                 OriginalTxid = GetOriginalTxid(engine),
                 GasForResponse = gasForResponse,
@@ -224,10 +225,14 @@ private async ContractTask Request(ApplicationEngine engine, string url, string
                 CallbackContract = engine.CallingScriptHash,
                 CallbackMethod = callback,
                 UserData = BinarySerializer.Serialize(userData, MaxUserDataLength, engine.Limits.MaxStackSize)
-            }));
+            };
+            engine.SnapshotCache.Add(CreateStorageKey(Prefix_Request, id), StorageItem.CreateSealed(request));
 
             //Add the id to the IdList
-            var list = engine.SnapshotCache.GetAndChange(CreateStorageKey(Prefix_IdList, GetUrlHash(url)), () => new StorageItem(new IdList())).GetInteroperable<IdList>();
+            using var sealInterop = engine.SnapshotCache.GetAndChange
+                (CreateStorageKey(Prefix_IdList, GetUrlHash(url)), () => new StorageItem(new IdList()))
+                .GetInteroperable(out IdList list);
+
             if (list.Count >= 256)
                 throw new InvalidOperationException("There are too many pending responses for this url");
             list.Add(id);
diff --git a/src/Neo/SmartContract/Native/TrimmedBlock.cs b/src/Neo/SmartContract/Native/TrimmedBlock.cs
index 8ada1c8ce3..67614084de 100644
--- a/src/Neo/SmartContract/Native/TrimmedBlock.cs
+++ b/src/Neo/SmartContract/Native/TrimmedBlock.cs
@@ -16,6 +16,7 @@
 using Neo.VM.Types;
 using System;
 using System.IO;
+using System.Linq;
 using Array = Neo.VM.Types.Array;
 
 namespace Neo.SmartContract.Native
@@ -47,6 +48,31 @@ public class TrimmedBlock : IInteroperable, ISerializable
 
         public int Size => Header.Size + Hashes.GetVarSize();
 
+        /// <summary>
+        /// Create Trimmed block
+        /// </summary>
+        /// <param name="block">Block</param>
+        /// <returns></returns>
+        public static TrimmedBlock Create(Block block)
+        {
+            return Create(block.Header, block.Transactions.Select(p => p.Hash).ToArray());
+        }
+
+        /// <summary>
+        /// Create Trimmed block
+        /// </summary>
+        /// <param name="header">Block header</param>
+        /// <param name="txHashes">Transaction hashes</param>
+        /// <returns></returns>
+        public static TrimmedBlock Create(Header header, UInt256[] txHashes)
+        {
+            return new TrimmedBlock
+            {
+                Header = header,
+                Hashes = txHashes
+            };
+        }
+
         public void Deserialize(ref MemoryReader reader)
         {
             Header = reader.ReadSerializable<Header>();
@@ -70,7 +96,7 @@ IInteroperable IInteroperable.Clone()
 
         void IInteroperable.FromReplica(IInteroperable replica)
         {
-            TrimmedBlock from = (TrimmedBlock)replica;
+            var from = (TrimmedBlock)replica;
             Header = from.Header;
             Hashes = from.Hashes;
         }
@@ -82,8 +108,8 @@ void IInteroperable.FromStackItem(StackItem stackItem)
 
         StackItem IInteroperable.ToStackItem(IReferenceCounter referenceCounter)
         {
-            return new Array(referenceCounter, new StackItem[]
-            {
+            return new Array(referenceCounter,
+            [
                 // Computed properties
                 Header.Hash.ToArray(),
 
@@ -99,7 +125,7 @@ StackItem IInteroperable.ToStackItem(IReferenceCounter referenceCounter)
 
                 // Block properties
                 Hashes.Length
-            });
+            ]);
         }
     }
 }
diff --git a/src/Neo/SmartContract/StorageItem.cs b/src/Neo/SmartContract/StorageItem.cs
index ae856e7eec..eba2a9d1cc 100644
--- a/src/Neo/SmartContract/StorageItem.cs
+++ b/src/Neo/SmartContract/StorageItem.cs
@@ -25,6 +25,16 @@ namespace Neo.SmartContract
     /// </summary>
     public class StorageItem : ISerializable
     {
+        private class SealInteroperable(StorageItem item) : IDisposable
+        {
+            public readonly StorageItem Item = item;
+
+            public void Dispose()
+            {
+                Item.Seal();
+            }
+        }
+
         private ReadOnlyMemory<byte> _value;
         private object? _cache;
 
@@ -84,6 +94,45 @@ public StorageItem(IInteroperable interoperable)
             _cache = interoperable;
         }
 
+        /// <summary>
+        /// Create a new instance from an sealed <see cref="IInteroperable"/> class. 
+        /// </summary>
+        /// <param name="interoperable">The <see cref="IInteroperable"/> value of the <see cref="StorageItem"/>.</param>
+        /// <returns><see cref="StorageItem"/> class</returns>
+        public static StorageItem CreateSealed(IInteroperable interoperable)
+        {
+            var item = new StorageItem(interoperable);
+            item.Seal();
+            return item;
+        }
+
+        /// <summary>
+        /// Returns true if the <see cref="IInteroperable"/> class is serializable
+        /// </summary>
+        /// <param name="interoperable">The <see cref="IInteroperable"/> value of the <see cref="StorageItem"/>.</param>
+        /// <returns>True if serializable</returns>
+        public static bool IsSerializable(IInteroperable interoperable)
+        {
+            try
+            {
+                _ = CreateSealed(interoperable);
+            }
+            catch
+            {
+                return false;
+            }
+            return true;
+        }
+
+        /// <summary>
+        /// Ensure that is Serializable and cache the value
+        /// </summary>
+        public void Seal()
+        {
+            // Assert is Serializable and cached
+            _ = Value;
+        }
+
         /// <summary>
         /// Increases the integer value in the store by the specified value.
         /// </summary>
@@ -166,6 +215,31 @@ public void FromReplica(StorageItem replica)
             return (T)_cache;
         }
 
+        /// <summary>
+        /// Gets an <see cref="IInteroperable"/> from the storage.
+        /// </summary>
+        /// <param name="interop">The <see cref="IInteroperable"/> in the storage.</param>
+        /// <typeparam name="T">The type of the <see cref="IInteroperable"/>.</typeparam>
+        /// <returns>The <see cref="IDisposable"/> that seal the item when disposed.</returns>
+        public IDisposable GetInteroperable<T>(out T interop) where T : IInteroperable, new()
+        {
+            interop = GetInteroperable<T>();
+            return new SealInteroperable(this);
+        }
+
+        /// <summary>
+        /// Gets an <see cref="IInteroperable"/> from the storage.
+        /// </summary>
+        /// <param name="interop">The <see cref="IInteroperable"/> in the storage.</param>
+        /// <param name="verify">Verify deserialization</param>
+        /// <typeparam name="T">The type of the <see cref="IInteroperable"/>.</typeparam>
+        /// <returns>The <see cref="IDisposable"/> that seal the item when disposed.</returns>
+        public IDisposable GetInteroperable<T>(out T interop, bool verify = true) where T : IInteroperableVerifiable, new()
+        {
+            interop = GetInteroperable<T>(verify);
+            return new SealInteroperable(this);
+        }
+
         public void Serialize(BinaryWriter writer)
         {
             writer.Write(Value.Span);
@@ -181,16 +255,6 @@ public void Set(BigInteger integer)
             _value = null;
         }
 
-        /// <summary>
-        /// Sets the interoperable value of the storage.
-        /// </summary>
-        /// <param name="interoperable">The <see cref="IInteroperable"/> value of the <see cref="StorageItem"/>.</param>
-        public void Set(IInteroperable interoperable)
-        {
-            _cache = interoperable;
-            _value = null;
-        }
-
         public static implicit operator BigInteger(StorageItem item)
         {
             item._cache ??= new BigInteger(item._value.Span);


Reviews:
user: login: vncoelho
body: LGTM
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-05T08:57:26Z
user: login: vncoelho
state: DISMISSED
author_association: MEMBER
submitted_at: 2025-03-05T09:12:08Z
user: login: AnnaShaleva
body: The fix itself LGTM, I agree that we don't need a hardfork for it since without this fix nodes just can't process this block.

> we can destroy contract as well in the same transaction.

I don't think that it's a problem with the current implementation, because the `deploy` function should end with contract serialization irrespectively if there's a call to `destroy` after it or not.

But we should probably check that we don't have other similar situations like this with delayed items serialization. The most suspicions place to me is Ledger's block serialization:
https://github.com/neo-project/neo/blob/bb153eb993d963b1a19336a1bfd26ed877053b9d/src/Neo/SmartContract/Native/LedgerContract.cs#L51

But likely consensus can't accept block that can't be serialized.

Still, it would be nice to double-check. Otherwise LGTM.
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-06T16:25:09Z
user: login: AnnaShaleva
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-07T11:36:45Z
user: login: shargon
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-07T14:32:23Z
user: login: shargon
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-07T14:33:30Z
user: login: AnnaShaleva
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-11T16:33:50Z
user: login: shargon
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-11T16:37:22Z
user: login: AnnaShaleva
body: LGTM. May I ask @superboyiii to test this implementation one more time on the "problem" block of Testnet and ensure that it still works properly, because the implementation has been slightly changed comparing to the initial fix.
state: APPROVED
author_association: MEMBER
submitted_at: 2025-03-17T11:12:43Z
user: login: Wi1l-B0t
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-03-17T13:01:34Z
user: login: Jim8y
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-03-20T08:38:31Z

Review comments:
diff_hunk: @@ -247,7 +247,7 @@ private async ContractTask<ContractState> Deploy(ApplicationEngine engine, byte[
 
             if (!contract.Manifest.IsValid(engine.Limits, hash)) throw new InvalidOperationException($"Invalid Manifest: {hash}");
 
-            engine.SnapshotCache.Add(key, new StorageItem(contract));
+            engine.SnapshotCache.Add(key, StorageItem.CreateSealed(contract));
path: src/Neo/SmartContract/Native/ContractManagement.cs
user: login: AnnaShaleva
body: To be honest, I liked the previous solution with explicit call to serializator. The problem of the current solution is that serialization happens in `CreateSealed` whereas it's expected to happen during attempt to `engine.SnapshotCache.Add`. For this particular line of code it doesn't matter, but if we're going to add sometihng like this:
```
var si = StorageItem.CreateSealed(contract);

... // do some additional actions or checks

engine.SnapshotCache.Add(key, si);
```
then the current implementation will fault the execution during the call to `StorageItem.CreateSealed`, whereas in fact execution should be faulted during the call to `engine.SnapshotCache.Add`. So this way of protection from delayed serialization looks more error-prone than the previous implementation to me.
created_at: 2025-03-07T11:32:54Z
author_association: MEMBER
diff_hunk: @@ -268,7 +268,10 @@ private ContractTask Update(ApplicationEngine engine, byte[] nefFile, byte[] man
 
             engine.AddFee(engine.StoragePrice * ((nefFile?.Length ?? 0) + (manifest?.Length ?? 0)));
 
-            var contract = engine.SnapshotCache.GetAndChange(CreateStorageKey(Prefix_Contract, engine.CallingScriptHash))?.GetInteroperable<ContractState>(false);
+            var contractState = engine.SnapshotCache.GetAndChange(CreateStorageKey(Prefix_Contract, engine.CallingScriptHash))
+                ?? throw new InvalidOperationException($"Updating Contract Does Not Exist: {engine.CallingScriptHash}");
+
+            using var sealInterop = contractState.GetInteroperable(out ContractState contract, false);
path: src/Neo/SmartContract/Native/ContractManagement.cs
user: login: AnnaShaleva
body: From the current implementation it looks to me like the disposer of `sealInterop` will be called by the moment when execution leaves the `using` statement which means that this line is trying to serialize old contract. But I'm likely mistaken, could you confirm? We need to serialize the updated contract.
created_at: 2025-03-07T11:36:36Z
author_association: MEMBER
diff_hunk: @@ -268,7 +268,10 @@ private ContractTask Update(ApplicationEngine engine, byte[] nefFile, byte[] man
 
             engine.AddFee(engine.StoragePrice * ((nefFile?.Length ?? 0) + (manifest?.Length ?? 0)));
 
-            var contract = engine.SnapshotCache.GetAndChange(CreateStorageKey(Prefix_Contract, engine.CallingScriptHash))?.GetInteroperable<ContractState>(false);
+            var contractState = engine.SnapshotCache.GetAndChange(CreateStorageKey(Prefix_Contract, engine.CallingScriptHash))
+                ?? throw new InvalidOperationException($"Updating Contract Does Not Exist: {engine.CallingScriptHash}");
+
+            using var sealInterop = contractState.GetInteroperable(out ContractState contract, false);
path: src/Neo/SmartContract/Native/ContractManagement.cs
user: login: shargon
body: It will be sealed on dispose, exactly 
created_at: 2025-03-07T14:32:23Z
author_association: MEMBER
diff_hunk: @@ -247,7 +247,7 @@ private async ContractTask<ContractState> Deploy(ApplicationEngine engine, byte[
 
             if (!contract.Manifest.IsValid(engine.Limits, hash)) throw new InvalidOperationException($"Invalid Manifest: {hash}");
 
-            engine.SnapshotCache.Add(key, new StorageItem(contract));
+            engine.SnapshotCache.Add(key, StorageItem.CreateSealed(contract));
path: src/Neo/SmartContract/Native/ContractManagement.cs
user: login: shargon
body: This implementation is safer, the previous one ensure that is serializable, but if you change it after this call, it could become unserialziable again, now, before leave the method it will be sealed, and the exactly value will be committed 
created_at: 2025-03-07T14:33:30Z
author_association: MEMBER
diff_hunk: @@ -268,7 +268,10 @@ private ContractTask Update(ApplicationEngine engine, byte[] nefFile, byte[] man
 
             engine.AddFee(engine.StoragePrice * ((nefFile?.Length ?? 0) + (manifest?.Length ?? 0)));
 
-            var contract = engine.SnapshotCache.GetAndChange(CreateStorageKey(Prefix_Contract, engine.CallingScriptHash))?.GetInteroperable<ContractState>(false);
+            var contractState = engine.SnapshotCache.GetAndChange(CreateStorageKey(Prefix_Contract, engine.CallingScriptHash))
+                ?? throw new InvalidOperationException($"Updating Contract Does Not Exist: {engine.CallingScriptHash}");
+
+            using var sealInterop = contractState.GetInteroperable(out ContractState contract, false);
path: src/Neo/SmartContract/Native/ContractManagement.cs
user: login: AnnaShaleva
body: But when exactly dispose happens? In the end of `Update` function or right after `using` statement?
created_at: 2025-03-11T16:33:50Z
author_association: MEMBER
diff_hunk: @@ -268,7 +268,10 @@ private ContractTask Update(ApplicationEngine engine, byte[] nefFile, byte[] man
 
             engine.AddFee(engine.StoragePrice * ((nefFile?.Length ?? 0) + (manifest?.Length ?? 0)));
 
-            var contract = engine.SnapshotCache.GetAndChange(CreateStorageKey(Prefix_Contract, engine.CallingScriptHash))?.GetInteroperable<ContractState>(false);
+            var contractState = engine.SnapshotCache.GetAndChange(CreateStorageKey(Prefix_Contract, engine.CallingScriptHash))
+                ?? throw new InvalidOperationException($"Updating Contract Does Not Exist: {engine.CallingScriptHash}");
+
+            using var sealInterop = contractState.GetInteroperable(out ContractState contract, false);
path: src/Neo/SmartContract/Native/ContractManagement.cs
user: login: shargon
body: this case at the end of `Update` methdod
created_at: 2025-03-11T16:37:22Z
author_association: MEMBER

PR comments:
user: login: dusmart
created_at: 2025-03-05T09:17:14Z
body: Thank you for your PR that resolved the testnet halt issue caused by the attack. We definitely support merging it.  

I’d like to propose a consideration that might lead to a new PR or issue: could we allow all possible outcomes after a successful simulation and block packaging, especially when the transaction simulation results in `VMState.HALT`? Instead of throwing exceptions, this could cover scenarios like those we encountered during `Storage.Commit`, or even within `OnPersist` and `PostPersist`, helping to mitigate similar attacks in the future.  

While the likelihood of an issue like today’s happening again is quite small, logging a high-level error could serve as an early warning. Take it into today’s case: the blockchain won't be halted by the attack, the affected contract might become unreadable due to deserialization failure — which, arguably, is a slightly better outcome. 

What do you think?  

author_association: NONE
user: login: dusmart
created_at: 2025-03-05T09:25:38Z
body: Additionally, I’m wondering if there might be other potential risks beyond the contract checks — could other `IInteroperable` instances also cause similar issues during Storage.Commit? @Jim8y, could you and your teammates help take a closer look?

Shargon has also found the `Oracle`'s problem. Great job!
author_association: NONE
user: login: shargon
created_at: 2025-03-05T09:27:47Z
body: > Thank you for your PR that resolved the testnet halt issue caused by the attack. We definitely support merging it.
> 
> I’d like to propose a consideration that might lead to a new PR or issue: could we allow all possible outcomes after a successful simulation and block packaging, especially when the transaction simulation results in `VMState.HALT`? Instead of throwing exceptions, this could cover scenarios like those we encountered during `Storage.Commit`, or even within `OnPersist` and `PostPersist`, helping to mitigate similar attacks in the future.
> 
> While the likelihood of an issue like today’s happening again is quite small, logging a high-level error could serve as an early warning. Take it into today’s case: the blockchain won't be halted by the attack, the affected contract might become unreadable due to deserialization failure — which, arguably, is a slightly better outcome.
> 
> What do you think?

Yes, we will deep into it
author_association: MEMBER
user: login: roman-khimov
created_at: 2025-03-05T09:37:41Z
body: > the testnet halt issue caused by the attack

It's certainly not an attack, just an accident. People write contracts, people try them on testnet.

> those we encountered during Storage.Commit

Each state change is bound to some execution context by design. The core of the problem is that this assumption is broken, transaction does something and fully succeeds, but then it turns out that the state change is incorrect and should've never happened.

> within OnPersist and PostPersist

I don't think you can do this. It's pretty similar to the problem of insufficient GAS balance during OnPersist processing. If this happens we're toast, there is no correct behavior when we have a correctly signed block that does something wrong. This can be recovered only via a fork, throwing away a block and creating a new one.
author_association: CONTRIBUTOR
user: login: dusmart
created_at: 2025-03-05T10:20:42Z
body: > > those we encountered during Storage.Commit
> 
> Each state change is bound to some execution context by design. The core of the problem is that this assumption is broken, transaction does something and fully succeeds, but then it turns out that the state change is incorrect and should've never happened.

Perhaps, at the very least, we should have a reminder for others developers — ensuring that every new `IInteroperable` includes a check to confirm it can be properly serialized if it will be committed to the storage during transaction simulation. Otherwise, wouldn’t this kind of issue be something only experienced developers like you and Shargon would think to catch?

Or perhaps we should consider simulating the commit process during transaction simulation as well for all kinds of txs. If any errors occur during commit, the simulation shouldn't return a HALT state.
author_association: NONE
user: login: shargon
created_at: 2025-03-05T10:54:42Z
body: > Or perhaps we should consider simulating the commit process during transaction simulation as well for all kinds of txs. If any errors occur during commit, the simulation shouldn't return a HALT state.

This can be done in consensus, although the transaction may give a different result (not care because it's a commit test).
author_association: MEMBER
user: login: superboyiii
created_at: 2025-03-06T03:25:09Z
body: We need `HF` flag here. @shargon 
author_association: MEMBER
user: login: roman-khimov
created_at: 2025-03-06T05:52:24Z
body: But why? You can't do
```
if old_node
    do_something()
else
    do_something_else()
```
here. The buggy node just stops processing and can't persist this block, nothing happened. Proper ones are perfectly aligned with this PR in behavior, they fail the transaction.
author_association: CONTRIBUTOR
user: login: shargon
created_at: 2025-03-06T06:58:15Z
body: Agree with @roman-khimov , old node doesn't work, so no hf needed
author_association: MEMBER
user: login: roman-khimov
created_at: 2025-03-06T08:12:26Z
body: It just occured to me that we can destroy contract as well in the same transaction. Can this be a problem?

Frankly I'd rather roll out the fix quickly for mainnet and be done with that.
author_association: CONTRIBUTOR
user: login: shargon
created_at: 2025-03-06T08:44:46Z
body: @neo-project/core please review
author_association: MEMBER
user: login: superboyiii
created_at: 2025-03-06T09:36:55Z
body: > But why? You can't do
> 
> ```
> if old_node
>     do_something()
> else
>     do_something_else()
> ```
> 
> here. The buggy node just stops processing and can't persist this block, nothing happened. Proper ones are perfectly aligned with this PR in behavior, they fail the transaction.

It's a different behaviour and different native contract, you don't know what may happen before next release. Even stateroot is also different when resync. No doubt, it's a hardfork already.
author_association: MEMBER
user: login: roman-khimov
created_at: 2025-03-06T09:50:43Z
body: IIUC you don't have any stateroot for testnet block 5557998 with the old node because to get a state root you need a state first and you don't have a state because there is an unserializable thing in the cache. Then if we're to tie this to Echidna the question is how to process testnet block 5557998, it's certainly pre-Echidna (unless you're to set it earlier and require reprocessing), but you can't have the old behavior because it just stops the node completely. Then also mainnet, I won't talk about it much here, but you understand the current state of it perfectly with the unpatched node.
author_association: CONTRIBUTOR
user: login: superboyiii
created_at: 2025-03-06T10:13:42Z
body: > IIUC you don't have any stateroot for testnet block 5557998 with the old node because to get a state root you need a state first and you don't have a state because there is an unserializable thing in the cache. Then if we're to tie this to Echidna the question is how to process testnet block 5557998, it's certainly pre-Echidna (unless you're to set it earlier and require reprocessing), but you can't have the old behavior because it just stops the node completely. Then also mainnet, I won't talk about it much here, but you understand the current state of it perfectly with the unpatched node.

So nothing breaks on mainnet, but a different storage from block 0 in v3.8.0. You think it works as expected?
author_association: MEMBER
user: login: roman-khimov
created_at: 2025-03-06T10:24:11Z
body: Why from block 0? In this particular PR.
author_association: CONTRIBUTOR
user: login: superboyiii
created_at: 2025-03-06T10:27:24Z
body: > Why from block 0? In this particular PR.

Don't add fork height means it's applied from the begining...

author_association: MEMBER
user: login: roman-khimov
created_at: 2025-03-06T10:31:01Z
body: Sure. And state is exactly the same.
author_association: CONTRIBUTOR
user: login: shargon
created_at: 2025-03-07T09:26:10Z
body: > But likely consensus can't accept block that can't be serialized.

Yes, they can, because one is a binary serialization, and the other a stack item serialization.


author_association: MEMBER
user: login: shargon
created_at: 2025-03-07T09:32:29Z
body: @AnnaShaleva please take a look to the new changes, it also checks block viability during prepare request phase
author_association: MEMBER
user: login: shargon
created_at: 2025-03-20T08:37:05Z
body: @Jim8y Merge?
author_association: MEMBER