Repository: JasperFx/marten (2026-07-29T18:24:51Z)

Issue:
Newtonsoft: persisting a lazy LINQ sequence writes an unreadable $type and breaks all later reads
Assigning a lazy LINQ sequence to a persisted member produces a document that **writes successfully but can never be read back** when the store uses Newtonsoft.

## What happens

`JsonNetSerializer` uses `TypeNameHandling.Auto`, which emits `$type` whenever a member's runtime type differs from its declared type. A LINQ operator returns a private iterator class, so:

```csharp
public sealed record Product
{
    public required IEnumerable<ProductCode> KnownCodes { get; set; }
}

// somewhere in a projection
KnownCodes = product.KnownCodes
    .Where(c => c.Value != newCode)
    .Append(new ProductCode(newCode));   // never materialised
```

persists as:

```json
"KnownCodes": {
    "$type": "System.Linq.Enumerable+AppendPrepend1Iterator`1[[...ProductCode, MyAssembly]], System.Linq",
    "$values": [ { "Value": "PROD-001" }, { "Value": "PROD-999" } ]
}
```

The write succeeds. Every subsequent read of that document throws:

```
Newtonsoft.Json.JsonSerializationException : Cannot create and populate list type
System.Linq.Enumerable+AppendPrepend1Iterator`1[...]. Path 'KnownCodes.$values'
```

`AppendPrepend1Iterator` is an internal implementation detail with no usable constructor, so the document is permanently unreadable. The same applies to any lazy operator — `Where`, `Select`, `Concat`, `Union`, `OrderBy` and so on all return iterator types.

System.Text.Json is unaffected: it ignores the runtime type and writes a plain array.

## Why this is worth guarding against

- **The failure is delayed and misattributed.** Nothing fails at write time. The exception arrives on a later read, in a different part of the system, pointing at a `System.Linq` type the user never mentioned.
- **The data is not recoverable through Marten.** The stored JSON is fine — only the `$type` is wrong — but every read path goes through the same deserializer.
- **It is easy to hit.** Any projection that builds a collection with LINQ and forgets `.ToArray()` / `.ToList()` is exposed. Documented Marten samples use `IEnumerable<T>` members freely.

## Found via

`DaemonTests.Aggregations.Bug_5041_natural_key_source_discovery` did exactly this. It went unnoticed because that suite has only ever run under System.Text.Json — its `[assembly: TestFramework]` named a type and assembly that no longer existed, so `DEFAULT_SERIALIZER` was never applied there (#5066). Materialising the sequences fixes the tests; this issue is about the underlying trap.

## Possible directions

1. Serialize `IEnumerable<T>` members as their element array type rather than the concrete iterator — i.e. suppress `$type` when the runtime type is a non-reconstructible framework iterator.
2. Use a `SerializationBinder` / `ContractResolver` that maps `System.Linq.Enumerable+*Iterator` to `T[]` on write.
3. Fail fast at write time with a message naming the member, instead of succeeding and breaking reads.
4. At minimum, document it.

(1) or (2) seem the most useful — the resulting JSON is already a plain array, so only the `$type` needs to change.



Discussion:

Patch:
diff --git a/src/Marten.Newtonsoft/Services/Json/JsonNetLazyEnumerableConverter.cs b/src/Marten.Newtonsoft/Services/Json/JsonNetLazyEnumerableConverter.cs
new file mode 100644
index 0000000000..74a961ad26
--- /dev/null
+++ b/src/Marten.Newtonsoft/Services/Json/JsonNetLazyEnumerableConverter.cs
@@ -0,0 +1,92 @@
+using System;
+using System.Collections;
+using System.Collections.Generic;
+using System.Linq;
+using Newtonsoft.Json;
+
+namespace Marten.Services.Json;
+
+/// <summary>
+///     Writes lazy LINQ sequences as plain JSON arrays.
+/// </summary>
+/// <remarks>
+///     <para>
+///     Marten's Newtonsoft settings use <see cref="TypeNameHandling.Auto"/>, which records
+///     <c>$type</c> whenever a value's runtime type differs from its declared type. LINQ operators
+///     return private iterator classes, so assigning one to a persisted member -- e.g.
+///     <c>IEnumerable&lt;T&gt; Codes { get; set; }</c> set from <c>.Where(...).Append(...)</c> --
+///     used to persist <c>$type</c> as something like
+///     <c>System.Linq.Enumerable+AppendPrepend1Iterator`1[[T]]</c>.
+///     </para>
+///     <para>
+///     Those types have no usable constructor, so the write succeeded and every later read of the
+///     document threw "Cannot create and populate list type". The stored payload was otherwise
+///     fine -- only the recorded type was unusable. See #5076.
+///     </para>
+///     <para>
+///     This converter only claims types that could never have been deserialized in the first
+///     place, so it turns a permanently unreadable document into a readable array and leaves
+///     every reconstructible collection (arrays, <c>List&lt;T&gt;</c>, user types) exactly as it
+///     was. Collection storage as a whole is a separate, opt-in concern -- see
+///     <see cref="JsonNetCollectionToArrayJsonConverter"/> and <c>CollectionStorage.AsArray</c>.
+///     </para>
+/// </remarks>
+public sealed class JsonNetLazyEnumerableConverter: JsonConverter
+{
+    public static readonly JsonNetLazyEnumerableConverter Instance = new();
+
+    /// <summary>
+    ///     Write only. These types never appear as a declared member type, so there is nothing to read.
+    /// </summary>
+    public override bool CanRead => false;
+
+    public override bool CanConvert(Type objectType)
+    {
+        if (objectType == null || !typeof(IEnumerable).IsAssignableFrom(objectType))
+        {
+            return false;
+        }
+
+        // Restricted to the framework's own non-public iterators. A user's internal collection
+        // type is left alone -- it may well be reconstructible, and suppressing its $type could
+        // break a legitimate polymorphic member.
+        if (objectType.Namespace is not "System.Linq")
+        {
+            return false;
+        }
+
+        return !objectType.IsPublic && !objectType.IsNestedPublic;
+    }
+
+    public override void WriteJson(JsonWriter writer, object? value, JsonSerializer serializer)
+    {
+        if (value is null)
+        {
+            writer.WriteNull();
+            return;
+        }
+
+        var items = ((IEnumerable)value).Cast<object>().ToArray();
+
+        // Materialise into an array of the element type rather than object[], so the element
+        // type matches what is written and Auto does not stamp $type onto each element.
+        var elementType = elementTypeOf(value.GetType()) ?? typeof(object);
+        var typed = Array.CreateInstance(elementType, items.Length);
+        for (var i = 0; i < items.Length; i++)
+        {
+            typed.SetValue(items[i], i);
+        }
+
+        serializer.Serialize(writer, typed, typed.GetType());
+    }
+
+    public override object ReadJson(JsonReader reader, Type objectType, object? existingValue,
+        JsonSerializer serializer) =>
+        throw new NotSupportedException(
+            $"{nameof(JsonNetLazyEnumerableConverter)} is write-only; {objectType} is never a declared member type.");
+
+    private static Type? elementTypeOf(Type type) =>
+        type.GetInterfaces()
+            .FirstOrDefault(i => i.IsGenericType && i.GetGenericTypeDefinition() == typeof(IEnumerable<>))
+            ?.GetGenericArguments()[0];
+}
diff --git a/src/Marten.Newtonsoft/Services/JsonNetSerializer.cs b/src/Marten.Newtonsoft/Services/JsonNetSerializer.cs
index 86fecc1515..32062c7068 100644
--- a/src/Marten.Newtonsoft/Services/JsonNetSerializer.cs
+++ b/src/Marten.Newtonsoft/Services/JsonNetSerializer.cs
@@ -45,7 +45,12 @@ public class JsonNetSerializer: ISerializer
         // ISO 8601 formatting of DateTime's is mandatory
         DateFormatHandling = DateFormatHandling.IsoDateFormat,
         MetadataPropertyHandling = MetadataPropertyHandling.ReadAhead,
-        ContractResolver = new JsonNetContractResolver()
+        ContractResolver = new JsonNetContractResolver(),
+
+        // TypeNameHandling.Auto would otherwise record a lazy LINQ iterator's concrete type,
+        // which cannot be reconstructed -- the document writes cleanly and then fails every
+        // read. See #5076.
+        Converters = { JsonNetLazyEnumerableConverter.Instance }
     };
 
     #endregion
@@ -56,7 +61,12 @@ public class JsonNetSerializer: ISerializer
     {
         TypeNameHandling = TypeNameHandling.Objects,
         DateFormatHandling = DateFormatHandling.IsoDateFormat,
-        ContractResolver = new JsonNetContractResolver()
+        ContractResolver = new JsonNetContractResolver(),
+
+        // Objects mode stamps $type onto everything, so a lazy LINQ sequence reaching this
+        // path -- PatchOperation writes polymorphic patch values through it -- hits the same
+        // unreadable-$type problem as the document serializer. See #5076.
+        Converters = { JsonNetLazyEnumerableConverter.Instance }
     };
 
     private readonly Lazy<JsonSerializer> _withTypes;


Test patch:
diff --git a/src/DocumentDbTests/Bugs/Bug_5076_lazy_enumerable_round_trip.cs b/src/DocumentDbTests/Bugs/Bug_5076_lazy_enumerable_round_trip.cs
new file mode 100644
index 0000000000..0f2f93fdb2
--- /dev/null
+++ b/src/DocumentDbTests/Bugs/Bug_5076_lazy_enumerable_round_trip.cs
@@ -0,0 +1,125 @@
+using System;
+using System.Collections.Generic;
+using System.Linq;
+using System.Threading.Tasks;
+using Marten;
+using Marten.Newtonsoft;
+using Marten.Patching;
+using Marten.Testing.Harness;
+using Shouldly;
+using Xunit;
+
+namespace DocumentDbTests.Bugs;
+
+/// <summary>
+/// #5076 — a lazy LINQ sequence assigned to a persisted member used to be written by
+/// Newtonsoft's TypeNameHandling.Auto as $type = the iterator's concrete type. The write
+/// succeeded and every later read threw, because types like
+/// System.Linq.Enumerable+AppendPrepend1Iterator cannot be constructed.
+/// </summary>
+public class Bug_5076_lazy_enumerable_round_trip: BugIntegrationContext
+{
+    public class Basket
+    {
+        public Guid Id { get; set; }
+
+        // Declared as IEnumerable<T> on purpose: that is what makes Newtonsoft record the
+        // runtime type, and it is a perfectly ordinary way to model a read-only sequence.
+        public IEnumerable<string> Items { get; set; } = [];
+    }
+
+    private void useNewtonsoft() => StoreOptions(opts =>
+    {
+        // Pinned so the test means the same thing on both CI serializer legs.
+        opts.UseNewtonsoftForSerialization();
+        opts.Schema.For<Basket>();
+    });
+
+    [Fact]
+    public async Task can_read_back_a_document_holding_an_appended_sequence()
+    {
+        useNewtonsoft();
+
+        var basket = new Basket { Items = new[] { "first" }.Append("second") };
+
+        await using (var session = theStore.LightweightSession())
+        {
+            session.Store(basket);
+            await session.SaveChangesAsync();
+        }
+
+        await using var query = theStore.QuerySession();
+        var loaded = await query.LoadAsync<Basket>(basket.Id);
+
+        loaded.ShouldNotBeNull();
+        loaded.Items.ShouldBe(new[] { "first", "second" });
+    }
+
+    [Fact]
+    public async Task can_read_back_a_document_holding_a_filtered_sequence()
+    {
+        useNewtonsoft();
+
+        var basket = new Basket { Items = new[] { "keep", "drop", "keep too" }.Where(x => x.StartsWith("keep")) };
+
+        await using (var session = theStore.LightweightSession())
+        {
+            session.Store(basket);
+            await session.SaveChangesAsync();
+        }
+
+        await using var query = theStore.QuerySession();
+        var loaded = await query.LoadAsync<Basket>(basket.Id);
+
+        loaded.ShouldNotBeNull();
+        loaded.Items.ShouldBe(new[] { "keep", "keep too" });
+    }
+
+    [Fact]
+    public async Task a_materialised_sequence_still_round_trips()
+    {
+        useNewtonsoft();
+
+        var basket = new Basket { Items = new List<string> { "one", "two" } };
+
+        await using (var session = theStore.LightweightSession())
+        {
+            session.Store(basket);
+            await session.SaveChangesAsync();
+        }
+
+        await using var query = theStore.QuerySession();
+        var loaded = await query.LoadAsync<Basket>(basket.Id);
+
+        loaded.ShouldNotBeNull();
+        loaded.Items.ShouldBe(new[] { "one", "two" });
+    }
+
+    [Fact]
+    public async Task can_read_back_after_patching_with_a_lazy_sequence()
+    {
+        useNewtonsoft();
+
+        var basket = new Basket { Items = new[] { "original" } };
+
+        await using (var session = theStore.LightweightSession())
+        {
+            session.Store(basket);
+            await session.SaveChangesAsync();
+        }
+
+        // Patch values go through the withTypes serializer, where TypeNameHandling.Objects
+        // stamps $type onto everything -- so a lazy sequence hits the same trap there.
+        await using (var session = theStore.LightweightSession())
+        {
+            session.Patch<Basket>(basket.Id).Set(x => x.Items, new[] { "a", "b", "skip" }.Where(x => x != "skip"));
+            await session.SaveChangesAsync();
+        }
+
+        await using var query = theStore.QuerySession();
+        var loaded = await query.LoadAsync<Basket>(basket.Id);
+
+        loaded.ShouldNotBeNull();
+        loaded.Items.ShouldBe(new[] { "a", "b" });
+    }
+}