Repository: gruelbox/transaction-outbox
Pull request #851: Increase robustness of outbox flushing with respect to outbox entry deserialization issues
Author: login: skastenholz
Created: 2025-04-30T05:53:55Z | Merged: 2025-05-16T17:40:11Z

Background:
- We are utilizing this framework in all our microservices for certain requests to external and internal downstream services.
- In doing so JacksonInvocationSerializer is used to serialize/deserialize invocations (in a most flexible way).

Problem:
- On rare occassions some outbox entries that were serialized cannot be deserialized successfully.
- For instance this happens when the id of multiple attributes are set to null. However, the concrete issue is not relevant in this context. Most importantly the the call of "deserializeInvocation" at JacksonInvocationSerializer (https://github.com/gruelbox/transaction-outbox/blob/master/transactionoutbox-jackson/src/main/java/com/gruelbox/transactionoutbox/jackson/JacksonInvocationSerializer.java#L46) fails with an IOException.

Analysis:
- As a result the IOException is wrapped in a RuntimeException and is populated to
  - https://github.com/gruelbox/transaction-outbox/blob/master/transactionoutbox-core/src/main/java/com/gruelbox/transactionoutbox/TransactionOutboxImpl.java#L129C53-L129C64
  - https://github.com/gruelbox/transaction-outbox/blob/master/transactionoutbox-core/src/main/java/com/gruelbox/transactionoutbox/TransactionOutboxImpl.java#L142
- Ultimately, the processing of outbox entries stops entirely - even in case the problems applies to a single outbox entry only as part of the batch. As a result no outbox entries are processed.
- In addition the same happens when the outbox is flushed again at a later point in time. 
- Finally, all outbox entries of the microservice are no longer processed.
- A similar observation was made in #628.

Solution:
- The approach suggested here is to handle the IOException at a later point in time at the DefaultPersistor.
- During deserialization of all relevant outbox entries the ones on which the problems applied are ignored (instead of failing the whole operation).
- In doing so deserializeable entries are processed and the processing of remaining outbox entries is not  blocked.

Open points:
- [ ] Please review the solution and provide your feedback.
- [ ] With respect to tests I would plan to extend AbstractPersistorTest, but am struggling since the InvocationSerializer is not exposed yet. Any hints would be welcome.




Diff:
diff --git a/transactionoutbox-core/src/main/java/com/gruelbox/transactionoutbox/DefaultInvocationSerializer.java b/transactionoutbox-core/src/main/java/com/gruelbox/transactionoutbox/DefaultInvocationSerializer.java
index 543af4b1..004e780d 100644
--- a/transactionoutbox-core/src/main/java/com/gruelbox/transactionoutbox/DefaultInvocationSerializer.java
+++ b/transactionoutbox-core/src/main/java/com/gruelbox/transactionoutbox/DefaultInvocationSerializer.java
@@ -1,16 +1,6 @@
 package com.gruelbox.transactionoutbox;
 
-import com.google.gson.Gson;
-import com.google.gson.GsonBuilder;
-import com.google.gson.JsonArray;
-import com.google.gson.JsonDeserializationContext;
-import com.google.gson.JsonDeserializer;
-import com.google.gson.JsonElement;
-import com.google.gson.JsonObject;
-import com.google.gson.JsonParseException;
-import com.google.gson.JsonSerializationContext;
-import com.google.gson.JsonSerializer;
-import com.google.gson.TypeAdapter;
+import com.google.gson.*;
 import com.google.gson.stream.JsonReader;
 import com.google.gson.stream.JsonToken;
 import com.google.gson.stream.JsonWriter;
@@ -106,8 +96,12 @@ public void serializeInvocation(Invocation invocation, Writer writer) {
   }
 
   @Override
-  public Invocation deserializeInvocation(Reader reader) {
-    return gson.fromJson(reader, Invocation.class);
+  public Invocation deserializeInvocation(Reader reader) throws IOException {
+    try {
+      return gson.fromJson(reader, Invocation.class);
+    } catch (JsonIOException | JsonSyntaxException exception) {
+      throw new IOException(exception);
+    }
   }
 
   private static final class InvocationJsonSerializer
diff --git a/transactionoutbox-core/src/main/java/com/gruelbox/transactionoutbox/DefaultPersistor.java b/transactionoutbox-core/src/main/java/com/gruelbox/transactionoutbox/DefaultPersistor.java
index 52a00f93..a12ffb9f 100644
--- a/transactionoutbox-core/src/main/java/com/gruelbox/transactionoutbox/DefaultPersistor.java
+++ b/transactionoutbox-core/src/main/java/com/gruelbox/transactionoutbox/DefaultPersistor.java
@@ -371,14 +371,20 @@ private TransactionOutboxEntry map(ResultSet rs) throws SQLException, IOExceptio
     if (rs.wasNull()) {
       sequence = null;
     }
+    // Reading invocationStream *must* occur first because some drivers (ex. SQL Server)
+    // implement true streams that are not buffered in memory. Calling any other getter
+    // on ResultSet before invocationStream is read will cause Reader to be closed
+    // prematurely.
     try (Reader invocationStream = rs.getCharacterStream("invocation")) {
+      Invocation invocation;
+      try {
+        invocation = serializer.deserializeInvocation(invocationStream);
+      } catch (IOException e) {
+        invocation = new FailedDeserializingInvocation(e);
+      }
       TransactionOutboxEntry entry =
           TransactionOutboxEntry.builder()
-              // Reading invocationStream *must* occur first because some drivers (ex. SQL Server)
-              // implement true streams that are not buffered in memory. Calling any other getter
-              // on ResultSet before invocationStream is read will cause Reader to be closed
-              // prematurely.
-              .invocation(serializer.deserializeInvocation(invocationStream))
+              .invocation(invocation)
               .id(rs.getString("id"))
               .uniqueRequestId(rs.getString("uniqueRequestId"))
               .topic("*".equals(topic) ? null : topic)
diff --git a/transactionoutbox-core/src/main/java/com/gruelbox/transactionoutbox/FailedDeserializingInvocation.java b/transactionoutbox-core/src/main/java/com/gruelbox/transactionoutbox/FailedDeserializingInvocation.java
new file mode 100644
index 00000000..b11e84d0
--- /dev/null
+++ b/transactionoutbox-core/src/main/java/com/gruelbox/transactionoutbox/FailedDeserializingInvocation.java
@@ -0,0 +1,24 @@
+package com.gruelbox.transactionoutbox;
+
+import java.io.IOException;
+import java.lang.reflect.InvocationTargetException;
+
+/** Represents an invocation those deserialization failed. */
+public class FailedDeserializingInvocation extends Invocation {
+
+  /**
+   * @return Indicates an Exception during De-Serialization, that is not persisted.
+   */
+  private final transient IOException exceptionDuringDeserialization;
+
+  public FailedDeserializingInvocation(IOException exceptionDuringDeserialization) {
+    super("", "", new Class<?>[] {}, new Object[] {});
+    this.exceptionDuringDeserialization = exceptionDuringDeserialization;
+  }
+
+  @Override
+  void invoke(Object instance, TransactionOutboxListener listener)
+      throws NoSuchMethodException, IllegalAccessException, InvocationTargetException {
+    throw new UncheckedException(exceptionDuringDeserialization);
+  }
+}
diff --git a/transactionoutbox-core/src/main/java/com/gruelbox/transactionoutbox/Invocation.java b/transactionoutbox-core/src/main/java/com/gruelbox/transactionoutbox/Invocation.java
index 3cc815d8..c620227c 100644
--- a/transactionoutbox-core/src/main/java/com/gruelbox/transactionoutbox/Invocation.java
+++ b/transactionoutbox-core/src/main/java/com/gruelbox/transactionoutbox/Invocation.java
@@ -6,7 +6,8 @@
 import java.util.Arrays;
 import java.util.Map;
 import java.util.concurrent.Callable;
-import lombok.Value;
+import lombok.*;
+import lombok.experimental.FieldDefaults;
 import lombok.extern.slf4j.Slf4j;
 import org.slf4j.MDC;
 
@@ -17,7 +18,10 @@
  * <p>Optimized for safe serialization via GSON.
  */
 @SuppressWarnings("WeakerAccess")
-@Value
+@ToString
+@EqualsAndHashCode
+@FieldDefaults(makeFinal = true, level = AccessLevel.PRIVATE)
+@Getter
 @Slf4j
 public class Invocation {
 
@@ -129,6 +133,7 @@ <T> T withinMDC(Callable<T> callable) throws Exception {
 
   void invoke(Object instance, TransactionOutboxListener listener)
       throws NoSuchMethodException, IllegalAccessException, InvocationTargetException {
+
     Method method = instance.getClass().getDeclaredMethod(methodName, parameterTypes);
     method.setAccessible(true);
     if (log.isDebugEnabled()) {
diff --git a/transactionoutbox-core/src/main/java/com/gruelbox/transactionoutbox/InvocationSerializer.java b/transactionoutbox-core/src/main/java/com/gruelbox/transactionoutbox/InvocationSerializer.java
index e7bb43a8..20af181f 100644
--- a/transactionoutbox-core/src/main/java/com/gruelbox/transactionoutbox/InvocationSerializer.java
+++ b/transactionoutbox-core/src/main/java/com/gruelbox/transactionoutbox/InvocationSerializer.java
@@ -1,5 +1,6 @@
 package com.gruelbox.transactionoutbox;
 
+import java.io.IOException;
 import java.io.Reader;
 import java.io.Writer;
 
@@ -40,5 +41,5 @@ static InvocationSerializer createDefaultJsonSerializer() {
    * @param reader The reader.
    * @return The deserialized invocation.
    */
-  Invocation deserializeInvocation(Reader reader);
+  Invocation deserializeInvocation(Reader reader) throws IOException;
 }
diff --git a/transactionoutbox-core/src/test/java/com/gruelbox/transactionoutbox/AbstractTestDefaultInvocationSerializer.java b/transactionoutbox-core/src/test/java/com/gruelbox/transactionoutbox/AbstractTestDefaultInvocationSerializer.java
index 5dc13fa0..3ef2f10b 100644
--- a/transactionoutbox-core/src/test/java/com/gruelbox/transactionoutbox/AbstractTestDefaultInvocationSerializer.java
+++ b/transactionoutbox-core/src/test/java/com/gruelbox/transactionoutbox/AbstractTestDefaultInvocationSerializer.java
@@ -1,7 +1,11 @@
 package com.gruelbox.transactionoutbox;
 
+import static org.junit.jupiter.api.Assertions.assertThrows;
+
+import java.io.IOException;
 import java.io.StringReader;
 import java.io.StringWriter;
+import java.io.UncheckedIOException;
 import java.time.*;
 import java.time.temporal.ChronoUnit;
 import java.util.*;
@@ -206,6 +210,12 @@ void testUUIDNull() {
     check(new Invocation(CLASS_NAME, METHOD_NAME, primitives, values));
   }
 
+  @Test
+  void testDeserializationException() {
+    assertThrows(
+        IOException.class, () -> serializer.deserializeInvocation(new StringReader("unparseable")));
+  }
+
   void check(Invocation invocation) {
     Invocation deserialized = serdeser(invocation);
     Assertions.assertEquals(deserialized, serdeser(invocation));
@@ -213,10 +223,14 @@ void check(Invocation invocation) {
   }
 
   Invocation serdeser(Invocation invocation) {
-    var writer = new StringWriter();
-    serializer.serializeInvocation(invocation, writer);
-    log.info("Serialised as: {}", writer);
-    return serializer.deserializeInvocation(new StringReader(writer.toString()));
+    try {
+      var writer = new StringWriter();
+      serializer.serializeInvocation(invocation, writer);
+      log.info("Serialised as: {}", writer);
+      return serializer.deserializeInvocation(new StringReader(writer.toString()));
+    } catch (IOException e) {
+      throw new UncheckedIOException(e);
+    }
   }
 
   enum ExampleCustomEnum {
diff --git a/transactionoutbox-jackson/src/main/java/com/gruelbox/transactionoutbox/jackson/CustomInvocationDeserializer.java b/transactionoutbox-jackson/src/main/java/com/gruelbox/transactionoutbox/jackson/CustomInvocationDeserializer.java
index 12bec212..79115373 100644
--- a/transactionoutbox-jackson/src/main/java/com/gruelbox/transactionoutbox/jackson/CustomInvocationDeserializer.java
+++ b/transactionoutbox-jackson/src/main/java/com/gruelbox/transactionoutbox/jackson/CustomInvocationDeserializer.java
@@ -5,6 +5,7 @@
 import com.fasterxml.jackson.databind.DeserializationContext;
 import com.fasterxml.jackson.databind.JsonNode;
 import com.fasterxml.jackson.databind.deser.std.StdDeserializer;
+import com.fasterxml.jackson.databind.jsontype.TypeDeserializer;
 import com.fasterxml.jackson.databind.node.ArrayNode;
 import com.gruelbox.transactionoutbox.Invocation;
 import java.io.IOException;
@@ -31,6 +32,13 @@ protected CustomInvocationDeserializer(Class<?> vc) {
     this(Invocation.class);
   }
 
+  @Override
+  public Invocation deserializeWithType(
+      JsonParser p, DeserializationContext ctxt, TypeDeserializer typeDeserializer)
+      throws IOException {
+    return deserialize(p, ctxt);
+  }
+
   @Override
   public Invocation deserialize(JsonParser p, DeserializationContext ctxt) throws IOException {
     JsonNode node = p.getCodec().readTree(p);
diff --git a/transactionoutbox-jackson/src/main/java/com/gruelbox/transactionoutbox/jackson/CustomInvocationSerializer.java b/transactionoutbox-jackson/src/main/java/com/gruelbox/transactionoutbox/jackson/CustomInvocationSerializer.java
index 217a8c14..01cd0daa 100644
--- a/transactionoutbox-jackson/src/main/java/com/gruelbox/transactionoutbox/jackson/CustomInvocationSerializer.java
+++ b/transactionoutbox-jackson/src/main/java/com/gruelbox/transactionoutbox/jackson/CustomInvocationSerializer.java
@@ -2,6 +2,7 @@
 
 import com.fasterxml.jackson.core.JsonGenerator;
 import com.fasterxml.jackson.databind.SerializerProvider;
+import com.fasterxml.jackson.databind.jsontype.TypeSerializer;
 import com.fasterxml.jackson.databind.ser.std.StdSerializer;
 import com.gruelbox.transactionoutbox.Invocation;
 import java.io.IOException;
@@ -16,6 +17,13 @@ protected CustomInvocationSerializer(Class<Invocation> t) {
     super(t);
   }
 
+  @Override
+  public void serializeWithType(
+      Invocation value, JsonGenerator gen, SerializerProvider serializers, TypeSerializer typeSer)
+      throws IOException {
+    serialize(value, gen, serializers);
+  }
+
   @Override
   public void serialize(Invocation value, JsonGenerator gen, SerializerProvider provider)
       throws IOException {
diff --git a/transactionoutbox-jackson/src/main/java/com/gruelbox/transactionoutbox/jackson/JacksonInvocationSerializer.java b/transactionoutbox-jackson/src/main/java/com/gruelbox/transactionoutbox/jackson/JacksonInvocationSerializer.java
index 23d08612..9f000ff2 100644
--- a/transactionoutbox-jackson/src/main/java/com/gruelbox/transactionoutbox/jackson/JacksonInvocationSerializer.java
+++ b/transactionoutbox-jackson/src/main/java/com/gruelbox/transactionoutbox/jackson/JacksonInvocationSerializer.java
@@ -43,24 +43,20 @@ public void serializeInvocation(Invocation invocation, Writer writer) {
   }
 
   @Override
-  public Invocation deserializeInvocation(Reader reader) {
-    try {
-      // read ahead to check if old style
-      BufferedReader br = new BufferedReader(reader);
-      if (checkForOldSerialization(br)) {
-        if (defaultInvocationSerializer == null) {
-          throw new UnsupportedOperationException(
-              "Can't deserialize GSON-format tasks without a "
-                  + DefaultInvocationSerializer.class.getSimpleName()
-                  + ". Supply one when building "
-                  + getClass().getSimpleName());
-        }
-        return defaultInvocationSerializer.deserializeInvocation(br);
+  public Invocation deserializeInvocation(Reader reader) throws IOException {
+    // read ahead to check if old style
+    BufferedReader br = new BufferedReader(reader);
+    if (checkForOldSerialization(br)) {
+      if (defaultInvocationSerializer == null) {
+        throw new UnsupportedOperationException(
+            "Can't deserialize GSON-format tasks without a "
+                + DefaultInvocationSerializer.class.getSimpleName()
+                + ". Supply one when building "
+                + getClass().getSimpleName());
       }
-      return mapper.readValue(br, Invocation.class);
-    } catch (IOException e) {
-      throw new RuntimeException(e);
+      return defaultInvocationSerializer.deserializeInvocation(br);
     }
+    return mapper.readValue(br, Invocation.class);
   }
 
   private boolean checkForOldSerialization(BufferedReader reader) throws IOException {
diff --git a/transactionoutbox-jackson/src/test/java/com/gruelbox/transactionoutbox/jackson/TestJacksonInvocationSerializer.java b/transactionoutbox-jackson/src/test/java/com/gruelbox/transactionoutbox/jackson/TestJacksonInvocationSerializer.java
index b9aa01fe..0ad39a0a 100644
--- a/transactionoutbox-jackson/src/test/java/com/gruelbox/transactionoutbox/jackson/TestJacksonInvocationSerializer.java
+++ b/transactionoutbox-jackson/src/test/java/com/gruelbox/transactionoutbox/jackson/TestJacksonInvocationSerializer.java
@@ -9,8 +9,10 @@
 import com.fasterxml.jackson.datatype.jsr310.JavaTimeModule;
 import com.gruelbox.transactionoutbox.DefaultInvocationSerializer;
 import com.gruelbox.transactionoutbox.Invocation;
+import java.io.IOException;
 import java.io.StringReader;
 import java.io.StringWriter;
+import java.io.UncheckedIOException;
 import java.time.*;
 import java.time.temporal.ChronoUnit;
 import java.util.*;
@@ -50,9 +52,13 @@ void check(Invocation invocation) {
   }
 
   Invocation serdeser(Invocation invocation) {
-    var writer = new StringWriter();
-    underTest.serializeInvocation(invocation, writer);
-    return underTest.deserializeInvocation(new StringReader(writer.toString()));
+    try {
+      var writer = new StringWriter();
+      underTest.serializeInvocation(invocation, writer);
+      return underTest.deserializeInvocation(new StringReader(writer.toString()));
+    } catch (IOException e) {
+      throw new UncheckedIOException(e);
+    }
   }
 
   @Test
@@ -148,7 +154,7 @@ void testJavaTimeClasses() {
   }
 
   @Test
-  void deserializes_old_representation_correctly() {
+  void deserializes_old_representation_correctly() throws IOException {
     StringReader reader =
         new StringReader(
             "{\"c\":\"com.gruelbox.transactionoutbox.jackson.Service\",\"m\":\"parseDate\",\"p\":[\"String\"],\"a\":[{\"t\":\"String\",\"v\":\"2021-05-11\"}],\"x\":{\"REQUEST-ID\":\"someRequestId\"}}");
diff --git a/transactionoutbox-spring/src/test/java/com/gruelbox/transactionoutbox/spring/example/EventuallyConsistentControllerTest.java b/transactionoutbox-spring/src/test/java/com/gruelbox/transactionoutbox/spring/example/EventuallyConsistentControllerTest.java
index e7d77284..6c5cbacb 100644
--- a/transactionoutbox-spring/src/test/java/com/gruelbox/transactionoutbox/spring/example/EventuallyConsistentControllerTest.java
+++ b/transactionoutbox-spring/src/test/java/com/gruelbox/transactionoutbox/spring/example/EventuallyConsistentControllerTest.java
@@ -12,6 +12,7 @@
 import org.springframework.boot.test.context.SpringBootTest;
 import org.springframework.boot.test.web.client.TestRestTemplate;
 import org.springframework.boot.test.web.server.LocalServerPort;
+import org.springframework.jdbc.core.JdbcTemplate;
 
 @SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
 class EventuallyConsistentControllerTest {
@@ -30,6 +31,8 @@ class EventuallyConsistentControllerTest {
   @Autowired
   private ExternalQueueService externalQueueService;
 
+  @Autowired private JdbcTemplate jdbcTemplate;
+
   @BeforeEach
   void setUp() throws Exception {
     this.base = new URL("http://localhost:" + port + "/");
@@ -52,13 +55,18 @@ void testCheckNormal() {
     assertTrue(template.postForEntity(url, tupac, Void.class).getStatusCode().is2xxSuccessful());
     assertTrue(template.postForEntity(url, jeff, Void.class).getStatusCode().is2xxSuccessful());
 
+    jdbcTemplate.execute(
+        "UPDATE txno_outbox SET invocation='non-deserializable invocation' WHERE invocation LIKE '%"
+            + neil.getLastName()
+            + "%'");
+
     await()
         .atMost(10, SECONDS)
         .pollDelay(1, SECONDS)
         .untilAsserted(
             () ->
                 assertThat(externalQueueService.getSent())
-                    .containsExactlyInAnyOrder(joe, dave, neil, tupac, jeff));
+                    .containsExactlyInAnyOrder(joe, dave, tupac, jeff));
   }
 
   @Test
@@ -77,12 +85,14 @@ void testCheckOrdered() {
     assertTrue(template.postForEntity(url, tupac, Void.class).getStatusCode().is2xxSuccessful());
     assertTrue(template.postForEntity(url, jeff, Void.class).getStatusCode().is2xxSuccessful());
 
+    jdbcTemplate.execute(
+        "UPDATE txno_outbox SET invocation='non-deserializable invocation' WHERE invocation LIKE '%"
+            + neil.getLastName()
+            + "%'");
+
     await()
         .atMost(10, SECONDS)
         .pollDelay(1, SECONDS)
-        .untilAsserted(
-            () ->
-                assertThat(externalQueueService.getSent())
-                    .containsExactly(joe, dave, neil, tupac, jeff));
+        .untilAsserted(() -> assertThat(externalQueueService.getSent()).containsExactly(joe, dave));
   }
 }
diff --git a/transactionoutbox-testing/src/main/java/com/gruelbox/transactionoutbox/testing/AbstractPersistorTest.java b/transactionoutbox-testing/src/main/java/com/gruelbox/transactionoutbox/testing/AbstractPersistorTest.java
index 65300037..01a78626 100644
--- a/transactionoutbox-testing/src/main/java/com/gruelbox/transactionoutbox/testing/AbstractPersistorTest.java
+++ b/transactionoutbox-testing/src/main/java/com/gruelbox/transactionoutbox/testing/AbstractPersistorTest.java
@@ -6,6 +6,7 @@
 import static org.junit.jupiter.api.Assertions.*;
 
 import com.gruelbox.transactionoutbox.*;
+import java.io.IOException;
 import java.math.BigDecimal;
 import java.time.Instant;
 import java.time.temporal.ChronoUnit;
@@ -166,6 +167,21 @@ public void testBlockedEntriesExcluded() throws Exception {
             tx -> assertThat(persistor().selectBatch(tx, 3, now.plusMillis(1)), hasSize(2)));
   }
 
+  @Test
+  public void testUnparseableEntriesExcluded() throws Exception {
+
+    txManager()
+        .inTransactionThrows(
+            tx -> {
+              persistor().save(tx, createEntry("FOO1", now, false));
+              persistor().save(tx, createEntry("FOO2", now, createUnparseableInvocation()));
+              persistor().save(tx, createEntry("FOO3", now, false));
+            });
+    txManager()
+        .inTransactionThrows(
+            tx -> assertThat(persistor().selectBatch(tx, 3, now.plusMillis(1)), hasSize(3)));
+  }
+
   static class TransactionOutboxEntryMatcher extends TypeSafeMatcher<TransactionOutboxEntry> {
     private final TransactionOutboxEntry entry;
 
@@ -417,6 +433,17 @@ private TransactionOutboxEntry createEntry(
         .build();
   }
 
+  private TransactionOutboxEntry createEntry(
+      String id, Instant nextAttemptTime, Invocation invocation) {
+    return TransactionOutboxEntry.builder()
+        .id(id)
+        .invocation(invocation)
+        .blocked(false)
+        .lastAttemptTime(null)
+        .nextAttemptTime(nextAttemptTime.truncatedTo(MILLIS))
+        .build();
+  }
+
   private Invocation createInvocation() {
     return new Invocation(
         "Foo",
@@ -425,6 +452,10 @@ private Invocation createInvocation() {
         new Object[] {1, BigDecimal.TEN, null});
   }
 
+  private Invocation createUnparseableInvocation() {
+    return new FailedDeserializingInvocation(new IOException());
+  }
+
   private void expectTobeInterrupted() {
     try {
       Thread.sleep(10000);


Reviews:
user: login: badgerwithagun
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-15T12:09:20Z
user: login: badgerwithagun
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-15T12:10:36Z
user: login: badgerwithagun
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-15T12:12:02Z
user: login: badgerwithagun
body: This is a really nice approach. I like how neatly it works when you push the exception down into the invocation. Great stuff.

Just a few tweaks on the implementation, but the approach and testing are sound.
state: CHANGES_REQUESTED
author_association: MEMBER
submitted_at: 2025-05-15T12:14:42Z
user: login: skastenholz
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-15T18:47:26Z
user: login: skastenholz
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-15T18:52:12Z
user: login: skastenholz
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-15T19:20:15Z
user: login: badgerwithagun
state: APPROVED
author_association: MEMBER
submitted_at: 2025-05-16T17:39:43Z

Review comments:
diff_hunk: @@ -89,6 +96,19 @@ public Invocation(
     this.parameterTypes = parameterTypes;
     this.args = args;
     this.mdc = mdc;
+    this.exceptionDuringDeserialization = Optional.empty();
+  }
+
+  /**
+   * @param exception The exception that occured during de-serialization.
+   */
+  public Invocation(IOException exception) {
+    this.className = "";
+    this.methodName = "";
+    this.parameterTypes = new Class<?>[] {};
+    this.args = new Object[] {};
+    this.mdc = null;
+    this.exceptionDuringDeserialization = Optional.of(exception);
path: transactionoutbox-core/src/main/java/com/gruelbox/transactionoutbox/Invocation.java
user: login: badgerwithagun
body: Rather than adding a new property here (which will impact on ser/deser of all `Invocation`s, not just this one), why not create a `FailedDeserializingInvocation`, something like this:

```
class FailedDeserializingInvocation extends Invocation {

  private final IOException e;

  FailedDeserializingInvocation(IOException e) {
    this.className = "";
    this.methodName = "";
    this.parameterTypes = new Class<?>[] {};
    this.args = new Object[] {};
    this.mdc = null;
    this.e= e;
  }

  @Override
  void invoke(Object instance, TransactionOutboxListener listener)
      throws NoSuchMethodException, IllegalAccessException, InvocationTargetException {
     throw new WrappedCheckedException(e);
  }

}
```

Should make the code elsewhere cleaner too.

[You will need to replace `@Value`](https://projectlombok.org/features/Value) on `Invocation` to make it nonfinal so this can be done:

```
@ToString
@EqualsAndHashCode
@AllArgsConstructor
@FieldDefaults(makeFinal = true, level = AccessLevel.PRIVATE)
@Getter
```
created_at: 2025-05-15T12:09:20Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,8 @@
+package com.gruelbox.transactionoutbox;
+
+/** Thrown when de-serialization of an invocation fails. */
+class DeserializationFailedException extends RuntimeException {
path: transactionoutbox-core/src/main/java/com/gruelbox/transactionoutbox/DeserializationFailedException.java
user: login: badgerwithagun
body: I think there's a standard exception somewhere in the core lib for generically wrapping checked exceptions, which is all that's going on here.
created_at: 2025-05-15T12:10:36Z
author_association: MEMBER
diff_hunk: @@ -43,24 +43,20 @@ public void serializeInvocation(Invocation invocation, Writer writer) {
   }
 
   @Override
-  public Invocation deserializeInvocation(Reader reader) {
-    try {
-      // read ahead to check if old style
-      BufferedReader br = new BufferedReader(reader);
-      if (checkForOldSerialization(br)) {
-        if (defaultInvocationSerializer == null) {
-          throw new UnsupportedOperationException(
-              "Can't deserialize GSON-format tasks without a "
-                  + DefaultInvocationSerializer.class.getSimpleName()
-                  + ". Supply one when building "
-                  + getClass().getSimpleName());
-        }
-        return defaultInvocationSerializer.deserializeInvocation(br);
+  public Invocation deserializeInvocation(Reader reader) throws IOException {
path: transactionoutbox-jackson/src/main/java/com/gruelbox/transactionoutbox/jackson/JacksonInvocationSerializer.java
user: login: badgerwithagun
body: Does this same problem affect `DefaultInvocationSerializer`?
created_at: 2025-05-15T12:12:02Z
author_association: MEMBER
diff_hunk: @@ -89,6 +96,19 @@ public Invocation(
     this.parameterTypes = parameterTypes;
     this.args = args;
     this.mdc = mdc;
+    this.exceptionDuringDeserialization = Optional.empty();
+  }
+
+  /**
+   * @param exception The exception that occured during de-serialization.
+   */
+  public Invocation(IOException exception) {
+    this.className = "";
+    this.methodName = "";
+    this.parameterTypes = new Class<?>[] {};
+    this.args = new Object[] {};
+    this.mdc = null;
+    this.exceptionDuringDeserialization = Optional.of(exception);
path: transactionoutbox-core/src/main/java/com/gruelbox/transactionoutbox/Invocation.java
user: login: skastenholz
body: Yes, makes sense. Done.
created_at: 2025-05-15T18:47:26Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,8 @@
+package com.gruelbox.transactionoutbox;
+
+/** Thrown when de-serialization of an invocation fails. */
+class DeserializationFailedException extends RuntimeException {
path: transactionoutbox-core/src/main/java/com/gruelbox/transactionoutbox/DeserializationFailedException.java
user: login: skastenholz
body: Thanks for the hint. Done.
created_at: 2025-05-15T18:52:12Z
author_association: CONTRIBUTOR
diff_hunk: @@ -43,24 +43,20 @@ public void serializeInvocation(Invocation invocation, Writer writer) {
   }
 
   @Override
-  public Invocation deserializeInvocation(Reader reader) {
-    try {
-      // read ahead to check if old style
-      BufferedReader br = new BufferedReader(reader);
-      if (checkForOldSerialization(br)) {
-        if (defaultInvocationSerializer == null) {
-          throw new UnsupportedOperationException(
-              "Can't deserialize GSON-format tasks without a "
-                  + DefaultInvocationSerializer.class.getSimpleName()
-                  + ". Supply one when building "
-                  + getClass().getSimpleName());
-        }
-        return defaultInvocationSerializer.deserializeInvocation(br);
+  public Invocation deserializeInvocation(Reader reader) throws IOException {
path: transactionoutbox-jackson/src/main/java/com/gruelbox/transactionoutbox/jackson/JacksonInvocationSerializer.java
user: login: skastenholz
body: According to JavaDoc it could so I applied the same pattern.
created_at: 2025-05-15T19:20:15Z
author_association: CONTRIBUTOR

PR comments:
user: login: badgerwithagun
created_at: 2025-05-08T14:36:02Z
body: This looks like a sensible first step.  I think, however, that we should follow normal error behaviour for anything that fails to deserialise:

* push back the `nextAttemptTime`
* increment the `attemptCount` (if the record has no `group`, i.e. is unordered)
* Fire the appropriate `TransactionOutboxListener` events if appropriate (with a synthetic`TransactionOutboxEntry` constructed from the normal fields on the record but with a dummy invocation)

This will avoid the same record getting re-read continously, allow blocking to progress as normal, and ensure that anyone who is using `TransactionOutboxListener` to monitor things to detect that there is a problem.


author_association: MEMBER
user: login: badgerwithagun
created_at: 2025-05-08T14:38:03Z
body: We're also going to need testts for these behaviours!
author_association: MEMBER
user: login: skastenholz
created_at: 2025-05-14T06:24:23Z
body: Alright, good point. I tried different approaches and came up with the one comitted. In doing so I needed to modify core classes and, in particular, you may have suggestions regarding the Invocation class. Also I added/modified some tests.

Please have a look and provide your feedback. Thanks.
author_association: CONTRIBUTOR
user: login: badgerwithagun
created_at: 2025-05-16T17:40:19Z
body: Merged! Thanks :) 
author_association: MEMBER