Knee-deep in Spring Boot, Transactional Event Listeners and CGLIB proxies
Tags: springboot, lessonslearned, developerstories, java
Published: 2021-05-07 15:40:42

A colleague and I spent two hours tracking down a strange bug in our Spring Boot application today. The cause was so interesting that I have to write about it here. Since I obviously can't write about customer projects here, I'm presenting the problem using a sample application instead. So here we go.

Let's say we have a domain-driven application with two aggregates: `Order` and `Invoice`. We also have an orchestrator that automatically should create new invoices once an order transitions into the `SHIPPED` state.

We start with the following application service for creating new orders:

```java
@Service
public class OrderService {

    private final OrderRepository orderRepository;

    public OrderService(OrderRepository orderRepository) {
        this.orderRepository = orderRepository;
    }

    @Transactional
    public Long createOrder() {
        var order = new Order();
        return orderRepository.saveAndFlush(order).getId();
    }

    @Transactional
    public void shipOrder(Long orderId) {
        orderRepository.findById(orderId).ifPresent(order -> {
            order.ship();
            orderRepository.saveAndFlush(order);
        });
    }
}
```

The `Order.ship()` method will change the state of the order to `SHIPPED` and publish an `OrderStateChangedEvent`.

Then we need another application service for creating new invoices:

```java
@Service
public class InvoiceService {
    private final InvoiceRepository invoiceRepository;

    InvoiceService(InvoiceRepository invoiceRepository) {
        this.invoiceRepository = invoiceRepository;
    }

    @Transactional
    Long createInvoiceForOrder(Order order) {
        var invoice = new Invoice(order);
        return invoiceRepository.saveAndFlush(invoice).getId();
    }
}
```

Finally, we need an orchestrator that creates the invoice when the order is shipped:

```java
@Component
class InvoiceCreationOrchestrator {

    private final OrderRepository orderRepository;
    private final InvoiceService invoiceService;

    InvoiceCreationOrchestrator(OrderRepository orderRepository, InvoiceService invoiceService) {
        this.orderRepository = orderRepository;
        this.invoiceService = invoiceService;
    }

    @TransactionalEventListener
    public void onOrderStateChangedEvent(OrderStateChangedEvent event) {
        if (event.getNewOrderState().equals(OrderState.SHIPPED)) {
            orderRepository
                .findById(event.getOrderId())
                .ifPresent(invoiceService::createInvoiceForOrder);
        }
    }
}
```

There! Now we just run the application, create a new order, ship it and... no invoice gets created. There are no exceptions in the log either. So what went wrong?

It turns out the problem is in the `@TransactionalEventListener`. It is by default configured to run after the transaction has been committed. This is exactly what we want, but there is a caveat in how Spring actually implements this.

Domain events are published using the ordinary application event publisher. Spring will actually catch them using an ordinary `@EventListener` as well. However, instead of invoking the transactional event listener directly, Spring will register a `TransactionSynchronization` with the `TransactionSynchronizationManager`. This will invoke the transactional event listener after the transaction has successfully committed, but before the transaction synchronization manager has cleaned itself up.

Now, our event listener is invoking the `createInvoiceForOrder` method, which has the `@Transactional` annotation. The default propagation for `@Transactional` is `REQUIRED`. This means that if there already is an active transaction, the method should participate in it; otherwise it should create its own transaction. 

Because this method is being invoked inside a `TransactionSynchronization`, there actually is an "active" transaction but it has already been committed. Thus, the call to `saveAndFlush` will result in a `TransactionRequiredException`. This exception is swallowed by `TransactionSynchronizationUtils` (another Spring class) and logged using the DEBUG level. Thus, the only way to detect this exception is by having DEBUG logging turned on for the `org.springframework.transaction.support` package.

The solution to this problem is to make sure that the `InvoiceService` always runs inside its own transaction. So we change the method like this:

```java
@Service
public class InvoiceService {
    @Transactional(propagation = REQUIRES_NEW)
    Long createInvoiceForOrder(Order order) {
      // Rest of the method omitted
    }
}
```

It is anyway a good practice to configure all your application services to *always* use `REQUIRES_NEW` since they are responsible for [controlling the transactions](https://dev.to/peholmst/domain-driven-design-and-the-hexagonal-architecture-2o87).

Now we run the application again and... it still does not work. The application behaves exactly the same. What's wrong now?

It turns out that `createInvoiceForOrder` is not actually running inside a transaction at all. The transaction is started and committed by the call to `saveAndFlush()` in the repository, and that method still uses `REQUIRED` transaction propagation. How come?

The `InvoiceService` is not implementing any interfaces, so Spring is using a CGLIB proxy to add the transaction interceptors. However, the method `createInvoiceForOrder` happens to have package visibility and the transaction interceptor is only applied to public methods. So we need to change the method to be public:

```java
@Service
public class InvoiceService {
    @Transactional(propagation = REQUIRES_NEW)
    public Long createInvoiceForOrder(Order order) {
      // Rest of the method omitted
    }
}
```

Now we run the application once more and it finally works!