Project: springxd
Issue XD-2722: Partitioned job throws: java.lang.RuntimeException: Could not serialize lambda
Running a partitioned jdbchdfs job with 12 partitions and 3 xd-containers. Some steps fail with the jdbc connection pool exception XD-2720. I also sometimes see a serialization exception. This results in the partitioner never getting the status for some of the steps, so it keeps running until it times out even though all steps are either complete of failed. {code} 2015-02-13 13:18:36,294 1.1.0.RELEASE ERROR inbound.files4.0-redis:queue-inbound-channel-adapter1 redis.RedisMessageBus$1 - Failed to deliver message; retries exhausted; message sent to queue 'ERRORS:name' org.springframework.messaging.MessageHandlingException: error occurred in message handler [files4.0.bridge.handler]; nested exception is com.esotericsoftware.kryo.KryoException: java.lang.RuntimeException: Could not serialize lambda Serialization trace: stepExecutions (org.springframework.batch.core.JobExecution) jobExecution (org.springframework.batch.core.StepExecution) org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:84) org.springframework.integration.dispatcher.AbstractDispatcher.tryOptimizedDispatch(AbstractDispatcher.java:116) org.springframework.integration.dispatcher.UnicastingDispatcher.doDispatch(UnicastingDispatcher.java:101) org.springframework.integration.dispatcher.UnicastingDispatcher.dispatch(UnicastingDispatcher.java:97) org.springframework.integration.channel.AbstractSubscribableChannel.doSend(AbstractSubscribableChannel.java:77) org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:277) org.springframework.xd.dirt.integration.redis.RedisMessageBus$1$1.doWithRetry(RedisMessageBus.java:267) org.springframework.xd.dirt.integration.redis.RedisMessageBus$1$1.doWithRetry(RedisMessageBus.java:263) org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:263) org.springframework.retry.support.RetryTemplate.execute(RetryTemplate.java:168) org.springframework.xd.dirt.integration.redis.RedisMessageBus$1.doSend(RedisMessageBus.java:263) org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:277) org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:239) org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:115) org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:45) org.springframework.messaging.core.AbstractMessageSendingTemplate.send(AbstractMessageSendingTemplate.java:95) org.springframework.integration.endpoint.MessageProducerSupport.sendMessage(MessageProducerSupport.java:101) org.springframework.integration.redis.inbound.RedisQueueMessageDrivenEndpoint.popMessageAndSend(RedisQueueMessageDrivenEndpoint.java:220) org.springframework.integration.redis.inbound.RedisQueueMessageDrivenEndpoint.access$300(RedisQueueMessageDrivenEndpoint.java:50) org.springframework.integration.redis.inbound.RedisQueueMessageDrivenEndpoint$ListenerTask.run(RedisQueueMessageDrivenEndpoint.java:314) org.springframework.integration.util.ErrorHandlingTaskExecutor$1.run(ErrorHandlingTaskExecutor.java:52) java.lang.Thread.run(Thread.java:745) Caused by: com.esotericsoftware.kryo.KryoException: java.lang.RuntimeException: Could not serialize lambda Serialization trace: stepExecutions (org.springframework.batch.core.JobExecution) jobExecution (org.springframework.batch.core.StepExecution) com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125) com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:528) com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:704) com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106) com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:528) com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:682) org.springframework.xd.dirt.integration.bus.serializer.kryo.PojoCodec.doDeserialize(PojoCodec.java:41) org.springframework.xd.dirt.integration.bus.serializer.kryo.AbstractKryoMultiTypeCodec$1.execute(AbstractKryoMultiTypeCodec.java:63) com.esotericsoftware.kryo.pool.KryoPoolQueueImpl.run(KryoPoolQueueImpl.java:43) org.springframework.xd.dirt.integration.bus.serializer.kryo.AbstractKryoMultiTypeCodec.deserialize(AbstractKryoMultiTypeCodec.java:60) org.springframework.xd.dirt.integration.bus.serializer.kryo.PojoCodec.deserialize(PojoCodec.java:30) org.springframework.xd.dirt.integration.bus.serializer.CompositeCodec.deserialize(CompositeCodec.java:72) org.springframework.xd.dirt.integration.bus.serializer.CompositeCodec.deserialize(CompositeCodec.java:78) org.springframework.xd.dirt.integration.bus.MessageBusSupport.deserializePayload(MessageBusSupport.java:588) org.springframework.xd.dirt.integration.bus.MessageBusSupport.deserializePayload(MessageBusSupport.java:573) org.springframework.xd.dirt.integration.bus.MessageBusSupport.deserializePayloadIfNecessary(MessageBusSupport.java:556) org.springframework.xd.dirt.integration.redis.RedisMessageBus.access$1000(RedisMessageBus.java:68) org.springframework.xd.dirt.integration.redis.RedisMessageBus$ReceivingHandler.handleRequestMessage(RedisMessageBus.java:465) org.springframework.integration.handler.AbstractReplyProducingMessageHandler.handleMessageInternal(AbstractReplyProducingMessageHandler.java:99) org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:78)	... 21 more Caused by: java.lang.RuntimeException: Could not serialize lambda com.esotericsoftware.kryo.serializers.ClosureSerializer.read(ClosureSerializer.java:52) com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:786) com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:116) com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:22) com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:704) com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)	... 40 more Caused by: java.lang.ArrayIndexOutOfBoundsException: -2 java.util.ArrayList.elementData(ArrayList.java:418) java.util.ArrayList.get(ArrayList.java:431) com.esotericsoftware.kryo.util.MapReferenceResolver.getReadObject(MapReferenceResolver.java:42) com.esotericsoftware.kryo.Kryo.readReferenceOrNull(Kryo.java:830) com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:680) com.esotericsoftware.kryo.serializers.ClosureSerializer.read(ClosureSerializer.java:49)	... 45 more {code}
Story points: 5