Project: springxd
Issue XD-347: Investigate Redis connection timeout issues when running performance test
With the performance test run, the numbers (messages sent/received per second) keep varying as there are "redis client connection timeout exceptions" (Caused by: org.jboss.netty.channel.ConnectTimeoutException: connection timed out) at both redis inbound/outbound channel adapters as I increase the total number of messages being processed (max. 10K/second). Some of the exception messages for the review: 1) With connection pool (at Redis outbound): Caused by: org.springframework.data.redis.connection.PoolException: Could not get a resource from the pool; nested exception is com.lambdaworks.redis.RedisException: Unable to connect at org.springframework.data.redis.connection.lettuce.DefaultLettucePool.getResource(DefaultLettucePool.java:95) at org.springframework.data.redis.connection.lettuce.DefaultLettucePool.getResource(DefaultLettucePool.java:36) at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory.createLettuceConnector(LettuceConnectionFactory.java:318) at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory.getConnection(LettuceConnectionFactory.java:109) at org.springframework.data.redis.core.RedisConnectionUtils.doGetConnection(RedisConnectionUtils.java:81) at org.springframework.data.redis.core.RedisConnectionUtils.getConnection(RedisConnectionUtils.java:53) at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:157) at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:137) at org.springframework.data.redis.core.AbstractOperations.execute(AbstractOperations.java:84) at org.springframework.data.redis.core.DefaultListOperations.leftPush(DefaultListOperations.java:71) at org.springframework.data.redis.core.DefaultBoundListOperations.leftPush(DefaultBoundListOperations.java:67) at org.springframework.xd.perftest.redis.outbound.RedisQOutboundChannelAdapter.handleMessageInternal(RedisQOutboundChannelAdapter.java:71) at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:73) ... 17 more Caused by: com.lambdaworks.redis.RedisException: Unable to connect at com.lambdaworks.redis.RedisClient.connect(RedisClient.java:176) at com.lambdaworks.redis.RedisClient.connectAsync(RedisClient.java:139) at org.springframework.data.redis.connection.lettuce.DefaultLettucePool$LettuceFactory.makeObject(DefaultLettucePool.java:252) at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:1181) at org.springframework.data.redis.connection.lettuce.DefaultLettucePool.getResource(DefaultLettucePool.java:93) ... 29 more Caused by: org.jboss.netty.channel.ConnectTimeoutException: connection timed out: localhost/127.0.0.1:6379 at org.jboss.netty.channel.socket.nio.NioClientBoss.processConnectTimeout(NioClientBoss.java:137) at org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:83) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312) at org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42) 2) Without connection pool (at Redis inbound): Caused by: com.lambdaworks.redis.RedisException: Unable to connect at com.lambdaworks.redis.RedisClient.connect(RedisClient.java:176) at com.lambdaworks.redis.RedisClient.connectAsync(RedisClient.java:139) at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory.createLettuceConnector(LettuceConnectionFactory.java:321) ... 12 more Caused by: org.jboss.netty.channel.ConnectTimeoutException: connection timed out: localhost/127.0.0.1:6379 at org.jboss.netty.channel.socket.nio.NioClientBoss.processConnectTimeout(NioClientBoss.java:137) at org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:83) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312) at org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42) ... 3 more
Story points: 2