Project: springxd
Issue XD-3613: Multiple module instances consuming from taps or topics get duplicate messages on redis Message Bus
If I deploy more than one instance of a module (eg using module.name.count > 1 or module.name.count =0) that consumes from a tap or topic then I get duplicate messages if I’m using Redis as the message bus. It looks like this is the same issue as XD-3100 but the fix for that only fixed Rabbit as the message bus. This is easy to reproduce on a 2 container cluster using a Redis Message Bus: Create and deploy streams as follows: {code} stream create --definition "http | log" --name httpLog stream deploy --name httpLog --properties "module.*.count=0" stream create --definition "tap:stream:httpLog > transform --expression='payload.toString() + \" TAPPED\"' | log" --name httpLogTap stream deploy --name httpLogTap --properties "module.*.count=0" {code} On container 1 send a message: {code} curl --data "test message 001" http://localhost:9000/httpLog {code} Container 1 logs are then: {code} 2015-10-13 14:16:28.853 INFO 22774 --- [ol-28-thread-18] xd.sink.httpLog : test message 001 2015-10-13 14:16:28.855 INFO 22774 --- [enerContainer-4] xd.sink.httpLogTap : test message 001 TAPPED {code} and container 2: {code} 2015-10-13 14:16:28.859 INFO 22719 --- [enerContainer-4] xd.sink.httpLogTap : test message 001 TAPPED {code} Ie the tapped message is duplicated (picked up by both tap module instances) Similarly for topics create and deploy these streams: {code} stream create --definition "http > topic:mytopic" --name httpTopic stream deploy --name httpTopic --properties "module.*.count=0" stream create --definition "topic:mytopic > transform --expression='payload.toString() + \" TOPIC CONSUMER 1\"' | log" --name topicConsumer1 stream deploy --name topicConsumer1 --properties "module.*.count=0" stream create --definition "topic:mytopic > transform --expression='payload.toString() + \" TOPIC CONSUMER 2\"' | log" --name topicConsumer2 stream deploy --name topicConsumer2 --properties "module.*.count=0" {code} On container 1 send a message: {code} curl --data "test message 002" http://localhost:9000/httpLog {code} Container 1 logs are then: {code} 2015-10-13 14:34:23.168 INFO 22774 --- [enerContainer-2] xd.sink.topicConsumer2 : test message 002 TOPIC CONSUMER 2 2015-10-13 14:34:23.172 INFO 22774 --- [enerContainer-2] xd.sink.topicConsumer1 : test message 002 TOPIC CONSUMER 1 {code} and container 2: {code} 2015-10-13 14:34:23.173 INFO 22719 --- [enerContainer-2] xd.sink.topicConsumer2 : test message 002 TOPIC CONSUMER 2 2015-10-13 14:34:23.177 INFO 22719 --- [enerContainer-2] xd.sink.topicConsumer1 : test message 002 TOPIC CONSUMER 1 {code} Ie the topic message is picked up by each instance of the module in each stream. In this case I would expect each stream to pick up the message once ie I would get a single output for each stream test message 002 TOPIC CONSUMER 2 once (on either container) test message 002 TOPIC CONSUMER 1 once (on either container)
Story points: 5