Repository: dotnetcore/CAP

Issue #861: RabbitMQ Connection close forced
Created: 2021-05-09T03:13:33Z
Hi, After upgrading from 5.0.0 to 5.0.1 or 5.0.2, cap cannot connect to the RabbitMQ server and reports "Connection close forced" exception.
When downgrade to 5.0.0, the exception does not occur.
RabbitMQ 3.8.14 Erlang 23.3.1
log:
2021-05-09T02:56:33.151799434Z fail: DotNetCore.CAP.Internal.ConsumerRegister[0]
2021-05-09T02:56:33.151825096Z       The AMQP operation was interrupted: AMQP close-reason, initiated by Application, code=200, text='Connection close forced', classId=0, methodId=0
2021-05-09T02:56:33.151833874Z       RabbitMQ.Client.Exceptions.OperationInterruptedException: The AMQP operation was interrupted: AMQP close-reason, initiated by Application, code=200, text='Connection close forced', classId=0, methodId=0
2021-05-09T02:56:33.151843336Z          at RabbitMQ.Client.Impl.SimpleBlockingRpcContinuation.GetReply(TimeSpan timeout)
2021-05-09T02:56:33.151850829Z          at RabbitMQ.Client.Impl.ModelBase.ModelRpc(MethodBase method, ContentHeaderBase header, Byte[] body)
2021-05-09T02:56:33.151858402Z          at RabbitMQ.Client.Framing.Impl.Model._Private_QueueBind(String queue, String exchange, String routingKey, Boolean nowait, IDictionary`2 arguments)
2021-05-09T02:56:33.151866082Z          at DotNetCore.CAP.RabbitMQ.RabbitMQConsumerClient.Subscribe(IEnumerable`1 topics)
2021-05-09T02:56:33.151873324Z          at DotNetCore.CAP.Internal.ConsumerRegister.<>c__DisplayClass17_1.<Start>b__2()
2021-05-09T02:56:33.158342427Z warn: DotNetCore.CAP.RabbitMQ.ConnectionChannelPool[0]
2021-05-09T02:56:33.158368148Z       RabbitMQ client connection closed! --> Connection close forced
2021-05-09T02:56:33.164347969Z warn: DotNetCore.CAP.RabbitMQ.ConnectionChannelPool[0]
2021-05-09T02:56:33.164397986Z       RabbitMQ client connection closed! --> Connection close forced
2021-05-09T02:56:33.170505352Z warn: DotNetCore.CAP.RabbitMQ.ConnectionChannelPool[0]
2021-05-09T02:56:33.170526837Z       RabbitMQ client connection closed! --> Connection close forced
2021-05-09T02:56:33.173584918Z warn: DotNetCore.CAP.RabbitMQ.ConnectionChannelPool[0]
2021-05-09T02:56:33.173598361Z       RabbitMQ client connection closed! --> Connection close forced
2021-05-09T02:56:33.173611282Z warn: DotNetCore.CAP.Internal.ConsumerRegister[0]
2021-05-09T02:56:33.173616072Z       RabbitMQ consumer cancelled. --> amq.ctag-LHdQUCASJam_u8IiBY2T5w
2021-05-09T02:56:33.173620416Z warn: DotNetCore.CAP.Internal.ConsumerRegister[0]
2021-05-09T02:56:33.173624933Z       RabbitMQ consumer shutdown. --> Connection close forced
2021-05-09T02:56:33.173927533Z fail: DotNetCore.CAP.Internal.ConsumerRegister[0]
2021-05-09T02:56:33.173950761Z       The AMQP operation was interrupted: AMQP close-reason, initiated by Application, code=200, text='Connection close forced', classId=0, methodId=0
2021-05-09T02:56:33.173957839Z       RabbitMQ.Client.Exceptions.OperationInterruptedException: The AMQP operation was interrupted: AMQP close-reason, initiated by Application, code=200, text='Connection close forced', classId=0, methodId=0
2021-05-09T02:56:33.173964031Z          at RabbitMQ.Client.Impl.SimpleBlockingRpcContinuation.GetReply(TimeSpan timeout)
2021-05-09T02:56:33.173970886Z          at RabbitMQ.Client.Impl.ModelBase.ModelRpc(MethodBase method, ContentHeaderBase header, Byte[] body)
2021-05-09T02:56:33.173976659Z          at RabbitMQ.Client.Framing.Impl.Model._Private_QueueBind(String queue, String exchange, String routingKey, Boolean nowait, IDictionary`2 arguments)
2021-05-09T02:56:33.173982449Z          at DotNetCore.CAP.RabbitMQ.RabbitMQConsumerClient.Subscribe(IEnumerable`1 topics)
2021-05-09T02:56:33.173987975Z          at DotNetCore.CAP.Internal.ConsumerRegister.<>c__DisplayClass17_1.<Start>b__2()
2021-05-09T02:56:33.179440061Z warn: DotNetCore.CAP.RabbitMQ.ConnectionChannelPool[0]
2021-05-09T02:56:33.179467580Z       RabbitMQ client connection closed! --> Connection close forced
2021-05-09T02:56:33.185306684Z warn: DotNetCore.CAP.RabbitMQ.ConnectionChannelPool[0]
2021-05-09T02:56:33.185332453Z       RabbitMQ client connection closed! --> Connection close forced
2021-05-09T02:56:33.191531529Z warn: DotNetCore.CAP.RabbitMQ.ConnectionChannelPool[0]
2021-05-09T02:56:33.191562287Z       RabbitMQ client connection closed! --> Connection close forced

Duplicate issue #918: DotnetCAP > Subscribers sometimes do not consume and queue is accumulating.
Created: 2021-06-23T19:55:12Z
Hi @yang-xiaodong,
We sometimes face a likely edge-case problem. We have consumers that developed via dotnet core web API. They are served on the cloud with CI/CD pipeline.
We made a deployment a week ago and everything was fine. However, our consumers that listen to RabbitMQ started to stop consuming suddenly. We still publish the messages successfully. We also see the messages in the queues when monitoring the RabbitMQ Management Dashboard (e.g, message rates and queued messages).
The only solution for this problem is to rebuild and redeploy them. Then, they again consume the messages successfully. In addition, we analyzed the errors thrown by DotnetCAP, but we could not find the core problem exactly.

Do you have any idea about this problem? What do you think if we upgrade the DotnetCAP version to 5.1.0?
Our configurations
Consumers: Dotnetcore 5.0 WebAPI
DotnetCAP: version 5.0.3

Closing comment (2021-06-24T01:29:14Z): @omeerkorkmazz Hi, the problem is similar to #861.
how many connections are in the Connections tab of RabbitMQ Management Dashboard?
The figure below is the correct state, each node has only one connection, and each connection has multiple channels.