Project: datamanagement
Issue DM-625: Too many connections from czar to zookeeper
I have just managed to crash qserv czar by running repeated queries against it. What I see in the logs: qserv-czar.log: {code} 2014-05-02 13:21:10,861:22525(0x7f76e37ab700):ZOO_ERROR@handle_socket_error_msg@1723: Socket [127.0.0.1:12181] zk retcode=-4, errno=104(Connection reset by peer): failed while receiving a server response 2014-05-02 13:21:10,861:22525(0x7f76e37ab700):ZOO_ERROR@handle_socket_error_msg@1723: Socket [127.0.0.1:12181] zk retcode=-4, errno=104(Connection reset by peer): failed while receiving a server response terminate called after throwing an instance of 'lsst::qserv::css::CssException_ConnFailure' what(): Failed to connect to persistent store. {code} and zookeeper.log: {code} 2014-05-02 13:21:10,861 [myid:] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:12181:NIOServerCnxnFactory@193] - Too many connections from /127.0.0.1 - max is 60 2014-05-02 13:21:10,861 [myid:] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:12181:NIOServerCnxnFactory@193] - Too many connections from /127.0.0.1 - max is 60 2014-05-02 13:21:14,585 [myid:] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:12181:NIOServerCnxn@357] - caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x145bdd1b8440015, likely client has closed socket at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228) at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208) at java.lang.Thread.run(Thread.java:744) 2014-05-02 13:21:14,585 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:12181:NIOServerCnxn@1007] - Closed socket connection for client /127.0.0.1:49913 which had 2014-05-02 13:21:14,585 [myid:] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:12181:NIOServerCnxn@357] - caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x145bdd1b8440016, likely client has closed socket at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228) at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208) at java.lang.Thread.run(Thread.java:744) 2014-05-02 13:21:14,586 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:12181:NIOServerCnxn@1007] - Closed socket connection for client /127.0.0.1:49939 which had {code}
Story points: 1