jubatus/jubatus — pull request #329 by tatsuhiko-inoue (2013-04-30 08:14:31+00:00)
Title: Dead lock in anomaly::add method.

When I executed "add" function of anomaly in parallel with ZooKeeper,
the following log message was printed.

W0415 16:46:56.774195 27919 anomaly_serv.cpp:159] cannot create 1th replica: 192.168.10.10:9208
W0415 16:46:56.774281 27919 anomaly_serv.cpp:161] request timed out

I searched the cause of this log message, and found the dead-lock in "add" function of jubaanomaly.

"add" function is defined as "#@update" function in IDL.
Thus "add" function acquires write-lock.
but "add" function calls "update_row" function in other servers.
"update_row" is the "#@update" function, thus it acquires write-lock.

That is, "add" function tries to acquire write-lock of another server, while it holds write-lock.
While 2 servers have held their own lock, when they try to acquire a partner's lock mutually, deadlock occurs.