hpcc-systems/HPCC-Platform — pull request #5498 by jakesmith (2014-03-14 16:42:12+00:00)
Title: HPCC-10698 - Avoid deadlock if retrying lock on superfiles

Two problems:
1) A change introduced in this commit:
19beb0cee074faf6fbde55233507eba897c6d41a

..caused an attempt to lock a file for a transaction, to unlock
the file (completely) pause and retry, this had the nasty side
effect if the file in question was a superfile, of unlocking
the super only, leaving all the sub files locked.
As a consequence competing transactions would still have
locked each other out.

2) A implict transaction (a super operation without a
start/finish super transaction call), used the same transaction
mechanism. If it failed to lock and retried, it cleared it's
locks and cache of files and paused to retry. On the next
attempt it would lookup the files it needed again and relock.
Unfortunately, in an implicit super operation, the caller is
the super file the op. is being performed on and as a
consequence, it is never released, but because the cache is,
the retry mechansim looks up the file again and gains a
2nd lock to the super (and all it's subfiles). As a result,
it deadlocks itself.

This fix:
+ removes the unlock(completely)/pause mechanism on
individual files, introduced by commoning up a method in:
19beb0cee074faf6fbde55233507eba897c6d41a
+ Checks if file is locked, without timeout, i.e. if locked
do not wait (transaction mechanism pauses and retries itself).
This avoids wasteful blocking time, when transaction has
locks to some, whilst blocking/waiting on others.
+ Ensures the superfile involved in implicit transactions,
remains in the file cache, so that on retry it is reused and
not looked up again causing deadlock.

Signed-off-by: Jake Smith