dCache/dcache — pull request #865 by gbehrmann (2014-10-07 12:38:23+00:00)
Title: pool: Fix HSM cancellation

A regression may cause a Berkeley DB operation to be interrupted, resulting in
an invalid pool state:

com.sleepycat.je.ThreadInterruptedException: (JE 6.0.11) /pool/meta java.lang.InterruptedException THREAD_INTERRUPTED: InterruptedException may cause incorrect internal state, unable to continue. Environment is invalid and must be closed.
at com.sleepycat.je.latch.LatchImpl.acquireExclusive(LatchImpl.java:62) ~[je-6.0.11.jar:6.0.11]
at com.sleepycat.je.tree.IN.latch(IN.java:539) ~[je-6.0.11.jar:6.0.11]
at com.sleepycat.je.dbi.CursorImpl.latchBIN(CursorImpl.java:441) ~[je-6.0.11.jar:6.0.11]
at com.sleepycat.je.dbi.CursorImpl.removeCursorAndPerformCacheEviction(CursorImpl.java:588) ~[je-6.0.11.jar:6.0.11]
at com.sleepycat.je.dbi.CursorImpl.close(CursorImpl.java:572) ~[je-6.0.11.jar:6.0.11]
at com.sleepycat.je.dbi.CursorImpl.close(CursorImpl.java:555) ~[je-6.0.11.jar:6.0.11]
at com.sleepycat.je.Cursor.close(Cursor.java:518) ~[je-6.0.11.jar:6.0.11]
at com.sleepycat.collections.CurrentTransaction.closeCursor(CurrentTransaction.java:484) ~[je-6.0.11.jar:6.0.11]
at com.sleepycat.collections.DataCursor.close(DataCursor.java:215) ~[je-6.0.11.jar:6.0.11]
at com.sleepycat.collections.StoredContainer.closeCursor(StoredContainer.java:422) ~[je-6.0.11.jar:6.0.11]
at com.sleepycat.collections.StoredContainer.getValue(StoredContainer.java:310) ~[je-6.0.11.jar:6.0.11]
at com.sleepycat.collections.StoredMap.get(StoredMap.java:240) ~[je-6.0.11.jar:6.0.11]
at org.dcache.pool.repository.meta.db.CacheRepositoryEntryImpl.getStorageInfo(CacheRepositoryEntryImpl.java:148) ~[dcache-core-2.10.3.jar:2.10.3]
at org.dcache.pool.repository.meta.db.CacheRepositoryEntryImpl.getFileAttributes(CacheRepositoryEntryImpl.java:156) ~[dcache-core-2.10.3.jar:2.10.3]
at org.dcache.pool.repository.v5.CacheEntryImpl.(CacheEntryImpl.java:35) ~[dcache-core-2.10.3.jar:2.10.3]
at org.dcache.pool.repository.v5.CacheRepositoryV5.setState(CacheRepositoryV5.java:837) ~[dcache-core-2.10.3.jar:2.10.3]
at org.dcache.pool.repository.v5.CacheRepositoryV5.destroyWhenRemovedAndUnused(CacheRepositoryV5.java:955) ~[dcache-core-2.10.3.jar:2.10.3]
at org.dcache.pool.repository.v5.ReadHandleImpl.close(ReadHandleImpl.java:52) ~[dcache-core-2.10.3.jar:2.10.3]
at org.dcache.pool.nearline.NearlineStorageHandler$FlushRequestImpl.failed(NearlineStorageHandler.java:604) ~[dcache-core-2.10.3.jar:2.10.3]
at org.dcache.pool.nearline.AbstractBlockingNearlineStorage$Task.run(AbstractBlockingNearlineStorage.java:230) ~[dcache-core-2.10.3.jar:2.10.3]
at org.dcache.util.CDCExecutorServiceDecorator$1.run(CDCExecutorServiceDecorator.java:104) [dcache-core-2.10.3.jar:2.10.3]
at org.dcache.util.BoundedExecutor$Worker.run(BoundedExecutor.java:211) [dcache-core-2.10.3.jar:2.10.3]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_51]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_51]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]
Caused by: java.lang.InterruptedException: null
at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireNanos(AbstractQueuedSynchronizer.java:1244) ~[na:1.7.0_51]
at java.util.concurrent.locks.ReentrantLock.tryLock(ReentrantLock.java:445) ~[na:1.7.0_51]
at com.sleepycat.je.latch.LatchImpl.acquireExclusive(LatchImpl.java:55) ~[je-6.0.11.jar:6.0.11]

This patch fixes the problem by preventing cancellation from killing the state
update.

The patch fixes a secondary problem in the same code: Cancellation could lead
to a minor memory leak due to requests not being removed from a map.

Target: trunk
Request: 2.10
Require-notes: yes
Require-nook: no
Acked-by: Tigran Mkrtchyan
Patch: https://rb.dcache.org/r/7334/
(cherry picked from commit 8bb52a87a61403a3b3cd84879365463fd0b126cc)