frerich/clcache — Issue #9: ObjectCache.clean is not safe for concurrent builds 
Author: halfy
Created: 2011-10-29 01:41:09+00:00
State: open

Running multiple builds simultaneously causes the occasional python crashes, due to ObjectCache.clean trying to stat() or rmtree() missing files.

I patched it thus:

def clean(self, stats, maximumSize):
currentSize = stats.currentCacheSize()
if currentSize < maximumSize:
return

objects = [os.path.join(root, "object")
for root, folder, files in os.walk(self.dir)
if "object" in files]

objectInfos = []
for fn in objects:
try:
objectInfos.append( (os.stat(fn), fn) )
except:
pass
objectInfos.sort(key=lambda t: t[0].st_atime, reverse=True)

for stat, fn in objectInfos:
try:
rmtree(os.path.split(fn)[0])
except:
pass
currentSize -= stat.st_size
if currentSize < maximumSize:
break

stats.setCacheSize(currentSize)