buffer: keep a list of tiles cache entries in handler

The whole tile cache was being walked linearly on every buffer destruction to
find the tiles a cache handler is responsible for. This made use of temporary
buffers during processing slow down with a constant factor depending on how
many tiles were in the cache.