Everything is cleaned up in https://github.com/mozilla/pdf.js/blob/master/web/viewer.js#L1604 which is called when a page is kicked out of the cache.