CacheManagingDrawTask: Fix up cache clear bugs