Make sure we can close down a filter context

Will be needed for filter unloading when the filter chain changes...