Clean up on Riemann plugin tasks