Clean up memory leaks from the write group structure