Add destructors for all of the layers

Add destructors for all the layers so that the dynamic parameter clients are destroyed as well.