Don't remove log parts if the resulting log is empty

Not quite sure if this is the best way to do it.

I'll add some tests once I figure out what the right behaviour is.