Since we're removing things from the legend then we might as well make the entire thing disappear if it is empty as well. extension of @04b3937 #202