Don't remove group aesthetic if geom can handle groups (as with ribbon, polygon, line, and path.)
Updated comment