BUG/FIX: groupby should raise on multi-valued filter

closes #7870
insert `item` abuse here!

Do we have to support this case? The docs do say it must return a bool. IMO it's pretty weird anyway... better the user thinks about whether there may be cases which are not single-valued rather than assume there isn't (i.e. raise and ask user to manually call item/all/any/whatever).