BUG: (GH3740) Groupby transform with item-by-item not upcasting correctly

closes #3740