BUG: (GH3740) Groupby transform with item-by-item not upcasting correctly
BUG: ensure float into function of series transform when item-by-item

     downcast result if needed
Merge pull request #3743 from jreback/groupby_int

BUG: (GH3740) Groupby transform with item-by-item not upcasting correctly