BUG: Bug in groupby transform with a datetime-like grouper (GH5712)

closes #5712