Remove Rebroadcast in the input of Join.

Add it only when needed in the grad.