In most cases, cross-play is limited to players on the same console. You can generally only play with Xbox friends on your PS4 if they are also playing the game on their Xbox console.