Question: is it grammatically correct to say that being said?
Answer: "That said" is an appropriate truncation of "that having been said", which is correct in that the clause refers back to what was just stated in the prior sentence. "That being said" is incorrect since the prior sentence is in the past. It is not in the process of being said.