Caviar is high in fat, so she cautions not to go overboard with your serving size or frequency. Lane notes no more than two times a week is recommended. “Most people don't consume caviar on a pretty regular basis because it's expensive and it's more considered a delicacy,” she says.