It’s totally up to you. I peel half of them and use the other half with their skin on. The purple skin of figs gives its color to the bread. So if you don’t want it, remove all of them.