You can, but I find it's not worth the extra effort, and the peels melt right into the jam as it cooks.