OF course not. Leave it out and just make it a pure pumpkin ice cream. You can also leave out the pecans, add a tablespoon of cocoa powder to the swirl, and make a pumpkin chocolate swirl ice cream.