Yes, you can, though fresh bell peppers will provide a better texture. Frozen peppers tend to be softer upon cooking.