Yes, you can skip the oil, but the result may be less crispy. Consider baking the coated bits in the oven at a low temperature for a longer time.