Yes, you can leave nutmeg out of a recipe if you don't have it on hand or prefer to avoid its flavor. However, keep in mind that nutmeg adds a distinctive warm and slightly sweet flavor, so the final dish may taste different without it.