Yes, you should use regular peanut butter for cupcakes. Peanut butter does not typically have any added sugars or other ingredients, which makes it an ideal choice for baking. You may also opt for natural varieties of peanut butter if you prefer.