Technically, yes, you can, but you'll have a harder time achieving the colour you want because the frosting is already a natural tan colour. If you must add colour, I suggest trying to make it a darker colour like brown or orange which will be easier, rather than trying to make it a lighter colour (I wouldn't even try to make it light blue, pink, yellow, etc!). I also suggest using food colour gels over water-based colours since they're bolder and won't thin your frosting out if you need to use a large amount. Two brands to check out are AmeriColor and Chefmaster. They’re both vegan AND gluten-free. Another option, if it's a cake you're decorating, is to make a small amount of vanilla buttercream, colour it how you want, frost the outside of your cake with the peanut butter frosting, chill it so it stiffens, and then apply a very thin layer of the coloured vanilla buttercream over the layer of peanut butter frosting.