Yes, you can use frozen bananas for the muffins. Thaw them before using and make sure to drain any excess liquid.