Yes, you do need to allow the brownies to cool fully. Otherwise, the frosting will melt, making the dessert mushy.