Marshmallows will soften and get gooey when baked. If baked at too high a temperature for too long, the sugar in the marshmallows can caramelize and harden.