There's no need to allow the brie to come to room temperature before baking. The oven will adequately heat the brie cheese.