No, the butter does not need to be room temperature. Either cold butter or room temperature butter will work just fine to make browned butter. Since you will be melting it over gentle heat, either one will work just fine.