A method of decoding a compressed Higher Order Ambisonics, HOA, representation of a sound or sound field, the method comprising: receiving a bit stream containing the compressed HOA representation corresponding to a plurality of hierarchical layers that include a base layer and two or more hierarchical enhancement layers, and containing basic side information that is associated with the base layer and enhancement side information that is associated with the two or more hierarchical enhancement layers, wherein the bit stream includes data payloads respectively corresponding to the plurality of hierarchical layers, wherein, for a respective one of the plurality of hierarchical layers, there is assigned thereto a plurality of groups of components of a basic compressed sound representation of the sound or sound field, a number of groups of components corresponding to a number of respective layers, and wherein the two or more hierarchical enhancement layers comprises a highest usable hierarchical enhancement layer, wherein each of the two or more hierarchical enhancement layers includes a portion of the enhancement side information including parameters for improving a basic reconstructed sound representation obtainable from data included in the respective layer and any layers lower than the respective layer; and wherein the method further comprises decoding the compressed HOA representation based on the basic side information that is associated with the base layer and based on a first portion of the enhancement side information that is associated with the highest usable hierarchical enhancement layer, and not based on a second portion of the enhancement side information that is associated with any other layer of the two or more hierarchical enhancement layers. receiving a bit stream containing the compressed HOA representation corresponding to a plurality of hierarchical layers that include a base layer and two or more hierarchical enhancement layers, and containing basic side information that is associated with the base layer and enhancement side information that is associated with the two or more hierarchical enhancement layers, wherein the bit stream includes data payloads respectively corresponding to the plurality of hierarchical layers, wherein, for a respective one of the plurality of hierarchical layers, there is assigned thereto a plurality of groups of components of a basic compressed sound representation of the sound or sound field, a number of groups of components corresponding to a number of respective layers, and wherein the two or more hierarchical enhancement layers comprises a highest usable hierarchical enhancement layer, wherein each of the two or more hierarchical enhancement layers includes a portion of the enhancement side information including parameters for improving a basic reconstructed sound representation obtainable from data included in the respective layer and any layers lower than the respective layer; and wherein the method further comprises decoding the compressed HOA representation based on the basic side information that is associated with the base layer and based on a first portion of the enhancement side information that is associated with the highest usable hierarchical enhancement layer, and not based on a second portion of the enhancement side information that is associated with any other layer of the two or more hierarchical enhancement layers.