A method for decoding digital media, wherein the digital media includes scalably coded video data and scalability information data including initializing scalability information data in a Scalability Information Supplemental Enhancement Information (SSEI) message and at least one change to said initializing scalability information data with time, the method comprising: at a decoder, using the scalability information data to inform the decoder about the presence and dependency structure of the layers contained in the coded video data thereby creating an SSEI information structure, wherein the at least one change to initializing scalability information data indicates that at least one of: at least one scalability layer is added to the bitstream from that time forward, encoded in a scalability_info_add Supplemental Enhancement Information message which adds the at least one scalability layer to the SSEI information structure, at least one scalability layer is removed from the bitstream from that time forward encoded in a scalability_info _delete SEI message which deletes the at least one scalability layer from the SSEI information structure, or the scalability information for at least one layer present in the bitstream is replaced with the scalability information contained in a scalability_info replace SEI message that contains the changes to initializing scalability information data from that time forward, wherein the scalability_info replace SEI message replaces the at least one layer of the SSEI information structure, wherein any scalability information data for a layer which is not explicitly coded in the changes to initializing scalability information data is assumed to retain the scalability information data's previous value, and wherein the scalable video data is coded according to H.264 Scalable Video Coding, the method further comprising applying the changes to the scalability information at the decoder cumulatively. at a decoder, using the scalability information data to inform the decoder about the presence and dependency structure of the layers contained in the coded video data thereby creating an SSEI information structure, wherein the at least one change to initializing scalability information data indicates that at least one of: at least one scalability layer is added to the bitstream from that time forward, encoded in a scalability_info_add Supplemental Enhancement Information message which adds the at least one scalability layer to the SSEI information structure, at least one scalability layer is removed from the bitstream from that time forward encoded in a scalability_info _delete SEI message which deletes the at least one scalability layer from the SSEI information structure, or the scalability information for at least one layer present in the bitstream is replaced with the scalability information contained in a scalability_info replace SEI message that contains the changes to initializing scalability information data from that time forward, wherein the scalability_info replace SEI message replaces the at least one layer of the SSEI information structure, at least one scalability layer is added to the bitstream from that time forward, encoded in a scalability_info_add Supplemental Enhancement Information message which adds the at least one scalability layer to the SSEI information structure, at least one scalability layer is removed from the bitstream from that time forward encoded in a scalability_info _delete SEI message which deletes the at least one scalability layer from the SSEI information structure, or the scalability information for at least one layer present in the bitstream is replaced with the scalability information contained in a scalability_info replace SEI message that contains the changes to initializing scalability information data from that time forward, wherein the scalability_info replace SEI message replaces the at least one layer of the SSEI information structure, wherein any scalability information data for a layer which is not explicitly coded in the changes to initializing scalability information data is assumed to retain the scalability information data's previous value, and wherein the scalable video data is coded according to H.264 Scalable Video Coding, the method further comprising applying the changes to the scalability information at the decoder cumulatively.