A method of decoding video data, the method comprising: entropy decoding prediction information and quantized transformation coefficients for a block of video data; determining a maximum possible value for a non-separable secondary transform index for the block of video data, wherein a context set is defined for each of a plurality of maximum possible values of the non-separable secondary transform index; determining a context set to be used to entropy decode the value for the non-separable secondary transform index to be the context set defined for the determined maximum value, and based on whether the block is a luminance block or a chrominance block, wherein determining the context set comprises determining a first context model for an ordinal first bit of the predetermined bits regardless of whether the block is the luminance block or the chrominance block; entropy decoding a predetermined number of bits of a value for the non-separable secondary transform index of the block using the determined context set and entropy decoding remaining bits of the value for the non-separable secondary transform index using bypass mode, to form a binarized value representative of the non-separable secondary transform for the block, wherein entropy decoding the predetermined number of bits comprises: entropy decoding the ordinal first bit of the predetermined number of bits using the first context model; when the block is a luminance block, entropy decoding the remaining bits of the predetermined number of bits following the ordinal first bit using a first set of contexts; and when the block is a chrominance block, entropy decoding the remaining bits of the predetermined number of bits using a second set of contexts that is different than the first set of contexts; reverse binarizing the value for the non-separable secondary transform index to determine the non-separable secondary transform for the block, wherein reverse binarizing comprises reverse truncated unary binarizing the value for the non-separable secondary transform index regardless of the determined maximum possible value; inverse quantizing the quantized transform coefficients to produce transform coefficients; inverse transforming transform coefficients of the block using the determined non-separable secondary transform to produce intermediate transform coefficients for the block of video data; and inverse transforming the intermediate transform coefficients, using a primary transform, to produce residual data, in a pixel domain, for the block of video data. entropy decoding prediction information and quantized transformation coefficients for a block of video data; determining a maximum possible value for a non-separable secondary transform index for the block of video data, wherein a context set is defined for each of a plurality of maximum possible values of the non-separable secondary transform index; determining a context set to be used to entropy decode the value for the non-separable secondary transform index to be the context set defined for the determined maximum value, and based on whether the block is a luminance block or a chrominance block, wherein determining the context set comprises determining a first context model for an ordinal first bit of the predetermined bits regardless of whether the block is the luminance block or the chrominance block; entropy decoding a predetermined number of bits of a value for the non-separable secondary transform index of the block using the determined context set and entropy decoding remaining bits of the value for the non-separable secondary transform index using bypass mode, to form a binarized value representative of the non-separable secondary transform for the block, wherein entropy decoding the predetermined number of bits comprises: entropy decoding the ordinal first bit of the predetermined number of bits using the first context model; when the block is a luminance block, entropy decoding the remaining bits of the predetermined number of bits following the ordinal first bit using a first set of contexts; and when the block is a chrominance block, entropy decoding the remaining bits of the predetermined number of bits using a second set of contexts that is different than the first set of contexts; entropy decoding the ordinal first bit of the predetermined number of bits using the first context model; when the block is a luminance block, entropy decoding the remaining bits of the predetermined number of bits following the ordinal first bit using a first set of contexts; and when the block is a chrominance block, entropy decoding the remaining bits of the predetermined number of bits using a second set of contexts that is different than the first set of contexts; reverse binarizing the value for the non-separable secondary transform index to determine the non-separable secondary transform for the block, wherein reverse binarizing comprises reverse truncated unary binarizing the value for the non-separable secondary transform index regardless of the determined maximum possible value; inverse quantizing the quantized transform coefficients to produce transform coefficients; inverse transforming transform coefficients of the block using the determined non-separable secondary transform to produce intermediate transform coefficients for the block of video data; and inverse transforming the intermediate transform coefficients, using a primary transform, to produce residual data, in a pixel domain, for the block of video data.