Tweet
Hi, As per the h264 specification, in case of bitdepth equal to 8, transform ouput should be within the
–2^(7 + BitDepthY) to 2^(7 + BitDepthY)–1, inclusive. But few streams generated by the JM encoder producing morethan 16 bit output as transform coefficients. JM encoder doesn't have limit checks on transform output. Can i assume that, the coefficient size after the transform is within the 16 bit in h264 decoder implementation (i.e. inverse quantization input)?
Thanks,
Sapthagiri
–2^(7 + BitDepthY) to 2^(7 + BitDepthY)–1, inclusive. But few streams generated by the JM encoder producing morethan 16 bit output as transform coefficients. JM encoder doesn't have limit checks on transform output. Can i assume that, the coefficient size after the transform is within the 16 bit in h264 decoder implementation (i.e. inverse quantization input)?
Thanks,
Sapthagiri