Fog/Smoke/Underwater - sometimes banded instead of smooth

[BRTACAN2]

I'm relatively pleased with 5.1 so far. I've done several encodes with it - Fellowship Of The Ring, Murder By Numbers and Hannibal. I'm seeing an increase in quality, [not big but noticable]. It takes 5.1, in slow mode, about 4X longer than 5.05, in slowest mode, with all other settings similar. I also see @15% speed boost by turning off the feedback window.

Anyway, one of the things I checked first with 5.1 was the smoothness of blended backgrounds like fog or smoke or underwater. I'm not liking what I'm seeing. Instead of smoothness, I see a noticable banded/stepped halo effect... as if the color palette is limited and can't display a smooth transition from color shade to shade. My bitrates are fine and bits/frame is better than 0.25.

I had this in 5.05, and had hoped it would disappear in 5.1. Prime examples are underwater scenes in U-571, the hilltop/foggy fight/chase scene in Rob Roy, and in Fellowship Of The Rings where they're running from the pub being chased by the horsemen... horse slides to a halt on the dock... then you see the other horsemen arrive in the background [there where you have a backlit fog].

I usually encode in home theater mode, at 704 by X to 640 by X, with relatively high bitrates of 1200 to 1800, keeping an eye on bits/frame, using Lanzos resize filter and minimum/fast PV. That's it, no QPEL, no GMC, etc.

Anybody else seen this? Post-processing settings can fix it? Recommendations?

Thanks for any input/suggestions.

[Enchanter]

That kind of problem is normally associated with having QPel enabled (which, I noticed, you have supposedly disabled). You should double-check to ensure that QPel has indeed been disabled.

Cheers.

[BRTACAN2]

Hi Enchanter: Nope. I've never used QPel. I always use Home Theater Mode. You have to deselect the standard modes to use QPel/GMC; and, I've never desired to do that. I stay within DivX spec - best way to avoid odd problems... well ummm... almost. I also never wanted to take the speed hit caused by using QPel anyway.

The banding isn't a huge problem. It's an annoyance. I try to maximize quality as opposed to stretching the codec to sqeeze a three hour 4:3 movie on 1/2 a CD, if you know what I mean.

The banding doesn't always manifest itself. My impression is that it occurs when there is fog/smoke/underwater that is backlit. For example, the underwater scenes in U-571 are obviously CG - likely using a background point light to highlight the sub. On the other hand, the clouds in the last scene of Crouching Tiger Hidden Dragon [when she jumps] look just fine, (lighting is more natural in that scene).

Thanks for expending the brain cells to give it some thought...

[electric!sheep]

Is your color depth 32bit? (Just making sure.) Consider turning up some of DivX's deblocking on the decoder.

[Enchanter]

You could also try XviD...

[BRTACAN2]

Yupp... my ATI CATs are set for 32bit. I may try XviD in the future. I looked at it in Gordian Knot before. The codex setup didn't seem as intuitive - to me anyway. So, I really haven't messed with it to date. I like Divx. I think it works great... it's just this one nagging issue that crops up sometimes. The occational lack of fog/smoke smoothness is livable. I notice it... but it's not like a big yellow macroblock or something. I'm surprised I'm the only one whom seems to have come across it. I'll try playing with decode deblocking lectricsheep sugests.

Thanks...

[electric!sheep]

Originally posted by BRTACAN2
The occational lack of fog/smoke smoothness is livable. I notice it... but it's not like a big yellow macroblock or something. I'm surprised I'm the only one whom seems to have come across it.

Well, I think most of us accept it as a feature of the codec, which is trying to save bits by representing swaths of similar color with a single plane. True gradients are really hard to compress, I suppose, since any reduction in quality makes them evidently chunkier.