When the JPEG folks were working on image compression, they didn’t create a better or faster LZ. Instead they developed new approaches. I see games growing >150GB and then graphs like this, and it’s obvious the game devs are going in the wrong direction:
https://aras-p.info/blog/2023/01/31/Float-Compression-2-Oodleflate/
(Note these benchmarks are great and extremely useful.)
Separating out the texture encoding stage from the lossless stage is a compromise. I first did this in my “crunch” library around 15 years ago. It was called “RDO mode”. You can swizzle the ASTC/BC1-7 bits before LZ, and precondition them, and that’ll help, but the two steps are still disconnected. Instead combine the texture and compression steps (like crunch’s .CRN mode – shipped by Unity for BC1-5.)
Alternatively: defer computing the GPU texture data until right before it’s actually needed and cache it. Ship the texture signal data using existing image compression technology, which at this point is quite advanced. For normal maps, customize or tune existing tech to handle them without introducing excessive angular distortion. I think both ideas are workable.
Also, these LZ codecs are too fast. They are designed for fast loading and streaming off SSD’s. Who cares about saving off a few hundred ms (or a second) when it takes hours or days to download the product onto the SSD?
Somebody could develop a 10x faster Oodle (or an Oodle that compresses 1-2% better) and we’re still going to wait many hours or days to actually use the product. And then there’s the constant updates. This approach doesn’t scale.
It’s fun and sexy to work on faster LZ but the real problem (and value add) doesn’t call for better or more lossless tech. This is a distraction. If trends continue the downloads and updates will be measured in terms of fractional or 1+ week(s).
Read More
Christeen Coby