Am I the only one who read the article, and has no idea what the hell they’re talking about?
The original source is clearer:
The launch version of PSSR framed almost the entire reconstruction pipeline as a single Color-Predicting Network. This gave the model a wide surface area for optimization, but it also implicitly asked the model to solve a number of disparate problems that already had efficient closed-form solutions. The talk will show how that generality became a cost, and how the upgrade walked much of it back: restoring explicit closed-form solutions where possible and focusing the model on the pattern recognition it is best suited for. The result improved visual quality and temporal stability while reducing runtime, memory, and training cost by giving the model less to do, not more.
So basically, they treated the whole render process as frame-in frame-out predictions. But by using traditional rendering techniques instead of pure ML the whole system gets faster and better (no surprise to me on that).
Games expect 3x upscaling now to meet performance targets, and ML requires exponentially more data to train a geberal model and test PSSR on a game — that’s the GPU month thing, if I had 30 GPUs it would take 1 day to train 1GPU month (ish — depends on gpu/etc).
The 540p thing is the training goes 4K -> downsample to 1080p -> downscale to 540p as an autencoder (model that tries to predict original input from progressively downscaled/bottle necked layers, with the idea it learns the best representation this way — not actually true though, not relevant here though).
The new model does the same ML but finishes with traditional compositing. That is to say, do you need an ML model to blend 50% blue onto yellow? No, you can use simple math and your ML can focus on the harder bits. So this all works faster and better with HDR. So now less work is done on that and more work is spent on upscaling.
The new model is also more deterministic, so it won’t hallucinate artifacts or turn your main character into a blond instagram model like NVidia DLSS.
Tl;dr, by replacing ML with traditional rendering you do less work, it’s faster to train and cheaper, and it does a better job overall.
The new Kernel Predicting Network places around 90% of its parameters at 540p or below and does little work at 4K
The ELI5(ish) is that they now do as much image enhancement work in low resolution (before upscaling) as possible so the AI has far fewer pixels to process.
The upgraded model also stops predicting the final color directly. Instead, the KPN generates blend weights while conventional filtering handles the actual blend. This avoids storing and blending HDR color in 8-bit form and allows the network to spend less capacity on those operations.
At the same time they aren’t using the expensive model for handling colour, because doing it the “old fashioned” way is more efficient and produces better results.
What, you don’t know what a “GPU month” is? Everyone knows what a GPU month is!