Awesome! Looking good!

Since it’s using temporal accumulation, perhaps the noise pattern should be randomized every frame to even out the dithering pattern 😄

Interesting! Do you have a reference for how this is done?

We want to include temporal accumulation in other effects like SSR as well so information about this topic are highly welcome.

Everywhere “frameNumber” is used here is for hacking in temporal jitter:
https://github.com/zalo/three.js/blob/2b2c012ed88d402e53c53c14823aff1295440367/examples/jsm/shaders/SSILVBShader.js#L90

The temporal jitter checkbox should enable it here:
https://raw.githack.com/zalo/three.js/feat-ssilvb-gtvbao/examples/webgl_postprocessing_ssilvb.html

I’m not 1,000,000% sure it will converge, but it might help 😅

I’d love to see a dropdown that switches between the no-SSGI color, the pure SSGI channel, and the composite!

This looks cool.
But the demo is scene used is missleading, as it has lightmaps with baked GI in them.
To present the technique in its actual quality, it is best to use scene with dynamic lights and shadow maps, but without lightmaps.

100% agreed!

I've just struggled to find a free, good looking glTF asset with an appropriate CC license and material/texture setup. Many assets at Sketchfab bake light and shadows directly into the diffuse textures so there is not even an option to remove the baked lighting which would be possible with separate light maps. I also wanted to avoid to reuse an existing asset in the repository so we can provide fresh visuals. Help in this context would be highly welcome. Maybe someone can recommend or even share an appropriate asset?

@zalo Thanks! Let's give this approach a go when the PR got merged.

I’d love to see a dropdown that switches between the no-SSGI color, the pure SSGI channel, and the composite!

The last entry in the GUI named output should hopefully do what you are looking for. Beauty is the unprocessed beauty pass (so no-SSGI color), GI is the pure SSGI channel and Default is the composite. Or are you referring to something else?

Congratulations @Mugen87 , quite amazing.

Looks very nice!

Would it be possible to also get a denoise pass in? Maybe the denoise pass from N8AO by @N8python would be suitable here? (https://github.com/N8python/n8ao/blob/master/src/N8AOPass.js#L433)

Or are you referring to something else?

Ah, nope! 😅 It was hidden on mobile until I scrolled down. 🫠

Would it be possible to also get a denoise pass in?

A denoise pass is already available as DenoiseNode.

The respective TSL function denoise() is documented here:

You would use denoise() right after the GI pass and then use the result for the composite:

const denoisePass = denoise( giPass, scenePassDepth, scenePassNormal, camera );

Hmm, @Mugen87 I think it might be too early to close #29668... I just found the time to sit down and test everything...

I think there may be some significant bugs in the SSGI implementation...

Here's the #29668 's (three.js GT-VBAO) implementation with 2 slices and 8 steps:

And here's the WebGPU TSL SSGI Implementation with 2 slices and 8 steps:

GT-VBAO 32 steps:

SSGI 32 steps:

The SSGI one seems to get exponentially worse as more steps are added... likewise, I don't think the temporal jitter is working as expected 💀
Perhaps it is worth adding some more intermediate debug views between the existing implementations and tracking down where they diverge 😅

I wish I could work with TSL as well as GLSL 🫠

Hm, the SSRT3 "three.js" implementation is here; which wasn't quite right to the Unity version either 😅
https://raw.githack.com/zalo/three.js/feat-ssilvb/examples/webgl_postprocessing_ssilvb.html
https://github.com/zalo/three.js/blob/01789a62f2abac47159c81ff9aa482441b519dfc/examples/jsm/shaders/SSILVBShader.js

Which implementation should I step through line by line to look for differences?

The implementation at https://raw.githack.com/zalo/three.js/feat-ssilvb/examples/webgl_postprocessing_ssilvb.html goes darker with higher AOSamples count, that is probably not intended, right?

@zalo I have used the code from https://github.com/cdrinmatane/SSRT3/tree/main/HDRP as a reference. When visually comparing SSGINode with your port, I have temporarily exchanged below code to match yours so the ray starts of the same (I've seen you have used the results of randf() in these lines):

But I did revert to the original version. The way you have computed the horizon did not seem plausible to me so I sticked to the original version as well.

Instead of reopening #29668, I would prefer to open a new issue focusing on SSGINode.

@cdrintherrieno Hi there 👋 ! Glady to report that SSRT3 founds its way in the main repo but it seems there is an issue in our AO implementation that makes the AO too dark the more samples are used (see #31839 (comment)). I could not reproduce this issue own my side so maybe it you could have a look at our demos?

You can use below URLs for testing:

https://rawcdn.githack.com/Mugen87/three.js/88e013b60298ed85d6557f4f954a57692122ca4a/examples/webgpu_postprocessing_ssgi.html
https://rawcdn.githack.com/mrdoob/three.js/8683d9657d3d9b83e9055025c95edcff71281be9/examples/webgpu_postprocessing_ssgi.html

Does the AO/GI look plausible to you? You can visually debug the results of the AO/GI by changing the output in the GUI. When doing so, it's best to disable temporal filtering so you get stable frames.

BTW: The code of the implementation is here:

https://github.com/mrdoob/three.js/blob/dev/examples/jsm/tsl/display/SSGINode.js

The actual shader code is implemented in the setup() method.

@zalo I have tested the other scene with SSGINode and I get a completely different result than in your post. It looks as expected to me:

Here is a link for testing using the latest code from dev, just with another scene setup in webgpu_postprocessing_ssgi:

https://rawcdn.githack.com/Mugen87/three.js/88e013b60298ed85d6557f4f954a57692122ca4a/examples/webgpu_postprocessing_ssgi.html

I also can't observe the mentioned darkening with higher step counts. Some areas are getting a more prominent AO, yes, but it does not look like in your second screenshot. It looks as expected to me.

Woah yeah that is completely different on my phone! I can see the OP’s example work correctly on my phone now… I’ll check to see what’s different about my code or how I called it in the morning in a few hours …. Perhaps 16-bit textures were getting used and messing up the bit counting?

In the new mostly correct version in the OP, I am noticing a leftward bias to the darkening (that goes away when you look at the scene from the top-down).

This behavior does not appear in the new Little Tokyo demo you linked! 🧐

@Mugen87 Very nice to see it implemented in ThreeJS! Indeed the AO seems wrong, also it flickers which indicates that the slice parametrization is not symmetric. If I had to guess based on a quick glance at the code, it's probably around these parts:

let frontBackHorizon = vec2( dot( pixelToSample, viewDir ), dot( pixelToSampleBackface, viewDir ) );
frontBackHorizon = GTAOFastAcos( clamp( frontBackHorizon, - 1, 1 ) );
frontBackHorizon = clamp( div( mul( samplingDirection, vec2( frontBackHorizon.x.negate(), frontBackHorizon.y ) ).sub( n.sub( HALF_PI ) ), PI ) ); // Port note: This line also required an update because of different uv conventions
frontBackHorizon = directionIsRight.equal( true ).select( frontBackHorizon.yx, frontBackHorizon.xy ); // Front/Back get inverted depending on angle

You can also compare with other implementations (although I'm not sure if they use the same coordinate system as WebGPU):
https://www.shadertoy.com/view/dsGBzW
https://www.shadertoy.com/view/XXGSDd
https://github.com/martymcmodding/iMMERSE/blob/main/Shaders/MartysMods_MXAO.fx#L592

I think there are two major differences in coordinate space conventions that a port must honor. In WebGPU, the uv (0,0) represents the top-left corner whereas in Unity it should be bottom-left. For matching uv offsets, three's implementation must compute the uv and sampling directions differently.

I believe this bit is correct. What I'm unsure about is the second difference in view space handedness. In WebGPU and WebGL, the camera looks down the negative Z-axis by default so (0,0,-1). In Unity, it should be (0,0,1) (is that right?). So our viewZ values are also negative. That's why I had to add a negate() here to fix view space sampling.

This difference in view space handedness could also affect the slice parametrization. Ideally, we don't get it right by trial and error but can understand the math differences in the shader. Can you think of a part of the frontBackHorizon computation that needs an update if the view space handedness is inverted?

Found one bug which is fixed via #31863. Keep working on potential horizon issue...

Okay, I'll check out GT-VBAO next when I'm happy with the SSRT3 code. In the meanwhile, I would appreciate any help in evaluating our port of SSRT3 so we know the coordinate space sensitive operations are correct. This will also help us when writing the TSL for GT-VBGI. BTW: I would prefer to do this manually and not with an AI so we can evaluate, understand and document (if necessary) each line.

@zalo Thank you for your help here ❤️ ! I'm convinced any three.js SSGI and AO implementation will benefit if we get our SSRT3 right. We can document all relevant port changes so it's clear for developers what parts of the reference code have been updated in what way.

Ahh, so the source of the major discrepancy was that it was falling back to WebGL when I did local development without https (and I guess in other weird contexts); it seems like there's something in here that breaks when emitting GLSL. When forced to https, it uses WebGPU and everything looks as expected.

Secondly, I think I found the AO bug! When I change the PI on this line to PI2, everything looks beautiful and non-flickery:

I'm not sure why, since that correlates with this line which still has it as PI:
https://github.com/cdrinmatane/SSRT3/blob/main/HDRP/Shaders/Resources/SSRTCS.compute#L348

Sorry for the compression making it rough...

Secondly, I think I found the AO bug! When I change the PI on this line to PI2, everything looks beautiful and non-flickery:

Indeed, that's it! Awesome! Would you like to make a PR with the fix?

Ahh, so the source of the major discrepancy was that it was falling back to WebGL when I did local development without https

Makes sense! Indeed, there is an issue on the GLSL side. In some areas, the AO looks darker than in the WGSL version. I'll have a look at this tomorrow unless you find the root cause ahead of me^^.

Comparing the "fixed" SSGI (SSRT3) shader with the three.js GT-VBAO:

While the SSRT3 SSGI is certainly an improvement on GTAO, I think GT-VBAO/GI (maybe with distant light sampling?) will be a significant quality improvement even on top of that, if we can work it.

Also, this is all raising the necessity for marking pixels "dirty" in the TRAA pass, to prevent ghosting...

#31873 fixed a bug in SSGINode that substantially compromised the bitfield. This bug was the actual root cause for the AO flickering and was also responsible for the mentioned quality issues.

The new version produces a noticeably better AO. I guess we need a new comparison video for #31839 (comment) ^^.

Here's the latest comparison between the two implementations, both set to 4 slices and 32 samples (other settings adjusted to be similar style):

The smoother/nicer one is the fixed SSGI 😄 I no longer think that the incremental improvement from GT-VBAO will make a huge difference...

Now there are just a couple smearing issues with the TRAA to solve... and I might commit some extra changes to the scene. The "Neutral Tonemapping" and Exposure set to 1.5 is crushing out a lot of the detail, and the "pre-baked" lighting room is hiding a lot of the benefit 😄

As far as the demo scene goes... the current room is a crime compared to some of these other models

Spaceship Hallway shows off both the AO and the GI really well, since it has no baked in lighting (this is JUST the SSGI postprocess):

EDIT: This scene does not look nearly as good when the GI compositing is fixed. Guess it was just lucky.

I'll look into making a custom demo scene to show it off in a few hours...

Hmm, there's another issue... it looks like the GI doesn't act like normal light, it just adds itself on top of the scene.

Since the Albedo/Normals/Roughness/etc. GBuffers are present, would it make sense to interpret the GI as PBR light when compositing?

At the very least, it should account for the underlying color/textures somehow...

Aha, I've got a fix

Just need to swap out:

const compositePass = vec4( add( scenePassColor.rgb, gi ).mul( ao ), scenePassColor.a );

for:

const compositePass = vec4( scenePassColor.rgb.mul( ao ).add( scenePassDiffuse.rgb.mul(gi)), scenePassColor.a );

I'll make a PR for it.

Considering a Cornell Box since it very succinctly shows the benefits of AO+GI and it looks nice with the settings dialed in:

(Here's the lit frame at full color depth)

// Walls
let geo = new THREE.PlaneGeometry(1, 1);
let mat = new THREE.MeshPhysicalMaterial({ color: "#ff0000" });
let mesh = new THREE.Mesh(geo, mat);
mesh.scale.set( 20, 15, 1 );
mesh.rotation.y = Math.PI * 0.5;
mesh.position.set(-10, 7.5, 0);
mesh.receiveShadow = true;
scene.add(mesh);

mat = new THREE.MeshPhysicalMaterial({ color: "#00ff00" });
mesh = new THREE.Mesh(geo, mat);
mesh.scale.set( 20, 15, 1 );
mesh.rotation.y = Math.PI * -0.5;
mesh.position.set(10, 7.5, 0);
mesh.receiveShadow = true;
scene.add(mesh);

mat = new THREE.MeshPhysicalMaterial({ color: "#fff" });
mesh = new THREE.Mesh(geo, mat);
mesh.scale.set( 20, 20, 1 );
mesh.rotation.x = Math.PI * -.5;
mesh.receiveShadow = true;
scene.add(mesh);

mesh = new THREE.Mesh(geo, mat);
mesh.scale.set( 15, 20, 1 );
mesh.rotation.z = Math.PI * -0.5;
mesh.position.set(0, 7.5, -10);
mesh.receiveShadow = true;
scene.add(mesh);

mesh = new THREE.Mesh(geo, mat);
mesh.scale.set( 20, 20, 1 );
mesh.rotation.x = Math.PI * 0.5;
mesh.position.set(0, 15, 0);
mesh.receiveShadow = true;
scene.add(mesh);

// Tall Box
geo = new THREE.BoxGeometry(5, 7, 5);
mesh = new THREE.Mesh(geo, mat);
mesh.rotation.y = Math.PI * 0.25;
mesh.position.set(-3, 3.5, -2);
mesh.castShadow = true;
mesh.receiveShadow = true;
scene.add(mesh);

// Short Box
geo = new THREE.BoxGeometry(4, 4, 4);
mesh = new THREE.Mesh(geo, mat);
mesh.rotation.y = Math.PI * -0.1;
mesh.position.set(4, 2, 4);
mesh.castShadow = true;
mesh.receiveShadow = true;
scene.add(mesh);

// Lampshade
mesh = new THREE.Mesh(new THREE.CylinderGeometry(2.5, 2.5, 1, 64), 
					  new THREE.MeshBasicMaterial());
mesh.position.y = 15;
scene.add(mesh);

// Main Light
let light = new THREE.PointLight("#ffffff", 100);
light.position.set(0, 13, 0);
light.distance = 100;
light.castShadow = true;
light.shadow.mapSize.width = 1024;
light.shadow.mapSize.height = 1024;
light.shadow.bias = -0.002;
scene.add(light);

// Ambient Light
light = new THREE.AmbientLight("#0c0c0c");
scene.add(light);

Also, it seems like there's another bug where the screen space number of steps decreases the effective radius because the step radius is getting divided by the number of steps twice; I put a fix for that into my PR as well.

I've got an improvement to the temporal noise coming down the pipe in #31890

And I took a stab at trying to fix the ghosting in the TRAA implementation, but persisting the previous frame's depth to the next frame was too much for me, so I made an issue at #31892

Lastly, I have this Cornell Box SSGI Example Scene, does this seem good? Should I make a PR for this?

Lastly, I have this Cornell Box SSGI Example Scene, does this seem good? Should I make a PR for this?

That would be great!

Unless we want to do a two part TRAA or have the SSGI take in Roughness/Specular to subsume SSR, then I think I'm done for this release. 😄

Is it possible to just use this AO and disable for performance reasons? This AO seems much smoother than the original ao node.

Is it possible to just use this AO and disable for performance reasons?

You mean disabling the GI? That is of course doable e.g. with a new flag. That would mean the rgb components of the SSGI texture would just hold the black clear color.

That said, GTAONode does currently not support temporal filtering. Adding it should definitely improve the quality similar to SSGINode. It requires the usage of TAA though. So that would be an option as well.

I already tried exclude the GI stuff but without success. Would you mind to adapt it accordingly? It would make sense to disable the whole GI computation if the intensity is set to 0.

If we want to do this right, we need larger changes to the component since "AO only" means you can use a smaller render target, less uniforms and less TSL.

TBH, improving the standalone AO node seems easier to maintain to me right now.

Is it possible to achieve similar results? The AO of the SSGI node is much better than of GTOA node.

SSGINode is based on GTAO so the underlying algorithms are similar. We could mirror some changes of SSGINode back to GTAONode to make the effect more similar but that must be done carefully. The code in GTAONode is more simple but thus more performant.

I would recommend to start with adding temporal filtering to GTAONode which will denoise/smooth the AO similar to SSGINode. We can then see what else is missing. In the meanwhile, try to use a manual denoise() with DenoiseNode to smooth the AO.

Let's discuss the addition of temporal filtering to GTAONode at a new issue though.