Day 213: SSAO

Today I finished implementing SSAO. The result is messier than expected, with a lot of visible noise. I knew I had to expect noise between pixels, but I at least hoped that pixels close to each other would behave similarly, making noise softer on the eye.

Also, I realized that SSAO should be applied only to ambient light, or at least have different weights, and only influence diffuse light in a small amount.

That being said, the steps required to have a proper, functional implementation of SSAO seem to be:

1) Blur the result to soften the noise
2) Feed the texture to the lighting calculations, so that the right amount of AO can be applied properly.

I'm not sure I'll go ahead with the implementation, as it requires some big changes to the pipeline, and I really just wanted to test the post processor. But I might change my mind in the future, as the graphics pipeline evolves.

In the picture, you can see the AO factor buffer on the top, and the result applied to the image on the bottom (particularly visible near corners and edges).

Day 212: Improved post-processor and some SSAO

Today I started by quickly refactoring the post processor. I made it faster, and factored most of the operations in an include shader.

Then I started to work on an implementation of SSAO in order to use the post processor for something useful. The implementation is still not great, and while I get some occlusion, I'm not yet pleased with the results. I hope I'll be able to improve it tomorrow.

Day 211: Tiled deferred rendering for spot lights

Today I completed the tiled deferred rendering implementation to fully support directional and spot lights as well.

For directional lights I just needed to refactor the shader to sample the G buffer just once, and use its result to process all the lights.

For spot lights, I also had to take care of culling. This is mostly done the same way as point lights, but the position and radius of the bounding sphere are preprocessed in order to calculate a fitting sphere that matches the cone as much as possible. There are probably more correct ways to tackle the problem, but this works well enough for my purpose. In the picture you can see the amount of spot lights processed for each tile, and as you can see the culling is more aggressive than with point lights.

I also wrote from scratch the code to get the outer fading of the spot lights, as the math was slightly off.

EDIT: the pic contains a bug with AABB culling, noticeable in the black edges in the bottom of the screen, which has been fixed in the evening.

Day 210: Tiled deferred rendering testing

Today I did some testing of the tiled deferred rendering under several different conditions, just to have a general idea of how it behaves. Some of the parameters I varied are radius size, number of lights, amount of overdraw, culling techniques and distance from camera.

I also noticed that accessing the light struct into the SSBO is still the most expensive operation, so I tried to put the whole light into the TBO. That makes caching less efficient and reduce performance, so I ended up discarding the change.

In the afternoon I spent some time putting together a [Video] to show the results and the process behind the implementation.

Day 209: Rendering 10.000 lights with TBOs

Today I decided to use a TBO instead of an SSBO for culling information.
Since the light structure is fairly big, I'm still using the SSBO for shading, but for culling I'm using a TBO that holds just the radius and the position of the light. I did expect an improvement in performance, but the result is far beyond expectations.As you can see from the picture, I'm rendering 10.000 lights at more than 50 fps. Also, the technique is more efficient with a grid size of 16x16, which is one of the problems I still had to address.

Tomorrow I plan to do some testing of the algorithm under different conditions, and I'm also planning on writing a short article about it, as I'm quite happy with the results.

Day 208: Alternative culling

Today I tried to find a solution to the bad performance of the 16x16 version of tiled deferred rendering. Alas, I couldn't come up with anything, as it's clear that the problem is the number of memory accesses, and there is no apparent way of reducing that. Using a TBO might help with cache coherence, but I'd rather not use that to store complex structs.

Anyway, to improve performance further (including for the 32x32 version), I decided to experiment with another type of culling, AABB culling. Some papers argue that the test is faster to do, and occasionally leads to better results. So I tried it and I agree it gets good performance, despite being slightly worse than the frustum culling. Another thing most paper do, is pick one of the two techniques, but I think that doing both has great benefits. In the picture you can see blue and red squared, which represents the differences in lights culled by the first and second methods (brighter = more than 1 light). As you can see, a LOT of lights gets treated differently, and I think that the extra computation to do both tests is justified (and profiling this specific scene seem to prove it, but it is definitely not a common scene").

Day 207: Improvements to tiled deferred rendering

Today I cleaned the code of the tiled deferred shader.
For example I noticed that shading each light on the compute shader would cause a new sampling of all textures in the gbuffer. So I wrote a function to sample them, build the relevant information, save them into a structure, and reuse them for lights calculation. This didn't massively help performance, probably because cache was already taking care of it, but I noticed some improvements.

Then I noticed that the technique runs at 55 fps with 2500 lights with 32x32 grid size, while it drops to 16 fps at 16x16 size. I suspect that bad latency-hiding is the cause, but running the program on Intel gets a better performance, so it might be a problem with the Nvidia driver. I'm still investigating the issue.

I also tried RenderDoc to profile the shader, and while it doesn't do real-time profiling (I think, didn't look too much into it yet), it is a great tool to analyze a single frame of rendering.

Day 206: Point light culling in tiled deferred rendering

Today has been an amazingly productive day. I planned to implement the full culling logic for point lights in the tiled deferred rendered, and I managed to do it. Also, I imposed myself to not look online for implementations, as I wanted to do all the math on my own, and I'm really happy I managed to pull it through fairly easily.
The shader might have room for optimization, as its performance is not very good in all situations. Here are some data on 1000 point lights

Normal deferred : ~55 fps
Tiled deferred with 16x16 grid: ~35 fps
Tiled deferred with 32x32 grid: ~62 fps

As you can see, the technique is an improvement when using the 32x32 cell size, which basically tests a single culling for each pixel. On 16x16 though, apparently the 4 tests required are killing performance, which is fairly surprising, as the most complext part of the shader, the fragment rendering, doesn't seem to give any problems. Tomorrow I'll debug and improve what I have.

In the picture is shown the rendering with a white overlay representing the number of point lights influencing that specific quad.

Day 205: Tiled deferred rendering complete (Culling excluded)

Today I finally completed the implementation of all three tiled deferred rendering systems. Now the system can switch between tiled and non-tiled depending on whether an object casts shadows.

The nasty bug I had yesterday was quite hard to track down. Basically the introspection in my shader class assigns a size to each type of uniform. However, I forgot to add the cube_shadow_map to the list, and so the point shadows were broken. This has always been a bug, but before I was storing uniforms into a map, and the value was stored in a unsafe area of memory, without ever giving problems. However, after changing the system to store uniforms in a contiguous area of memory, this led to overflows and overwrites with the other uniforms, making the problem evident.

In the evening I also changed the light formula for specular ligthing, as it was wrong to a certain extent as the diffuse light one. In the picture you can see all ligths working together in tiled mode.

Tomorrow I'll start to work on the culling for point lights. It seems fairly easy and I'm very curious about profiling the results.

Day 204: Work on tiled deferred rendering, and new light attenuation system.

In the morning, I addressed a problem I had, which is that tiled and non-tiled rendering were slightly different. Turns out that the sampling was off by half a pixel in the regular one, and that the tiled one was leaking some colors outside the attenuation radius.

In the afternoon I continued the implementation of tiled deferred rendering for both Point and Spot lights. A couple of nasty bugs kept me busy for quite some time, and I still need to solve the last one, which makes the point lights not work in regular deferred rendering.

In the evening, while looking for the solution of the bug, I noticed that the attenuation logic of my lights was completely wrong, including the calculation of the bounding box size - at the time of the implementation my 3D math skills were quite basic. So I fixed it, and now lights behave more realistically and have better performance.

Day 203: SSBO for all lights, and point lights tiled rendering in progress

Today I started to implement compute shaders for point and spot lights as well (although I didn't finish the latter yet).
As soon as the point lights were working, I noticed a bad performance issue with UBOs. Basically I was sending an UBO for each different light, and this was not good for performance. So i decided to use an SSBO instead. I wa already using one for the compute shader, now I'm using another one for the normal lights. Depending on whether a light has shadows, its information are added to either SSBO.

At the end of the day, I managed to have dir and point lights working. Since point lights don't have any culling yet, the scene with 1000 point light is running at ~5 fps, but I expect it to become extremely fast after culling is done.

Day 202: Important optimizations and polishing

Before proceeding with the implementation of tiled deferred rendering on point and spot lights, I wanted to clean the code for directional light as much as possible. I also wanted to write from scratch the memory solution used to hold uniforms in a material, because of a leaking problem and bad performance.

First, I tackled the uniform problem. I was still using a map to hold the data, with a flawed solution to account for the different size of uniforms. I used a different system instead, which allocates a chunk of memory, and stores the uniforms at different addresses. This is not leaking memory, it is faster and helps cache coherence.
I also had to rewrite the compare_uniforms function used for sorting, in order to account for the changes. It became much simpler, and the speed up is incredible. I tested the sorting time of 10000 objects with the old and the new system, and from ~9 ms the time dropped at ~0.9 ms.

In the afternoon I decided to make the compute / non-compute techniques even more similar. Instead of sending uniforms to the light shader, I replaced them with an UBO. This is more efficient, makes the code much simpler, and reuses the same structure used by the compute shader (with the difference that the latter takes an array of lights).
With this optimization the code for directional lights has become short, readable and simple, despite the fact that it's supporting both deferred and tiled deferred shading. Tomorrow I'll mimic the same behavior for point and spot lights, and once that is done, I'll start to work on culling.

Day 201: Shader Preprocessor

Today I decided to refactor the code of the compute and regular shaders I'm using for directional lights. I immediately found out that OpenGL has no way to include header files, so I took some time to write my own preprocessor in order to achieve that.
It is now possible to start a shader with one (or more) of the following commands:

E(path)
G(path)

The first one loads a .glinc file (custom extension I'm using) with a path relative to the engine folder, the second relative to the game folder. The loading operation involves replacing the command with the actual code contained in the file.
With the preprocessor in action I managed to simplify tremendously both shaders, which now share most of the code in a "DirectionalLight" library, and also several things from a "Global" library, in which I'll put useful functions and UBOs that need to be shared by every shader.

This work led to a refactoring of the deferred renderer as well, and now the directional light code is simple, well organized and reusable. However, I found a very bad memory leak in the Material class, and in how it handles uniforms. I'll need to make a huge refactoring there, as I'm not happy at all with the amount of allocations it performs, as well as the fact that it is leaking.
Tomorrow if I've time I'll also start refactoring point and spot lights to do the same as the directional ones. The new shader preprocessor will make it very easy and clean to add culling, as it will be just a function at the beginning of the compute shader, while the light code is in the library.

Day 200: Compute Shaders

I decided to embark in the challenge of implementing tiled deferred rendering. The task would not be too hard to be implemented from scratch, but the engine is growing big, and despite my efforts to keep the codebase clean, lots of care need to be taken when changing techniques drastically.
I decided to start from directional lights, as they don't require culling, so I would just need to focus on the implementation of a simple compute shader.
A major decision was keeping both deferred and tiled deferred rendering. The first for shadow-casting lights, the second for non-shadow-casting lights. This is because tiled deferred rendering would require to precalculate all the shadow maps and store them into an array. The trade-off might be worth it, but for now I prefer to just calculate the lighting into two different ways and reuse a single shadow map. Tomorrow I plan to wrap all the lighting logic into functions that will be shared by both shaders (the regular one and the new compute one), in order to avoid code duplication.
The compute shader itself gets its data from an SSBO filled with light structures. Unfortunately, the dynamic nature of the engine forces to refill it every frame.
In general there are some problems with the code, and I'd love to make it more clean, so tomorrow I'll probably focus on  polishing and refactoring.

In the picture, directional lighting obtained through the compute shader. The grid pattern its just a mask applied in the compute shader to show the blocks of threads.I opted for 32 x 32 to have 1024 parallel thread, which results in a good number of warps (thus hiding memory access latency). When culling will be a priority (i.e. spot lights) I'll use 16x16.

Day 199: Post Processor Manager

Today I spent the morning fixing some tiny problems I noticed with the engine, as well as some minor improvements (like adding the screen size to the global UBO).

In the afternoon, I moved the logic of the post processor to a manager class. Each scene holds its own post processor, so the user can select a different behavior for each scene.The post processor simply holds a vector of materials (that is, a shader plus its arguments), and the renderer iterates through the list to apply them. The result of each pass is saved alternatively to two render textures, and the next step samples it.
Finally I added command to the shell to enable and disable the post processor dinamically.

In the picture, two postprocessing effects working at the same time: the first masks away the green channel, and the second flips the uv coordinates.

Day 198: Post Processor

In the last few days I've been studying some rendering techniques, and a few of them were basically full-screen post processing effects (for example, screen-space reflections). So I decided to implement a versatile post processor for the engine, as it is fairly easy to do, and will allow me to test some of the aforementioned techniques.

I added a pass to the rendering pipeline, and also I needed to add another texture in the FBO. This is because now I will need two textures to use alternately when applying post processing techniques.
The shader used for this pass uses a structure to hold all the most important data sampled from the screen. The struct is filled directly in the shader with a function that will be available to every post processing shader. Colors, normals, depth, position and so on will be available easily and immediately. If information of adjacent pixels are needed, they'll have to be calculated manually.
An awful amount of time was pent tracking down a bug that would cause a black screen. In the end, I didn't specify a filtering option for the two new textures, and OpenGL refused to assume a default value.
My goal for the following days, is to create a PostProcess manager, in which the user can add and sort techniques.

In the picture, a simple post proces effect to invert the color of the image.

Day 197: UBOs

I've not been updating the engine for a few months, but in the meantime I learned a lot about modern C++, graphics techniques and game engines. So I decided to continue this project, in order to improve things I did in the past, as well as experiment with new techniques.
I spent some time just going through the code, as my memories of it are a bit blurry, and I immediately noticed something I was doing in an inefficient way: all the uniforms in the game were sent individually to each shader. This is clearly not ideal, as it adds some bandwidth cost.So I did some research, and found out about uniform buffer obejects, which I never used in the past. They are a simple, read-only area of memory on the GPU, which can be accessed by all shaders. So I created a UBO containing all the global uniforms that most shaders rely on: matrices, time and camera positions for now. I updated all shaders to use them and the engine is now working with them.
I'm now thinking of adding a way to create custom UBOs, so that user-level shaders can benefit from the same technique. Basically what Unreal Engine does with the Collections of parameters.