Tankenstein NPC spawning and POIs

Hi.

We recently pushed the 0.3 update for Tankenstein which included a big upgrade to the terrain generation and spawning system. Because of those changes and because we've had some comments about tank spawning, I decided to write up some detail about how the terrain streaming works, and how tanks are spawned as part of that system.  

I've also included some information about points of interest (POIs), which govern how the AI navigates across the landscape after spawning.

In order to explain how tanks spawn it's necessary to know the basics of how the world streaming system works in general, because there isn't a separate spawn system for tanks. 

Spawning

Tankenstein uses an in house tool called Panzer Terrain. This handles real-time, procedural generation of terrain and game object spawning. As the player traverses the world, the landscape is spawned in as ~1km² tiles. The height map is generated on the GPU using compute shaders, then handed off to the CPU to generate collision data.

 

GPU terrain generation based on camera position.

Once the height map and collision data have been generated for a tile, it is handed off to the game object spawning system which creates all the objects that appear on that terrain tile. This system works by subdividing the tile into smaller and smaller cells and attempting to spawn an object in each cell. This structure is known as a quad-tree and by testing for collisions at each node of that tree we can efficiently ensure spawned objects don't collide with each other.

Objects are spawned in the following order. Large compounds -> small compounds -> tanks -> wrecks -> scattered cover -> decals. There's a few extra steps such as flattening the terrain around compounds and generating the navigation mesh, but that is the order for the spawning of objects. Some objects such as grass, stones, rocks and trees are dynamically streamed, so they only appear near the player. But anything gameplay related is handled by the spawning system.

The spawning sequence slowed down to show each step.

Functionally this means that well before a player gets to a terrain tile, all the objects that can spawn on that tile have already streamed in. This includes tanks, wrecks and any other objects that can be scanned by the radar. 

For performance reasons tanks and compounds will remain inactive until a player gets within 1000m of them. Once the player enters that radius, they will wake up and start running their AI. This leads on to POIs, and how tanks use them to navigate the map.

POIs

Many of the objects that spawn are flagged as points of interest (POIs). This is a tag that tells the AI that an object is of interest to them. POIs are wrecks (including destroyed tanks), scrappers, recombinators and compounds. When out of combat, tanks will pick nearby POIs to investigate. They will travel to that POI, investigate, then pick another.

 

A compound showing points of interest highlighted in blue

 

If an area has many POIs in it, such as a compound after a fight, then it becomes more likely that a tank will pick a POI in that area, by virtue of there being many POIs. In fact, compounds are some of the more dangerous areas of the map because of this.

 

A tank investigate a wreck.

The combination of the tanks spawning with terrain and investigating POIs means that as the player traverses the wasteland, there will always be new tanks driving around, investigating POIs and occasionally fighting each other*. They never remain idle, because the aim of this system is to populate the world with challenging enemies that behave in a (mostly) intelligent but predictable way. 

 

Hopefully this gives a little more detail about how the spawning system works and how tanks navigate through the world after spawning. If you have any questions, I'm happy to answer them here, on steam or on discord. 

 

* We had to limit this to a 10% chance for a tank to start a fight, any higher than that and tanks would destroy each other before the player could get involved.

 

Tankenstein

 

This has been on my bucket list for a long time! I have finally released a game on steam after many years of game dev. 

Tankenstein is a 3rd person ARPG where the player controls a disheveled tank in a scrapyard looking for new parts to survive and upgrade itself. All of the other tanks in the scrapyard have the same goal, which makes them hostile in this survival of the fittest arena. 

Available now on Steam

[OpenCV / OpenGL] Facial Albedo Approximation in Constrained Environments

This is a writeup of some of the work I did during my post graduate studies. The purpose of this research was to find solutions to the problem of facial reflectance capture in realtime on constrained hardware.Where facial reflectance is albedo & roughness textures for physically based BRDFs. 

Constrained hardware refers to the use of webcams or phone cameras, in addition to laptop hardware or low power mobile chips, with limited compute and graphics processing capability. 

Albedo Approximation

The first step was to extract albedo information from images. I used pre-existing renders for this process, with accurate normals rendered for each face.  

Test images were rendered in Blender using HDRIs for lighting.

Using the normal and color data, spherical harmonics can be extracted from the image. Lighting extraction can the be performed by using inverse rendering (Marschner, Guenter and Raghupathy, 2000).

Image = Diffuse * Albedo + Specular.

Rearranging the equation by dividing by the diffuse component results in an image that contains only the Albedo and specular. Spherical harmonics are used to approximate the diffuse component and have been shown to be up to 98% accurate for this task (Ramamoorthi, 2006) . 

 Image / Diffuse = Albedo + Specular.

The result of diffuse removal is shown above. The bright highlights on grazing angles can be removed by accounting for the fresnel affect manually. 

However, a more powerful technique is to combine highlight removal into a single step with a process called corrective fields. (Ichim et al., 2015)

Corrective fields remove a lot of the artifacts introduced by grazing angles and as a bonus help to reduce the specular component that is still present in the image. 

There are still two issues with the calculated result.  

1. The image still contains specular information. 
2. The image is limited to a single angle so the texture is stretched on the sides of faces, or missing if part of the face is occluded.

Multi-view Merging

The issues resulting from simple inverse rendering can both be solved by combining the results of multiple viewpoints. 

For the 2nd issue, multi-view merging means more areas of the face a visible to the system and thus areas are less likely to be missed.

An example of multiple viewpoints merged into a single texture.

The above image combines many angles to obtain the final result. Diffuse lighting has been removed from the image, however specular highlights are still clearly visible. 

Specular Removal

An interesting fact about specular highlights is that they are view dependent, whereas albedo is not. Using the reverse rendering via spherical harmonic technique as described above. A texture containing albedo + specular can be obtained. Because albedo is not affected by viewing angle, whereas specular is. Then any variation in luminosity can be attributed to a change in specular intesity.  

In theory, by choosing the minimum value of a point on the surface given multiple viewing angles, we can find the angle with the lowest specular response and use that value to get the most accurate estimation for the surfaces albedo. 

Choosing a minimum is technically correct, but in practice, limitations of the capture process and errors in the spherical harmonic estaimation, mean that niavely choosing the darkest pixel, often results in very visible seams in the image.

Seams appear when niavely sampling based on minimum pixel intensity.

Combining techniques

The trick to obtaining the best result and to removing the specular response is reversing the above process and merging images before attempting to remove lighting information from them. 

Whilst this makes little sense from a theoretical standpoint, it ultimately proves to result in better results that are free from seams and specular highlights. 

pre-merge extration (top) vs post merge extraction (bottom)

 

Future work

I obtained some very good results when changing the problem from one of reverse rendering to a linear regression problem. However it is difficult to optimise this kind of solution to low power devices, as a sufficiently powerful CPU is required.

Weighted least squares linear solve

 

 

References 

Ghosh, A. et al. (2011) ‘Multiview face capture using polarized spherical gradient illumination’, Proceedings of the 2011 SIGGRAPH Asia Conference on - SA ’11, 30(6), p. 1. doi: 10.1145/2024156.2024163.

Ramamoorthi, R. (2006) ‘Modeling Illumination Variation with Spherical Harmonics’, Face Processing: Advanced Modeling Methods, pp. 385–424.

IchiIchim, A. E., Bouaziz, S., & Pauly, M. (2015). Dynamic 3D avatar creation from hand-held video input. ACM Transactions on Graphics, 34(4), 45:1-45:14. https://doi.org/10.1145/2766974