Enabling Direct3D12 and Vulkan renderers

With 1.10 a major cleanup of the rendering code came about, specially around texturing and shaders. I was then able to diagnose what was preventing the game from using the existing BGFX Direct3D12 and Vulkan renderers. For Vulkan, I discovered the game was running out of shader uniform space, due to the fact the Vulkan renderer using the default Vulkan memory layout rather than a custom one with a custom allocator. I was able to cut down on the number of uniforms (they are used as a material LUT so they can get large), but limited the fix to Vulkan.

For Direct3D12 I discovered an actual bug in how BGFX uploads dynamic texture arrays, which are used very intensively by the game. I developed a fix and submitted it upstream.

Didn’t make it: rounded line caps

During October I keep implementing features in the new map that had less priority or were technical in nature, but one of them was fully implemented and wasn’t enabled in the end: rounded line caps.

These are rendered using the same SDF line code as line corners, by extending the line end point a bit, and are as pixel perfect as the corners. They look very good in other words. Unfortunately the game OSM database strips continuity information between objects, so ways at different layers were displayed like this:

Even if that information was kept in the game data, there is nothing in the OSM data model that forces OSM mappers to keep a discipline with their node IDs and continuity between ways, since it’s a database designed for viewing, not for planning. It’s likely this would keep happening in some instances.

Proper stitching of track segments in curves

A little detail which has irked me for a long time: track segments are not 100% seamless, and that’s visible on short radius curves. But with the new 1.10 line drawing system this detail can now be fixed:

Texture compression

Starting from 1.10 the game now uses GPU texture compression, for VRAM savings of 4x for textures. The choice of codec is very conservative, with only BC1 and BC3 supported, which are 20 year old formats. BC7 was also a higher quality possibility and in theory all DirectX11 GPUs are mandated to support it, but I’ve seen enough strange stuff coming from low end GPUs that I have my doubts about “support” meaning CPU decompression and uncompressed VRAM upload in this case.

Texture compression has been applied to train textures, which are very large but spend most of the time displayed in a small size with little detail visible. In the same vein the same 2-tier resolution system used for tracks is now also used for trains, with the low resolution texture array having space for thousands of simultaneous train textures, but the high resolution only for 256. The high resolution texture is mostly used in the UI, with the map view almost always set to a zoom level which allows to use the low resolution array.

Texture VRAM usage has fallen more than 50% in 1.10 compared to a freshly started 1.9, and even more after a saved game is loaded and the map is scrolled around. This will hopefully help players with less capable GPUs or with integrated graphics.

Blueprint clipboard and collection fixes

A long standing issue with how blueprints are copied into memory and files has been solved in 1.10. The original code for this feature had the right idea: do not store the coordinates (lon-lat or Mercator, does not matter) of the track points. Instead find the geometrical center of the clip and store a direction vector and distance measure for every node. And this would work if the Earth was flat. And if would work in our pseudo spherical Earth if the proper re-projection was done on both copy and paste. But it wasn’t, so track nodes accumulated more and more error the further from the center.

In 1.10 track clips are stored in the same way, but this time proper formulas are used to respect the projection. The “aviator’s bearing” formula is used for direction, and the haversine distance formula for the distance. In essence the game now stores the clip track nodes as if they were on a spherical cap, as it should. This now allows to paste clips with very little distortion, both in the same original spot or moving them anywhere in the world.

“Any train” pax boarding

Remember the pre-1.5 “any train” pax pathfinder? A distant relative of it is now deciding what train pax board in 1.10. The 1.5 pathfinder is still deciding the overall path a pax takes. So pax on a certain station will always want to reach their next stop, never changing their mind about the next stop (unless time passes and it becomes sub-optimal). But when faced with a specific train, pax now can ignore the exact train/run/stop determined by the 1.5 pathfinder. They will instead perform a new kind of mini-pathfind, and ask the train:

• Can you take me to my next stop without any train changes? If yes:
• Can you do it before the departure of my next connection train? If yes:
• Can you do it without starting any technical line in the way? If yes, then the train is valid.

After implementing some small caches this new mini-pathfind became fast enough, and it is now always enabled.

Didn’t make it: satellite photo layer

I am dissatisfied with how the medium zoom levels look in the game. LOD 0 looks good thanks to OSM and the new map style, and LOD 5+ looks okay with the land texture at its 1:1 resolution. But the intermediate LOD levels don’t look very good. The land texture is too low resolution for them. And I had to remove a lot of OSM data from LOD 1 and higher to reduce the map file size. So I decided to try to fit a satellite photo layer at LOD 3 with a 400x400 resolution, which would give 100m per pixel, acceptable for a cosmetic layer. This could fit in a 2GB budget if using JPG compression. Again, it’s only for cosmetics, so JPG was okay.

It turned out that the free sources of global satellite photos are unprocessed. This means that it’s often the raw images from the satellite sensors. Immensely detailed, immensely huge, very slow to process, and very often with clouds and shadows. Processed cloud-free datasets are often commercial-only. But in the end I found a processed photo dataset from Sentinel-2 data from the same researchers that made the population layer already used in the game. Indeed, it is the source for some of their data. And they offered downloads with OVR (overview) files already processed, so I would not need to wait for multiple days while I downscaled the 10m sources into my required 100m resolution, or require some multi TB storage in my server. Here’s the result:

The flaws were immediately visible in North America and a few other places. I didn’t detect them in my initial testing, since I was downloading only a few tiles and those were not affected. It turns out some of the OVR files are corrupted in the server. I would have needed to identify them and re-create them from the source data. I decided instead to shelve the project, and maybe resurrect it in the future. It’s not gameplay, and the new map system allows me to add or change map layers without a full re-download. In any case I took some screenshots to show it off:

Another possible un-fixable flaw is that mountains in the photos are shadowed, and these shadows do not necessarily match the game hillshading, but it does look good anyway. I also experimented with tweaking the map color balance, making a HSV saturation and value controls available in the map options to obtain results like this, more in line with the rest of the game palette:

Another possibility I will consider is purchasing a commercial dataset. I saw “outdated” ones (2016) starting at 1500 EUR. I assume they are cloud-free, and if they are also reasonably shadow-free in mountain ranges, it could be an acceptable alternative for the price, since then I avoid reprocessing the damaged tiles.