NimbyScript progress

NimbyScript development continued in June. I started work in one big item, memory ownership, because it is closely related to the already implemented borrow checker, but it was giving me a lot of trouble, so I decided to first tackle some other smaller features which were much required anyway.

• Formalize a standalone compiler build (as standalone as possible) to run the test suite. NimbyScript can now be built as its own compiler binary, independent of (most) of the game, but not completely. A standalone build already existed, but it was more like a special build of the game. This is now instead a proper independent build which uses a few source files from the main game tree, and even these files have been structured to be separated in the future if I wanted.

• Greatly expand the test suite. The standalone build is used to run the test suite of the language, which is a mix of manual tests and automated repetitive tests. There’s now hundreds of tests, although at the moment only the compiler semantics are being tested, not the execution of the code itself. This is not a bad thing; for example some tests have compiler errors, to make sure not just perfect code is being tested, and the test runner verifies the compiler gives the expected error. A test expecting an error but being compiled fine is considered a bug in the compiler!

• All the 8 bit to 64 bit integer types, and 32 and 64 bit float types are now implemented, with proper type checking for operators. I decided to make math operators have strict type matching, so you cannot add together i8 and i32, this is considered an error and you must first convert one of the two operands (with provided compiler built-in functions). For floating point to integer conversion you must also explicitly choose between floor, ceil and round. For comparisons I decided to be more lax, and you don’t need conversion most of the time.

• Final form of variable declaration:

let name = exp; // previously var name = exp;
let name &= exp; // previously ref name = exp;
let name &mut= exp; // previously ref mut name = exp;

I finally settled on a form of variable declaration that didn’t look and feel like its own type system, compared to actual type declarations. It is now closer to Rust, with the important difference that I don’t want to have reference operators, so the reference specifier is at the left of the equals (it’s a different token so let name &mut = exp is just as valid).

• Bit-copyable plain C structs. As a previous step to restarting work on ownership semantics, I implemented support for simple bit-copy of structs. If a struct is declared from the C++ side as bit-copyable the compiler now allows to copy it like any regular old C struct. Some small types which don’t own memory can benefit from this simplified handling, and don’t need any extra APIs to be manipulated.

NimbyScript pause

After going back to thinking about and prototyping memory owning semantics, I just gave up. I was about to enter another mutli-week cycle of design-prototype-test-delete like I did with borrow checking, but this time I was a lot more aware about the problems I was going to find. The design I wanted to implement was like a simplified version of move-only types with auto drop, similar to Rust, but less fancy, so you would still need to manually drop and move values in some cases if the compiler could not automatically figure things out (if/else, looking at you).

But this is a lot harder than borrow checking, as strange as it sounds. With borrow checking I can get away with mandating rules like “if a reference exists at some point inside a scope, it is always valid to the end of the scope”, but move/drop-able type means any bind (variable or reference) can now become invalid from things like being a parameter to a function call, in their own scope (the reference-only equivalent of this problem always happens in a nested scope, which makes it a lot easier to check lifetimes).

I have implemented let else (only for pointer checking) so NimbyScript already has some degree of support for invalidating or at least hiding bindings, but then I also had to think about dropping, and construction and destructors. It is a huge problem so I decided to stop for now. What is going to happen? After 1.17 I will revisit this idea, or implement “plan B”: NimbyScript cannot own memory. Instead some hidden C++ context object owns all memory allocated by script vectors/list/etc, which can never survive a single invocation context. This could be a dumb, very fast bump allocator, for example.

NimbyScript is, at the moment, “half a language”. It’s missing fundamental building blocks (cannot have runtime-sized types, cannot do looping, cannot properly express iterators), but it’s perfectly capable of being used for signal logic as long as it delegates the heavy lifting into C++ APIs. Which I believe it’s not a bad place to stop the design at, after adding a few more things. I’m not planning to rewrite the game in NimbyScript, and I don’t plan on supporting “total conversion” mods written in NimbyScript. Unlike some C# games, which expose the entire guts of the game logic, NimbyScript will always be limited by a set of purposely designed APIs, set in stone, and access to game objects will be limited and read-only in many cases.

1.17: Non-player testing of NimbyScript

How to test NimbyScript before it’s ready for players? In 1.17 path signal logic will be implemented in NimbyScript, with an internal script source which can be viewed in the new Control editor, but cannot be edited. This will test all of the existing pieces of NimbyScript and its game integration, without opening the floodgates for player scripts while the language is not finalized.

1.17 is going to be a weird version. There’s huge, risky internal changes but no major new or changed playable systems. It’s a small version from players POV, even tiny, but very needed to make sure the whole NimbyScript idea is technically sound in its current prototype form, and what issues can arise from runtime compiled-and-loaded C code. It will also be a performance test, to make sure existing saves do not regress when signal logic (at least the top level) is moved to NimbyScript, in absence of any player customization of said logic.

But to make the version a bit less boring, I’ve also implemented a couple all time favorite feature requests, which are going to make some players very happy (I think).

1.17: Recolorable UI

It is now possible to change the colors of the UI:

To make this work with the UI icons, most of the icons are now being processed as an alpha mask rather than a full color icon (the decision to keep them monochrome just paid off immensely here), which enables them to be recolored with the custom text color. A few of the icons were already recolorable for other reasons, like the line icon, so these remain hardcoded to black. Also on-map UI elements are a lot tricker to change so they also remain non-recolorable.

The button 9-patch image, the basis for almost every element in the UI, was also full color, so it was not recolorable. To make it work I’ve changed to a different design which recolors OK as long as it’s not a very dark color. For dark mode styles, consider keeping buttons a little bit brighter, like I propose in the previous screenshot.

1.17: Alert listing

Alerts can now be viewed in a listing:

All queued alerts are immediately available in the listing, and older alerts are also kept there, past the new alerts and any “missed” alert (shown but not interacted with). There’s a limit of 200 alerts, and they are automatically cleared as new ones are pushed at the front, or you can just clear them manually.

1.17: Committed to a July release

After deciding to pause and prepare NimbyScript for a non-player public test, the scope of 1.17 has been reduced greatly, so a July release is now possible. It’s been awhile so I am committing to a July release. There’s still a few more things I want to do in 1.17, and I will decide in the coming days if they go in 1.17.1 or into the beta builds, but in any case 1.17 is not too far off now.