The Spatials V3 devblog 2015-09-28

Internal development notes, very slightly cleaned up and commented a week later.

TL;DR: In the past week the logistics system was improved, pallets for selective stockpiling were implemented, a litter plus recycler system was prototyped, and it was made clear that the current JobsSystem is not flexible enough, so it is being replaced with generic state machines.

- logistics phase 3: stock + stock delivery system (scheme)
    stock-component + station-stock-system: proto on flexible pallet object
        0 >= min-amount >= max-amount
        first sum amounts of all non-Child or ChildC poiting to us itemC over the ent rect
        under min-amount: add a ChildC to every non-child ItemC ent on the rect to block picking
        over min-amount: remove ChildC if ChildC points to us
        under max-amount: add a job ent to gather items from elsewhere
            new job comp/sys: gather-job + station-gather-system
        over max-amount: remove the job ent
    the itemC-stockC rel is not real, just a spatial query over its area, ignore based on ChildC (for passing workers, etc)
    handling of 0 sizing (must remove item ent): not required, ItemPick() already deletes depleted stacks
- enhance logistics logic in JobsSys to be able to say "pick any >0 amount, but only on a single pickup"
    set -1 on #@requiredAmount
- fix: delivery jobs for a pallet are using the same pallet as a source
    Entity* TileMapSystem::closestEntityWithPathAvoidRect(float x, float y, AspectInst* ai, int rx, int ry, int rw, int rh);
        pass pallet boundary in JobComponent::avoidItemsInRect the avoid rect params, JobsSystem will use closestEntityWithPathAvoidRect
        also ItemPickRectAvoid
- pallet: need to be able to select if the delivery job should pick from the floor, from other pallets, both, or be disabled
    item ents need a comp+sys to flag if they are over a pallet or not (in-storage-component/sys)
        low phase, point (faster) spatial query
- add an aspect in job for extra options when looking for item ents
    set it up when creating a delivery job for a pallet based on the pallet options (deliver-from-storage, deliver-from-floor)

The pallet logic system basically monitors whatever stack(s) of items are on top of itself, and in case they are of the kind it is supposed to control stock of, it will either lock them down with a ChildComponent (ItemPickXXX helpers avoid item stacks that are owned, that's it, that have a ChildComponent), or it will remove their ChildComponents and create a job whose only purpose is to use the JobsSystem logistic support in order to transport and dump any unclaimed item stack, up to the pallet stock limit. It can also be instructed to pick items from other stacks, not just from the ground, in order to enable transfer logistics (for example you could have a pallet in a central warehouse with a large max-amount, then a smaller one in a kitchen, with a lower max-amount, in order to have a cooking ingredient nearby).

This required adding more special cases to the logistic support in JobsSystem, starting to make it very clear too much unrelated functionality was being added in a centralized system.

- comp that indicates max carry capacity for entities, so -1 deliveries dont carry arround huge stacks
    only respected by requiredAmount -1, otherwise job search would need to take it into account (for now!)
    fix picking so it passes arround the max carry for the picker ent, only when requiredAmount allows to pick any possible amount (refill jobs)

After enabling pallets to take from other pallets silly things started happening, like an officer picking up the entire stock of a pallet and dumping it in another one. For this reason entities can now optionally report a max carry limit with a component.

- (define* (ecs-query world count by-distance rect circle point (must '()) (one '()) (exclude '()) path-from filters)
    migrate all core/* scheme code to use ecs-query instead of lower level C++ calls

A much more Scheme-friendly world query function was added. Prior to ecs-query, systems logic was calling the low level C++ functions, and lack of proper RAII from the Scheme side was making this a bit painful, having to track and delete temporary C++ objects. ecs-query neatly wraps both general and spatial queries, keeping track of all required temporary cleanup, and allowing to pass custom filter lambdas. It also has specialized support for the most expensive filter, path-from, so it's left for last, even after the (optional) distance sorting.

- clean up some debug tool inspector stuff and enhance it
    write support for scheme values
    filter out useless stuff (dirtyTickStamp, serializable, bridgeSerialized)
    write support for C++ values
- hover picker hl: visually paint the PosC boundary
    prototype in the debug picker for now
    add some TintC: bugs out bodies in officers, do not use TintC for this, or any other setColor stuff, for the final info cursor

All development of station features has avoided, for now, coding any kind of object-specific sidebar UI. This work is waiting for more systems to be implemented in order to be sure of what shape the interface needs. In the meantime there is a super ugly "debug inspector" that gets the job done. It has now gained write support and I also played a bit with it in order to test some object selection/cursor ideas.

Debug inspector

The screenshot shows it displaying a component from a pallet. Since the component is implemented in Scheme, its C++ class is BridgedComponent. The previously discussed amount limits and picking preferences can be clicked to input a new value, which in this case is then eval-ed as a Scheme expression.

- proto litter system
    add litter comp
    random sprite, with random h flip
    it is also ItemC? watch out the display sys
        just exclude *PoseComponent_Kind in the DS
- litter cleaning sys/job
    recycler gives the job, it's just a resource gathering system
    resources awaiting refactor, for now just peg it to an existing resource
    add picking animation??
        this doesnt feel good, at all

One of the planned V3 systems is having your station inhabitants drop litter on the floor during the course of their daily activities. One of the facets of this system is having to keep your station clean, so part of your work force should be ready to find and pick up this litter. I started to prototype this system by tackling it on top of the existing JobsSystem, since the act of finding and picking litter is similar to that of resource logistics. It worked, but when I wanted to add a custom picking animation for litter, it just feel wrong adding yet another specific toggle/field to JobComponent and the extra check/code in the JobsSystem. It was clear at this point JobsSystem was picking up too much stuff (heh) so it needed to be refactored or replaced.

- prototype a generic, simple state machine library in scheme
    args for passed args per-execution
    state machine-level vars for the serializable state (this is returned by init-state-machine, must be passed to run-state-machine)
    node-level vars eval-ed at every run of a single node
- state machine: "extend" for running a node before another one, and keeping vars eval-ed
    if nothing matched, node resumes its execution
    original node keeps the vars from the extended node exec
    super useful for invariants, like keeping track of valid external state
    and for refactoring vars
- port litter system to a state machine use state machine in litter system
- fix extend: really take into account state changes made by extend
- endless loop in litter, why?
    going back to find-litter, counters still at max -> easy to fix
    find-litter finds everything ready: worker at right pos, and litter still there
        why? removeEntity removes from world set, calls remove on SIS, but SIS needs applyChanges to actually update its index
        so ecs-query will still find the same litter just there
        add check in SIS::queryCoarse in order to bring it to the same level of freshness as non-spatial queries
    second bug: BusyC was "sticky" in recycler ent
        super core bug: Entity::getByKind was checking components before add, so it was returning the old, removed comp
            consequence of implicity assumptions about lifetimes
- litter: heading lookat

The choice was made to replace JobsSystem with specialized state machines, making sure they can be parametrized and extended when required, and adding as many ECS helper procedures as required to keep the code concise. The new system looks much better and it will make it much easier to add specialized code, while allowing related objects (for example, shops) to share the same state machines.

The litter-recycler system was rewritten using the new state machine library, and the resulting code is very readable and concise. All world-effecting code is pure component-entity manipulation, keeping the ECS decoupling intact.