The Spatials V3 devblog 2015-09-28

Internal development notes, very slightly cleaned up and commented a week later.

TL;DR: In the past week the logistics system was improved, pallets for selective stockpiling were implemented, a litter plus recycler system was prototyped, and it was made clear that the current JobsSystem is not flexible enough, so it is being replaced with generic state machines.

- logistics phase 3: stock + stock delivery system (scheme)
	stock-component + station-stock-system: proto on flexible pallet object
		resource
		0 >= min-amount >= max-amount
		first sum amounts of all non-Child or ChildC poiting to us itemC over the ent rect
		under min-amount: add a ChildC to every non-child ItemC ent on the rect to block picking
		over min-amount: remove ChildC if ChildC points to us
		under max-amount: add a job ent to gather items from elsewhere
			new job comp/sys: gather-job + station-gather-system
		over max-amount: remove the job ent
	the itemC-stockC rel is not real, just a spatial query over its area, ignore based on ChildC (for passing workers, etc)
	handling of 0 sizing (must remove item ent): not required, ItemPick() already deletes depleted stacks
- enhance logistics logic in JobsSys to be able to say "pick any >0 amount, but only on a single pickup"
	set -1 on #@requiredAmount
- fix: delivery jobs for a pallet are using the same pallet as a source
	Entity* TileMapSystem::closestEntityWithPathAvoidRect(float x, float y, AspectInst* ai, int rx, int ry, int rw, int rh);
		pass pallet boundary in JobComponent::avoidItemsInRect the avoid rect params, JobsSystem will use closestEntityWithPathAvoidRect
		also ItemPickRectAvoid
- pallet: need to be able to select if the delivery job should pick from the floor, from other pallets, both, or be disabled
	item ents need a comp+sys to flag if they are over a pallet or not (in-storage-component/sys)
		low phase, point (faster) spatial query
- add an aspect in job for extra options when looking for item ents
	set it up when creating a delivery job for a pallet based on the pallet options (deliver-from-storage, deliver-from-floor)

The pallet logic system basically monitors whatever stack(s) of items are on top of itself, and in case they are of the kind it is supposed to control stock of, it will either lock them down with a ChildComponent (ItemPickXXX helpers avoid item stacks that are owned, that’s it, that have a ChildComponent), or it will remove their ChildComponents and create a job whose only purpose is to use the JobsSystem logistic support in order to transport and dump any unclaimed item stack, up to the pallet stock limit. It can also be instructed to pick items from other stacks, not just from the ground, in order to enable transfer logistics (for example you could have a pallet in a central warehouse with a large max-amount, then a smaller one in a kitchen, with a lower max-amount, in order to have a cooking ingredient nearby).

This required adding more special cases to the logistic support in JobsSystem, starting to make it very clear too much unrelated functionality was being added in a centralized system.

- comp that indicates max carry capacity for entities, so -1 deliveries dont carry arround huge stacks
	only respected by requiredAmount -1, otherwise job search would need to take it into account (for now!)
	fix picking so it passes arround the max carry for the picker ent, only when requiredAmount allows to pick any possible amount (refill jobs)

After enabling pallets to take from other pallets silly things started happening, like an officer picking up the entire stock of a pallet and dumping it in another one. For this reason entities can now optionally report a max carry limit with a component.

- (define* (ecs-query world count by-distance rect circle point (must '()) (one '()) (exclude '()) path-from filters)
	migrate all core/* scheme code to use ecs-query instead of lower level C++ calls

A much more Scheme-friendly world query function was added. Prior to ecs-query, systems logic was calling the low level C++ functions, and lack of proper RAII from the Scheme side was making this a bit painful, having to track and delete temporary C++ objects. ecs-query neatly wraps both general and spatial queries, keeping track of all required temporary cleanup, and allowing to pass custom filter lambdas. It also has specialized support for the most expensive filter, path-from, so it’s left for last, even after the (optional) distance sorting.

- clean up some debug tool inspector stuff and enhance it
	write support for scheme values
	filter out useless stuff (dirtyTickStamp, serializable, bridgeSerialized)
	write support for C++ values
- hover picker hl: visually paint the PosC boundary
	prototype in the debug picker for now
	add some TintC: bugs out bodies in officers, do not use TintC for this, or any other setColor stuff, for the final info cursor

All development of station features has avoided, for now, coding any kind of object-specific sidebar UI. This work is waiting for more systems to be implemented in order to be sure of what shape the interface needs. In the meantime there is a super ugly “debug inspector” that gets the job done. It has now gained write support and I also played a bit with it in order to test some object selection/cursor ideas.

The screenshot shows it displaying a component from a pallet. Since the component is implemented in Scheme, its C++ class is BridgedComponent. The previously discussed amount limits and picking preferences can be clicked to input a new value, which in this case is then eval-ed as a Scheme expression.

- proto litter system
	add litter comp
	random sprite, with random h flip
	it is also ItemC? watch out the display sys
		just exclude *PoseComponent_Kind in the DS
- litter cleaning sys/job
	recycler gives the job, it's just a resource gathering system
	resources awaiting refactor, for now just peg it to an existing resource
	add picking animation??
		this doesnt feel good, at all

One of the planned V3 systems is having your station inhabitants drop litter on the floor during the course of their daily activities. One of the facets of this system is having to keep your station clean, so part of your work force should be ready to find and pick up this litter. I started to prototype this system by tackling it on top of the existing JobsSystem, since the act of finding and picking litter is similar to that of resource logistics. It worked, but when I wanted to add a custom picking animation for litter, it just feel wrong adding yet another specific toggle/field to JobComponent and the extra check/code in the JobsSystem. It was clear at this point JobsSystem was picking up too much stuff (heh) so it needed to be refactored or replaced.

- prototype a generic, simple state machine library in scheme
	args for passed args per-execution
	state machine-level vars for the serializable state (this is returned by init-state-machine, must be passed to run-state-machine)
	node-level vars eval-ed at every run of a single node
- state machine: "extend" for running a node before another one, and keeping vars eval-ed
	if nothing matched, node resumes its execution
	original node keeps the vars from the extended node exec
	super useful for invariants, like keeping track of valid external state
	and for refactoring vars
- port litter system to a state machine use state machine in litter system
- fix extend: really take into account state changes made by extend
- endless loop in litter, why?
	going back to find-litter, counters still at max -> easy to fix
	find-litter finds everything ready: worker at right pos, and litter still there
		why? removeEntity removes from world set, calls remove on SIS, but SIS needs applyChanges to actually update its index
		so ecs-query will still find the same litter just there
		add check in SIS::queryCoarse in order to bring it to the same level of freshness as non-spatial queries
	second bug: BusyC was "sticky" in recycler ent
		super core bug: Entity::getByKind was checking components before add, so it was returning the old, removed comp
			consequence of implicity assumptions about lifetimes
- litter: heading lookat

The choice was made to replace JobsSystem with specialized state machines, making sure they can be parametrized and extended when required, and adding as many ECS helper procedures as required to keep the code concise. The new system looks much better and it will make it much easier to add specialized code, while allowing related objects (for example, shops) to share the same state machines.

The litter-recycler system was rewritten using the new state machine library, and the resulting code is very readable and concise. All world-effecting code is pure component-entity manipulation, keeping the ECS decoupling intact.