All notable changes to this project will be documented in this file. This project adheres to Semantic Versioning.
x/platform/syscall has a scope of its own, and its open-flag table is an
export its readers import (#719, step 4). The per-OS O_* table was
%file-modes, a private name three other files read from the root. It is
now file-modes, a plain export: x/sys/file and x/sys/posix import it at
their top, (import x/platform/syscall file-modes), and the boot loader’s
%module-mode imports it when it runs, since boot/module.x loads long
before the table does. (File file-modes) answers as before.
os-linux?, arch-arm64? and arch-x86-64? without
providing them, and other files use all three. They are in the provide
list now, marked (global ...) with the file’s other sanctioned names.(import NAME), so (import NAME sym ...) in an amalgamated file
would have been left unspliced and its module loaded from the platform at
boot. It splices the module as for a bare import – ahead of a scoped
importer, in place in an unscoped one – and keeps the line, because binding
the names is the line’s other half and the load is a no-op once the splice
has marked the module loaded. The three library amalgams were byte-identical
before and after the generator change; the amalgam smoke check covers both
placements on a throwaway tree.check-bare-globals counts a scoped module’s marked names only. A
scoped module’s other bare definitions are its own, exported for import or
not, and need no manifest row; file-modes is the first such name in the
files it scans. A scoped file that defines a sanctioned name and stops
exporting it is refused, which is the mistake the three predicates above
would have been.%-budget row is retired.A compile that carries fvars declares its calling world. With no third
argument compile-asm read the fvar table to choose between an integer
function and an analyse callback – present meant analyser – though an fvar
also names a callee the body calls (#603), so the table cannot say which. The
reading stayed as a default for x-python’s tokenizer, whose compiles were
undeclared and adopted under a guard that would have hidden a refusal. That
bundle now declares every compile
(x-python#144), and its specs
require the JIT to be active wherever the lane compiles an analyser
(x-python#138), so the door
stops guessing: fvars with no third argument refuse, naming both declarations.
A compile with no fvars is an integer function, as before.
x/type/vector, x/protocol/str/str8 and x/type/type have scopes of
their own (#719, step 4), and two doors replace the five private names
other files read from the first two. (Convert to-int v what) is a value
as an INT, converting through the registry and raising what when there is
no conversion: the coercion x/type/array, x/type/gen and x/protocol/str/utf8
had each borrowed from a neighbour’s private (%vec->int, %str8->int).
The hot callers test for an INT themselves and reach the door only to
convert. (Str8 %check v what) is the string check StrUtf8’s own paths
used through str8’s private, homed on the class as (Iter %check) is.
x/doc/doc-gen renders with (Str str ...), which is what str8’s
%str-build did. The type-convert spec asks (Type of #(0)) for the vector
type instead of vector’s private handle. Four rows of the private-read
budget go down: array to none, utf8 by two, gen and doc-gen by one each.
Cross-file reads of private names may only decrease (#719). Until a
module is scoped, its % names are globals, and other files read them:
127 files read 954 such names today, most of them the number tower reading
its neighbours, the compiler’s stages resolving free names in the root, and
the dev tools driving internals. Step 4 turns each read into a door, and
check-private-reads keeps the count from growing meanwhile: for every
reader file, tools/contract/private-reads.x budgets the distinct %
names it reads that another unscoped file defines at its top level and it
does not define itself. A file over its budget fails and names the reads;
a file under it fails until the row is lowered. It runs with the fast
gates, so a pull request sees it.
Bracket colours carry across the lines of a multi-line entry. Each line of
an entry was marked and painted on its own, so a continuation line started
again at depth 0, a close paren that closed a paren on an earlier line showed
bold red as unmatched, and a line continuing a string was painted as though a
new string began at its closing quote. Line read takes the lines already
entered as an optional second argument, and the REPL’s continuation read
passes them. The marker is asked about the whole entry and the editor keeps the
marks that fall on the line being edited; the painter is handed the text that
comes before what it paints, the earlier lines and any part of the line
scrolled out of view, and starts from whatever string or comment that text
leaves open. The painter’s cache compares that text by identity, so moving the
cursor still repaints nothing. %repl-marks now receives the whole entry, and
%repl-paint a third argument, which a painter written for fewer ignores.
A session started without -l has the line editor. lib/x.x, the
default entry, boots helium, but its launcher lacked the
(import x/repl/line) that lib/he.x, lib/xe.x and lib/rn.x carry, so
a bare sh x.sh read lines through the plain loop, without the editor’s
keys, history, Tab or colour. Its launcher is now the same as theirs.
A scoped module’s plain exports stay its own (#719). provide
records every export in the registry, where a selective import finds it,
and now binds in the root only what docs/namespaces.md says it should: a
class, and a name the provide list marks (global NAME). The marks are for
the sanctioned bare set, and check-bare-globals holds them to it both
ways. Any other export of a scoped module is reached with
(import MODULE NAME), which copies it into the importer’s own frame.
quasi, iter, def-record, def-trait, trait?, def-generic, on,
throws? and raised.x/tool/cov’s and x/tool/profile’s functions are imported by the tool
script and the specs that call them, and x/reader/indent stops
exporting its four catalog functions.check-dup-defs counts,
for a scoped module, only what reaches the root.x/sys/posix has a scope of its own (#719, step 4), hiding
thirty-two names. Three other files read two of them. x/repl/term and
x/sys/socket sign-folded their libc int returns with %sys-fold; they now
call (Sys %sign-fold r), a static on the file’s own class, one class call
beside each syscall. x/tool/compile fetched libc’s unlink with %resolve
to remove its generated C source; it now calls (File unlink), inside a
guard because File raises on a failure the raw call ignored.
The arithmetic refusal costs a boot nothing again. (Type
refuse-arithmetic!), added above, installed its thirty handlers through
(Type push-op) and (List for-each), a class call per operator, and that
added 0.7% to the evaluations of every x-core boot. It now calls the push-op
primitive itself in a plain loop, and an x-core boot takes 22,437,175
evaluations, fewer than the 22,451,999 it took before the door existed,
counted with the engine’s profiling build. The messages are unchanged.
The %-budget and the duplicate-definition gate look only at what stays
global (#719, step 5). A scoped module’s top-level definitions bind in
its own environment, so its %-names never reach the global tree.
check-percent-globals no longer counts a file whose first form is
(module NAME), and the thirty-seven rows such files held, 427 names, are
retired. check-dup-defs checks a scoped module for the names it provides
and nothing else: two scoped modules may share a private name, and a scoped
module may reuse a global’s, while a provided name that an unscoped file
also defines is still refused. The third part of the step needed no change:
lint already reports another module’s private name as undefined for every
scoped module, because that module’s environment is not the linter’s.
x/type/str-utf8 has a scope of its own, and its list conversions are
catalog doors (#719, step 4). The file loads before the object system,
so there is no class to carry what other files need from it. It publishes
its two list conversions beside (str cp-len) instead: (str from-list)
builds a UTF-8 string from a list of code-point characters, and
(str ->list) decodes a string into one. x/type/char-io, x/type/convert
and x/type/iter each fetch the one they use once, at load, where they used
to read the file’s private names. With that, the file carries the
(module NAME) header and hides fourteen names.
A non-numeric type refuses arithmetic through one door. (Type
refuse-arithmetic! ts tname) registers a raising handler for each of
+ - * / % < on a type struct, so (+ 1 "a") answers “no + for STRING”
rather than falling through to integer arithmetic on the string’s pointer
(#52’s refusal, now a method on the class that owns the type registry).
x/core/op-guard calls it for strings, lists, pairs and vectors, and
x/type/bool for BOOL. Before this, bool installed its refusals with three
private names of op-guard’s; op-guard now defines no global at all, and a
lang that defines a type of its own can call the same door (#719, step 4).
Six more modules have a scope of their own (#757). x/reader/analyser,
x/tool/cov, x/reader/indent, x/type/list, x/type/char and
x/type/ptr carry the (module NAME) header (#719, step 4), hiding
forty-one private names between them. None needed a door: nothing outside
them reads a name of theirs, apart from per-file catalog aliases that an
unscoped file still binds in the root, and every public name other code
uses was already provided. x/tool/cov walks whatever environment its
caller hands it, so what it reports is unchanged. Booting costs 0.02% to
0.05% more evaluations, counted rather than timed: 22,428,516 to
22,440,486 for the x-core amalgam, 60,240,457 to 60,252,679 for x-base,
and 22,624,448 to 22,634,898 from source.
The assembler loads and stores at every width (#756). x/tool/asm
had eight-byte ldr/str and one-byte ldrb/strb; it now has the
widths between, on both backends: ldrh, ldrsh and strh for two
bytes, ldrw, ldrsw and strw for four, and ldrsb for a byte
sign-extended. The names are arm64’s own, except the four-byte pair,
which arm64 writes as LDR and STR of a W register: the portable operands
are X registers only, so the width goes in the name, as it does for a
byte. A load fills all 64 bits of its register, and a store writes its
width and nothing past it. Every one is a table entry, and
asm-widths.spec.md runs on both hosts.
Four more modules have a scope of their own (#755). x/type/iter,
x/sys/stream, x/sys/socket and x/sys/file carry the (module NAME)
header (#719, step 4), hiding sixty-two private names between them.
Three needed nothing: nothing outside them read a name of theirs. The
fourth did. %as-list in x/core/list drained an iterator through
x/type/iter’s %i-empty? and %i-next, and both of those are fetches of
engine primitives, so list.x fetches them itself rather than reading
another module’s names; the duplicate-definition gate allows several files
fetching one catalog entry, and the budget for core/list.x goes from 21
%-globals to 23. In the same spirit the Ansi class carries inverse
beside bold and dim, and x/repl/paint takes the cursor-pair colour
from the class, as it already took every other colour. Boot time is
unchanged: best of ten alternating runs against the base commit.
The r5rs three are walked to a release (#754). The note this file and
tools/contract/langs.x carried named x-engine-c v0.2.11 for “an
environment is a value”, taken from the entry below, which names the pin
#720 ends on rather than the release the feature shipped in. The engine’s
own changelog puts it in 0.2.10 (x-engine-c#49, closing #46); 0.2.11 is the
follow-up that makes a name found by identity (#52). Both rows now say
0.2.10, and both carry the walk that settles it – x-r5rs held at dd3ba79,
each x-lang commit built against the engine its own pin names:
x-lang pin result what it is
v0.14.0 v0.2.8 667/0 the last green pairing
18f0b530^ v0.2.8 667/667 source already migrated, pin behind
18f0b530 v0.2.10 667/3 the pin that takes the new env model
edaabc2a v0.2.11 667/3
d6fdd68d v0.2.12 667/3
ef0cc505 v0.2.13 667/3 where this tree sits
Nothing since v0.2.10 has moved the number in either direction. The two 667/667 rows are the limit of the method and are recorded as such: v0.14.0’s source does not run on v0.2.10 and the migrated source does not run on v0.2.8, so the engine release and this tree’s adaptation to it land together and pinning cannot separate them. This names a release, not yet a line.
Two lang budgets ratchet to zero, and the r5rs three are ours (#752).
tools/contract/langs.x recorded one failure for x-python and two for
x-ash; both bundles have fixed theirs, and the rows now say 0. The test
counts are left where they are. They are floors, not censuses, and the file
says why: a floor set to a moving high-water mark cries wolf.
x-cc’s row is unchanged too, and the reason is worth recording. The gate
reported its suite had shrunk 119 -> 117 and asked for a re-record. It had
not shrunk. check-langs reads ../languages, and every one of the
thirteen checkouts there was behind its remote – x-coreutils by 30 commits,
x-cc by 24, and five of them parked on a shared feature branch rather than
main. Measured against fresh checkouts at each bundle’s origin/main,
x-cc runs 153 specs, x-coreutils 444 and x-ash 974. Re-recording 117 would
have written a stale working copy into a contract that may only shrink.
x-r5rs is red at 667/3 and the budget stays 0, because the three are this
platform’s. They are the R5RS pitfall 3.2 cases, which assert that a
definition a macro introduces does not escape the let it was written in.
x-r5rs fixed exactly those in its own repository on 2026-09-08 and measured
667/0 against x-lang main on x-engine-c v0.2.8, the engine v0.14.0 pins.
This tree pins v0.2.13, and the three arrive with “an environment is a
value” – x-engine-c#49, released in v0.2.10 and carried here by #720: a
def binds in the current environment, an eval with an environment makes
it current with the binding staying put, and the frame marks, shadow list
and local boundary that those fixes were written against were retired with
it. x-r7rs gained the same three in the same window. A budget raised to 3
would record our own regression as the bundle’s.
check-langs has never run in CI and does not now: make gates runs on
main only, a CI checkout has no ../languages, and the gate prints SKIPPED
and exits 0. It is a local gate, and these numbers are only ever as good as
the checkouts beside the tree.
The numeric tower is right at the most negative integer (#748).
LONG_MIN is the one int whose negation is not an int, and every promotion
route reached it through its own magnitude. %bigint-from-int took (- 0 n)
first, which for LONG_MIN hands back LONG_MIN; the limb loop then divided
a negative number, whose remainders are negative too, so each limb came out
carrying the sign. The bigint that built printed as -9-223372036-854775808
and did arithmetic to match: (+ LONG_MIN 18446744073709551616) answered
n itself and are negated one
at a time, which is always in range because a limb is smaller than the base.%would-overflow-mul? compared magnitudes through the same wrapping abs, and
read LONG_MIN as negative on both sides. On the left that only promoted
products that did not need it, and they demoted again; on the right a
negative operand never exceeds a bound, so the check passed and the product
wrapped – (* 2 LONG_MIN) was 0. It now divides the bound the product would
cross by b, after answering the two cases that no division can: -1 is the
one multiplier that overflows LONG_MIN, and LONG_MIN / -1 the one
division that would overflow.
Unary - took a fast path that negated a plain int in place, which left
(- LONG_MIN) as LONG_MIN; it now goes through the same binary the
variadic fold uses, so it takes that binary’s overflow check. Binary / had
no overflow check at all, and LONG_MIN / -1 is undefined in C – the prim
under it evaluates a plain a / b, which answers LONG_MIN on arm64 and
traps on x86. That pair now promotes like any other overflow. Num abs and
Num negate are right as a consequence: both are (- 0 n).
(% LONG_MIN -1) is left alone. It answers 0 on arm64, and % reaches the
prim with no wrapper on purpose; the undefined division under it is the
engine’s to guard.
Nine more modules have a scope of their own, and amalgams hoist a scoped
file’s imports. Eight boot files, x/type/dict, x/type/bool,
x/type/err-io, x/type/char-io, x/type/hash, x/core/math,
x/protocol/str/utf8 and x/type/regex, and the run-time x/codec/json,
carry the (module NAME) header (#719, step 4). Nothing may be spliced
inside a scoped file in an amalgam, since its header reads every form after
it into the module. The generator used to refuse such a splice. It now
splices the files a scoped file includes once or imports at top level
ahead of it, in order, and the lines inside the module become the usual
“inlined above” comments. That is what lets x/type/dict join: the bitwise
app imports it before x/type/hash is loaded, and hash is now spliced
ahead of dict. A plain include inside a scoped file is still refused, and
the amalgam smoke check covers both cases. A header under a plain include
now takes the module it names, as it already did under include-once: the
tower dialects load x/type/hash and x/type/regex with include from
boot/tower-compiled.x. include takes the module the caller expects as a
second argument, which is how import asks for one, so the mark a header
reads belongs to the one file being loaded and an import’s name is never
asked of the files that file loads in turn. Boot time is unchanged: best of
ten alternating runs against main, the source boot, the x-core amalgam and
the x-base amalgam differ by 0.0%, 0.0% and -0.7%, and loops over dict,
strings, display, Num and regex are within 4% either way.
The first boot files have a scope of their own. x/core/quasi,
x/type/record, x/type/err, x/type/assoc and x/type/gen carry the
(module NAME) header (#719, step 4). x-core loads most of them with include-once,
which names no module, so a header under include-once takes the module
it names. In the amalgams a scoped boot file is spliced in place, and the
generator now writes (%module-expecting! (lit NAME)) before it and
(%module-end) after it; the header reads up to that marker, since the end
of the spliced text is not the end of the stream. The generator refuses to
splice another file inside a scoped one, because that file would load into
the module, and the amalgam smoke check proves the refusal on a throwaway
tree. x/type/dict waits: the bitwise app imports it before anything has
loaded x/type/hash, so its amalgam would splice hash inside dict.
x/core/boolean, x/core/control and x/sys/pact wait too: every global
name looked up inside a scoped module’s code checks the module’s
environment first, and those three hold and, or, if, let and the
number tower’s contracts, which run constantly. Scoped, they made the
x-base amalgam boot about 5% slower; the five above cost nothing measurable.
The declaration generator no longer fills the disk on a profile cycle.
tools/contract/gen-engine-xon.sh flattened each profile in work files
named after the profiles, appending one profile’s file to another’s. A row
of %feature-profiles that named its own profile, or two rows that named
each other, could make it append a file to itself; that append never
reaches end of file, so the file grew until the disk was full.
check-engine-contract reports such rows and then runs the generator. Each
profile is now expanded at most once, a row’s own name counts as expanded
from the start, and no work file is named after a profile. A profile that
names one defined after it now takes that profile’s atoms, which were left
out before. The declaration for the committed vocabulary is unchanged.
The engine-contract gate reads the library as forms (#739). The
checks check-engine-contract makes of the library run in x, in
tools/check/engine-contract.x: the partition of the reference ISA into
capability groups, the profiles, the rows of tools/contract/requires.x,
and the parameter values tools/contract/constraints.x binds. The checks
that judge the candidate engine stay in shell, so an engine that cannot run
x is still refused by name, and the gate prints what it printed before.
Deriving requires.x from forms corrects two of its rows:
lib/x/type/err.x needs isa/ffi-call (its errno lookup calls
ffi dlopen, ffi dlsym and ptr call, sites the text pattern did not
resolve), and lib/x/rn.x does not need isa/syscall (the pattern matched
a comment). The declared profile is still posix. The gate needs the built
engine, and its library half takes about 15 seconds where the shell took 3.
A clean answer is kept in build/ under a digest of everything it read, so
check-second-engine, which asks the gate about three more engines, and a
tree that has not changed read it instead of deriving it again.
The tool scripts stop on an interrupt or a TERM. Twenty scripts under
tools/, the conformance runner and the pre-push hook set their cleanup as
a trap on EXIT INT TERM with no exit in it. On INT or TERM the
cleanup ran and the script went on without its scratch files, so an
interrupt ended only the command in progress, and a gate run under timeout
went on into its later checks. Each now sets the cleanup on EXIT alone,
with trap 'exit 130' INT and trap 'exit 143' TERM beside it, so the
cleanup runs once and the script exits with the status a shell reports for
that signal. tools/check/asan-boot.sh runs each boot under a timeout of
its own, which puts itself in a separate process group, so the signal did
not reach a boot in progress and the gate waited for it to finish, minutes
under ASan. It now runs the boot in the background, and its INT and TERM
traps stop the boot’s process group before the gate exits. It still puts
back the JIT byte cache it sets aside, and ignores a further signal while it
does; one that arrived during the restore left the entries not yet moved in
the gate’s work directory. Fourteen more scripts set their cleanup on EXIT
with no trap on INT or TERM. Such a script stops on either signal, but
dash does not run the EXIT trap of a shell a signal ends, so under dash they
left their scratch files behind. They now set the same two traps. x.sh
writes the prefix a state image is built from in a temporary directory, and a
signal during the write, which takes seconds, left that directory behind. It
now sets the same three traps around that section and clears them at its end,
so the boot that follows runs with the signal dispositions the wrapper was
started with.
A wall-time ceiling an interrupt can stop. timeout SECONDS COMMAND
cannot be stopped by the script that runs it: GNU timeout puts itself in a
process group of its own, so a signal sent to the script’s group does not
reach the command, and the script waits for the command before it acts on the
signal. tools/lib/guard.sh runs the timeout in the background and waits for
it, where a trapped signal interrupts the wait, and its INT and TERM traps
pass TERM to the timeout’s process group before exiting. The command’s own
status is passed through, timeout’s 124 for a run that hit the ceiling
included, and with neither timeout nor gtimeout on PATH the command replaces
the guard, where the caller’s signal reaches it directly.
tools/check/pin-smoke.sh, which runs the wrapper 73 times under a
120-second ceiling, and tools/dev/cov-lib.sh, which runs the profile engine
once per spec file under a 60-second one, run theirs through it: a signal now
ends pin-smoke where it stands, 4.0 seconds against 7.0 to 8.2 in the same
runs.
A scoped module’s header reads the rest of its file. The loader used to
read a prefix of every imported file into a string to find out whether it
was scoped, then read a scoped file whole into a string and tokenize all of
it before evaluating any form. Now import loads every file with include
and names the module it is loading, and a file’s (module NAME) header,
finding that name, makes the module’s environment and reads the rest of the
file into it itself, one form at a time, with the reader. Nothing is read
into a string, and no file is read twice. Each form carries the file and
its line, so an error while a scoped module loads now names its file as
well as the line; a form is read after the forms before it have run, so it
can read the forms after it; and a literal () in a module is a form like
any other, since read answers the EOF sentinel at end of input
(x-engine-c v0.2.13). A header naming a module other than the one being
imported is refused, naming both. module.x sheds the peek, the whole-file
read and the tokenizer path: five fewer private definitions, and no use of
the raw file-reading doors.
End of input is a value of its own (x-engine-c v0.2.13, x-sweet
v0.1.6). The engine’s read primitive answered nil at end of input, and
nil is also what a literal () reads as, so a loop that read until nil
stopped at the first () in its input. The pinned engine answers the EOF
sentinel, %token-eof, instead
(x-engine-c#57).
(Io read) still answers () at end of input, so its callers are
unchanged. The library’s own callers of the primitive stop at the sentinel:
the vector reader, and the lint driver, which no longer stops early at a
() among the forms it lints. The sweet bundle moves to v0.1.6, whose
readers stop at either
(x-sweet#15).
The spec runner’s NUL escaper is x, and runs only for a spec that asserts
a zero byte (#733). It was perl, which is not part of this project; it is
now tools/dev/nul-escape.x, run through the wrapper and installed beside the
runner so a bundle’s suite finds it too. It reads the caller’s stdin from fd
3, where the wrapper keeps it while the engine reads its program from fd 0.
The runner also stopped appending an escaper to every batch: the awk runner
reads every spec before it runs any batch, so it knows whether one writes
<<NUL>>, and a suite that measured 162 escaper runs now makes one.
Eighteen more run-time modules have a scope of their own (#719, step
4). x/codec/base64, x/codec/csv, x/codec/zlib, x/doc/emit,
x/doc/emit-man, x/net/http, x/net/rest, x/net/tls, x/repl/edit,
x/sys/opts, x/test/assert, x/tool/contract, x/tool/pin,
x/type/array, x/type/counter, x/type/deque, x/type/path and
x/type/pq carry the (module NAME) header. No file outside any of them
reaches one of their private names, in this tree, the language checkouts or
the released bundles. Four public names that other code already used were
missing from a provide list, and would have become private: DocEmit,
DocMd and DocMan now appear in their modules’ lists, and x/sys/opts,
which had no provide at all, provides Opts. Importing a scoped module is
no slower than importing it unscoped: the same five imports timed in-process
ran between 2 and 10 percent faster with the header.
The first library modules have a scope of their own. Nine modules
that load through import at run time now carry a (module NAME) header
and keep their private names to themselves: x/tool/highlight,
x/repl/paint, x/repl/line, x/repl/term, x/type/generic,
x/type/trait, x/num/random, x/sys/date and x/tool/profile
(#719, step 4). Their provide lists are now complete, since an unlisted
name is private; Paint and Line gained the provide they never had.
The one seam another module reached through a private name is a door now:
the numeric tower asked generic’s %g-absorbs? at its miss handler, and
asks (Generic absorbs? k1 k2) instead.
The header follows the file’s comment banner rather than preceding it, so
a scoped file opens the way every library file does; the loader’s peek
skips the banner, finishing a long one from the whole file. A test that
deliberately drives a module’s private seam reaches it through the
module’s environment, (eval (lit NAME) (module x/repl/line)), which the
repl specs now do.
A collect inside a guard body no longer frees the enclosing handlers (x-engine-c v0.2.12, engine x-engine-c#54). Installing a guard’s handler took the previous handler out of the error-handler slot and kept it in a C local, so a collect inside the body swept the enclosing guard’s handler and the base-eval handler under it, and the next raise went through freed memory. The handler now carries the handler it displaced. The library never collected inside a guard on its own, so nothing met this until #728 swept after each module load during an image write and the tower’s JIT probe, a guard around a lazy import, put that sweep inside one; the x-base, xe and rn image writers segfaulted on macOS CI. With this engine the same write is clean under ASan. The shortest program that showed it, on the previous engine, was a nested guard whose body collects and then raises to the outer guard.
A module can have a scope of its own. A file whose first form is
(module NAME) is evaluated in an environment of its own, a child of the
root: its top-level definitions are private to it, and only what provide
lists reaches the root (#719, design note #718). A private helper stays
in the module, two modules may share a private name, and a name is exported
once — provide refuses a second module that defines a name another owns,
while tolerating a re-export of the same object. A selective (import NAME
sym ...) copies an export into the importer’s own environment, by name or
under an alias, so the importer holds the value and a later global rebind
does not reach it; (module NAME) as an expression denotes the module’s
environment. Scoping is opt-in and per file: a file with no header loads
through include in the root exactly as before, which is the whole boot
floor and standard library, and the loader tells the two apart by the first
bytes so an unscoped module is never re-read. Built on first-class
environments (x-engine-c v0.2.11): a scoped module is an environment, and
import/provide move bindings between it and the root.
-l is repeatable. The first names the lang that owns the prompt, as it
always has; each further one names a bundle to load beside it, resolved
exactly as the first is and read ahead of its entry, so x -l xe -l python
opens at xenon’s prompt with Python registered and (lang python) a call
away. %lang-lead, a new bundle-class seam, carries the first name into
the pipe so a bundle’s entry knows whether to take the prompt or only
register what it is. The state image of such a boot is its own, named for
every lang in it.
An environment is a value (x-engine-c v0.2.11, #720; design note
#718). The pinned engine’s environment is one pair, bindings and parent:
the root’s bindings are a tree, every other environment keeps an alist and
the environment it was made in. A call makes a child, an operative body
runs in a child and receives the caller’s environment as a value, def
binds in the current environment, and eval with an environment makes it
current with the binding staying put. So a definer written as an operative
now defines for its caller inside a frame: (eval (list 'def n v) e) binds
in e, and doc, def-class, def-record and a lang’s define work in
a closure body the way they always did at top level (#527). The engine
retired the frame marks, the shadow list, the local boundary, the tree a
closure carried and the top-level bracket’s frame stripping with it, and
its base layout’s env group is now env and env-root. What read the old
rows by name moved with the pin: the sandbox and ISA specs, the image
reader and inspector, the normative example in the spec, the sandboxing
tutorial and the layout passages in the architecture, type-system,
glossary and state-image documents.
A session switches languages. x/repl/lang keeps the REPL’s seams as a
named bundle – the prompt, the continuation prompt, the printer, the
painter, the bracket marks, Tab’s candidate source and what a finished line
means – and (lang NAME) installs one, so the next line the editor reads
is that lang’s, coloured as that lang. Two of those seams are new:
%repl-eval-line, which is what the editor does with a finished line and
the piece that lets a lang keep the editor instead of replacing repl, and
%repl-complete, which is where (Line completer) now lives.
%repl-prompt-more moved beside %repl-prompt. x-lang’s own bundle is
"x", assembled by the files that own its parts as they load. All of it
joins the lang contract’s seam table.
A bare atom at the REPL prints. The line editor hands a finished line
to the reader without its newline, and the reader drops a final atom that
nothing terminates (#161): name at the prompt read as no forms and printed
nothing, and 1 2 lost the 2, while (def name 1) was fine because the
paren closes it. The line evaluator now appends the newline before reading,
as the painter appends a space for the same reason.
The linter reads a set! body as a definition body. (def NAME ())
followed by (set! NAME (fn ...)) is how a self-referential function is
written – the forward declaration lets the body name itself – and the walk
scanned only the def spelling, so the ladder and shape rules never saw
those bodies. Every tokenizer state machine in lib/x/num/ is written that
way and reported nothing; decimal.x held two four-deep chains while
reporting zero ladders. %arity-collect already read the two spellings
together, so this brings the two rules into line with it.
The assembler lane lowers match. The arms are tried in order: a
comparison test folds into cmp and one conditional branch, any other test
is evaluated and tested with cbz, a literal #t test takes its arm
unconditionally, and with no arm taken the value is nil. if reaches the
lane as a match of one or two arms, the way lib/x/core/control.x derives
it, so both forms share one lowering and if emits the instructions it did.
A compiled analyser state written with match used to raise
asm-compile: unsupported form: match inside the tower’s guard and run
interpreted with nothing reported. The byte cache’s codegen epoch moves to
g2, since the emitter accepts a form it refused.
The REPL colours parens by nesting depth. Both halves of a pair share
a colour, cycling yellow, magenta, cyan from the outside in, the way editors
colour bracket pairs; a close paren with nothing to close is bold red; the
pair beside the cursor is drawn inverse on top of its colour, a close just
before the cursor first so a pair lights as its close is typed, and removed
again when the line is submitted, so the transcript keeps the colours and not
the inverse. An open paren not yet closed is simply its depth’s colour, the
state of every line while it is being typed. The depths come from one walk over the whole line that steps
over strings, comments and character literals with the scan’s own rules, so
#\( is not an open paren and a paren inside a string is not counted, and a
line that has scrolled sideways still colours correctly.
%repl-marks joins %repl-prompt, %repl-print and %repl-paint as the
fourth customisation point: a function from the line and the cursor offset to
(offset depth focused) marks, installed and guarded the way %repl-paint is,
so a lang whose brackets are not x-lang’s sets its own or leaves it nil. The
editor passes the marks to %repl-paint as a second argument, translated into
the window; a painter written for one argument ignores it. (Paint marks line
at) and (Paint line text marks) expose the two halves for use apart from the
editor. The scan also treats #\ and the glyph after it as one character
literal. On a 70-byte line the depth walk costs about 4ms and the coloured
render about 10ms over a plain one, and the walk runs on every redraw; a
redraw whose text and marks are unchanged is served from the cache as before.
A class names each of its methods once. (help x/type/list) printed
map, sort-by, times and every other block-wrapped selector twice, and
so did the listing for every other collection: Block method! installs its
operative through def-static!, and %class-add! prepends, so the newest
registration wins at dispatch and leaves the row it shadows behind it in the
cold alist. class-methods and class-static-methods answered with that
alist’s keys as they stood, so a selector defined a second time was named a
second time, and help – which walks those names – had an entry for each
row. The accessors strain their keys through %names-minus now, which keeps
the first occurrence of a name: for a method alist that is the row a dispatch
reaches. The alist and the dispatch order are unchanged, and the member
accessors need nothing – their alists are written through %box-put!,
which replaces an entry in place.
Tab’s candidates are a seam a lang can fill. %repl-paint answers the
colour question for a lang’s session; completion had no such answer.
%ln-candidates prefix-searches the doc registry, which holds what x-lang
modules document, so a lang that parses its own syntax has none of its names
there and Tab at its prompt offers x-lang’s. (Line completer f) installs a
function from the Edit buffer to (typed . names), and () turns Tab off.
Filling a unique answer, extending to the common prefix and listing on the
second Tab are the same job whatever the syntax is, and stay whichever
completer is installed; %ln-complete! tests before it walks, since a nil
completer has nothing to destructure and first/rest are unchecked prims.
line.x’s %-global budget is 36, and crafting-a-lang.md §7 carries both
seams for a lang author.
List and Iter raise on a value they cannot walk. List from-seq is
the door every basic normalizes through, and it hands anything that is not
nil or a pair to (Iter new). That answers nil for a value whose type
carries no iter slot, and the driver prims dispatch on a type handle a nil
does not have, so passing one on ends the process. (Iter new) raises
type now rather than answering nil, which is what gives (List length 5)
and (List length (%type-alist)) an error; each public Iter door also
checks its argument once on entry, so (Iter ->list ()) raises too. The
drain loops carry their iterator through unchanged and their per-element
prim calls are unaffected. Iter %check is homed on the class under the
classes-are-namespaces rule in tools/check/percent-globals.sh.
A lang bundle can no longer change what the library’s containers mean by
equal. equal? is a bare global, so a session may rebind it, and lang
bundles do – x-sweet ships (def equal? eq?) as part of its Scheme shim.
Every container that compares by content read that name: Dict’s bucket
search, Assoc find, List index-of/includes?/uniq/uniq-by, and a
record’s =?. A rebinding did not make them fail; it made them answer a
different question for the rest of the session, in code that never mentioned
equality. Under -l sweet, (Dict get "k") on a dict holding "k" answered
nil, and (List includes? "a" (list "a" "b")) answered #f, with the bytes,
the FNV hash and the bucket all unchanged. Library internals now read
%equal?, captured beside equal? where it is defined, which is the split
protocol/str/utf8.x draws between Str, the
rebindable ambient alias, and Str8, the fixed name its own internals use.
What is captured is the closure, not the behaviour: its body reads
%equal-others at call time, so a module extending equality through that hook
(x/type/vector does) still reaches every one of those seats. Only rebinding
the name stops working.
The painter reports a vocabulary it cannot read. x/repl/paint builds its
construct set from lib/x/constructs.x behind a guard
whose handler returned an empty Dict and dropped the error with it. An empty
vocabulary looks the same as a session in which nothing happens to be a
construct – a line with no colour on it, either way. The fallback is
unchanged, because losing a colour is not worth refusing to start a session
over, but the handler now writes one line to stderr naming what could not be
read.
A compiled function could be handed an argument back instead of an answer.
compile-asm emits for two calling worlds – an integer function called from x,
whose arguments arrive unevaluated and whose result is boxed, and an analyse
callback invoked from C with live values, where nothing evaluates and the
leading params stay pointers – and with no third argument the door reads the
fvar table to decide which: present means analyser. An fvar also names a callee
the body calls, so a body that calls a prim that way compiles as a tokenizer
state. Its first ordered comparison then reads a param as a pointer, takes the
branch on that, and the unboxed result is one of the arguments; other argument
shapes dereference an argument expression as an integer and reach a SIGSEGV.
The misuse refuses rather than the undeclared call. Refusing the two-argument
form would not raise in a bundle: x-python compiles its tokenizer states that
way and adopts them under a guard, so the refusal would pin the interpreted
states and keep them, and a bundle spec cannot assert that the JIT is active, so
nothing would report it. What refuses is the shape. In analyser mode the leading
one or two params are x_obj_t*, and arithmetic, a shift or an ordered
comparison on one is not something an analyser means – every state in the tower
and in the bundles uses its object params as trampoline arguments and nothing
else. The refusal names the parameter and the declaration that makes the compile
an integer function. =, not, and and or stay legal on an object param:
testing a pointer for equality or for truth is meaningful, and (= buffer ())
asks a real question. A comparison in an if test is checked separately,
because if folds one into its branch and it does not reach the call emitter.
The byte cache key names the compiler. It named the machine and the engine
release, so an entry outlived a change to the emitter: a hit never reaches a
compiler, and a compile whose acceptance or output has changed was served the
bytes an earlier version produced. #597 made this argument for the engine half
of the key, after #590’s cc cache served ABI-stale objects that misread 2.5 as
2 and the symbol .5. The key now carries x-lib-version, which covers a
consumer, and a codegen epoch that covers development: bumping it is part of
changing what the emitter accepts, refuses or emits, the rule the record
format’s magic already states. One boot per machine per version pays for it –
a xenon boot costs 7.8s against a cold cache and 1.7s against a warm one.
Every compile-asm call in the repository now declares its calling world, and
the bodies with no free variable pass an empty fvar table instead of an unused
one.
The session has a line editor, and rlwrap is no longer the answer.
sh x.sh with a terminal now gives arrow keys, the readline chords, history
that outlives the process, Tab completion over every documented name, and
colour applied to what you type as you type it. The documentation has told
people to wrap the session in rlwrap since the REPL existed; that advice is
gone from the README and the tutorial, replaced by docs/repl.md.
It is four modules and only one of them touches a descriptor – x/repl/edit
is the buffer, the cursor and the history walk with no terminal in it,
x/repl/term is raw mode and byte-to-key decoding, x/repl/paint colours a
half-typed line, x/repl/line is the loop that joins them. The split is what
makes it testable: Edit is pure and Term key takes a byte-reading function
rather than a descriptor, so 45 cases run in the ordinary spec harness with no
pty anywhere. Installing it is the seam the langs already use – repl is a
plain global and x-python and x-ash both replace it – so x/repl/line
replaces it too, only when there is a terminal to drive, and a pipe, -f,
-c or a spec harness reaches the C reader’s loop unchanged and never loads
any of this.
The reader’s type alist reaches this from ordinary code. It is a C-built
spine, so pair? answers #f and from-seq treats it as a non-list; the
catalog carries it as type/alist, and (Base cell 'type-alist) addresses
the same structure. Such spines are walked with the bare first/rest
accessors, the rule docs/sandboxing-tutorial.md states for handler spines;
type/struct.x, (Base cell) and (List from-seq) point at it.
The wrapper checks that a pinned amalgam is the file the lock pinned. The
boot-time pairing guards compare recorded strings — a row in the lock against a
stamp beside the installed library — and each of them describes the amalgam the
lock names rather than the bytes on disk. An amalgam replaced after the lock was
written satisfies all of them and still reaches the engine as another release’s
boot, where a base layout that has moved is a SIGSEGV in the first form that
walks a base cell. The lock’s three-element (boot "he.x" "sha256:…") row is
the claim, and the wrapper now reads it: one digest of one amalgam, on a path
that already parses those bytes, before the release reach rather than after it,
since a reach hands the whole invocation to the release the lock names. The
two-element row predates the digest and is skipped, and with no sha256 tool
available the wrapper reports the identity as unchecked instead of passing in
silence. (Pin verify) checks the same row on demand and in CI.
A lang can colour its own lines. %repl-paint joins %repl-prompt and
%repl-print as the REPL’s third customisation point: a function from the
line’s text to the text to display for it, which the line editor calls in the
one place it paints.
It is a hook because colouring is the part of an interactive session that depends on the language. Reading a key, moving a cursor and remembering a line are the same job whatever the syntax is, and repl/line.x already does them for every lang that has not replaced the loop; the sweet bundle inherits the editor without knowing it exists. Where the tokens begin and end is not the same job, and the langs differ on where that answer lives: x-logo registers LOGO-OPEN and LOGO-BLOCK as token types on the base, while x-python parses Python in x-lang and never touches the base tokenizer. One painter cannot serve both.
The install follows the rule repl/ansi.x states for the printer and repl/line.x for the loop: over nil, or over the painter x/repl/paint last installed itself, and over nothing else. A bundle’s entry runs before the launcher that imports the editor, so an unconditional install would take a lang’s painter away and colour its lines as x-lang. A lang’s painter survives whether it is set before the editor loads or after, and across a state image reload.
Nil means no painter installed, not no colour. --no-color, NO_COLOR and
TERM=dumb answer the colour question, and the platform painter honours all
three by returning its argument untouched; setting this to nil to mean off does
not work, because the next install reads nil as nobody having set one. A
painter that raises is caught at the redraw and the line is drawn unpainted for
that keystroke.
The digest engine is built for the input in hand, not for a running
total. Sha256 hex built the compiled engine once 64KB had been
digested in a session, cumulatively – a bar set when the build cost 12
seconds and pure-x ran at 2.3KB/s, and a shape that paid the build for
the wrong input: Pin vendor’s lockfile is two files, 16KB then 2KB,
and with the bar lowered to what the build costs now the first digested
pure-x, just under, and the second built the whole engine to digest 2KB,
12s where pure-x alone was 10. A total says nothing about what is left
to digest; the length of this input does. The rule is per input now:
build when the input is %sha-jit-threshold bytes or more, and the bar
is the measured breakeven – pure-x at 2.4KB/s against a 4.5s build
(#679), so 12KB is the first size at which the build is ahead within the
call that paid for it. Pin fetch and Pin bundle read the same bar
instead of carrying their own 65536. Pin vendor of x/type/dict: 6.6s
from 10.5 on a 12-core arm64 box (the build’s 4.5s, then two digests
in milliseconds where the first alone was 6.7s); a spec case pins that two inputs just under the bar stay
pure-x and one over it builds.
CI keeps the sanitizer engine it built. The gates job’s asan-boot and
the asan job each cloned the pinned engine sources and built x-bin-asan
from them, 55 seconds on both runners before a dialect booted – the same
bytes every run, because a tag is immutable and asan-boot.sh already says
so (“the build never goes stale”). deps/engine-src is an
actions/cache entry now, keyed on the engine pin and the runner image:
the image because the engine is built with gcc, which links libasan
dynamically, so a binary is only as good as its image’s runtime. On a hit
asan-boot finds the build and skips the clone; the asan job’s source
fetch reuses the tree the same way. What is left in asan-boot is the two
tower boots, 1m33s each on the x86-64 runner, and they stay serial:
measured there on 2026-09-13, each peaks at 10.0GB on a 15GB machine.
The state image writer’s host boots from an image. A write is a helium
process running tools/dev/image-write.x over a child that loads the
library from source, and the host booted with --no-image – from source,
every time – so that a host missing its own image could not write one and
land back in the builder that started it. That was 2.3s of a 5.7s x.x
write on arm64, 29 times over in make images (6m43s of a CI specs job on
the Linux runner, where a source boot is slower still), and once per
pinned project’s first boot, where it was the whole of pin-smoke’s
remaining cost: a pinned miss was 6.8s against 3.3s from source. The
wrapper has the mode the recursion actually needs, X_IMAGE_NO_WRITE: a
current image is booted from, a stale or absent one is neither written nor
used, and the boot is from source. image-build.sh runs the host under
it, and tools/dev/images.sh boots helium once before its 29 writers so
that every host hits on a fresh checkout. An x.x write is 3.7s from 5.7, a
pinned project’s first boot 5.5s from 7.35, and the 29 images at four jobs
on a 12-core arm64 box 56s from 72; pin-smoke checks the mode both ways.
The pin gate’s digests cost what they weigh. tools/check/pin-smoke.sh
was ten minutes of the ubuntu gates job, and the job tripped its 20-minute
cap in nine of the last thirteen main runs; measured under a timestamped
trace on a 12-core arm64 box, the smoke’s fifty wrapper runs took 261 of
its 264 seconds, and three costs owned most of them, none of them the
tests. First, Pin fetch and Pin boot digested the engine’s 15KB
isa.x in pure x-lang – 2.4KB/s, six seconds a run – to print a drift
notice, when the fact is already written down twice: an install tree’s
contract/isa.sha256 stamp and a checkout engine’s (isa "sha256:...")
row in x-engine.xon. Pin %pin-tree-isa reads one of those now, the way
the wrapper’s own boot guard compares recorded strings, and says the
pairing is unchecked when neither is readable; a fetch is 2.3s, from 7.7.
Second, (Sha256 jit!) cost twelve seconds in every process that called
it, nine of them compiling the engine’s 12,241-node fill body, because the
assembler cache stood aside above 128 nodes – a cap sized for an
interpreted printer the cache had already stopped using: its key text is
spelled by the C write-to-str door and hashed by FNV, and the fill body
prints in 0.6s and hashes in 0.1s. The cap is gone, and with it the
slurp’s 64KB single read that called a full buffer a miss (the fill body’s
record file is 82KB, so the first cut of this change stored it every time
and never read it): the slurp reads in rounds now and an entry is whatever
size it is. The build is 4.5s from 12, of which 1.8s is relocating the fill
body’s 2,945 sites and 1.4s the differential check. Third, Pin bundle
called jit! unconditionally where Pin fetch had guarded it behind a
64KB size since #324, so the smoke’s six bundle runs and two install runs
each built the engine to verify a few hundred bytes; the guard is the same
now. Pinned by three spec cases: a 300-node body is keyed and its second
compile is a load, an entry longer than one read round loads whole, and
the tree’s fingerprint answers the engine’s declaration. The smoke is 154s
from 264 on the same box, all of it still in the wrapper runs; what is
left is nine pinned throwaway projects at 7 to 9 seconds each, the state
image their first boot writes (#669) against 3.3s for the same boot with
--no-image, which is the writer’s cost, not the pin’s, and the next
thing to measure.
The socket specs let the kernel pick their ports. Four cases bound
fixed ports, two of them inside the ephemeral range macOS hands to any
process that asks, and on 2026-09-12 Spotify held UDP 49364 when the
release tag’s pre-push suite ran: bind: Address already in use, the
gate red on a port the test had no claim to. There was no way to bind
port 0 and learn what was chosen, so Socket local-port is new:
getsockname on a bound or listening fd, the port read from bytes 2-3
of the sockaddr, TCP and UDP alike, a cold (%sk ...) resolve like
sendto so the module’s %-globals budget is unchanged. Every fixed port
in lib/socket.spec.md is now a bind to 0 followed by local-port,
which also makes the eaddrinuse, frees-on-close and econnrefused cases
say what they claim rather than “47913 happened to be free”; the door
has two cases of its own, one for the failure path.
The assembler lane refuses a form it cannot spell on this engine, instead
of calling address 0. An optional JIT symbol – jit_score_variant,
jit_buffer_last_char – binds as 0 on an engine that lacks it, so that
every other form keeps compiling; but a form that needed one was emitted
anyway, as blr 0, and died arbitrarily far from the cause: x-python’s
compiled number states declared their variant on an engine without the
symbol and the first number token after the swap segfaulted (its main-lane
CI, 3 of 5 runs). %emit-call! now refuses an address of 0 with a 'state
Err, and %score-variant! refuses by name the way a call through a value
already did, so a bundle’s probe hears no and its guard keeps the
interpreted twin. Pinned in ext/jit-optional-symbol.spec.md on every
engine, by standing in for the missing symbol.
A spec can assert a NUL byte in captured output. The shared runner reads
the interpreter’s stdout with cmd | getline, and awk is a C-string
language: a zero byte TERMINATES a record, so a test’s output was truncated
at the first one and compared short – a\0b arrived as a, with nothing
said about the rest. On the one-true-awk that record’s length is 1,
measured. The byte is now escaped before awk reads it, to the literal text
<<NUL>> in the same in-band style as the harness’s own <<SEP>>, and a
spec asserts one by writing <<NUL>> in its expected block; every other
byte passes through untouched, so no existing spec sees a difference. perl
does the escaping, being the only tool to hand that is both byte-clean and
able to expand one byte into several – BSD sed cannot express a NUL match
(“first RE may not be empty”) and tr is strictly 1:1. Without perl the
runner warns once and the truncation stands, so a spec asserting the byte
fails rather than passing quietly, and SPEC_NUL_FILTER overrides the
command. Because the escaper is the pipeline’s last stage, the engine’s own
exit code is now read from a file written inside the pipeline rather than
from close() – which makes a mid-batch death report its status on the
one-true-awk, where that code was previously only ever visible on
gawk/mawk. x-python wanted this: its bytes carries NULs and its str is
becoming a code point list that can hold one, so a bare print of such a
value had to be pinnable at all. tests/spec-format.md states the
contract, meta/multiline.spec.md holds it.
An error’s classifying symbol is its TAG, not its “kind”. Err grew
up saying kind: (Err kind-of e), (e kind? 'io), the kind field,
(Err make kind msg data), and every doc string that promised “a kind-‘io
Err”. Kind is not a term this tree uses for a classifier – a token’s
classification is its variant, an error’s is its tag – so the API now says
so: (Err tag e) is the total accessor (the noun applied to the value, the
way (List length lst) reads), (e tag) the field, (e tag? 'io) the
predicate, and make / raise take a tag. No alias is kept: a guard
that matched on (Err kind-of e) now writes (Err tag e). (Err code-of
and File stat’s kind key are untouched: the first is an engine raise’s
message literal, the second names a file’s kind – ‘file ‘dir ‘link.)
A reader hears which state accepted, instead of rescanning to find out.
An analyser state knows which of its states accepted and threw it away; the
type’s reader then rescanned the text to learn what it had just been told.
The engine now hangs a variant cell off the score cell (x-engine-c#43),
records the winning handler’s variant at the accept, and hands it to the
reader as its second argument; this release is the library’s two ends of
that channel. %score-variant! is the writing end, called by a state at
its accept, and %read-variant the reading end – the integer, or nil when
no state declared one. The variant travels as a raw atom cell because an
int object only exists relative to a base that registered the int type, and
a tokenizer base has none by design. The assembler lane compiles
%score-variant! through an optional JIT symbol, so an engine without it
keeps compiling every state that does not use one. The capability is
tok/variant in the contract, claimed by the engine, and the two spec files
that need it say so with # @requires: the interpreted twin in
lib/reader-variant.spec.md and the compiled one in
ext/jit-analyser-variant.spec.md, split out of jit-analyser-self so
that file keeps its self-param coverage on an engine without the door. The
runner honoured only the FIRST # @requires line of a file, so one needing
two capabilities was gated on one and would have run, and failed, where the
other was absent; every line gates now, and a skip names the capability it
was missing.
The core boot reclaims its garbage as it goes. The engine never
collects on its own – mark and sweep run from the heap prims and nowhere
else – and the dialect bodies collect once, after the whole boot, so a
source boot’s footprint was the sum of every include’s garbage: the image
writer for lib/x-core.x peaked at 3.05GB on arm64 and 2.9GB on x86-64
for a heap that holds 85K objects when it is done, and on x86-64 the
x-base writer reached 6.2GB and the tower harness 7.7GB, which is why the
Linux runner writes its images one at a time. x-core.x now collects at
fourteen group boundaries between its own includes; it may, because it is
included at the top level of the dialect bodies and the harnesses and
imported by nothing, so nothing of an includer’s is in flight – the case
the module rule guards against does not arise. Measured on a 12-core
arm64 box, interleaved against main: a source boot 1.92s to 2.29s, the
x-core writer 3.05GB to 1.15GB, x-base 4.14GB to 2.37GB, the tower harness
3.92GB to 2.58GB; a boot from a state image pays nothing. The tower’s own
load burst inside boot/tower-compiled.x is what remains, and stays,
because that file is importable. The asan-boot gate, the full suite and
doctest passed on it.
A pinned project boots from a state image of its own. The wrapper
imaged every boot but one: a project with a pin.xon paid the full source
traversal on every run – seven seconds for a helium REPL that boots from
an image in under one – because “what a pin arms is per-directory state
the key does not see”. It sees it now. A manifest’s (root ...) rows are
armed inside the imaged prefix (pin_arm imports x/tool/pin, which reads
the manifest and import-path!s each root), so the manifest is a key path,
hashed as one file, and an edited manifest is a different image. The image
lives beside the manifest, in the project’s own .images/, the way a
bundle’s does – the per-user cache keeps one file per dialect per install
root, and two pinned projects would have taken turns overwriting it. The
overlay’s modules are not in the key and need not be: they load after the
loader, on import, from the roots the image armed. What stays unimaged is
a pinned boot amalgam (--boot, or a manifest’s (boot ...)): another
release’s boot, which this tree’s loader has no business standing in for.
tools/check/pin-smoke.sh grows the case: the image lands beside the
manifest, the second run boots from it with the overlay resolving, a
touched manifest misses, --no-image boots from source, and a manifest
with a (boot ...) row is refused by --image as before. Add .images/
to a pinned project’s .gitignore, as the bundles do.
A bundle boots from its image again after the platform is reinstalled.
A bundle’s image key carries lib/ and the engine, so one platform
reinstall staled every installed bundle’s image at once, and the wrapper’s
rule – never write into a bundle behind its installer’s back – left what
followed as a source boot on every run, with nothing on stderr: x -l
python took 16.4s a boot against 0.98s from the image its installer HAD
written, until the user thought to re-run each bundle’s make install. The
rule is kept, and the fallback the wrapper already had for every other boot
now applies to bundles too: the installer’s image is preferred, and on a
miss the user’s own is written into the per-user cache, announced, and
used; the key gains the bundle’s directory so two checkouts of one lang do
not overwrite each other’s. Found beside it: tools/dev/image-build.sh
defaulted its wrapper to ./x.sh, which exists only in a checkout, and the
six lang bundles that call it from their spec runners set nothing else, so
from an installed tree not one of them wrote an image at all – the failure
went into a log beside the image nobody reads, and the suite booted from
source. The default now finds a wrapper – ./x.sh in a checkout, else the
install’s bin/x, else the x on PATH – and says so rather than failing
into a log when there is none; only the write looks, a check still answers
from the key alone. Measured on x-python’s suite, one spec file: 22s before,
1.5s after.
The state images are written in parallel. make images wrote its 29
images one after another, and that loop was the longest phase of a CI
specs job: 6m43s on the 4-core Linux runner and 9m30s on the 3-core macOS
one, ahead of the 8-minute suite the images exist to speed up. The writes
are independent, so tools/dev/images.sh runs them under xargs -P. The
job count is bounded by memory, not cores: a writer boots a library from
source and images the child, and measured one at a time x-base peaks at
2.0GB, x-core at 3.2GB and the tower harness at 3.9GB – a first cut that
took one job per core put twelve writers on a 16GB box and took it down.
On arm64 a job is budgeted 3.5GB: four on a 16GB box, two on the 7GB
macOS runner, where the 29 images now take five minutes against nine and
a half. On x86-64 the same writer is bigger – the heap costs ~64 bytes an
object there against ~29 – and four jobs killed the 16GB Linux runner 49
seconds in, so until its peak is measured x86-64 budgets 9GB a job, which
keeps that 16GB runner serial. A box whose size cannot be read gets one, and
IMG_JOBS overrides. Measured on a 12-core arm64 box: 271s serially, 84s
at four jobs, with the suite green behind it.
CI’s specs job has room to breathe. Its 20-minute cap was set when the macOS run took eight; across the last eight runs of main it took 8, 13, 15, 16, 17, 18 and 19, and PR #665’s first attempt was cancelled at 20 with every suite green, fifteen seconds into doc-examples. The cap is 30 now – a slow runner is not a failed change.
Vector build takes a block. The one callable-taking builder without
one: (Vector build 3 (i) (* i i)) now stands beside (Vector build 3 (fn (_
i) (* i i))), wired at position 1 as List times is, count first. It
slipped the collection-wide sweep because its callback is declared ANY,
not CALLABLE, and the sweep keyed on the type token; every other builder
on an iterable either has its block already (List times, List iterate,
Gen iterate, Gen make, Iter make) or takes no callable.
A list dispatches to List at the value. ((List of 1 2 3) filter (x)
(> x 1)) answered Unbound SYMBOL 'filter': the engine’s list call reads its
first argument as an index, and a vector’s or a string’s value form – (v
filter (x) (> x 1)) – had no counterpart for the one collection every other
one is built from. x/type/list now binds List OVER that handler the way
Vector and Str are bound, so a symbol selector sends to the class,
subject-last, and both call shapes ride it: (xs map (i x) (* i x)) and (xs
map (fn (_ x) (* x 10))). Everything the handler underneath did it still
does – (xs 0) indexes, (xs 1 3) slices, and nested list data the iterator
re-evaluates keeps its #69 echo, since a list head followed by a non-symbol
delegates to the prior handler unchanged. The linter learned the value form
with it: it bound a block’s names only under a class head, so (v map (x)
...) reported x undefined for a vector as much as for a list – a block
send binds its names at any head now, still keyed by the selector table. The
full suite ran with the binding in place before it was kept: 2961 tests, 0
failed.
A comma is not a token delimiter, and the owl’s nose is one. The quote
family’s shared delimit hook ended a token on ', ` and , – so
foo'bar reads as foo then 'bar, which is right, and a comma glued to a
token split it, which was not: {O,O} was two tokens. Unquote is recognised
only where a token BEGINS, so with a space or a paren before it the comma
already starts its own token; the delimiter changed exactly one thing, and
it is gone. Bitwise’s nose is {O,O} – every costume, the frozen parity
fixtures and the twin’s renderings, the site bar and footer, and x-lang’s
own mark and banner follow; only the auk keeps its > bill, for the
wordplay. x-lang’s reference line is (def owl "{O,O}") – the bare datum
was splitting on the comma until the reader stopped treating one as a
delimiter.
A wrapped method’s help shows its block form. (help Vector for-each)
answered the applicative signature and nothing else – true, and incomplete:
the block form is a second call shape a reader could not discover from it.
The wrap is the one place that knows the shape, so it adds the note itself,
onto the method’s own entry, pending or committed; wrapping twice does not
say it twice. It shows the method’s own head with the block in the
callable’s seat – (Vector filter (x) body ... v), (for-each (p) body
...) on a Dict, (List times n (x) body ...) where the callback is not
first – and the shape’s other spelling as a suffix, (i x), (acc i x),
(k v). “In place of f” was not discernible; the head is. One fact stated where it is
decided, instead of fifty (doc ...) forms repeating it and drifting. The
generated reference carries it too: the generator loads the module it
documents (guarded; boot files and the dialect toolboxes skipped) and merges
each method’s live registry notes into its page, deduplicated against the
source form’s own – so a page shows what (help) shows.
A block’s index comes first. (v for-each (i x) ...), not (x i): the
order of Gen enumerate’s (index . value), which this library had already
fixed, and the one a hand reaches for – the prototype was index-first for
that reason, and the shipped order was a crossed wire. A fold’s optional
index precedes the element it indexes, (acc i x). Breaking for any
two-name block written against 0.13.0; the language cannot catch a swapped
pair, since both names are just symbols, so the fix is to swap them.
A bundle can be swept, and the assembler is preloaded whole. make
lint-x sweeps lib/ and apps/; a lang bundle under languages/ was
swept by nothing, so every rule the linter knows was advice those bundles
never heard. The platform ships the gate, bundles do not vendor it:
tools/lang-kit/lint.sh drives the linter at a bundle, with an overridable
root (X_LINT_ROOT/X_BIN) for one outside the checkout, a non-module
sibling recognised as a fragment (x-coreutils is one module assembled from
twenty-one include-once‘d files with no provide), a clean file’s
advisory warnings no longer dropped with its dot, and string=? known to
the ladder rule – 476 uses across eight bundles that %ladder-cmp? could
not spell. Pointing it at the bundles found the next two: an assembler’s own
top level was missed (cc/base.x defines %cc-x-write; five fragments call
it; an import cannot reach it), and a group’s preload omitted its own first
file. The assembler is preloaded whole now, in the order the bundle really
uses, which also drops the order reconstruction; %cc-x-write resolves and
two phantom go/N ladders vanish. Swept: x-coreutils, x-awk, x-grep,
x-make, x-cc.
Every callable-taking method takes a block, and two shapes joined. The
block form shipped in 0.13.0 on 35 selectors; this wires every remaining
higher-order method but one – List’s count-if none? reject find-index
uniq-by drop-while fold-right scan zip-with iterate, Gen’s none?
drop-while scan zip-with iterate make, Iter make, and Assoc’s map
(the callback gets the value) and filter (it gets the whole assoc): two
shapes on one class, which is why shape is declared per selector. Two
mechanism extensions covered the stragglers. POSITION: List’s
constructor-count rule puts the count ahead of the callback, so (List
times 4 (i) (* i i)) and (List adjust 0 (x) (* x 100) lst) wrap at
position 1, the forms ahead evaluating in the caller’s env. THUNK: () is
a binding list, and get-or-else takes it – (d get-or-else () (expensive)
k), the body a default run only on a miss; an empty binding list in any
other shape now fails with its count. Left out on purpose: List unfold
takes three callables, and a block for one of them would confuse more than
it saves – it is the spec’s opt-in proof now; PQ make waits on its doc.
The linter accepts the binding list at position 0 or 1 and ().
Interpolated strings are spelled #"…". The $"…" spelling (0.4.0)
was a mistake: every other reader extension – #t, #\a, #(…), #/…/
– wears the #, and interpolation was the one form that did not. It does
now: #"a{x}b" reads exactly as $"a{x}b" did, into (Str8 str "a" x "b"),
a #"…" nests inside a hole the same way, and {{, }} and \{ mean what
they meant. The rest of the # family is untouched, and the spec pins the
neighbours: x_token_analyse runs every handler from the token’s first
character independently, so the literal’s analyser declining on #t or
#\" costs their own readers nothing – and the read guard now checks the
whole #" opener, so #\", three bytes that start with # and end with
", can never be mistaken for an empty literal. The library, tools, specs
and docs are respelled; the highlighter, the formatter and Xon arm-source!
follow the reader; the downstream langs never used the form. BREAKING: a
$"…" left in source fails loudly – an unbound symbol $"…", or past its
first space an unterminated string – and the fix is one character.
The REPL printer is only Ansi’s to move. Ansi install runs again
after a state image loads – that is what put colour back in an imaged
session – and it ended by enabling or disabling the highlighted printer
unconditionally, so a lang’s own printer was replaced by the platform’s:
x-krn’s suite from an image printed ('b 'c) for (b c), four specs
failing on nothing but the printer, and the quieter half was worse –
bundles whose values print the same either way passed while using the
platform’s printer instead of theirs. The class now remembers what it
installed and moves %repl-print only when it holds the platform’s own
printer or that one; anything else belongs to someone who asked for it.
x-krn from an image: 74 tests, 4 failed, to 0. Found wiring the remaining
bundles’ suites to boot from an image.
The image spec’s scratch files are per checkout. The state-image spec
kept them in a fixed /tmp/x-image-spec, world-writable and shared by
every checkout and worktree on the machine, and they are not inert: pre.x
carries this run’s %IMG-PATH, and one test writes one binding
%IMG-VERBOSE. A concurrent run in another checkout supplied them, so the
header probe booted the PEER’s image, verbose, and captured the loader’s
count line where it expected header ok – the leftover file named the
other session’s worktree. Intermittent, one run in three, and it presented
as an unrelated red that blocked git push through the pre-push hook. The
scratch dir lives under X_IMG_DIR now, which is per checkout.
A KEY-PATH is a bundle’s source, and only the bundle spells it.
image-build.sh keyed a caller’s KEY-PATH with find -name '*.x' and
nothing else, so a lang whose modules are written in anything but .x had
them silently outside its own image key: editing one left the key unchanged,
the builder answered “is current”, and the suite went on testing the
library that was there before, out of the image, while its from-source
control tested the one on disk – both legs green, at two different
libraries. Measured on x-r7rs, whose scm/ layer IS its library: before,
CURRENT after an edit; after, NEEDS REBUILD. Reworked after review: the
first version put x scm in the platform’s default, a bundle’s fact living
in the platform, and it is not there any more.
libm addresses live in cells, not inside closures. float.x resolved
each libm entry once at load and closed over the result – a raw address
inside the closure’s own frame, where the transient rule cannot reach it:
%image-transients names globals, so clearing %fsin emptied the global
and left the frame the closure still held. Seventeen survived the child’s
collect and the writer refused the image on unnameable: 16, every one a
foreign address, and X_IMG_WHO=1 named the holder in one word: sym. It
refused on Linux only because only there the words could not be NAMED –
glibc keeps a dlopen’d libm out of the global scope, where macOS’s
libSystem folds it in – so a clean unnameable: 0 on a mac was this bug
passing quietly, not its absence. The address lives in a one-slot cell the
closure reads at call time, and a thunk among the transients empties every
cell in the child before the walk.
Opts asks the declaration before it assumes. Two fixes, one fault –
the class assuming something it had been told – both found by x-coreutils
within hours of adopting the library. (Opts on? o "-m") answered #f for
a flag that HAD been given, when it was declared as taking an argument:
on? read only the standalone list, so a caller asking “was -m given” had
to know which of its own two lists it went into. It answers presence for
either kind now; a value flag is still read with (Opts value ...). And
-5 read as an operand because it looks like a number – decided by shape
alone, before consulting the declaration – so comm(1), which declares
-5, could not take it. The declaration is asked first; the heuristic
still stands where nothing claims the token (sort -5 is an operand, not
five unknown flags).
The documentation site wears the owl. Every page under
jonruttan.github.io/x-lang published as bare jekyll-theme-primer – a blue
x-lang text link above the H1, and nothing else, across 143 pages of which
113 are generated and nobody edits. Nothing here is designed: the marks
already exist, apps/bitwise draws them from bitwise.xon, and
sha256("x-lang") picked the 261.6 degree hue, so this only carries them
onto the pages. A bar with {O,O} x-lang and three links, a footer, that
hue on every prose link, and a favicon where there was none – the avatar
format, generated by the same tool rather than drawn by hand, so it stays a
function of the name. The owl in the bar is TEXT, not bitwise-mark.svg:
the drawn glyphs are Roboto Mono outlines over their field, and rasterised at
30px they anti-alias into a grey smudge, while {O,O} set as type is crisp
at any size and costs no request. The frame is
docs/theme/_layouts/default.html, an override of the theme’s layout of that
name – Primer’s supported hook reaches <head> only, and a bar and a footer
are markup in <body> – kept beside the pages it frames and moved to the
site root by the workflow, where Jekyll looks for it. One thing beyond the
chrome: “Improve this page” no longer appears on the generated reference,
whose pages are never committed, so the link had been sending a reader to
github.com to CREATE a file the next build overwrites.
The documentation catches up with the code. width is pinned in the
glossary as the code-point count, with bytes and true display columns
named as the two things it is not; the formatter’s comment above its width
function had described a (s) call the code no longer made and a
str-length it had never called. standard-library.md documents all
fourteen Vector methods where it listed seven, points List map at the
block form, and reads 0.12.0 where it said 0.5.2; object-system.md
says what a send with no selector does.
Opts: the command line, parsed against a declaration. Every bundle was writing this by hand – x-grep’s and x-make’s option readers were byte identical apart from an error tag, and x-coreutils had grown nine of its own. The cost was never the duplication but the drift between the check and the read. A bundle declares its options and Opts answers them.
Block form across the collections. A higher-order method takes a callable,
and the call site can now write the callback’s parameter names and body
directly – (List map (x) (* x 10) xs) beside the applicative
(List map (fn (_ x) (* x 10)) xs), with a second name binding the index.
Both forms stay live on the same selector. List group-by, List partition
and Dict map join them.
A string’s value-call indexes. ("ab") answered 2, which made a bare
string call a second, unrelated operation wearing the indexing syntax. It
indexes now, and a length is asked for by name.
A send that names nothing errors instead of crashing. ((new P v 1)),
(P) and (#(1 2)) each took the interpreter down with nothing on stderr.
(first ()) can be made to raise. It is undefined by spec and
dereferences nil in practice, so an ordinary typo takes the process down
with no diagnostic. (import x/tool/safe-access) guards both accessors.
Opt-in: the library walks its own lists through them, and the guard costs
1.4x-1.7x.
A boot from a state image keeps its terminal. Three things a boot decides
from the process it is in, which an image cannot carry, all found by driving
langs from one. Colour is the visible one: every colour is a string baked when
x/repl/ansi.x loaded, and the writer’s child has a pipe for stdout, so an
imaged session had colour off for good – (help) came out plain in a terminal
that could show it. Detection and installation are one Ansi install now, run
again by an image recache hook. Measured on a pty, escape codes in (help):
source 63, image 0 before, image 63 after; a pipe still gets 0.
A boot from a state image keeps ctrl-c, and a bundle keeps its session.
Two things a boot does that an image cannot carry, both found by driving
x-logo from one. The SIGINT handler is the process’s, not the heap’s:
sigint-install ran in x-core.x at boot and never again, so every image boot
ran with SIGINT at its default and ctrl-c killed the session outright instead
of raising STOP – in every lang and every dialect. It is now an image recache
hook. And %batch? was reset for a bundle only when $file was empty, which
for a bundle it never is (it holds the bundle’s own entry), so an entry that
chooses between a REPL and a batch reader on %batch? always read the
--batch the loader needs: x-logo booted from an image read its own launcher
as a Logo program and exited without a prompt. The reset now keys on the
user’s file, the same rule bundle_form uses.
The image writer refuses instead of dying. Five lang bundles crashed it
on first contact, and a crash says nothing. The transient walk now runs
from the writer’s base with primitive objects rather than as a form the
child evaluates (r5rs rebinds fn), and a raise inside the child stops the
write with its message – a swallowed raise left the child’s root chain
pointing at C frames that were gone, and its next collect walked freed
stack, which was every silent SIGSEGV. A reference word the writer cannot
place is reported by its holder, never read; a child past a million live
objects is refused with the count; an entry that reads stdin at load (the
engine’s program and the child’s stdin are one descriptor) is refused as
“ended the writer”, and the child is told it is being imaged through
%image-writing; a signal in the log is called a crash. r5rs images
(144K objects, 0.5s to boot against 7s); logo and python refuse with three
and two unplaced references into the second base each makes at load.
The recache walk is not shaped like R5RS iteration. (%image-recache!)
remakes what an image could not carry – a dlopen handle, a JIT trampoline,
the colour detection – and its hook walk was spelled (do ((first l)) ...),
which sequences correctly here and nowhere else: do is late-bound, and a
lang that gives it Scheme’s meaning reads that as a loop and calls no hook at
all. x-r5rs did, silently, and the first thing to fail was thousands of forms
from the cause.
A lang boots from its image. x -l NAME paid the same boot on every
run – six seconds for awk, eleven for xe – although its suite had been
booting from a state image since 0.11.1. The wrapper now images the prefix
it pipes ahead of the user’s program (the root and param forms, the entry,
the pin arming, a bundle’s root and entry) through the same
tools/dev/image-build.sh and the same key, and when the image is current
the loader stands in for the prefix: x -l awk is 0.7s, x -l xe 0.9s,
an installed x -l he 0.4s. The engine’s own args survives the install
(the loader rebinds it, since the writer’s child had none), and what the
prefix would have decided by evaluating – the %batch? reset, the
launcher – is emitted after the loader. A bundle’s image is its
installer’s: make install runs x --image -l NAME into the bundle’s
.images/, and a stale one means boot from source, quietly. Every other
boot writes its own into ~/.cache/x/images/<tree>/ on a miss and says so
on stderr, because that run pays a boot twice. --no-image boots from
source; a pinned boot is never imaged. make install ships the image
tools and the layout contracts under the share tree, so an installed x
does all of this too.
The first cold load of an x-base image found one more transient: the compiler’s trampoline addresses are integers, resolved when the module loads, and an integer names nothing, so a compile on the far side of a load called the writer’s process’s addresses. They register as transients now and the recache hook resolves them again.
The engine pin rides to x-engine-c v0.2.8, and the three JIT dialects
are in the images list. v0.2.8 carries x-engine-c #41: a def scopes by
the live frame rather than by an empty save stack, so a def in a closure’s
tail position is frame-local like the one a form earlier. That is what
had refused x-base, xe and rn: the asm lane’s temporaries (hit,
cell, buf) leaked into bare globals, and though the pinned v0.2.7
imaged them with a warm asm byte cache, a cold one took the compiler’s
miss path, whose own tail defs leaked too – and a CI runner is always
cold. On v0.2.8 the three write clean from a cold cache, so make images
lists them after x-core, x and he: 29 images, none refused, and the
lang bundles’ harnesses, which all boot x-base.x, get the image they
were waiting for.
The JIT dialects can be imaged, and a lang bundle’s suite can boot from
an image. A state image could not carry the tower’s compiled analysers
– native code in a page the writing process mapped – so x-base, xe
and rn were refused outright and every lang bundle’s harness, all of
which boot x-base.x, with them. boot/tower-compiled.x now records each
compile as a site: what it displaced, where it went, and how to make it
again. The writer runs it inside the child before its walk – a thunk listed
among %image-transients, the second half of the transient rule in
boot/reflect.x – which puts the interpreted twins back; the loader’s recache hook compiles them anew
in boot order. The x-base smoke and reader specs run from the image in 1s
a file against 6s from source, and a load with every analyser native
again is 0.77s. What refused the three after that was the asm lane: a
def in a closure’s TAIL position bound globally (the engine decided
top-level by an empty save stack, and the frame is popped before a
deferred tail runs), so every compile left its last function, self-cell
and read buffer in bare globals (hit, cell, buf) the writer cannot
name. The writer now says so: when anything is unnameable its census is
followed by the holders, one path up to a named spine node, which is how
those four were found. The engine fix – def scopes by the live frame,
eval! evaluates as a top-level form, an operative’s restore sheds an
inner frame – is x-engine-c’s feat/def-lexical-scope, and with it the
three write clean from the real library. On the pinned engine they image
only with a warm asm byte cache (a cold one takes the compiler’s miss path,
whose tail defs leak too), so make images waited for that fix to be
pinned (the entry above). The writer’s holder chase is opt-in,
X_IMG_WHO=1: it is minutes on a dialect-sized heap. docs/state-images.md carries the
measurements and the open item.
For the bundles: tools/dev/image-build.sh takes extra key paths, so a
bundle’s image is keyed on its own modules as well as the platform;
tests/spec-runner.sh finds the loader from the platform root rather
than the library’s directory and roots the loader’s repo-relative
includes there, since a bundle’s suite runs in the bundle; and make
boot emits a launcher-free x-core.x beside x-base.x, the amalgam a
helium-weight harness should have been loading all along –
docs/lang-contract.md says which to load and why. x-awk is the worked
example: 167 tests, 38s from x-base.x to 18s from an image, one file per
job, and IMG=0 as the from-source control.
The engine pin had first moved to x-engine-c v0.2.7, and with it #599
closed: a type registered on another base outlives the collector.
base-make-type pinned the type’s objects by marking from the target
base’s tree root, and a base’s root is born SHARED, so the walk stopped on
its first node and pinned nothing – a defect that surfaced one collect
late and read as a collector bug. A lang that brings its own tokenizer
base (x-ash) could not take the per-turn sweep and ran its suite with the
per-snippet collect off; both workarounds went with that pin. The
crafting-a-lang.md entry that described the defect asked to be deleted
when it closed, and is.
The release gate reads ci.yml job by job. It polled the tag’s run
until nothing was in flight, which meant waiting on make test-asan –
CI’s own ~32-minute extra, not part of make test – and tripping the
cap when the run had queued behind another main push. The verdict is now
the jobs that cover the suite, each judged as it lands.
Lint children sweep before every file. A batch child lints up to
eight files in one boot and x collects only when asked, so its peak was
the sum of every file’s analysis garbage – four such children OOM-killed
the 16 GB release runner twice. The sweep runs before each file, the
first included: the single-file children (class.x, pin.x,
asm-compile.x) start on 17-19M live objects of boot and reader residue,
and that was the peak that mattered.
The two long spec batches are cut along their fixture chains:
tools/pin into six files sharing two on-demand fixture trees,
bitwise-parity in two, none over 7 s.
x-coreutils’ contract advances to 103 applets, 0 failed.
The tool tier got its doors. A bundle could read and write files and
little else, and the small tools stopped at that wall: no chmod, no
ln, no readlink, no df, no uname, no way to ask who is running.
File gains chmod, chown, link, symlink, readlink, utimes,
mkfifo and statfs — syscalls, in that module’s own idiom, and only
the DARWIN table needed new numbers because the Linux index tables
already name every one. Two of those numbers are not the obvious ones:
statfs64 (345) rather than statfs (157), whose 32-bit-inode struct
reads f_bsize back as zero; and utimes takes only the clock, because
explicit stamps want a packed pair of timevals and that module holds no
pointer prims to build one. Sys gains getuid, geteuid, getgid,
getegid, getgroups, uname, cpu-count, sync, fsync, nice
and chroot — all libc through %resolve, so no syscall number appears
there at all. uname reads the five utsname fields as C strings at
their own offsets rather than through the Struct codec, because
posix.x loads mid-x-core before that codec exists; the array WIDTH is
the whole per-OS difference, 256 against 65, as _SC_NPROCESSORS_ONLN
is 58 against 84. Thirteen specs, each checked against the system tool
it mirrors: getgroups against id -G, uname against uname -smrn,
cpu-count against hw.ncpu, statfs’s block counts against df -k.
What they unblock is x-coreutils’ second half — the nineteen applets
from chmod to whoami that take it to busybox parity at 92.
A compiled function can call something other than itself. The
compile-asm lane refused every head but the function’s own name, so a
compiled function could recurse and nothing else: anything a lane function
needed from another function had to be inlined into it, which cannot reach a
callee that loops. Two shapes join the self-call. A name bound to an fvar
holding a prim compiles to a call to that prim, its address baked at
generation (#603). (%call HEAD arg ...) takes an operand as its head
— a callback parameter, a pointer read out of a dispatch table — so which
prim runs is decided at run time (#604); C function pointers are exactly
that shape, and a compiled caller of one no longer has to drop back to the
interpreter. Both build the argument list the self-call already built and
hand (callee arg0 ...) to a new jit_call_value trampoline, which
checks the head is a callable prim before it branches: on anything else
the word it would jump through is a length or a character, and the crash
would have no relation to the call site. A head the emitter cannot resolve
— an unbound name, an fvar holding a non-prim, an unimplemented operator —
still refuses at generation. Analyser mode is now declared rather than
inferred: compile-asm takes an optional third argument, defaulting to the
old fvars-present guess, so an integer function may carry a callee fvar
without its params silently ceasing to evaluate. The byte cache’s key
carries that mode in its own right for the same reason it carries the fvar
table’s shape: the two worlds emit different bodies for one source text,
and a key that could not tell them apart would serve the wrong one as a hit.
A lang can word the engine’s errors. (no-such-binding) reached x-lang as
a bare, type-less atom holding one pre-flattened English string — “Unbound
SYMBOL ‘no-such-binding’” — with the symbol already concatenated in and thrown
away. A lang that wanted to say it differently had nothing to work with: no
type, so no dispatch stack to push a handler onto, and no structure, so its
only recourse was to pattern-match English.
The engine now raises a typed ERR carrying (code . subject) — the raise
site’s message literal and what it was about, unflattened — and
x/type/err-io.x pushes the default wording onto that type’s write/display
stacks. The wording is byte-for-byte what C emitted; the difference is that
it is now a handler, so a lang pushes its own over it and pops it again, the
same shape char-io.x has always used for CHARACTER:
(%type-push-display ERR
(fn (_ e) (display (Str8 append "symbole non liée : " (Err subject-of e)))))Err learned the vocabulary: (Err kind-of e) answers 'engine for an
engine raise rather than lumping it in with 'user, and (Err code-of e) /
(Err subject-of e) return the two facts as strings. The engine’s whole
raise vocabulary is five codes, so keying a translation off them is
tractable. boot/printer.x lost the identity test it used to need (#54): a
nil-typed atom had to be recognised by pointer to print at all, and a typed
value simply dispatches — two %-globals gone with it.
Uncaught errors still word themselves in C and read exactly as before: that path runs before any library is loaded, and nothing on a fatal path should be calling into x-lang.
Migrating a guard that reads the message. The value a guard receives for an ENGINE raise is now a structured ERR rather than an atom whose bytes are the message, so the old lifts out of that atom no longer work — they read a slot as a character pointer and hand back garbage rather than failing loudly, which is the one unkind part of this change:
| was | now |
|---|---|
(symbol->str e) |
(%display-to-str e) for the whole sentence |
(Str8 append "" e) |
(Err code-of e) for just the code |
| — | (Err subject-of e) for just the name it is about |
Guards that only re-raise or match on (Err kind-of e) are unaffected,
and (error "msg") still delivers the string itself exactly as before.
The documentation answers a machine now. AGENTS.md (with CLAUDE.md
symlinked to it, so there is one file and not a copy to rot) is the briefing
a coding agent needs and the prose docs could not be: how to run it, the Lisp
assumptions that fail here — no car, no print, subject-last dispatch,
self as argument 0 — the three calls that answer “what exists” without
reading anything else, and which dialect has what. llms.txt indexes the
published documentation for the same audience, listing entry points rather
than every page, so it cannot go stale behind them.
A failed dispatch suggests what you meant. (List mp …) answered no
such static member mp and stopped; it now adds -- did you mean map?.
Selectors within one edit of the miss, or a proper prefix of it, are
proposed, three at most, privacy-wrapped entries skipped — a typo
self-corrects, a wrong concept still does not, and a miss with nothing near
it reads exactly as it did before. Both dispatch-miss sites share the one
cold-path helper; the hot path never reaches it.
Two ways in that were missing, and a silent one that lied. x.sh gained
-c/--eval: evaluate an expression and exit, repeatable, expressions running
in order, so a definition and its use fit in one command. Before it, asking
the language a one-line question — what does (help Str8/split) say, what
does this expression evaluate to — cost a temporary file, which is why
nothing scripted ever asked. The obvious alternative was worse than missing:
echo '(write 1)' | sh x.sh printed a prompt, evaluated nothing, and
exited 0. The pipe the wrapper builds is the engine’s stdin, so the REPL
reaches the caller’s only by reclaiming fd 3 — and repl/loop.x reclaims it
only (when (Sys isatty 3)), which a pipe is not. A non-terminal stdin is
now program text, appended after the library: the wrapper’s spelling of the
cat lib/x.x - | ./x-bin the README has always documented. make
check-wrapper is the gate that keeps all of it honest — the spec suite talks
to the engine, so nothing tested the wrapper at all.
x-lang’s own spec fences say x, not scheme. 2,735 of them, across 154
files. The tag was decoration to the runner — it collects any fenced block the
same way, and only ` ```output ` means anything to it — but it is a claim to
every reader, and it named a language this is not. A reader who trusts it
reaches for car, which is unbound. x is what the rest of the tree already
uses: it is the tag the Pages build highlights (Rouge has no x-lang lexer, so
tools/dev/highlight-sweep.sh supplies one) and the tag the hand-written docs
are written with. The scheme row stays for the langs that are Scheme
dialects.
x runs C, and compiles it. x-cc registered at 27 specs and left this
release at 117, 0 failed — the language arc’s final tier. The front end is
whole and the evaluator is a cell machine, with every expectation an oracle
row against /usr/bin/cc: twin agreement is the spec, so a first
divergence is loud. The C surface grew across the release to pointers,
structs (fields as cell offsets, -> and ., arrays of structs, scaled
pointer steps, typedef, copy), switch with fallthrough, function-like
macros with argument text and rescan, enum, union, function pointers,
initializer lists and the #ifdef/#elif/#undef family.
The half that makes it more than an interpreter is build: eligible integer
functions lower through the engine’s own compile-asm lane to native code
with no external toolchain. Loops become tail self-recursion — params and
accumulators alike ride the self-call, body locals substitute away, if/else
merges as a ternary, return/break/continue are guarded exits, and
nested loops two deep run as a state machine over the one self-call. Pointers
make the program’s memory one raw buffer that the interpreter and the native
twin address alike, so arrays cross the boundary and a native bubble sort
sorts main’s array. A 2M-iteration loop costs 79s interpreted and 9.5s
built.
The JIT compiles each function once per machine, not once per process.
compile-asm emits the same bytes every time it is asked for the same
expression, and a xenon boot asked eleven times — 7.2M evals of a 52M boot,
none of it per-process except the addresses the code bakes in. Those are
recorded now, so the bytes are kept and poured into a fresh buffer with each
address re-encoded for the process loading them: warm boot 46.0M evals
(-11.5%), -12.6% wall; cold pays +5.8% to fill the cache.
Two things had to be true for that to be worth having. Nothing on this path
may walk bytes — hex-encoding the code cost ~24,000 evals per byte and
parsing a text record file by hand ~880, so the code moves through libc
write(2)/read(2) straight out of and into the mmap’d buffer and the
record file is fixed-stride binary. And the cache is the DOOR, with the
compiler as its fallback: loading asm-compile.x costs 2.5M evals before it
emits an instruction, and a warm process never loads it at all.
Correctness rests on the key, because the failure this replaces was a cache key blind to engine identity serving ABI-stale objects that silently misread numbers. The key carries machine, engine release and the fvar table’s shape, and the whole key text is stored in the entry and compared before a byte is trusted — so a hash collision costs a recompile, not a wrong function. Every doubt is a miss rather than an error. Expressions past 128 nodes are not keyed at all: naming one means printing it, and the printer is superlinear (1.4K evals per node at 35 nodes, 8.9K at 400), so a generated body takes the uncached path it always had.
Fixed alongside: the assembler’s relocation slot was never traced by the
collector — asm-new makes seven slots and set-units! said six — so one
collection turned three live records into a single nil. Quiet since it
landed, because nothing yet held those records across an allocation.
The reader’s hottest hook is compiled too. %macro-delimit runs on every
character of every symbol-shaped token — the C symbol analyser calls it per
char to ask whether ' ` , terminates the token — and it was the
single largest per-character reader cost while the numeric analysers beside
it were already compiled. It is the same shape, so it goes through the same
lane, reaching the new jit_buffer_last_char trampoline as
%buffer-last-char. It stays safe on an engine that predates that
trampoline: the compiled twin is built ONLY when the address resolved, and
asm-compile.x resolves it through a %jit-addr-optional that records no
miss — recording one would trip compile-asm’s whole-runtime “JIT
unavailable” refusal (#201) for every compile and drop even the numeric
analysers to interpreted on a fully JIT-capable engine.
A compiled analyser may now call itself. The assembler lane’s self-call
path was written for integer recursion and had never been run by an analyser,
which breaks all three of its integer assumptions: bare params were unboxed
with atomint, self-call arguments were marshalled through jit_mkint, and
the self-call’s result was unboxed the same way — but the tokenizer protocol
fixes the leading params as OBJECTS and a handler returns an object. Any one
is a segfault on the first self-call, which is why a self-recursive analyser
had never worked: it died before returning. The lane now knows which
parameter positions are object-kinded and honours that in the loader, the
boxing and the result. Library-only — no engine change, no new trampoline.
The core tool set is complete at forty-four applets. x-coreutils grew
from 24 specs to 48, taking the bundle from the measured core
(sort/tr/cut/join/comm, sha256sum as FIPS 180-4 in pure x) to the full
busybox shape with the scripting set beside it — echo printf seq test [
diff cmp ls mv touch install mktemp which xargs env date and friends. The
install tree gained the half the shim migration had been waiting on:
tools/lang-kit/spec-gate.sh now ships, so kit-gated bundles stop answering
“no lang kit … upgrade x” against an installed tree — the v0.10.0 fan-out
surfaced it with five bundles red on the missing file and zero on their
suites.
The linter reports shape, on measured criteria. Three advisory rules,
one finding per definition with the numbers carried in the name: ladder
(a nested if chain branching on one variable), ladder-dict (the same,
string-keyed, ≥15 arms) and shape (depth ≥12 and ≥500 nodes). match is
an engine primitive and measures faster than the chain it replaces (605ms vs
897ms over 40 arms and 10k lookups), so ladder has no hot-path exemption —
hot code converts first, not last — while a Dict wins only on string keys
(2.75s vs 6.90s at 25 arms) and is 5x SLOWER than match on integer keys,
which is why the fix rides the key type. shape counts NODES, not lines:
the linter reads forms as data and density runs 4.8–9.7 nodes/line, so a
line count would partly measure the formatter. The 500 is calibrated — at
250 the rule found 83 definitions in a smooth decay with no natural gap.
docs/code-quality.md records the criteria and, deliberately, the plausible
rules that failed, so they do not come back.
Two chapters written from debugging that had cost real time.
docs/crafting-a-lang.md gains garbage collection: NOTHING COLLECTS UNLESS
YOU ASK — every sweep in this tree is a hand-placed (Heap collect), so a
lang that replaces the REPL loop inherits the per-turn sweep as a DUTY and a
long session or a -f script grows without bound otherwise. And an isolated
tokenizer base does not survive collection, which turns that rule into a
dilemma for any lang bringing its own base; the doc carries a verified
seven-line reproduction and the consequence for a suite’s alloc ceiling.
apps/ has a second occupant, and a project has a face. Bitwise draws
the ASCII owl from every source header, set from Roboto Mono outlines over a
field that sha256(name) seeds, in a costume from langs.json
(x -l bitwise -- --all --png). Every quantity is an integer in micro-units,
so the picture is a function of the name alone and the gallery’s browser twin
computes the identical bytes — pinned by parity specs that compare gen.x
against the twin’s checked-in renderings.
The exact tower is exact at every magnitude. Rational arithmetic
silently answered wrong values once cross products passed 2^63 (two ~1e13
denominators multiply to ~1e26): the %rat-* internals used the raw C
binaries, which wrap. They now go through the public promoting operators,
and %gcd reduces by %int% instead of reconstructing a - b*(a/b) (the
reconstruction product wrapped for bigint-sized operands). Two bigint
defects fell out of the same probe: %would-overflow-add?’s negative
threshold itself wrapped for subtrahends past 2, promoting nearly every
negative subtraction; and demotion only ever fired for single-limb
results, so any bigint ≥ the limb base — including 17-19 digit literals
straight from the reader — stayed a stealth bigint that printed like an
int but failed eq? and raw slot ops (str->number’s own overflow verify
among them: an 11-digit all-nines parse raised a spurious “integer
overflow”). Subtraction gets its own exact %would-overflow-sub?, the
demotion window now spans every length that could round-trip, and both
bigint construction paths route through it. Pinned in
tests/x/specs/ext/rational.spec.md and ext/bigint.spec.md. The op
arbitration gained the #584 guard beside it: an undeclared typed pair
with handlers on both sides now raises the lattice’s teaching error at
the operator door instead of falling through to payload-word reads.
16K+ is no longer a cliff. The boot layer’s non-tail recursions each
put one C eval frame group per element on the stack, so any walk over
~16K elements segfaulted outright — first surfaced by (Str8 make 16384
c) in x-awk’s stdin slurp, then found across the family. Str8 make
delegates to repeat’s binary doubling; boot %map1/%mapn-go/
%append2/%filter-go are tail-shaped; %as-list’s iterator drain
actually drains; Iter ->list (behind List from-seq) is tail; the
regex quantifier state collectors got the same cure. Each is pinned by a
16K spec. One behavior change fell out and is deliberate: Str8 make
refuses a NUL fill loudly. Strings are C strings, so a NUL-filled
string is "" — the old list-encode path silently allocated n bytes
behind that empty string, and x-awk’s read wrapper leaned on the accident
as an allocation door until the doubling rewrite collapsed it to one byte
and a raw kernel read overran it (heap corruption, found and fixed the
same day). The teaching error names the real door: the str make prim
(make-str), n writable space-filled bytes.
The boot burst compiles on the engine’s own JIT, never a system
toolchain. Requiring cc at runtime was always against the law of the
tree; the burst now goes through compile-asm, with the cc lane as the
fallback where the engine ships headers, and the compile cache keyed by
engine identity so one engine’s artifacts cannot poison another’s.
Compiled analysers grew up alongside: object params, the self-param
resolved as arg slot 0, no eval in analyser mode, a negative score-set
sign that survives the trip to x1, and the lazy compile-asm stub
forwarding the fvar table.
The core tool set rides the contract now. x-awk arrived at 69 specs and
left feature-complete at 167 — the full POSIX surface: arrays, printf,
field assignment, sub/gsub, getline, RS incl. paragraph mode, user
functions, the CLI front (x -l awk -- ... over files and stdin), and
both pipe forms with SIGPIPE held off; its stdin slurp reads 64K chunks
through the engine’s raw make door. x-grep (29 specs), x-sed (21),
x-make (23), and x-coreutils (24) registered beside it, completing the
core set.
Release-day motions became tools. fan-out-langs.sh runs every
bundle’s suite against a candidate x and stages the requires-release
bumps for the green ones; bump-pin.sh rewrites the engine pin from a
release’s sidecars; diff-engines fuzzes the C and rust engines
differentially and reports the form the diverging line printed. The spec
harness hardened in passing: a batch that dies mid-batch reports the
engine’s last words, a wrapperless @lib run is refused loudly, and
SPEC_SEAM_COLLECT is the run-level door out of the seam collect.
x.sh emits %platform-release beside %install-root, and two docs
joined the tree: crafting-a-lang (the practice beside the contract) and
the bootstrap tool closure scorecard.
Logo left, and the platform grew the row that let it. apps/ is empty:
its only occupant is x-logo now,
arriving green at 83 tests / 0 failed with its examples and its pty contract
intact. Minor rather than patch, and the reason is one seam addition —
%lang-root, the bundle’s own directory — plus %batch? recovering the
meaning it is documented to have. Both are new surface a lang may rely on,
which is what a minor bump is for.
Removing a lang from apps/ moves the payload fingerprint, so this release
is the extraction rather than a tidy that followed one. x-logo declares
(requires-release "v0.9.0") because nothing earlier can serve its viewer.
Logo left the tree, and is x-logo.
It was the last occupant of apps/, and the reason that directory exists —
-l resolving apps/NAME/run.x was added for it. Its own README always
described it as “a second surface language”, which is a lang’s job
description, so the lang contract named it as the
obvious extraction the day that document was written.
It arrived green: 83 tests, 0 failed — the same 83 that ran here as
lib/logo.spec.md, against the same turtle kernel, through the bundle’s own
harness. The pty contract came too and reports what it always did (7 ok, 1
known-fail), and the examples/logo/ch1.logo pin came with it. Nothing about
the language changed; what changed is that it is now acquired, pinned, and
run against a matrix of platform releases rather than only against the tip of
this tree.
apps/ stays, empty. The second resolution step is documented behaviour
and still gated; what left is its only occupant, not the mechanism. See
apps/README.md for what belongs there and what should be a
bundle instead — the short version is that an app entry is self-booting and
therefore nailed to this tree, so anything wanting its own version number
wants to be a lang.
A bundle can find the data it ships: %lang-root (x.sh, a bundle
row in tools/contract/seam.x). import answers where do my modules come
from, and a ./-relative include-once reaches a sibling source file.
Neither means the bytes of that file, and Logo is the first lang to need
that — serve.x hands viewer.html to a browser.
The three ways to do it without this row are all wrong in the same
direction: a cwd-relative literal works only from the tree root, a path
joined onto %install-root finds the platform’s tree rather than the
bundle’s, and reading %import-roots-cell is a platform internal the seam
does not cover. The first of those is not hypothetical — it is exactly how
Logo’s viewer came to be broken in every installed tree, which is the one
environment nothing ran in.
So the wrapper says it, once, because the wrapper is the only thing that
ever knows: it is what searched langs/*/lang.xon and found the directory.
Emitted for the bundle whose entry is about to run and never for a
(requires-lang …) dependency — to the bundle that needs it, a required
lang is a library.
The gate needed a bundle, so there is one. %lang-root is bound only
while a bundle is loaded, which is a third class beside always and
installed, and a class nothing can probe is documentation rather than a
contract. tools/contract/bundles/seamprobe/ is nine lines whose only job
is to be loaded: check-seam now asserts the name is absent from a bare
he/xe/rn and present under -l seamprobe.
Arbitrary-precision decimal floating-point, the sixth tower stage
(x/num/decimal.x, class Decimal, literal suffix d). It exists because
the tower had no exact answer for a decimal fraction. Bigint takes the
integers as far as they go and float is a double, so (+ 0.1 0.2) is
0.30000000000000004 and always will be — 0.1 is not a binary fraction.
(+ 0.1d 0.2d) is 0.3d, exactly, and stays exact however long the sum
runs.
A value is a significand times a power of ten, stored canonically: trailing
zeros are stripped and the exponent raised to match, so 1.50d and 1.5d
are the same pair, zero has one spelling, and = is a pair compare rather
than a walk. The significand is an ordinary exact integer, so it promotes to
bigint on its own — the type is decimal floating point, not a fixed number
of digits.
What rounds is stated, and it is nearly nothing. +, -, * and %
are EXACT: the significand grows to whatever the answer needs. / cannot be
— 1/3 has no finite decimal — so division is the operation that rounds, to
(Decimal precision) significant digits, half-even; sqrt rounds for the
same reason. This is deliberately not Python’s decimal, which rounds
every operation to the context: a context that silently truncates an addition
is a footgun in a language whose whole numeric story is that the tower widens
rather than wraps.
It sits above float in the promotion chain, and the reason is that the
widening is exact rather than conventional: every finite double is a finite
decimal, because m * 2^-k is m * 5^k * 10^-k. So (Decimal from 0.1) is
0.1000000000000000055511151231257827021181583404541015625d — the double’s
true value, not its 17-digit short form — and a mixed pair promotes without
inventing digits. Complex still absorbs decimal, as it absorbs every real.
The literal takes a trailing d and only scores when the suffix is there, so
it never contests a token the float reader wants: 1.5 is a float, 1.5d is
a decimal, and the four-character claim beats the three-character one on the
same digits. write keeps the suffix, for the reason floats keep their point
— a printed 1.5 would read back as a double. (Decimal ->str x) is the
suffix-free text.
x/num/tower carries it across all seven generics, and x/num/decimal
registers rational and complex through the pact rather than importing them,
so the module stands on bigint and float alone.
ln, exp and log10 come with it, as series rather than as a door.
Float’s twenty math methods are each one dlsym, because libm computes in
doubles; nothing computes in 34-digit decimals, so these are implemented
here. All three reduce the argument into a window where a series converges
quickly, run at the precision plus ten guard digits, and round once at the
end — exp by halving until the Taylor series is forty terms and squaring
back (dividing a decimal by a power of two is exact), the logarithms by
folding the significand into [0.3, 3) and running one shared atanh.
They agree with the published constants to the last digit: (Decimal exp
1d) is e to 34 places, (Decimal ln 2d) is 0.6931471805599453094172321214581766.
Two decisions in there are worth naming. The fold window is [0.3, 3) and
not the obvious [1, 10) because inside it the exponent is zero: an
argument near 1, where ln m and e·ln 10 would cancel completely and the
answer’s leading digits would be guard digits, never meets ln 10 at all.
And an exact power of ten answers log10 from its own exponent, because
canonical storage has already stripped it to a significand of 1 — (Decimal
log10 1000d) is 3d, not three followed by thirty-three zeros and a doubt.
Two rewrites underneath them, together a 75x. ln 2d at 34 digits took
9.8s when it first worked; it takes 0.13s.
The first was rounding, and it made everything else faster too. Rounding a
magnitude to N significant digits was q = m / 10^(d-N) with the
remainder — two of bigint’s general multi-limb long divisions, measured at
129ms per call where the multiply feeding it took 3.5. Every series term
rounds once, so that one line was the entire cost of a logarithm. A divisor
below bigint’s base is a single limb and takes its fast path, so the
discarded digits now come off a limb’s worth at a time. That alone took
ln 2d to 0.9s, and /, sqrt, round and rescale ride the same path.
The second was the series themselves, which no longer compute in decimals.
A term is an integer at a fixed scale — v stands for v · 10^S, one S
per series — so addition is integer addition with no alignment, no exponent
arithmetic, no instance to build and canonicalise, and no rounding, and a
term that divides down to zero has fallen off the working precision, which
is the stopping condition for free. Only multiplication does anything: the
product sits at 2S and comes back with one drop. A term went from ~19ms to
~3.
The scale is the accuracy argument, and the two series choose differently.
atanh scales relative to its argument, so a z near zero — ln of an
x near 1 — keeps a full working precision instead of being measured
against a 1 it is nowhere near. exp scales absolutely, because its
accumulator starts at 1 and stays within a factor of e of it; its squarings
are the one place fixed point is wrong (each doubles the magnitude, which
only an exponent can carry for free) and they stayed in decimals.
Both rewrites were checked against the code they replaced rather than assumed: the rounder over 500 random and deliberately tie-heavy magnitudes, the series over 140 random arguments spanning 1e-40 to 1e3 including the near-zero scale case. No disagreement in either, and every published constant still lands on the same digits.
%batch? means “a file was supplied” again, for bundles too. banner.x
documents it that way and every dialect entry uses it that way, but the
bundle path passes --batch unconditionally — it is what keeps the
dialect’s launcher quiet while the bundle loads on top — so every bundle
saw true whether or not the user named a file.
The two questions were the same question until bundles existed. x-sweet’s
(unless %batch? (%banner)) has silently never fired for want of the
distinction, and x-logo’s entry has to choose between its REPL and its batch
reader on exactly this fact: with the wrong answer it consumes the launcher
x.sh appended and reads it as a Logo program, which is silent and prints
nothing. x.sh now restores the flag’s documented meaning after the dialect
boots, rather than by withholding --batch, which is still doing its other
job.
The engine pin moves to x-engine-c v0.1.6, and x-r7rs goes from 43 failures to 27 without a line changing in it.
v0.1.5 adds (base def-global): an operative can define for its caller,
whatever the frame depth. def decides global-versus-local by save-stack
depth, so a define written as an operative — which Scheme’s and Kernel’s
both are — binds in its own frame and loses the binding when that frame pops.
The sixteen are all of error, error objects and guard. guard is the
live case: R7RS guard and x’s guard are different forms sharing a name,
so providing one means shadowing the other, and shadowing interposes exactly
the frame that broke the eval! workaround. x-r7rs changed nothing — it
already preferred the primitive and fell back when it was absent.
tools/contract/langs.x ratchets that budget to 27.
The pin names v0.1.6 rather than v0.1.5, and the extra release is one
line. v0.1.5 added the primitive to the engine’s isa.x without regenerating
its x-engine.xon, so it shipped the digest of the previous manifest and
check-engine-contract refused it — the gate doing exactly its job, on the
first pin bump that could reach it. v0.1.6 is that file regenerated; nothing
in the built engine differs.
tests/x/fixtures/engine-min grows the same row. The paper engine declares
core and no more, def-global is tagged spine, and a group missing one
coordinate is not a capability — so the fixture stopped claiming core the
moment the vocabulary grew. check-second-engine predicted this in its own
header, which is the argument for fixtures over remembering.
syntax-rules works. The reader stopped claiming the dot, and x-r5rs went
green without a line changing in it.
The engine pin moves to x-engine-c v0.1.4, and x-r5rs goes green without a line changing in it — 667/9 to 667/0.
v0.1.4 stopped the dot being a token kind. It sat in
X_SEXP_LIST_CHARS_STR beside the brackets, so the analyser scored it on
sight: correct for ( and ), which really are always single-character
tokens, and false for ., which separates a pair only when nothing follows
it. Every token beginning with a dot was taken whole as the separator, and
the reader’s internal marker was returned to the caller as a value — so
(first (rest (lit (a ... b)))) segfaulted on a raw C satom.
All nine of x-r5rs’s remaining failures were ellipsis patterns, and R5RS’s
macro layer is written entirely in them, so one reader fix moved every one.
tools/contract/langs.x ratchets that budget to 0: a zero is a claim, not a
hope, and it is what makes a tenth failure loud.
The engine takes no view on ... — it is a symbol the reader does not
recognise and passes through, exactly like .foo. One reading changes with
it: (a.b) is the symbol a.b rather than an improper list, which was an
accident of the delimiter set rather than deliberate syntax.
The lang kit: the platform ships a bundle’s checks, as it already ships the
spec runner (x-lang#546). tools/lang-kit/ installs to
share/x/tools/lang-kit, beside tests/spec-runner.sh and outside the
payload fingerprint for that file’s own reason — the digest is library bytes
and answers which release is this, while a tool ships with the wrapper and
the engine.
docs/lang-contract.md settled the ruling for the runner: the platform
ships it; bundles do not vendor it. The runner was the first thing every
bundle would otherwise have copied, and it is not the only one. Across the
six published bundles tools/bundle.sh is 103 lines and differs between any
two by exactly 12 — every one of them the lang’s name or its release URL —
and tests/spec-gate.sh differs by a single line of header comment.
The first member is release-refs.sh, chosen because its evidence is the
freshest rather than because it is the largest: it was written twice within a
day, and the second copy needed three fixes backported the same afternoon —
an issue reference mistaken for a release, a greedy regex window pairing a
name with the wrong version, and a look-back that spanned a neighbouring
pair. A seventh bundle would have been a seventh copy of all three.
It is generic over the manifest rather than per bundle: the table of versions
to check is read from lang.xon — (requires-release …) plus any
(requires-lang …) — so one file serves a bundle declaring one and a bundle
declaring two.
Two ways to reach it, and the second is not a convenience. A bundle’s
specs job has already checked x-lang out; its contract job has not, and does
not need to, because this check reads a manifest and greps a tree. Making a
one-second text check depend on a built engine would be a regression dressed
as consistency. So X_LANG_KIT names a checkout’s tools/lang-kit directly,
and everything else asks x --share-dir — the addressing the runner already
uses, and the failure the contract calls “addressing, not sharing”.
For bundle authors: a bundle that sources the kit declares
(requires-release "v0.8.0") or later. Earlier releases do not carry it, and
the shim says so by name rather than failing obscurely.
(Num expt b -1) no longer takes the machine down (x-lang#545). The
parameter was documented “Non-negative integer exponent” and nothing checked
it. A negative exponent never reaches the (= exp 0) base case — it walks
away from zero — and every even step squares the base, so the recursion
allocated bignums that doubled in width until memory was gone. That is what
made it dangerous rather than merely wrong: a plain infinite recursion trips
the spec runner’s timeout, while this one exhausts memory first, and
alloc-limit! is calibrated in objects, so a handful of enormous bignums
reaches far more RAM than the object count implies.
It was reachable by accident: 2 ** -1 is ordinary Python and (expt 2 -1)
is ordinary Scheme, and any lang implementing exponentiation on top of
Num expt inherited it.
expt now raises err:value on a negative exponent, the same shape
Num isqrt already used for a negative input. The doc stated the contract;
this enforces it rather than changing it — a caller wanting Python’s or
Scheme’s answer needs a float, which is the caller’s decision to make and not
a silent reinterpretation here.
The engine pin moves to x-engine-c v0.1.3, and x-ash goes from unusable
to two failures without a line changing in it. v0.1.2’s
x_prim_make_token_base segfaulted on the first character of any input:
it assigned to the boolean singletons’ cells rather than through them, and
never created the read buffer make_base gives itself. An isolated tokenizer
base is the one thing x-ash cannot do without, so nearly its whole suite was
red for a reason that was never x-ash’s.
| before | after | |
|---|---|---|
| x-ash | 80 failed | 2 |
x-ash’s two are its own string readers, a bundle bug its README documents.
v0.1.3 also names the buffer sizes it had scattered as bare literals — the
reader’s buffer was 256 bytes in make-base and 65536 in the CLI, filling
the same field.
tools/contract/langs.x corrects two attributions and one number, and
the corrections are the point of a file whose budgets are supposed to be
measured rather than transcribed.
x-r5rs’s 37 → 9 was credited to this pin. It was not the engine: v0.1.3
changed four files, none of them the reader, and
src/x-token/sexp/list.c is byte-identical to v0.1.2’s. The 28 came from
x-r5rs itself — R5RS §6.6 ports rewritten against File (21), and exactness
under §6.2.5 (7). The nine that remain say so: they are the ellipsis
group, which is exactly what a pair-dot fix would have removed.
(base def-global) was described as shipping in v0.1.3. It is not in v0.1.3;
it is proposed and unmerged.
x-r7rs’s budget moves 58 → 43. 58 is what that suite reports when
something else is running — an orphaned engine holding a core has made it say
49, 58 and 247 for one unchanged tree, with batches dying mid-run.
check-langs runs six suites in sequence, which is precisely that condition.
Measured twice on a quiet machine it is 43, on both engines.
0.6.0 let a lang leave the tree. This one is about langs that build on each other, and about the platform noticing when it breaks one.
A lang can now name another in its manifest, at an exact version, and be
refused at startup rather than discovering the gap as an unbound symbol
somewhere inside itself. make check-langs runs every bundle’s own suite
against this working tree, which is the first time anything here notices a
behaviour change a bundle cares about — measured when the gate was written:
x-lang green at 2590/0 while the six bundles carried 175 failures between them,
with nothing here saying so. And the two lines at the top of every lang
extraction — a second (include "lib/x-core.x") — no longer segfault, which is
the first wall a new bundle hits and the one that told it nothing.
Indent is the same story one layer down: Logo and x-sweet each owned a copy
of the indentation algorithm, disagreeing at the edges, and now drive one
module.
Minor rather than patch. A new public reader module, a new manifest row, a new wrapper flag and a new gate — new surface of this size belongs in the minor under the SemVer this file adheres to, and nothing published depends on what changed underneath it.
make check-langs — every lang bundle’s suite, against your working tree.
check-seam catches a rename in eight seconds and cannot catch anything else:
a behaviour change, an arity change, a reader that scores a tie differently
all leave this tree green while a bundle in its own repository breaks, and
each bundle’s CI runs on its own schedule against a release, so the break
surfaces weeks later as somebody else’s mystery. Measured when the gate was
written: x-lang green at 2590/0 while the six bundles carried 175 failures
between them, with nothing here saying so. Budgets live in
tools/contract/langs.x and may only shrink, the rule percent-globals.x
runs on. Advisory about presence — a bundle not on the disk is announced and
skipped, because this tree must build for someone who cloned nothing else —
and strict about regression. Deep tier; LANGS='krn sweet' runs a subset and
X_LANGS_DIR moves where bundles are found.
Indent — the stack discipline under indentation-sensitive grouping
(lib/x/reader/indent.x). Logo and x-sweet each owned a copy of it, reached
by different routes and disagreeing at the edges; both now drive this one.
Two layers: measurement (advance / scan / measure) answers what
column a line begins at, and the stack (make / feed / close-all)
answers what opened and what closed. feed returns zero or more close
events followed by exactly one open or same, so no caller counts levels
itself — the loop both previous implementations owned. The two policy
questions the surfaces disagreed on are constructor parameters, not
assumptions: the tab stop, and what a dedent matching no open level means
(open, close or error). Defaults are SRFI-110’s, which are also
Python’s. advance / scan / measure / classify are registered under
catalog ns indent for per-character callers who must not dispatch, the
discipline x/reader/analyser’s terminators already use. (#520)
A lang may require another, and lang.xon says so —
(requires-lang "NAME"). x.sh resolves it exactly as it resolves -l,
depth-first and transitively with a cycle guard, and arms each required
root before the requiring bundle’s own — import-path! prepends, so a
bundle still wins any name it shares with something it builds on. A
missing lang is a refusal at startup naming what is wanted and who asked,
rather than an unbound symbol somewhere inside it. Before this, x-r7rs
found the x-r5rs it extends by probing sibling paths in two separate
places, which recorded nothing about what was needed and would have been
re-derived by every dependent. The same argument (dialect ...) already
makes: a requirement belongs in the manifest. (#526)
A required lang may be pinned to an exact version —
(requires-lang "r5rs" "v0.1.0"), compared for equality and never
parsed, as (requires-release ...) already is. Ordering and ranges would
need a version algebra and, with it, a resolver; equality plus
--allow-lang-skew is smaller and says what it means. What it compares
against is derived, not declared: a bundle carries no version row of
its own, because such a row can only be true at the one commit that gets
tagged — before the tag the tree claims a release that does not exist, and
after it every commit claims one it is not. Instead a bundle’s
make install and tools/bundle.sh stamp a version file from
git describe and from the tag, the same split as this tree’s own
$(X_RELEASE) → <lib>/contract/release. A lang that reports no version
is refused as its own case, because it means a checkout rather than an
install and the fix is different. (#526)
apps/logo/indent.x keeps what
was ever Logo’s — what a block is, and where the tokens go — and its
(indent-level . tokens-reversed) stack and %pop-to are gone. Its two
answers are stated rather than implied by a loop: a tab is one column (a tab
stop of 1), and a dedent to a column no open block sits at opens a block
there (open). apps/logo/types.x measures the leading run with the shared
scan, which hands back the column and the end index separately — the loop it
replaces stepped an index and returned it as a column, which is correct only
while a tab is worth one. (#520)<space><tab>x, +8 says column 9 and a tab stop
says 8. Neither suite had a tab case in it, so nothing caught it in either
direction. The shared module carries the corrected reading, and it subsumes
Logo’s answer rather than overruling it — the next multiple of 1 after n is
n+1. (#520)The release that lets a surface language leave the tree. A lang — Kernel, Scheme, Logo — can now be built, published, acquired, installed and run from its own repository, and the platform is held by a gate to what it promises one. The five 2024-era personalities rotted because nothing held either side to anything; this is the machinery that would have caught all three ways they died.
(Pin bundle "deps/langs") reads a
project’s lang.pin.xon, downloads the tarball its (bundle "sha256:…"
"URL") row names, digests it before tar runs, and publishes it only
once the whole tree verifies. A mismatch quarantines the bytes as
.rejected rather than deleting them. The terms both sides are held to are
The Lang Contract. (#521, #533)x --install-lang URL, with nothing cloned — fetches a published
lang.pin.xon, then the tarball it names, and installs to
<install-root>/langs/<name>. Install and pin answer different questions
and the contract now says so: an install is one unversioned copy for the
machine and the prompt, a pin is a digest-frozen tree for one project. A
failed upgrade leaves the working install untouched. (#534)-l NAME runs an acquired lang — the third resolution step after
lib/NAME.x and apps/NAME/run.x. Unlike those, a bundle does not boot
itself: the wrapper boots the dialect its lang.xon declares, arms the
bundle’s module root, and loads it on top — the shape -F has always had.
So a bundle needs no root-relative literals at all. A dialect this tree
cannot supply, and two bundles claiming one name, are refused before
anything boots. (#530)tools/contract/seam.x declares the eleven names a lang
may rely on, and make check-seam holds the running platform to them in
every dialect. A lang lives in its own repository, so a rename here that
drops %repl-prompt or import-path! breaks it silently while this tree
stays green. Declared rather than derived, because a lang’s call sites are
not in this tree and never will be. (#531)(Sha256 hex-n s n) — the digest, with the length given. hex bounds
itself by Str8 length, which has strlen semantics, so binary input digests
as its leading fragment: a gzip whose fourth byte is a NUL digested as three
bytes. Both engines were already length-driven and byte-accurate; only the
derivation was narrow. The JIT’s adoption check gained two binary vectors,
since proving the two engines agree on text proved nothing about the case
this was added for. (#524)x --share-dir and x --engine-path — a tool outside this repository
can ask where x reads its tree from and which engine it runs, instead of
guessing. The shared spec runner now installs to <share>/tests/, so a lang
runs its own suite with the platform’s runner rather than vendoring 865
lines of it. (#514, #532)%lang-name all along. surface was unavailable: it is already
load-bearing for the C surface the engine contract is built on, and taking
it would give one word two concepts. The Linux personality syscall keeps
its name, because it is the kernel’s and not ours. (#530)deps/, not build/ — the engine, its sources
and lang bundles. build/ is what this project produces; none of these is
built here, and filing them under output is why engine needed a symlink to
be findable at all. Sixteen files reach the engine through engine/ or
$(ENGINE_DIR) and not one of them changed — the link’s own promise, kept.
(#533)SPEC_HEAVY_JOBS is 2 only
where there is memory for two big heaps (~13GB), 1 below 24GB, and 1 when
the size cannot be established. The runner’s own comment said to lower it on
a small box; a default that is safe only when the caller remembers a comment
is a note, not a guard. CI is unaffected: both legs set it explicitly.
(#529)serve.x read its
template from a cwd-relative "apps/logo/viewer.html", which resolves only
when the cwd is the repo root. Every test and every developer run found it;
no installed user did. The path-literal ratchet could not see it, because it
matches include forms and a data path is neither — so that gate grew a
section for app data paths. (#512)--share-dir refused to answer from outside the tree, which is the one
thing it exists for. Mode detection is cwd-based by design, so from outside
a checkout it took the installed branch and computed a share/x no checkout
has. Found by the first lang to use it, which had worked around it by
cd-ing to the wrapper’s directory first. (#532)eval! — x-krn sets
%repl-print on its first line of real work, and its $define! turns on
eval!: 59 of 72 specs fail without it. The table had been written by
reading what the platform offers; a lang reaches for what it needs.
(#532)git archive
--prefix=NAME/ is how a publisher rolls one, and every tarball the feature
was developed against was flat. Descended one level, and only when
unambiguous. (#533)The first release shaped by a second engine: two undeclared assumptions became declared capabilities, eight unwritten laws became written ones with conformance checks, and a pinned project now runs the release it names instead of reporting an errand.
$XDG_CACHE_HOME/x/releases) — verified before unpacked, staged and
published atomically — and hands the whole invocation to it. Fetching asks
consent: --fetch-release, or one question on a terminal. Nothing global
changes; the re-exec’d wrapper runs its own guards against its own matched
tree, and --allow-release-skew still means “run THIS install”. The
refusal survives for the unresolvable cases and now names the flag. (#499)provide shadowing the
lower. Chains (@3 over @2 over @1) fall out of the same rule; outside
a self-load the loud version contract is unchanged. (#503)native/cc (the
engine ships its C headers, so x/tool/compile.x can build against them)
and the jit/asm lane are now stated in the engine contract instead of
assumed. The compiled tower falls back to the interpreted one on an engine
that hosts no C, and the jit/asm specs skip against an engine that never
claimed the lane — both assumptions had silently encoded “the engine is
x-engine-c”.docs/engine-laws.md and checked by conformance: the stamping law (a value
carries its base’s type), the evaluator’s state rides the base, evaluation
and application are type handlers, the callable state spine, and the rest.-ts suffix retires with the
phrase it abbreviated. The engine contract’s covers rows use the ISA’s own
spellings. (#497, #498)first/rest walk encoded x-engine-c’s base spine (the one thing
decision L1 says a caller must not do) and pruned through the wrong cell on
any other layout. The (B cell 'type-alist) door resolves whichever engine
is underneath.(Pin boot "v0.5.0") failed for every project — pin: unknown
release-manifest form. v0.5.0’s pin.release.xon carries the
(engine-release …) row the release script now emits, but the pin
tool’s manifest parser predated the row and refused the whole
manifest. The parser reads it now, optional like layout — a manifest
without the row still parses, a missing value reads back as nil. And
since the pin tool ships inside releases, the fix has to be a
release: v0.5.0 itself cannot be pinned, and this line exists so the
v0.5.x series can be.The object model’s second architecture: one routing model with four doors, and the composition features that shape dispatch tables without changing how routing works.
def-trait defines a behaviour bundle and with composes it into a class; delegates makes forwarding a declared relationship rather than hand-written pass-through methods.def-generic and on give multi-argument dispatch as the cold path beside message passing’s hot one: pointwise specificity, from-lattice tie-break, and teaching errors when no method applies. x/num/tower is the worked example, and the tower’s mixed-type policy on generics is now stated rather than implied.def-record for lightweight named-field data types; Tls ported onto it as the first occupant.def-method! and def-static! add to a class after its definition, with a %missing hook for selectors that resolve to nothing.(private …) and (protected …) blocks, enforced at the dispatch door. Reflection is the documented escape hatch; a leading % stays what it always was, a naming convention.The C engine is a separate repository, and this tree carries no engine at all — x-engine-c, with its own version line and published releases. make engine reads tools/engine/engine.pin.xon — a declared release tag plus one artifact row per platform, digests taken from the release’s own sha256 sidecars — fetches the tarball, verifies it, and unpacks it; make engine-source clones the sources instead for platforms with no published artifact. x-bin is copied to this repo’s root as before, so the wrapper, the spec runners and every tools/check/*.sh script are unchanged. This release pins x-engine-c v0.1.2, whose v0.1.1 and v0.1.2 lines are this release’s own finds: a prim walked off a dotted argument list (#487) and the FFI called a nil function pointer (the #171 class) – both uncatchable SIGSEGVs, both now catchable raises.
The contract manifests travel with the engine, in its tools/contract/: an engine’s description of itself is stronger there than as a copy the library keeps, because a private layout copy is right about some engine and not necessarily the one running. lib/x-core.x includes base-paths.x and obj-layout.x from there as the first things it loads, and pin.x reads isa.x at runtime. check-isa, check-obj-layout and check-base-paths are the engine’s own gates now; check-prim-coverage stays here, because most primitives are reachable only through the library and the honest answer needs both spec suites.
The engine declares its own release: x -V prints the library’s release and the engine’s side by side, and a released engine ships no C — the C-suite gates announce-and-skip here, its own repository having ratcheted them on the build that produced the artifact.
x-lang reaches its engine through one path — engine, a .gitignored symlink in the repo root that make points at whatever the tree builds against: a fetched release, a checkout named by X_ENGINE_DIR, or any unpacked engine directory. Everything downstream uses that one spelling — the boot’s contract includes, the JIT’s -I flags, the gates, the conformance runner — so a second implementation needs no edit to lib/. An engine directory either has sources (built here) or ships a binary (used as-is); the build asks which rather than assuming. Once pointed somewhere explicitly the link stays there, and make install stamps the ISA fingerprint from the engine it actually built against, not from a hardcoded path.
method-of as the sanctioned de-dispatch door; value-call subject-last is routing sugar into the same door; generic functions are the multi-argument cold path; and the C ops cell’s seven spellings shim into the tower’s generics, so promotion has one authority. Composition (with/delegates), contracts, records, open classes and %missing are table-shaping features that never change routing. Measured: a 110-entry static back-hit went 231 → ~100 µs/call, and method-of-hoisted calls sit at ~31 µs against a plain fn’s ~21.%this-class box replaces the %super-class binding; member and static writes go in place through %box-put!; the dispatch arg frame is tail-evaluated.Library version 0.5.0 — x-lib-version had read 0.3.0 since before v0.4.0 shipped, so the banner and x -V under-reported the library by two releases while claiming precision. docs/spec.md, docs/standard-library.md and the README’s maturity line move with it.
The pin records which engine a project was verified against — a lock’s
(engine-release …), compared before a pinned amalgam boots and refused on a
mismatch (--allow-release-skew waives it, loudly). x-lang no longer stamps
its own tag onto the engine it builds: x-engine-c has a version line of its
own, so an installed tree now carries two facts — contract/release for the
library and contract/engine-release for the engine — and each refusal names
which one failed. Both rows are optional; a lock written before them announces
that the pairing is unchecked rather than being refused.
ext/x-engine-c submodule is
gone. make engine fetches the release tools/engine/engine.pin.xon names
for your platform and verifies it against a recorded digest; make
engine-source clones that release and builds it, which is what a platform
with no published engine (the Pi, 32-bit) takes automatically and what the
sanitizer, coverage and engine-hacking flows ask for by name. A tree that has
never acquired one prints those options instead of failing obscurely. Cloning
x-lang no longer needs --recursive, and building it no longer needs a C
compiler.(= 1.5 1.5) segfaulted, uncatchably, from ordinary source text: with no float module loaded 1.5 reads as the dotted pair (1 . 5), and a primitive walking its argument spine tested only for the proper ending. A proper list bottoms out at nil; an improper one bottoms out at an atom, which was then read as a pair — the tail integer’s value word dereferenced as a pointer. No guard could catch it, because a prim call never enters the applicative walk #69 guarded. Which primitives crashed was decided by the library, not the engine: + and < raised cleanly because the tower shadows them, while the unshadowed %isa-keep entries =, eq? and same? reached C directly. Fixed in the engine (x-engine-c#5) by hoisting #69’s own structural test into one guard every C spine walker shares, and picked up here by pinning v0.1.1. A satisfied arity still ignores a junk tail, and a non-pair argument remains the unchecked contract it always was.lib/ and apps/ decides everything, because an amalgam is self-contained only over its include closure and resolves its imports against the installed tree as it boots. See #467.) Three things now carry release identity: the engine reports its own tag as x-release (x -V prints it), an installed tree is stamped with that tag and with a payload fingerprint — one digest over everything the release ships as library — and pin.release.xon publishes the same payload fingerprint beside the ISA one. The wrapper compares the lock’s release tag against the engine’s before the amalgam reaches it, and refuses the pair, naming both tags and both remedies; --allow-release-skew, or (allow-release-skew) in the manifest, waives it loudly for anyone who means it. (Pin verify) reports the same comparison without enforcing it. The tag comparison works on locks written long before this change, because Pin boot has always recorded it.Pin sync classifies a project’s own modules and never vendors them (#223) — and check’s audit half arms the project’s own roots too, so a project whose sources import its own modules can finally sync and check. Previously sync copied the project’s modules into its own overlay, where the stale copies shadowed the real files.Pin boot writes the lock where the guard will find it (#313’s write side) — the boot pin’s engine-pairing guard no longer silently stops firing when overlay roots are reordered.The first release since 0.2.0: the 0.3.x development line and the 0.3.1 release candidates, shipped together. Reproducibility is the theme — a project can now freeze the language it runs on, and a release ships the engine to run it.
pin.xon manifest (root overlay, src scan tree, boot entry) and the Pin verbs over it: init, boot, sync, check, vendor, verify, closure, fetch, resolve, unused. Two tiers, because there are two kinds of drift: an overlay pin freezes the library modules a project imports — vendored closure-wise, digests in a lockfile — and a boot pin freezes the language itself by committing a released amalgam. The lockfile records the release tag, that release’s ISA fingerprint, and the amalgam’s digest; the wrapper arms both pins, announces them on stderr, and refuses to boot an amalgam whose fingerprint doesn’t match the running engine.import-version-once / import-version select among sibling @-suffixed version files by spec string ("1.3", "1.3.*", "^1", "*"). Files are append-only, so a fix is a new patch file and every import whose spec admits it picks the fix up on its next run; dedup keys the base name, and a loaded version that doesn’t satisfy a later spec is a loud error naming both sides. Pin resolve is the dry run, Pin unused the safe-removal answer (#214, #215, #216)..sha256 sidecar, so a release runs with no toolchain and no compile. bootstrap.sh covers the from-source path in one command: clone, acquire the engine the pin names, build it (a ~4-second C89 compile), and optionally install under a user prefix.Socket (#374), then the REST tier on top: https via libssl FFI, DNS, the verbs, and Rest (#412), with basic auth, bearer-token auth, and auto-followed redirects.File stat/lstat adopting it (#371), and base64 and hex (#362).$"…" string interpolation — holes hold code, and the literal scans as one token instead of shattering into fragments (#292); the quote family’s char codes become char literals. Adopted across the library wherever an output call was a text template (#291).Path (#225), a Proc tier over Sys with one correct spawn shape (#226), File read-all/write-all, and the coverage tail: Pq, Deque, Counter, Random uuid, Str8 wrap/fill (#375), from-iso, UDP and unix sockets, Proc options, walk, glob, relpath, copy, temp (#364).lib/he.x, light/default — the old lib/x.x surface, byte for byte), xenon (lib/xe.x, stable full tower — the old x-and), and radon (lib/rn.x, experimental — the old x-or). Atomic weight = library weight, radioactivity = instability; and the governing rule: dialects may differ in what surface is loaded, never in what a shared spelling means. x-lang reverts to being the language’s name only; banners show the full element word (xenon v0.3.0 on x-lang); -l flags stay terse (-l xe). The module layer follows: x/and → x/xe, x/or → x/rn.-l x-and / -l x-or and (import x/and) / (import x/or) no longer resolve (transitional shims existed only within this release cycle); an unknown -l name fails with the wrapper’s inventory listing. lib/x.x remains as the default pointer (bare sh x.sh boots helium), and check-doc-vocab now ratchets the retired spellings out of lib/.lib/x/boot/{helium,xenon,radon}.x; each entry is a body-include plus the top-level launcher (a (repl) cannot ride a nested include: it would read the included file’s EOF instead of the session’s stdin).examples/and/ → examples/xe/, examples/or/ → examples/rn/.lib/x/platform/x86_64.x) — cmp (rr/ri), the six conditional branches (b/eq b/ne b/lt b/ge b/gt b/le as Jcc rel32, sharing arm64’s mnemonic names), a b alias for jmp, and per-arch asm-prologue!/asm-epilogue! (SysV frame + rbx/r12-r14) and asm-load-imm64!. The JIT codegen module (asm-compile.x) remains arm64-only (registers hard-wired) — tracked separately.Arch-tagged specs — the spec runner skips <name>.<arch>.spec.md files on non-matching hosts (uname -m, arm64/aarch64 and x86_64/amd64 normalized); asm specs split into .arm64./.x86_64. variants since the scenarios are ABI-specific (A64’s x0 arg-and-return duality vs SysV’s rdi-in/rax-out)
heap-mark-hook!, heap-free-hook!, heap-mark-root! primitives wired through to x-expr’s heap-group extensible lists; lib/x/sys/gc.x is now a thin re-export layeropt/ — first occupant is opt/x-prim/signal.c; gated by X_SIGNAL (default on), make X_SIGNAL= drops the module and compiles the eval poll outexamples/logo/ch1.logo — Chapter-1 programs from Turtle Geometry (ARCR/ARCL, RAY, POLY/NEWPOLY, POLYSPI/POLYSPII, INSPI)tests/x/specs/applicative/gc-hooks.spec.md (STRESS-only)Object-oriented class system (lib/x/type/class.x) — classes are themselves callable %class objects; instances are %object. Message-passing dispatch with literal selectors ((obj name args), no quotes — the call handler is an operative), single inheritance with super, and a (static …) block of static methods + class-wide members so a class doubles as a namespace ((Class name), (Class new …)). Members are declared directly in the class body (no wrapper) with a uniform form — name |
(name default) |
(name default "desc") — identical in the static block; instance members gain optional default values. Access is encapsulated (external reads/writes only via dispatch; method-internal (member 'm)/(set-member! 'm v) for the private-data pattern). (help Class) lists members and methods grouped static-vs-instance, merged across the inheritance chain and sorted by name. Spec: tests/x/specs/ext/object.spec.md; guide: docs/object-system.md |
lib/x/type/lit-reader.x) — 'expr is reader shorthand for (lit expr) ('sym, '(a b), ''x, and ' as a terminating macro char). The analyser is JIT-compiled in x/and and x/or so it doesn’t slow tokenizing. Spec: tests/x/specs/core/quote-reader.spec.mdBignum → Bigint (#356), Token → Analyser and StrUTF8 → StrUtf8 (#359), the cross-class verbs unified (#358), and the R7RS method names retired (#357). Pre-1.0 surface churn, done in one pass rather than a drip.make no longer mutates the binary — the strip is stamp-gated (#367), and per-variant object suffixes retired the clean-obj brackets (#329).x_base_* → x_interp_* across the interpreter source tree; the file formerly at src/x-base.c is now src/x-interp.c. x_base_* names are reserved for x-expr’s library-level skeleton (file descriptors, hooks, heap-group); x_interp_* covers the environment/control/extras half this project fills in.extras group into x-expr’s heap-group — one canonical storage location for everything GC, registered by name via x_heap_{mark,free}_hook_add() / x_heap_mark_root_add() instead of raw (rest (rest …)) path-walking from x-lang(doc …) forms stash raw metadata at load time; the full processor runs only on first (help)/(apropos)/(modules) invocation (~1s startup savings)(list (lit name) ;N …) to (lit (name name …)); ~1000 lines lighter, same in-memory shapelib/x-and / lib/x-or module-loading layer tightened — drop duplicate posix re-imports (x-core already loads it); pre-compile quasi/unquote reader analysers in x-or so subsequent file parses aren’t ~20% slower; make x/or’s system extensions (syscall/file/socket) opt-in to save ~660 lines per startupmake test-asan promoted to a hard CI gate — the AddressSanitizer baseline reached zero (112 findings at the gate’s introduction → 0): the under-read fixes below plus pinned ASAN_OPTIONS (detect_stack_use_after_return=0; stack-copying call/cc is fundamentally incompatible with ASan’s fake stack, as with any fiber library). A red ASan job is now a real memory-safety regression.H shuffles fold into the compiled loop, and the build is gated on payload size so a small artifact still verifies in pure x (#123, #324).List family walks become iterative inner loops with one normalization (#336), the regex matcher stops rebuilding group splices (#337), bigint division carries an MSB view instead of re-reversing per digit (#341), and/or become nested if in per-element loops (#343), int-only sites fetch cached int prims (#335), and the doc, fmt, cov and lint walks shed per-node work (#338, #339, #342, #344).** glob hole closed (#321, #322).-7.5 was never claimed by the float type and fragmented into -7, a stray ., and 5. A - entry state (requiring a digit next, so the minus operator stays a symbol) fixes it; pure x-lang, no C.(do 7 . 5) / (begin 7 . 5) (e.g. from a malformed literal under a lib without the float type) walked the improper tail into the unchecked rest prim and evaluated a value word as an expression (SIGSEGV). The check lives at the x level, in do/begin’s boot walker: the C core is the processor and does not bounds-check; the walker that accepts the program does. Boot-safe via catalog-fetched type prims and per-dialect probe handles (reader cells and pair-prim cells carry different types).% on floats returned garbage — the float type registered + - * / < = ops but not %, so (% 1.2 1.4) fell through to x_prim_mod’s integer fallback: value-word modulo on two float payload pointers ((gcd 1.2 1.4) famously yielded 8). Floats now dispatch % to a new d%d FFI convention (fmod, matching %’s truncated-division semantics; -lm added for Linux). Rational and complex still lack % ops and inherit the garbage fallback — noted for the tower’s next pass.included the whole lint tool (~150 MB of objects each, never collected: the harness doesn’t GC between snippets), so the batch OOM’d any small-RAM box. macOS appeared fine only because memory compression hid it (peak footprint told the truth). Now loads once per batch via # @lib ../tests/x/lib/lint.x, matching every other tool spec. Investigation notes: object counts and sizes are identical across platforms, and jemalloc matched glibc byte-for-byte — there was no leak and no allocator pathology, just honest accounting on Linux.type?, type-name, units, length, and the write/display hook dispatch) navigated x_type_field_* on whatever sat in an object’s type slot. A child base’s slot holds the x_eval_obj sentinel (a static atom tagging the raw string "BASE"), so e.g. (pair? (Base make)) read 8 bytes past the tag string (ASan global-buffer-overflow) and worked only because the garbage compared unequal. All six now use x_type_op_try’s documented guard: only a pair-tree type has fields; sentinel-typed objects get defined fallbacks (type? → #f, atom units/length, default repr).&local ordering against the stack base) and the segment copies tripped the memcpy interceptor on other frames’ redzones. The capture/restore functions are now exempt (no_sanitize_address, which also keeps the setjmp frame in the captured segment) and copy through an uninstrumented byte loop under ASan; plain builds are unchanged (memcpy, empty attribute).%collect-methods tested (eq? (first (first forms)) (lit method)) without a pair? guard, so a bare member name (a symbol) had its name buffer dereferenced as an object — an out-of-bounds read that 64-bit malloc tolerates (garbage compares unequal, so bare members were skipped by luck) but ASan flags and 32-bit/Pi can segfault on. This was the tracked “eq?/match under-read” blocking make test-asan from hard-gating.&local, missing frame slots the compiler placed below it (gcc spills p_base/cont there); clang’s register allocation masked it. Capture now bounds from a non-inlinable callee frame, and the restore descent keeps a two-pad margin so the memcpy can’t clobber the live restore frame.%asm-arm64? matched only Darwin’s “arm64” spelling, loading the x86_64 backend on aarch64 Linux(op …) definition time, not the caller’s environment at call time. Co-issue: a C-spec for procedure_call / operative_call was updated to match.x_alist_bst_insert no longer path-copies, so fn closures that captured a BST snapshot at definition time stay valid as later globals are added. This was the root cause of an intermittent turtle test failure (>= unbound during include-once of float.x).syscall-id self-parameter — was declared (fn (call) …) which left the actual argument slot empty; one-arg call sites were working by accident. Now (fn (_ call) …) per x-lang fn convention.x_heap_*_hook_add and x_heap_mark_root_add were replacing the whole stack-cell slot instead of pushing into its current list. After one registration, the slot was a one-deep cons cell whose first IS the hook, and the collector walk crashed on the first non-pair internal field. Fixed to push into first(cell).(heap-collect) is now atomic — the env/ctrl/extras base-tree cells and eval-list scratch cells are allocated X_OBJ_FLAG_NONE, so they survive a sweep only by being marked. The old heap-collect = (applicative heap-mark heap-sweep) let x_eval_body push a fresh eval-list cell between the mark and the sweep; that cell was allocated after the mark (so nothing marked it — conservative C-stack scanning is part of the mark and had already run), got freed mid-traversal, and the next pop dereferenced freed memory (any (heap-collect) invoked from inside a begin/do/op body — including the spec runner’s per-test wrapping — could SIGSEGV). heap-collect is now a single C primitive doing mark+sweep with no allocation between; its mark marks its own in-flight frame. The raw (heap-mark)/(heap-sweep) remain exposed but are low-level. GC hooks are now driven through the TCO trampoline so a value-returning hook body doesn’t leave a half-finished call for the sweep to free.x_obj_push_field/x_obj_pop_field exports + OOM-via-x_obj_error in x_obj_alloc, static-library build (make lib) + make install/uninstall + make doc, README rewrite with quick-start + API table + examples/hello.c, GitHub issue/PR templates, contributing guide, and the per-pass GC hook/root list fields plus their registration helpers and the push-into-first fix above.provide/import with include-once deduplication and module registry; (modules) discovery command#/pattern/ literal syntax and compiled pattern matcher(doc ...) forms with (param ...), (returns ...), (note ...) metadata; auto-generated Markdown referencedocs/spec.md) mapping 1:1 to 1229 test cases#() reader syntaxp_base-as-nil to nil = NULL; () parses to NULLlib/x.x into 50+ modular files under lib/x/operatives, data, string, module)cons/car/cdr renamed to pair/first/rest throughout; string-* renamed to str-*fn (applicative), op (operative), wrap/unwrapguard