All notable changes to this project will be documented in this file. This project adheres to Semantic Versioning.
A heap mark can be made more than once in a process. (#1243) image-clear! in
x/tool/image/walk now takes the borrowed flag off the base object as well
as the allocation chain. The engine’s (heap chain-clear!) never reaches
the base object, and a flag left there made every later mark from the base
flag nothing, silently; the module’s “one mark per process” rule is gone.
An object function keeps the arguments it has evaluated rooted. (#1239)
compile-asm’s object world evaluated each argument into a raw stack
slot, so a collect run while a later argument evaluated freed an earlier
one. With (obj set!) compiled, Vector build over a filler that
collects lost its index and stored into a reused word: x-coreutils’ sort,
shuf, comm and the -z reader crashed (exit 139/138), ran past 3.5 GB or
hit the allocation limit. An object function of two or more parameters
now holds them in root-chain cells, as the function world does; one
parameter keeps its raw slot. No measurable cost on (obj ref) and
(obj set!) over 6M calls.
Every Syntax, Theme, Compiled and profile entry shows a worked example
under help. (#1234) The 71 entries of (help Syntax ...), (help Theme ...),
(help Compiled ...), the x/tool/profile functions and their three
modules that answered with a description alone now carry an executable
example or a sample, run by make doctest: the doctest spec grows to
1341.
Term’s speed lookup works on a 32-bit word. (#1233) Term %baud, which
(Term modes fd) uses for ispeed and ospeed, raised on 32-bit Linux;
it now masks with 0xFFFFFFFF.
Every REPL line-editor entry shows a worked example under help. (#1233)
The 49 entries of (help Term ...) and (help Edit ...) that
answered with a description alone now carry an executable example or
a sample, run by make doctest: the doctest spec reaches 1345.
Unlzma decompresses lzma in x, with no liblzma. (#1228) (Unlzma decode s start n) answers (OUT N USED) for one .lzma “alone” stream, the format busybox’s unlzma and lzcat read: the header’s lc, lp, pb and dictionary size, then the range coder’s bytes to the end marker or to the length the header declares. The window grows to the dictionary size as the output does rather than taking 64MB up front, and a distance naming bytes the stream never wrote raises a label 'value whose data holds (lzma . KIND) where busybox reads uninitialized memory. Checked against streams xz --format=lzma wrote: a short input, nothing at all, 300 varied bytes, a run, a repeating input, 16KB over a 4KiB dictionary so the window wraps four times, and a header that declares its length.
The linter fails a wide decimal literal given to a bit operation. (#1242) A
decimal integer that does not fit a 4-byte word, given to &, |, ^,
~, <<, >> or an integer door, or bound by a def, is reported as
wide-decimal and fails make lint-x; a bit pattern is written in hex,
which reads the same on either word. Six such masks in the library and
apps/bitwise are hex now.
help no longer documents names that are not bound. (#1236) (help set-first!),
(help str-length), (help number->str), (help peek-char) and the rest
of twelve retired names now answer “No documentation” instead of
describing a function that raises Unbound SYMBOL. (help eval) says the
environment argument is one pair, (bindings . parent), as an operative
receives it, not an alist.
x/num/fixed-literal adds a reader literal for fixed-point values. (#1235) 0.75q16 reads as the integer 49152 once x/num/fixed-literal is loaded; x/num/fixed itself registers no reader, so a program that only calls the arithmetic does not carry one.
A cache hit no longer rebuilds an entry’s site table. (#1237) An entry’s .asm file now carries the table in the layout a group file uses, so a hit copies the sites in one block instead of matching every relocation record against the slot list, and the entry is held with that table rather than building it a second time. Compiling the linter’s 82 closures from a warm cache drops from 3.02 s to 1.66 s. Every cached entry recompiles once, since both file magics move.
(Sdl poll) reads the mouse’s place on a 4-byte word. (#1238) The float SDL gives for it is taken apart on the integer doors, with bit 31 as its sign.
Sum sysv answers the same checksum on a 4-byte word. (#1167)
x/num/fixed (#1167): fixed-point arithmetic on plain integers, exact through 64 bits on a 4-byte word and an 8-byte one. x/audio/mixer renders on a 4-byte word.
(Sdl audio-queued) answers on a 4-byte word. (#1167)
The engine is x-engine-c v0.2.31 (#1167). A base made with (Base make) allocates on its maker’s chain, so a value bound into it survives the maker’s collects, and (heap-collect) inside it collects. An engine built for 32-bit ARM declares arch arm.
Added: the contract’s arch vocabulary spells arm (32-bit ARM).
Hash fnv-1a is FNV-1a 32 on a 4-byte word; the SHA-3, (#1167) SHA-512, GHASH, X25519 and Ed25519 engines, DES crypt and RSA key generation work on a 4-byte word.
Bzip2 and Bunzip2 and their JIT engines run on 32-bit ARM. (#1167)
The Inflate and Deflate JIT engines build, prove and adopt on 32-bit ARM. (#1167)
bcrypt and its JIT engine run on 32-bit ARM. (#1167)
The AES, SHA-256, SHA-1 and MD5 JIT engines build, prove and adopt on 32-bit ARM. (#1167)
The ChaCha20 and Poly1305 JIT engines build, prove and adopt on 32-bit ARM. (#1167)
The scalar double spec runs on 32-bit ARM through the VFP lowerings. (#1167)
Struct, statfs and set-times read and write by the platform word on 32-bit ARM. (#1167)
x/num/u32 converts a 32-bit unsigned value to and from a (#1167) word’s bits on any word; Crc32, Cksum, Zlib crc32 and Png give the same values on a 32-bit word as on a 64-bit one, and the CRC-32, cksum and sum engines compile on 32-bit ARM.
str->number, rational reading and division, Float’s (#1167) predicates, equal? on vectors, Struct’s 64-bit fields and Host all answer the same values on a 32-bit word as on a 64-bit one; an exact integer quotient past the word is now exact; number? accepts bigints and rationals; joining many pieces no longer hangs on a 32-bit word.
x/sys/sdl clears its event and pixel regions on a 4-byte word, so a posted key reads back as a press, not a repeat. (#1167)
(Sys time-of-day) reads the microseconds under musl on 32-bit (#1167) hosts, where it answered 0.
The terminal screen’s diff compares and copies cells by their (#1167) 4-byte halves on a 32-bit host; a cell whose style alone changed was not redrawn there.
Sys zone reads struct tm at the machine’s word size and (#1167) deflate’s token tables are word-wide; on 32-bit Linux, musl’s stack protector killed git commit, and zone read the wrong fields.
(Socket resolve) reads struct addrinfo at the machine’s word (#1167) size; on 32-bit hosts it answered no IPv4 address for every name.
Deflate’s pure-x matcher compares a word at a time at the (#1167) machine’s width, and Zlib’s streams lay out z_stream by the word; on 32-bit arm deflate’s multi-block output and zlib’s streams were wrong.
(Random sw) draws on a 4-byte word, the same stream as on an 8-byte one. (#1167)
A state image of a library that used the mont-jit engine at load drops its compiled code and rebuilds it on first use. (#1167)
The ssh crypto codecs (SHA-512/384, ChaCha20, Poly1305, GHASH, Montgomery, X25519, Ed25519) run on a 4-byte word; pure-x SHA-512 on a 64-bit host is about 1.6x slower (2.57 s to 4.08 s on its bench). (#1167)
SHA-1, SHA-256 and MD5 on 32-bit. (#1167) The pure-x digests give the right answer on a 4-byte word: their rotates shift logically, and their constants are spelled in hex.
32-bit arm: floats, codecs and the clock. (#1167) The arm32 backend’s encodings are spelled in hex, so they hold once bigint is loaded; the JIT engines written for an 8-byte word decline a 4-byte one and keep their pure-x path; (Sys time-of-day) reads the 32-bit timeval.
Compiled code on 32-bit arm. (#1167) x/tool/asm/arm32 lowers the compiler’s whole vocabulary to A32, so compile-asm, the JIT engines, the first/rest guards and Callback run natively there. The byte cache’s site table is sized by the word.
Floats on a 4-byte word. (#1167) A Float’s payload is its IEEE 754 bits in an INTEGER where the word holds them and in a VECTOR of two words, the low word first, where it does not; Float bits->str, bits->int, int->bits and str->bits take and give that payload. A 32-bit arm backend, x/tool/asm/arm32, emits the float stubs there, with a double as two words in and an answer through a pointer. The printer rounds on the digit string and sizes its limbs to the word.
A hex string reads as the word’s bits. (#1167) str->number with a 0x prefix or radix 16 fills the word from the low end, so "ffffffff" is -1 on a 32-bit word and 4294967295 on a 64-bit one; a pattern with more significant digits than the word has nibbles raises hex wider than the word. Decimal keeps its overflow check.
The JIT lane refuses plainly on a machine with no assembler backend. (#1167) compile-asm and the Callback stub raise no assembler backend for this platform before emitting, instead of ending on an unbound asm-prologue!.
State images write on a 32-bit machine. (#1167) The namer read dladdr’s symbol name at a 64-bit offset and the name tables counted words with a shift by 3; both now take the width from %word-size. asm-backend? answers whether the assembler has a backend for this machine, and the tower loads with empty float cells where it has none instead of ending at the first stub.
x-lang boots on 32-bit arm Linux. (#1167) The platform layer names linux-arm for an engine that declares arm: the arm EABI syscall table, stand-ins for the stat64, chown32 and fcntl64 forms, the kernel’s stat64 layout, and arm’s open flags. Where no assembler backend exists the JIT lane refuses rather than loading x86-64’s, so the tower, the lexers and the bundles keep their interpreted states. A non-gating linux/arm/v7 leg runs the suite in a 32-bit container on the arm64 runner.
Sdl raise! gives a window the keyboard, and Sdl poll reports the mouse. (#1230) (Sdl raise! win) brings the window to the front with the keyboard, and fullscreen! raises the window it switches, so a game that starts fullscreen on a Mac takes keys without a click. (Sdl poll) answers (mouse-down BUTTON X Y), (mouse-up BUTTON X Y) and (mouse-motion X Y HELD), buttons named by (Sdl mouse-buttons); (Sdl poll win) gives X Y in the picture’s pixels.
Every Host and Sys entry shows a worked example under help. (#1227) The 110
entries of (help Host ...), (help Sys ...) and the x/sys/posix
module that answered with a description alone now carry an executable
example or a sample, run by make doctest: the doctest spec grows to
1213.
Every Http and image-walk entry shows a worked example under help. (#1226)
The 66 entries of x/net/http and x/tool/image/walk that answered
help with a description alone now carry an executable example or a
sample; the doctest spec runs 1190.
Added x/sys/web: a window in a browser tab, answering Sdl’s messages. (#1224) (Web open title w h scale) listens on 127.0.0.1 (ports 8080 to 8089, or what Web listen! names) and serves a page whose canvas shows the RGBA picture and whose Web Audio plays the signed 16-bit streams; a WebSocket carries frames and samples to it and keys and gamepad input back as the events Sdl poll answers, with SDL’s keycodes and button names. A program written to Sdl runs in a browser by naming Web instead, with nothing outside x.
Http url-decode undoes percent-encoding, the inverse of url-encode. (#1220)
Added sound: audio streams in x/sys/sdl, and x/audio/mixer and x/audio/tracker (#1217). Sdl audio-open, audio-put!, audio-queued, audio-clear! and audio-close push signed 16-bit frames to SDL3 over the dlopen FFI. x/audio/mixer synthesises voices from wave tables, with envelopes, slides, low-pass, pan and echoes, and mixes them to stereo a block at a time; its seven per-frame loops are compiled (Mixer jit!) and checked against pure x, and a 1024-frame block of 12 voices and two echoes costs 1.5 ms. x/audio/tracker plays songs written as data, their tempo and layers following a tension. The int-form loop machinery is shared as x/tool/int-form and x/tool/int-form-jit (formerly x/gfx/canvas-jit).
compile-fn! carries pair? and symbol? in place (#1213). A compiled closure tests an operand’s type by reading its type name, as type? does, rather than calling the interpreted predicate. A tree walk testing pair? and symbol? on every node goes from 2113 ms to 214 ms.
The linter boots from a state image (#1212). In a checkout, tools/dev/lint.sh writes .images/lint-image.x.ximg (the library and the linter, keyed on both and the engine) and boots every lint child from it. make lint-x NPROC=2 on arm64: 479 s -> 188 s, the same 226 verdicts and the same findings. One child: lib/x/core/logic.x 3.65 s -> 0.46 s, lib/x/codec/fe25519.x 5.44 s -> 2.11 s; the image write is 7 s, once. X_LINT_IMAGE=0 boots from source.
compile-fn! compiles a program’s own closure where it stands (#1206). (compile-fn! NAME F), from x/tool/fn-compile, rewrites the closure F into compile-asm’s new function world, (compile-asm EXPR FVARS (lit function)), and installs the compiled version into F itself, so every holder of F calls it; the interpreted version stays on the Compiled list. Parameters and locals live in cells on the root chain, so a collect inside a callee keeps the frame. A body with a form the rewrite cannot carry stays interpreted, with the reason recorded. (Env cell e ‘name) answers a name’s binding cell. fib 22 goes from 95 ms to 14 ms; x-chess perft 3, with its board functions compiled, from 2096 ms to 741 ms.
Bundle CI runs the main leg off pull requests, and takes the runners as an input (#1211). .github/workflows/bundle.yml runs a pull request against the declared release only; the x-lang main leg runs on a push to the bundle’s default branch, on its schedule and on a manual dispatch. The new os input names the runners as a JSON list and defaults to the ones x-lang’s ci.yml tests.
An installed x writes state images again. (#1208) Since 0.48.0 make install
left out the engine’s base-slots.x contract, which the image writer
includes, so an installed x could not write an image for a dialect or a
bundle and booted from source on every run. Every contract file the engine
ships is installed now; x-roguelike’s image writes again (188,592 objects).
A window’s picture can fill it at any size. (#1209) (Sdl scaling! win 'fit)
shows the picture as large as the window allows, its shape kept and bars
on two sides only; 'whole keeps the whole multiples open starts with.
Sdl reports gamepad buttons, axes and hot-plug from poll, and fullscreen! switches a window to desktop fullscreen and back. (#1207)
A loop helper’s parameter that its call fills with the outer variable of the same name is no longer reported as a shadow. (#1205) (def go (fn (self seed acc) …)) followed by (go seed ()) carries seed on, as (let ((x (f x)))) refines x; a helper fed anything else still shadows. Over lib/ and apps/ the shadow warnings go from 121 names to 38.
The linter no longer reports a name as unused or shadowed when it is not. (#1203) Uses inside a callback passed to a block-form selector, in a match clause, or in a provide list now count, a body’s nested imports are known, and a self slot named for its definition or a parameter named like a sibling method is no shadow. Over lib/ and apps/ the unused warnings go from 163 names to 71 and the shadow warnings from 207 to 121.
Type and the class introspection doors show worked examples under
help. (#1202) Type’s lookups and cell navigators, object?, class?, class-of,
class-name, class-parent, instance-of?, the class-*-fields/-methods
listers, class-static-ref and method-of now carry examples run by make
doctest; the handler-wiring entries carry samples. The doctest spec
grows to 1059.
The core primitives show worked examples under help. (#1200) (help eval),
(help call/cc), (help null?), (help >) and the rest of 43 entries
that answered with a description alone now carry an example, run by make
doctest, or a sample for the ones that print or load: the doctest spec
grows to 1049.
Added x/gfx/canvas: RGBA canvases with fill, copy, blend, stamp, nine-piece, text and light operations, compiled (Canvas jit!) and checked against pure x. (#1198)
The engine is x-engine-c v0.2.28 (#1180), whose release carries the base-slots contract file.
Help shows a worked example for every method of Num, Float, Decimal and Complex. (#1185)
x/sys/sdl opens a window and shows an RGBA buffer in it, through the host’s SDL3. (#1193) Sdl open makes a resizable window for a w x h picture drawn at whole-number scale; Sdl show! puts a buffer on screen; Sdl poll answers key, quit and close events; Sdl read-pixels reads the drawn frame back. SDL3 is loaded with dlopen, as Zlib loads libz, and Sdl load! names a copy to try first.
Bundle CI checks out the langs a bundle requires. (#1191) .github/workflows/bundle.yml clones each (requires-lang "NAME" ["vX.Y.Z"]) row of lang.xon beside the bundle as x-NAME, so a bundle written on top of another lang runs its suite on CI.
null? runs compiled. (#1190) x/tool/obj-compiled compiles it beside
(obj ref) and (obj set!) and installs it into its own closure object,
so every list walk calls the compiled version; it answers as the
interpreted one on every kind of value, the integer 0 and the character 0
included. A call costs what a C primitive’s does: 200k calls take 466 ms
against 525 ms interpreted and 488 ms for first. On x-chess, timed in one
process with only null? switched, perft depth 3 takes 1274 ms against
1295 ms interpreted.
Char, Assoc and Set show worked examples under help. (#1184) The 48
entries of (help Char method), (help Assoc method) and (help Set
method) that answered with a description alone now carry an executable
example or a sample, run by make doctest: the doctest spec grows to
861.
A cold Lexer make no longer spends quadratic memory recording its
cache plan. (#1187) %plan finds states, cells and buffers by address and
reads each state’s held names as the state is made, so it is one pass
over the states. On x-python’s 254-state lexer the plan step allocates
1.22M objects instead of 6.53M; a cold make of a 686-state lexer peaks at
1,019 MB instead of 2,717 MB (9 s instead of 11 s). The plan text is
unchanged.
A compiled routine can replace an engine routine in a running base. (#1180)
compile-asm takes (run TYPE ...) as its calling world: a routine with
the engine’s one signature, whose parameters are the words of the
argument run the engine passes through a slot. (Base slots),
(Base slot b name) and (Base slot! b name routine) read and write the
base’s slot vector by the names in the engine’s base-slots contract. A
compiled heap-sweep routine installed this way is called by (Heap
collect), and storing the engine’s address back restores it. Needs
x-engine-c v0.2.28, whose release carries the contract file.
(obj ref) and (obj set!) run compiled. (#1186) Every vector, record and
field access reads or writes a data word through one of the two, and
both were interpreted closures. x/tool/obj-compiled compiles each with
compile-asm and installs it into the closure object itself, through the
new (Compiled in-place ...), so a door bound before the import calls
the compiled version. Both are switched to interpreted before a state
image is written and compiled again after a load, like every other
Compiled entry. x-base, xenon and radon import the module at boot.
Measured on x-chess (helium, which imports it itself at a cost of about
310 ms): perft depth 3 from 3476 ms to 1411 ms, and a depth-3 search
from 241 ms to 141 ms.
Fn’s combinators and Gen’s consumers show a worked example under
help. (#1183) 45 entries of (help Fn method), (help Gen method) and the
boot list walkers that answered with a description alone now carry an
executable example or a sample, run by make doctest: the doctest
spec grows from 802 to 849.
A declared digit option is an option (#1182) (x/sys/opts): a token led by - and a digit the declaration names, such as ssh’s -4 or scp’s -3, alone or leading a cluster, is read as that option; before, any such token was an operand, as a negative number is. An undeclared digit is still an operand, so sort -5 reads as before.
*%str-concat no longer copies totaln/2 bytes for many small pieces.** (#1181)
Below 256 KB the fold runs in spans of total/sqrt(n) bytes, so a join of
n pieces copies total*sqrt(n) bytes, and every Str8 join and string
builder that funnels through it leaves that much garbage instead. A
200K-piece, 150 KB join (x-cc’s object file) passed 3.1 GB and was
stopped; it now finishes in 1.2-2.3 s at a 1.65-1.8 GB peak for the
whole process, 1.05 GB of which is building the pieces. A 2000-piece
8 KB join: 10.1 -> 9.3-10.2 ms, and 200 of them peak at 1.0-1.2 GB
instead of 2.3-2.5 GB. Counting the pieces costs a 4-piece join about
2-3 us (20.2-21.4 -> 22.0-24.4 us).
The arithmetic and comparison primitives show worked examples under
help. (#1179) (help +), (help <), (help <<) and the rest of the 13
entries that answered with a description alone now carry an executable
example, run by make doctest: the doctest spec grows to 802.
A grammar’s class function may answer its token’s pieces (#1178) (x/tool/syntax): each (offset length class) inside the token, in order, so one token can be drawn in several classes – a $var inside a shell string, the name in name=value, a make target before its colon. A class symbol or nil works as before.
Every Dict method shows a worked example under help. (#1177) The 13
entries of (help Dict method) that answered with a description alone
now carry an executable example or a sample, run by make doctest:
the doctest spec grows from 763 to 778.
The engine is x-engine-c v0.2.27 (#1175). Every object’s extra meta words start at zero, from x-expr itself (x-engine-c#116, #119), so a form read from standard input keeps its stamp under any allocator: lib/source.spec.md passes under AddressSanitizer, and with #1165 the ASan run on main is green again.
Every Vector method shows a worked example under help. (#1174) The 8
entries of (help Vector method) that answered with a description
alone now carry an executable example, run by make doctest: the
doctest spec grows from 754 to 763.
CI’s macOS legs run on macos-15. (#1173) GitHub retired the macos-14 image on 2026-11-02.
The string classes show a worked example under help. (#1171) Str8/iter and
the four StrUtf8 codec methods that answered help with a description
alone now carry an executable example, run by make doctest: the doctest
spec grows from 692 to 697.
The x86-64 backend takes bl as a second name for call (#1172), as it
takes b for jmp, so a call into another object keeps the one
vocabulary on both hosts.
A link library, x/tool/link, and the assembler’s hand-off to a linker. (#1172)
An object is (SECTIONS SYMBOLS RELOCS): sections (NAME ALIGN SIZE BYTES),
symbols (NAME SECTION OFFSET SCOPE), local or global, and relocations
(SECTION OFFSET KIND NAME ADDEND) whose kinds are the caller’s. link
lays objects out as one program, section by section in object order and
each at its alignment, asks the caller where each output section will be,
resolves every name (a local shadows a global; an undefined or a twice-
defined name refuses) and calls the caller’s patcher at every relocation;
link-undefined lists the names no object defines; link-write! and
link-read keep an object file, the records as text after a first line
that says how long they are, then the bytes. The assembler gained
asm-unresolved!, the patches naming labels it never placed, taken out
so the code finalizes as a section and the linker resolves them, with
page-relative patches taken as well on request; asm-labels;
asm-patch-at! and asm-reloc-at!, the backend’s patcher and the
64-bit relocator on code no assembler holds; and asm-adr-far!, an
address in reach of a whole executable – adrp and an add with the
patch types arm64-adrp and arm64-lo12 on arm64, whose encoders
asm-patch-adrp! and asm-patch-lo12! a linker calls with real
addresses, and lea from RIP on x86-64. The label index finalize built
inline is now asm-label-finder, shared. x-cc’s objects and linker are
the first client.
Every List method shows a worked example under help. (#1168) 55 entries of
(help List method) that answered with a description alone now carry an
executable example, run by make doctest: the doctest spec grows from
692 to 749.
The include-paths spec restores the import roots it arms, so a batched run no longer leaks them into the next file. (#1165) make test-asan had failed on module-imports’ roots message on every main run since that case landed.
x/codec/png decodes PNG, and x/tui/picture draws an image as terminal text. (#1164) Png decode and Png size take every colour type, depth, interlacing and transparency, to RGBA, with a compiled engine for the row loops (Png jit!); Picture lines draws an RGBA image as half-block text in 24-bit or 256 colours, and Picture fit sizes it.
A Lexer make in an image writer’s child sweeps after each state it compiles, so a cold-cache write no longer holds every compile’s garbage at once. (#1162) x-ash’s image write in a fresh x-os build container went from 2290 MB to 954 MB peak.
(Base64 encode-region REGION) encodes a region (BUF START LEN) read by count, NULs included, and answers the text as a region. (#1161) A compiled encoder is built on its own for inputs of 4 KB or more, and (Base64 encode-jit!) builds it on demand; output is the same either way. 1 MB encodes in about 0.2 s, where encode-bytes takes minutes.
Bzip2 compresses to bzip2 in x, with no libbz2. (#1160) (Bzip2 compress s level) answers (OUT N) for one stream at a block size digit of 1 to 9; (Bzip2 stream) and feed write one a piece at a time, a block as the pieces fill it. The streams are byte for byte the ones busybox’s compressor writes – the run-length coder, busybox’s fallback sort, move-to-front, zero-run coding and up to six Huffman tables, a table for each group of fifty symbols – checked against what bzip2 1.0.8 wrote at -1 and -9. The run, sort, move-to-front, selection and writing loops are compiled ((Bzip2 jit!), past 4KB on its own, always for a stream) and adopted only after writing the check streams exactly as the pure-x ones do.
make bench counts the boot alone. (#1159) Each row’s counters are taken before
x/tool/profile is imported, so the profiler’s own load no longer lands in
them; profile-snapshot returns the counters and profile-dump prints one
it is given. Evals, before -> after: x-core 24.21M -> 22.17M, r5rs 55.38M
-> 50.81M, r7rs 59.91M -> 55.31M, sweet 27.03M -> 23.98M, r5rs-img 10.55M
-> 5.98M, r7rs-img 10.59M -> 5.99M, sweet-img 6.08M -> 2.03M.
A selective import into a scoped module asks the engine whether the name is already bound, rather than walking the module’s tree in the library. (#1157) The check behaves the same; %module-import-one! is nine lines shorter.
A scoped module’s names cost a lookup from inside it one tree search, not a walk. (#1154) The engine is x-engine-c v0.2.26, whose #114 keeps a tree behind
an environment the language makes with pair, as a module’s is, and the
library reads it as one: Env bindings and Env names flatten a module’s
frame as they do the root’s, and a selective import’s check for a name
already bound walks the tree. x-coreutils’s vi, a module of 337 names,
goes from 2.6 back to 1.0 ms a keystroke.
make bench compares like with like: every bundle row boots from source, and -img rows time the state-image boot. (#1153) The r5rs, r7rs and sweet rows had been loading x.sh’s cached state image while x-core booted from source, so the bundles read as faster than the core they run on (r5rs 1.06s against x-core 2.80s). From source, on engine v0.2.25: x-core 2.73s, r5rs 5.92s, r7rs 6.69s, sweet 2.91s; from the image: r5rs 1.09s, r7rs 1.11s, sweet 0.66s.
Socket tcp-listen and tcp-listen-on take a list or Gen of candidate ports. (#1149) (Socket tcp-listen (list 8080 (Gen range 9000 9010) 0)) binds 8080 if it is free, else the first free port from 9000 to 9009, else one the kernel chooses; (Socket local-port fd) reads which. Only a port in use moves on to the next candidate. When all are in use the eaddrinuse Err names the ports tried. An INTEGER behaves as before.
(Gen flatten v) walks lists and generators in their place, lazily. (#1148) (Gen flatten (list 8080 (Gen range 9000 9010) 0)) yields 8080, then 9000 to 9009, then 0, and steps the range only when the walk reaches it. Strings, vectors and other objects are single values, as in List flatten; (g flatten) walks a generator’s own values.
An image written from a library that aliases first or rest at load is no longer refused. (#1146) The writer’s child leaves the prims raw (the guards of #1135 are compiled and no image can name them); the globals are armed after a load. x-ash’s boot image wrote unnameable: 2 on x-lang main.
help on a class with no summary of its own shows its module’s doc. (#1145) It adds a “module:” line; x/tool/pin and x/tool/compile/pipeline have module docs.
x/tui/window passes over a link that takes no place on the page. (#1144) Tab on a page holding an empty link, or a link whose only text is an image, segfaulted the engine; next-link and prev-link now skip such a link, and Window select of one selects none and answers nil.
The tutorial’s list examples run. (#1143) map, filter and fold were shown as bare names and are unbound; they are (List map ...), (List filter ...) and (List fold ...). The tutorial, README and AGENTS.md also lose a CLAUDE.md that no longer exists, a missing hook link, make lint (it is make lint-x), the ctrl-c text the REPL contradicts, and hand-copied counts that had drifted; docs/index.md links the engine laws.
help says what a name with no entry is. (#1142) A module not yet loaded answers with the import that loads it; (help map) lists the documented names ending in /map; a bound but undocumented name says so. apropos hides %-private names unless the pattern starts with %, and prints a line when nothing matches. An example with no result prints no ` => `.
AES-GCM: (Aes gcm-seal ...) and (Aes gcm-open ...), with GHASH and compiled engines for AES and GHASH. (#1139) Galois/counter mode (NIST SP 800-38D) with 128-, 192- and 256-bit keys and a 96-bit IV, as SSH’s aes128-gcm@openssh.com and aes256-gcm@openssh.com use it. gcm-open answers nil when the tag does not match. (Ghash hash ...) is the hash on its own. (Aes jit!) and (Ghash jit!) build the compiled engines, which are also built on their own for large inputs; output is the same either way.
X_LANG_DIR finds the bundles with or without a trailing slash. (#1141) The wrapper appended */ to it as given, so the X_LANG_DIR=/path/to/x-NAME/.. line in the bundle READMEs found nothing; it ends the path in a slash now.
x --help lists every option the parser accepts. (#1140) --install-lang, --share-dir, --engine-path and -- were parsed and never listed; the usage line now reads [OPTION]... [-- ARG...], and -l says what it takes (a dialect, a library or app, or an installed lang). The wrapper gate reads the parser’s option labels and fails on one the help omits.
The library’s engine doors are one module. (#1138) x/core/doors provides each hot catalogue door once – int+, int<, str-byte-ref, obj-ref, ptr-set!, type-of and 50 more – and every library file imports the ones it calls instead of fetching its own, 1,129 fewer prim-ref lines across 136 files; a door imported costs what a local def cost, 0 objects a call for int< where < is a closure call. x/codec/flate-codes provides the DEFLATE length and distance tables Inflate and Deflate each carried.
HTML forms reach the page. (#1137) (Html parse) keeps a form’s fields as inlines beside the links, (field TYPE NAME VALUE ON OPTIONS FORM), with the form’s number, action and method on each; x/tui/page draws them as lynx does, in one piece, and files them with the links, so a window selects a field as it selects a link. An Http response carries a url entry, where it came from after redirects.
The second-engine check shows its DERIVED context portably. (#1136) It printed the lines after a DERIVED: finding with sed’s ,+4 address, which GNU and BSD sed read but POSIX, busybox and x-sed do not; an awk counter now prints the same lines on every awk.
(first ()) and (rest ()) raise instead of taking the process down. (#1135) The engine’s prims stay unchecked; the globals the library binds are compiled guards (x/core/first-rest, object functions in the assembler lane: a nil test and the read), so a list walk costs what the raw prims do (4M steps: 10.2 s raw, 9.8-10.0 s guarded) and (first ()) raises first: () has no first. compile-asm takes the symbol object as its third argument for an object function called from x: arguments evaluate and stay objects, the result is one. x/tool/safe-access, the opt-in closure guard at 3.3x, is retired.
The library carries what the bundles copied. (#1133) Err text (the message of any error value), Sys setsid, Proc quiet! (a daemon’s descriptors) and Lang symbols-bare! (write symbols without the quote), each replacing copies in the bundles.
An interactive lang session runs with no guard around it again. (#1132) In v0.42.0 and v0.43.0, x -l LANG with no program ran the lang’s session under the batch loop’s guard, so a ctrl-c partway through an entry could end the session with Error [...]: STOP where it should have discarded the entry. The batch loop now wraps only a program: a -f or -F file, a piped stdin or -c.
Highlighting grammars come from the bundles. (#1129) A lang.xon’s (grammar NAME (module M) (files ...) (names ...) (interpreters ...)) forms are a catalogue x/tool/syntax reads, importing a grammar’s module the first time a file needs it. Syntax for-file takes the file’s text too and picks the grammar a vi modeline names, else the one for the #! line’s interpreter, else the one for the file’s name; Syntax catalogue lists them.
The engine is x-engine-c v0.2.25 (#1126). (base def-global) is gone from the engine; a root definition is (eval! (list (lit def) n (list (lit lit) v))), and the engine-min fixture and the sandbox spec no longer mention the primitive.
The release packager and the ASan boot gate call make with no GNU-only flag. (#1127) tools/release/package.sh drops --no-print-directory, and tools/check/asan-boot.sh builds the sanitizer engine with cd DIR && make in place of make -C DIR, so both run under BSD make and x-make as they do under GNU make.
%str-concat raises a type Err on a non-string piece instead of reading raw memory. (#1125) Each piece is type-checked in the length walk that already visits it, with one (type ?) call: no heap objects added (197 a call over 20,000 four-piece calls, before and after), about 3% CPU on the bare call, and Str8 join/append/str, Json emit, Xon emit and Markdown html unchanged within noise. A bundle that walked its own check before calling it (188 objects a call) can drop the walk.
Path join costs 257 heap objects a two-part call, down from 6,275. (#1124) The seams run on the str, char and int doors (two byte scans, slices only when a slash is cut, two concats) instead of a fold of str=?, Str8 ref and Char =? sends; 551 -> 24 us a call under the tower, under Str8 append’s 810 objects. The root seam now holds the documented contract: (Path join "/" "etc") is "/etc" (was "//etc") and doubled slashes collapse to one; a non-string part raises 'type.
The Makefile needs no reading only GNU make has. (#1120) Every GNU-only idiom is replaced by the form GNU make, BSD make and POSIX 2024 read alike, and every target keeps its commands:
$(shell git describe ...) for X_RELEASE is a backquoted command the shell runs at the recipe; every use sits inside double quotes, where it runs, and a value given on the command line passes through as a string. (!= would be GNU 4 and BSD only; macOS ships GNU 3.81.)$(if $(X_ENGINE_DIR),...,$(shell readlink ...)) for ENGINE_SRC is the same, an if/elif in a backquoted command; ENGINE_ENSURE reads it once into a shell variable.$(MAKE) --no-print-directory -C engine TARGET, seven places, is cd engine && $(MAKE) TARGET; the two coverage targets put their copy on a recipe line of its own, since the cd is the last thing in the macro’s shell. The flag is dropped from the four recursions into this tree.$< and $*, are one explicit rule a variant; $< in the x-bin copy rule is the prerequisite’s path written out, since POSIX and BSD make define it only in inference rules.ifeq ($(IMG),0) around test-x is one rule whose prerequisites and recipe come from a computed name: IMG=0 selects the _SRC pair and any other value the image pair, as the conditional did.$(CURDIR) and $(abspath $(IMG_DIR)) are $(pwd) and $(cd DIR && pwd) read at the recipe; images has made the directory by then.$(patsubst %,*/lib/%.spec.md,...) for ASAN_CRYPTO_SPECS is the 34 globs written out.$(if $(LIB),-l $(LIB)) $(if $(ROWS),-n $(ROWS)) in profile-x is the argument list built by the shell with set --.$(MAKEFILE_LIST) in help is Makefile.
The install stamp printf '%s\n' '$(X_RELEASE)' is double-quoted so the command runs there too.A lang’s -f file is text for its reader, not an include form. (#1121) #1117 emitted every -f file as an include form, which handed a Logo or Python program to the x reader: x -l logo -f prog.logo printed nothing. With a lang leading the file is cat’d whole again; with a dialect leading the include form stays, and a raise in a program with no file reads bare Error: ... rather than a line relative to the form just read.
Html parses a page into Markdown’s blocks, and Entity reads character references. (#1122) (Html parse text) answers the blocks (Markdown parse) answers (headings, paragraphs, pre, blockquote, nested lists, hr, a table as a paragraph a row) with a, b/strong, i/em, code, br and an img’s alt text as inlines, tolerant of unclosed p, li, td and th and of attributes in either quote or none; head, script, style and comments are dropped and white space collapsed. (Html title text) answers the <title>. x/codec/entity holds HTML 4’s named references and every numeric form for both codecs; Markdown now decodes all 252 names, not 62.
x-lang’s own tree no longer calls (base def-global). (#1114) The image writer, the image loader and the specs bind in the root with a def evaluated by eval! (engine law 10), the value under lit; the engine primitive is retired now that the lang bundles have moved (x-krn#26, x-r5rs#35, x-sweet#33, x-r7rs#35).
A raised Err displays as the sentence it was raised with. (#1118) The REPL showed a library-raised error as Error: #<err:type no + for STRING>, the inspection form; it now shows Error: type: no + for STRING, the sentence (Err raise ...) wrote and what the engine printed for the same error uncaught. Err gains %str (display) beside %repr (write), and (display err) or (Str8 str err) answers the sentence anywhere.
A program’s raise is reported with its file and line. (#1117) A raise in -f, -c or a piped program printed *** ERROR: Unbound SYMBOL 'frob from the engine’s own loop and exited. lib/x/repl/batch.x reads a program the way the interactive session does, one form under a guard, so the same raise reads Error [prog.x:3]: Unbound SYMBOL 'frob', in the library’s wording; the run still ends with status 1. x.sh emits (batch) ahead of the program and the user’s -f/-F files as (include ...) forms, which is what gives their forms a file and a line; bundle entries are cat’d as before.
An import of a module no root holds is refused by name. (#1116) (import x/nope/missing) reached the engine’s include and raised include: cannot open, with no module name, path or roots. import probes the resolved path first and refuses with import: no module x/nope/missing -- searched lib, the roots in search order; the loaded mark is set after the probe, so the name can be imported once it exists.
Term reads and sets a terminal’s modes by name. (#1115) (Term modes fd) answers (NAME . VALUE) for each mode RFC 4254 names that this OS has – a control character’s code (nil when turned off), a flag’s BOOL, whether the size is cs7 or cs8, and ispeed and ospeed in baud – or nil when fd is no terminal; (Term modes! fd modes) sets them in order, passing over a name the OS lacks, and answers whether tcsetattr took them.
An entry may start on the Changelog: line. (#1113) The collector reads the rest of that line as the entry’s first line, so a commit written that way is collected and passes the gate rather than refusing every later push.
Bunzip2 decompresses bzip2 in x, with no libbz2. (#1112) (Bunzip2 decode s start n) answers (OUT N USED) for one stream; (Bunzip2 stream), feed, finish, done?, ok? and rest decode one a block at a time as the input completes it. A malformed stream raises a label 'value whose data holds (bunzip . KIND). The symbol, link and walk loops are compiled ((Bunzip2 jit!), past 4KB on its own) and adopted only after agreeing with the pure-x ones.
docs/crafting-a-lang.md no longer sends a lang author to the base/def-global door. (#1110) The passage on def at depth described the save-stack rule x-engine-c v0.2.10 retired; it now states engine law 10 (def binds in the current environment, eval! evaluates in the root) and names eval!, with the value under lit, for a definition made at depth that must bind for the session.
Lexer until takes a close of any length. (#1109) The close is matched through a chain of states, each falling back to the longest start of the close the bytes read still match, so """ and ` ``` ` close a span where only one or two bytes could before; a close of two bytes or more is always part of the span. Specs cover a three-byte close, a close whose start repeats, and a close that starts like the open.
Syntax highlighting colours are a theme, and a theme file changes them. (#1108) x/tool/syntax gains Theme: each class’s Style as data, (Theme current) the one Syntax ansi, sgr and code draw with. The default theme is the old table, byte for byte. $X_THEME, else $XDG_CONFIG_HOME/x/theme.x, else ~/.config/x/theme.x, holds (theme NAME (CLASS ATTRIBUTES FG BG) ...) forms, each over the theme of that name or the default, and (use NAME); it is read on first use and never evaluated. The REPL, the documentation highlighter and vi all follow it.
Cksum and Sum: the checksums of cksum and sum(1). (#1107) (Cksum of s) is POSIX cksum’s CRC, the length folded in; (Sum bsd s) and (Sum sysv s) are sum’s BSD and System V checksums. Each reads by count, so a NUL is a byte like any other, takes an optional START and LENGTH, and builds a compiled engine (JIT) for 4 KB or more as Crc32 does. Over 1 MB the three sums cost 0.044 s once their engines are built, which costs about 0.5 s each a process.
The operator guards, when and unless test with match, not if. (#1106) The guards on <, &, |, ^, <<, >> and ~, the derived >, <= and >=, and when and unless are match clauses: < costs 10 objects a call over the door (was 25), >, <= and >= 20 (was 69-86), and when and unless 8 (was 19 and 26). A missing operand and a nil one raise one message, <: needs two operands, neither nil.
Term key names Page Up and Page Down. (#1105) Term key answers 'page-up and 'page-down for ESC [ 5 ~ and ESC [ 6 ~.
Ansi highlight colours code with x/tool/syntax. (#1100) The example is shown byte for byte as written ('rdonly stays 'rdonly) in the same classes as the REPL painter, where it re-rendered the parsed form with a keyword list of its own.
x/tool/syntax: lexical grammars, their spans, and renderers for a terminal, a web page and a screen editor. (#1100) A grammar is a Lexer whose rules read every byte and a table from each token’s tag to a class in one vocabulary (comment string char number keyword constant class private name punct operator escape error). spans answers (offset length class) for a stretch of text, reading from a line the grammar’s sync test passes, so a string opened above the stretch colours its start; Syntax ansi, html and cells draw spans as SGR sequences, Rouge span markup, or a class byte a text byte for an editor that lays out its own rows. (Syntax x) is x-lang’s grammar, its atoms classed by what the reader reads them as; register!, named and for-file find a grammar. x/repl/paint and x/tool/highlight draw (Syntax x)’s spans in place of their own scanners, and Paint lexer reads through (Syntax over l colours): Paint classify answers the shared classes ('keyword, 'name and 'constant where it answered 'construct, 'symbol and 'bool), Highlight source, spans and transcript take no keyword list, and Highlight keywords is gone. A line paints as before, a bold colour now one SGR sequence (1;35) where it was two, and costs the same (~11 ms a 60-byte line); the documentation’s 395 highlighted blocks still strip back to their bytes. The REPL pays for the Lexer: loading the line editor 1.01 -> 1.39 s, and the first painted line of a session ~0.5 s (the asm-cache import and the grammar’s Lexer make).
x/tui: a styled-cell screen, Markdown pages, windows and tabs. (#1103) Screen holds rows of styled cells and renders only what changed since the last frame. Page lays out a Markdown document at a width with bold, italic, code and links styled; Window scrolls a page and selects its links; Tabs holds windows with a bar listing them. Width and Style give a code point’s columns and a style’s escape sequence.
let costs half what it did. (#1102) A plain let is built as the fn application it stands for, ((fn (_ a b) body) 1 2), as the named form already was, and tail-evaluated, instead of evaluating a fn into a closure and applying it to a list of values. One binding costs 58 objects a run instead of 120, three 96 instead of 192; named let 134 instead of 144. str->number on a signed decimal costs a third less. There is one let, in x/core/control.
Markdown: a CommonMark parser with HTML out. (#1101) (Markdown parse text) answers the document as blocks led by their CommonMark names (heading, paragraph, code-block, block-quote, list, thematic-break, html-block) with inlines as strings and emph, strong, code, link, image, html-inline, softbreak and hardbreak lists; (Markdown html blocks) renders them as the CommonMark reference does. 574 of 595 cases agree with pandoc’s CommonMark reader, the rest being pandoc’s own choices and two gaps (named entities beyond the common ones; Unicode case folding of reference labels). A 10 KB document parses in ~350 ms.
import, include-once and import-version load through the include binding again. (#1098) A lang that wraps include with set! sees every file the module system loads, as it did before v0.37.0; the wrapper is called with the path alone, and the file still loads into the module it belongs to.
(Doc names) builds its list once until a name is documented. (#1097) The sorted names are kept beside the registry list they came from and answered again while it is unchanged, so apropos and Tab completion no longer sort every documented name on each call. Helium, after the first call: a Tab completion on “S” 2.11M -> 315K objects; (apropos “split”) 2.75M -> 0.96M.
(Doc names) and (Doc entry) allocate less. (#1096) Doc names compares names on the engine’s instructions instead of through a class call per comparison, and Doc entry reads an entry’s parts in one walk. Over the 732 names a helium boot documents: (Doc names) 2.31M -> 1.39M objects; (Doc entry ‘Str8/split) 19.7K -> 17.1K, most of what remains being the registry lookup.
A libm closure held by a scoped module survives an image write. (#1095) libm-fn’s held rows are remade through their cells alone; the caller’s name, which may be a module’s, is no longer set in float’s frame. A scoped module binding a libm function (x-python’s core) wrote no state image before this.
The linter resolves a scoped module’s heads in the module’s environment. (#1093) A send on a module-private value, (%lexer read-str s) in a file headed (module NAME), is read as a send, so its selector is no longer reported undefined; before, the head was evaluated in the root, which does not see a module’s private names. The driver’s undefined-name filter reads the same environment.
Http request bodies may be runs. (#1091) Http open, Http request and the verbs on them take a body as a run, (STRING . COUNT), as well as a string, so a binary body with NUL bytes in it goes out whole, over http or https. Content-Length is the run’s count.
Source reads where a form was read. (#1090) (Source line form) and (Source file form) read the reader’s stamp on a form, (Source path id) and (Source files) the registry of included files, and (Source here) the reader’s live position as (path . line). A form built at run time answers (). The help entry for current-line, which no module bound, is gone.
libm-fn makes stubs for a double and an integer or pointer. (#1088) The labels "di->d" and "dp->d" call a libm function such as ldexp(x, n) or frexp(x, &e): a double’s bits and an INTEGER in, a double’s bits out.
File stat and File lstat answer the owner, the access time and the link count. (#1086) The alist adds uid, gid, atime and nlink to size, mode, file-type and mtime.
ctrl-c during a method dispatch no longer loses a class member. (#1085) Moving a member found deep in a class’s dispatch table to the front takes three writes, and a ctrl-c between two of them dropped the member from the table for the rest of the process, which then reported “no such static member” for a method that exists (x-logo’s tty test t02, Base raw-of). The move now runs with the error handler set aside, so the interrupt is raised once the move is whole. Objects allocated, helium: 500 calls that each move a member 95,822 -> 110,293; 500 hits in the first two cells and 100 full-table walks unchanged (56,031 and 41,120).
A state image of a library that loads a file into a module writes clean. (#1083) The loader include and include-once use inside a module claimed meta words on its read buffer that a fresh base – the image writer’s child – does not give, and wrote past them; writing the image crashed with a bus error, so a bundle with a module in several files (x-coreutils’s vi) could not image its harness. The buffer now takes what its base gives, as the engine’s include does. A harness importing a two-file module is imaged by every suite run.
An included file is evaluated where the include stands, so a module may be written in several files. (#1081) include and include-once evaluate a file’s forms in the including environment – the root from an unscoped file, the module from a scoped one – through a loader written in x-lang over the base’s input stack, with the engine’s own include serving the root. A module’s own file includes its other files, each headed with the module’s name, and they share its private names; a file headed with a loaded module’s name joins it, and a module’s file loaded again by path reloads it in place. The amalgamator splices an included file where the include stands, and no longer refuses a plain include inside a scoped module. docs/modules.md, “A module in several files”.
The linter reads a call through a local named after a construct as a call. (#1078) A parameter or let variable called op, first or any other construct name, applied in its scope, is a call of that local; before, the dispatcher parsed it as the construct and the engine died on the shape, with no verdict for the file. x-awk’s awk/eval.x lints now.
Pty runs a program on a pseudo-terminal. (#1077) (Pty open) gives a master and its slave’s path, (Pty window! fd cols rows) sets the window, (Pty attach! path cols rows) makes the slave a forked child’s controlling terminal, (Pty fork cols rows) does the three as forkpty(3) does, and (Pty spawn program args cols rows) runs a program on a new terminal, answering (PID . MASTER).
Rsa generate makes an RSA key, and ECDSA runs over P-521. (#1076) (Rsa generate bits) answers a key’s (N E D IQMP P Q). Ecdsa verify, verify-der, sign and public take p521.
The lang kit’s dup-defs refuses one name defined two ways in one file, a scoped module’s private names included. (#1075) A later top-level def silently replaces an earlier one; x-ssh’s ssh/session.x carried (def %ASKED 4) and (def %ASKED 7) and wrote a Vector past its end. The scanner also now ends a form where its paren closes, mid-line or not, so (def %A 1) (def %B 2) is read as two defs, and drops a stray ) at depth 0.
Rsa sign-pkcs1, Ecdsa sign and Ecdsa public make the signatures x-lang already checked. (#1074) (Rsa sign-pkcs1 n d hash digest) is RSASSA-PKCS1-v1_5, as SSH’s rsa-sha2-256 and rsa-sha2-512 and certificates are signed. (Ecdsa sign curve d hash) answers (R S) on P-256 or P-384 with RFC 6979’s deterministic nonce – the same signature OpenSSL’s nonce-type:1 makes – and (Ecdsa public curve d) the key’s point, 04 X Y. Both are variable time.
Assembling a large program resolves its labels in linear time. (#1072) asm-finalize! indexes 32 or more labels by symbol before resolving its patches, where it walked the whole label list for each one: 2,000 labels and branches resolve in 0.16 s at 652 objects a patch, where they passed 8 GB. A patch is read by its pairs, and the arm64 and x86_64 resolvers work on the integer doors.
The lang kit’s lint reads a bundle’s constructs and its required langs, and holds the module rule to the files the entry loads. (#1073) A lang’s constructs.x declares its binding forms to the linter, including a new letrec scope; a module’s import from a required lang resolves; and a file nothing loads is not held to the module rule.
Doc names and Module names drop duplicates after sorting. (#1071) The quadratic List distinct walk is gone from both: over the 701 documented names of a helium session, (Doc names) allocates 2.4 million objects where List distinct alone ran past 34 million; the lazy doc commit it triggers costs 1.1 million.
Doc reads the documentation registry as data. (#1071) (Doc entry 'name) answers a record with name, desc, returns, params, examples, sees, notes and samples, as the doc form wrote them, or () for a name nothing is documented under; (Doc names) lists every documented name, sorted.
Env reads an environment by name, and Module reads the module registry as data. (#1069) (Env bindings e), (Env names e), (Env parent e) and (Env has? e 'name) read any environment, (Env current) answers the one a form is evaluated in and (Env root) the running base’s root. (Module names), (Module loaded? 'm), (Module exports 'm), (Module owner 'sym) and (Module env 'm) answer what (modules) and (help x/some/module) print.
Fn label, Fn params, Fn body and Fn env read what a callable is made of. (#1068) (Fn label f) answers 'fn, 'op, 'wrap or 'prim, and () for anything else. (Fn params f) is the parameter list as written, (Fn body f) the body forms, and (Fn env f) the environment a closure or an operative was made in, one pair (bindings . parent) that eval accepts. A wrapped combiner and a primitive answer () from the three readers; unwrap answers for the one and the other has no state.
help answers a value that is not a name instead of crashing, leaves % members out of a class listing, and apropos returns the matching names. (#1067) (help 42) segfaulted; it now prints “No documentation for 42”, and (help 'name) finds what (help name) finds. A class listing omits members named with a leading %, which (help Class %member) still answers. (apropos "gcd") answers ('Num/gcd) after printing its listing, and () when nothing matches.
The engine is x-engine-c v0.2.24 (#1066). A guard’s error record holds no raw word, so a continuation captured inside a guard survives the collect after the raise; call/cc specs the case.
The lang kit’s strict lint holds a bundle to its module headers and bare provides. (#1065) tools/lang-kit/lint.sh --strict reports a file the lang loads whose first form is not (module ...), and a provide list naming a %-private, beside the linter’s structural findings; the entry and the bundle’s tests/, tools/ and docs/ are exempt from the module rule. The provide scan ships with the kit, and make check-provide-names runs the same one over lib/.
A bundle’s files are scoped modules and its name tables are classes. (#1063) The lang contract states the three rules for a bundle: every file but the entry is a (module ...), a name chosen at run time is a class rather than a Dict, an assoc list or a chain of string compares, and a prefix is not a namespace. Ruling 3 of docs/lang-scale.md is settled by pinning rather than by a pre-release run of every bundle.
(Cont capture f) is call/cc for a continuation that waits (#1058). A generator or coroutine captured through it keeps what its suspended frames hold across a collect; with bare call/cc those objects were swept and the resumed frame read freed memory (#283). The marker beside the continuation walks the saved root chain in the stack copy, through the engine’s closure mark handlers (x-engine-c#93).
The engine is x-engine-c v0.2.23 (#1061). A type made with type make may give its mark handler as a closure, and call/cc’s state list carries stack-lo, stack-size and stack-copy; both for x/sys/cont.
Vector ref and set! read an in-range index at once. (#1060) An INT within a vector’s length, either sign, is read or written after one check, where conversion, the receiver check and negative normalization each took a call: (Vector ref 5 v) 236 -> 198 objects, (Vector set! 5 x v) 243 -> 218, (v 5) 352 -> 334. Every other index takes the checked road it took before, with the same errors.
Term window falls back to COLUMNS and LINES. (#1057) Where TIOCGWINSZ reports no window – standard input a pipe or a file – the window is COLUMNS and LINES from the environment, as documented, and 80x24 only when they are unset or not positive numbers; before, it was always 80x24.
Aes and Bcrypt open what OpenSSH seals a key file with. (#1056) (Aes ctr key size iv s [start len]) is AES in counter mode for 16-, 24- and 32-byte keys, the 128-bit big-endian counter OpenSSH’s aes-ctr and OpenSSL use, and (Aes encrypt-block key size block) one block. (Bcrypt pbkdf password password-len salt salt-len rounds key-len) is OpenBSD’s bcrypt-pbkdf, what OpenSSH derives a key file’s cipher key and IV from; its Blowfish key schedule runs on a compiled engine built on first use and adopted only after it answers Go’s bcrypt_hash vector, about 0.1 s a round. Regions are read by their lengths, so salts and keys with NUL bytes are whole.
The engine is x-engine-c v0.2.22 (#1053). A typed list marks without a C frame a pair (x-engine-c#90): the collector follows a list’s rest instead of recursing into it, so a collect with some 35,000 pairs live no longer overflows the C stack. The library needs no change.
Tls13 accept is the server’s side of a TLS 1.3 session. (#1054) (Tls13 accept in out (list (pair 'pem text))) runs the handshake over descriptors already open – an accepted socket, or standard input and output under tcpsvd – with the key and certificates from PEM text: X25519, ChaCha20-Poly1305, and CertificateVerify signed with an RSA key (RSA-PSS, about 1.3 s for 2048 bits) or an Ed25519 key. A client that offers no X25519 share or no ChaCha20-Poly1305 is refused with a handshake_failure alert: no HelloRetryRequest is sent. send, recv-run, pending? and close work as on a client’s session.
Rsa signs, and X509 reads a private key. (#1052) (Rsa sign-pss n d hash digest) makes an RSASSA-PSS signature (RFC 8017) with a fresh salt as long as the hash, as a TLS 1.3 server signs its CertificateVerify: about 1.3 s for a 2048-bit key, variable time. (X509 private-key text) reads the first private key in PEM text: an RSA key as PKCS#8 or PKCS#1, answering (rsa N E D), or an Ed25519 key as PKCS#8, answering (ed25519 SEED).
Syslog sends a line to the system log as syslog(3) does, over new unix datagram sockets on Socket. (#1050) (Socket unix-dgram-bind path) and (Socket unix-dgram-connect path) are the unix pair udp-bind and udp-connect are for IPv4. (Syslog send ident facility severity text) builds the RFC 3164 line the C libraries send – <PRI>Mmm dd hh:mm:ss IDENT[PID]: TEXT – and sends it to /dev/log (/var/run/syslog on Darwin), answering #f where no daemon listens; (Syslog line ...) builds it alone.
A static field read walks no list. (#1049) A class’s table entry for a static field holds the field’s value cell, so a read no longer walks the class’s data and its statics for the field: (Lexer %rec-on) is 724 -> 285 objects over the send, (Lexer %i+) 422 -> 119, and x-ash’s held Lexer remake 330K -> 319K objects. A write to an own field sets the same cell; a write through a subclass still shadows into its own box.
The TLS codecs’ bit operations use the int doors. (#1048) HMAC, RSA’s padding checks, DER, the TLS key schedule, X509 and base64 no longer send their bit work through the numeric tower: a verified TLS connect allocates 2.4% less, and the pure-x base64 decoder much less.
x/boot/tower-compiled can be imported from any directory. (#1047) Its includes are relative to its own file, so a bundle that imports it for the compiled numeric analysers boots from source wherever an installed x runs, not only from the x root; tools/release/amalgamate.sh and the boot-order check resolve ./ and ../ includes against the file that holds them.
(Tls13 pending? session) says what the session has read ahead. (#1046) Bytes a smaller recv-run left, or a whole record already in its buffer: what poll() on the descriptor cannot see, so a loop that polls receives while it holds.
(Tls13 over in out) runs a TLS 1.3 session on descriptors already open. (#1045) The handshake and records read one descriptor and write the other – a socket’s, or a pair of pipes to a program, as busybox’s ssl_client -e has it – and close leaves them to the caller; connect is a TCP connect handed to the same code.
A TLS record costs half what it did. (#1044) ChaCha20-Poly1305 makes one ChaCha20 pass a record for the one-time key and the data together, and ChaCha20 and Poly1305 do their bit work through the int doors rather than the numeric tower: a sealed record allocates 20 K objects where it allocated 41 K, and the pure-x fallbacks are faster for the same reason.
A replayed Lexer state is poured with no free variable matched by name. (#1043) A pour takes the values of its entry’s fvar slots in slot order, which the Lexer’s plan now keeps; a caller with named fvars has them matched once, as before. x-ash’s held Lexer remake is 33.4 -> 30.2 ms and 361K -> 330K objects. asm-cache-held-fvar-names lists names in slot order, and asm-cache-entry-pour takes values in that order rather than as (name . value) pairs; the Lexer’s plan version is 5.
TLS works again from a state image. (#1042) The record cipher, HMAC and RSA no longer hold zero-filled buffers made as their modules load, which an image brought back empty; wget in an x-coreutils bundle image failed every handshake on main.
A cached state pours with fewer objects. (#1041) The asm cache counts its lists on the integer doors rather than with the boot layer’s %length, a fold over the generic +: a pour counts its slots every time, and x-ash’s held Lexer remake is 37 -> 33 ms and 412K -> 361K objects.
X25519 runs its whole ladder in one compiled call. (#1040) The 255 ladder steps and the inverse are two compiled functions built from the field engine’s own expressions, where each field operation was a call from x: an X25519 multiplication allocates 41 K objects where it allocated 157 K, two of them a TLS handshake.
X509 verify-chain accepts certificates already parsed. (#1039) Each may be DER or what X509 parse answered, so a caller that read the leaf for its key does not parse it twice; the TLS client now does exactly that, one certificate parse less a connect.
A TLS connect allocates 40% less. (#1038) HMAC, HKDF, the TLS key schedule, ChaCha20-Poly1305, RSA’s padding checks, Der oid and X509’s name handling build their bytes by memcpy and touch them on the prims: a verified connect to an RSA site is 2.75 M objects where it was 4.4 M, a certificate parses in 158 K, and an HMAC is 12 K.
A Lexer remade after an image load is a third cheaper. (#1037) Its replay pours each state from the cache entry it found once for the plan, which the image carries, instead of finding the entry and noting it by its key text every pour: x-ash’s held remake is 66 -> 42 ms and 672K -> 412K objects, and its image-booted process starts with 3.08M -> 2.81M objects allocated. The new doors asm-cache-held-entry and asm-cache-entry-pour find and pour an entry; a cache group whose thunk noted nothing and set no extra keeps its file as it was rather than writing it empty.
Deflate and Inflate take a stream a piece at a time, and the CRC-32 is compiled. (#1035) (Deflate stream) and (Deflate feed w s start n final) write a raw DEFLATE stream from pieces; (Inflate stream), feed, done? and rest read one back, a block at a time as the input completes it, so gzip and unzip need no libz and never hold a file whole. (Crc32 jit!) compiles the CRC-32’s byte loop (past 4KB on its own), and Deflate builds its engine past 16KB rather than 32KB.
A method send costs half what it did. (#1034) Class and instance dispatch skip the quote unwrap for a symbol selector, read a built hot record in place and sort the table entry with one eq?, one pair? and one nil test before calling the method: a static send is 14.7 -> 8.2 us and 180 -> 91 objects.
The Lexer’s number rule reads hexadecimal floating constants. (#1033) 0x1.8p3, 0x.8p-1 and 0X1P+4 – a hex fraction or a p exponent after 0x and hex digits – are one token labelled 2, as a decimal fraction is, any suffix after it taken as before; a hex integer stays labelled 3, and %plan-version is 4. C’s hexadecimal floating constants read whole for x-cc, and Python’s float.fromhex text for a reader that wants it.
(Tls13 ready!) builds every engine a connect uses. (#1032) One call, poured from the cache when they are there, answering #t when all seven are compiled: what connect does before dialling, now available to a build that warms a cache with no server in sight.
Processes that load floats start about 50 ms sooner. (#1027) After a state
image loaded, x/num/float assembled all thirty of its libm and arithmetic
stubs whether or not the process did any float arithmetic. Each is now
made at its first call; a stub handed out by libm-fn or exported is still made at the
load. Xenon starts in 0.32 s of CPU instead of 0.38 s, and a coreutils
applet in 0.30 s instead of 0.34 s.
Curve engines build in milliseconds and point formulas run in one call. (#1025) x/codec/mont-jit is one looping engine for every modulus now, with a compiled runner that executes a whole program of modular operations from a table in the scratch: mont-jit-program writes the steps once, mont-jit-run! is one call. Five functions serve every curve and RSA key, poured once a process, where each curve built and poured twelve of its own: a P-256 engine builds in 5 ms where it took 300, and an ECDSA verification is 40% faster.
List and Str8 methods shed their internal sends. (#1024) List’s methods call their shared walkers, from-seq and fold, as module functions rather than sends back into the class, test a list’s end with eq? and count on the integer doors, and Str8’s trims walk bytes on the doors: on ten elements List length is 3,783 -> 539 objects (325 -> 40 us), reverse 2,650 -> 539, sum 2,432 -> 516, take 1,451 -> 825; (Str8 trim s) on 25 bytes 9,197 -> 684 objects (857 -> 58 us) and (Str8 split " " s) 3,981 -> 2,121.
A bundle’s spec gate no longer reads the x-lang checkout its CI makes inside the bundle. (#1023) tools/lang-kit/dup-defs.sh leaves out directories whose names start with a dot, so .x-lang and .x-r5rs beside a bundle’s sources are not scanned as the bundle’s own files.
Host regions: a process’s memory one record a region, for pmap. (#1022)
(Host regions pid) answers the process’s mappings in address order, each
((start . ADDR) (size . B) (mode . "r-xp") (name . PATH) (pss . B)
(private-dirty . B) (swap . B)), name nil for an anonymous region. Linux
reads /proc/PID/smaps as BusyBox’s pmap does; Darwin walks proc_pidinfo’s
PROC_PIDREGIONPATHINFO and reports no pss. Host maps keeps its sums.
TLS connects in half the time. (#1019) Hmac digests answer bytes straight from the hash’s words, Der reads on the prims, X509 from-pem decodes PEM blocks by memmem and a streaming Base64 decode-region (new, the lossless door into a buffer), and the compiled engines no longer load the C compiler to replay cached code: a verified handshake is 1.6 s of CPU where it was 3.1, and every process that pours a cached engine starts a third of a second sooner.
A bundle’s spec gate fails when two of its files define one global name differently. (#1021) tools/lang-kit/dup-defs.sh, the scanner check-dup-defs holds the platform to, ships with the kit, and tools/lang-kit/spec-gate.sh runs it over the bundle’s own sources before the suite. Each applet ran in a process of its own, so x-coreutils passed every spec with %ts-verbose meaning a tar accessor in one file and tcpsvd’s counter in another; the gate names both files. Fetches of one catalogue entry, the same definition written twice and private names in scoped modules pass, as on the platform.
A socket send to a peer that has closed raises an 'io failure instead of ending the process. (#1020) Socket send and send-run pass MSG_NOSIGNAL, so the kernel answers EPIPE rather than SIGPIPE. A Tls close to a server that hung up first, as wget’s https reads met, no longer kills the program.
Starting a bundle costs fewer processes. (#1018) x.sh read each installed
bundle’s lang.xon name with sed | head, two processes a bundle on every
launch; it reads them with the shell’s own read now, and quotes a value
with sed only when it holds a single quote. With sixteen bundles
installed a coreutils applet starts in 0.50 s of CPU instead of 0.56 s.
Vectors index, build and fold several times faster. (#1017) Index and length arithmetic in x/type/vector runs on the integer doors – an index is INT once converted, a length is slot 0’s INT – so (Vector make 100 0) is 3.15 -> 0.40 ms and 39,397 -> 4,419 objects, a 100-element (Vector fold ...) 1.76 -> 0.41 ms, (Vector ref i v) 100.6 -> 70.5 us and (v i) 69.7 -> 39.6 us; make and build convert their length once, as ref converts its index.
Every process started from a state image starts about 0.15 s sooner. (#1016) The image loader found each struct a stored object names by walking the type list and, for every type, searching the base’s path table for the type-name row; it looks that row up once now. Helium starts in 0.21 s of CPU instead of 0.36 s, and a coreutils applet in 0.41 s instead of 0.56 s.
Method sends cost a third of what they did. (#1014) Class dispatch and every method body bind with def in their own activation rather than the let operative, which entered a fresh closure each time: a static send’s own cost falls from about 46 us and 561 objects to 15 us and 169, every method call sheds about 11 us and 118 objects, and (Vector ref i v) falls 100.6 -> 60.7 us.
Host holders reads maps devices from any dialect. (#1013) %maps-dev, which
prints a device as /proc/PID/maps does for (Host holders ...)’s Linux
walk, divides on the int doors; from a bundle whose / is the tower’s
division it raised “&: operands must be integers”.
<, >, <= and >= are three times faster. (#1012) Their nil and arity guards test with eq? against nil and if rather than null? and the or operative, so (< a b) is 10.2 -> 3.3 us and 122 -> 24 objects a call, > 20.3 -> 6.8 us (264 -> 68), <= 28.7 -> 8.3 us (365 -> 85) and >= 26.5 -> 7.7 us (337 -> 78), with the same answers and messages; (Vector ref i v) falls 100.6 -> 82.3 us with it.
RSA verification is the compiled power and little else. (#1009) x/codec/mont-jit’s looping engine reads big-endian bytes into a slot and writes a slot out as bytes in compiled loops (mont-jit-load-bytes!, mont-jit-store-bytes), reads the modulus’s limbs the same way, and sets up R^2 and its one in the scratch, so a 2048-bit verification is tens of milliseconds where it was a quarter second of limb conversion. x/codec/mont makes a modulus’s limbs when first asked.
(obj ref) and (obj set!) are 1.64x and 1.16x faster. (#1008) The slot accessors address data word i through its view – the object’s address plus i words, read by first and written at the first-word offset – so no (int ->ptr) or (ptr ->int) cast and no %data-word-off frame sits between: a read is 2,742 -> 1,669 ns and 21 -> 12 objects, a write 2,951 -> 2,538 ns and 24 -> 16 objects, with no new primitive.
make check-boot-order no longer runs past 12 GB on a cold run. (#1007) The
lint read all of lib/ and kept every file’s reader garbage until it
finished; it now sweeps the heap after each file it reads, and a run with
no cached verdict peaks at 3.6 GB of footprint.
A bundle reached through a symbolic link gets a new state image when
its source changes. (#1007) tools/dev/image-build.sh walked a key path with
find, which on BSD lists a link to a directory and nothing under it, so
a linked bundle’s edits never changed its image’s key and the old image
was loaded on every run. It walks with find -H now.
Sys wait waits on through a caught signal, and raises when it cannot wait. (#1006) A signal caught without SA_RESTART – the engine’s SIGINT, any catch-signal – interrupted waitpid, and Sys wait answered a status read from a buffer waitpid never filled (160) and left the child a zombie; it now waits again on EINTR. Any other failure raises the io Err from errno ('echild for a pid that is no child of this process) instead of answering a made-up status.
Host files reads a process’s mapped files in time linear in its regions. (#1002) A process mapping thousands of files, Firefox among them, passed 5 GB and did not return; it now takes under a second.
Host holders: the processes that hold a path, or anything on its file
system. (#1005) (Host holders PATH [mount?]) answers the pids, ascending, that
hold a path as their working directory, root or executable, through a
descriptor or a mapping. Darwin asks the kernel with proc_listpidspath
(0.1-0.3 s where walking every process’s files took about 70 s on a Mac with
773 processes); Linux compares each process’s files records’ device and
inode. Busybox’s fuser reads it for each FILE operand.
File set-times stamps a path’s access and modification times with given seconds. (#1004) (File set-times path atime mtime) packs a pair of timevals, a word of seconds then a zero word each, and calls utimes; the zero fraction makes the same bytes a pair of timespecs where utimensat stands in for utimes, which the Linux stand-in table now says. File utimes keeps stamping the clock. Coreutils’ touch -d and cp -p, and x-ssh’s scp -p sink, have a door for a given time.
Crypt md5, Crypt sha256, Crypt sha512: crypt(3)’s $1$, $5$
and $6$ hashes. (#1000) (Crypt md5 KEY SALT) and (Crypt sha256|sha512 KEY
SALT) give the hashes busybox’s cryptpw -m md5|sha256|sha512 makes,
rounds=N$ included; every spec vector is busybox’s.
RSA signatures verify, so TLS reaches servers with RSA chains. (#1003) (Rsa verify-pkcs1 n e hash digest sig) and (Rsa verify-pss ...) (x/codec/rsa) check RSASSA-PKCS1-v1_5 and RSASSA-PSS signatures (RFC 8017) under 2048- and 4096-bit keys; X509 verify-chain follows sha256/384/512WithRSAEncryption signatures and Tls13 connect checks an rsa_pss_rsae CertificateVerify, so www.debian.org and every other Let’s Encrypt RSA chain authenticates against Mozilla’s roots. Beneath them x/codec/mont-jit gains a looping Montgomery engine generic over the modulus, one cache group for every RSA key, and mont-jit-pow! for any exponent.
Md5 jit!: MD5’s compiled engine. (#998) (Md5 jit!) builds and adopts a
compiled MD5 digest (ARM64 and x86-64), verified against the pure-x one;
an input of 8KB or more builds it on its own. About 60 times the pure-x
speed on a short input.
Host files reports what a process has open. (#1001) (Host files pid) answers a record each for its current and root directories, its executable, its file descriptors and its mapped files: use, fd, dev, ino, type, name and id, and for an internet socket its protocol, family and local port. Name uses to read only those: (Host files pid 'fd). Linux reads the /proc/PID links and maps and /proc/net; Darwin proc_pidinfo and proc_pidfdinfo.
Compiled engines replay from the cache with nothing printed or
proven. (#999) x/tool/jit-group builds an engine’s functions as one cache
group and records a plan a later process pours them by, so the six
engines a TLS connection needs no longer regenerate, print, re-prove or
collect after their expressions in every process, and a Montgomery
modulus no longer computes R^2 in pure x: a process’s first https
connection drops from 17 s to about 8 s on arm64.
Md5, and bytes-n on Md5, Sha256 and Sha512. (#997) (Md5 hex s)
is the MD5 digest (RFC 1321) in pure x-lang; (Md5 hex-n s n) digests a
byte region. (Sha256 bytes-n s n), (Sha512 bytes-n s n) and
(Md5 bytes-n s n) give a digest as a buffer of its raw bytes.
FAIL: lines with sed under the caller’s locale; on macOS under UTF-8 a
failing case that printed a raw 0xff made sed exit with “illegal byte
sequence”, and the gate exited 1 without printing the run or naming the
failure. Its text processing now runs under LC_ALL=C; the runner itself
keeps the caller’s locale. make check-lang-kit-spec-gate, in gates and
gates-fast, holds it on a fixture suite that prints 0xff beside a failure.Crc32: the CRC-32 of ISO 3309 in pure x. (#994) (Crc32 of s) and (Crc32 update crc s), the sum gzip, zip and git’s pack indexes carry, checked against libz’s; for a system that has no libz.
Tls is TLS 1.3 written in x, the server checked against Mozilla’s
roots. (#993) (Tls connect quad port (list (pair 'host NAME))) now runs
Tls13, which checks the server’s CertificateVerify and its certificate
chain for NAME against Mozilla’s roots (lib/x/net/mozilla-roots.pem,
regenerated by tools/roots/update.sh); ('insecure) skips the check.
No C library is opened, so https works where there is none. Servers
must offer TLS 1.3 with ChaCha20-Poly1305 and sign with ECDSA; RSA is
not written yet. (Bytes find hay needle) searches binary data.
A Linux process record costs half the objects it did. (#992) Host processes,
process and threads parse each field through the integer primitives,
and the real uid and gid are found in /proc/PID/status in place: a record
with a full status file fell from 643K objects to 324K.
X509 and Der: certificate chains checked to a trusted root. (#991)
(X509 verify-chain certs host now roots) checks a TLS server’s chain –
the host against the leaf’s names, dates, CA constraints and ECDSA
signatures up to a trusted root – and raises naming the first thing that
fails; parse and from-pem read certificates. x/codec/der reads DER
over regions.
An image written after a compile is no longer refused. (#990) The last compile’s code buffer and relocation records, native code and its facts, are dropped before an image is written. A process that compiled before its image was written – a Lexer made from an empty compile cache – left two unnameable primitives, and the image was refused.
Ecdsa: P-256 and P-384 signature verification. (#989) (Ecdsa verify
curve public hash r s) and verify-der check SEC 1 ECDSA signatures,
0.25 s for P-256 and 0.40 s for P-384 once the curve’s engines are built.
Beneath it, x/codec/mont is Montgomery arithmetic modulo any odd
number in pure x, and x/codec/mont-jit compiles it per modulus,
checked against the pure arithmetic before it is adopted.
A replayed Lexer make pours each state with only the free variables its code uses. (#988) The plan keeps those, named as the text the pour matches, and the cache group key is printed once per rule list. x-ash’s tokenizer lexer, 73 states, remade after an image load: 1.11M objects and ~100 ms, to 0.74M and 65-69 ms.
Deflate’s engine compiles its matcher, counter and Adler-32 as loops. (#987) With compiled tail self-calls, the matcher runs over the whole input in one call, and a block’s counting and the zlib wrapper’s Adler-32 are compiled loops too; the engine (built on (Deflate jit!) or for 32KB of input or more) is adopted only after compressing four check inputs to the pure-x encoder’s bytes. 10 KB of near-random input compresses in 0.14 s where pure x takes 1.24 s; 100 KB of text in 0.13 s where it takes 0.98 s.
A Lexer with a nested or word rule no longer keeps a boot image from being written. (#984) Its image transient drops the reset states with the rest of its compiled states, and the load remakes them. A process that made such a lexer before its image was written booted from source every time.
Host reads the kernel’s system-wide totals: counters, disks, interfaces and interrupts. (#982) Context switches, paging, faults and open files; per-disk I/O; per-interface traffic; per-source interrupt counts on each processor. Linux reads /proc, Darwin sysctl, the Mach VM statistics and IOKit; a field the kernel does not report is nil. cpu and cpus also report guest and guest-nice time.
make check-pin is green again. (#981) The smoke’s bundle declares
(requires-lang ...) since #977, and its load case now supplies the lang it
names instead of failing on the wrapper’s missing-requirement refusal.
A compiled function loops: a self-call in tail position is a jump. (#980) compile-asm gives an integer function stack slots for its parameters, read once on entry, and compiles a self-call in tail position – the body, a do’s last form, an if or match arm – as a store over the slots and a branch to the top: constant stack and no allocation per turn (a million turns in 4 ms). A self-call whose value is used is still a call. A parameter is now evaluated once, on entry, where it was evaluated at every read.
Tls13: a TLS 1.3 client written in x. (#979) (Tls13 connect quad port
opts), send, recv-run, recv-bytes and close, as Tls has them,
over X25519, ChaCha20-Poly1305 and SHA-256, with x/net/tls-keys the key
schedule checked against RFC 8448. The server’s certificate is not yet
checked, so ('insecure) is required. Bytes copies through memcpy, a
few objects a copy whatever its length.
Deflate writes its tokens through a compiled engine, and reckons a block’s costs from its frequencies. (#978) The token writer is compiled (JIT, ARM64 and x86-64) and adopted on (Deflate jit!) or for 32KB of input or more, only after writing byte for byte what the pure-x writer writes: 15 times fewer objects and 7 times less CPU writing a block. A block’s costs come from its frequency tables rather than two more passes over its tokens, so the pure path is cheaper too.
x --install-lang installs a bundle that requires another lang. (#977) The
pin reader refused (requires-lang ...) in a bundle’s lang.xon, so
x-r7rs and x-sed could not be installed through it. make check-langs
also keeps each bundle’s suite output when LANGS_LOG_DIR is set.
Opts parse lists a cluster’s flags in the order given. (#973) -bt was
listed as -t then -b, so a caller where the later of two flags wins –
busybox md5sum’s -b and -t, cp’s -H/-L/-P, chmod’s -c/-v – read the
wrong one; Opts on? asks only membership and is unchanged.
Bytes and ChaCha20Poly1305. (#974) x/codec/bytes carries binary data
as regions (BUF START LEN), with a growable writer, a bounds-checked
reader, constant-time comparison and (Bytes random n) from
/dev/urandom, since a string’s own length stops at its first NUL.
(ChaCha20Poly1305 seal key nonce aad pt) and open are RFC 8439’s
AEAD, the record protection of a TLS 1.3 client written in x.
A replayed Lexer make patches its states in one compiled call. (#975) A cached entry keeps its relocation sites as a table of distinct addresses and site offsets, patched by a compiled patcher where the pour walked each site in x, and a group file carries the table as bytes. (Lexer skip) names its type SKIP, so one rule list is one group key and a make met again in a heap reads no file and no plan. x-ash’s tokenizer, 73 rules: a fresh process’s make 0.44 s to 0.26 s, a make whose group is held 0.40 s to 0.12 s.
The image writer’s child sweeps each file’s load garbage, so x-coreutils’s
boot image writes at last. (#976) The loader’s include collects after each file
while %image-writing is true, in the writer’s child alone; outside the
writer the loader still never collects. x-coreutils’s write went from past
12GB (killed, no image) to 1.25GB, a 20.8MB image applets boot from in 0.57s;
x-logo’s from 1.60GB to 1.13GB.
Hmac, Hkdf and Sha384. (#972) (Hmac mac 'sha256 key msg) answers
the RFC 2104 tag over SHA-256, SHA-384 or SHA-512, (Hkdf extract ...)
and (Hkdf expand ...) are RFC 5869, and (Sha384 hex s) is FIPS
180-4’s SHA-384 on SHA-512’s digest and compiled engine. Binary inputs
are regions (STRING START LEN); answers are buffers of exactly the
digest’s bytes. They are the key schedule of a TLS 1.3 client written in
x.
Deflate raw and Deflate zlib compress. (#971) LZ77 over a 32K window through a hash chain and Huffman codes a block – its own, the fixed ones or stored, whichever is fewest bits – written in pure x and read back by libz byte for byte. What git writes a loose object as, and what gzip wants.
A replayed Lexer make reads its plan in one walk. (#970) The plan a cache group keeps is words, each list led by its length, read back by one walk over its bytes where the s-expression reader took 42 us a byte. x-ash’s tokenizer, 73 states: a make booted from x-ash’s image, 0.60 s to 0.40 s.
Crypt des: crypt(3)’s traditional DES password hash. (#969) (Crypt des
KEY SALT) gives the thirteen-character hash, salt first, that
/etc/passwd and busybox’s cryptpw use. Pure x-lang, 166ms a hash;
every spec vector is busybox’s.
Sha3: SHA-3, with a compiled engine. (#968) (Sha3 hex WIDTH s) and
(Sha3 hex-n WIDTH s n) give the FIPS 202 digest at 224, 256, 384 or 512
bits, or another multiple of 8 below 800. A compiled Keccak engine is
adopted after it agrees with the pure-x digest; (Sha3 jit!) builds it,
and so does an input of 4KB or more: 64KB in 0.3s, against 47s pure-x.
The X25519 and Ed25519 engines build in about 2 s together and take a
signature’s field work (#967): x/codec/fe25519-jit builds the field’s
operations once a process, proven against the pure-x field directly
rather than through a whole pure-x exchange, and adds the inverse and
the square root’s power as compiled chains; each curve’s engine checks
the rest against its RFC’s bytes. (Ed25519 jit!) now also covers a
point’s encoding and decoding, and Ed25519’s scalars are reduced as
ref10’s sc_reduce does. Measured, arm64: X25519’s build 7.3 s to 2.0 s,
Ed25519’s 6.7 s to 0.25 s; a signature 2.4 s to 0.73 s, a check 1.6 s
to 0.47 s; a reduction mod L 314 ms to 18 ms.
Inflate decodes 3-11x cheaper in pure x. (#966) The hot paths – the bit reader, the decoder, the table builder, the codes loop, the copy, the dynamic header’s code lengths and the Adler-32 – are primitives alone (def in a body, if, do, self-calls of top-level functions, the int doors), and a code is decoded through a lookup table over its next bits, sized to its longest length up to nine, falling back to puff.c’s bit-at-a-time walk for a longer one. 200 bytes of text cost 2.04M objects and 215 ms, now 603K and 82 ms; 2000 bytes of dynamic codes 4.6M and 448 ms, now 807K and 113 ms; the Adler-32 of 300 stored bytes 370K, now 34K. The compiled engine agrees with the rewritten reference on every check stream.
A - before a character class’s closing ] is a literal (#964). [+-]
was read as a range from + to ], so the class swallowed the rest of the
pattern up to the next ]: [+-]?([0-9]+) never matched 12. Found running
the AWK book’s number regexes through x-awk.
A /proc stat line shorter than a kernel’s reads; the fields past its end are nil. (#963)
The Linux reader’s processor (stat field 39) indexed past the end of a
line that stops sooner, and the read raised; (Host process pid) on such a
line now answers the record with processor nil.
Host maps and Host threads cost a process a small fraction of what they did, and Host memory reports anon, mapped, slab, dirty and writeback. (#961) A process’s maps fell from 5.1M objects to 174K on Darwin (its threads from 217K to 63K), each region and thread read through the resolved proc_pidinfo and the int doors rather than a class call a field; Linux’s smaps figures are scanned as bytes rather than trimmed and measured as strings. The new memory fields are /proc/meminfo’s AnonPages, Mapped, Slab, Dirty and Writeback, which BusyBox’s top -m prints; Darwin reports anon (its anonymous pages) and nil for the rest.
(Socket send-run fd s n): the first n bytes of s sent, NULs
included (#960) – the lossless door beside recv-run’s, since send
measures its string to the first NUL and a binary packet has them
inside. What x-ssh sends its packets through.
Ed25519: signatures (RFC 8032) in pure x, with a
differentially-verified compiled engine, and the field GF(2^255 - 19)
as a module of its own (#959). (Ed25519 public seed), (Ed25519 sign seed
m [start len]) and (Ed25519 verify pub sig m [start len]), over
x/codec/sha512 and the new x/codec/fe25519 – the ten-limb field X25519
now imports too, with x/codec/fe25519-jit its compiled operations for
both engines. (Ed25519 jit!) builds the scalar multiplication –
a signature in 2.4s and a check in 2.3s, the hashes, the inverse and the scalars mod L still pure-x, the build 7.9s of which six are the one reference multiplication its check runs – against 14s a signature and 27s a check pure-x. The
host and user keys of x-ssh.
Sha512: the SHA-512 digest (FIPS 180-4) in pure x, with a
differentially-verified compiled engine (#958). (Sha512 hex s) and
(Sha512 hex-n s n) as Sha1’s; pure-x on 32-bit halves so the
arithmetic fits the engine’s int, and (Sha512 jit!) builds the engine
on the lane’s 64-bit words – 64KB in 83ms against 520us a byte pure-x, a 1.4s build on a warm asm cache – which hex also builds
for an input of 4KB or more. What Ed25519 hashes with, for x-ssh.
X25519: Diffie-Hellman on Curve25519 (RFC 7748) in pure x, with a
differentially-verified compiled engine (#957). (X25519 scalarmult k u)
and (X25519 base k) answer 32 bytes, on ref10’s ten-limb field so the
arithmetic fits the engine’s int; (X25519 jit!) builds the engine –
the multiply, add, subtract, times-121666 and swap each a compiled
function, the ladder in x over them – an exchange in 135ms against 5.0s pure-x, the build 7s of which five are the one reference exchange its check runs. The key
exchange of curve25519-sha256, for x-ssh.
Term key names the function and keypad keys, and Line read-until ends a line on a key or stops it from an idle function. (#956) Term key answers 'f1 to 'f12 for the function keys in their SS3, modified-CSI and ESC [ N ~ spellings, and 'kp0 to 'kp9 for a keypad in application mode; ESC [ N ~ is now read by its number, where a byte test had read F5 to F8 as Home. (Line read-until prompt ends . idle) reads a line as read does, answers (key . text) when a key in ends ends it, and with idle as (tenths . fn) calls fn in the cooked terminal each time that long passes with no key: true stops the read as ('stopped . text), a string becomes the prompt.
Poly1305: the one-time authenticator (RFC 8439 2.5) in pure x, with
a differentially-verified compiled engine (#955). (Poly1305 mac key s
[start len]) is the 16-byte tag of a byte region under a one-time key,
on 26-bit limbs so the arithmetic fits the engine’s int; (Poly1305
jit!) builds the engine – the block loop as one compiled function, a
block a call – which mac also builds for a region of 48KB or more:
64KB in 12ms against 24us a byte pure-x, a 1.1s build on a warm asm
cache. The MAC of chacha20-poly1305@openssh.com, for x-ssh.
make check-changelog-commits reads a branch that merged main as one
change. (#954) A fix commit after a merge of main is judged with the entry the
branch already carries, not as a change of its own.
ChaCha20: the stream cipher (RFC 8439) in pure x, with a
differentially-verified compiled engine (#953). (ChaCha20 xor key iv s
[start len]) encrypts or decrypts a byte region and (ChaCha20 block
key iv) is a keystream block; the iv is state words 12..15, so RFC
8439’s and openssh’s nonce shapes are one cipher. (ChaCha20 jit!)
builds the engine – one compiled function a block, a double round a
call, whole blocks XORed as eight 64-bit words – which xor also
builds for a region of 8KB or more: 64KB in 49ms against 280us a byte
pure-x, a 2.3s build on a warm asm cache. The first codec for x-ssh.
A Lexer make replays the plan its cache group holds. (#951) A make with every
state compiled records, in the group beside the code, which entry each
state is and what its free variables are; a later make of the same rules
pours the states from that plan with no form made, printed or checked, in
the same heap or in a process that reads the group file. vi’s 21-state
address lexer: a warm (Lexer make) 104-110 -> 58-88 ms in a fresh
process, a remake in the same heap 86-91 -> 40 ms; x-ash’s 73-state
tokenizer and x-awk’s record lexers replay the same way. The replayed
field says which make ran.
The environment model is the tenth engine law (#946).
docs/environment-model.md opened with “Nothing in it is implemented”; the
model, an environment as one pair (bindings . parent), shipped in
x-engine-c v0.2.10 and the conformance suite tests it. Its rules are
behaviour the language requires of an engine that the manifest cannot
express, so they are law 10 in docs/engine-laws.md, and the file with its
history, costs, sequence and open questions is gone. docs/lang-scale.md’s
status line says which of its rulings are built, three of five, and that the
two that are not are still asked for; the one built ruling the lang contract
did not record, the release-refs gate, has its paragraph there.
Host reports what top needs: tasks, cpus, threads, maps, utmp and a processor field. (#952) (Host tasks) answers /proc/loadavg’s running and total counts and last pid (nil on Darwin, which keeps none); (Host cpus) a cpu record for each processor, from /proc/stat’s cpuN lines or host_processor_info; (Host threads pid) a record a thread, from /proc/PID/task or proc_pidinfo’s thread info; (Host maps pid) the mapping sums BusyBox’s top -m prints, from /proc/PID/smaps (read a piece at a time) or a proc_pidinfo region walk; (Host utmp) every utmpx entry with a user name and its type, of which (Host users) is now the user-process ones; and each process record gains processor, the CPU it last ran on (nil on Darwin). Darwin’s process CPU times are whole nanoseconds again under -l xe, where a 125/3 timebase made them fractions.
A lookup by name is a class, and docs/code-quality.md carries no
measurements (#948). Rule 1.2 said a chain of fifteen or more string
comparisons becomes a Dict of functions built at load; that is a dispatch
table written by hand. A chain that selects a function by name is a
dispatch, and the class system is the dispatcher: the names become static
methods of a class and the call is the lookup, at three names or thirty. The
linter’s warning for a string-keyed chain is dispatch, with no size
threshold, and the kit’s strict mode fails on it as it did on ladder-dict.
The document keeps its rules and loses the timing tables, the lists of worst
cases and the dated baseline behind them.
make check-changelog-commits refuses a branch that adds to CHANGELOG.md (#947).
The gate already required every feat, fix or perf commit since the tag
to carry its entry after a Changelog: line; now it also fails a branch whose
net difference from the merge base with origin/main adds lines to
CHANGELOG.md, since the file is written at release and a pull request that
adds its paragraph puts every other open one in conflict. A release branch,
carrying a release: commit, is the one that may add.
(Lexer state form fvars): an analyser state for a type built by hand (#947).
The Lexer realizes one state from a form in the assembler lane’s dialect, as
its rules do – compiled to native code when the lane is open, the interpreted
twin otherwise – and answers (STATE . COMPILED?). For a tokenizer base the
rules cannot express: typed instances, a live stream, a read that recurses
(x-logo’s). The caller holds its states and makes them again after an image
load.
docs/state-images.md is the design’s reasoning, and the developer
documents carry no measurements (#950). The state-images document said
the image format, the writer and the reader were still design; all three
shipped, and the document now says why each is shaped as it is, with the
measurements, probe scripts and accounts of abandoned approaches gone. The
format document owns the layout. docs/lang-scale.md, docs/modules.md,
docs/crafting-a-lang.md and docs/lang-contract.md lose their timings, counts and bug accounts the same
way and keep their rules; lang-contract.md’s compiled-harness sentence
names the registration rule (lib/x/tool/compiled.x) instead of an engine
defect that is fixed.
Deflate: DEFLATE streams written in pure x, stored blocks first (#949).
(Deflate stored s [start len]) writes a raw stream (RFC 1951) of stored
blocks and (Deflate zlib-stored s [start len]) the same in zlib’s wrapper
(RFC 1950), each answering (OUT N); the spec reads them back through
Inflate and through the system libz. What git writes a loose object as
when nothing compresses it; a compressing writer is next.
A held asm-cache group reads no file (#939). A group met again while its
entries are held – the same rules remade, or a process booted from an image
the group was made in – is served without reading its file, and its path is
hashed only when the file is read or written. The loops that run once per
relocation record use the integer doors. vi’s 21-state address lexer: a warm
(Lexer make) 123-128 -> 104-110 ms, a remake in the same heap 86-91 ms.
The library reference is generated, and the prose says where it is
(#940). docs/standard-library.md wrote 144 entries by hand, 129 of
them a second copy of the (doc …) form the API reference is generated
from, under a version line that said 0.12.0. It is a page of 79 lines now:
how the library is laid out, the shape of a call, how to find a name, and
how the reference is written. The booted forms in docs/primitives.md, the
third of its three parts, were the same kind of copy and are a paragraph
that says where they live; the C instruction set and the coordinates stay,
held to the engine’s manifest as before.
(Paint lexer l colours): a painter made from a Lexer (#935). It
answers a %repl-paint painter that colours a line by the tokens of a Lexer
whose rules read every byte, each token in the style its tag names: an Ansi
style name, an escape string, or a function of the token’s text. A lang whose
syntax a Lexer reads colours its lines with no span scan of its own.
(Sys args), and the launcher writes -- between its options and the program’s (#934).
x.sh starts the engine as x-bin OPTION... -- ARG.... (Sys args) answers
the arguments as the engine was given them, and (Sys args 'program) the
engine’s path and what follows the first --, so a program no longer drops
words it takes for the launcher’s, nor loses its own: --no-color given to a
program reaches the program and leaves the REPL’s colour alone. --batch,
--quiet and --no-color are read from the launcher’s side of the --
alone. With no -- nothing is left out of either side.
A paste into the line editor no longer slows with each byte, or dies (#938). The redraw measured the line a byte at a time through class calls, three walks a keystroke; it now walks it over the cached prims and the integer doors, and the loop sweeps within a line once the heap has grown a set amount, where it swept only between lines. An 85-byte paste under x-ash’s painter: 15.6 s to 2.6 s; a 340-byte one that hit the allocation ceiling finishes.
Module scope’s rules live with the module system (#933).
docs/namespaces.md, the proposal for module scope with the record of
building it appended, is gone. What a developer needs from it is in
docs/modules.md under “Module scope”: how a module’s names are reached
from outside, the rule for each name conflict, the files that are not
scoped with the reason and the measured cost for each, and the gates. The
counts from before the design, the sequence, the alternatives and the
measurement narratives went with the file. Every reference points at
docs/modules.md.
Sha1: SHA-1 in pure x, with a compiled engine sharing SHA-256’s driver (#936).
(Sha1 hex s), (Sha1 hex-n s n) and (Sha1 jit!), as Sha256 has them, for
naming content as git does; SHA-1 is not for trust. x/codec/sha256-jit is
x/codec/sha-jit now: one fill, driver and differential check serve both
digests. The engine builds on its own for an input of 8KB or more, SHA-1’s
measured breakeven: pure-x 3.8KB/s, the build 2.2s, the engine 470KB/s.
Inflate: DEFLATE and zlib streams in pure x (#937).
(Inflate raw s [start len]) and (Inflate zlib s [start len]) answer (OUT N
USED), USED the input the stream took, so a reader of streams laid end to end,
as a git pack lays them, goes on from there; (Inflate adler32 s n) is the
checksum alone. The decoder is puff.c’s: stored, fixed and dynamic blocks,
codes read a bit at a time, ~7.6KB/s of output on arm64. It is the reference
a compiled engine will be checked against.
Inflate has a compiled engine: the codes loop and the Adler-32 (#943).
x/codec/inflate-jit compiles a step that decodes one symbol or copies
eight bytes of a back-reference, and one that sums sixteen bytes of the
checksum; the driver in x grows the output between calls. Built for an
input of 4KB or more or on (Inflate jit!), adopted only after decoding and
summing two embedded streams exactly as the pure-x decoder does. The zlib
path: 16KB of output in 97ms through the engine, 2.22s in pure x; 64KB in
135ms.
The engine is x-engine-c v0.2.21 (#930). A NUL is an ordinary byte in
the tokenizer: a zero byte in a buffer the engine reads is a byte like any
other, not the end of the text, and tok read-str takes a span.
nested and word take to-end (#927). (Lexer nested TAG OPEN
START CONTEXTS 'to-end) and (Lexer word TAG START CONTEXTS STOP 'to-end)
make a span still open at the end of the text a token, the end text cut off
it, as until’s to-end does: a shell’s ${X or 'a'"b left open at the
end is the word to the end. Rules without the flag read as before.
(Lexer record TAG SHAPES) and (l read-span s start len): self-sized binary records (#930).
A record’s first byte picks a shape, and the shape’s steps read its length from its
own bytes: (bytes N), (fields K SIZES) (2-bit fields, stop ending them),
(flag MASK N) and (units W BIT); its token is (tag bytes length), the length
taken from the buffer, since the bytes may hold a NUL. read-span reads a span of
bytes, NULs included, with nothing appended; it needs an engine whose tok
read-str takes a span (x-engine-c#88) and raises a label 'lexer Err on one that
does not.
(Lexer pattern tag steps): a rule of steps, and the end text is never part of a token (#929).
A pattern is steps in order, each (CLASS FEWEST MOST) – a class, the fewest
bytes of it and the most, nil for no bound, #t for every byte – and reads as
many of each as there are: a printf directive is %, any flags, width and
precision, then one conversion byte; an octal escape is \ and one to three
digits. Each byte costs one compiled state, the next step’s opening decision
written into the one before it. And every token the engine took at the end
of the text is cut back by the end text read-str appended, not only a span
that ran to the end: a trailing \ reads as the escape \, a trailing % as
the directive %, what the text held.
(Socket tcp-listen-on host port): a TCP listener bound to one local address (#928).
What tcp-listen does, with bind(host, port) in place of INADDR_ANY, so only
connections to that address reach it – busybox’s httpd -p IP:PORT. An address
no interface holds is a label 'io Err from bind (eaddrnotavail); a host that
is not a dotted quad raises 'value before any syscall. tcp-listen is
tcp-listen-on with every interface.
(Host exe pid) and (Sys signals): a process’s executable, and this kernel’s signal table (#926).
Host exe answers the executable’s absolute path, or nil: Linux reads
/proc/PID/exe, Darwin asks proc_pidpath, which answers for any process.
Sys signals answers signals 1 to 31 as (NAME . NUMBER) in number order,
named as busybox’s kill prints them; Darwin numbers BUS, SYS, USR1, USR2
and the job-control signals apart from Linux.
run takes an optional follow class (#925).
(Lexer run TAG FIRST REST FOLLOW) is a token only when the byte after the
run is in FOLLOW, that byte left for the next token: the digits of a shell
descriptor such as 13>, which alone would be a word. Rules without it read
as before.
libm-fn’s "ii->d" stubs a C function of two integers or pointers answering a double (#924).
strtod(s, &end) is the case: the arguments travel in the integer
registers and the answer comes back as the double’s bits, as "s0->d"’s
does.
Zlib streams deflate and inflate buffers in place (#921). (Zlib
deflater LEVEL FORMAT) and (Zlib inflater FORMAT) make a stream for zlib’s
wrapper, gzip’s, or raw deflate; (Zlib step S IN OFF N OUT AT ROOM FINISH)
runs one call over a stretch of each buffer and answers how much it used and
made and whether the stream has ended; (Zlib end S) frees it, and (Zlib
crc32 CRC BUF N) is gzip’s CRC. A file goes through a block at a time, never
read whole.
The engine is x-engine-c v0.2.20 (#922). ptr call passes up to
eight arguments (x-engine-c#86), so a C function that takes eight, such as
zlib’s deflateInit2_, can be called through it; more than eight are still
ignored. The library needs no change.
until takes its close, or runs to the end of the text (#920).
(Lexer until TAG OPEN CLOSE 'take 'to-end): take makes a one-byte close
part of the token, so /ab/ is one token; to-end makes a span no close
ended a token running to the end of the text, the end text cut off it, so
ex’s /pat with no closing / is read whole. Both are optional and refused
when misspelt; rules without them read as before.
A short Lexer read costs a fifth of what it did (#919). remake! builds
the read once, as a closure over the tokenizing door, the raw base, the end
text and the dropped marker, and read-str calls it: ten bytes read in
182 us instead of 1,049 us, of which the engine’s tokenizing was 61 us. Only
a rule list with a dropped span filters, and a nested or word rule’s depth
reset runs inside the same closure.
Csv parse reads Lexer tokens instead of walking bytes (#918). A lexer
made on the first parse splits the text into separators, newlines (CR and LF
apart), quoted fields, plain runs and stray quotes; the parser joins doubled
quotes and refuses the same malformed quoting it always did. On 16 KB of rows
a parse falls from 359 ms to 137 ms under helium and from 560 ms to 142 ms
under xenon; the first parse in a process makes the lexer, about 80 ms with
the byte cache warm.
The Lexer reads a word whose bytes hold spans (#917). (Lexer word TAG
START CONTEXTS STOP) is a nested span with no opening literal: its first byte
is read in its start context, and it ends before a byte of STOP met at depth
0, which is left for the next token. Spans opened inside it read as nested
reads them, so a shell word with its escapes, quotes and command
substitutions is one token. Every nested and word rule puts its return stack
back at depth 0 before each read and when an opener goes past its depth.
A compile that hits the byte cache loads the code buffers, not the
assembler (#915). lib/x/tool/asm-code.x holds what a function’s bytes
need once they exist – the mapping and protection, the relocation records,
the relocators, finalize and free – and the cache imports it alone. asm.x
and the backend’s opcode table now load with the compiler, on a miss, or
with anything that emits. Under helium, importing the cache door falls from
290 ms to 210 ms; a hit still never loads the compiler. Code that emits
instructions imports x/tool/asm itself: the cache no longer brings it.
The Lexer reads nested spans and escapes (#914). (Lexer nested TAG OPEN
START CONTEXTS) reads a span whose body holds spans of its own: each context
names its closing byte, its escape byte and the literals that enter other
contexts, and a context’s close returns to the one it was entered from, so a
shell’s "..." holding $(...) holding "..." again reads as one token. The
return stack is a scratch buffer through the lane’s %mem-* forms, so each
context is one compiled state and contexts nest 63 deep. (Lexer escape TAG
BYTE) reads a byte and the byte after it as one token.
A lexer starts in half the time (#913). The Lexer imports only the
compile cache’s door, not compile.x and its C lane, so importing it costs
130 ms instead of 630 ms. The arm64 relocator works through the integer
primitives, so a cache hit re-encodes thirty baked addresses in 0.57 ms
instead of 3.3 ms; every cached compile gains. (compile asm-cache-group)
keeps a set of compiled functions in one file under a caller’s key, and a
later process loads them into the heap in one read, so each compile hits
without hashing its key or opening a file; every entry is still matched by
its whole key text. The Lexer groups its states under its rules and end
text. Making the JSON lexer in a fresh process, imports included, falls
from about 1,100 ms to about 570 ms under helium and from about 270 ms to
about 220 ms under xenon.
A Darwin Host process record costs thousands of objects, not hundreds of thousands (#912).
A record read each field through a class call and re-resolved its libc
symbols per call: 187K objects a record, and a table of ~820 processes
passed 3.5 GB. It now reads off primitives and symbols resolved once a
process, cleared by the image recache hook: 14.5K a record, 11.8M the
table. Host args asks for kern.procargs2’s size instead of taking
kern.argmax’s 1 MB, 89.5K a call to 24K; reading a C string, 73K to 9K.
Socket sends and receives binary datagrams (#910). (Socket
send-to-run fd run host port) sends a run’s COUNT bytes as one datagram,
and (Socket recv-from-run fd n) answers ((STRING . COUNT) . (host .
port)), the payload with its NUL bytes and the sender to answer. A DNS or
TFTP packet could not cross send-to and recv-from, whose strings end at
the first NUL.
(Callback make f nargs) is a C function pointer that calls an x function (#909).
x/sys/callback writes a native stub a C library calls like any other –
qsort’s comparator, atexit’s handler – with up to four integer or pointer
arguments; f gets each as an integer and its integer answer is the C
result. (cb address) is the pointer to hand the library; the Callback
keeps f alive while it is held, and the stub is freed before a state image
is written and written again after a load.
A process record carries its group, session, real ids and tty halves (#908).
Host’s process record gains pgid, sid, gid (effective), ruid,
rgid, tty-major and tty-minor, the fields busybox’s ps -o prints that
it lacked. Linux reads them as busybox does, from /proc/PID/stat, the
/proc/PID directory’s group and /proc/PID/status; Darwin reads
kinfo_proc and asks getsid(2) for the session, which kinfo_proc does
not carry. tty stays the device number; its halves are separate fields
because each kernel packs them differently.
A command’s options and its help text are one declaration (#905).
(Opts declare NAME SYNOPSIS SUMMARY ROWS) takes rows built by flag,
arg, hidden and text, and the rows are both what parse accepts and
what usage prints, so an option cannot be accepted and undocumented, nor
documented and refused. parse and parse-leading take a declaration in
place of the two lists, and a row’s spellings are one option: on? and
value answer for whichever was given. help? asks busybox’s question,
--help as the first argument, unless the declaration turns help off as
test, true, false and echo do. usage lays the text out as busybox does,
the description column the first tab stop past the widest option unless
(column . N) sets it; 108 of busybox’s 121 option tables follow that
rule. The caller chooses the stream and the status.
(Sys local->unix ...) and (Date local->unix date) turn a local time into unix seconds (#906).
The inverse of Sys zone and Date local: the C library’s mktime over the
local fields, with daylight time left to it, after tzset. TZ chooses the
zone, and a time a daylight change skips or repeats lands where mktime puts
it.
The Lexer’s child base registers nothing, and every state is compiled and
rooted (#903). Tokens are built in the base that made the lexer, through
base-eval: an integer label made in the tokenizer base registered INTEGER
there with its s-expression analyser, and the child then read +1 as an
integer. The two states that were interpreted, a quoted literal’s escape
and a two-byte closer’s second byte, hand back through (first cell) and
compile, and the lexer holds those cells, which a collect had freed under
the compiled code. A dropped until span scores positive, so a comment
beats the / that opens it, and answers a marker read-str filters out. A
table of any size compiles; the trie’s names stopped at 26. (Lexer any
TAG), listed last, takes a byte no rule reads as a one-byte token; it
refuses the skip rules’ bytes and the end text’s, which is now make’s
optional second argument. Building tokens in the parent costs about 2.5 µs
a token: 0.8 µs a byte on words, 2.0 on C-like text.
A tokenizer base is built from data rules, its analysers compiled (#901).
(Lexer make rules) (lib/x/reader/lexer.x) registers one tokenizer type a
rule on a (Base make-tok) child, in list order, so the first rule wins an
equal-length tie. A rule is run, skip, table (longest match), quoted
(an escape byte; the text is kept raw), until (a span, kept or dropped) or
number (labels 1 integer, 2 fraction or exponent, 3 hex, and a suffix
class). Every analyser state is one form in the assembler lane’s dialect,
compiled through compile-asm when the lane is open and evaluated as the
interpreted twin otherwise; the base and its states are dropped before a
state image is written and made again after a load. On 20 KB of words a
byte walk in x costs 39 µs a byte through the raw string prims and 130
through Str8 ref, a base with interpreted analysers 62, and the Lexer’s
compiled states 0.29, the engine’s own reader’s speed; C-like text with
fifty-one compiled states reads at 1.2 µs a byte. docs/crafting-a-lang.md
carries the Lexer and the tier numbers.
Sys waits on several descriptors (#900). (Sys poll fds timeout)
is poll(2): it answers each ready descriptor with what it is ready for –
in, out, hup, err, nval – or nil once the timeout passes, and a
signal that interrupts the wait answers nil too. (Sys nonblock! fd) sets
O_NONBLOCK and (Sys nonblock! fd #f) clears it, through fcntl reached as
a syscall, since fcntl is variadic and Apple arm64 passes a variadic
argument on the stack; Darwin’s syscall table gains fcntl (#92). (Socket
shutdown fd) closes the write side of a connection, the peer then reading
end of input while its replies still arrive, and (Socket peer fd) names
the other end as (QUAD . PORT).
(Sys zone secs) and (Date local secs) give local time (#899).
Sys zone answers the zone in force at a unix second, from the C library’s
localtime_r after tzset: its offset in seconds east of UTC, its
abbreviation and whether it is daylight time. Date local is from-unix
moved by that offset, with offset and zone added; the calendar is still
Date’s own. TZ chooses the zone, as it does for every C program.
A program can catch a signal and read its arrival (#897). (Sys
catch-signal N) has the engine record signal N’s arrival and do nothing
more (x-engine-c#84); (Sys take-signal N) answers whether it arrived since
the last take and clears the record. A read or poll waiting when it arrives
is interrupted, so the program sees it at once. Sys names sigwinch,
sigttou, sigtstp and sigstop, the last two for the system they run
on. (Term raw-with-signals! fd) makes the terminal raw with its signal
keys still signalling, as an editor that catches them wants, and (Term
measure fd) answers the window’s size as the terminal reports it, or nil
when it reports none; window falls back from it as before.
The engine is x-engine-c v0.2.19 (#897). It carries the signal catch and take above (x-engine-c#84). A call through the base’s slot vector skips the test for a base and a vector where both are always present, and a call form whose head is a primitive goes straight to the callable-call slot (x-engine-c#83); the engine measures the slot vector’s cost over v0.2.17 at 1.6% on a helium boot and on a call-heavy loop, down from 2.7% and 3.3%.
str->number reads one to fifteen decimal digits in one walk (#896).
Text of plain decimal digits, with no radix passed, is read on the integer
doors; any other text is read by the parser as before. Reading "3" costs
6 µs where the parser costs about 390, and convert from a string to an
integer rides on it.
Host (x/sys/host) reports the machine the same way on Linux and Darwin (#895).
(Host boot-time), load, memory, cpu, processes, process, args
and users answer alists with unix-second times, nanosecond CPU times and
byte sizes, so a program such as uptime, free, ps or top formats them
without asking which kernel answered. Linux reads what BusyBox reads:
sysinfo(2), /proc/meminfo, /proc/stat and /proc/PID/stat. Darwin reads
sysctl, the Mach host statistics and proc_pidinfo over the dlopen FFI. Both
read utmpx through libc. A field the kernel does not report is nil; Darwin
gives another user’s process memory, CPU time and run state only to root.
Http streams a response’s body (#894). (Http open method url
headers body) reads the head and answers a stream – (s status),
(s headers), (s head) – whose body (Http read s n) hands out a piece
of at most n bytes at a time, framing removed, until nil; (Http close s)
ends it. A piece is a run, (STRING . COUNT): a buffer and how many of its
bytes count, NUL bytes included, written with (File write fd (first r)
(rest r)). (Socket recv-run fd n) and (Tls recv-run session n) read
runs, and the stream copies parts of them with libc’s memcpy, so a piece
costs a few objects whatever its size where a byte list costs about 74 a
byte to build. A 4 MB body reads in under a second with no growth in the
heap, where request holds all of it. The options are (redirects . N)
and (insecure), which skips https certificate verification. The stream
is the one response reader: request, %parse-response and %dechunk
all drain it. Tls connect no longer raises new: init key without a
value after a good handshake, which had broken every https exchange, and
the Host header carries the port when it is not the scheme’s own (RFC
9110 7.2).
(File type path) and (File ltype path) answer a path’s file type alone (#893).
Each answers the file type File stat or File lstat gives under
file-type, or nil when nothing is there, and never raises: it reads the
mode alone, and a miss builds no Err. A file that is there costs 32 µs
against File stat’s 97, and a path that is not there 16 µs against 355
for File stat under a guard.
The engine is x-engine-c v0.2.18 (#891). The engine’s base has two units, the tree first and a slot vector second: the evaluator, environment lookup, the reader, calling, allocation and the collector’s phases are each a function pointer at a fixed position, and storing another function in a slot replaces that routine in that base alone (x-engine-c#82). The library reads the base through its routes and needs no change; an image does not carry the slot vector, which is process state. The engine costs about 5% more user time than v0.2.17 on a helium boot and on a call-heavy loop.
A bundle’s CI runs its suite on arm64 Linux too (#892). The bundle
workflow’s specs matrix gains ubuntu-24.04-arm, against the declared release
and against main, as ci.yml’s specs matrix has it. The leg reports and does
not gate, as ci.yml’s does; its make engine clones the pinned engine and
builds it, since the pin declares no linux arm64 artifact.
The linter collects between a preload’s imports (#890). Each
top-level import in a lint child’s preload is followed by a collect, so the
preload costs its heaviest import rather than the sum of them.
lib/x/num/tower.x, which imports the whole numeric tower, measured a 9.4 GB
footprint before and 3.4 GB after; lint-x reports the same findings with
the same CPU. A module still never collects: these collects run at the top
level of the stream the engine is fed, where nothing is in flight.
A project’s label comes from its bitwise.xon (#887). The line above
a project’s name on a banner is the costume’s (label "..."), read by
(Bitwise render ...) when the caller passes none, instead of a table in
cli.x keyed by directory; a costume without one draws “an x project”.
Every project’s file carries the text the table gave it.
The linter sweeps within a file (#889). A lint child sweeps between
top-level forms, between a class’s methods and between the forms it reads
once more than 20M objects are allocated, where it swept only between files.
x-coreutils’s lint step, 31 files in one child, measured an 8,029 MB
footprint before and 1,436 MB after, with the same findings and no more CPU
(215 s before, 192 s after). The count is the base’s alloc-count cell;
(Heap count) walks the heap to answer it. A file’s preload is loaded
before the linter runs, so lib/x/num/tower.x, which imports the whole
numeric tower, still peaks at 9.4 GB.
Bitwise has one generator (#886). The JavaScript copy of gen.x – the
gallery page, its build and parity scripts, the thirty pictures it rendered,
the five bitwise-parity spec files and the costume fixtures only they read
– is removed. (Bitwise diff a b) stays. --all creates the parents of
--out, so its spec case passes in a checkout with no build/ yet.
Bitwise’s --kind is --label (#884). The text Bitwise draws above a
project’s name on a banner says which sort of project it is, so it is the
project’s label. The option is --label TEXT, and the index the app writes
holds it under "label". No alias is kept.
Catalogue, in every file (#882). The glossary’s word is catalogue, and
the tree’s comments, doc strings, specs and docs spell it so.
image-foreign-catalogue was image-foreign-catalog, and the conformance
spec is core/catalogue-ops.spec.md. No alias is kept.
A compiled state reaches another through a cell (#881). The assembler
lane lowers (first CELL) and (rest CELL), in analyser mode, to
jit_firstobj and jit_restobj. CELL is an fvar holding a pair, read when
the code runs, so two states that hand to each other both compile: the
caller puts each state in its cell with %set-first! once every state
exists. The operand is an fvar, an object parameter, or another first or
rest; a number, an fvar that holds nil, and an integer function refuse at
generation. The collector reaches a state through the cell that holds it.
The engine is unchanged.
The documentation takes the glossary’s words (#880). A discriminant is
a label, a tier is a profile, a layer, a pin or visibility, and shape is a
block form’s and nothing else. docs/type-system.md’s flags table loses the
six simple-type codes, which are x-expr’s. Prose only: nothing a program
calls is renamed.
The engine-contract gate calls an ISA row’s justification its label
(#879). Its failure lines read TOTAL: COORD (label X) and claimed both by
label X; they said tag. features.x and the two generators follow, and an
engine’s x-engine.xon comes out as before.
The seam gate calls a row’s always, installed or bundle its label
(#878). It refuses an unknown one with seam: unknown label(s) in FILE: NAME;
the line said class(es). tools/contract/seam.x gives a row as
(seam LABEL NAME "what it is"). The rows and the three values are unchanged.
The assembler and the float stubs work in single precision (#877). A
single (a C float) is the low 32 bits of a d register, the bits above it
zero on both backends. fcvt/d converts one to a double, fcvt/s a double
to one, and scvtf/s an integer to one in a single rounding: FCVT and SCVTF
on arm64; CVTSS2SD, CVTSD2SS and CVTSI2SS on x86-64, where PSLLQ and PSRLQ
clear the bits the last two leave. libm-fn with no name makes the new
stubs "d->f", "f->d" and "i->f".
The specs, the runner and the tools take the glossary’s words (#874).
Comments, spec prose and spec headings under tests/ and tools/ say
label for a discriminant, and pattern, layout, structure or case where
they said shape. A spec with an arch in its file name is arch-labelled.
Nothing a program calls is renamed.
The foreign census runs again, and the image inspector is gone (#873).
tools/dev/image-foreign.x called three helpers that left the tree on
2026-09-04, when the writer began to recognise a type’s call pointer by the
type. The census makes the writer’s test, then asks the naming map and the
linker, in the writer’s order, and reports the dlopen handle on a line of
its own. tools/dev/image-inspect.x read a header and called
(image rebuild!) as they were before that day, and is deleted; the loader
under %IMG-VERBOSE prints the counts and the externals that did not
resolve. tools/dev/README.md describes the writer, the loader and the
census as they are.
A state image carries compiled code (#872). The asm cache keeps each
entry in the heap as well as in its two files: the code in an object of type
ASM-CODE, whose first unit is its length and whose payload is word
units, and the relocation records as a list. Nothing in a held entry is an
address, so an image carries it. compile-asm looks for the held entry
first and pours from it with one block copy, then probes the files, then
compiles. The entries on Compiled’s list are switched to interpreted
before a write and compile again after a load, as before, and the compile
after a load is a copy into a fresh page and a patch, with no file and no
compiler. The cache’s files stay, for boots from source.
On x-engine-c v0.2.17, darwin/arm64, a xenon image loaded with the cache directory empty:
| time | cache files afterwards | |
|---|---|---|
| with held entries | 0.59s | 0 |
| before, cache warm | 0.70s | 48 |
| before, cache directory empty | 1.81s | 48 |
The image grows from 8,845,204 to 9,043,065 bytes. A process holds its entries whether or not it writes an image.
The library’s last loose uses of kind, shape, family and tag are reworded
(#871). On arm64 Linux the call made in a name’s place is a stand-in:
linux-generic-syscall-stand-ins was linux-generic-syscall-shapes, and the
two private door builders follow. Comments that said “two kinds of”, “the
same shape”, “the scalar double family” and “tagged” name what they mean.
No alias is kept, and no bundle uses the names.
The image tools say label and unit labels (#869). image-unit-label
and image-type-label were image-unit-kind and image-type-kind, and
image-unit-labels-count, -mask, -desc and -static? were
image-shape-count, -mask, -desc and -static?. A foreign table
entry’s 1 to 8 is its label, and the reader prints unresolved label for
one it cannot place. No alias is kept, and no bundle uses any of them.
The library’s private names and comments take the glossary’s words
(#866). Names that said kind or tag for the value that says which variant
something is say label, %apply-label, %field-label and %asm-cache-label-
among them, and a type’s named leaves are fields, as in %type-proc-fields.
Two instance fields follow: Random’s kind and Indent’s mode are
label. Two raised messages read Float: no such stub label and
x86_64: unsupported 3-address arrangement (dst==src2). No alias is kept,
and no bundle uses any of them.
The list of functions that have a compiled version is Compiled
(#859). x/sys/swap and its class Swap are x/tool/compiled and
Compiled, with names that say what each thing is. An entry holds a
function’s interpreted version, the function that does the compile, the
function that does the install, and the compiled version once there is
one; its state is compiled, interpreted or failed. No alias is kept.
| was | is |
|---|---|
(Swap site! name twin maker seat) |
(Compiled make name interpreted compile install) |
(Swap in-env name env), (Swap in-cell pair) |
(Compiled into-name name env), (Compiled into-cell pair) |
(Swap down!), (Swap up!) |
(Compiled interpret-all!), (Compiled compile-all!) |
(s down!), (s up!), (s value) |
(c interpret!), (c compile!), (c compiled) |
(Swap rows) |
(Compiled list) |
states up, twin, down, refused |
compiled, interpreted, interpreted, failed |
(Swap site-on-demand! ...), new in 0.17.0, is (Compiled make-on-demand ...).
The compiled SHA-256 engine’s entry is made with it, as before.
The linter’s depth warning is named depth (#864). A definition at
least 12 deep and 500 nodes large was reported as shape; a lint run now
prints depth: for it. The lang kit’s --strict matches the new name, so a
bundle’s strict lint fails on the same findings as before.
Five reads of a borrowed catalog alias take the public doors (#861).
tools/dev/fmt.x, bench-sha256.x, cov-report.x and doc.x, and
type/block.x, called %cvt or %str->symbol without binding it; the name
was in the root only because another file had fetched it. The tools call
(Convert to ...) and (Str8 ->sym ...), and block.x fetches the
primitive where it uses it. check-private-reads also stops counting a
member that a class body declares, (%size 8192).
A tool script is a reader of private names and never an owner (#867): the
gate takes its owners from lib/ and apps/ alone, and refuses a file of
lib/ or apps/ that loads a file under tools/, which is what that rests
on. A tool that reads a library’s private name is counted as before. With
both, tools/contract/private-reads.x holds no row; the count was 954 on
2026-09-21.
What Paint classify answers is a label (#862). (Paint labels) lists
the nine symbols classify can answer; it was (Paint classes). classify
keeps its name, and 'class stays among the nine, the label of an atom that
names a class. No alias is kept, and no bundle uses the old name.
The image tools’ heap walk and foreign namer are library modules (#857).
tools/dev/image-walk.x and tools/dev/image-name.x are
x/tool/image/walk and x/tool/image/name, with provide lists, and
image-write.x, image-inspect.x and image-foreign.x take what they use
by selective import. An export is a bare name, image-walk,
image-over-units, image-name-map, and a name left private is
%image-NAME. The walk’s periodic collect is switched with
image-walk-collect!. Both modules are unscoped, the walk being per object,
and a header was not measured. tools/dev/image-read.x is as before, a
file included by path, since it loads where there is no module system. An
install carries the two modules with lib/. The image scripts’ private-read
rows go from 64 names to none.
The type word is a label, and (obj relabel!) writes it (#860). The
word in an object’s header that says which type it is was called the type
tag. (obj relabel!) was (obj retag!), and its refusal reads
relabel!: unknown type handle. Type cast!’s doc strings and the
library’s comments say type label. No alias is kept, and no bundle uses
the old name.
A site that was never brought up is down (#842). x/sys/swap documents
state, reason and the static all with their defaults,
(doc (NAME default) "..."), which the reference can name since #833. A
site made with new is down with an empty reason until it is brought up;
both were nil. (Swap site! ...) brings a site up as it makes one, and is
as before.
The counted private reads are the ones that still want a door (#852).
Three changes to what check-private-reads counts, 183 reads in 40 files
before and 78 in 11 after. A name in the selector’s place of a send,
(self %walk ...), is a message and no longer a read. The names other files
read from the files that stay unscoped are listed in
tools/contract/shared-privates.x, each with what it is: a door in front of
a name that stays in the root would hide nothing and cost a dispatch. And
seven readers of %stderr, five tool scripts and the line editor’s two
files, write through (Stream with-fd 2 thunk); the name is listed for the
three readers that cannot load that module. What is left is the tool
scripts reading one another and a few catalog aliases.
The profiler says which functions evaluation goes to (#835). The
profiling engine that x-engine-c ships from v0.2.15 counts, in each object, how
many times evaluation reached it. x/tool/profile now reads those counts by
function: profile-fn gives a function’s calls and the evaluation its body
did, profile-report lists every function on the heap by the file and line its
body starts on, and make profile-x FILE=program.x profiles a program. Between
clearing the counts and reading them, the profiler runs only the engine’s forms
and primitives, so it adds nothing of its own; the reader, which is x code, is
counted when it reads the program. make x-bin-profile takes the shipped binary
from an unpacked release, and the image tools mark with the engine’s
%obj-flag-trace instead of 1024.
A site can be made on demand, and the SHA-256 engine’s is (#854).
(Swap site-on-demand! name twin maker seat) records a site and brings it up
as site! does, and adds no recache hook: the site goes down with every other
before a state image is written, and stays down after one is loaded until its
owner sends it up!. It is for a value that is dear to make and not always
wanted. The compiled SHA-256 engine is one, so a process that loads an image
holding its site does not build the engine as it loads; (Sha256 jit!) or an
input of 12KB or more builds it again. (Sha256 jit!) on a site that is down
therefore builds and answers #t, where it answered #f. A refused build is
still not tried again.
core/fn.x has a module header (#846). The apply door’s eight private
names are the module’s own, and the root keeps apply and the Fn class.
The file had been measured and left unscoped: a call through the library’s
apply compares 351 bindings with the header where it compared 190, with
evaluations and allocations unchanged. It is scoped by decision. The
library’s own callers hold the engine’s apply, so the cost falls only on a
call through the door. One %-row is retired, and 79 of the library’s 138
files are scoped.
The reference names a field documented with a default (#833). A field’s
doc form wraps its declaration, (doc NAME "...") or
(doc (NAME default) "..."), and a static field’s does the same.
(help Class/NAME) read both, and the generated reference read the first:
handed the second, it passed the declaration, a pair, to symbol->str, which
is unchecked, and the entry’s heading and its man page name came out as bytes
that are not text. macOS’s awk stops on such bytes, so make doc-x and
make check-man failed there for a module that used the form, and passed on
Linux. The generator now names the entry by its declaration’s name.
A static field is noted in the reference as “Static field: data held by C itself, not by its instances.” It was noted as data carried by a C instance, as a field is.
The compiled SHA-256 engine is a site (#849). x/codec/sha256 kept its
own record of the engine (Sha256 jit!) builds: nothing, the symbol failed,
or the engine. It makes a site of x/sys/swap instead, named sha256, the
first time a build is asked for, with the pure-x digest as its twin. A build
that raises is refused by the site, which keeps the raise’s text, so
(Swap report) shows why an engine is not in use. %sha-jit-engine is gone;
(Sha256 jit!), the digests and the 12KB bar are unchanged. A process that
loads a state image holding the site builds the engine again as it loads.
The reference gate checks a documented field (#841).
tools/check/doc-forms.sh checks that every field and static field declared
in a class body under lib/ has an entry on its module’s page in
docs/ref/x. A declaration wrapped in a doc form, (doc NAME "...") or
(doc (NAME default) "..."), was skipped, because the gate read the form’s
head, doc, as the class’s own description. The walker,
tools/check/doc-forms.x, now reports such a form’s declared name in a fourth
column, and the gate checks it under that name. Three fields were skipped
before: Base raw, Type handle and Type raw.
A field documented with no description, (doc NAME), which (help Class/NAME)
reads as an empty description, stopped the reference generator with a
segmentation fault. The generator read the description with an unchecked
first of the empty list. It now reads an absent description as the empty
string, and the entry keeps its heading and its note. No module under lib/
used the form yet.
The reference renders (doc) as the field it declares (#847). A doc
form with no argument, (doc), is neither a class’s description nor a field’s
doc, and lib/x/type/class.x declares it as a field named doc. The reference
generator read its argument without a check and stopped with a segmentation
fault. It now renders the form as a field named doc, and the doc-forms gate
checks it under that name. Nothing under lib/ writes the form.
An instance’s data is its fields (#829). A field is a named component of
an instance or a record, and a member is a field or a method. Inside a method,
(field 'name) reads an instance’s storage and (set-field! 'name v) writes
it; they were (member 'name) and (set-member! 'name v). (class-fields c)
lists a class’s own field names; it was (class-members c). (help Class)
heads an instance’s data fields:, and the generated reference notes each one
as a field. No alias is kept.
Five messages follow: is not a field of, use bare field names,
declare fields directly, expected a field name and duplicate field.
A class’s own data is its static fields (#840). (class-static-fields c)
lists them; it was (class-static-members c). (help Class) heads them
fields: under static:, where it said members:. A static member is a
static field or a static method, so no such static member reads as it did.
One message follows: call with no selector -- name a field or method. No
alias is kept.
The doc-forms gate counts fields, and its lines say so (#844). Its
lines read doc-forms: ok (87 fields, all rendered),
doc-forms: FAIL -- fields declared but never rendered: and
FILE class NAME field NAME (absent from PAGE); they said members and
member. The exit codes and what is checked are unchanged.
The tokenizer’s channel is the label channel (#851). The integer an
accepting state declares for its type’s reader is a label: the value that
says which variant the state saw. %score-label! writes it and
%read-label reads it; they were %score-variant! and %read-variant. The
assembler lane’s message reads a state that declares a label cannot be
compiled, and the two spec files are lib/reader-label.spec.md and
ext/jit-analyser-label.spec.md. No alias is kept. x-python, the one bundle
that reads the pair, takes either name since x-python#184.
A unit’s ref, word, bytes or foreign is its label (#855). The
helpers behind (Type set-unit-labels!) said kind for the label and shape
for the declaration. %unit-label and %unit-label-mask were %kind-code
and %kind-mask, the rows are %type-unit-label-rows in
lib/x/type/unit-label-rows.x, which was %type-shape-rows in
shape-rows.x, and an unknown label raises
'type-unit-labels-unknown-label. The image loader’s copies follow, under
%img-unit-label- and %img-declare-. No alias is kept, and no bundle
uses any of them.
A stat record holds a file’s type under file-type (#823).
(File stat p) and (File lstat p) answer
((size . N) (mode . M) (file-type . K) (mtime . T)); the third key was
kind. The values are the same: 'file, 'dir, 'link, 'char, 'block,
'fifo, 'socket and 'unknown. No alias is kept: a caller that read
(Assoc get 'kind st) now reads (Assoc get 'file-type st).
type/promise.x and repl/ansi.x have module headers (#824). The
promise type’s handle and five other % names, and ansi’s twenty-six, are
their files’ own now. The root keeps the Promise and Ansi classes and
delay. x-r7rs defines a %promise of its own at the root, which until now
rebound the name the platform’s class and delay read. repl/ansi.x still
sets the names other files own, the REPL’s printer, doc.x’s colour stubs
and the image recache hooks, through its module’s parent, and a boot from a
state image prints in colour as before. The ansi spec reaches %code-sugar
through the module. Two %-rows are retired, and apps/bitwise/gen.x’s
private-reads row falls by one, 674 to 673: its own %esc method had been
counted as a read of ansi’s. core/fn.x was measured for a header and is
left as it is: a call through apply would compare 351 bindings where it
compares 190, and docs/namespaces.md has the numbers.
The names read across files by decision are listed, and not counted
(#836). tools/contract/shared-privates.x lists the private names other
files may read, with what each is: fifty-nine names of the eight boot files,
the eighteen walkers of core/list.x and core/alist.x, and the four names
of the analyse protocol. check-private-reads does not count a read of a
listed name, nor of a % name of the seam, which it takes from
tools/contract/seam.x. It refuses a read of any other private name of a
boot file, whatever the reader’s budget, so what the boot layer shares grows
only by an edit to the list. A row is held to the tree in turn: its file has
to define the name, another file has to read it, and a name a document
promises has to be mentioned there. The budget rows fall from 673 reads in
121 files to 183 in 40, the reads that still want a door. No library file
changes.
Sites: a slow value set aside for a faster one (#827). x/sys/swap is
new. A site records one replacement of a slow value by a faster one: the seat
the value sits in, the slow twin that belongs there, and the maker of the fast
value. (Swap site! name twin maker seat) records one and brings it up. A
maker that raises is refused, and one that answers the twin has declined; the
twin stays seated either way, and the site’s state is up, twin or
refused, with the raise’s text as its reason. (Swap report) prints every
site, so a compile that was refused shows. (Swap in-env name env) and
(Swap in-cell pair) make the two seats the library has needed, and any
function of one value is a seat.
Every site is put down before a state image is written and comes up again
after one is loaded, in the order the sites were made. The tower’s compiled
analysers (boot/tower-compiled.x) are sites of the module, where the record
and its two walks were the tower’s own: %tower-sites, %tower-unjit! and
%tower-rejit! are gone, with the record’s accessors. The tower’s makers no
longer catch a refused compile, so the raise reaches the site. A type’s entry
analyser is pushed as its interpreted twin and compiled in place, where the
compiled one was pushed and, before an image write, removed, so an image holds
the twin in the type’s list.
boot/reflect.x files two doors in the catalog, (image transient!) and
(image recache-hook!), which add to the image writer’s transients and to the
loader’s recache hooks.
An error’s classifying symbol is its label (#821). (Err label e) is the
total accessor, (e label) the field and (e label? 'io) the predicate, and
make and raise take a label. They were (Err tag e), (e tag) and
(e tag? 'io). The glossary’s word for the value that says which variant
something is, is label, and an error’s 'io is one. No alias is kept: a guard
that matched on (Err tag e) now writes (Err label e).
A bundle that runs on both sides of the rename asks the platform once, at load, which name it has, as x-coreutils and x-python do:
(def %cu-err-label
(guard (_ (fn (_ e) (Err tag e)))
(do (Err label "probe")
(fn (_ e) (Err label e)))))docs/namespaces.md opens by saying the design is built (#839). It
opened as a proposal with nothing implemented. It now says what it is, the
record of the design’s reasoning, points to docs/modules.md for how a
scoped module is written, and dates the counts of its first section to the
tree before any of it. The index no longer calls it a proposal.
The r5rs three and the r7rs six are answered in the bundles (#831).
tools/contract/langs.x recorded x-r5rs at 667/3 and x-r7rs at 33 failures
on this tree, read as the platform’s debt since the engine made an
environment a value. x-r5rs now evaluates a macro’s expansion in a child of
the use site’s environment (x-r5rs#17, its v0.2.4), and x-r7rs binds
let-values’ formals in one (x-r7rs#19). Measured on v0.15.0 and on main,
x-r5rs is 667/0 and x-r7rs 637/27, the 27 it records by name, and its row
here follows the count from 30 to 27.
The six large files were measured for a module header, and none takes
one (#837). The line was decided on 2026-09-27: a header goes on when it
adds under 0.1% to the environment comparisons of an x-core boot and under
10% to those of the file’s own work. With a header, type/class.x makes an
x-core boot compare 3.8 times as many bindings and a static call 10.5 times;
doc/doc.x adds 0.98% to the boot and makes a doc form compare 5.8 times
as many; tool/asm.x doubles an instruction emitted; tool/lint.x adds 33%
to a lint and codec/sha256.x 37% to a digest of 1 KB. The evaluations and
allocations are the same either way. boot/tower-compiled.x is a load
sequence, and a scoped file holds no plain include. docs/namespaces.md
has the numbers, and records that a development tool reads a module’s
private name as a spec does, (eval (lit NAME) (module M)). No code
changes.
A spec may not rebind the library’s root %-definitions (#818). A spec
snippet is evaluated at the root, so its top-level defs outlive it, and
check-spec-globals refuses one that takes a name the shared vocabulary owns.
That vocabulary was the engine’s bare and keep names and
tools/contract/bare-globals.x; it now also holds the top-level %-defs of every
library file without a (module NAME) header, 930 names in all where the gate
held 137. Rebinding one breaks the rest of the spec’s own file as well as later
files in its batch: + and * call %fold past two arguments. Four specs
whose local helpers took such a name (%find, %str->ptr, %ptr->int,
%ptr-set!, %ptr-ref, %obj->ptr) now prefix them with their subject.
Doc annotations spell the runtime type’s name (#817). An annotation, the
T of a (param NAME T ...) or (returns T ...) doc form, names a runtime type
as registered, a class as spelled, or one of ANY, NUMBER, CALLABLE,
ALIST, TYPE and NIL. The library spelled seven of them short, and 540
annotations now say INTEGER for INT, CHARACTER for CHAR, POINTER for
PTR, STRING for STR, PRIMITIVE for PRIM, Gen for GEN and Dict
for DICT. (help ...) and the generated reference show the new spellings.
make check-doc-annotations holds the rule, and make doc-x runs it. It asks
the platform it runs on for its runtime types, reads the classes from the
source’s def-class forms, and takes the six other names from
tools/contract/doc-annotations.x, so nothing is edited when a type or a class
is added. make check-doc-vocab is removed, with the four lists it carried.
A system call’s result folds on the integer primitives (#812). Every
Sys method that calls libc passes its result through %sys-fold, which turns
the top half of the u32 range back into negatives: Linux hands an int return
back zero-extended, so its -1 arrives as 4294967295. The test and the fold ran
on the library’s generic > and -, about 290 objects a call. The result is
always the integer the FFI returns, so they now run on the engine’s integer <
and -: the fold allocates 7 objects, and 8 when it folds, and a Sys getpid
through a method-of door allocates 135 where it took 417.
A spec case that writes a file writes a fresh one (#843). A case that
wrote a fixed /tmp path and then checked it failed when two runs of
make test-x overlapped on one machine: one run unlinked the other’s file
between its write and its check, and ext/io.spec.md’s “file-exists? tracks
create then unlink” answered (#f #f #f) where it expects (#f #t #f). Those
cases in ext/io, ext/posix, ext/compile, ext/jit-cache,
lib/file-ergo, lib/socket and lib/zlib take their path from
(File temp PREFIX); a case that needs a path that does not exist yet unlinks
its temp file and reuses the name. Every assertion is unchanged.
The sh-open-write spec case leaves no file behind (#848). ext/posix.spec.md’s
“opens and closes without error” opened /tmp/x-test-open.txt and never
removed it. It takes a fresh name from (File temp PREFIX) and unlinks it
after the close.
The assembler takes a label’s address (#834). (adr Xd (label L)) sets
Xd to the address of the label L, counted from where the instruction sits,
so code that runs wherever it is loaded can hand out addresses inside itself.
On ARM64 it is ADR, which reaches a label less than a megabyte away in either
direction, and asm-finalize! refuses one farther; on x86-64 it is a lea
from rip, which reaches two gigabytes either way. A register holding a
label’s address is called with blr.
A directory listing decodes on the integer primitives (#811). File
list-dir on a 50-name directory allocated about 111,000 objects: the dirent
decoder read each record a byte at a time through the tower’s +, *, =
and <, . and .. were dropped with List reject, and the directory was
opened and closed through two class dispatches. The decoder now works on the
integer primitives and reads a Darwin name by its namlen, one walk drops the
dot entries and keeps the order, and the method opens and closes the directory
through the module’s syscall doors. The same listing allocates about 11,300,
and a 52-entry batch decodes in about 7,000 where it took 38,900.
The ASan boot gate is safe beside another run of itself (#820).
tools/check/asan-boot.sh made its boots cold by moving every /tmp/x-asm-*
entry into its scratch directory and back under set -e, so a second run that
moved the same file first ended it with status 1 and nothing printed.
compile-asm’s byte cache takes its directory from X_ASM_CACHE_DIR, and from
/tmp when that is unset or empty, and the gate gives each dialect’s boot an
empty directory of its own: it moves no files and shares no entries. Every boot
compiles cold, rn’s included; xe and rn compile the same 24 units, and a cache
shared by the run left rn none to compile. The boots pass --no-image, because
rewriting a stale state image boots the dialect first and fills the cache before
the boot under test.
A block-wrapped method tests for a block send without a class dispatch
(#815). Every send of a selector Block method! wraps – List map,
filter, fold, sort and the rest, plain applicative sends included –
first asked whether it was a block send, through four Block class
dispatches, and, or, let and the tower’s < and -. The test is made
once per wrap now, as closures over the integer primitives, and answers as
before. A wrapped method called through its table entry costs about 210
objects a send where it cost 16,300; with the dispatch table’s reordering
as well, (List map f ()) costs about 1,900 where it cost 25,300.
List sort recurses on plain functions made once (#816). The sort
re-created its split, merge and reverse-onto closures on every call and
recursed through recur, re-entering the method and from-seq’s dispatch
at every level. They are module functions now, the sort proper is one
function recursing on itself, and the input is converted once; sort-by
calls the same function. Fifty integers sort in about 56,200 objects where
they took 204,500, and in about 24,700 with the dispatch table’s reordering
and the block wrap’s test as well; an empty sort costs about 1,600 with all
three, where it cost 33,200.
The engine pin moves to x-engine-c v0.2.15 (#810). Every engine release
now ships x-bin-profile beside x-bin, and under X_PROFILE each object’s
flags word counts how many times evaluation reached it (x-engine-c#66). The
count takes bits 11 to 30, above the trace bit, 1024, that the image writer
marks with, which the engine now names X_OBJ_FLAG_TRACE; the plain x-bin
is unchanged. The engine’s tools/contract/obj-layout.x gains the trace and
count rows, and the declaration’s layout digest changes with them.
The engine pin moves to x-engine-c v0.2.16 (#838). v0.2.16 carries the
engine’s contract in the glossary’s words (x-engine-c#69) and %seq over any
number of forms (x-engine-c#70), and x-lang follows its names: the ISA
manifest’s block is %isa-catalogue, (type set-shape!) is
(type set-unit-labels!) and Type set-shape! is Type set-unit-labels!, the
compiled states’ door is jit_score_label, and the tokenizer’s claim is
tok/label. obj-layout.x no longer carries x-expr’s simple-type codes, and
the base’s field routes end in -fields.
Two selectors of one class sent in turn settle at the front of its
dispatch table (#814). A lookup that found its selector deeper than
second place swapped it with the head, which sent the head’s entry to the
hit’s cell, so a method and a helper of the same class sent in turn –
List sort and the List from-seq inside it, File open and File close
– traded places at that depth on every send, and each lookup walked the
table that far. A deep hit now moves to the head, the head’s entry to second
place and the second’s to the hit’s cell, and the walk is a plain loop.
(List sort < ()) allocates about 4,100 objects where it took 33,200, List
map, filter and fold on an empty list about 4,300 where they took 21,400
to 25,400, and a File open and File close pair about 2,200 where it took
15,800.
Constructing an object fills its fields without derived forms
(#825). (new C ...) ran every field through let frames, unless,
do, not and if in %instantiate, %init-fields, %check-init-keys
and %opt-cell; the same steps run on def, match and an applied fn.
A three-field (new Err ...) allocates about 1,050 objects where it took
3,500, and (Err make ...) about 1,740 where it took 4,200.
A data-slot access does no generic arithmetic (#809). (obj ref),
(obj set!) and the pair mutators address data word i by one formula, the data
offset plus i words, and it was computed with the + and * bound at the root:
the variadic wrappers under helium, the numeric tower’s generic dispatch under
xenon. The formula now runs on the engine’s integer + and *, fetched once,
and %set-first! and %set-rest! write at the first and rest offsets computed
at load, so they do no arithmetic at all. Under xenon a %set-first! allocates
13 objects where it took 237, %set-rest! 13 where it took 293, and (obj ref)
and (obj set!) 22 and 25 where they took 280 and 283; under helium the four
took 51, 51, 38 and 41.
do builds only the nest a body needs (#822). do checked each cell of
its body with a function call and rebuilt the whole body as nested two-form
%seq calls on every run: 19 objects for one form, 36 for two and 87 for five.
The checks are now match tests over the type prims, which allocate nothing. One
form is tail-evaluated as it is, two are handed to %seq in the body’s own
cells, and a longer body builds its nest with the body’s last cell innermost.
Every cell is still checked before any form runs. One form now allocates 5
objects, two 6, three 23 and five 43, and (do (def x 1) x) 8 where it took 38.
do is the engine’s %seq (#832). do and begin are bound to %seq,
which sequences any number of forms (x-engine-c#70): each form but the last is
evaluated in the caller’s environment, the last in tail position, no forms
answer nil, and a dotted body raises before any form runs, with the engine’s
message, call: improper argument list (dotted tail). A do body allocates what
a function body with the same forms does: nothing for (do 1 2 3 4 5), where it
took 43, and 2 for (do (def x 1) x), where it took 8. lib/img.x binds its
do the same way, and the asm and cc compile lanes compile %seq as they
compile do.
The image loader sequences its install with atomic (#850).
tools/dev/image-read.x installs the image’s cells and rebinds args inside
one form of primitive calls, and carries on in the image’s environment. It
sequenced the two with %seq. From x-engine-c#72, %seq makes the environment
it started in current again when its last form is done, which would put the
loader’s own back. atomic evaluates each form itself and leaves the
environment as the forms leave it. Images load as before on v0.2.16.
The engine is x-engine-c v0.2.17 (#853). Its %seq, which do is,
walks its forms as an operative body is walked (x-engine-c#72). A dotted body
raises where the walk reaches the dot, after the forms before it have run,
with the same message, call: improper argument list (dotted tail). The
environment %seq started in is current again when its last form is done.
do still allocates nothing a call.
File stat decodes through a Struct reader made once (#813). A stat
allocated about 51,000 objects, 40,600 of them decoding three fields: Struct
unpack compiled its field spec on every call and assembled each byte with
the tower’s *, + and <, three Assoc entry sends followed, and the
decode was a second class dispatch. The codec’s readers now work on the
integer primitives, a reader’s fields carry their offsets with the pads
dropped, and File makes its stat reader when it loads: a stat allocates
about 1,800. File exists? tests the call’s result instead of raising and
catching an Err on a miss, about 980 objects either way where a miss took
25,600. A machine integer wraps at 64 bits, so an i64 field reads as its
two’s-complement value with the numeric tower loaded too, where -1 read back
as 18446744073709551615, and a pre-1970 modification time no longer depends
on the dialect. A reader raises type for a record offset that is not an
integer.
A failed call’s errno is read without per-call lookups (#826). (Err
errno-of) fetched its three pointer primitives with prim-ref on every
call, and (Err from-errno) built its message through a Str8 class send;
the primitives are resolved when the module loads and the message is joined
directly. errno-of allocates about 700 objects where it took 1,630; with
the construction change as well, a failed File stat allocates about 4,500
where it took 9,650.
A call through a syscall door walks nothing (#807). A door from
syscall-door found each of the six arguments it hands the primitive by
walking the list it was called with, on every call, so a call cost 400 objects
in x-core where the syscall-id lookup it replaced cost 220. A door is now
resolved when it is made, into a function of six arguments whose body is the
call: a plain door hands its arguments to the primitive as they are, and a
shaped door is the same form with its shape’s slots in place, built and
evaluated once. A call through a door allocates 20 objects, and File read and
File write 977 each, where they took 1,357 since #801 and 1,170 and 1,187
in v0.15.0.
The REPL searches its history. ctrl-r searches backward for what is typed
after it and ctrl-s forward, as readline’s incremental search does: the prompt
becomes (reverse-i-search)query’: , the newest entry holding the query is
shown with the cursor on the match, and each key extends the query. ctrl-r or
ctrl-s again moves on to the next entry that holds it, passing over a line
identical to the one shown. Backspace steps back to the match before the last
key, Enter runs the match, ctrl-g abandons the search and leaves the line as it
was, and any other key keeps the match for editing and is then handled as
usual, so Up and Down carry on from the entry that was found. ctrl-r on an
empty query repeats the last search, which (Line last-search)` holds.
The search is Edit search and Edit jump!, both pure: search reports where
a match is without moving there, and jump! moves there only once the match is
accepted, stashing the line being typed so that Down walks back to it. A query
that no entry holds is not held however it is extended, so typing after a
failed search, or repeating it in the same direction, looks nothing up. Every
language whose prompt reads through the line editor has the search, x-ash and
x-python included.
CI runs the specs on arm64 Linux (#803). The specs job gains an
ubuntu-24.04-arm leg, which reports without gating: the run’s verdict stays
with macOS and x86-64 Linux. The pin declares no linux arm64 engine, so the
leg builds the pinned release from source, and the state-image cache key now
carries the runner’s architecture. The one case the leg failed,
tools/tests/specs/cov/meta.spec.md’s read of an unwritten meta slot, now
builds its pair at width 0, the branch it names.
The engine pin moves to x-engine-c v0.2.14, which has no ffi-call
(#804). v0.2.14 removes the (ffi call) primitive and its ISA row
(x-engine-c#63), which #796 had already stopped using, so
tools/contract/features.x drops ffi/call from the isa/ffi-call group,
which keeps ffi/dlopen, ffi/dlsym and ptr/call, and the conformance
case for the signature-driven double call goes with it. The release also
stops a primitive called with too few arguments, such as (eval), (-) or
(fn), at the missing argument instead of segfaulting (x-engine-c#62), and
adds a tenth profiling counter, profile-env-steps, at the end of the base’s
profile group (x-engine-c#59); the library reads base cells by name, so
nothing here moves with it. dlopen and dlsym are now the CLI’s, which
files them in the root base’s catalog only, so the child base the image
writer loads a library into had neither, and every image write crashed in
err.x’s errno lookup. tools/dev/image-write.x files both in the child’s
catalog, as it already binds the CLI’s include and syscall there.
type/convert.x has a module header (#802). Its six type handles and
its fourteen other % names are its own now. The root keeps the Convert
class, and the catalog keeps (convert to), which is how every hot caller
reaches the dispatcher. Nothing read the handles by name any longer: the
library’s readers moved to (Type named ...) in #799, and the lang
bundles in their own repositories. Counted with the lookup counter on an
x-core boot, the header adds 0.02% to the environment comparisons. A
conversion through the dispatcher compares about 740 more bindings, 9% more
on a loop of symbol to string and 5% more on integer to string and back,
with evaluations and allocations unchanged; docs/namespaces.md records
the numbers beside core/arithmetic.x’s. The file’s %-row of 20 is retired.
File runs on arm64 Linux (#801). syscall-id gave every host that was
not Darwin the x86-64 numbers. arm64 Linux uses the generic table, where those
numbers name other calls and fourteen of the path calls – open, stat, lstat,
mkdir, unlink, rmdir, rename, chmod, chown, link, symlink,
readlink, utimes, mknod – have no number at all, their work being done
by the -at forms. x/platform/syscall now selects Darwin’s table, x86-64’s
or the generic one, and exports syscall-door: a call resolved by name, once,
to a function that makes it, through the -at form where the platform needs
one. File and the module loader’s directory reader call through doors. The
stat struct’s layout is platform data beside the open flags, exported as
stat-layout; arm64’s has mode at 16 where x86-64’s has it at 24. arm64
Linux has its own open-flag table, since direct, largefile, directory
and nofollow trade values there.
A Linux host that is neither x86-64 nor arm64 raises unsupported-platform
when the platform layer loads, where it ran on x86-64’s numbers. syscall-id
answers -1 for the fourteen names on arm64 Linux; a caller making one of them
wants a door. The Linux sync open flag is 1052672, O_SYNC as <fcntl.h>
has it, where the table held the bare __O_SYNC bit, 1048576.
The library’s last readers of type/convert.x’s handles use the door
(#799). %int, %string and %ptr were still read by name in
codec/sha256.x, type/block.x, tool/lint.x, five dev tools, the
engine-contract checker, the cov spec and a line of docs/contributing.md.
Each takes the handle from (Type named ...) now: held by the one function
that uses it where the read is warm, asked for in place where it is cold,
and kept as a class static in the contract checker. tool/lint.x takes its
handle from the catalog’s (type of), as it takes the two type structs
beside it, and no longer caches the two type prims it used at load alone, so
its %-row falls from 89 to 88. Eight private-reads rows fall by eleven
between them, 686 to 675. type/convert.x keeps its handles and takes its
module header when the lang bundles have moved; each of the eight that read
them has a pull request open in its own repository.
Num modulo is right when the divisor passes half the integer range. It
computed ((a % b) + b) % b, and the sum wrapped once b was past half the
range: (Num modulo 5 LONG_MAX) answered -9223372036854775804, and
(Num modulo -1 LONG_MIN) answered LONG_MAX. It now adds the divisor only
when the truncated remainder has the other sign, where the sum cannot
overflow. A call costs about 9 objects more.
No division reaches the machine with a divisor it traps on. The C
division under %int/ and %int% tests nothing, and x86-64 traps on a zero
divisor and on LONG_MIN over -1, where arm64 answers: 0 for a zero divisor,
LONG_MIN for the pair. The #80 zero guard covered only core/arithmetic.x’s
/ and %. Under the tower rational.x’s / divided before its zero check,
so (/ 1 0) trapped on x86, and (/ 0 0), a bigint over 0 and (/ 1+2i 0)
answered 0 on arm64. bigint.x’s / dropped the guard it replaced, and
Num quotient, remainder, modulo and divmod had none. Random int with
0, which an empty range passes, trapped on x86 and never returned on arm64.
A zero divisor now raises in each, and Random int refuses any bound below 1,
as its documentation says. At -1, % and the Num methods answer without
dividing: (% LONG_MIN -1) is 0, and (Num quotient LONG_MIN -1) promotes
under the tower. Helium’s (/ LONG_MIN -1) wraps to LONG_MIN, as its *
and - do there.
x86-64 add and sub with an immediate read their source register
(#797). The x86-64 backend encoded (add Xd Xn (imm N)) and its sub as
the two-address form on Xd and never read Xn, so with the two different it
computed Xd plus N where ARM64 computes Xn plus N. It now moves Xn into Xd
first, as the register form does. asm-three-address.spec.md runs the form
on both backends.
Floats no longer need the engine’s ffi-call (#796). x-engine-c is
removing (ffi call): the engine does no floating point, and float work is
the language’s. lib/x/num/float.x now emits each double operation as a small
machine-code stub through the library’s own assembler and calls it with
(ptr call), which passes a double’s bit pattern in a general register as
it passes any integer. A stub moves the bits into the FP registers,
operates, and moves the result back: arithmetic, the two comparisons, the
int casts, strtod, and every libm function, whose stub calls the
dlsym‘d address. The stubs are process-local, so they live in the same
cells the libm pointers did, emptied for a state image and remade when one
loads. libm-fn keeps its kinds, "d->d", "dd->d" and "ptr".
Printing (%.15g, which ffi-call did with sprintf) is x: exact digits
from bigint, rounded to fifteen significant digits, ties to even. The
assembler gains the scalar double family on both backends – fmov/d,
fmov/x, fadd, fsub, fmul, fdiv, scvtf, fcvtzs, flt, feq,
with d0-d7 naming arm64’s d registers and x86-64’s xmm registers.
(Ffi call), which wrapped ffi-call and had no callers, is removed.
Works on the pinned engine, v0.2.13, which still has ffi-call; dropping
ffi/call from tools/contract/features.x waits for the engine release
without it.
The kit’s lint reads a scoped bundle’s imports (#795). The linter lints
a bundle’s directory of modules as one batch, with a preload that imported
each sibling bare and ran the imports of only the first file. A scoped
module answers a bare import with its classes alone, so every name a scoped
file imported from a sibling read as Undefined – x-cc’s 53, once its parts
became modules. The preload now runs every file’s own imports, read whole
though they run over several lines, and is the same whichever file it is
computed from. check-lang-kit-lint holds it on a two-module fixture, and
holds that a name nothing defines is still reported.
The lang kit ships whole (#794). make install copied three of the
kit’s files by name and not gen-harness.sh, so the 0.15.0 release, which
is the install tree, carried no harness generator, and a bundle reaching
the kit through x --share-dir could not generate its spec harness on an
installed x. The install now copies every file in tools/lang-kit/,
diffing each, so a file added to the kit ships without a line of its own.
The base types have a public door: (Type named INTEGER) (#719,
step 4). A lookup by registered name through the type registry, it answers
the handle (Type of) answers for a value of that type, for any type
registered at the time, and nil for a name nothing carries; the set of types
is open, so the door is a lookup, not a list of statics (a first cut named
one static per base type and was withdrawn the same day). The name is
written bare or quoted, (Type named INTEGER) or (Type named "INTEGER"),
and any other form is evaluated and must answer a symbol or a string; the
static is an operative for that, added through the class’s open door. The
linter learns the general rule – an operative static’s arguments are not
references – through a new accessor, class-static-ref, which answers a
class’s static method as defined where method-ref answers it bound. Until
now the only spelling of a base type’s handle was a private cache in
type/convert.x – %int, %string, %symbol, %char, %ptr, %pair –
read by fourteen library files, the tools, the specs and ten lang bundles.
The twelve scoped readers fetch from the door once at load ((def %string
(Type named STRING))) and the specs write it out ((Convert to 42 (Type
named STRING))); nineteen private reads leave the scan (705 to 686). The
private handles stay in convert.x, and so do the reads of them by the
unscoped files still at their %-budget and by the bundles, until a release
carries the door and the bundles’ requires-release pins move; the file
takes its header when the last bundle has. docs/namespaces.md records the
plan.
No provide list names a %-private, and the gate that keeps it so
(#719, step 4, three decisions of 2026-09-24). type/class.x exported the
last two %-named names in the library, the hooks that give a type’s values
method dispatch: they are class-call-handler and bind-call-over! now,
bare, documented, and named that way in the eleven and five files that
install them, in the specs that mention them and in the contributing guide.
A new gate, make check-provide-names, reads every provide form under
lib/ and refuses an export that starts with %; it runs in the fast gates,
so the pre-push sees it. The other decision holds still: the boot-layer
walkers of core/list.x and core/alist.x (%fold, %map, %reverse,
%length, %append, %assoc-get, %assq and their kin) are a sanctioned
shared vocabulary, the private layer the List and Assoc classes stand
on that some forty files read at the root for speed; docs/namespaces.md
and the private-reads contract record that their rows hold and are not
doors owed. The third decision was to measure core/arithmetic.x before
scoping it, with the lookup counter the six hot modules were measured with:
a module header adds 3.6% to the environment comparisons of an x-core boot
and 4.3 times to those of a 200,000-step loop of =, - and +, 124 more
bindings compared per wrapper call, with evaluations and allocations
unchanged. It stays unscoped with the six; docs/namespaces.md has the
numbers.
The number modules, type/hash, core/math and the coverage report fetch
the integer primitives themselves (#719, step 4). core/arithmetic.x
saves the binary C operators as %int+, %int-, %int*, %int/,
%int%, %int< and %int= before it wraps the bare names, and seven
library files and one tool read those saved names across the file
boundary; the catalog files the same C functions under int, so each
reader fetches its own, (prim-ref (lit int) (lit +)), as the image walk
already did. %int-number?, the machine-INT test as number? was before
float widened it, has no catalog entry; the four readers that need it test
the INT type directly. Thirty-six private reads leave the scan (754 to 718),
and core/arithmetic.x has no reader outside itself. It stays unscoped for
now: its operator wrappers are the most called functions in the library, and
a module frame would put its scan in front of every name their bodies
resolve, which is the cost the six hot modules were measured for before
staying put. docs/namespaces.md records that the seven boot files loading
before boot/module.x cannot be modules at all.
The JIT compiler, its byte cache and the compiled SHA-256 engine are
modules of their own (#719, step 4, the JIT batch). x/tool/asm-compile,
x/tool/asm-cache and x/codec/sha256-jit carry (module ...) headers,
which hides 181 private names; nothing outside them read one except two
specs, which reach the cache’s five functions and the engine’s two
expressions through the modules’ environments now. Their two %-named
exports are bare: asm-compile-fresh (the uncached compile the cache falls
back to) and sha-jit-make, which the codec reaches through a catalog
entry, (prim-ref (lit sha256) (lit jit-make)), since it loads the engine
inside the function that builds it and the linter cannot see a name
imported there. The compiler’s image transients are one thunk in place of
symbols, as float’s became in #783: the writer clears a symbol in the
child’s root, and the JIT library handle and the trampoline addresses live
in the module’s frame. The assembler’s two architecture backends stay
unscoped, and docs/namespaces.md says why: their bare names are one
interface with two implementations, chosen by asm.x at load, which a
reader cannot import from. Two %-named exports remain in the library, both
in type/class.x.
apply applies a wrapped combiner as the combiner it wraps.
(apply (wrap c) vals) answered nil. A wrapped combiner is a procedure whose
parameters and body are nil and whose environment slot holds c, and the
engine’s apply bound it like a closure with nothing to run. The library’s
apply now reads the wrap flag and applies c to the values, as the direct
call ((wrap c) ...) hands c the values it evaluated, and a call handler
that is a wrapped combiner is applied the same way. One call, %apply-label (released as %apply-kind),
now tells the door what it was handed, with type tests and a flag read
fetched into its own closure, in place of procedure? and operative?.
x/num/float exports its libm door as libm-fn. Since the float file
became a module, %libm-fn – the door that binds a libm function as a row a
state image re-resolves – has lived in its frame, unexported, and a bundle
that binds a libm function the float file does not could no longer reach it:
x-python binds erf, tgamma and the hyperbolics through it, and its load
died on Unbound SYMBOL '%libm-fn' against main. The module exports the same
door as libm-fn, reached by a selective import. Its rows are the file’s own
rows, cleared by its image thunk and remade by its recache hook, so libm is
still opened once.
The pin specs no longer share a vendor overlay. The closure file and
the lockfile file both vendored into build/pin-spec/out, and the runner
has the two in flight together, so a verify in the lockfile file could count
the tree the closure file was rewriting; it did once, on the macOS leg of
#784’s run (one file of five). The lockfile file vendors into lout now,
and the shared fixture writes a file only when its bytes differ, since the
six files build the fixture together as they start.
x/num/rational, x/num/complex and x/num/decimal are modules of
their own (#719, step 4, the second batch of the number modules; the first
is below). Their forty-two cross-file names – the tower’s twenty-five
operations and handles, the eighteen analyser states the compiled tower
rebinds or reads, %rational, %rat?, %make-rational and
%make-complex – are bare exports now. num/tower.x takes the operations
by selective import at its top, as it does bigint’s and float’s;
boot/tower-compiled.x reaches each module’s states through its frame, the
way float’s are reached, with one %tower-state helper for the four modules
and its nine interpreted twins gone (its %-budget row falls 53 to 47);
complex imports rational’s handle and predicate; decimal reads
make-complex from the module when its pact with complex fires, so loading
decimal alone still loads no complex. complex? is a wrapper over number?
rather than a copy of it, which retires decimal’s re-pointing of the alias
after its load. Rational, Complex and Decimal gain a type static like
Float and Bigint, and Rational a make, which is what the specs used
%rational and %make-rational for; the tower-states spec asks each site
whether the module’s binding is still the interpreted twin. Forty-six
private reads leave the scan (801 to 755) and the three files’ %-budget
rows retire. Every number module is scoped now.
apply takes a value’s call handler, and refuses a value without one.
The engine’s apply calls through whatever it is handed, and handed a value
that is not a closure, an operative or a primitive it jumps into the value’s
first slot: a vector, a list, a string, a class instance, a generic, a number,
or a make-type instance with a call handler each took the process down
with a bus error, past the reach of any guard, while the direct call
(v args...) dispatched through the type’s handler. The library now binds
apply over the engine’s, in x/core/fn: a value applies through its type’s
call handler, so (apply v vals) and (v ...) reach the same handler, and a
handler that evaluates its operands is handed each value quoted, so a symbol or
a list among the values arrives as itself. A value with no handler raises a
type error, as does an apply with no argument list. The engine’s apply
stays under %apply for what the library built or reads from a type cell:
let, the class dispatcher, the printer and the reflect layer keep it, so
they pay for no check they have no use for and a lang that binds apply over
the library’s door cannot retarget them. The image writer names that primitive
as it names include’s, and the isa spec’s library declares %apply as an
alias of the bare name. The engine is unchanged: a C primitive jumps where it
is told, and the guard is the language’s. x-python’s
%py-apply-any had grown an arm per callable type to keep clear of this;
those arms can go once the bundle pins this version.
x/num/bigint and x/num/float are modules of their own (#719,
step 4, the first batch of the number modules). Their operations were
%-named root globals – %big-add, %f-add, %ensure-float,
%float-type and forty more – read by the tower’s dispatcher, by the
compiled tower’s JIT sites, by the other three number modules, by the json
codec and by the specs. They are bare exports now (big-add, f-add,
ensure-float, float-type, …) and each reader imports what it uses.
num/tower.x, which stays unscoped as a hot path, takes the eighteen
operations it wires by a selective import at its top, so they are root
bindings by import, listed name by name; docs/namespaces.md records that
door under “import binds”. boot/tower-compiled.x imports bigint’s five
analyser states, which it reads and never rebinds, and reads float’s four
through the module’s frame, since it rebinds three of them with their
compiles: a JIT site now records the environment its name is set in, and
%tower-jit-global! takes that environment as an optional sixth argument
(the module’s at the three float sites, the caller’s own otherwise), so
taking a site down and up sets the module’s binding rather than a root copy.
num/rational, num/complex and num/decimal, unscoped until the second
batch, import the handles and operations they read; codec/json reads the
float type through the module, as does float’s one read of bigint-base
inside its pact callback. Float and Bigint gain a type static that
returns the type handle, which is what the specs used %float and %bigint
for ((Convert to 42 (Float type))). real? moves from float to
core/predicates, beside number?: complex and decimal extend it in the
root with set!, and a definer inside a module frame keeps calling its own
copy (Float real? did, and answered #t for a complex), so a root name
that other files extend in place is defined in the root. Helium binds
real? now. Fifty private reads leave
the scan (851 to 801), the two files’ %-budget rows retire, and
boot/tower-compiled.x’s row falls 55 to 53: the four interpreted twins it
named are the module’s own bindings now.
Rationals and decimals are right at the most negative integer. #748 gave
the bigint routes to LONG_MIN a promotion, but rational.x and decimal.x
still negated plain ints with the raw %int-, which hands LONG_MIN back
unchanged. A rational with LONG_MIN as divisor or denominator came out with
the wrong sign and a negative denominator – (/ 1 LONG_MIN) was
-1/-9223372036854775808 – and a decimal significand of LONG_MIN printed as
--9223372036854775808d and divided by -1 to itself. (/ LONG_MIN -1) reached
the raw C division, which answers LONG_MIN on arm64 and traps on x86: under
the tower / is rational.x’s, so #748’s guard in bigint.x’s / is not the
one that runs. The magnitudes and negations in both files now take the public
promoting -, as their cross products and significand arithmetic already did,
and a divisor of -1 is negation. A negative magnitude costs about 90 objects
more; positive values cost what they did.
The six hot core modules stay unscoped, by decision (#719, step 4).
core/boolean, core/control, core/syntax, core/predicates, sys/pact
and num/tower are the last files the scan calls ready, and their operatives
are looked up from everywhere: measured with the lookup counter of
x-engine-c#59, a scope on boolean and control alone adds 34-42% to the
environment comparisons of a boot, with evaluations unchanged. Their 44
private names are read by nothing outside them, so a scope would buy only the
frame. docs/namespaces.md records the numbers under “What to measure first”.
The quote and quasiquote readers are modules of their own, and so is the
xon codec (#719, step 4, the reader batch). x/reader/lit-reader and
x/reader/quasi-reader exported their analysers, readers and states under
% names – %lit-analyse, %quasi-read, %unquote-after-comma and seven
more – which the tower, the formatter and lit-reader itself read from the
root. They are bare exports now (lit-analyse, quasi-read,
unquote-after-comma, …), plus interp-analyse and interp-after-hash
for the #"..." literal, and every reader imports what it uses:
lit-reader seats quasi-reader’s four handlers, boot/tower-compiled.x imports
the nine entry tests and states it compiles and swaps by identity,
tools/dev/fmt.x imports the seven it arms on its scratch base, and
x/codec/xon – a module of its own now, with its own fetches of the type
and buffer prims – imports the literal’s entry analyser. The tower-jit spec
reaches macro-delimit through the module. Thirty private names leave the
root; two %-budget rows are retired (21 and 9) and five private-read rows
go down by thirty-nine between them.
The compiler is three modules of its own: x/tool/compile,
x/tool/compile/emit and x/tool/compile/pipeline (#719, step 4).
Ninety-two private names leave the root. The three shared what they needed
by reading each other’s root names; they import it now: emit exports the C
generator’s helpers, pipeline the generation stages and the writer brackets
it used to keep, and compile re-exports the public face. Three of emit’s
names were variables the driver, the pipeline and x/tool/asm-compile
assigned around a compile – the free-variable alist, the parameter list and
the nested-function holder – and an import copies a value, so those are
reached through accessors (compile-fvars, compile-fvars-set!,
compile-fvar-lookup, compile-params-set!, compile-fns,
compile-fns-set!). For the same reason compile-emitters is a function
answering the emitter table, and compile-add-emitter! grows the table it
answers.
compile-asm is one function. x/tool/compile keeps the door that
loads the assembler on first use; x/tool/asm-cache used to redefine the
same name at the root behind it, which left a session holding two objects
under one name once the toolchain had loaded and refused any later import
of it. asm-cache’s function is asm-compile-cached now, beside
asm-compile’s %asm-compile-fresh, and the door dispatches to it.compile-hosted? is an export: the tower’s JIT probe imports it, with
compile and compile-asm, instead of reading a private. x/codec/sha256-jit
imports compile-asm, x/tool/asm-compile imports the two accessors it
needs, and the compile spec’s fixture imports the API it exercises.tools/contract/requires.x records that the pipeline needs isa/ffi-call
in its own right, for the dlsym that patches the fvar table. Three
%-budget rows are retired (57, 25 and 10) and five private-read rows go
down by thirty-one between them. The type-system guide’s analyser example
passes the free variables as compile’s argument, which is what the API
always took, rather than setting the compiler’s state.A closed tower probe leaves a tower that reads. The symbol type’s delimiter
hook took the full ladder, so with the probe closed on an engine that ships its
C headers it went to the cc rung, which compiled it without refusing into a
hook that misread the source read after it, and the x-base boot failed
(Unbound SYMBOL '34, or Str8 append: not a string, depending on what was
read next). The hook calls the asm lane’s trampolines, %buffer-last-char and
%buffer-unread, like the states on %tower-asm-only, and now compiles the way
they do: the asm lane or its interpreted twin.
tests/x/specs/e2e/tower-fallback.spec.md takes the path a state image takes on
a host whose lane refuses – every site down, the probe closed, every site back
up through its maker – and requires the tower that results to read a sample of
every tower literal as the compiled one does.
The symbol type’s delimiter hook compiles on every boot that can.
%c-macro-delimit compiled only where %jit-buffer-last-char was bound, and
that is asm-compile.x’s name, bound once a byte-cache miss has loaded the
compiler. A boot whose compiles all hit the cache never loads it, so after a
machine’s first boot per version the symbol type kept the interpreted
%macro-delimit; CI, which starts cold, compiled it. The hook now asks the
engine whether it exports jit_buffer_last_char, with dlsym on the process,
the way the lane resolves its trampolines. tests/x/specs/e2e/tower-jit.spec.md
requires it compiled wherever the probe is open and the trampoline is
exported, and now checks the swap sites with the others.
Three scoped modules fetch their own engine primitives (#719, step 4).
x/protocol/str/str8 called the raw integer, pointer and memory primitives
through boot/string.x’s caches (%sc-int+, %sc-int<, %n2s-int-,
%mem-cmp, …), x/repl/line printed and collected through repl/loop.x’s
(%repl-write-to-str, %repl-collect), and x/doc/doc-gen interned symbols
through convert.x’s %str->symbol: ten reads of another file’s private
name, each a cache of a catalog entry any file may fetch. Each module fetches
its own now, under its own names, in its own frame – the rule since #755 for
an engine primitive, which needs no door. Three private-read rows go down by
ten between them, and the engine-contract manifest records that x/repl/line
needs isa/gc in its own right: it is the turn loop when the line editor is
installed, and sweeps at the top of each turn as repl/loop does.
A bigint product walks the shorter operand. %limb-mul, the schoolbook
multiply under every bigint product, walks the limbs of its first operand and
adds each shifted partial product into the whole accumulator, so its cost
grows with the square of that operand’s length and linearly with the other’s.
%big-mul passed the operands in the order it was given them, and now passes
the shorter limb list first; the product does not depend on the order. Both
orders cost what the cheaper one did: (* (Num expt 2 990) 3) allocates 22K
objects where it allocated 216K, and a 34-limb bigint times an 11-limb one
174K where it allocated 342K. Equal lengths keep their order, and a square
pays only for the length test, under 1%. Num expt squares factors of the
same length, and its odd steps already put the smaller factor first; its last
step multiplies the fully squared base by 1, and (Num expt 2 990) allocates
409K objects where it allocated 461K. The bigint spec pins both orders for a
one-limb int, a negative int, two bigints of different lengths, and a product
that demotes to a native int.
x/type/struct has a scope of its own (#719, step 4). The type
mechanism’s 48 helpers were %-private names in the root that the file filed
in the catalog under ns type for consumers to fetch at load; with the scope
the catalog is the one door, as the file’s header always said. Three names
were reached from the root instead of through it: %type-offset by
x/tool/compile, %type-units-cell by the image tools and a spec, and
%type-alist by two specs. The first two are filed now, (type offset) and
(type units-cell), and every reader fetches from the catalog; (type alist)
was there already.
%-budget row goes up for the fetches: x/tool/compile fetches the
offset in the three functions that use it, the two image tools that read a
type’s units once per type do so through the public door,
(Type cell ts (lit type-units)), and the image walk, which reads them per
object, keeps the fetched prim in a closure. The file’s own row of 48 is
retired, and five private-read rows go down by one.The containers that compare by content import equal? by name
(#719, step 4). x/type/dict, x/type/list, x/type/record and
x/type/assoc called %equal?, a second name for equal? that
x/core/logic defined so that a session rebinding the bare global (x-sweet’s
Scheme shim does) would not retarget a bucket search or index-of. A
selective import is that protection without the second name: each of the
four has (import x/core/logic equal?) at its top, holds the closure in its
own frame and calls it as equal?, and a rebinding of the global does not
reach it. %equal? is gone.
x/type/vector installs its elementwise handler on equal?’s extension
hook, which it read from the root as %equal-others. The cell is in the
catalog now, (prim-ref (lit logic) (lit equal-others)), and vector fetches
it there.x/core/logic’s
%-budget row goes from 2 to 1.The tower’s JIT probe is a state in the form its states take.
%tower-jit? decided the whole compiled burst by compiling (fn (_ x) (+ x k))
as an integer function, a mode none of the tower’s states use, so it could open
or close independently of them. It is now an analyser state – a loop through
the self param and a handoff through an fvar, declared #t – at boot and again
in %tower-rejit!, the probe x-python’s tokenizer uses
(x-python#138). The four
states that carried an unused _u fvar to force analyser mode pass an empty
table, their mode being declared. A closed probe or a state that kept its
interpreted twin was silent, since a twin answers what its compiled state
answers: tests/x/specs/e2e/tower-jit.spec.md asks the lane directly whether it
compiles an analyser, and where it does requires the probe to be open and every
global and push site to hold compiled code.
x/codec/utf8, x/tool/fmt and x/doc/doc-gen have scopes of their
own (#719, step 4), hiding sixty-one private names between them, and the
seven names other files read from two of them become exports those files
import. The UTF-8 codec’s five functions – utf8-decode, utf8-encode,
utf8-width, utf8-seq-len, utf8-cp-at – were private names that the two
string layers, x/protocol/str/utf8 and x/type/str-utf8, read from the
root; both decode inside tokenizer callbacks, where a class call is not
allowed, so each imports the ones it uses by name and calls them directly,
at no new cost. doc-build-lookup and doc-walk-with-prims are the
generator’s two exports for its driver, tools/dev/doc.x, which imports
them; the doc-gen spec reaches the one internal it exercises through the
module’s environment. x/tool/fmt needed nothing: its only outside mention
was prose. The private-read rows for the two string layers and the doc
driver go down, and the three files’ %-budget rows are retired.
Tab inserts a tab where there is nothing to complete. With only
whitespace before the cursor and no candidate to fill in, Tab puts a tab in
the line, so an indented block in a lang that reads indentation, x-python’s
for one, is typed at the prompt as it is in a file. ctrl-v is readline’s
quoted-insert: the key after it goes into the line as typed, so ctrl-v Tab
inserts a tab after text. The editor draws a tab to the next multiple of
eight columns, and measures the cursor and the scrolled window with the
column step x/reader/indent measures a line by.
x/platform/syscall has a scope of its own, and its open-flag table is an
export its readers import (#719, step 4). The per-OS O_* table was
%file-modes, a private name three other files read from the root. It is
now file-modes, a plain export: x/sys/file and x/sys/posix import it at
their top, (import x/platform/syscall file-modes), and the boot loader’s
%module-mode imports it when it runs, since boot/module.x loads long
before the table does. (File file-modes) answers as before.
os-linux?, arch-arm64? and arch-x86-64? without
providing them, and other files use all three. They are in the provide
list now, marked (global ...) with the file’s other sanctioned names.(import NAME), so (import NAME sym ...) in an amalgamated file
would have been left unspliced and its module loaded from the platform at
boot. It splices the module as for a bare import – ahead of a scoped
importer, in place in an unscoped one – and keeps the line, because binding
the names is the line’s other half and the load is a no-op once the splice
has marked the module loaded. The three library amalgams were byte-identical
before and after the generator change; the amalgam smoke check covers both
placements on a throwaway tree.check-bare-globals counts a scoped module’s marked names only. A
scoped module’s other bare definitions are its own, exported for import or
not, and need no manifest row; file-modes is the first such name in the
files it scans. A scoped file that defines a sanctioned name and stops
exporting it is refused, which is the mistake the three predicates above
would have been.%-budget row is retired.A compile that carries fvars declares its calling world. With no third
argument compile-asm read the fvar table to choose between an integer
function and an analyse callback – present meant analyser – though an fvar
also names a callee the body calls (#603), so the table cannot say which. The
reading stayed as a default for x-python’s tokenizer, whose compiles were
undeclared and adopted under a guard that would have hidden a refusal. That
bundle now declares every compile
(x-python#144), and its specs
require the JIT to be active wherever the lane compiles an analyser
(x-python#138), so the door
stops guessing: fvars with no third argument refuse, naming both declarations.
A compile with no fvars is an integer function, as before.
x/type/vector, x/protocol/str/str8 and x/type/type have scopes of
their own (#719, step 4), and two doors replace the five private names
other files read from the first two. (Convert to-int v what) is a value
as an INT, converting through the registry and raising what when there is
no conversion: the coercion x/type/array, x/type/gen and x/protocol/str/utf8
had each borrowed from a neighbour’s private (%vec->int, %str8->int).
The hot callers test for an INT themselves and reach the door only to
convert. (Str8 %check v what) is the string check StrUtf8’s own paths
used through str8’s private, homed on the class as (Iter %check) is.
x/doc/doc-gen renders with (Str str ...), which is what str8’s
%str-build did. The type-convert spec asks (Type of #(0)) for the vector
type instead of vector’s private handle. Four rows of the private-read
budget go down: array to none, utf8 by two, gen and doc-gen by one each.
Cross-file reads of private names may only decrease (#719). Until a
module is scoped, its % names are globals, and other files read them:
127 files read 954 such names today, most of them the number tower reading
its neighbours, the compiler’s stages resolving free names in the root, and
the dev tools driving internals. Step 4 turns each read into a door, and
check-private-reads keeps the count from growing meanwhile: for every
reader file, tools/contract/private-reads.x budgets the distinct %
names it reads that another unscoped file defines at its top level and it
does not define itself. A file over its budget fails and names the reads;
a file under it fails until the row is lowered. It runs with the fast
gates, so a pull request sees it.
Bracket colours carry across the lines of a multi-line entry. Each line of
an entry was marked and painted on its own, so a continuation line started
again at depth 0, a close paren that closed a paren on an earlier line showed
bold red as unmatched, and a line continuing a string was painted as though a
new string began at its closing quote. Line read takes the lines already
entered as an optional second argument, and the REPL’s continuation read
passes them. The marker is asked about the whole entry and the editor keeps the
marks that fall on the line being edited; the painter is handed the text that
comes before what it paints, the earlier lines and any part of the line
scrolled out of view, and starts from whatever string or comment that text
leaves open. The painter’s cache compares that text by identity, so moving the
cursor still repaints nothing. %repl-marks now receives the whole entry, and
%repl-paint a third argument, which a painter written for fewer ignores.
A session started without -l has the line editor. lib/x.x, the
default entry, boots helium, but its launcher lacked the
(import x/repl/line) that lib/he.x, lib/xe.x and lib/rn.x carry, so
a bare sh x.sh read lines through the plain loop, without the editor’s
keys, history, Tab or colour. Its launcher is now the same as theirs.
A scoped module’s plain exports stay its own (#719). provide
records every export in the registry, where a selective import finds it,
and now binds in the root only what docs/namespaces.md says it should: a
class, and a name the provide list marks (global NAME). The marks are for
the sanctioned bare set, and check-bare-globals holds them to it both
ways. Any other export of a scoped module is reached with
(import MODULE NAME), which copies it into the importer’s own frame.
quasi, iter, def-record, def-trait, trait?, def-generic, on,
throws? and raised.x/tool/cov’s and x/tool/profile’s functions are imported by the tool
script and the specs that call them, and x/reader/indent stops
exporting its four catalog functions.check-dup-defs counts,
for a scoped module, only what reaches the root.x/sys/posix has a scope of its own (#719, step 4), hiding
thirty-two names. Three other files read two of them. x/repl/term and
x/sys/socket sign-folded their libc int returns with %sys-fold; they now
call (Sys %sign-fold r), a static on the file’s own class, one class call
beside each syscall. x/tool/compile fetched libc’s unlink with %resolve
to remove its generated C source; it now calls (File unlink), inside a
guard because File raises on a failure the raw call ignored.
The arithmetic refusal costs a boot nothing again. (Type
refuse-arithmetic!), added above, installed its thirty handlers through
(Type push-op) and (List for-each), a class call per operator, and that
added 0.7% to the evaluations of every x-core boot. It now calls the push-op
primitive itself in a plain loop, and an x-core boot takes 22,437,175
evaluations, fewer than the 22,451,999 it took before the door existed,
counted with the engine’s profiling build. The messages are unchanged.
The %-budget and the duplicate-definition gate look only at what stays
global (#719, step 5). A scoped module’s top-level definitions bind in
its own environment, so its %-names never reach the global tree.
check-percent-globals no longer counts a file whose first form is
(module NAME), and the thirty-seven rows such files held, 427 names, are
retired. check-dup-defs checks a scoped module for the names it provides
and nothing else: two scoped modules may share a private name, and a scoped
module may reuse a global’s, while a provided name that an unscoped file
also defines is still refused. The third part of the step needed no change:
lint already reports another module’s private name as undefined for every
scoped module, because that module’s environment is not the linter’s.
x/type/str-utf8 has a scope of its own, and its list conversions are
catalog doors (#719, step 4). The file loads before the object system,
so there is no class to carry what other files need from it. It publishes
its two list conversions beside (str cp-len) instead: (str from-list)
builds a UTF-8 string from a list of code-point characters, and
(str ->list) decodes a string into one. x/type/char-io, x/type/convert
and x/type/iter each fetch the one they use once, at load, where they used
to read the file’s private names. With that, the file carries the
(module NAME) header and hides fourteen names.
A non-numeric type refuses arithmetic through one door. (Type
refuse-arithmetic! ts tname) registers a raising handler for each of
+ - * / % < on a type struct, so (+ 1 "a") answers “no + for STRING”
rather than falling through to integer arithmetic on the string’s pointer
(#52’s refusal, now a method on the class that owns the type registry).
x/core/op-guard calls it for strings, lists, pairs and vectors, and
x/type/bool for BOOL. Before this, bool installed its refusals with three
private names of op-guard’s; op-guard now defines no global at all, and a
lang that defines a type of its own can call the same door (#719, step 4).
Six more modules have a scope of their own (#757). x/reader/analyser,
x/tool/cov, x/reader/indent, x/type/list, x/type/char and
x/type/ptr carry the (module NAME) header (#719, step 4), hiding
forty-one private names between them. None needed a door: nothing outside
them reads a name of theirs, apart from per-file catalog aliases that an
unscoped file still binds in the root, and every public name other code
uses was already provided. x/tool/cov walks whatever environment its
caller hands it, so what it reports is unchanged. Booting costs 0.02% to
0.05% more evaluations, counted rather than timed: 22,428,516 to
22,440,486 for the x-core amalgam, 60,240,457 to 60,252,679 for x-base,
and 22,624,448 to 22,634,898 from source.
The assembler loads and stores at every width (#756). x/tool/asm
had eight-byte ldr/str and one-byte ldrb/strb; it now has the
widths between, on both backends: ldrh, ldrsh and strh for two
bytes, ldrw, ldrsw and strw for four, and ldrsb for a byte
sign-extended. The names are arm64’s own, except the four-byte pair,
which arm64 writes as LDR and STR of a W register: the portable operands
are X registers only, so the width goes in the name, as it does for a
byte. A load fills all 64 bits of its register, and a store writes its
width and nothing past it. Every one is a table entry, and
asm-widths.spec.md runs on both hosts.
Four more modules have a scope of their own (#755). x/type/iter,
x/sys/stream, x/sys/socket and x/sys/file carry the (module NAME)
header (#719, step 4), hiding sixty-two private names between them.
Three needed nothing: nothing outside them read a name of theirs. The
fourth did. %as-list in x/core/list drained an iterator through
x/type/iter’s %i-empty? and %i-next, and both of those are fetches of
engine primitives, so list.x fetches them itself rather than reading
another module’s names; the duplicate-definition gate allows several files
fetching one catalog entry, and the budget for core/list.x goes from 21
%-globals to 23. In the same spirit the Ansi class carries inverse
beside bold and dim, and x/repl/paint takes the cursor-pair colour
from the class, as it already took every other colour. Boot time is
unchanged: best of ten alternating runs against the base commit.
The r5rs three are walked to a release (#754). The note this file and
tools/contract/langs.x carried named x-engine-c v0.2.11 for “an
environment is a value”, taken from the entry below, which names the pin
#720 ends on rather than the release the feature shipped in. The engine’s
own changelog puts it in 0.2.10 (x-engine-c#49, closing #46); 0.2.11 is the
follow-up that makes a name found by identity (#52). Both rows now say
0.2.10, and both carry the walk that settles it – x-r5rs held at dd3ba79,
each x-lang commit built against the engine its own pin names:
x-lang pin result what it is
v0.14.0 v0.2.8 667/0 the last green pairing
18f0b530^ v0.2.8 667/667 source already migrated, pin behind
18f0b530 v0.2.10 667/3 the pin that takes the new env model
edaabc2a v0.2.11 667/3
d6fdd68d v0.2.12 667/3
ef0cc505 v0.2.13 667/3 where this tree sits
Nothing since v0.2.10 has moved the number in either direction. The two 667/667 rows are the limit of the method and are recorded as such: v0.14.0’s source does not run on v0.2.10 and the migrated source does not run on v0.2.8, so the engine release and this tree’s adaptation to it land together and pinning cannot separate them. This names a release, not yet a line.
Two lang budgets ratchet to zero, and the r5rs three are ours (#752).
tools/contract/langs.x recorded one failure for x-python and two for
x-ash; both bundles have fixed theirs, and the rows now say 0. The test
counts are left where they are. They are floors, not censuses, and the file
says why: a floor set to a moving high-water mark cries wolf.
check-pin’s drift marker is exported from the module ([#749]). The
gate’s drift case appends a marker to a vendored copy of type/dict.x and
read it at the top level, which a scoped module does not expose; the marker
is provided beside its definition.
x-cc’s row is unchanged too, and the reason is worth recording. The gate
reported its suite had shrunk 119 -> 117 and asked for a re-record. It had
not shrunk. check-langs reads ../languages, and every one of the
thirteen checkouts there was behind its remote – x-coreutils by 30 commits,
x-cc by 24, and five of them parked on a shared feature branch rather than
main. Measured against fresh checkouts at each bundle’s origin/main,
x-cc runs 153 specs, x-coreutils 444 and x-ash 974. Re-recording 117 would
have written a stale working copy into a contract that may only shrink.
x-r5rs is red at 667/3 and the budget stays 0, because the three are this
platform’s. They are the R5RS pitfall 3.2 cases, which assert that a
definition a macro introduces does not escape the let it was written in.
x-r5rs fixed exactly those in its own repository on 2026-09-08 and measured
667/0 against x-lang main on x-engine-c v0.2.8, the engine v0.14.0 pins.
This tree pins v0.2.13, and the three arrive with “an environment is a
value” – x-engine-c#49, released in v0.2.10 and carried here by #720: a
def binds in the current environment, an eval with an environment makes
it current with the binding staying put, and the frame marks, shadow list
and local boundary that those fixes were written against were retired with
it. x-r7rs gained the same three in the same window. A budget raised to 3
would record our own regression as the bundle’s.
check-langs has never run in CI and does not now: make gates runs on
main only, a CI checkout has no ../languages, and the gate prints SKIPPED
and exits 0. It is a local gate, and these numbers are only ever as good as
the checkouts beside the tree.
The numeric tower is right at the most negative integer (#748).
LONG_MIN is the one int whose negation is not an int, and every promotion
route reached it through its own magnitude. %bigint-from-int took (- 0 n)
first, which for LONG_MIN hands back LONG_MIN; the limb loop then divided
a negative number, whose remainders are negative too, so each limb came out
carrying the sign. The bigint that built printed as -9-223372036-854775808
and did arithmetic to match: (+ LONG_MIN 18446744073709551616) answered
n itself and are negated one
at a time, which is always in range because a limb is smaller than the base.%would-overflow-mul? compared magnitudes through the same wrapping abs, and
read LONG_MIN as negative on both sides. On the left that only promoted
products that did not need it, and they demoted again; on the right a
negative operand never exceeds a bound, so the check passed and the product
wrapped – (* 2 LONG_MIN) was 0. It now divides the bound the product would
cross by b, after answering the two cases that no division can: -1 is the
one multiplier that overflows LONG_MIN, and LONG_MIN / -1 the one
division that would overflow.
Unary - took a fast path that negated a plain int in place, which left
(- LONG_MIN) as LONG_MIN; it now goes through the same binary the
variadic fold uses, so it takes that binary’s overflow check. Binary / had
no overflow check at all, and LONG_MIN / -1 is undefined in C – the prim
under it evaluates a plain a / b, which answers LONG_MIN on arm64 and
traps on x86. That pair now promotes like any other overflow. Num abs and
Num negate are right as a consequence: both are (- 0 n).
(% LONG_MIN -1) is left alone. It answers 0 on arm64, and % reaches the
prim with no wrapper on purpose; the undefined division under it is the
engine’s to guard.
Nine more modules have a scope of their own, and amalgams hoist a scoped
file’s imports. Eight boot files, x/type/dict, x/type/bool,
x/type/err-io, x/type/char-io, x/type/hash, x/core/math,
x/protocol/str/utf8 and x/type/regex, and the run-time x/codec/json,
carry the (module NAME) header (#719, step 4). Nothing may be spliced
inside a scoped file in an amalgam, since its header reads every form after
it into the module. The generator used to refuse such a splice. It now
splices the files a scoped file includes once or imports at top level
ahead of it, in order, and the lines inside the module become the usual
“inlined above” comments. That is what lets x/type/dict join: the bitwise
app imports it before x/type/hash is loaded, and hash is now spliced
ahead of dict. A plain include inside a scoped file is still refused, and
the amalgam smoke check covers both cases. A header under a plain include
now takes the module it names, as it already did under include-once: the
tower dialects load x/type/hash and x/type/regex with include from
boot/tower-compiled.x. include takes the module the caller expects as a
second argument, which is how import asks for one, so the mark a header
reads belongs to the one file being loaded and an import’s name is never
asked of the files that file loads in turn. Boot time is unchanged: best of
ten alternating runs against main, the source boot, the x-core amalgam and
the x-base amalgam differ by 0.0%, 0.0% and -0.7%, and loops over dict,
strings, display, Num and regex are within 4% either way.
A lang bundle’s spec harness is generated from the dialect it declares
([#744]). Every bundle had loaded x-base.x whatever its lang.xon said;
the lang kit writes the harness from the declared dialect, so a helium
bundle is tested without a tower it never loads, and a xenon or radon
bundle with its dialect’s library set.
The first boot files have a scope of their own. x/core/quasi,
x/type/record, x/type/err, x/type/assoc and x/type/gen carry the
(module NAME) header (#719, step 4). x-core loads most of them with include-once,
which names no module, so a header under include-once takes the module
it names. In the amalgams a scoped boot file is spliced in place, and the
generator now writes (%module-expecting! (lit NAME)) before it and
(%module-end) after it; the header reads up to that marker, since the end
of the spliced text is not the end of the stream. The generator refuses to
splice another file inside a scoped one, because that file would load into
the module, and the amalgam smoke check proves the refusal on a throwaway
tree. x/type/dict waits: the bitwise app imports it before anything has
loaded x/type/hash, so its amalgam would splice hash inside dict.
x/core/boolean, x/core/control and x/sys/pact wait too: every global
name looked up inside a scoped module’s code checks the module’s
environment first, and those three hold and, or, if, let and the
number tower’s contracts, which run constantly. Scoped, they made the
x-base amalgam boot about 5% slower; the five above cost nothing measurable.
The declaration generator no longer fills the disk on a profile cycle.
tools/contract/gen-engine-xon.sh flattened each profile in work files
named after the profiles, appending one profile’s file to another’s. A row
of %feature-profiles that named its own profile, or two rows that named
each other, could make it append a file to itself; that append never
reaches end of file, so the file grew until the disk was full.
check-engine-contract reports such rows and then runs the generator. Each
profile is now expanded at most once, a row’s own name counts as expanded
from the start, and no work file is named after a profile. A profile that
names one defined after it now takes that profile’s atoms, which were left
out before. The declaration for the committed vocabulary is unchanged.
The engine-contract gate reads the library as forms (#739). The
checks check-engine-contract makes of the library run in x, in
tools/check/engine-contract.x: the partition of the reference ISA into
capability groups, the profiles, the rows of tools/contract/requires.x,
and the parameter values tools/contract/constraints.x binds. The checks
that judge the candidate engine stay in shell, so an engine that cannot run
x is still refused by name, and the gate prints what it printed before.
Deriving requires.x from forms corrects two of its rows:
lib/x/type/err.x needs isa/ffi-call (its errno lookup calls
ffi dlopen, ffi dlsym and ptr call, sites the text pattern did not
resolve), and lib/x/rn.x does not need isa/syscall (the pattern matched
a comment). The declared profile is still posix. The gate needs the built
engine, and its library half takes about 15 seconds where the shell took 3.
A clean answer is kept in build/ under a digest of everything it read, so
check-second-engine, which asks the gate about three more engines, and a
tree that has not changed read it instead of deriving it again.
The tool scripts stop on an interrupt or a TERM. Twenty scripts under
tools/, the conformance runner and the pre-push hook set their cleanup as
a trap on EXIT INT TERM with no exit in it. On INT or TERM the
cleanup ran and the script went on without its scratch files, so an
interrupt ended only the command in progress, and a gate run under timeout
went on into its later checks. Each now sets the cleanup on EXIT alone,
with trap 'exit 130' INT and trap 'exit 143' TERM beside it, so the
cleanup runs once and the script exits with the status a shell reports for
that signal. tools/check/asan-boot.sh runs each boot under a timeout of
its own, which puts itself in a separate process group, so the signal did
not reach a boot in progress and the gate waited for it to finish, minutes
under ASan. It now runs the boot in the background, and its INT and TERM
traps stop the boot’s process group before the gate exits. It still puts
back the JIT byte cache it sets aside, and ignores a further signal while it
does; one that arrived during the restore left the entries not yet moved in
the gate’s work directory. Fourteen more scripts set their cleanup on EXIT
with no trap on INT or TERM. Such a script stops on either signal, but
dash does not run the EXIT trap of a shell a signal ends, so under dash they
left their scratch files behind. They now set the same two traps. x.sh
writes the prefix a state image is built from in a temporary directory, and a
signal during the write, which takes seconds, left that directory behind. It
now sets the same three traps around that section and clears them at its end,
so the boot that follows runs with the signal dispositions the wrapper was
started with.
A wall-time ceiling an interrupt can stop. timeout SECONDS COMMAND
cannot be stopped by the script that runs it: GNU timeout puts itself in a
process group of its own, so a signal sent to the script’s group does not
reach the command, and the script waits for the command before it acts on the
signal. tools/lib/guard.sh runs the timeout in the background and waits for
it, where a trapped signal interrupts the wait, and its INT and TERM traps
pass TERM to the timeout’s process group before exiting. The command’s own
status is passed through, timeout’s 124 for a run that hit the ceiling
included, and with neither timeout nor gtimeout on PATH the command replaces
the guard, where the caller’s signal reaches it directly.
tools/check/pin-smoke.sh, which runs the wrapper 73 times under a
120-second ceiling, and tools/dev/cov-lib.sh, which runs the profile engine
once per spec file under a 60-second one, run theirs through it: a signal now
ends pin-smoke where it stands, 4.0 seconds against 7.0 to 8.2 in the same
runs.
A scoped module’s header reads the rest of its file. The loader used to
read a prefix of every imported file into a string to find out whether it
was scoped, then read a scoped file whole into a string and tokenize all of
it before evaluating any form. Now import loads every file with include
and names the module it is loading, and a file’s (module NAME) header,
finding that name, makes the module’s environment and reads the rest of the
file into it itself, one form at a time, with the reader. Nothing is read
into a string, and no file is read twice. Each form carries the file and
its line, so an error while a scoped module loads now names its file as
well as the line; a form is read after the forms before it have run, so it
can read the forms after it; and a literal () in a module is a form like
any other, since read answers the EOF sentinel at end of input
(x-engine-c v0.2.13). A header naming a module other than the one being
imported is refused, naming both. module.x sheds the peek, the whole-file
read and the tokenizer path: five fewer private definitions, and no use of
the raw file-reading doors.
End of input is a value of its own (x-engine-c v0.2.13, x-sweet
v0.1.6). The engine’s read primitive answered nil at end of input, and
nil is also what a literal () reads as, so a loop that read until nil
stopped at the first () in its input. The pinned engine answers the EOF
sentinel, %token-eof, instead
(x-engine-c#57).
(Io read) still answers () at end of input, so its callers are
unchanged. The library’s own callers of the primitive stop at the sentinel:
the vector reader, and the lint driver, which no longer stops early at a
() among the forms it lints. The sweet bundle moves to v0.1.6, whose
readers stop at either
(x-sweet#15).
The spec runner’s NUL escaper is x, and runs only for a spec that asserts
a zero byte (#733). It was perl, which is not part of this project; it is
now tools/dev/nul-escape.x, run through the wrapper and installed beside the
runner so a bundle’s suite finds it too. It reads the caller’s stdin from fd
3, where the wrapper keeps it while the engine reads its program from fd 0.
The runner also stopped appending an escaper to every batch: the awk runner
reads every spec before it runs any batch, so it knows whether one writes
<<NUL>>, and a suite that measured 162 escaper runs now makes one.
Eighteen more run-time modules have a scope of their own (#719, step
4). x/codec/base64, x/codec/csv, x/codec/zlib, x/doc/emit,
x/doc/emit-man, x/net/http, x/net/rest, x/net/tls, x/repl/edit,
x/sys/opts, x/test/assert, x/tool/contract, x/tool/pin,
x/type/array, x/type/counter, x/type/deque, x/type/path and
x/type/pq carry the (module NAME) header. No file outside any of them
reaches one of their private names, in this tree, the language checkouts or
the released bundles. Four public names that other code already used were
missing from a provide list, and would have become private: DocEmit,
DocMd and DocMan now appear in their modules’ lists, and x/sys/opts,
which had no provide at all, provides Opts. Importing a scoped module is
no slower than importing it unscoped: the same five imports timed in-process
ran between 2 and 10 percent faster with the header.
A reusable CI workflow runs a lang bundle’s suite on every platform x-lang is tested on ([#736]).
The first library modules have a scope of their own. Nine modules
that load through import at run time now carry a (module NAME) header
and keep their private names to themselves: x/tool/highlight,
x/repl/paint, x/repl/line, x/repl/term, x/type/generic,
x/type/trait, x/num/random, x/sys/date and x/tool/profile
(#719, step 4). Their provide lists are now complete, since an unlisted
name is private; Paint and Line gained the provide they never had.
The one seam another module reached through a private name is a door now:
the numeric tower asked generic’s %g-absorbs? at its miss handler, and
asks (Generic absorbs? k1 k2) instead.
The header follows the file’s comment banner rather than preceding it, so
a scoped file opens the way every library file does; the loader’s peek
skips the banner, finishing a long one from the whole file. A test that
deliberately drives a module’s private seam reaches it through the
module’s environment, (eval (lit NAME) (module x/repl/line)), which the
repl specs now do.
A collect inside a guard body no longer frees the enclosing handlers (x-engine-c v0.2.12, engine x-engine-c#54). Installing a guard’s handler took the previous handler out of the error-handler slot and kept it in a C local, so a collect inside the body swept the enclosing guard’s handler and the base-eval handler under it, and the next raise went through freed memory. The handler now carries the handler it displaced. The library never collected inside a guard on its own, so nothing met this until #728 swept after each module load during an image write and the tower’s JIT probe, a guard around a lazy import, put that sweep inside one; the x-base, xe and rn image writers segfaulted on macOS CI. With this engine the same write is clean under ASan. The shortest program that showed it, on the previous engine, was a nested guard whose body collects and then raises to the outer guard.
The bootstrap closure is rescored ([#727], [#729], [#731]).
docs/bootstrap-closure.md reports 37 of 46 rows implemented, splits the
closure’s call sites into those reachable before and after self-hosting,
and re-measures the 45 tools at 20,398 invocations.
x-cc runs 119 specs ([#730]): the contract row for the bundle, with two new cases that call a compiled twin by name.
The engine link is refused when the pin does not name it ([#726]). A
tree on the wrong engine built, booted and passed the suite, and failed
only when it wrote a state image; make engine-link checks the link
against the pin first.
A module can have a scope of its own. A file whose first form is
(module NAME) is evaluated in an environment of its own, a child of the
root: its top-level definitions are private to it, and only what provide
lists reaches the root (#719, design note #718). A private helper stays
in the module, two modules may share a private name, and a name is exported
once — provide refuses a second module that defines a name another owns,
while tolerating a re-export of the same object. A selective (import NAME
sym ...) copies an export into the importer’s own environment, by name or
under an alias, so the importer holds the value and a later global rebind
does not reach it; (module NAME) as an expression denotes the module’s
environment. Scoping is opt-in and per file: a file with no header loads
through include in the root exactly as before, which is the whole boot
floor and standard library, and the loader tells the two apart by the first
bytes so an unscoped module is never re-read. Built on first-class
environments (x-engine-c v0.2.11): a scoped module is an environment, and
import/provide move bindings between it and the root.
The library’s if ladders are match forms ([#707], [#711], [#722],
[#724]). The linter’s ladder rule reads any chain of three or more nested
ifs, whatever their tests compare, and reads a set! body as the
definition it is; the 38 chains it found across lib/ and apps/, the
sixteen the set! rule surfaced and the tower’s five compiled states are
written with match.
-l is repeatable. The first names the lang that owns the prompt, as it
always has; each further one names a bundle to load beside it, resolved
exactly as the first is and read ahead of its entry, so x -l xe -l python
opens at xenon’s prompt with Python registered and (lang python) a call
away. %lang-lead, a new bundle-class seam, carries the first name into
the pipe so a bundle’s entry knows whether to take the prompt or only
register what it is. The state image of such a boot is its own, named for
every lang in it.
The JIT call spec roots its dispatch table’s callables ([#721]). The case stored three compiled closures only as addresses in a scratch buffer, which a collection between the store and the call could free.
An environment is a value (x-engine-c v0.2.11, #720; design note
#718). The pinned engine’s environment is one pair, bindings and parent:
the root’s bindings are a tree, every other environment keeps an alist and
the environment it was made in. A call makes a child, an operative body
runs in a child and receives the caller’s environment as a value, def
binds in the current environment, and eval with an environment makes it
current with the binding staying put. So a definer written as an operative
now defines for its caller inside a frame: (eval (list 'def n v) e) binds
in e, and doc, def-class, def-record and a lang’s define work in
a closure body the way they always did at top level (#527). The engine
retired the frame marks, the shadow list, the local boundary, the tree a
closure carried and the top-level bracket’s frame stripping with it, and
its base layout’s env fields are now env and env-root. What read the old
rows by name moved with the pin: the sandbox and ISA specs, the image
reader and inspector, the normative example in the spec, the sandboxing
tutorial and the layout passages in the architecture, type-system,
glossary and state-image documents.
Two design notes: namespaces for the library, and first-class
environments beneath them (#718). docs/namespaces.md and
docs/environment-model.md; the environment model was built in
x-engine-c#49 and its readers moved in #720.
A session switches languages. x/repl/lang keeps the REPL’s seams as a
named bundle – the prompt, the continuation prompt, the printer, the
painter, the bracket marks, Tab’s candidate source and what a finished line
means – and (lang NAME) installs one, so the next line the editor reads
is that lang’s, coloured as that lang. Two of those seams are new:
%repl-eval-line, which is what the editor does with a finished line and
the piece that lets a lang keep the editor instead of replacing repl, and
%repl-complete, which is where (Line completer) now lives.
%repl-prompt-more moved beside %repl-prompt. x-lang’s own bundle is
"x", assembled by the files that own its parts as they load. All of it
joins the lang contract’s seam table.
A bare atom at the REPL prints. The line editor hands a finished line
to the reader without its newline, and the reader drops a final atom that
nothing terminates (#161): name at the prompt read as no forms and printed
nothing, and 1 2 lost the 2, while (def name 1) was fine because the
paren closes it. The line evaluator now appends the newline before reading,
as the painter appends a space for the same reason.
The linter reads a set! body as a definition body. (def NAME ())
followed by (set! NAME (fn ...)) is how a self-referential function is
written – the forward declaration lets the body name itself – and the walk
scanned only the def spelling, so the ladder and depth rules never saw
those bodies. Every tokenizer state machine in lib/x/num/ is written that
way and reported nothing; decimal.x held two four-deep chains while
reporting zero ladders. %arity-collect already read the two spellings
together, so this brings the two rules into line with it.
The assembler lane lowers match. The arms are tried in order: a
comparison test folds into cmp and one conditional branch, any other test
is evaluated and tested with cbz, a literal #t test takes its arm
unconditionally, and with no arm taken the value is nil. if reaches the
lane as a match of one or two arms, the way lib/x/core/control.x derives
it, so both forms share one lowering and if emits the instructions it did.
A compiled analyser state written with match used to raise
asm-compile: unsupported form: match inside the tower’s guard and run
interpreted with nothing reported. The byte cache’s codegen epoch moves to
g2, since the emitter accepts a form it refused.
The REPL colours parens by nesting depth. Both halves of a pair share
a colour, cycling yellow, magenta, cyan from the outside in, the way editors
colour bracket pairs; a close paren with nothing to close is bold red; the
pair beside the cursor is drawn inverse on top of its colour, a close just
before the cursor first so a pair lights as its close is typed, and removed
again when the line is submitted, so the transcript keeps the colours and not
the inverse. An open paren not yet closed is simply its depth’s colour, the
state of every line while it is being typed. The depths come from one walk over the whole line that steps
over strings, comments and character literals with the scan’s own rules, so
#\( is not an open paren and a paren inside a string is not counted, and a
line that has scrolled sideways still colours correctly.
%repl-marks joins %repl-prompt, %repl-print and %repl-paint as the
fourth customisation point: a function from the line and the cursor offset to
(offset depth focused) marks, installed and guarded the way %repl-paint is,
so a lang whose brackets are not x-lang’s sets its own or leaves it nil. The
editor passes the marks to %repl-paint as a second argument, translated into
the window; a painter written for one argument ignores it. (Paint marks line
at) and (Paint line text marks) expose the two halves for use apart from the
editor. The scan also treats #\ and the glyph after it as one character
literal. On a 70-byte line the depth walk costs about 4ms and the coloured
render about 10ms over a plain one, and the walk runs on every redraw; a
redraw whose text and marks are unchanged is served from the cache as before.
A class names each of its methods once. (help x/type/list) printed
map, sort-by, times and every other block-wrapped selector twice, and
so did the listing for every other collection: Block method! installs its
operative through def-static!, and %class-add! prepends, so the newest
registration wins at dispatch and leaves the row it shadows behind it in the
cold alist. class-methods and class-static-methods answered with that
alist’s keys as they stood, so a selector defined a second time was named a
second time, and help – which walks those names – had an entry for each
row. The accessors strain their keys through %names-minus now, which keeps
the first occurrence of a name: for a method alist that is the row a dispatch
reaches. The alist and the dispatch order are unchanged, and the field
accessors need nothing – their alists are written through %box-put!,
which replaces an entry in place.
Tab’s candidates are a seam a lang can fill. %repl-paint answers the
colour question for a lang’s session; completion had no such answer.
%ln-candidates prefix-searches the doc registry, which holds what x-lang
modules document, so a lang that parses its own syntax has none of its names
there and Tab at its prompt offers x-lang’s. (Line completer f) installs a
function from the Edit buffer to (typed . names), and () turns Tab off.
Filling a unique answer, extending to the common prefix and listing on the
second Tab are the same job whatever the syntax is, and stay whichever
completer is installed; %ln-complete! tests before it walks, since a nil
completer has nothing to destructure and first/rest are unchecked prims.
line.x’s %-global budget is 36, and crafting-a-lang.md §7 carries both
seams for a lang author.
List and Iter raise on a value they cannot walk. List from-seq is
the door every basic normalizes through, and it hands anything that is not
nil or a pair to (Iter new). That answers nil for a value whose type
carries no iter slot, and the driver prims dispatch on a type handle a nil
does not have, so passing one on ends the process. (Iter new) raises
type now rather than answering nil, which is what gives (List length 5)
and (List length (%type-alist)) an error; each public Iter door also
checks its argument once on entry, so (Iter ->list ()) raises too. The
drain loops carry their iterator through unchanged and their per-element
prim calls are unaffected. Iter %check is homed on the class under the
classes-are-namespaces rule in tools/check/percent-globals.sh.
A lang bundle can no longer change what the library’s containers mean by
equal. equal? is a bare global, so a session may rebind it, and lang
bundles do – x-sweet ships (def equal? eq?) as part of its Scheme shim.
Every container that compares by content read that name: Dict’s bucket
search, Assoc find, List index-of/includes?/uniq/uniq-by, and a
record’s =?. A rebinding did not make them fail; it made them answer a
different question for the rest of the session, in code that never mentioned
equality. Under -l sweet, (Dict get "k") on a dict holding "k" answered
nil, and (List includes? "a" (list "a" "b")) answered #f, with the bytes,
the FNV hash and the bucket all unchanged. Library internals now read
%equal?, captured beside equal? where it is defined, which is the split
protocol/str/utf8.x draws between Str, the
rebindable ambient alias, and Str8, the fixed name its own internals use.
What is captured is the closure, not the behaviour: its body reads
%equal-others at call time, so a module extending equality through that hook
(x/type/vector does) still reaches every one of those seats. Only rebinding
the name stops working.
The painter reports a vocabulary it cannot read. x/repl/paint builds its
construct set from lib/x/constructs.x behind a guard
whose handler returned an empty Dict and dropped the error with it. An empty
vocabulary looks the same as a session in which nothing happens to be a
construct – a line with no colour on it, either way. The fallback is
unchanged, because losing a colour is not worth refusing to start a session
over, but the handler now writes one line to stderr naming what could not be
read.
A compiled function could be handed an argument back instead of an answer.
compile-asm emits for two calling worlds – an integer function called from x,
whose arguments arrive unevaluated and whose result is boxed, and an analyse
callback invoked from C with live values, where nothing evaluates and the
leading params stay pointers – and with no third argument the door reads the
fvar table to decide which: present means analyser. An fvar also names a callee
the body calls, so a body that calls a prim that way compiles as a tokenizer
state. Its first ordered comparison then reads a param as a pointer, takes the
branch on that, and the unboxed result is one of the arguments; other argument
forms dereference an argument expression as an integer and reach a SIGSEGV.
The misuse refuses rather than the undeclared call. Refusing the two-argument
form would not raise in a bundle: x-python compiles its tokenizer states that
way and adopts them under a guard, so the refusal would pin the interpreted
states and keep them, and a bundle spec cannot assert that the JIT is active, so
nothing would report it. The refusal is structural. In analyser mode the leading
one or two params are x_obj_t*, and arithmetic, a shift or an ordered
comparison on one is not something an analyser means – every state in the tower
and in the bundles uses its object params as trampoline arguments and nothing
else. The refusal names the parameter and the declaration that makes the compile
an integer function. =, not, and and or stay legal on an object param:
testing a pointer for equality or for truth is meaningful, and (= buffer ())
asks a real question. A comparison in an if test is checked separately,
because if folds one into its branch and it does not reach the call emitter.
The byte cache key names the compiler. It named the machine and the engine
release, so an entry outlived a change to the emitter: a hit never reaches a
compiler, and a compile whose acceptance or output has changed was served the
bytes an earlier version produced. #597 made this argument for the engine half
of the key, after #590’s cc cache served ABI-stale objects that misread 2.5 as
2 and the symbol .5. The key now carries x-lib-version, which covers a
consumer, and a codegen epoch that covers development: bumping it is part of
changing what the emitter accepts, refuses or emits, the rule the record
format’s magic already states. One boot per machine per version pays for it –
a xenon boot costs 7.8s against a cold cache and 1.7s against a warm one.
Every compile-asm call in the repository now declares its calling world, and
the bodies with no free variable pass an empty fvar table instead of an unused
one.
The session has a line editor, and rlwrap is no longer the answer.
sh x.sh with a terminal now gives arrow keys, the readline chords, history
that outlives the process, Tab completion over every documented name, and
colour applied to what you type as you type it. The documentation has told
people to wrap the session in rlwrap since the REPL existed; that advice is
gone from the README and the tutorial, replaced by docs/repl.md.
It is four modules and only one of them touches a descriptor – x/repl/edit
is the buffer, the cursor and the history walk with no terminal in it,
x/repl/term is raw mode and byte-to-key decoding, x/repl/paint colours a
half-typed line, x/repl/line is the loop that joins them. The split is what
makes it testable: Edit is pure and Term key takes a byte-reading function
rather than a descriptor, so 45 cases run in the ordinary spec harness with no
pty anywhere. Installing it is the seam the langs already use – repl is a
plain global and x-python and x-ash both replace it – so x/repl/line
replaces it too, only when there is a terminal to drive, and a pipe, -f,
-c or a spec harness reaches the C reader’s loop unchanged and never loads
any of this.
Five specs are weighted by their measured footprint ([#696]), so the runner’s heavy-job cap sees them.
Comments in tools/ and the linter’s preload rules are stated plainly
([#691], [#694]).
The reader’s type alist reaches this from ordinary code. It is a C-built
spine, so pair? answers #f and from-seq treats it as a non-list; the
catalog carries it as type/alist, and (Base cell 'type-alist) addresses
the same structure. Such spines are walked with the bare first/rest
accessors, the rule docs/sandboxing-tutorial.md states for handler spines;
type/struct.x, (Base cell) and (List from-seq) point at it.
The wrapper checks that a pinned amalgam is the file the lock pinned. The
boot-time pairing guards compare recorded strings — a row in the lock against a
stamp beside the installed library — and each of them describes the amalgam the
lock names rather than the bytes on disk. An amalgam replaced after the lock was
written satisfies all of them and still reaches the engine as another release’s
boot, where a base layout that has moved is a SIGSEGV in the first form that
walks a base cell. The lock’s three-element (boot "he.x" "sha256:…") row is
the claim, and the wrapper now reads it: one digest of one amalgam, on a path
that already parses those bytes, before the release reach rather than after it,
since a reach hands the whole invocation to the release the lock names. The
two-element row predates the digest and is skipped, and with no sha256 tool
available the wrapper reports the identity as unchecked instead of passing in
silence. (Pin verify) checks the same row on demand and in CI.
A lang can colour its own lines. %repl-paint joins %repl-prompt and
%repl-print as the REPL’s third customisation point: a function from the
line’s text to the text to display for it, which the line editor calls in the
one place it paints.
The lang kit’s linter is corrected in six places ([#683], [#687],
[#695], [#699]; [#686] and [#689] were reverted by [#693]). A batch’s
lint-known declarations come from every file, not the first; the boot
amalgam is found in a checkout as well as an install; the walk’s write
comes from the catalogue, not the global; an assembler is recognised by
its including a sibling .x file, and the cause of an engine death is kept.
It is a hook because colouring is the part of an interactive session that depends on the language. Reading a key, moving a cursor and remembering a line are the same job whatever the syntax is, and repl/line.x already does them for every lang that has not replaced the loop; the sweet bundle inherits the editor without knowing it exists. Where the tokens begin and end is not the same job, and the langs differ on where that answer lives: x-logo registers LOGO-OPEN and LOGO-BLOCK as token types on the base, while x-python parses Python in x-lang and never touches the base tokenizer. One painter cannot serve both.
The install follows the rule repl/ansi.x states for the printer and repl/line.x for the loop: over nil, or over the painter x/repl/paint last installed itself, and over nothing else. A bundle’s entry runs before the launcher that imports the editor, so an unconditional install would take a lang’s painter away and colour its lines as x-lang. A lang’s painter survives whether it is set before the editor loads or after, and across a state image reload.
Nil means no painter installed, not no colour. --no-color, NO_COLOR and
TERM=dumb answer the colour question, and the platform painter honours all
three by returning its argument untouched; setting this to nil to mean off does
not work, because the next install reads nil as nobody having set one. A
painter that raises is caught at the redraw and the line is drawn unpainted for
that keystroke.
The digest engine is built for the input in hand, not for a running
total. Sha256 hex built the compiled engine once 64KB had been
digested in a session, cumulatively – a bar set when the build cost 12
seconds and pure-x ran at 2.3KB/s, and a rule that paid the build for
the wrong input: Pin vendor’s lockfile is two files, 16KB then 2KB,
and with the bar lowered to what the build costs now the first digested
pure-x, just under, and the second built the whole engine to digest 2KB,
12s where pure-x alone was 10. A total says nothing about what is left
to digest; the length of this input does. The rule is per input now:
build when the input is %sha-jit-threshold bytes or more, and the bar
is the measured breakeven – pure-x at 2.4KB/s against a 4.5s build
(#679), so 12KB is the first size at which the build is ahead within the
call that paid for it. Pin fetch and Pin bundle read the same bar
instead of carrying their own 65536. Pin vendor of x/type/dict: 6.6s
from 10.5 on a 12-core arm64 box (the build’s 4.5s, then two digests
in milliseconds where the first alone was 6.7s); a spec case pins that two inputs just under the bar stay
pure-x and one over it builds.
CI keeps the sanitizer engine it built. The gates job’s asan-boot and
the asan job each cloned the pinned engine sources and built x-bin-asan
from them, 55 seconds on both runners before a dialect booted – the same
bytes every run, because a tag is immutable and asan-boot.sh already says
so (“the build never goes stale”). deps/engine-src is an
actions/cache entry now, keyed on the engine pin and the runner image:
the image because the engine is built with gcc, which links libasan
dynamically, so a binary is only as good as its image’s runtime. On a hit
asan-boot finds the build and skips the clone; the asan job’s source
fetch reuses the tree the same way. What is left in asan-boot is the two
tower boots, 1m33s each on the x86-64 runner, and they stay serial:
measured there on 2026-09-13, each peaks at 10.0GB on a 15GB machine.
The state image writer’s host boots from an image. A write is a helium
process running tools/dev/image-write.x over a child that loads the
library from source, and the host booted with --no-image – from source,
every time – so that a host missing its own image could not write one and
land back in the builder that started it. That was 2.3s of a 5.7s x.x
write on arm64, 29 times over in make images (6m43s of a CI specs job on
the Linux runner, where a source boot is slower still), and once per
pinned project’s first boot, where it was the whole of pin-smoke’s
remaining cost: a pinned miss was 6.8s against 3.3s from source. The
wrapper has the mode the recursion actually needs, X_IMAGE_NO_WRITE: a
current image is booted from, a stale or absent one is neither written nor
used, and the boot is from source. image-build.sh runs the host under
it, and tools/dev/images.sh boots helium once before its 29 writers so
that every host hits on a fresh checkout. An x.x write is 3.7s from 5.7, a
pinned project’s first boot 5.5s from 7.35, and the 29 images at four jobs
on a 12-core arm64 box 56s from 72; pin-smoke checks the mode both ways.
The pin gate’s digests cost what they weigh. tools/check/pin-smoke.sh
was ten minutes of the ubuntu gates job, and the job tripped its 20-minute
cap in nine of the last thirteen main runs; measured under a timestamped
trace on a 12-core arm64 box, the smoke’s fifty wrapper runs took 261 of
its 264 seconds, and three costs owned most of them, none of them the
tests. First, Pin fetch and Pin boot digested the engine’s 15KB
isa.x in pure x-lang – 2.4KB/s, six seconds a run – to print a drift
notice, when the fact is already written down twice: an install tree’s
contract/isa.sha256 stamp and a checkout engine’s (isa "sha256:...")
row in x-engine.xon. Pin %pin-tree-isa reads one of those now, the way
the wrapper’s own boot guard compares recorded strings, and says the
pairing is unchecked when neither is readable; a fetch is 2.3s, from 7.7.
Second, (Sha256 jit!) cost twelve seconds in every process that called
it, nine of them compiling the engine’s 12,241-node fill body, because the
assembler cache stood aside above 128 nodes – a cap sized for an
interpreted printer the cache had already stopped using: its key text is
spelled by the C write-to-str door and hashed by FNV, and the fill body
prints in 0.6s and hashes in 0.1s. The cap is gone, and with it the
slurp’s 64KB single read that called a full buffer a miss (the fill body’s
record file is 82KB, so the first cut of this change stored it every time
and never read it): the slurp reads in rounds now and an entry is whatever
size it is. The build is 4.5s from 12, of which 1.8s is relocating the fill
body’s 2,945 sites and 1.4s the differential check. Third, Pin bundle
called jit! unconditionally where Pin fetch had guarded it behind a
64KB size since #324, so the smoke’s six bundle runs and two install runs
each built the engine to verify a few hundred bytes; the guard is the same
now. Pinned by three spec cases: a 300-node body is keyed and its second
compile is a load, an entry longer than one read round loads whole, and
the tree’s fingerprint answers the engine’s declaration. The smoke is 154s
from 264 on the same box, all of it still in the wrapper runs; what is
left is nine pinned throwaway projects at 7 to 9 seconds each, the state
image their first boot writes (#669) against 3.3s for the same boot with
--no-image, which is the writer’s cost, not the pin’s, and the next
thing to measure.
The socket specs let the kernel pick their ports. Four cases bound
fixed ports, two of them inside the ephemeral range macOS hands to any
process that asks, and on 2026-09-12 Spotify held UDP 49364 when the
release tag’s pre-push suite ran: bind: Address already in use, the
gate red on a port the test had no claim to. There was no way to bind
port 0 and learn what was chosen, so Socket local-port is new:
getsockname on a bound or listening fd, the port read from bytes 2-3
of the sockaddr, TCP and UDP alike, a cold (%sk ...) resolve like
sendto so the module’s %-globals budget is unchanged. Every fixed port
in lib/socket.spec.md is now a bind to 0 followed by local-port,
which also makes the eaddrinuse, frees-on-close and econnrefused cases
say what they claim rather than “47913 happened to be free”; the door
has two cases of its own, one for the failure path.
The assembler lane refuses a form it cannot spell on this engine, instead
of calling address 0. An optional JIT symbol – jit_score_label (released
as jit_score_variant),
jit_buffer_last_char – binds as 0 on an engine that lacks it, so that
every other form keeps compiling; but a form that needed one was emitted
anyway, as blr 0, and died arbitrarily far from the cause: x-python’s
compiled number states declared their label on an engine without the
symbol and the first number token after the swap segfaulted (its main-lane
CI, 3 of 5 runs). %emit-call! now refuses an address of 0 with a 'state
Err, and %score-label! (released as %score-variant!) refuses by name
the way a call through a value
already did, so a bundle’s probe hears no and its guard keeps the
interpreted twin. Pinned in ext/jit-optional-symbol.spec.md on every
engine, by standing in for the missing symbol.
A spec can assert a NUL byte in captured output. The shared runner reads
the interpreter’s stdout with cmd | getline, and awk is a C-string
language: a zero byte TERMINATES a record, so a test’s output was truncated
at the first one and compared short – a\0b arrived as a, with nothing
said about the rest. On the one-true-awk that record’s length is 1,
measured. The byte is now escaped before awk reads it, to the literal text
<<NUL>> in the same in-band style as the harness’s own <<SEP>>, and a
spec asserts one by writing <<NUL>> in its expected block; every other
byte passes through untouched, so no existing spec sees a difference. perl
does the escaping, being the only tool to hand that is both byte-clean and
able to expand one byte into several – BSD sed cannot express a NUL match
(“first RE may not be empty”) and tr is strictly 1:1. Without perl the
runner warns once and the truncation stands, so a spec asserting the byte
fails rather than passing quietly, and SPEC_NUL_FILTER overrides the
command. Because the escaper is the pipeline’s last stage, the engine’s own
exit code is now read from a file written inside the pipeline rather than
from close() – which makes a mid-batch death report its status on the
one-true-awk, where that code was previously only ever visible on
gawk/mawk. x-python wanted this: its bytes carries NULs and its str is
becoming a code point list that can hold one, so a bare print of such a
value had to be pinnable at all. tests/spec-format.md states the
contract, meta/multiline.spec.md holds it.
An error’s classifying symbol is its LABEL, not its “kind”. Err grew
up saying kind: (Err kind-of e), (e kind? 'io), the kind field,
(Err make kind msg data), and every doc string that promised “a kind-‘io
Err”. Kind is not a term this tree uses for a classifier – a token’s
classification is its variant, an error’s is its label – so the API now
says so: (Err label e) is the total accessor (the noun applied to the
value, the way (List length lst) reads), (e label) the field,
(e label? 'io) the predicate, and make / raise take a label. No
alias is kept: a guard that matched on (Err kind-of e) now writes
(Err label e) (released as (Err tag e), now (Err label e)).
(Err code-of and File stat’s file-type key, released as kind, are
untouched: the first is an engine raise’s message literal, the second names
a file’s type – ‘file ‘dir ‘link.)
A reader hears which state accepted, instead of rescanning to find out.
An analyser state knows which of its states accepted and threw it away; the
type’s reader then rescanned the text to learn what it had just been told.
The engine now hangs a label cell off the score cell (x-engine-c#43),
records the winning handler’s label at the accept, and hands it to the
reader as its second argument; this release is the library’s two ends of
that channel. %score-label! is the writing end, called by a state at
its accept, and %read-label the reading end – the integer, or nil when
no state declared one; they were released as %score-variant! and
%read-variant. The label travels as a raw atom cell because an
int object only exists relative to a base that registered the int type, and
a tokenizer base has none by design. The assembler lane compiles
%score-label! through an optional JIT symbol, so an engine without it
keeps compiling every state that does not use one. The capability is
tok/label (released as tok/variant) in the contract, claimed by the
engine, and the two spec files
that need it say so with # @requires: the interpreted twin in
lib/reader-label.spec.md and the compiled one in
ext/jit-analyser-label.spec.md, split out of jit-analyser-self so
that file keeps its self-param coverage on an engine without the door. The
runner honoured only the FIRST # @requires line of a file, so one needing
two capabilities was gated on one and would have run, and failed, where the
other was absent; every line gates now, and a skip names the capability it
was missing.
The core boot reclaims its garbage as it goes. The engine never
collects on its own – mark and sweep run from the heap prims and nowhere
else – and the dialect bodies collect once, after the whole boot, so a
source boot’s footprint was the sum of every include’s garbage: the image
writer for lib/x-core.x peaked at 3.05GB on arm64 and 2.9GB on x86-64
for a heap that holds 85K objects when it is done, and on x86-64 the
x-base writer reached 6.2GB and the tower harness 7.7GB, which is why the
Linux runner writes its images one at a time. x-core.x now collects at
fourteen batch boundaries between its own includes; it may, because it is
included at the top level of the dialect bodies and the harnesses and
imported by nothing, so nothing of an includer’s is in flight – the case
the module rule guards against does not arise. Measured on a 12-core
arm64 box, interleaved against main: a source boot 1.92s to 2.29s, the
x-core writer 3.05GB to 1.15GB, x-base 4.14GB to 2.37GB, the tower harness
3.92GB to 2.58GB; a boot from a state image pays nothing. The tower’s own
load burst inside boot/tower-compiled.x is what remains, and stays,
because that file is importable. The asan-boot gate, the full suite and
doctest passed on it.
A pinned project boots from a state image of its own. The wrapper
imaged every boot but one: a project with a pin.xon paid the full source
traversal on every run – seven seconds for a helium REPL that boots from
an image in under one – because “what a pin arms is per-directory state
the key does not see”. It sees it now. A manifest’s (root ...) rows are
armed inside the imaged prefix (pin_arm imports x/tool/pin, which reads
the manifest and import-path!s each root), so the manifest is a key path,
hashed as one file, and an edited manifest is a different image. The image
lives beside the manifest, in the project’s own .images/, the way a
bundle’s does – the per-user cache keeps one file per dialect per install
root, and two pinned projects would have taken turns overwriting it. The
overlay’s modules are not in the key and need not be: they load after the
loader, on import, from the roots the image armed. What stays unimaged is
a pinned boot amalgam (--boot, or a manifest’s (boot ...)): another
release’s boot, which this tree’s loader has no business standing in for.
tools/check/pin-smoke.sh grows the case: the image lands beside the
manifest, the second run boots from it with the overlay resolving, a
touched manifest misses, --no-image boots from source, and a manifest
with a (boot ...) row is refused by --image as before. Add .images/
to a pinned project’s .gitignore, as the bundles do.
A bundle boots from its image again after the platform is reinstalled.
A bundle’s image key carries lib/ and the engine, so one platform
reinstall staled every installed bundle’s image at once, and the wrapper’s
rule – never write into a bundle behind its installer’s back – left what
followed as a source boot on every run, with nothing on stderr: x -l
python took 16.4s a boot against 0.98s from the image its installer HAD
written, until the user thought to re-run each bundle’s make install. The
rule is kept, and the fallback the wrapper already had for every other boot
now applies to bundles too: the installer’s image is preferred, and on a
miss the user’s own is written into the per-user cache, announced, and
used; the key gains the bundle’s directory so two checkouts of one lang do
not overwrite each other’s. Found beside it: tools/dev/image-build.sh
defaulted its wrapper to ./x.sh, which exists only in a checkout, and the
six lang bundles that call it from their spec runners set nothing else, so
from an installed tree not one of them wrote an image at all – the failure
went into a log beside the image nobody reads, and the suite booted from
source. The default now finds a wrapper – ./x.sh in a checkout, else the
install’s bin/x, else the x on PATH – and says so rather than failing
into a log when there is none; only the write looks, a check still answers
from the key alone. Measured on x-python’s suite, one spec file: 22s before,
1.5s after.
The state images are written in parallel. make images wrote its 29
images one after another, and that loop was the longest phase of a CI
specs job: 6m43s on the 4-core Linux runner and 9m30s on the 3-core macOS
one, ahead of the 8-minute suite the images exist to speed up. The writes
are independent, so tools/dev/images.sh runs them under xargs -P. The
job count is bounded by memory, not cores: a writer boots a library from
source and images the child, and measured one at a time x-base peaks at
2.0GB, x-core at 3.2GB and the tower harness at 3.9GB – a first cut that
took one job per core put twelve writers on a 16GB box and took it down.
On arm64 a job is budgeted 3.5GB: four on a 16GB box, two on the 7GB
macOS runner, where the 29 images now take five minutes against nine and
a half. On x86-64 the same writer is bigger – the heap costs ~64 bytes an
object there against ~29 – and four jobs killed the 16GB Linux runner 49
seconds in, so until its peak is measured x86-64 budgets 9GB a job, which
keeps that 16GB runner serial. A box whose size cannot be read gets one, and
IMG_JOBS overrides. Measured on a 12-core arm64 box: 271s serially, 84s
at four jobs, with the suite green behind it.
CI’s specs job has room to breathe. Its 20-minute cap was set when the macOS run took eight; across the last eight runs of main it took 8, 13, 15, 16, 17, 18 and 19, and PR #665’s first attempt was cancelled at 20 with every suite green, fifteen seconds into doc-examples. The cap is 30 now – a slow runner is not a failed change.
Vector build takes a block. The one callable-taking builder without
one: (Vector build 3 (i) (* i i)) now stands beside (Vector build 3 (fn (_
i) (* i i))), wired at position 1 as List times is, count first. It
slipped the collection-wide sweep because its callback is declared ANY,
not CALLABLE, and the sweep keyed on the type token; every other builder
on an iterable either has its block already (List times, List iterate,
Gen iterate, Gen make, Iter make) or takes no callable.
A list dispatches to List at the value. ((List of 1 2 3) filter (x)
(> x 1)) answered Unbound SYMBOL 'filter': the engine’s list call reads its
first argument as an index, and a vector’s or a string’s value form – (v
filter (x) (> x 1)) – had no counterpart for the one collection every other
one is built from. x/type/list now binds List OVER that handler the way
Vector and Str are bound, so a symbol selector sends to the class,
subject-last, and a call rides it with a block or with a function:
(xs map (i x) (* i x)) and (xs
map (fn (_ x) (* x 10))). Everything the handler underneath did it still
does – (xs 0) indexes, (xs 1 3) slices, and nested list data the iterator
re-evaluates keeps its #69 echo, since a list head followed by a non-symbol
delegates to the prior handler unchanged. The linter learned the value form
with it: it bound a block’s names only under a class head, so (v map (x)
...) reported x undefined for a vector as much as for a list – a block
send binds its names at any head now, still keyed by the selector table. The
full suite ran with the binding in place before it was kept: 2961 tests, 0
failed.
A comma is not a token delimiter, and the owl’s nose is one. The delimit
hook quote, quasiquote and unquote share ended a token on ', ` and
, – so
foo'bar reads as foo then 'bar, which is right, and a comma glued to a
token split it, which was not: {O,O} was two tokens. Unquote is recognised
only where a token BEGINS, so with a space or a paren before it the comma
already starts its own token; the delimiter changed exactly one thing, and
it is gone. Bitwise’s nose is {O,O} – every costume, the frozen parity
fixtures and the twin’s renderings, the site bar and footer, and x-lang’s
own mark and banner follow; only the auk keeps its > bill, for the
wordplay. x-lang’s reference line is (def owl "{O,O}") – the bare datum
was splitting on the comma until the reader stopped treating one as a
delimiter.
A wrapped method’s help shows its block form. (help Vector for-each)
answered the applicative signature and nothing else – true, and incomplete:
the block form is a second way to write the call, which a reader could not
discover from it.
The wrap is the one place that knows the shape, so it adds the note itself,
onto the method’s own entry, pending or committed; wrapping twice does not
say it twice. It shows the method’s own head with the block in the
callable’s seat – (Vector filter (x) body ... v), (for-each (p) body
...) on a Dict, (List times n (x) body ...) where the callback is not
first – and the shape’s other spelling as a suffix, (i x), (acc i x),
(k v). “In place of f” was not discernible; the head is. One fact stated where it is
decided, instead of fifty (doc ...) forms repeating it and drifting. The
generated reference carries it too: the generator loads the module it
documents (guarded; boot files and the dialect toolboxes skipped) and merges
each method’s live registry notes into its page, deduplicated against the
source form’s own – so a page shows what (help) shows.
A block’s index comes first. (v for-each (i x) ...), not (x i): the
order of Gen enumerate’s (index . value), which this library had already
fixed, and the one a hand reaches for – the prototype was index-first for
that reason, and the shipped order was a crossed wire. A fold’s optional
index precedes the element it indexes, (acc i x). Breaking for any
two-name block written against 0.13.0; the language cannot catch a swapped
pair, since both names are just symbols, so the fix is to swap them.
A bundle can be swept, and the assembler is preloaded whole. make
lint-x sweeps lib/ and apps/; a lang bundle under languages/ was
swept by nothing, so every rule the linter knows was advice those bundles
never heard. The platform ships the gate, bundles do not vendor it:
tools/lang-kit/lint.sh drives the linter at a bundle, with an overridable
root (X_LINT_ROOT/X_BIN) for one outside the checkout, a non-module
sibling recognised as a fragment (x-coreutils is one module assembled from
twenty-one include-once‘d files with no provide), a clean file’s
advisory warnings no longer dropped with its dot, and string=? known to
the ladder rule – 476 uses across eight bundles that %ladder-cmp? could
not spell. Pointing it at the bundles found the next two: an assembler’s own
top level was missed (cc/base.x defines %cc-x-write; five fragments call
it; an import cannot reach it), and a batch’s preload omitted its own first
file. The assembler is preloaded whole now, in the order the bundle really
uses, which also drops the order reconstruction; %cc-x-write resolves and
two phantom go/N ladders vanish. Swept: x-coreutils, x-awk, x-grep,
x-make, x-cc.
Every callable-taking method takes a block, and two shapes joined. The
block form shipped in 0.13.0 on 35 selectors; this wires every remaining
higher-order method but one – List’s count-if none? reject find-index
uniq-by drop-while fold-right scan zip-with iterate, Gen’s none?
drop-while scan zip-with iterate make, Iter make, and Assoc’s map
(the callback gets the value) and filter (it gets the whole assoc): two
shapes on one class, which is why shape is declared per selector. Two
mechanism extensions covered the stragglers. POSITION: List’s
constructor-count rule puts the count ahead of the callback, so (List
times 4 (i) (* i i)) and (List adjust 0 (x) (* x 100) lst) wrap at
position 1, the forms ahead evaluating in the caller’s env. THUNK: () is
a binding list, and get-or-else takes it – (d get-or-else () (expensive)
k), the body a default run only on a miss; an empty binding list in any
other shape now fails with its count. Left out on purpose: List unfold
takes three callables, and a block for one of them would confuse more than
it saves – it is the spec’s opt-in proof now; PQ make waits on its doc.
The linter accepts the binding list at position 0 or 1 and ().
Interpolated strings are spelled #"…". The $"…" spelling (0.4.0)
was a mistake: every other reader extension – #t, #\a, #(…), #/…/
– wears the #, and interpolation was the one form that did not. It does
now: #"a{x}b" reads exactly as $"a{x}b" did, into (Str8 str "a" x "b"),
a #"…" nests inside a hole the same way, and {{, }} and \{ mean what
they meant. The other #-prefixed readers are untouched, and the spec pins the
neighbours: x_token_analyse runs every handler from the token’s first
character independently, so the literal’s analyser declining on #t or
#\" costs their own readers nothing – and the read guard now checks the
whole #" opener, so #\", three bytes that start with # and end with
", can never be mistaken for an empty literal. The library, tools, specs
and docs are respelled; the highlighter, the formatter and Xon arm-source!
follow the reader; the downstream langs never used the form. BREAKING: a
$"…" left in source fails loudly – an unbound symbol $"…", or past its
first space an unterminated string – and the fix is one character.
The REPL printer is only Ansi’s to move. Ansi install runs again
after a state image loads – that is what put colour back in an imaged
session – and it ended by enabling or disabling the highlighted printer
unconditionally, so a lang’s own printer was replaced by the platform’s:
x-krn’s suite from an image printed ('b 'c) for (b c), four specs
failing on nothing but the printer, and the quieter half was worse –
bundles whose values print the same either way passed while using the
platform’s printer instead of theirs. The class now remembers what it
installed and moves %repl-print only when it holds the platform’s own
printer or that one; anything else belongs to someone who asked for it.
x-krn from an image: 74 tests, 4 failed, to 0. Found wiring the remaining
bundles’ suites to boot from an image.
The image spec’s scratch files are per checkout. The state-image spec
kept them in a fixed /tmp/x-image-spec, world-writable and shared by
every checkout and worktree on the machine, and they are not inert: pre.x
carries this run’s %IMG-PATH, and one test writes one binding
%IMG-VERBOSE. A concurrent run in another checkout supplied them, so the
header probe booted the PEER’s image, verbose, and captured the loader’s
count line where it expected header ok – the leftover file named the
other session’s worktree. Intermittent, one run in three, and it presented
as an unrelated red that blocked git push through the pre-push hook. The
scratch dir lives under X_IMG_DIR now, which is per checkout.
A KEY-PATH is a bundle’s source, and only the bundle spells it.
image-build.sh keyed a caller’s KEY-PATH with find -name '*.x' and
nothing else, so a lang whose modules are written in anything but .x had
them silently outside its own image key: editing one left the key unchanged,
the builder answered “is current”, and the suite went on testing the
library that was there before, out of the image, while its from-source
control tested the one on disk – both legs green, at two different
libraries. Measured on x-r7rs, whose scm/ layer IS its library: before,
CURRENT after an edit; after, NEEDS REBUILD. Reworked after review: the
first version put x scm in the platform’s default, a bundle’s fact living
in the platform, and it is not there any more.
libm addresses live in cells, not inside closures. float.x resolved
each libm entry once at load and closed over the result – a raw address
inside the closure’s own frame, where the transient rule cannot reach it:
%image-transients names globals, so clearing %fsin emptied the global
and left the frame the closure still held. Seventeen survived the child’s
collect and the writer refused the image on unnameable: 16, every one a
foreign address, and X_IMG_WHO=1 named the holder in one word: sym. It
refused on Linux only because only there the words could not be NAMED –
glibc keeps a dlopen’d libm out of the global scope, where macOS’s
libSystem folds it in – so a clean unnameable: 0 on a mac was this bug
passing quietly, not its absence. The address lives in a one-slot cell the
closure reads at call time, and a thunk among the transients empties every
cell in the child before the walk.
Opts asks the declaration before it assumes. Two fixes, one fault –
the class assuming something it had been told – both found by x-coreutils
within hours of adopting the library. (Opts on? o "-m") answered #f for
a flag that HAD been given, when it was declared as taking an argument:
on? read only the standalone list, so a caller asking “was -m given” had
to know which of its own two lists it went into. It answers presence
either way now; a value flag is still read with (Opts value ...). And
-5 read as an operand because it looks like a number – a structural test
alone, before consulting the declaration – so comm(1), which declares
-5, could not take it. The declaration is asked first; the heuristic
still stands where nothing claims the token (sort -5 is an operand, not
five unknown flags).
The documentation site wears the owl. Every page under
jonruttan.github.io/x-lang published as bare jekyll-theme-primer – a blue
x-lang text link above the H1, and nothing else, across 143 pages of which
113 are generated and nobody edits. Nothing here is designed: the marks
already exist, apps/bitwise draws them from bitwise.xon, and
sha256("x-lang") picked the 261.6 degree hue, so this only carries them
onto the pages. A bar with {O,O} x-lang and three links, a footer, that
hue on every prose link, and a favicon where there was none – the avatar
format, generated by the same tool rather than drawn by hand, so it stays a
function of the name. The owl in the bar is TEXT, not bitwise-mark.svg:
the drawn glyphs are Roboto Mono outlines over their field, and rasterised at
30px they anti-alias into a grey smudge, while {O,O} set as type is crisp
at any size and costs no request. The frame is
docs/theme/_layouts/default.html, an override of the theme’s layout of that
name – Primer’s supported hook reaches <head> only, and a bar and a footer
are markup in <body> – kept beside the pages it frames and moved to the
site root by the workflow, where Jekyll looks for it. One thing beyond the
chrome: “Improve this page” no longer appears on the generated reference,
whose pages are never committed, so the link had been sending a reader to
github.com to CREATE a file the next build overwrites.
The documentation catches up with the code. width is pinned in the
glossary as the code-point count, with bytes and true display columns
named as the two things it is not; the formatter’s comment above its width
function had described a (s) call the code no longer made and a
str-length it had never called. standard-library.md documents all
fourteen Vector methods where it listed seven, points List map at the
block form, and reads 0.12.0 where it said 0.5.2; object-system.md
says what a send with no selector does.
Opts: the command line, parsed against a declaration. Every bundle was writing this by hand – x-grep’s and x-make’s option readers were byte identical apart from an error label, and x-coreutils had grown nine of its own. The cost was never the duplication but the drift between the check and the read. A bundle declares its options and Opts answers them.
Block form across the collections. A higher-order method takes a callable,
and the call site can now write the callback’s parameter names and body
directly – (List map (x) (* x 10) xs) beside the applicative
(List map (fn (_ x) (* x 10)) xs), with a second name binding the index.
Both forms stay live on the same selector. List group-by, List partition
and Dict map join them.
A string’s value-call indexes. ("ab") answered 2, which made a bare
string call a second, unrelated operation wearing the indexing syntax. It
indexes now, and a length is asked for by name.
A send that names nothing errors instead of crashing. ((new P v 1)),
(P) and (#(1 2)) each took the interpreter down with nothing on stderr.
(first ()) can be made to raise. It is undefined by spec and
dereferences nil in practice, so an ordinary typo takes the process down
with no diagnostic. (import x/tool/safe-access) guards both accessors.
Opt-in: the library walks its own lists through them, and the guard costs
1.4x-1.7x.
A boot from a state image keeps its terminal. Three things a boot decides
from the process it is in, which an image cannot carry, all found by driving
langs from one. Colour is the visible one: every colour is a string baked when
x/repl/ansi.x loaded, and the writer’s child has a pipe for stdout, so an
imaged session had colour off for good – (help) came out plain in a terminal
that could show it. Detection and installation are one Ansi install now, run
again by an image recache hook. Measured on a pty, escape codes in (help):
source 63, image 0 before, image 63 after; a pipe still gets 0.
A boot from a state image keeps ctrl-c, and a bundle keeps its session.
Two things a boot does that an image cannot carry, both found by driving
x-logo from one. The SIGINT handler is the process’s, not the heap’s:
sigint-install ran in x-core.x at boot and never again, so every image boot
ran with SIGINT at its default and ctrl-c killed the session outright instead
of raising STOP – in every lang and every dialect. It is now an image recache
hook. And %batch? was reset for a bundle only when $file was empty, which
for a bundle it never is (it holds the bundle’s own entry), so an entry that
chooses between a REPL and a batch reader on %batch? always read the
--batch the loader needs: x-logo booted from an image read its own launcher
as a Logo program and exited without a prompt. The reset now keys on the
user’s file, the same rule bundle_form uses.
The image writer refuses instead of dying. Five lang bundles crashed it
on first contact, and a crash says nothing. The transient walk now runs
from the writer’s base with primitive objects rather than as a form the
child evaluates (r5rs rebinds fn), and a raise inside the child stops the
write with its message – a swallowed raise left the child’s root chain
pointing at C frames that were gone, and its next collect walked freed
stack, which was every silent SIGSEGV. A reference word the writer cannot
place is reported by its holder, never read; a child past a million live
objects is refused with the count; an entry that reads stdin at load (the
engine’s program and the child’s stdin are one descriptor) is refused as
“ended the writer”, and the child is told it is being imaged through
%image-writing; a signal in the log is called a crash. r5rs images
(144K objects, 0.5s to boot against 7s); logo and python refuse with three
and two unplaced references into the second base each makes at load.
The recache walk does not iterate the way R5RS does. (%image-recache!)
remakes what an image could not carry – a dlopen handle, a JIT trampoline,
the colour detection – and its hook walk was spelled (do ((first l)) ...),
which sequences correctly here and nowhere else: do is late-bound, and a
lang that gives it Scheme’s meaning reads that as a loop and calls no hook at
all. x-r5rs did, silently, and the first thing to fail was thousands of forms
from the cause.
A lang boots from its image. x -l NAME paid the same boot on every
run – six seconds for awk, eleven for xe – although its suite had been
booting from a state image since 0.11.1. The wrapper now images the prefix
it pipes ahead of the user’s program (the root and param forms, the entry,
the pin arming, a bundle’s root and entry) through the same
tools/dev/image-build.sh and the same key, and when the image is current
the loader stands in for the prefix: x -l awk is 0.7s, x -l xe 0.9s,
an installed x -l he 0.4s. The engine’s own args survives the install
(the loader rebinds it, since the writer’s child had none), and what the
prefix would have decided by evaluating – the %batch? reset, the
launcher – is emitted after the loader. A bundle’s image is its
installer’s: make install runs x --image -l NAME into the bundle’s
.images/, and a stale one means boot from source, quietly. Every other
boot writes its own into ~/.cache/x/images/<tree>/ on a miss and says so
on stderr, because that run pays a boot twice. --no-image boots from
source; a pinned boot is never imaged. make install ships the image
tools and the layout contracts under the share tree, so an installed x
does all of this too.
The first cold load of an x-base image found one more transient: the compiler’s trampoline addresses are integers, resolved when the module loads, and an integer names nothing, so a compile on the far side of a load called the writer’s process’s addresses. They register as transients now and the recache hook resolves them again.
The engine pin rides to x-engine-c v0.2.8, and the three JIT dialects
are in the images list. v0.2.8 carries x-engine-c #41: a def scopes by
the live frame rather than by an empty save stack, so a def in a closure’s
tail position is frame-local like the one a form earlier. That is what
had refused x-base, xe and rn: the asm lane’s temporaries (hit,
cell, buf) leaked into bare globals, and though the pinned v0.2.7
imaged them with a warm asm byte cache, a cold one took the compiler’s
miss path, whose own tail defs leaked too – and a CI runner is always
cold. On v0.2.8 the three write clean from a cold cache, so make images
lists them after x-core, x and he: 29 images, none refused, and the
lang bundles’ harnesses, which all boot x-base.x, get the image they
were waiting for.
The JIT dialects can be imaged, and a lang bundle’s suite can boot from
an image. A state image could not carry the tower’s compiled analysers
– native code in a page the writing process mapped – so x-base, xe
and rn were refused outright and every lang bundle’s harness, all of
which boot x-base.x, with them. boot/tower-compiled.x now records each
compile as a site: what it displaced, where it went, and how to make it
again. The writer runs it inside the child before its walk – a thunk listed
among %image-transients, the second half of the transient rule in
boot/reflect.x – which puts the interpreted twins back; the loader’s recache hook compiles them anew
in boot order. The x-base smoke and reader specs run from the image in 1s
a file against 6s from source, and a load with every analyser native
again is 0.77s. What refused the three after that was the asm lane: a
def in a closure’s TAIL position bound globally (the engine decided
top-level by an empty save stack, and the frame is popped before a
deferred tail runs), so every compile left its last function, self-cell
and read buffer in bare globals (hit, cell, buf) the writer cannot
name. The writer now says so: when anything is unnameable its census is
followed by the holders, one path up to a named spine node, which is how
those four were found. The engine fix – def scopes by the live frame,
eval! evaluates as a top-level form, an operative’s restore sheds an
inner frame – is x-engine-c’s feat/def-lexical-scope, and with it the
three write clean from the real library. On the pinned engine they image
only with a warm asm byte cache (a cold one takes the compiler’s miss path,
whose tail defs leak too), so make images waited for that fix to be
pinned (the entry above). The writer’s holder chase is opt-in,
X_IMG_WHO=1: it is minutes on a dialect-sized heap. docs/state-images.md carries the
measurements and the open item.
For the bundles: tools/dev/image-build.sh takes extra key paths, so a
bundle’s image is keyed on its own modules as well as the platform;
tests/spec-runner.sh finds the loader from the platform root rather
than the library’s directory and roots the loader’s repo-relative
includes there, since a bundle’s suite runs in the bundle; and make
boot emits a launcher-free x-core.x beside x-base.x, the amalgam a
helium-weight harness should have been loading all along –
docs/lang-contract.md says which to load and why. x-awk is the worked
example: 167 tests, 38s from x-base.x to 18s from an image, one file per
job, and IMG=0 as the from-source control.
The engine pin had first moved to x-engine-c v0.2.7, and with it #599
closed: a type registered on another base outlives the collector.
base-make-type pinned the type’s objects by marking from the target
base’s tree root, and a base’s root is born SHARED, so the walk stopped on
its first node and pinned nothing – a defect that surfaced one collect
late and read as a collector bug. A lang that brings its own tokenizer
base (x-ash) could not take the per-turn sweep and ran its suite with the
per-snippet collect off; both workarounds went with that pin. The
crafting-a-lang.md entry that described the defect asked to be deleted
when it closed, and is.
The release gate reads ci.yml job by job. It polled the tag’s run
until nothing was in flight, which meant waiting on make test-asan –
CI’s own ~32-minute extra, not part of make test – and tripping the
cap when the run had queued behind another main push. The verdict is now
the jobs that cover the suite, each judged as it lands.
Lint children sweep before every file. A batch child lints up to
eight files in one boot and x collects only when asked, so its peak was
the sum of every file’s analysis garbage – four such children OOM-killed
the 16 GB release runner twice. The sweep runs before each file, the
first included: the single-file children (class.x, pin.x,
asm-compile.x) start on 17-19M live objects of boot and reader residue,
and that was the peak that mattered.
The two long spec batches are cut along their fixture chains:
tools/pin into six files sharing two on-demand fixture trees,
bitwise-parity in two, none over 7 s.
x-coreutils’ contract advances to 103 applets, 0 failed.
The tool tier got its doors. A bundle could read and write files and
little else, and the small tools stopped at that wall: no chmod, no
ln, no readlink, no df, no uname, no way to ask who is running.
File gains chmod, chown, link, symlink, readlink, utimes,
mkfifo and statfs — syscalls, in that module’s own idiom, and only
the DARWIN table needed new numbers because the Linux index tables
already name every one. Two of those numbers are not the obvious ones:
statfs64 (345) rather than statfs (157), whose 32-bit-inode struct
reads f_bsize back as zero; and utimes takes only the clock, because
explicit stamps want a packed pair of timevals and that module holds no
pointer prims to build one. Sys gains getuid, geteuid, getgid,
getegid, getgroups, uname, cpu-count, sync, fsync, nice
and chroot — all libc through %resolve, so no syscall number appears
there at all. uname reads the five utsname fields as C strings at
their own offsets rather than through the Struct codec, because
posix.x loads mid-x-core before that codec exists; the array WIDTH is
the whole per-OS difference, 256 against 65, as _SC_NPROCESSORS_ONLN
is 58 against 84. Thirteen specs, each checked against the system tool
it mirrors: getgroups against id -G, uname against uname -smrn,
cpu-count against hw.ncpu, statfs’s block counts against df -k.
What they unblock is x-coreutils’ second half — the nineteen applets
from chmod to whoami that take it to busybox parity at 92.
A compiled function can call something other than itself. The
compile-asm lane refused every head but the function’s own name, so a
compiled function could recurse and nothing else: anything a lane function
needed from another function had to be inlined into it, which cannot reach a
callee that loops. Two call forms join the self-call. A name bound to an fvar
holding a prim compiles to a call to that prim, its address baked at
generation (#603). (%call HEAD arg ...) takes an operand as its head
— a callback parameter, a pointer read out of a dispatch table — so which
prim runs is decided at run time (#604); C function pointers match that
form exactly, and a compiled caller of one no longer has to drop back to the
interpreter. Both build the argument list the self-call already built and
hand (callee arg0 ...) to a new jit_call_value trampoline, which
checks the head is a callable prim before it branches: on anything else
the word it would jump through is a length or a character, and the crash
would have no relation to the call site. A head the emitter cannot resolve
— an unbound name, an fvar holding a non-prim, an unimplemented operator —
still refuses at generation. Analyser mode is now declared rather than
inferred: compile-asm takes an optional third argument, defaulting to the
old fvars-present guess, so an integer function may carry a callee fvar
without its params silently ceasing to evaluate. The byte cache’s key
carries that mode in its own right for the same reason it carries the fvar
table’s layout: the two worlds emit different bodies for one source text,
and a key that could not tell them apart would serve the wrong one as a hit.
A lang can word the engine’s errors. (no-such-binding) reached x-lang as
a bare, type-less atom holding one pre-flattened English string — “Unbound
SYMBOL ‘no-such-binding’” — with the symbol already concatenated in and thrown
away. A lang that wanted to say it differently had nothing to work with: no
type, so no dispatch stack to push a handler onto, and no structure, so its
only recourse was to pattern-match English.
The engine now raises a typed ERR carrying (code . subject) — the raise
site’s message literal and what it was about, unflattened — and
x/type/err-io.x pushes the default wording onto that type’s write/display
stacks. The wording is byte-for-byte what C emitted; the difference is that
it is now a handler, so a lang pushes its own over it and pops it again, the
way char-io.x has always used for CHARACTER:
(%type-push-display ERR
(fn (_ e) (display (Str8 append "symbole non liée : " (Err subject-of e)))))Err learned the vocabulary: (Err label e) (released as
(Err kind-of e), later (Err tag e), now (Err label e)) answers
'engine for an engine raise rather than lumping it in with 'user, and
(Err code-of e) / (Err subject-of e) return the two facts as strings.
The engine’s whole raise vocabulary is five codes, so keying a translation
off them is tractable. boot/printer.x lost the identity test it used to
need (#54): a nil-typed atom had to be recognised by pointer to print at
all, and a typed value simply dispatches — two %-globals gone with it.
Uncaught errors still word themselves in C and read exactly as before: that path runs before any library is loaded, and nothing on a fatal path should be calling into x-lang.
Migrating a guard that reads the message. The value a guard receives for an ENGINE raise is now a structured ERR rather than an atom whose bytes are the message, so the old lifts out of that atom no longer work — they read a slot as a character pointer and hand back garbage rather than failing loudly, which is the one unkind part of this change:
| was | now |
|---|---|
(symbol->str e) |
(%display-to-str e) for the whole sentence |
(Str8 append "" e) |
(Err code-of e) for just the code |
| — | (Err subject-of e) for just the name it is about |
Guards that only re-raise or match on (Err label e) (released as
(Err kind-of e)) are unaffected, and (error "msg") still delivers the
string itself exactly as before.
The documentation answers a machine now. AGENTS.md (with CLAUDE.md
symlinked to it, so there is one file and not a copy to rot) is the briefing
a coding agent needs and the prose docs could not be: how to run it, the Lisp
assumptions that fail here — no car, no print, subject-last dispatch,
self as argument 0 — the three calls that answer “what exists” without
reading anything else, and which dialect has what. llms.txt indexes the
published documentation for the same audience, listing entry points rather
than every page, so it cannot go stale behind them.
A failed dispatch suggests what you meant. (List mp …) answered no
such static member mp and stopped; it now adds -- did you mean map?.
Selectors within one edit of the miss, or a proper prefix of it, are
proposed, three at most, privacy-wrapped entries skipped — a typo
self-corrects, a wrong concept still does not, and a miss with nothing near
it reads exactly as it did before. Both dispatch-miss sites share the one
cold-path helper; the hot path never reaches it.
Two ways in that were missing, and a silent one that lied. x.sh gained
-c/--eval: evaluate an expression and exit, repeatable, expressions running
in order, so a definition and its use fit in one command. Before it, asking
the language a one-line question — what does (help Str8/split) say, what
does this expression evaluate to — cost a temporary file, which is why
nothing scripted ever asked. The obvious alternative was worse than missing:
echo '(write 1)' | sh x.sh printed a prompt, evaluated nothing, and
exited 0. The pipe the wrapper builds is the engine’s stdin, so the REPL
reaches the caller’s only by reclaiming fd 3 — and repl/loop.x reclaims it
only (when (Sys isatty 3)), which a pipe is not. A non-terminal stdin is
now program text, appended after the library: the wrapper’s spelling of the
cat lib/x.x - | ./x-bin the README has always documented. make
check-wrapper is the gate that keeps all of it honest — the spec suite talks
to the engine, so nothing tested the wrapper at all.
x-lang’s own spec fences say x, not scheme. 2,735 of them, across 154
files. The tag was decoration to the runner — it collects any fenced block the
same way, and only ` ```output ` means anything to it — but it is a claim to
every reader, and it named a language this is not. A reader who trusts it
reaches for car, which is unbound. x is what the rest of the tree already
uses: it is the tag the Pages build highlights (Rouge has no x-lang lexer, so
tools/dev/highlight-sweep.sh supplies one) and the tag the hand-written docs
are written with. The scheme row stays for the langs that are Scheme
dialects.
x runs C, and compiles it. x-cc registered at 27 specs and left this
release at 117, 0 failed — the language arc’s final tier. The front end is
whole and the evaluator is a cell machine, with every expectation an oracle
row against /usr/bin/cc: twin agreement is the spec, so a first
divergence is loud. The C surface grew across the release to pointers,
structs (fields as cell offsets, -> and ., arrays of structs, scaled
pointer steps, typedef, copy), switch with fallthrough, function-like
macros with argument text and rescan, enum, union, function pointers,
initializer lists and the #ifdef, #elif and #undef.
The half that makes it more than an interpreter is build: eligible integer
functions lower through the engine’s own compile-asm lane to native code
with no external toolchain. Loops become tail self-recursion — params and
accumulators alike ride the self-call, body locals substitute away, if/else
merges as a ternary, return/break/continue are guarded exits, and
nested loops two deep run as a state machine over the one self-call. Pointers
make the program’s memory one raw buffer that the interpreter and the native
twin address alike, so arrays cross the boundary and a native bubble sort
sorts main’s array. A 2M-iteration loop costs 79s interpreted and 9.5s
built.
The JIT compiles each function once per machine, not once per process.
compile-asm emits the same bytes every time it is asked for the same
expression, and a xenon boot asked eleven times — 7.2M evals of a 52M boot,
none of it per-process except the addresses the code bakes in. Those are
recorded now, so the bytes are kept and poured into a fresh buffer with each
address re-encoded for the process loading them: warm boot 46.0M evals
(-11.5%), -12.6% wall; cold pays +5.8% to fill the cache.
Two things had to be true for that to be worth having. Nothing on this path
may walk bytes — hex-encoding the code cost ~24,000 evals per byte and
parsing a text record file by hand ~880, so the code moves through libc
write(2)/read(2) straight out of and into the mmap’d buffer and the
record file is fixed-stride binary. And the cache is the DOOR, with the
compiler as its fallback: loading asm-compile.x costs 2.5M evals before it
emits an instruction, and a warm process never loads it at all.
Correctness rests on the key, because the failure this replaces was a cache key blind to engine identity serving ABI-stale objects that silently misread numbers. The key carries machine, engine release and the fvar table’s layout, and the whole key text is stored in the entry and compared before a byte is trusted — so a hash collision costs a recompile, not a wrong function. Every doubt is a miss rather than an error. Expressions past 128 nodes are not keyed at all: naming one means printing it, and the printer is superlinear (1.4K evals per node at 35 nodes, 8.9K at 400), so a generated body takes the uncached path it always had.
Fixed alongside: the assembler’s relocation slot was never traced by the
collector — asm-new makes seven slots and set-units! said six — so one
collection turned three live records into a single nil. Quiet since it
landed, because nothing yet held those records across an allocation.
The reader’s hottest hook is compiled too. %macro-delimit runs on every
character of every symbol token — the C symbol analyser calls it per
char to ask whether ' ` , terminates the token — and it was the
single largest per-character reader cost while the numeric analysers beside
it were already compiled. It takes the same path, so it goes through the same
lane, reaching the new jit_buffer_last_char trampoline as
%buffer-last-char. It stays safe on an engine that predates that
trampoline: the compiled twin is built ONLY when the address resolved, and
asm-compile.x resolves it through a %jit-addr-optional that records no
miss — recording one would trip compile-asm’s whole-runtime “JIT
unavailable” refusal (#201) for every compile and drop even the numeric
analysers to interpreted on a fully JIT-capable engine.
A compiled analyser may now call itself. The assembler lane’s self-call
path was written for integer recursion and had never been run by an analyser,
which breaks all three of its integer assumptions: bare params were unboxed
with atomint, self-call arguments were marshalled through jit_mkint, and
the self-call’s result was unboxed the same way — but the tokenizer protocol
fixes the leading params as OBJECTS and a handler returns an object. Any one
is a segfault on the first self-call, which is why a self-recursive analyser
had never worked: it died before returning. The lane now knows which
parameter positions are object-typed and honours that in the loader, the
boxing and the result. Library-only — no engine change, no new trampoline.
The core tool set is complete at forty-four applets. x-coreutils grew
from 24 specs to 48, taking the bundle from the measured core
(sort/tr/cut/join/comm, sha256sum as FIPS 180-4 in pure x) to the full
set laid out like busybox, with the scripting set beside it —
echo printf seq test [
diff cmp ls mv touch install mktemp which xargs env date and friends. The
install tree gained the half the shim migration had been waiting on:
tools/lang-kit/spec-gate.sh now ships, so kit-gated bundles stop answering
“no lang kit … upgrade x” against an installed tree — the v0.10.0 fan-out
surfaced it with five bundles red on the missing file and zero on their
suites.
The linter reports depth, on measured criteria. Three advisory rules,
one finding per definition with the numbers carried in the name: ladder
(a nested if chain branching on one variable), ladder-dict (the same,
string-keyed, ≥15 arms) and depth (released as shape; depth ≥12 and
≥500 nodes). match is
an engine primitive and measures faster than the chain it replaces (605ms vs
897ms over 40 arms and 10k lookups), so ladder has no hot-path exemption —
hot code converts first, not last — while a Dict wins only on string keys
(2.75s vs 6.90s at 25 arms) and is 5x SLOWER than match on integer keys,
which is why the fix rides the key type. depth counts NODES, not lines:
the linter reads forms as data and density runs 4.8–9.7 nodes/line, so a
line count would partly measure the formatter. The 500 is calibrated — at
250 the rule found 83 definitions in a smooth decay with no natural gap.
docs/code-quality.md records the criteria and, deliberately, the plausible
rules that failed, so they do not come back.
Two chapters written from debugging that had cost real time.
docs/crafting-a-lang.md gains garbage collection: NOTHING COLLECTS UNLESS
YOU ASK — every sweep in this tree is a hand-placed (Heap collect), so a
lang that replaces the REPL loop inherits the per-turn sweep as a DUTY and a
long session or a -f script grows without bound otherwise. And an isolated
tokenizer base does not survive collection, which turns that rule into a
dilemma for any lang bringing its own base; the doc carries a verified
seven-line reproduction and the consequence for a suite’s alloc ceiling.
apps/ has a second occupant, and a project has a face. Bitwise draws
the ASCII owl from every source header, set from Roboto Mono outlines over a
field that sha256(name) seeds, in a costume from langs.json
(x -l bitwise -- --all --png). Every quantity is an integer in micro-units,
so the picture is a function of the name alone and the gallery’s browser twin
computes the identical bytes — pinned by parity specs that compare gen.x
against the twin’s checked-in renderings.
The exact tower is exact at every magnitude. Rational arithmetic
silently answered wrong values once cross products passed 2^63 (two ~1e13
denominators multiply to ~1e26): the %rat-* internals used the raw C
binaries, which wrap. They now go through the public promoting operators,
and %gcd reduces by %int% instead of reconstructing a - b*(a/b) (the
reconstruction product wrapped for bigint-sized operands). Two bigint
defects fell out of the same probe: %would-overflow-add?’s negative
threshold itself wrapped for subtrahends past 2, promoting nearly every
negative subtraction; and demotion only ever fired for single-limb
results, so any bigint ≥ the limb base — including 17-19 digit literals
straight from the reader — stayed a stealth bigint that printed like an
int but failed eq? and raw slot ops (str->number’s own overflow verify
among them: an 11-digit all-nines parse raised a spurious “integer
overflow”). Subtraction gets its own exact %would-overflow-sub?, the
demotion window now spans every length that could round-trip, and both
bigint construction paths route through it. Pinned in
tests/x/specs/ext/rational.spec.md and ext/bigint.spec.md. The op
arbitration gained the #584 guard beside it: an undeclared typed pair
with handlers on both sides now raises the lattice’s teaching error at
the operator door instead of falling through to payload-word reads.
16K+ is no longer a cliff. The boot layer’s non-tail recursions each
put several C eval frames per element on the stack, so any walk over
~16K elements segfaulted outright — first surfaced by (Str8 make 16384
c) in x-awk’s stdin slurp, then found in the rest of the boot layer’s
non-tail recursions. Str8 make
delegates to repeat’s binary doubling; boot %map1/%mapn-go/
%append2/%filter-go recurse in tail position; %as-list’s iterator drain
actually drains; Iter ->list (behind List from-seq) is tail; the
regex quantifier state collectors got the same cure. Each is pinned by a
16K spec. One behavior change fell out and is deliberate: Str8 make
refuses a NUL fill loudly. Strings are C strings, so a NUL-filled
string is "" — the old list-encode path silently allocated n bytes
behind that empty string, and x-awk’s read wrapper leaned on the accident
as an allocation door until the doubling rewrite collapsed it to one byte
and a raw kernel read overran it (heap corruption, found and fixed the
same day). The teaching error names the real door: the str make prim
(make-str), n writable space-filled bytes.
The boot burst compiles on the engine’s own JIT, never a system
toolchain. Requiring cc at runtime was always against the law of the
tree; the burst now goes through compile-asm, with the cc lane as the
fallback where the engine ships headers, and the compile cache keyed by
engine identity so one engine’s artifacts cannot poison another’s.
Compiled analysers grew up alongside: object params, the self-param
resolved as arg slot 0, no eval in analyser mode, a negative score-set
sign that survives the trip to x1, and the lazy compile-asm stub
forwarding the fvar table.
The core tool set rides the contract now. x-awk arrived at 69 specs and
left feature-complete at 167 — the full POSIX surface: arrays, printf,
field assignment, sub/gsub, getline, RS incl. paragraph mode, user
functions, the CLI front (x -l awk -- ... over files and stdin), and
both pipe forms with SIGPIPE held off; its stdin slurp reads 64K chunks
through the engine’s raw make door. x-grep (29 specs), x-sed (21),
x-make (23), and x-coreutils (24) registered beside it, completing the
core set.
Release-day motions became tools. fan-out-langs.sh runs every
bundle’s suite against a candidate x and stages the requires-release
bumps for the green ones; bump-pin.sh rewrites the engine pin from a
release’s sidecars; diff-engines fuzzes the C and rust engines
differentially and reports the form the diverging line printed. The spec
harness hardened in passing: a batch that dies mid-batch reports the
engine’s last words, a wrapperless @lib run is refused loudly, and
SPEC_SEAM_COLLECT is the run-level door out of the seam collect.
x.sh emits %platform-release beside %install-root, and two docs
joined the tree: crafting-a-lang (the practice beside the contract) and
the bootstrap tool closure scorecard.
Logo left, and the platform grew the row that let it. apps/ is empty:
its only occupant is x-logo now,
arriving green at 83 tests / 0 failed with its examples and its pty contract
intact. Minor rather than patch, and the reason is one seam addition —
%lang-root, the bundle’s own directory — plus %batch? recovering the
meaning it is documented to have. Both are new surface a lang may rely on,
which is what a minor bump is for.
Removing a lang from apps/ moves the payload fingerprint, so this release
is the extraction rather than a tidy that followed one. x-logo declares
(requires-release "v0.9.0") because nothing earlier can serve its viewer.
Logo left the tree, and is x-logo.
It was the last occupant of apps/, and the reason that directory exists —
-l resolving apps/NAME/run.x was added for it. Its own README always
described it as “a second surface language”, which is a lang’s job
description, so the lang contract named it as the
obvious extraction the day that document was written.
It arrived green: 83 tests, 0 failed — the same 83 that ran here as
lib/logo.spec.md, against the same turtle kernel, through the bundle’s own
harness. The pty contract came too and reports what it always did (7 ok, 1
known-fail), and the examples/logo/ch1.logo pin came with it. Nothing about
the language changed; what changed is that it is now acquired, pinned, and
run against a matrix of platform releases rather than only against the tip of
this tree.
apps/ stays, empty. The second resolution step is documented behaviour
and still gated; what left is its only occupant, not the mechanism. See
apps/README.md for what belongs there and what should be a
bundle instead — the short version is that an app entry is self-booting and
therefore nailed to this tree, so anything wanting its own version number
wants to be a lang.
A bundle can find the data it ships: %lang-root (x.sh, a bundle
row in tools/contract/seam.x). import answers where do my modules come
from, and a ./-relative include-once reaches a sibling source file.
Neither means the bytes of that file, and Logo is the first lang to need
that — serve.x hands viewer.html to a browser.
The three ways to do it without this row are all wrong in the same
direction: a cwd-relative literal works only from the tree root, a path
joined onto %install-root finds the platform’s tree rather than the
bundle’s, and reading %import-roots-cell is a platform internal the seam
does not cover. The first of those is not hypothetical — it is exactly how
Logo’s viewer came to be broken in every installed tree, which is the one
environment nothing ran in.
So the wrapper says it, once, because the wrapper is the only thing that
ever knows: it is what searched langs/*/lang.xon and found the directory.
Emitted for the bundle whose entry is about to run and never for a
(requires-lang …) dependency — to the bundle that needs it, a required
lang is a library.
The gate needed a bundle, so there is one. %lang-root is bound only
while a bundle is loaded, which is a third class beside always and
installed, and a class nothing can probe is documentation rather than a
contract. tools/contract/bundles/seamprobe/ is nine lines whose only job
is to be loaded: check-seam now asserts the name is absent from a bare
he/xe/rn and present under -l seamprobe.
Arbitrary-precision decimal floating-point, the sixth tower stage
(x/num/decimal.x, class Decimal, literal suffix d). It exists because
the tower had no exact answer for a decimal fraction. Bigint takes the
integers as far as they go and float is a double, so (+ 0.1 0.2) is
0.30000000000000004 and always will be — 0.1 is not a binary fraction.
(+ 0.1d 0.2d) is 0.3d, exactly, and stays exact however long the sum
runs.
A value is a significand times a power of ten, stored canonically: trailing
zeros are stripped and the exponent raised to match, so 1.50d and 1.5d
are the same pair, zero has one spelling, and = is a pair compare rather
than a walk. The significand is an ordinary exact integer, so it promotes to
bigint on its own — the type is decimal floating point, not a fixed number
of digits.
What rounds is stated, and it is nearly nothing. +, -, * and %
are EXACT: the significand grows to whatever the answer needs. / cannot be
— 1/3 has no finite decimal — so division is the operation that rounds, to
(Decimal precision) significant digits, half-even; sqrt rounds for the
same reason. This is deliberately not Python’s decimal, which rounds
every operation to the context: a context that silently truncates an addition
is a footgun in a language whose whole numeric story is that the tower widens
rather than wraps.
It sits above float in the promotion chain, and the reason is that the
widening is exact rather than conventional: every finite double is a finite
decimal, because m * 2^-k is m * 5^k * 10^-k. So (Decimal from 0.1) is
0.1000000000000000055511151231257827021181583404541015625d — the double’s
true value, not its 17-digit short form — and a mixed pair promotes without
inventing digits. Complex still absorbs decimal, as it absorbs every real.
The literal takes a trailing d and only scores when the suffix is there, so
it never contests a token the float reader wants: 1.5 is a float, 1.5d is
a decimal, and the four-character claim beats the three-character one on the
same digits. write keeps the suffix, for the reason floats keep their point
— a printed 1.5 would read back as a double. (Decimal ->str x) is the
suffix-free text.
x/num/tower carries it across all seven generics, and x/num/decimal
registers rational and complex through the pact rather than importing them,
so the module stands on bigint and float alone.
ln, exp and log10 come with it, as series rather than as a door.
Float’s twenty math methods are each one dlsym, because libm computes in
doubles; nothing computes in 34-digit decimals, so these are implemented
here. All three reduce the argument into a window where a series converges
quickly, run at the precision plus ten guard digits, and round once at the
end — exp by halving until the Taylor series is forty terms and squaring
back (dividing a decimal by a power of two is exact), the logarithms by
folding the significand into [0.3, 3) and running one shared atanh.
They agree with the published constants to the last digit: (Decimal exp
1d) is e to 34 places, (Decimal ln 2d) is 0.6931471805599453094172321214581766.
Two decisions in there are worth naming. The fold window is [0.3, 3) and
not the obvious [1, 10) because inside it the exponent is zero: an
argument near 1, where ln m and e·ln 10 would cancel completely and the
answer’s leading digits would be guard digits, never meets ln 10 at all.
And an exact power of ten answers log10 from its own exponent, because
canonical storage has already stripped it to a significand of 1 — (Decimal
log10 1000d) is 3d, not three followed by thirty-three zeros and a doubt.
Two rewrites underneath them, together a 75x. ln 2d at 34 digits took
9.8s when it first worked; it takes 0.13s.
The first was rounding, and it made everything else faster too. Rounding a
magnitude to N significant digits was q = m / 10^(d-N) with the
remainder — two of bigint’s general multi-limb long divisions, measured at
129ms per call where the multiply feeding it took 3.5. Every series term
rounds once, so that one line was the entire cost of a logarithm. A divisor
below bigint’s base is a single limb and takes its fast path, so the
discarded digits now come off a limb’s worth at a time. That alone took
ln 2d to 0.9s, and /, sqrt, round and rescale ride the same path.
The second was the series themselves, which no longer compute in decimals.
A term is an integer at a fixed scale — v stands for v · 10^S, one S
per series — so addition is integer addition with no alignment, no exponent
arithmetic, no instance to build and canonicalise, and no rounding, and a
term that divides down to zero has fallen off the working precision, which
is the stopping condition for free. Only multiplication does anything: the
product sits at 2S and comes back with one drop. A term went from ~19ms to
~3.
The scale is the accuracy argument, and the two series choose differently.
atanh scales relative to its argument, so a z near zero — ln of an
x near 1 — keeps a full working precision instead of being measured
against a 1 it is nowhere near. exp scales absolutely, because its
accumulator starts at 1 and stays within a factor of e of it; its squarings
are the one place fixed point is wrong (each doubles the magnitude, which
only an exponent can carry for free) and they stayed in decimals.
Both rewrites were checked against the code they replaced rather than assumed: the rounder over 500 random and deliberately tie-heavy magnitudes, the series over 140 random arguments spanning 1e-40 to 1e3 including the near-zero scale case. No disagreement in either, and every published constant still lands on the same digits.
%batch? means “a file was supplied” again, for bundles too. banner.x
documents it that way and every dialect entry uses it that way, but the
bundle path passes --batch unconditionally — it is what keeps the
dialect’s launcher quiet while the bundle loads on top — so every bundle
saw true whether or not the user named a file.
The two questions were the same question until bundles existed. x-sweet’s
(unless %batch? (%banner)) has silently never fired for want of the
distinction, and x-logo’s entry has to choose between its REPL and its batch
reader on exactly this fact: with the wrong answer it consumes the launcher
x.sh appended and reads it as a Logo program, which is silent and prints
nothing. x.sh now restores the flag’s documented meaning after the dialect
boots, rather than by withholding --batch, which is still doing its other
job.
The engine pin moves to x-engine-c v0.1.6, and x-r7rs goes from 43 failures to 27 without a line changing in it.
v0.1.5 adds (base def-global): an operative can define for its caller,
whatever the frame depth. def decides global-versus-local by save-stack
depth, so a define written as an operative — which Scheme’s and Kernel’s
both are — binds in its own frame and loses the binding when that frame pops.
The sixteen are all of error, error objects and guard. guard is the
live case: R7RS guard and x’s guard are different forms sharing a name,
so providing one means shadowing the other, and shadowing interposes exactly
the frame that broke the eval! workaround. x-r7rs changed nothing — it
already preferred the primitive and fell back when it was absent.
tools/contract/langs.x ratchets that budget to 27.
The pin names v0.1.6 rather than v0.1.5, and the extra release is one
line. v0.1.5 added the primitive to the engine’s isa.x without regenerating
its x-engine.xon, so it shipped the digest of the previous manifest and
check-engine-contract refused it — the gate doing exactly its job, on the
first pin bump that could reach it. v0.1.6 is that file regenerated; nothing
in the built engine differs.
tests/x/fixtures/engine-min grows the same row. The paper engine declares
core and no more, def-global is labelled spine, and a group missing one
coordinate is not a capability — so the fixture stopped claiming core the
moment the vocabulary grew. check-second-engine predicted this in its own
header, which is the argument for fixtures over remembering.
syntax-rules works. The reader stopped claiming the dot, and x-r5rs went
green without a line changing in it.
The engine pin moves to x-engine-c v0.1.4, and x-r5rs goes green without a line changing in it — 667/9 to 667/0.
v0.1.4 stopped the dot being one of the single-character tokens. It sat in
X_SEXP_LIST_CHARS_STR beside the brackets, so the analyser scored it on
sight: correct for ( and ), which really are always single-character
tokens, and false for ., which separates a pair only when nothing follows
it. Every token beginning with a dot was taken whole as the separator, and
the reader’s internal marker was returned to the caller as a value — so
(first (rest (lit (a ... b)))) segfaulted on a raw C satom.
All nine of x-r5rs’s remaining failures were ellipsis patterns, and R5RS’s
macro layer is written entirely in them, so one reader fix moved every one.
tools/contract/langs.x ratchets that budget to 0: a zero is a claim, not a
hope, and it is what makes a tenth failure loud.
The engine takes no view on ... — it is a symbol the reader does not
recognise and passes through, exactly like .foo. One reading changes with
it: (a.b) is the symbol a.b rather than an improper list, which was an
accident of the delimiter set rather than deliberate syntax.
The lang kit: the platform ships a bundle’s checks, as it already ships the
spec runner (x-lang#546). tools/lang-kit/ installs to
share/x/tools/lang-kit, beside tests/spec-runner.sh and outside the
payload fingerprint for that file’s own reason — the digest is library bytes
and answers which release is this, while a tool ships with the wrapper and
the engine.
docs/lang-contract.md settled the ruling for the runner: the platform
ships it; bundles do not vendor it. The runner was the first thing every
bundle would otherwise have copied, and it is not the only one. Across the
six published bundles tools/bundle.sh is 103 lines and differs between any
two by exactly 12 — every one of them the lang’s name or its release URL —
and tests/spec-gate.sh differs by a single line of header comment.
The first member is release-refs.sh, chosen because its evidence is the
freshest rather than because it is the largest: it was written twice within a
day, and the second copy needed three fixes backported the same afternoon —
an issue reference mistaken for a release, a greedy regex window pairing a
name with the wrong version, and a look-back that spanned a neighbouring
pair. A seventh bundle would have been a seventh copy of all three.
It is generic over the manifest rather than per bundle: the table of versions
to check is read from lang.xon — (requires-release …) plus any
(requires-lang …) — so one file serves a bundle declaring one and a bundle
declaring two.
Two ways to reach it, and the second is not a convenience. A bundle’s
specs job has already checked x-lang out; its contract job has not, and does
not need to, because this check reads a manifest and greps a tree. Making a
one-second text check depend on a built engine would be a regression dressed
as consistency. So X_LANG_KIT names a checkout’s tools/lang-kit directly,
and everything else asks x --share-dir — the addressing the runner already
uses, and the failure the contract calls “addressing, not sharing”.
For bundle authors: a bundle that sources the kit declares
(requires-release "v0.8.0") or later. Earlier releases do not carry it, and
the shim says so by name rather than failing obscurely.
(Num expt b -1) no longer takes the machine down (x-lang#545). The
parameter was documented “Non-negative integer exponent” and nothing checked
it. A negative exponent never reaches the (= exp 0) base case — it walks
away from zero — and every even step squares the base, so the recursion
allocated bignums that doubled in width until memory was gone. That is what
made it dangerous rather than merely wrong: a plain infinite recursion trips
the spec runner’s timeout, while this one exhausts memory first, and
alloc-limit! is calibrated in objects, so a handful of enormous bignums
reaches far more RAM than the object count implies.
It was reachable by accident: 2 ** -1 is ordinary Python and (expt 2 -1)
is ordinary Scheme, and any lang implementing exponentiation on top of
Num expt inherited it.
expt now raises err:value on a negative exponent, just as
Num isqrt already used for a negative input. The doc stated the contract;
this enforces it rather than changing it — a caller wanting Python’s or
Scheme’s answer needs a float, which is the caller’s decision to make and not
a silent reinterpretation here.
The engine pin moves to x-engine-c v0.1.3, and x-ash goes from unusable
to two failures without a line changing in it. v0.1.2’s
x_prim_make_token_base segfaulted on the first character of any input:
it assigned to the boolean singletons’ cells rather than through them, and
never created the read buffer make_base gives itself. An isolated tokenizer
base is the one thing x-ash cannot do without, so nearly its whole suite was
red for a reason that was never x-ash’s.
| before | after | |
|---|---|---|
| x-ash | 80 failed | 2 |
x-ash’s two are its own string readers, a bundle bug its README documents.
v0.1.3 also names the buffer sizes it had scattered as bare literals — the
reader’s buffer was 256 bytes in make-base and 65536 in the CLI, filling
the same field.
tools/contract/langs.x corrects two attributions and one number, and
the corrections are the point of a file whose budgets are supposed to be
measured rather than transcribed.
x-r5rs’s 37 → 9 was credited to this pin. It was not the engine: v0.1.3
changed four files, none of them the reader, and
src/x-token/sexp/list.c is byte-identical to v0.1.2’s. The 28 came from
x-r5rs itself — R5RS §6.6 ports rewritten against File (21), and exactness
under §6.2.5 (7). The nine that remain say so: they are all ellipsis
patterns, exactly what a pair-dot fix would have removed.
(base def-global) was described as shipping in v0.1.3. It is not in v0.1.3;
it is proposed and unmerged.
x-r7rs’s budget moves 58 → 43. 58 is what that suite reports when
something else is running — an orphaned engine holding a core has made it say
49, 58 and 247 for one unchanged tree, with batches dying mid-run.
check-langs runs six suites in sequence, which is precisely that condition.
Measured twice on a quiet machine it is 43, on both engines.
0.6.0 let a lang leave the tree. This one is about langs that build on each other, and about the platform noticing when it breaks one.
A lang can now name another in its manifest, at an exact version, and be
refused at startup rather than discovering the gap as an unbound symbol
somewhere inside itself. make check-langs runs every bundle’s own suite
against this working tree, which is the first time anything here notices a
behaviour change a bundle cares about — measured when the gate was written:
x-lang green at 2590/0 while the six bundles carried 175 failures between them,
with nothing here saying so. And the two lines at the top of every lang
extraction — a second (include "lib/x-core.x") — no longer segfault, which is
the first wall a new bundle hits and the one that told it nothing.
Indent is the same story one layer down: Logo and x-sweet each owned a copy
of the indentation algorithm, disagreeing at the edges, and now drive one
module.
Minor rather than patch. A new public reader module, a new manifest row, a new wrapper flag and a new gate — new surface of this size belongs in the minor under the SemVer this file adheres to, and nothing published depends on what changed underneath it.
make check-langs — every lang bundle’s suite, against your working tree.
check-seam catches a rename in eight seconds and cannot catch anything else:
a behaviour change, an arity change, a reader that scores a tie differently
all leave this tree green while a bundle in its own repository breaks, and
each bundle’s CI runs on its own schedule against a release, so the break
surfaces weeks later as somebody else’s mystery. Measured when the gate was
written: x-lang green at 2590/0 while the six bundles carried 175 failures
between them, with nothing here saying so. Budgets live in
tools/contract/langs.x and may only shrink, the rule percent-globals.x
runs on. Advisory about presence — a bundle not on the disk is announced and
skipped, because this tree must build for someone who cloned nothing else —
and strict about regression. Deep tier; LANGS='krn sweet' runs a subset and
X_LANGS_DIR moves where bundles are found.
Indent — the stack discipline under indentation-sensitive grouping
(lib/x/reader/indent.x). Logo and x-sweet each owned a copy of it, reached
by different routes and disagreeing at the edges; both now drive this one.
Two layers: measurement (advance / scan / measure) answers what
column a line begins at, and the stack (make / feed / close-all)
answers what opened and what closed. feed returns zero or more close
events followed by exactly one open or same, so no caller counts levels
itself — the loop both previous implementations owned. The two policy
questions the surfaces disagreed on are constructor parameters, not
assumptions: the tab stop, and what a dedent matching no open level means
(open, close or error). Defaults are SRFI-110’s, which are also
Python’s. advance / scan / measure / classify are registered under
catalog ns indent for per-character callers who must not dispatch, the
discipline x/reader/analyser’s terminators already use. (#520)
A lang may require another, and lang.xon says so —
(requires-lang "NAME"). x.sh resolves it exactly as it resolves -l,
depth-first and transitively with a cycle guard, and arms each required
root before the requiring bundle’s own — import-path! prepends, so a
bundle still wins any name it shares with something it builds on. A
missing lang is a refusal at startup naming what is wanted and who asked,
rather than an unbound symbol somewhere inside it. Before this, x-r7rs
found the x-r5rs it extends by probing sibling paths in two separate
places, which recorded nothing about what was needed and would have been
re-derived by every dependent. The same argument (dialect ...) already
makes: a requirement belongs in the manifest. (#526)
A required lang may be pinned to an exact version —
(requires-lang "r5rs" "v0.1.0"), compared for equality and never
parsed, as (requires-release ...) already is. Ordering and ranges would
need a version algebra and, with it, a resolver; equality plus
--allow-lang-skew is smaller and says what it means. What it compares
against is derived, not declared: a bundle carries no version row of
its own, because such a row can only be true at the one commit that gets
tagged — before the tag the tree claims a release that does not exist, and
after it every commit claims one it is not. Instead a bundle’s
make install and tools/bundle.sh stamp a version file from
git describe and from the tag, the same split as this tree’s own
$(X_RELEASE) → <lib>/contract/release. A lang that reports no version
is refused as its own case, because it means a checkout rather than an
install and the fix is different. (#526)
apps/logo/indent.x keeps what
was ever Logo’s — what a block is, and where the tokens go — and its
(indent-level . tokens-reversed) stack and %pop-to are gone. Its two
answers are stated rather than implied by a loop: a tab is one column (a tab
stop of 1), and a dedent to a column no open block sits at opens a block
there (open). apps/logo/types.x measures the leading run with the shared
scan, which hands back the column and the end index separately — the loop it
replaces stepped an index and returned it as a column, which is correct only
while a tab is worth one. (#520)<space><tab>x, +8 says column 9 and a tab stop
says 8. Neither suite had a tab case in it, so nothing caught it in either
direction. The shared module carries the corrected reading, and it subsumes
Logo’s answer rather than overruling it — the next multiple of 1 after n is
n+1. (#520)The release that lets a surface language leave the tree. A lang — Kernel, Scheme, Logo — can now be built, published, acquired, installed and run from its own repository, and the platform is held by a gate to what it promises one. The five 2024-era personalities rotted because nothing held either side to anything; this is the machinery that would have caught all three ways they died.
(Pin bundle "deps/langs") reads a
project’s lang.pin.xon, downloads the tarball its (bundle "sha256:…"
"URL") row names, digests it before tar runs, and publishes it only
once the whole tree verifies. A mismatch quarantines the bytes as
.rejected rather than deleting them. The terms both sides are held to are
The Lang Contract. (#521, #533)x --install-lang URL, with nothing cloned — fetches a published
lang.pin.xon, then the tarball it names, and installs to
<install-root>/langs/<name>. Install and pin answer different questions
and the contract now says so: an install is one unversioned copy for the
machine and the prompt, a pin is a digest-frozen tree for one project. A
failed upgrade leaves the working install untouched. (#534)-l NAME runs an acquired lang — the third resolution step after
lib/NAME.x and apps/NAME/run.x. Unlike those, a bundle does not boot
itself: the wrapper boots the dialect its lang.xon declares, arms the
bundle’s module root, and loads it on top — the same way -F has always
worked.
So a bundle needs no root-relative literals at all. A dialect this tree
cannot supply, and two bundles claiming one name, are refused before
anything boots. (#530)tools/contract/seam.x declares the eleven names a lang
may rely on, and make check-seam holds the running platform to them in
every dialect. A lang lives in its own repository, so a rename here that
drops %repl-prompt or import-path! breaks it silently while this tree
stays green. Declared rather than derived, because a lang’s call sites are
not in this tree and never will be. (#531)(Sha256 hex-n s n) — the digest, with the length given. hex bounds
itself by Str8 length, which has strlen semantics, so binary input digests
as its leading fragment: a gzip whose fourth byte is a NUL digested as three
bytes. Both engines were already length-driven and byte-accurate; only the
derivation was narrow. The JIT’s adoption check gained two binary vectors,
since proving the two engines agree on text proved nothing about the case
this was added for. (#524)x --share-dir and x --engine-path — a tool outside this repository
can ask where x reads its tree from and which engine it runs, instead of
guessing. The shared spec runner now installs to <share>/tests/, so a lang
runs its own suite with the platform’s runner rather than vendoring 865
lines of it. (#514, #532)%lang-name all along. surface was unavailable: it is already
load-bearing for the C surface the engine contract is built on, and taking
it would give one word two concepts. The Linux personality syscall keeps
its name, because it is the kernel’s and not ours. (#530)deps/, not build/ — the engine, its sources
and lang bundles. build/ is what this project produces; none of these is
built here, and filing them under output is why engine needed a symlink to
be findable at all. Sixteen files reach the engine through engine/ or
$(ENGINE_DIR) and not one of them changed — the link’s own promise, kept.
(#533)SPEC_HEAVY_JOBS is 2 only
where there is memory for two big heaps (~13GB), 1 below 24GB, and 1 when
the size cannot be established. The runner’s own comment said to lower it on
a small box; a default that is safe only when the caller remembers a comment
is a note, not a guard. CI is unaffected: both legs set it explicitly.
(#529)serve.x read its
template from a cwd-relative "apps/logo/viewer.html", which resolves only
when the cwd is the repo root. Every test and every developer run found it;
no installed user did. The path-literal ratchet could not see it, because it
matches include forms and a data path is neither — so that gate grew a
section for app data paths. (#512)--share-dir refused to answer from outside the tree, which is the one
thing it exists for. Mode detection is cwd-based by design, so from outside
a checkout it took the installed branch and computed a share/x no checkout
has. Found by the first lang to use it, which had worked around it by
cd-ing to the wrapper’s directory first. (#532)eval! — x-krn sets
%repl-print on its first line of real work, and its $define! turns on
eval!: 59 of 72 specs fail without it. The table had been written by
reading what the platform offers; a lang reaches for what it needs.
(#532)git archive
--prefix=NAME/ is how a publisher rolls one, and every tarball the feature
was developed against was flat. Descended one level, and only when
unambiguous. (#533)The first release built alongside a second engine: two undeclared assumptions became declared capabilities, eight unwritten laws became written ones with conformance checks, and a pinned project now runs the release it names instead of reporting an errand.
$XDG_CACHE_HOME/x/releases) — verified before unpacked, staged and
published atomically — and hands the whole invocation to it. Fetching asks
consent: --fetch-release, or one question on a terminal. Nothing global
changes; the re-exec’d wrapper runs its own guards against its own matched
tree, and --allow-release-skew still means “run THIS install”. The
refusal survives for the unresolvable cases and now names the flag. (#499)provide shadowing the
lower. Chains (@3 over @2 over @1) fall out of the same rule; outside
a self-load the loud version contract is unchanged. (#503)native/cc (the
engine ships its C headers, so x/tool/compile.x can build against them)
and the jit/asm lane are now stated in the engine contract instead of
assumed. The compiled tower falls back to the interpreted one on an engine
that hosts no C, and the jit/asm specs skip against an engine that never
claimed the lane — both assumptions had silently encoded “the engine is
x-engine-c”.docs/engine-laws.md and checked by conformance: the stamping law (a value
carries its base’s type), the evaluator’s state rides the base, evaluation
and application are type handlers, the callable state spine, and the rest.-ts suffix retires with the
phrase it abbreviated. The engine contract’s covers rows use the ISA’s own
spellings. (#497, #498)first/rest walk encoded x-engine-c’s base spine (the one thing
decision L1 says a caller must not do) and pruned through the wrong cell on
any other layout. The (B cell 'type-alist) door resolves whichever engine
is underneath.(Pin boot "v0.5.0") failed for every project — pin: unknown
release-manifest form. v0.5.0’s pin.release.xon carries the
(engine-release …) row the release script now emits, but the pin
tool’s manifest parser predated the row and refused the whole
manifest. The parser reads it now, optional like layout — a manifest
without the row still parses, a missing value reads back as nil. And
since the pin tool ships inside releases, the fix has to be a
release: v0.5.0 itself cannot be pinned, and this line exists so the
v0.5.x series can be.The object model’s second architecture: one routing model with four doors, and the composition features that shape dispatch tables without changing how routing works.
def-trait defines a behaviour bundle and with composes it into a class; delegates makes forwarding a declared relationship rather than hand-written pass-through methods.def-generic and on give multi-argument dispatch as the cold path beside message passing’s hot one: pointwise specificity, from-lattice tie-break, and teaching errors when no method applies. x/num/tower is the worked example, and the tower’s mixed-type policy on generics is now stated rather than implied.def-record for lightweight named-field data types; Tls ported onto it as the first occupant.def-method! and def-static! add to a class after its definition, with a %missing hook for selectors that resolve to nothing.(private …) and (protected …) blocks, enforced at the dispatch door. Reflection is the documented escape hatch; a leading % stays what it always was, a naming convention.The C engine is a separate repository, and this tree carries no engine at all — x-engine-c, with its own version line and published releases. make engine reads tools/engine/engine.pin.xon — a declared release tag plus one artifact row per platform, digests taken from the release’s own sha256 sidecars — fetches the tarball, verifies it, and unpacks it; make engine-source clones the sources instead for platforms with no published artifact. x-bin is copied to this repo’s root as before, so the wrapper, the spec runners and every tools/check/*.sh script are unchanged. This release pins x-engine-c v0.1.2, whose v0.1.1 and v0.1.2 lines are this release’s own finds: a prim walked off a dotted argument list (#487) and the FFI called a nil function pointer (the #171 class) – both uncatchable SIGSEGVs, both now catchable raises.
The contract manifests travel with the engine, in its tools/contract/: an engine’s description of itself is stronger there than as a copy the library keeps, because a private layout copy is right about some engine and not necessarily the one running. lib/x-core.x includes base-paths.x and obj-layout.x from there as the first things it loads, and pin.x reads isa.x at runtime. check-isa, check-obj-layout and check-base-paths are the engine’s own gates now; check-prim-coverage stays here, because most primitives are reachable only through the library and the honest answer needs both spec suites.
The engine declares its own release: x -V prints the library’s release and the engine’s side by side, and a released engine ships no C — the C-suite gates announce-and-skip here, its own repository having ratcheted them on the build that produced the artifact.
x-lang reaches its engine through one path — engine, a .gitignored symlink in the repo root that make points at whatever the tree builds against: a fetched release, a checkout named by X_ENGINE_DIR, or any unpacked engine directory. Everything downstream uses that one spelling — the boot’s contract includes, the JIT’s -I flags, the gates, the conformance runner — so a second implementation needs no edit to lib/. An engine directory either has sources (built here) or ships a binary (used as-is); the build asks which rather than assuming. Once pointed somewhere explicitly the link stays there, and make install stamps the ISA fingerprint from the engine it actually built against, not from a hardcoded path.
method-of as the sanctioned de-dispatch door; value-call subject-last is routing sugar into the same door; generic functions are the multi-argument cold path; and the C ops cell’s seven spellings shim into the tower’s generics, so promotion has one authority. Composition (with/delegates), contracts, records, open classes and %missing are table-shaping features that never change routing. Measured: a 110-entry static back-hit went 231 → ~100 µs/call, and method-of-hoisted calls sit at ~31 µs against a plain fn’s ~21.%this-class box replaces the %super-class binding; field and static writes go in place through %box-put!; the dispatch arg frame is tail-evaluated.Library version 0.5.0 — x-lib-version had read 0.3.0 since before v0.4.0 shipped, so the banner and x -V under-reported the library by two releases while claiming precision. docs/spec.md, docs/standard-library.md and the README’s maturity line move with it.
The pin records which engine a project was verified against — a lock’s
(engine-release …), compared before a pinned amalgam boots and refused on a
mismatch (--allow-release-skew waives it, loudly). x-lang no longer stamps
its own tag onto the engine it builds: x-engine-c has a version line of its
own, so an installed tree now carries two facts — contract/release for the
library and contract/engine-release for the engine — and each refusal names
which one failed. Both rows are optional; a lock written before them announces
that the pairing is unchecked rather than being refused.
ext/x-engine-c submodule is
gone. make engine fetches the release tools/engine/engine.pin.xon names
for your platform and verifies it against a recorded digest; make
engine-source clones that release and builds it, which is what a platform
with no published engine (the Pi, 32-bit) takes automatically and what the
sanitizer, coverage and engine-hacking flows ask for by name. A tree that has
never acquired one prints those options instead of failing obscurely. Cloning
x-lang no longer needs --recursive, and building it no longer needs a C
compiler.(= 1.5 1.5) segfaulted, uncatchably, from ordinary source text: with no float module loaded 1.5 reads as the dotted pair (1 . 5), and a primitive walking its argument spine tested only for the proper ending. A proper list bottoms out at nil; an improper one bottoms out at an atom, which was then read as a pair — the tail integer’s value word dereferenced as a pointer. No guard could catch it, because a prim call never enters the applicative walk #69 guarded. Which primitives crashed was decided by the library, not the engine: + and < raised cleanly because the tower shadows them, while the unshadowed %isa-keep entries =, eq? and same? reached C directly. Fixed in the engine (x-engine-c#5) by hoisting #69’s own structural test into one guard every C spine walker shares, and picked up here by pinning v0.1.1. A satisfied arity still ignores a junk tail, and a non-pair argument remains the unchecked contract it always was.lib/ and apps/ decides everything, because an amalgam is self-contained only over its include closure and resolves its imports against the installed tree as it boots. See #467.) Three things now carry release identity: the engine reports its own tag as x-release (x -V prints it), an installed tree is stamped with that tag and with a payload fingerprint — one digest over everything the release ships as library — and pin.release.xon publishes the same payload fingerprint beside the ISA one. The wrapper compares the lock’s release tag against the engine’s before the amalgam reaches it, and refuses the pair, naming both tags and both remedies; --allow-release-skew, or (allow-release-skew) in the manifest, waives it loudly for anyone who means it. (Pin verify) reports the same comparison without enforcing it. The tag comparison works on locks written long before this change, because Pin boot has always recorded it.Pin sync classifies a project’s own modules and never vendors them (#223) — and check’s audit half arms the project’s own roots too, so a project whose sources import its own modules can finally sync and check. Previously sync copied the project’s modules into its own overlay, where the stale copies shadowed the real files.Pin boot writes the lock where the guard will find it (#313’s write side) — the boot pin’s engine-pairing guard no longer silently stops firing when overlay roots are reordered.The first release since 0.2.0: the 0.3.x development line and the 0.3.1 release candidates, shipped together. Reproducibility is the theme — a project can now freeze the language it runs on, and a release ships the engine to run it.
pin.xon manifest (root overlay, src scan tree, boot entry) and the Pin verbs over it: init, boot, sync, check, vendor, verify, closure, fetch, resolve, unused. Two pin mechanisms, because drift takes two forms: an overlay pin freezes the library modules a project imports — vendored closure-wise, digests in a lockfile — and a boot pin freezes the language itself by committing a released amalgam. The lockfile records the release tag, that release’s ISA fingerprint, and the amalgam’s digest; the wrapper arms both pins, announces them on stderr, and refuses to boot an amalgam whose fingerprint doesn’t match the running engine.import-version-once / import-version select among sibling @-suffixed version files by spec string ("1.3", "1.3.*", "^1", "*"). Files are append-only, so a fix is a new patch file and every import whose spec admits it picks the fix up on its next run; dedup keys the base name, and a loaded version that doesn’t satisfy a later spec is a loud error naming both sides. Pin resolve is the dry run, Pin unused the safe-removal answer (#214, #215, #216)..sha256 sidecar, so a release runs with no toolchain and no compile. bootstrap.sh covers the from-source path in one command: clone, acquire the engine the pin names, build it (a ~4-second C89 compile), and optionally install under a user prefix.Socket (#374), then the REST tier on top: https via libssl FFI, DNS, the verbs, and Rest (#412), with basic auth, bearer-token auth, and auto-followed redirects.File stat/lstat adopting it (#371), and base64 and hex (#362).$"…" string interpolation — holes hold code, and the literal scans as one token instead of shattering into fragments (#292); the quote, quasiquote and unquote’s char codes become char literals. Adopted across the library wherever an output call was a text template (#291).Path (#225), a Proc layer over Sys with one correct way to spawn (#226), File read-all/write-all, and the coverage tail: Pq, Deque, Counter, Random uuid, Str8 wrap/fill (#375), from-iso, UDP and unix sockets, Proc options, walk, glob, relpath, copy, temp (#364).lib/he.x, light/default — the old lib/x.x surface, byte for byte), xenon (lib/xe.x, stable full tower — the old x-and), and radon (lib/rn.x, experimental — the old x-or). Atomic weight = library weight, radioactivity = instability; and the governing rule: dialects may differ in what surface is loaded, never in what a shared spelling means. x-lang reverts to being the language’s name only; banners show the full element word (xenon v0.3.0 on x-lang); -l flags stay terse (-l xe). The module layer follows: x/and → x/xe, x/or → x/rn.-l x-and / -l x-or and (import x/and) / (import x/or) no longer resolve (transitional shims existed only within this release cycle); an unknown -l name fails with the wrapper’s inventory listing. lib/x.x remains as the default pointer (bare sh x.sh boots helium), and check-doc-vocab now ratchets the retired spellings out of lib/.lib/x/boot/{helium,xenon,radon}.x; each entry is a body-include plus the top-level launcher (a (repl) cannot ride a nested include: it would read the included file’s EOF instead of the session’s stdin).examples/and/ → examples/xe/, examples/or/ → examples/rn/.lib/x/platform/x86_64.x) — cmp (rr/ri), the six conditional branches (b/eq b/ne b/lt b/ge b/gt b/le as Jcc rel32, sharing arm64’s mnemonic names), a b alias for jmp, and per-arch asm-prologue!/asm-epilogue! (SysV frame + rbx/r12-r14) and asm-load-imm64!. The JIT codegen module (asm-compile.x) remains arm64-only (registers hard-wired) — tracked separately.Arch-labelled specs — the spec runner skips <name>.<arch>.spec.md files on non-matching hosts (uname -m, arm64/aarch64 and x86_64/amd64 normalized); asm specs split into .arm64./.x86_64. variants since the scenarios are ABI-specific (A64’s x0 arg-and-return duality vs SysV’s rdi-in/rax-out)
heap-mark-hook!, heap-free-hook!, heap-mark-root! primitives wired through to x-expr’s heap-fields (released as heap-group) extensible lists; lib/x/sys/gc.x is now a thin re-export layeropt/ — first occupant is opt/x-prim/signal.c; gated by X_SIGNAL (default on), make X_SIGNAL= drops the module and compiles the eval poll outexamples/logo/ch1.logo — Chapter-1 programs from Turtle Geometry (ARCR/ARCL, RAY, POLY/NEWPOLY, POLYSPI/POLYSPII, INSPI)tests/x/specs/applicative/gc-hooks.spec.md (STRESS-only)Object-oriented class system (lib/x/type/class.x) — classes are themselves callable %class objects; instances are %object. Message-passing dispatch with literal selectors ((obj name args), no quotes — the call handler is an operative), single inheritance with super, and a (static …) block of static methods + static fields so a class doubles as a namespace ((Class name), (Class new …)). Fields are declared directly in the class body (no wrapper) with a uniform form — name |
(name default) |
(name default "desc") — identical in the static block; instance fields gain optional default values. Access is encapsulated (external reads/writes only via dispatch; method-internal (field 'm)/(set-field! 'm v), released as (member 'm)/(set-member! 'm v), for the private-data pattern). (help Class) lists fields and methods grouped static-vs-instance, merged across the inheritance chain and sorted by name. Spec: tests/x/specs/ext/object.spec.md; guide: docs/object-system.md |
lib/x/type/lit-reader.x) — 'expr is reader shorthand for (lit expr) ('sym, '(a b), ''x, and ' as a terminating macro char). The analyser is JIT-compiled in x/and and x/or so it doesn’t slow tokenizing. Spec: tests/x/specs/core/quote-reader.spec.mdBignum → Bigint (#356), Token → Analyser and StrUTF8 → StrUtf8 (#359), the cross-class verbs unified (#358), and the R7RS method names retired (#357). Pre-1.0 surface churn, done in one pass rather than a drip.make no longer mutates the binary — the strip is stamp-gated (#367), and per-variant object suffixes retired the clean-obj brackets (#329).x_base_* → x_interp_* across the interpreter source tree; the file formerly at src/x-base.c is now src/x-interp.c. x_base_* names are reserved for x-expr’s library-level skeleton (file descriptors, hooks, heap-fields, released as heap-group); x_interp_* covers the environment/control/extras half this project fills in.extras group into x-expr’s heap-fields (released as heap-group) — one canonical storage location for everything GC, registered by name via x_heap_{mark,free}_hook_add() / x_heap_mark_root_add() instead of raw (rest (rest …)) path-walking from x-lang(doc …) forms stash raw metadata at load time; the full processor runs only on first (help)/(apropos)/(modules) invocation (~1s startup savings)(list (lit name) ;N …) to (lit (name name …)); ~1000 lines lighter, same in-memory shapelib/x-and / lib/x-or module-loading layer tightened — drop duplicate posix re-imports (x-core already loads it); pre-compile quasi/unquote reader analysers in x-or so subsequent file parses aren’t ~20% slower; make x/or’s system extensions (syscall/file/socket) opt-in to save ~660 lines per startupmake test-asan promoted to a hard CI gate — the AddressSanitizer baseline reached zero (112 findings at the gate’s introduction → 0): the under-read fixes below plus pinned ASAN_OPTIONS (detect_stack_use_after_return=0; stack-copying call/cc is fundamentally incompatible with ASan’s fake stack, as with any fiber library). A red ASan job is now a real memory-safety regression.H shuffles fold into the compiled loop, and the build is gated on payload size so a small artifact still verifies in pure x (#123, #324).List family walks become iterative inner loops with one normalization (#336), the regex matcher stops rebuilding group splices (#337), bigint division carries an MSB view instead of re-reversing per digit (#341), and/or become nested if in per-element loops (#343), int-only sites fetch cached int prims (#335), and the doc, fmt, cov and lint walks shed per-node work (#338, #339, #342, #344).** glob hole closed (#321, #322).-7.5 was never claimed by the float type and fragmented into -7, a stray ., and 5. A - entry state (requiring a digit next, so the minus operator stays a symbol) fixes it; pure x-lang, no C.(do 7 . 5) / (begin 7 . 5) (e.g. from a malformed literal under a lib without the float type) walked the improper tail into the unchecked rest prim and evaluated a value word as an expression (SIGSEGV). The check lives at the x level, in do/begin’s boot walker: the C core is the processor and does not bounds-check; the walker that accepts the program does. Boot-safe via catalog-fetched type prims and per-dialect probe handles (reader cells and pair-prim cells carry different types).% on floats returned garbage — the float type registered + - * / < = ops but not %, so (% 1.2 1.4) fell through to x_prim_mod’s integer fallback: value-word modulo on two float payload pointers ((gcd 1.2 1.4) famously yielded 8). Floats now dispatch % to a new d%d FFI convention (fmod, matching %’s truncated-division semantics; -lm added for Linux). Rational and complex still lack % ops and inherit the garbage fallback — noted for the tower’s next pass.included the whole lint tool (~150 MB of objects each, never collected: the harness doesn’t GC between snippets), so the batch OOM’d any small-RAM box. macOS appeared fine only because memory compression hid it (peak footprint told the truth). Now loads once per batch via # @lib ../tests/x/lib/lint.x, matching every other tool spec. Investigation notes: object counts and sizes are identical across platforms, and jemalloc matched glibc byte-for-byte — there was no leak and no allocator pathology, just honest accounting on Linux.type?, type-name, units, length, and the write/display hook dispatch) navigated x_type_field_* on whatever sat in an object’s type slot. A child base’s slot holds the x_eval_obj sentinel (a static atom tagging the raw string "BASE"), so e.g. (pair? (Base make)) read 8 bytes past the tag string (ASan global-buffer-overflow) and worked only because the garbage compared unequal. All six now use x_type_op_try’s documented guard: only a pair-tree type has fields; sentinel-typed objects get defined fallbacks (type? → #f, atom units/length, default repr).&local ordering against the stack base) and the segment copies tripped the memcpy interceptor on other frames’ redzones. The capture/restore functions are now exempt (no_sanitize_address, which also keeps the setjmp frame in the captured segment) and copy through an uninstrumented byte loop under ASan; plain builds are unchanged (memcpy, empty attribute).%collect-methods tested (eq? (first (first forms)) (lit method)) without a pair? guard, so a bare field name (a symbol) had its name buffer dereferenced as an object — an out-of-bounds read that 64-bit malloc tolerates (garbage compares unequal, so bare fields were skipped by luck) but ASan flags and 32-bit/Pi can segfault on. This was the tracked “eq?/match under-read” blocking make test-asan from hard-gating.&local, missing frame slots the compiler placed below it (gcc spills p_base/cont there); clang’s register allocation masked it. Capture now bounds from a non-inlinable callee frame, and the restore descent keeps a two-pad margin so the memcpy can’t clobber the live restore frame.%asm-arm64? matched only Darwin’s “arm64” spelling, loading the x86_64 backend on aarch64 Linux(op …) definition time, not the caller’s environment at call time. Co-issue: a C-spec for procedure_call / operative_call was updated to match.x_alist_bst_insert no longer path-copies, so fn closures that captured a BST snapshot at definition time stay valid as later globals are added. This was the root cause of an intermittent turtle test failure (>= unbound during include-once of float.x).syscall-id self-parameter — was declared (fn (call) …) which left the actual argument slot empty; one-arg call sites were working by accident. Now (fn (_ call) …) per x-lang fn convention.x_heap_*_hook_add and x_heap_mark_root_add were replacing the whole stack-cell slot instead of pushing into its current list. After one registration, the slot was a one-deep cons cell whose first IS the hook, and the collector walk crashed on the first non-pair internal field. Fixed to push into first(cell).(heap-collect) is now atomic — the env/ctrl/extras base-tree cells and eval-list scratch cells are allocated X_OBJ_FLAG_NONE, so they survive a sweep only by being marked. The old heap-collect = (applicative heap-mark heap-sweep) let x_eval_body push a fresh eval-list cell between the mark and the sweep; that cell was allocated after the mark (so nothing marked it — conservative C-stack scanning is part of the mark and had already run), got freed mid-traversal, and the next pop dereferenced freed memory (any (heap-collect) invoked from inside a begin/do/op body — including the spec runner’s per-test wrapping — could SIGSEGV). heap-collect is now a single C primitive doing mark+sweep with no allocation between; its mark marks its own in-flight frame. The raw (heap-mark)/(heap-sweep) remain exposed but are low-level. GC hooks are now driven through the TCO trampoline so a value-returning hook body doesn’t leave a half-finished call for the sweep to free.x_obj_push_field/x_obj_pop_field exports + OOM-via-x_obj_error in x_obj_alloc, static-library build (make lib) + make install/uninstall + make doc, README rewrite with quick-start + API table + examples/hello.c, GitHub issue/PR templates, contributing guide, and the per-pass GC hook/root list fields plus their registration helpers and the push-into-first fix above.provide/import with include-once deduplication and module registry; (modules) discovery command#/pattern/ literal syntax and compiled pattern matcher(doc ...) forms with (param ...), (returns ...), (note ...) metadata; auto-generated Markdown referencedocs/spec.md) mapping 1:1 to 1229 test cases#() reader syntaxp_base-as-nil to nil = NULL; () parses to NULLlib/x.x into 50+ modular files under lib/x/operatives, data, string, module)cons/car/cdr renamed to pair/first/rest throughout; string-* renamed to str-*fn (applicative), op (operative), wrap/unwrapguard