buffer.c File Reference

Buffer and tokenizer primitives – buffer-*, token-read(-string), buffer-token, buffer-last-char. More...

#include "x-prim.h"
#include "x-eval-slots.h"
#include "x-eval.h"
#include "x-heap.h"
#include "x-type.h"
#include "x-token.h"
#include "x-type/buffer.h"
#include "x-type/char.h"
#include "x-type/int.h"
#include "x-type/iter.h"
#include "x-type/list.h"
#include "x-type/prim.h"
#include "x-type/str.h"

Functions
static x_obj_t *	x_prim_buffer_token (x_obj_t *p_base, x_obj_t *p_args)
	Extract the consumed portion of a buffer as a string.
static x_obj_t *	x_prim_buffer_last_char (x_obj_t *p_base, x_obj_t *p_args)
static x_obj_t *	x_prim_token_read_string (x_obj_t *p_base, x_obj_t *p_args)
	Tokenize a string using a token base's registered types.
static x_obj_t *	x_prim_token_read (x_obj_t *p_base, x_obj_t *p_args)
static x_obj_t *	x_prim_buffer_make (x_obj_t *p_base, x_obj_t *p_args)
static x_obj_t *	x_prim_buffer_reset (x_obj_t *p_base, x_obj_t *p_args)
static x_obj_t *	x_prim_buffer_retain (x_obj_t *p_base, x_obj_t *p_args)
static x_obj_t *	x_prim_buffer_append (x_obj_t *p_base, x_obj_t *p_args)
static x_obj_t *	x_prim_buffer_read (x_obj_t *p_base, x_obj_t *p_args)
static x_obj_t *	x_prim_buffer_read_text (x_obj_t *p_base, x_obj_t *p_args)
x_obj_t *	x_prim_buffer_register (x_obj_t *p_base, x_obj_t *p_args)

Detailed Description

Buffer and tokenizer primitives – buffer-*, token-read(-string), buffer-token, buffer-last-char.

Author

Jon Ruttan (jonru.nosp@m.ttan.nosp@m.@gmai.nosp@m.l.co.nosp@m.m)

Copyright

2026 Jon Ruttan @license MIT No Attribution (MIT-0)

Function Documentation

x_prim_buffer_append()

static x_obj_t * x_prim_buffer_append ( x_obj_t *  p_base, x_obj_t *  p_args  )

static

x-lang (buffer-append buffer char): write one char at the write cursor.

x_prim_buffer_last_char()

static x_obj_t * x_prim_buffer_last_char ( x_obj_t *  p_base, x_obj_t *  p_args  )

static

Return the last character consumed by the tokenizer buffer.

x-lang form:

(buffer-last-char buffer) 

Parameters

p_base	Base (execution context).
p_args	Unevaluated: (self buffer).

Returns

Integer character code, or NULL if buffer is empty.

x_prim_buffer_make()

static x_obj_t * x_prim_buffer_make ( x_obj_t *  p_base, x_obj_t *  p_args  )

static

x-lang (buffer-make str [flags]): a BUFFER viewing str's bytes. NON-OWNING (the wrap rule): the buffer's cursors walk the string's own storage, so the string must outlive the buffer and (str make) is the natural backing store. Both cursors start at the base – append then read, or fill the string first and walk the read cursor. Optional flags (descriptor names, e.g. obj-flag-ro): a read-only buffer returns EOF at exhaustion instead of extending from stdin.

x_prim_buffer_read()

static x_obj_t * x_prim_buffer_read ( x_obj_t *  p_base, x_obj_t *  p_args  )

static

x-lang (buffer-read buffer): read one char (extending from input), or nil.

x_prim_buffer_read_text()

static x_obj_t * x_prim_buffer_read_text ( x_obj_t *  p_base, x_obj_t *  p_args  )

static

x-lang (buffer-read-text buffer): like buffer-read, NUL counts as EOF.

x_prim_buffer_register()

x_obj_t * x_prim_buffer_register	(	x_obj_t *	p_base,
		x_obj_t *	p_args
	)

Register the buffer and tokenizer primitives.

x_prim_buffer_reset()

static x_obj_t * x_prim_buffer_reset ( x_obj_t *  p_base, x_obj_t *  p_args  )

static

x-lang (buffer-reset buffer): empty the buffer (cursors back to base).

x_prim_buffer_retain()

static x_obj_t * x_prim_buffer_retain ( x_obj_t *  p_base, x_obj_t *  p_args  )

static

x-lang (buffer-retain buffer): compact unread data to the front.

x_prim_buffer_token()

static x_obj_t * x_prim_buffer_token ( x_obj_t *  p_base, x_obj_t *  p_args  )

static

Extract the consumed portion of a buffer as a string.

x-lang form:

(buffer-token buffer) 

Reads the buffer's current length (bytes consumed by the tokenizer) and copies that prefix, every byte of it, into a new owned string. A NUL in the token ends the string's text, but the bytes past it are in its storage, where byte-ref reaches them.

Parameters

p_base	Base (execution context).
p_args	Unevaluated: (self buffer).

Returns

New string containing the consumed buffer content.

x_prim_token_read()

static x_obj_t * x_prim_token_read ( x_obj_t *  p_base, x_obj_t *  p_args  )

static

Read the next expression from a tokenizer buffer.

Exposes x_token_read to x-lang so that custom reader hooks (defined via make-type) can recursively read sub-expressions from the stream.

x-lang form:

(token-read buffer) 

Parameters

p_base	Base (execution context).
p_args	Unevaluated: (self buffer).

Returns

Parsed expression, or NULL on EOF.

x_prim_token_read_string()

static x_obj_t * x_prim_token_read_string ( x_obj_t *  p_base, x_obj_t *  p_args  )

static

Tokenize a string using a token base's registered types.

x-lang form:

(token-read-string token-base string [start len]) 

Copies the input string into a read-only buffer, then repeatedly calls x_token_read against the token base to produce a linked list of token objects. If metadata tracking is active, the buffer is marked with initial line number 1.

Without start and len the input is the string's text, up to its first NUL. With them it is the len bytes at byte offset start, NULs included – a span of a binary buffer, as str-byte-sub addresses one. The span is the caller's to keep inside the string's storage.

Parameters

p_base	Calling execution context (tokens allocated here).
p_args	Unevaluated: (self token-base string [start len]).

Returns

Linked list of token objects, or NULL for empty input.

Note

The token base should have tokenizer types registered via make-token-base + base-make-type.