LIBFYAML-MISC(3) libfyaml LIBFYAML-MISC(3)

libfyaml-misc - libfyaml miscellaneous API

This header provides the low-level building blocks shared by all other libfyaml public headers. It has no libfyaml dependencies of its own and is safe to include in isolation.

Platform portability

  • ssize_t shim for MSVC / Windows toolchains
  • struct iovec shim for Windows (sys/uio.h substitute)
  • FY_EXPORT / FY_DEPRECATED / FY_FORMAT — symbol visibility and compiler-attribute portability macros
  • FY_UNUSED, FY_ALWAYS_INLINE, FY_DEBUG_UNUSED — compiler hint wrappers with safe fallbacks on non-GCC/Clang toolchains
  • FY_CONSTRUCTOR / FY_DESTRUCTOR — GCC-style constructor/destructor attributes with availability guards (FY_HAS_CONSTRUCTOR, FY_HAS_DESTRUCTOR)

Core constants and sentinel values

  • FY_BIT(x) — single-bit mask (1U << x)
  • FY_NT — sentinel meaning “null-terminated”; pass as a length argument to indicate that the string length should be inferred via strlen

Memory helpers

  • FY_ALLOCA_COPY_FREE / FY_ALLOCA_COPY_FREE_NO_NULL — copy a malloc-returned string onto the stack and immediately free the heap allocation; produces an alloca-lifetime copy in a single expression
  • fy_vsprintfa / fy_sprintfa — printf-style formatting into a stack-allocated (alloca) buffer; the result is valid for the lifetime of the enclosing function

Container and array helpers

  • container_of — recover a pointer to an enclosing struct from a pointer to one of its members
  • ARRAY_SIZE — number of elements in a stack-allocated array

Type safety and compile-time checks

  • FY_SAME_TYPE / FY_CHECK_SAME_TYPE — assert two expressions share the same C type (wraps __builtin_types_compatible_p with a fallback)
  • FY_COMPILE_ERROR_ON_ZERO — produce a compile-time error when the argument evaluates to zero; used for macro-level static assertions

Overflow-safe arithmetic

FY_ADD_OVERFLOW, FY_SUB_OVERFLOW, FY_MUL_OVERFLOW — wrappers around __builtin_*_overflow with portable fallback implementations

Miscellaneous

  • FY_IMPOSSIBLE_ABORT — mark unreachable code paths; aborts at runtime
  • FY_STACK_SAVE / FY_STACK_RESTORE — save and restore the stack pointer for alloca-based temporary arenas
  • Lambda / closure macros and a CPP metaprogramming framework for building variadic generic APIs
  • Diagnostic helpers and floating-point precision constants

This section implements a portable FY_CPP_* macro toolkit that enables applying an operation to every argument in a __VA_ARGS__ list — the equivalent of a compile-time map() function.

The core challenge: The C preprocessor does not support recursion. A macro cannot call itself. The standard workaround is deferred evaluation: a macro is “postponed” so that it is not expanded in the current scan pass, only in a future one. Multiple passes are forced by the FY_CPP_EVALn ladder which re-expands the token stream a power-of-two number of times.

Evaluation levels (FY_CPP_EVAL1 .. FY_CPP_EVAL):

  • FY_CPP_EVAL1 — one pass (no re-expansion)
  • FY_CPP_EVAL2 — 2 passes
  • FY_CPP_EVAL4 — 4 passes
  • FY_CPP_EVAL8 — 8 passes
  • FY_CPP_EVAL16 — 16 passes (GCC only; Clang craps out at 8)
  • FY_CPP_EVAL — full depth (32 passes on GCC, 16 on Clang)

Each doubling allows mapping over twice as many arguments. FY_CPP_EVAL supports lists of up to ~32 elements (GCC) or ~16 elements (Clang) in practice.

Argument accessors — extract positional arguments from __VA_ARGS__:

  • FY_CPP_FIRST(...) — first argument (0 if list is empty)
  • FY_CPP_SECOND(...) — second argument
  • FY_CPP_THIRD(...) — third argument
  • FY_CPP_FOURTH(...) — fourth argument
  • FY_CPP_FIFTH(...) — fifth argument
  • FY_CPP_SIXTH(...) — sixth argument
  • FY_CPP_REST(...) — all arguments after the first (empty if < 2)

Map operations:

  • FY_CPP_MAP(macro, ...) — expand macro(x) for each argument x.
  • FY_CPP_MAP2(a, macro, ...) — expand macro(a, x) for each x, threading a fixed first argument a through every call.

Utility:

  • FY_CPP_VA_COUNT(...) — number of arguments (integer expression).
  • FY_CPP_VA_ITEMS(_type, ...) — compound-literal array of _type from varargs.
  • FY_CPP_EMPTY() — deferred empty token (used to postpone macros).
  • FY_CPP_POSTPONE1(macro) — postpone a one-argument macro one pass.
  • FY_CPP_POSTPONE2(a, macro) — postpone a two-argument macro one pass.

Short-name mode (FY_CPP_SHORT_NAMES):

By default the implementation uses abbreviated internal names (_E1, _FM, etc.) to reduce token-expansion buffer pressure. Define FY_CPP_SHORT_NAMES_DEBUG before including this header to force the original long-name implementation (FY_CPP_EVAL1, _FY_CPP_MAP_ONE, etc.) which is easier to read in expansion traces.

Example:

// Count and collect variadic integer arguments into a C array:
int do_sum(int count, int *items) { ... }
#define do_sum_macro(...) \
    do_sum(FY_CPP_VA_COUNT(__VA_ARGS__), \
           FY_CPP_VA_ITEMS(int, __VA_ARGS__))
// do_sum_macro(1, 2, 5, 100) expands to:
// do_sum(4, ((int [4]){ 1, 2, 5, 100 }))

Portable wrappers around GCC/Clang #pragma GCC diagnostic directives. Use FY_DIAG_PUSH() and FY_DIAG_POP() to bracket a region where a specific warning is suppressed:

FY_DIAG_PUSH
FY_DIAG_IGNORE_ARRAY_BOUNDS
... code that triggers -Warray-bounds ...
FY_DIAG_POP

On compilers other than GCC/Clang all macros in this group expand to nothing.

Portable wrappers for the mantissa-digit and decimal-digit counts of float, double, and long double. Each macro tries three sources in order:

1.
The standard <float.h> macro (e.g. FLT_MANT_DIG).
2.
The GCC/Clang predefined macro (e.g. __FLT_MANT_DIG__).
3.
A conservative hard-coded fallback.

These constants are used when formatting floating-point values as YAML scalars to ensure round-trip fidelity.

__has_builtin(x)
Fallback for compilers that do not provide __has_builtin.
x – The builtin name to query.

Always evaluates to 0 (false) so that all #if __has_builtin(...) guards safely fall back to the portable implementation.

0.

Produce an unsigned bitmask with bit x set.
x – Zero-based bit position (0–31 for a 32-bit result).

1U << x.

Sentinel value meaning “null-terminated; compute length at runtime”.

Pass as the len argument to any libfyaml function that accepts a (const char *str, size_t len) pair to indicate that str is a NUL-terminated C string whose length should be determined with strlen().

Value: (size_t)-1

Mark a symbol as part of the shared-library public ABI.

On GCC/Clang (version >= 4) expands to __attribute__((visibility("default"))), overriding -fvisibility=hidden for the annotated symbol. On other compilers expands to nothing (all symbols are visible by default).

Mark a function or variable as deprecated.

On GCC/Clang expands to __attribute__((deprecated)), causing a compile-time warning whenever the annotated symbol is used. On other compilers expands to nothing.

Annotate a function with printf-style format checking.
  • _t – Format type token (e.g. printf, scanf, strftime).
  • _x – 1-based index of the format-string parameter.
  • _y – 1-based index of the first variadic argument (0 for va_list wrappers).

On GCC/Clang expands to __attribute__((format(_t, _x, _y))), enabling the compiler to type-check the format string and variadic arguments.

Copy a heap string onto the stack and free the original.
  • _str – Heap-allocated string to copy and free (may be NULL).
  • _len – Length in bytes, or FY_NT to use strlen().

1.
Copies _str (up to _len bytes, or the full NUL-terminated length when _len is FY_NT) into a NUL-terminated stack buffer via alloca().
2.
Calls free(@_str) to release the heap allocation.
3.
Evaluates to a const char * pointing to the stack copy.

When _str is NULL the macro evaluates to NULL and performs no copy or free.

The stack buffer is valid only for the lifetime of the enclosing function. Do not call this in a loop — each invocation grows the stack frame.

const char * to the stack copy, or NULL if _str was NULL.

Like FY_ALLOCA_COPY_FREE() but returns “” for NULL.
  • _str – Heap-allocated string to copy and free (may be NULL).
  • _len – Length in bytes, or FY_NT to use strlen().

Identical to FY_ALLOCA_COPY_FREE but substitutes an empty string literal "" when _str is NULL, so callers never receive a NULL pointer.

const char * to the stack copy, or "" if _str was NULL.

Recover a pointer to a containing struct from a member pointer.
  • ptr – Pointer to the member field.
  • type – Type of the containing struct.
  • member – Name of the member field within type.

Given a pointer ptr to a field named member inside a struct of type type, returns a pointer to the enclosing type instance.

Uses __typeof__ to catch type mismatches at compile time (GCC/Clang).

Pointer to the containing struct of type type.

Compute the number of elements in a stack-allocated array.
x – The array expression.

Evaluates to a compile-time constant. Only valid for arrays with a known size at compile time (not pointers or VLAs).

Number of elements, as a size_t.

Suppress “unused variable/parameter” warnings.

On GCC/Clang (version >= 4) expands to __attribute__((unused)). On other compilers expands to nothing.

Use on parameters or local variables that are intentionally unreferenced, e.g. in debug-only code paths.

Run a function automatically before main().

On GCC/Clang expands to __attribute__((constructor)). Also defines FY_HAS_CONSTRUCTOR so callers can detect support at compile time. On other compilers expands to nothing and FY_HAS_CONSTRUCTOR is not defined.

Run a function automatically after main() (or on exit()).

On GCC/Clang expands to __attribute__((destructor)). Also defines FY_HAS_DESTRUCTOR so callers can detect support at compile time. On other compilers expands to nothing and FY_HAS_DESTRUCTOR is not defined.

Mark a variable as potentially unused in non-debug builds.

Expands to __attribute__((unused)) on GCC/Clang when NDEBUG is defined, silencing warnings for variables that are only referenced inside assert() calls (which disappear in release builds). In debug builds (NDEBUG not set) expands to nothing.

Assert that an unreachable code path has been reached.
void – no arguments

Calls assert(0) followed by abort() to terminate the process immediately. Use to mark code paths that must never execute in a correct program, such as the default branch of a switch that covers all enum values.

The double invocation ensures termination even when assertions are disabled (NDEBUG).

Trigger a compile error if expression _e is zero.
_e – Compile-time expression that must be non-zero.

Evaluates _e at compile time. If _e is zero, sizeof(char[-1]) is ill-formed and the build fails. If _e is non-zero, the expression is a no-op void cast.

Prefer FY_CHECK_SAME_TYPE or static_assert for clearer error messages; use this primitive when a compile-time boolean is needed in a macro context.

Test whether two expressions have the same type.
  • _a – First expression.
  • _b – Second expression.

On GCC/Clang uses __builtin_types_compatible_p to compare the types of _a and _b (via __typeof__). On compilers without this builtin always evaluates to true to avoid false negatives.

Non-zero (true) if the types match, zero (false) otherwise.

Trigger a compile error if two expressions have different types.
  • _a – First expression.
  • _b – Second expression.

Combines FY_SAME_TYPE and FY_COMPILE_ERROR_ON_ZERO. Use in macros that require two operands to have the same type (e.g. overflow-safe arithmetic).

Checked addition: detect signed/unsigned integer overflow.
  • _a – First operand.
  • _b – Second operand (must have the same type as _a).
  • _resp – Pointer to receive the result (written even on overflow).

On GCC/Clang maps directly to __builtin_add_overflow(_a, _b, _resp), which is a single compiler intrinsic with full type generality.

  • Requires _a and _b to have the same type (enforced at compile time via FY_CHECK_SAME_TYPE).
  • Computes *_resp = _a + _b and returns true if the addition wrapped.

true if the addition overflowed, false otherwise.

Checked subtraction: detect signed/unsigned integer overflow.
  • _a – Minuend.
  • _b – Subtrahend (must have the same type as _a).
  • _resp – Pointer to receive the result (written even on overflow).

On GCC/Clang maps to __builtin_sub_overflow(_a, _b, _resp). The portable fallback requires _a and _b to have the same type.

true if the subtraction overflowed, false otherwise.

Checked multiplication: detect signed/unsigned integer overflow.
  • _a – First factor.
  • _b – Second factor (must have the same type as _a).
  • _resp – Pointer to receive the result (written even on overflow).

On GCC/Clang maps to __builtin_mul_overflow(_a, _b, _resp). The portable fallback requires _a and _b to have the same type and detects overflow by dividing the product back and comparing with the original operand.

true if the multiplication overflowed, false otherwise.

Format a string into a stack buffer using inline arguments.
  • _fmt – printf-style format string.
  • ellipsis (ellipsis) – Format arguments.

Like fy_vsprintfa() but accepts arguments directly (uses __VA_ARGS__). Two calls to snprintf() are made: the first with a NULL buffer to measure the required length, the second to write the result into a stack-allocated buffer.

The returned pointer is valid only for the lifetime of the enclosing function.

char * to a NUL-terminated stack buffer.

Force one additional macro-expansion pass.
ellipsis (ellipsis) – Token sequence to re-expand.

Expands its argument list once. Use as a building block for deeper evaluation levels; prefer FY_CPP_EVAL() for most uses.

Force two additional macro-expansion passes.
ellipsis (ellipsis) – Token sequence to re-expand.

Force four additional macro-expansion passes.
ellipsis (ellipsis) – Token sequence to re-expand.

Force eight additional macro-expansion passes.
ellipsis (ellipsis) – Token sequence to re-expand.

Force 16 additional macro-expansion passes (GCC only).
ellipsis (ellipsis) – Token sequence to re-expand.

Not defined on Clang, which exhausts its expansion buffer before reaching 16 levels. Wrap FY_CPP_MAP() / FY_CPP_MAP2() in FY_CPP_EVAL() instead of calling this directly.

Force maximum macro-expansion depth.
ellipsis (ellipsis) – Token sequence to fully expand.

On GCC performs 32 expansion passes (supports lists up to ~32 elements). On Clang performs 16 passes (supports lists up to ~16 elements).

Always use this (rather than a specific EVALn) unless you have a known bound on the number of arguments.

Produce an empty token sequence (deferred).
void – no arguments

Expands to nothing. Used in combination with FY_CPP_POSTPONE1() / FY_CPP_POSTPONE2() to defer macro expansion by one scan pass, breaking the preprocessor’s blue-paint recursion guard.

Defer a single-argument macro by one expansion pass.
macro – Macro token to postpone.

Expands to the macro name macro followed by a deferred FY_CPP_EMPTY(), so the macro call is not completed until the next pass.

Defer a two-argument macro by one expansion pass.
  • a – Fixed first argument (carried along, not expanded yet).
  • macro – Macro token to postpone.

Like FY_CPP_POSTPONE1() but for macros that take a leading fixed argument a.

Extract the first argument from a variadic list.
ellipsis (ellipsis) – Variadic argument list.

Returns the first argument, or 0 if the list is empty.

First argument token, or 0.

Extract the second argument from a variadic list.
ellipsis (ellipsis) – Variadic argument list.

Returns the second argument, or 0 if fewer than two arguments are present.

Second argument token, or 0.

Extract the third argument.
ellipsis (ellipsis) – Variadic argument list. Returns: Third argument, or 0.

Extract the fourth argument.
ellipsis (ellipsis) – Variadic argument list. Returns: Fourth argument, or 0.

Extract the fifth argument.
ellipsis (ellipsis) – Variadic argument list. Returns: Fifth argument, or 0.

Extract the sixth argument.
ellipsis (ellipsis) – Variadic argument list. Returns: Sixth argument, or 0.

Return all arguments after the first.
ellipsis (ellipsis) – Variadic argument list.

Expands to the tail of the variadic list with the first element removed. Expands to nothing if the list has zero or one element.

Apply a macro to every argument in a variadic list.
  • macro – Single-argument macro to apply.
  • ellipsis (ellipsis) – Arguments to map over.

Expands macro(x) for each argument x in __VA_ARGS__, concatenating all the results. The argument list must be non-empty. Uses FY_CPP_EVAL() internally, so the list length is bounded by the evaluation depth.

For example:

#define PRINT_ITEM(x)  printf("%d\n", x);
FY_CPP_MAP(PRINT_ITEM, 1, 2, 3)
// expands to: printf("%d\n", 1); printf("%d\n", 2); printf("%d\n", 3);

Count the number of arguments in a variadic list.
ellipsis (ellipsis) – Variadic argument list.

Evaluates to a compile-time integer expression equal to the number of arguments. Returns 0 for an empty list.

Integer expression giving the argument count.

Build a compound-literal array from variadic arguments.
  • _type – Element type of the resulting array.
  • ellipsis (ellipsis) – Values to place in the array.

Expands to a compound literal of type _type[N] (where N is the number of arguments) initialised with the provided values. Useful for passing a variadic argument list as an array to a function.

For example:

// FY_CPP_VA_ITEMS(int, 1, 2, 5, 100)
// expands to: ((int [4]){ 1, 2, 5, 100 })

Apply a binary macro to every argument, threading a fixed first argument.
  • a – Fixed first argument threaded into every call.
  • macro – Two-argument macro to apply.
  • ellipsis (ellipsis) – Arguments to map over.

Expands macro(a, x) for each argument x in __VA_ARGS__. The fixed argument a is passed as the first argument to every invocation.

Token-paste two arguments after full macro expansion.
  • _a – Left token (expanded before pasting).
  • _b – Right token (expanded before pasting).

Unlike the raw ## operator, this macro forces both _a and _b to be fully expanded before concatenation, so macro arguments are substituted correctly.

Generate a unique identifier using __COUNTER__.
_base – Identifier prefix.

Concatenates _base with the current value of the __COUNTER__ preprocessor counter, which increments by one for each use in a translation unit. Guarantees unique names across multiple macro expansions in the same file.

Generate a unique identifier using __LINE__.
_base – Identifier prefix.

Like FY_UNIQUE() but uses the current source line number instead of __COUNTER__. Sufficient when only one such identifier is needed per source line; prefer FY_UNIQUE() in general.

Save the current stack pointer.
void – no arguments

On compilers that provide __builtin_stack_save() (GCC, Clang) returns the current stack pointer as a void *, allowing it to be restored later with FY_STACK_RESTORE(). On other compilers returns (void *)NULL.

Use together with FY_STACK_RESTORE() to bound the stack growth of a loop that calls alloca() on each iteration.

Opaque stack pointer value, or NULL if unsupported.

Restore the stack pointer to a previously saved value.
_x – Value previously returned by FY_STACK_SAVE().

On compilers that provide __builtin_stack_restore() rewinds the stack pointer to the value captured by FY_STACK_SAVE(). On other compilers this is a no-op that discards _x.

Defined when the compiler supports anonymous functions.

  • Clang is compiling with -fblocks (Blocks extension) — also sets FY_HAVE_BLOCK_LAMBDAS; or
  • GCC is in use — also sets FY_HAVE_NESTED_FUNC_LAMBDAS (nested functions).

Code using lambdas should guard against this macro and provide a fallback for environments where it is not defined.

Defined when Clang Block lambdas are available.

Set on Clang when compiled with -fblocks. Implies FY_HAVE_LAMBDAS. Block lambdas use the ^(args){ body } syntax.

Defined when GCC nested-function lambdas are available.

Set on GCC. Implies FY_HAVE_LAMBDAS. Nested function lambdas are defined as local functions inside the enclosing function and cannot outlive it.

Save the current diagnostic state onto the compiler’s stack.

Always paired with FY_DIAG_POP.

Restore the diagnostic state saved by the last FY_DIAG_PUSH.

Suppress a specific warning by pragma string.
_warn – A string literal suitable for use in a _Pragma() call, e.g. "GCC diagnostic ignored \"-Wsomething\"" .

Suppress -Warray-bounds in the current region.

Suppress -Wunused-variable in the current region.

Suppress -Wunused-parameter in the current region.

Number of base-2 mantissa digits in a float.

Typically 24 (IEEE 754 single precision). Fallback: 9.

Number of base-2 mantissa digits in a double.

Typically 53 (IEEE 754 double precision). Fallback: 17.

Number of base-2 mantissa digits in a long double.

Varies by platform (64-bit x87 extended: 64; MSVC/ARM == double: 53). Fallback: 17.

Decimal digits required for a round-trip float.

The minimum number of significant decimal digits such that converting a float to decimal and back recovers the original value exactly. Typically 9. Fallback: 9.

Decimal digits required for a round-trip double.

Typically 17. Fallback: 17.

Decimal digits required for a round-trip long double.

Varies by platform; same as double on most non-x87 targets. Fallback: 17.

This header exposes libfyaml’s pluggable allocator subsystem. Rather than using malloc/free directly, the library routes certain internal allocations through a struct fy_allocator. This lets callers trade memory footprint, speed, and deduplication behaviour to match their workload.

Available strategies (select by name when calling fy_allocator_create()):

  • "linear" — bump-pointer arena. Allocation is O(1) and near-zero overhead; individual frees are a no-op. Ideal for parse-and-discard workflows where the entire arena is released at once.
  • "malloc" — thin wrapper around the system malloc/free. Familiar semantics; useful when individual node lifetimes vary. Should never be used for regular application builds. The presence of it is for having a ASAN/valgrind compatible allocator where buffer overflows can be detected easily.
  • "mremap" — growable linear arena backed by mremap(2). Avoids copying when the arena needs to grow. While mremap is Linux specific, this allocator can be configured to use mmap() or malloc() areas where they work for other platforms.
  • "dedup" — content-addressed store built on xxhash hashing. Stores each unique byte sequence exactly once and returns a shared pointer to all callers. Dramatically reduces memory use when parsing documents with many repeated keys or values (e.g. large YAML configurations, test-suite corpora).
  • "auto" — heuristic selection: given a policy. It usually can do the right thing and is a safe bet.

Tags partition an allocator’s address space. Obtain a tag with fy_allocator_get_tag() and pass it to every fy_allocator_alloc() / fy_allocator_store() call; release the whole tag’s memory in one shot with fy_allocator_release_tag(). This maps naturally to document lifetimes.

In-place variants (fy_linear_allocator_create_in_place(), fy_dedup_allocator_create_in_place()) initialise the allocator inside a caller-supplied buffer — zero dynamic setup cost, useful in stack-allocated or shared-memory contexts.

Iterate over available allocator names
prevp (const char**) – The previous allocator iterator pointer

This method iterates over all the available allocator names. The start of the iteration is signalled by a NULL in *prevp.

The next allocator name in sequence or NULL at the end.

Check if an allocator is available
name (const char*) – The name of the allocator to check

Check if the named allocator is available.

true if the allocator is available, false otherwise

Create an allocator.
  • name (const char*) – The name of the allocator
  • cfg (const void*) – The type specific configuration for the allocator, or NULL for the default.

Creates an allocator of the given type, using the configuration argument provided. The allocator may be destroyed by a corresponding call to fy_allocator_destroy().

You can retrieve the names of available allocators with the fy_allocator_get_names() method.

A pointer to the allocator or NULL in case of an error.

Destroy the given allocator
a (struct fy_allocator*) – The allocator to destroy

Destroy an allocator created earlier via fy_allocator_create(). Tracking allocators will release all memory allocated using them.

Create a linear allocator in place
  • buffer (void*) – The memory buffer to use for both storage and the allocator
  • size (size_t) – The size of the memory buffer

Creates a linear allocator in place, using the buffer provided. No memory allocations will be performed, so it’s safe to embed. There is no need to call fy_allocator_destroy for this allocator.

A pointer to the allocator, or NULL if there is no space

Create a dedup allocator in place
  • buffer (void*) – The memory buffer to use for both storage and the allocator
  • size (size_t) – The size of the memory buffer

Creates a dedup allocator in place, using the buffer provided. No memory allocations will be performed, so it’s safe to embed. There is no need to call fy_allocator_destroy for this allocator. The parent allocator of this will be a linear allocator.

A pointer to the allocator, or NULL if there is no space

Get a tag from an allocator
a (struct fy_allocator*) – The allocator

The allocator interface requires all allocation to belong to a tag. This call creates a tag and returns its value, or an error if not available.

If an allocator only provides a single tag (like the linear allocator for instance), the same tag number, usually 0, is returned.

The created tag or -1 in case of an error.

Release a tag from an allocator
  • a (struct fy_allocator*) – The allocator
  • tag (int) – The tag to release

Releases a tag from an allocator and frees all memory it allocated (if such an operation is provided by the allocator).

Get the maximum number of tags a allocator supports
a (struct fy_allocator*) – The allocator

Get the maximum amount of tags an allocator supports.

If an allocator only provides a single tag (like the linear allocator for instance), 1 will be returned.

The number of tags, or -1 on error

Set the maximum number of tags a allocator supports
  • a (struct fy_allocator*) – The allocator
  • count (unsigned int) – The amount of tags the allocator should support

Sets the maximum amount of tags an allocator supports. If the set allocator tag count is less than the current the additional tags will be released.

0 on success, -1 on error

Trim a tag
  • a (struct fy_allocator*) – The allocator
  • tag (int) – The tag to trim

Trim a tag, that is free any excess memory it allocator, fitting it to the size of the content it carries. Allocators that cannot perform this operation treat it as a NOP.

Reset a tag
  • a (struct fy_allocator*) – The allocator
  • tag (int) – The tag to reset

Reset a tag, that is free any content it carries, but do not release the tag.

Allocate memory from an allocator
  • a (struct fy_allocator*) – The allocator
  • tag (int) – The tag to allocate from
  • size (size_t) – The size of the memory to allocate
  • align (size_t) – The alignment of the object

Allocate memory from the given allocator tag, satisfying the size and align restrictions.

A pointer to the allocated memory or NULL

Free memory allocated from an allocator
  • a (struct fy_allocator*) – The allocator
  • tag (int) – The tag used to allocate the memory
  • ptr (void*) – The pointer to the memory to free

Attempt to free the memory allocated previously by fy_allocator_alloc() Note that non per object tracking allocators treat this as a NOP

Store an object to an allocator
  • a (struct fy_allocator*) – The allocator
  • tag (int) – The tag used to allocate the memory
  • data (const void*) – The pointer to object to store
  • size (size_t) – The size of the object
  • align (size_t) – The alignment restriction of the object

Store an object to an allocator and return a pointer to the location it was stored. When using a deduplicating allocator no new allocation will take place and a pointer to the object already stored will be returned.

The return pointer must not be modified, the objects stored are idempotent.

A constant pointer to the object stored, or NULL in case of an error

Store an object to an allocator (scatter gather)
  • a (struct fy_allocator*) – The allocator
  • tag (int) – The tag used to allocate the memory from
  • iov (const struct iovec*) – The I/O scatter gather vector
  • iovcnt (int) – The number of vectors
  • align (size_t) – The alignment restriction of the object

Store an object to an allocator and return a pointer to the location it was stored. When using a deduplicating allocator no new allocation will take place and a pointer to the object already stored will be returned.

The object is created linearly from the scatter gather io vector provided.

The return pointer must not be modified, the objects stored are immutable.

A constant pointer to the object stored, or NULL in case of an error

Lookup for object in an allocator.
  • a (struct fy_allocator*) – The allocator
  • tag (int) – The tag used to locate the memory
  • data (const void*) – The pointer to object to store
  • size (size_t) – The size of the object
  • align (size_t) – The alignment restriction of the object

Lookup for the exact contents of an object stored in an allocator and return a pointer to the location it was stored. The allocator must have the FYACF_CAN_LOOKUP capability.

A constant pointer to the object stored, or NULL if the object does not exist

Lookup for object in an allocator (scatter gather)
  • a (struct fy_allocator*) – The allocator
  • tag (int) – The tag used to search into
  • iov (const struct iovec*) – The I/O scatter gather vector
  • iovcnt (int) – The number of vectors
  • align (size_t) – The alignment restriction of the object

Lookup for the exact contents of an object stored in an allocator and return a pointer to the location it was stored. The allocator must have the FYACF_CAN_LOOKUP capability.

The scatter gather vector is used to recreate the object.

A constant pointer to the object stored, or NULL in case the object does not exist

Dump internal allocator state
a (struct fy_allocator*) – The allocator

Allocator capability flags

enum fy_allocator_cap_flags {
    FYACF_CAN_FREE_INDIVIDUAL,
    FYACF_CAN_FREE_TAG,
    FYACF_CAN_DEDUP,
    FYACF_HAS_CONTAINS,
    FYACF_HAS_EFFICIENT_CONTAINS,
    FYACF_HAS_TAGS,
    FYACF_CAN_LOOKUP
};

Allocator supports freeing individual allocations
Allocator supports releasing entire tags
Allocator supports deduplication
Allocator can report if it contains a pointer (even if inefficiently)
Allocator can report if it contains a pointer (efficiently)
Allocator has individual tags or not
Allocator supports lookup for content

These flags describe what operations an allocator supports.

Get allocator capabilities
a (struct fy_allocator*) – The allocator

Retrieve the capabilities of an allocator.

The capabilities of the allocator

Check if a allocator contains a pointer
  • a (struct fy_allocator*) – The allocator
  • tag (int) – Tag to search in, -1 for all
  • ptr (const void*) – The object pointer

Report if an allocator contains the pointer

true if the pointer ptr is contained in the allocator, false otherwise

Get the linear size of an allocator tag
  • a (struct fy_allocator*) – The allocator
  • tag (int) – The tag

Retrieve the linear size of the content of a tag. That is the size of a buffer if one was to copy the content of the tag in that buffer in a linear manner.

The linear size of the content stored in the tag or -1 in case of an error.

Get the linear extend of a tag
  • a (struct fy_allocator*) – The allocator
  • tag (int) – The tag
  • sizep (size_t*) – Pointer to a variable that will be filled with the size.

If a tag stores it’s content in a single linear buffer, retrieve it directly. This is possible only under careful arrangement of allocator configuration, but it is an important optimization case.

A pointer to the linear content of the tag, or NULL if othersize.

linear allocator configuration

struct fy_linear_allocator_cfg {
    void *buf;
    size_t size;
}

A pointer to a buffer that will be used, or NULL in order to allocate
Size of the buffer in bytes

The mremap allocator arena types

enum fy_mremap_arena_type {
    FYMRAT_DEFAULT,
    FYMRAT_MALLOC,
    FYMRAT_MMAP
};

Use what’s optimal for this platform
Use malloc/realloc arena type (not recommended)
Use mmap/mremap arena type

mremap allocator configuration

struct fy_mremap_allocator_cfg {
    size_t big_alloc_threshold;
    size_t empty_threshold;
    size_t minimum_arena_size;
    float grow_ratio;
    float balloon_ratio;
    enum fy_mremap_arena_type arena_type;
}

Threshold for immediately creating a new arena.
The threshold under which an arena is moved to the full list.
The minimum (and starting size) of an arena.
The ratio which an arena will try to grow if full (>1.0)
The multiplier for the vm area first allocation
The arena type

If any of the fields is zero, then the system will provide (somewhat) reasonable defaults.

dedup allocator configuration

struct fy_dedup_allocator_cfg {
    struct fy_allocator *parent_allocator;
    unsigned int bloom_filter_bits;
    unsigned int bucket_count_bits;
    size_t dedup_threshold;
    unsigned int chain_length_grow_trigger;
    size_t estimated_content_size;
    float minimum_bucket_occupancy;
}

The parent allocator (required)
Number of bits of the bloom filter (or 0 for default)
Number of bits for the bucket count (or 0 for default)
Number of bytes over which dedup takes place (default 0=always)
Chain length of a bucket over which a grow takes place (or 0 for auto)
Estimated content size (or 0 for don’t know)
The minimum amount that a tag bucket must be full before growth is allowed (default 50%, or 0.0)

auto allocator scenario type

enum fy_auto_allocator_scenario_type {
    FYAST_PER_TAG_FREE,
    FYAST_PER_TAG_FREE_DEDUP,
    FYAST_PER_OBJ_FREE,
    FYAST_PER_OBJ_FREE_DEDUP,
    FYAST_SINGLE_LINEAR_RANGE,
    FYAST_SINGLE_LINEAR_RANGE_DEDUP
};

only per tag freeing, no individual obj free
per tag freeing, dedup obj store
object freeing allowed, tag freeing still works
per obj freeing, dedup obj store
just a single linear range, no frees at all
single linear range, with dedup

auto allocator configuration

struct fy_auto_allocator_cfg {
    enum fy_auto_allocator_scenario_type scenario;
    size_t estimated_max_size;
}

Auto allocator scenario
Estimated max content size (or 0 for don’t know)

This header provides a simple, portable thread pool built on POSIX threads. It is used internally by the BLAKE3 hasher and the generic type system’s parallel map/filter/reduce operations, and is also available as a public API for application use.

Two operational modes are supported:

Work-stealing mode (FYTPCF_STEAL_MODE): the recommended mode for data-parallel loops. Submit a batch of work items with fy_thread_work_join(); the pool distributes items across threads and the caller participates in the execution. About 30% faster than reservation mode for typical workloads.

Reservation mode: explicitly reserve a thread with fy_thread_reserve(), submit a single work item with fy_thread_submit_work(), continue doing other work in the calling thread, then synchronise with fy_thread_wait_work(). Release the thread afterwards with fy_thread_unreserve().

Three convenience wrappers over fy_thread_work_join() cover the most common data-parallel patterns:

  • fy_thread_args_join() — array of heterogeneous argument pointers
  • fy_thread_arg_array_join() — flat array of equal-sized argument items
  • fy_thread_arg_join() — same argument broadcast to N invocations

An optional fy_work_check_fn callback lets each call site decide at runtime whether a given item is worth offloading to a thread or running inline.

Work exec function
arg (void*) – The argument to the method

The callback executed on work submission

Work check function
arg (const void*) – The argument to the method

Work checker function to decide if it’s worth to offload to a thread.

true if it should offload to thread, false otherwise

Work submitted to a thread for execution

struct fy_thread_work {
    fy_work_exec_fn fn;
    void *arg;
    struct fy_work_pool *wp;
}

The execution function for this work
The argument to the fn
Used internally, must be set to NULL on entry

This is the structure describing the work submitted to a thread for execution.

Thread pool configuration flags

enum fy_thread_pool_cfg_flags {
    FYTPCF_STEAL_MODE
};

Enable steal mode for the thread pool

These flags control the operation of the thread pool. For now only the steal mode flag is defined.

thread pool configuration structure.

struct fy_thread_pool_cfg {
    enum fy_thread_pool_cfg_flags flags;
    unsigned int num_threads;
    void *userdata;
}

Thread pool configuration flags
Number of threads, if 0 == online CPUs
A userdata pointer

Argument to the fy_thread_pool_create() method.

Create a thread pool
cfg (const struct fy_thread_pool_cfg*) – The configuration for the thread pool

Creates a thread pool with its configuration cfg The thread pool may be destroyed by a corresponding call to fy_thread_pool_destroy().

A pointer to the thread pool or NULL in case of an error.

Destroy the given thread pool
tp (struct fy_thread_pool*) – The thread pool to destroy

Destroy a thread pool created earlier via fy_thread_pool_create(). Note that this function will block until all threads of the pool are destroyed.

Get the number of threads
tp (struct fy_thread_pool*) – The thread pool

Returns the actual number of created threads.

> 0 for the number of actual threads created, -1 on error

Get the configuration of a thread pool
tp (struct fy_thread_pool*) – The thread pool

The configuration of the thread pool

Reserve a thread from the pool.
tp (struct fy_thread_pool*) – The thread pool

Reserve a thread from the pool and return it. Note this is only valid for a non-work stealing thread pool. You release the thread again via a call to fy_thread_unreserve.

A reserved thread if not NULL, NULL if no threads are available.

Unreserve a previously reserved thread
t (struct fy_thread*) – The thread

Unreserve a thread previously reserved via a call to fy_thread_reserve() Note this is only valid for a non-work stealing thread pool.

Submit work for execution
  • t (struct fy_thread*) – The thread
  • work (struct fy_thread_work*) – The work

Submit work for execution. If successful the thread will start executing the work in parallel with the calling thread. You can wait for the thread to terminate via a call to fy_thread_wait_work(). The thread must have been reserved earlier via fy_thread_reserve()

Note this is only valid for a non-work stealing thread pool.

0 if work has been submitted, -1 otherwise.

Wait for completion of submitted work
t (struct fy_thread*) – The thread

Wait until submitted work to the thread has finished. Note this is only valid for a non-work stealing thread pool.

0 if work finished, -1 on error.

Check whether no threads are free
tp (struct fy_thread_pool*) – The thread pool

true if all threads are currently reserved, false otherwise.

Check whether at least one thread is reserved
tp (struct fy_thread_pool*) – The thread pool

true if any thread is currently reserved, false otherwise.

Submit works for execution and wait
  • tp (struct fy_thread_pool*) – The thread pool
  • works (struct fy_thread_work*) – Pointer to an array of works sized work_count
  • work_count (size_t) – The size of the works array
  • check_fn (fy_work_check_fn) – Pointer to a check function, or NULL for no checks

Submit works for possible parallel execution. If no offloading is possible at the time execute in the current context. It is possible to use in both stealing and non-stealing mode with the difference being that stealing mode is about 30% faster.

Execute function in parallel using arguments as pointers
  • tp (struct fy_thread_pool*) – The thread pool
  • fn (fy_work_exec_fn) – The function to execute in parallel
  • check_fn (fy_work_check_fn) – Pointer to a check function, or NULL for no checks
  • args (void**) – An args array sized count of argument pointers
  • count (size_t) – The count of the args array items

Execute fn possibly in parallel using the threads in the thread pool. The arguments of the function are provided by the args array.

Execute function in parallel using argument array
  • tp (struct fy_thread_pool*) – The thread pool
  • fn (fy_work_exec_fn) – The function to execute in parallel
  • check_fn (fy_work_check_fn) – Pointer to a check function, or NULL for no checks
  • args (void*) – An args array of argsize items
  • argsize (size_t) – The size of each argument array item
  • count (size_t) – The count of the args array items

Execute fn possibly in parallel using the threads in the thread pool. The arguments of the function are provided by the args array.

Execute function in parallel with the same argument
  • tp (struct fy_thread_pool*) – The thread pool
  • fn (fy_work_exec_fn) – The function to execute in parallel
  • check_fn (fy_work_check_fn) – Pointer to a check function, or NULL for no checks
  • arg (void*) – The common argument
  • count (size_t) – The count of executions

Execute fn possibly in parallel using the threads in the thread pool. The argument of the functions is the same.

This header exposes libfyaml’s embedded BLAKE3 hasher. BLAKE3 is a modern, highly parallelisable cryptographic hash function producing 256-bit (32-byte) output.

Three hashing modes are supported, selected via struct fy_blake3_hasher_cfg at creation time:

  • Standard: plain BLAKE3 hash (default when key and context are NULL)
  • Keyed: MAC-like hash using a 32-byte key
  • Key derivation: derive a subkey from an application context string

The hasher can be used in streaming fashion (update / finalize) or for one-shot hashing of memory regions (fy_blake3_hash()) and files (fy_blake3_hash_file()). File hashing uses mmap by default for large files and can be further parallelised via a thread pool.

Runtime SIMD backend selection is automatic: the best available backend (SSE2, SSE4.1, AVX2, AVX512 on x86; NEON on ARM; portable C otherwise) is used unless a specific backend name is requested in the config. Use fy_blake3_backend_iterate() to enumerate available backends.

The hasher object is reusable: call fy_blake3_hasher_reset() to start a new hash without reallocating the object.

Iterate over the supported BLAKE3 backends
prevp (const char**) – The previous backend pointer, or NULL at start

This method iterates over the supported BLAKE3 backends. The start of the iteration is signalled by a NULL in *prevp.

The default backend is always the last in sequence, so for example if the order is [ “portable”, “sse2”, NULL ] the default is “sse2”.

The next backend or NULL at the end.

BLAKE3 hasher configuration

struct fy_blake3_hasher_cfg {
    const char *backend;
    size_t file_buffer;
    size_t mmap_min_chunk;
    size_t mmap_max_chunk;
    bool no_mmap;
    const uint8_t *key;
    const void *context;
    size_t context_len;
    struct fy_thread_pool *tp;
    int num_threads;
}

NULL for default, or a specific backend name
Use this amount of buffer for buffering, zero for default
Minimum chunk size for mmap case
Maximum chunk size for mmap case
Disable mmap for file access
pointer to a FY_BLAKE3_KEY_LEN area when in keyed mode. NULL otherwise.
pointer to a context when in key derivation mode. NULL otherwise.
The size of the context when in key derivation mode. 0 otherwise.
The thread pool to use, if NULL, create a private one
Number of threads to use - 0 means default: NUM_CPUS * 3 / 2 - > 0 specific number of threads - -1 disable threading entirely

Argument to the fy_blake3_hasher_create() method which is the fyaml’s user facing BLAKE3 API. It is very minimal, on purpose, since it’s meant to be exposing a full blown BLAKE3 API.

Create a BLAKE3 hasher object.
cfg (const struct fy_blake3_hasher_cfg*) – The configuration for the BLAKE3 hasher

Creates a BLAKE3 hasher with its configuration cfg The hasher may be destroyed by a corresponding call to fy_blake3_hasher_destroy().

A pointer to the BLAKE3 hasher or NULL in case of an error.

Destroy the given BLAKE3 hasher
fyh (struct fy_blake3_hasher*) – The BLAKE3 hasher to destroy

Destroy a BLAKE3 hasher created earlier via fy_blake3_hasher_create().

Update the BLAKE3 hasher state with the given input
  • fyh (struct fy_blake3_hasher*) – The BLAKE3 hasher
  • input (const void*) – Pointer to the input
  • input_len (size_t) – Size of the input in bytes

Updates the BLAKE3 hasher state by hashing the given input.

Finalize the hash and get output
fyh (struct fy_blake3_hasher*) – The BLAKE3 hasher

Finalizes the BLAKE3 hasher and returns the output

A pointer to the BLAKE3 output (sized FY_BLAKE3_OUT_LEN), or NULL in case of an error.

Resets the hasher
fyh (struct fy_blake3_hasher*) – The BLAKE3 hasher

Resets the hasher for re-use

BLAKE3 hash a memory area
  • fyh (struct fy_blake3_hasher*) – The BLAKE3 hasher
  • mem (const void*) – Pointer to the memory to use
  • size (size_t) – The size of the memory in bytes

Hash a memory area and return the BLAKE3 output.

A pointer to the BLAKE3 output (sized FY_BLAKE3_OUT_LEN), or NULL in case of an error.

BLAKE3 hash a file.
  • fyh (struct fy_blake3_hasher*) – The BLAKE3 hasher
  • filename (const char*) – The filename

Hash the given file (possibly using mmap)

A pointer to the BLAKE3 output (sized FY_BLAKE3_OUT_LEN), or NULL in case of an error.

This header provides portable utilities for working with memory alignment requirements, from compile-time attributes to runtime allocation and pointer-rounding helpers. It has no libfyaml dependencies.

Compile-time attributes

  • FY_ALIGNED_TO(x) — apply an alignment attribute to a variable or type; expands to __attribute__((aligned(x))) on GCC/Clang and __declspec(align(x)) on MSVC
  • FY_CACHELINE_ALIGN — shorthand for cache-line (64-byte) alignment, useful for preventing false sharing between fields accessed concurrently by different threads

Value rounding

  • FY_ALIGN(align, x) — round an integer up to the next multiple of align (must be a power of two)
  • FY_CACHELINE_SIZE_ALIGN(x) — round up to the next cache-line boundary
  • fy_ptr_align(p, align) — round a pointer up to alignment
  • fy_size_t_align(sz, align) — round a size_t value up to alignment

Heap allocation

  • fy_align_alloc(align, size) / fy_align_free(p) — allocate and free memory with an explicit alignment (posix_memalign on POSIX, _aligned_malloc on Windows)
  • fy_cacheline_alloc(size) / fy_cacheline_free(p) — convenience wrappers that fix the alignment at FY_CACHELINE_SIZE

Stack allocation

fy_alloca_align(sz, align) — allocate a block on the stack with a specified alignment; uses plain alloca when alignment fits within max_align_t, otherwise over-allocates and advances the pointer

Declare that a variable or type has a minimum alignment.
x – Required alignment in bytes (must be a power of two).

Expands to the appropriate compiler-specific alignment attribute: - GCC/Clang: __attribute__((aligned(x))) - MSVC: should be __declspec(align(x)) but MSVC does not support trailing alignment… - Other: empty (no enforced alignment)

Round _x up to the next multiple of _align.
  • _align – Alignment boundary (power of two).
  • _x – Value to round up.

_align must be a power of two. The result is always >= _x and is the smallest multiple of _align that is >= _x.

_x rounded up to the nearest multiple of _align.

Size of a CPU cache line in bytes.

Universally 64 bytes on all currently supported architectures (x86, x86_64, ARM, ARM64, PowerPC).

Round _x up to the next cache-line boundary.
_x – Value to round up.

_x rounded up to the nearest multiple of FY_CACHELINE_SIZE.

Alignment attribute for cache-line-aligned objects.

Apply to a variable or struct field to ensure it starts on a cache-line boundary, preventing false sharing between concurrent readers/writers.

Example:

struct my_data {
    FY_CACHELINE_ALIGN int hot_counter;
    int cold_field;
};

Allocate memory with a specific alignment.
  • align (size_t) – Required alignment in bytes (must be a power of two and >= sizeof(void *) ).
  • size (size_t) – Number of bytes to allocate.

Allocates size bytes rounded up to the nearest multiple of align, with the returned pointer guaranteed to be a multiple of align.

Uses posix_memalign() on POSIX systems and _aligned_malloc() on Windows. Free the result with fy_align_free().

Pointer to the allocated block, or NULL on failure.

Free memory allocated by fy_align_alloc().
p (void*) – Pointer previously returned by fy_align_alloc(), or NULL.

A NULL p is silently ignored.

Allocate cache-line-aligned memory.
size (size_t) – Number of bytes to allocate.

Equivalent to fy_align_alloc(FY_CACHELINE_SIZE, size). Free the result with fy_cacheline_free().

Cache-line-aligned pointer, or NULL on failure.

Free memory allocated by fy_cacheline_alloc().
p (void*) – Pointer previously returned by fy_cacheline_alloc(), or NULL.

A NULL p is silently ignored.

Round a pointer up to the next multiple of align.
  • p (void*) – Pointer to align.
  • align (size_t) – Alignment boundary in bytes (must be a power of two).

Does not allocate any memory; the caller is responsible for ensuring the underlying buffer extends at least (@align - 1) bytes past p.

p rounded up to the nearest multiple of align.

Round a size_t value up to the next multiple of align.
  • size (size_t) – Value to round up.
  • align (size_t) – Alignment boundary in bytes (must be a power of two).

size rounded up to the nearest multiple of align.

Stack-allocate a buffer with a specific alignment.
  • _sz – Number of bytes to allocate.
  • _align – Required alignment in bytes (must be a power of two).

Expands to a statement expression (GCC extension) that allocates _sz bytes on the stack, aligned to _align bytes. When _align <= sizeof(max_align_t) a plain alloca() is used; otherwise _sz + _align - 1 bytes are allocated and the pointer is advanced with fy_ptr_align(). This macro does not work on MSVC.

The result is valid only for the lifetime of the enclosing function; do not return or store it beyond that scope.

Stack pointer aligned to _align.

This header provides a portable way to include the platform’s byte-order detection headers and ensures the following macros are always defined:

  • __BYTE_ORDER — the byte order of the current platform
  • __BIG_ENDIAN — big-endian sentinel value
  • __LITTLE_ENDIAN — little-endian sentinel value

Usage:

#include <libfyaml/fy-internal-endian.h>
#if __BYTE_ORDER == __LITTLE_ENDIAN
    // little-endian path
#else
    // big-endian path
#endif

Supported platforms: - Linux / Cygwin / OpenBSD / GNU Hurd / Emscripten — via <endian.h> - macOS / iOS — via <libkern/OSByteOrder.h> + <machine/endian.h> - NetBSD / FreeBSD / DragonFly BSD — via <sys/endian.h> - Windows (MSVC) — via <winsock2.h> (+ <sys/param.h> for MinGW)

The non-standard BYTE_ORDER, BIG_ENDIAN, and LITTLE_ENDIAN spellings are aliased to the double-underscore variants if needed.

Byte-swap an 8-bit value (no-op).
x – The 8-bit value.

Defined for symmetry with bswap_16(), bswap_32(), and bswap_64(). Swapping a single byte is always a no-op.

x unchanged.

Encodes unsigned integer sizes into a compact, self-delimiting byte stream. The encoding is modelled after the variable-length quantity (VLQ / LEB128 big-endian variant) used in MIDI and other binary formats:

  • Each byte carries 7 bits of payload in bits 6..0.
  • Bit 7 (MSB) is a continuation flag: 1 means more bytes follow, 0 means this is the last byte.
  • Exception: the final (maximum-length) byte is always 8 bits of payload with no continuation bit, allowing the full 64-bit / 32-bit range.

64-bit encoding (up to 9 bytes):

bytes   bits   value range
1       7      0 .. 127
2       14     128 .. 16383
3       21     16384 .. 2097151
4       28     2097152 .. 268435455
5       35     268435456 .. 34359738367
6       42     ..
7       49     ..
8       56     ..
9       64     full uint64_t range

32-bit encoding (up to 5 bytes):

bytes   bits   value range
1       7      0 .. 127
2       14     128 .. 16383
3       21     16384 .. 2097151
4       28     2097152 .. 268435455
5       32     full uint32_t range (top 4 bits of byte 0 ignored)

The native-width fy_encode_size() / fy_decode_size() family selects the 64-bit or 32-bit variant based on SIZE_MAX.

Each family provides four operations:

  • _bytes() — compute the encoded length without writing anything
  • encode() — write the encoding into a bounded buffer
  • decode() — read and validate from a bounded buffer
  • decode_nocheck() — read without bounds checking (caller guarantees room)
  • skip() — advance past an encoded value in a bounded buffer
  • skip_nocheck() — advance without bounds checking

Maximum encoded length of a 64-bit size value.

A 64-bit value requires at most 9 bytes: 8 x 7-bit groups plus one final unconstrained byte = 7 x 8 + 8 = 64 bits.

Maximum encoded length of a 32-bit size value.

A 32-bit value requires at most 5 bytes: 4 x 7-bit groups plus one final unconstrained byte = 7 x 4 + 4 = 32 bits.

Compute the encoded byte count for a 32-bit size.
size (uint32_t) – The value whose encoded length is queried.

Returns the number of bytes that fy_encode_size32() would write for size, without actually writing anything. Useful for pre-allocating buffers.

Number of bytes required (1–5).

Encode a 32-bit size into a buffer.
  • p (uint8_t*) – Start of the output buffer.
  • bufsz (uint32_t) – Available space in bytes.
  • size (uint32_t) – Value to encode.

Writes the variable-length encoding of size into the buffer [@p, p+@bufsz).

Decode a variable-length 32-bit size from a buffer.
  • start (const uint8_t*) – Start of the encoded data.
  • bufsz (size_t) – Available bytes to read.
  • sizep (uint32_t*) – Output: the decoded value. Set to (uint32_t)-1 on error.

Reads bytes from [@start, start+@bufsz) and reconstructs the encoded 32-bit value. Stops after FYVL_SIZE_ENCODING_MAX_32 bytes at most.

was exhausted before a complete value was found.

Decode a 32-bit size without bounds checking.
  • start (const uint8_t*) – Start of the encoded data.
  • sizep (uint64_t*) – Output: the decoded value as a uint64_t for uniform handling.

Like fy_decode_size32() but assumes the buffer is large enough to hold a complete encoding. The caller must guarantee at least FYVL_SIZE_ENCODING_MAX_32 bytes are available at start.

Pointer to one past the last consumed byte.

Skip past a variable-length 32-bit size in a buffer.
  • start (const uint8_t*) – Start of the encoded data.
  • bufsz (size_t) – Available bytes.

Advances past the encoded value without decoding it. Useful when the value itself is not needed.

was exhausted before a complete encoding was found.

Skip a 32-bit encoded size without bounds checking.
p (const uint8_t*) – Start of the encoded data.

Like fy_skip_size32() but assumes the buffer is large enough. The caller must guarantee at least FYVL_SIZE_ENCODING_MAX_32 bytes are readable.

Pointer to one past the last consumed byte.

Compute the encoded byte count for a 64-bit size.
size (uint64_t) – The value whose encoded length is queried.

Returns the number of bytes that fy_encode_size64() would write for size, without writing anything.

Number of bytes required (1–9).

Encode a 64-bit size into a buffer.
  • p (uint8_t*) – Start of the output buffer.
  • bufsz (size_t) – Available space in bytes.
  • size (uint64_t) – Value to encode.

Writes the variable-length encoding of size into the buffer [@p, p+@bufsz).

Decode a variable-length 64-bit size from a buffer.
  • start (const uint8_t*) – Start of the encoded data.
  • bufsz (size_t) – Available bytes to read.
  • sizep (uint64_t*) – Output: the decoded value. Set to (size_t)-1 on error.

Reads bytes from [@start, start+@bufsz) and reconstructs the encoded 64-bit value. Stops after FYVL_SIZE_ENCODING_MAX_64 bytes at most.

was exhausted before a complete value was found.

Decode a 64-bit size without bounds checking.
  • start (const uint8_t*) – Start of the encoded data.
  • sizep (uint64_t*) – Output: the decoded value.

Like fy_decode_size64() but assumes the buffer is large enough. The caller must guarantee at least FYVL_SIZE_ENCODING_MAX_64 bytes are readable.

Pointer to one past the last consumed byte.

Skip past a variable-length 64-bit size in a buffer.
  • start (const uint8_t*) – Start of the encoded data.
  • bufsz (size_t) – Available bytes.

Advances past the encoded value without decoding it.

was exhausted before a complete encoding was found.

Skip a 64-bit encoded size without bounds checking.
p (const uint8_t*) – Start of the encoded data.

Like fy_skip_size64() but assumes the buffer is large enough. The caller must guarantee at least FYVL_SIZE_ENCODING_MAX_64 bytes are readable.

Pointer to one past the last consumed byte.

Compute encoded byte count for a native size_t.
size (size_t) – The value whose encoded length is queried.

Selects fy_encode_size64_bytes() or fy_encode_size32_bytes() based on SIZE_MAX.

Number of bytes required.

Encode a native size_t into a buffer.
  • p (uint8_t*) – Start of the output buffer.
  • bufsz (size_t) – Available space in bytes.
  • size (size_t) – Value to encode.

Selects fy_encode_size64() or fy_encode_size32() based on SIZE_MAX.

Pointer to one past the last written byte, or NULL on overflow.

Decode a native size_t from a buffer.
  • start (const uint8_t*) – Start of the encoded data.
  • bufsz (size_t) – Available bytes.
  • sizep (size_t*) – Output: the decoded value.

Selects fy_decode_size64() or fy_decode_size32() based on SIZE_MAX.

Pointer to one past the last consumed byte, or NULL on error.

Decode a native size_t without bounds checking.
  • start (const uint8_t*) – Start of the encoded data.
  • sizep (size_t*) – Output: the decoded value.

Selects the 64-bit or 32-bit nocheck variant based on sizeof(size_t). The caller must guarantee enough bytes are available at start.

Pointer to one past the last consumed byte.

Skip a native size_t encoding in a buffer.
  • start (const uint8_t*) – Start of the encoded data.
  • bufsz (size_t) – Available bytes.

Selects fy_skip_size64() or fy_skip_size32() based on SIZE_MAX.

Pointer to one past the last consumed byte, or NULL on error.

Skip a native size_t encoding without bounds checking.
start (const uint8_t*) – Start of the encoded data.

Selects the 64-bit or 32-bit nocheck variant based on SIZE_MAX. The caller must guarantee enough bytes are available.

Pointer to one past the last consumed byte.

Maximum encoded length for a native size_t.

Equals FYVL_SIZE_ENCODING_MAX_64 on 64-bit platforms, FYVL_SIZE_ENCODING_MAX_32 on 32-bit platforms.

This header provides a thin, portable abstraction over C11 <stdatomic.h> with graceful fallback for compilers that support _Atomic as an extension (GCC without -std=c11, Clang) and a last-resort non-atomic fallback for toolchains that support neither.

Detection macros (defined by this header, not meant for direct use):

the standard atomic_* functions and types are available.
from <stdatomic.h> or as a compiler extension.
maps to a real atomic type). When not defined, _Atomic(_x) expands to plain _x and all operations are non-atomic single-threaded stubs.
memory-ordered (i.e. <stdatomic.h> is in use). Without this, operations are performed as plain loads/stores with no memory barriers.

Public API — all fy_atomic_* macros delegate to the selected backend:

  • FY_ATOMIC() — qualify a type as atomic
  • fy_atomic_flag — boolean flag type
  • fy_atomic_load() / fy_atomic_store()
  • fy_atomic_exchange()
  • fy_atomic_compare_exchange_strong() / fy_atomic_compare_exchange_weak()
  • fy_atomic_fetch_add() / fy_atomic_fetch_sub()
  • fy_atomic_fetch_or() / fy_atomic_fetch_xor() / fy_atomic_fetch_and()
  • fy_atomic_flag_clear() / fy_atomic_flag_set() / fy_atomic_flag_test_and_set()
  • fy_cpu_relax() — emit a CPU relaxation hint (PAUSE/YIELD)
  • fy_atomic_get_and_clear_counter() — atomically read and subtract a counter

Qualify a type as atomic.
_x – The underlying C type.

Expands to _Atomic(_x) when FY_HAVE_ATOMICS is defined, or to plain _x otherwise (non-atomic fallback).

Example:

FY_ATOMIC(uint64_t) refcount;

A boolean flag that can be set/cleared/tested atomically.

Backed by atomic_flag from <stdatomic.h> when available, or a plain bool in the fallback path.

Atomically load the value at _ptr.
_ptr – Pointer to an FY_ATOMIC-qualified variable.

The current value.

Atomically store _v at _ptr.
  • _ptr – Pointer to an FY_ATOMIC-qualified variable.
  • _v – Value to store.

Atomically replace the value at _ptr with _v.
  • _ptr – Pointer to an FY_ATOMIC-qualified variable.
  • _v – New value to store.

The old value that was at _ptr before the exchange.

Strong CAS: replace _ptr‘s value if it equals _e.
  • _ptr – Pointer to an FY_ATOMIC-qualified variable.
  • _e – Pointer to the expected value (updated on failure).
  • _d – Desired value to store on success.

If *_ptr == *_e, stores _d into _ptr and returns true. Otherwise, loads the current value into _e and returns false. The strong variant never spuriously fails.

true if the exchange succeeded, false otherwise.

Weak CAS: may spuriously fail.
  • _ptr – Pointer to an FY_ATOMIC-qualified variable.
  • _e – Pointer to the expected value (updated on failure).
  • _d – Desired value to store on success.

Like fy_atomic_compare_exchange_strong() but may fail even when *_ptr == *_e. Prefer in retry loops where a spurious failure is harmless and performance matters.

true if the exchange succeeded, false otherwise.

Atomically add _v to _ptr and return the old value.
  • _ptr – Pointer to an FY_ATOMIC-qualified integer variable.
  • _v – Value to add.

The value of _ptr before the addition.

Atomically subtract _v from _ptr and return the old value.
  • _ptr – Pointer to an FY_ATOMIC-qualified integer variable.
  • _v – Value to subtract.

The value of _ptr before the subtraction.

Atomically OR _v into _ptr and return the old value.
  • _ptr – Pointer to an FY_ATOMIC-qualified integer variable.
  • _v – Value to OR in.

The value of _ptr before the operation.

Atomically XOR _v into _ptr and return the old value.
  • _ptr – Pointer to an FY_ATOMIC-qualified integer variable.
  • _v – Value to XOR in.

The value of _ptr before the operation.

Atomically AND _v into _ptr and return the old value.
  • _ptr – Pointer to an FY_ATOMIC-qualified integer variable.
  • _v – Value to AND in.

The value of _ptr before the operation.

Atomically clear a flag (set to false).
_ptr – Pointer to an fy_atomic_flag.

Atomically set a flag (set to true).
_ptr – Pointer to an fy_atomic_flag.

this is a libfyaml extension; standard <stdatomic.h> only provides atomic_flag_test_and_set(). In the fallback path this is implemented as a plain store.

Atomically set a flag and return its old value.
_ptr – Pointer to an fy_atomic_flag.

Sets the flag to true and returns the value it held before the operation. This is the standard test-and-set primitive.

true if the flag was already set, false if it was clear.

Emit a CPU relaxation hint inside a spin-wait loop.
void – no arguments

Reduces power consumption and improves hyper-threading performance on x86/x86_64 (PAUSE), signals a yield on AArch64/ARM (YIELD), and emits a low-priority hint on PowerPC (or 27,27,27). Falls back to a compiler memory barrier on unsupported architectures.

Use inside tight spin loops to avoid memory-ordering penalties and allow sibling hardware threads to make progress:

while (!fy_atomic_load(&ready))
    fy_cpu_relax();

2019-2026, Pantelis Antoniou

March 15, 2026