The core language

The core is what stage0/*.c and src/*.mc implement between them: the language mc compiles before any teaching happens. Everything else in this reference — #rule, syntax, pass, backend — builds on top of it and is described in directives.md and hooks.md.

The organising rule is that the core only has to be big enough to compile one program, src/mc.mc. Every "why is there no X?" below has that answer.

1. Lexical structure #

Tokens #

The lexer produces (id, value, lexeme, file, line). Seven ids are token classes:

Everything else is a lexeme in the token table, numbered from 256 in registration order. The core registers 45 of them at start-up (tok_init): the seven type words, if else loop break continue return extern, the punctuation, and the operators. #token, #rule's dispatch literal, and every Tier 3 registration append to the same table — which is why --dump-tokens shows ids above 300 in a taught program, and why the ids 256..269 (u8 through extern) must never move: registering a token before tok_init would shift them and break the core.

Punctuation is matched by longest prefix, scanning the table linearly. >>= would win over >> if it existed; p_resplit_punct is the one way to undo such a match (hooks.md).

Comments #

// to end of line and /* ... */. Block comments do not nest; an unclosed one is unterminated comment.

Literals #

• Integer — decimal or 0x hexadecimal. Stored as i64; 0xFFFFFFFFFFFFFFFF is fine.
• Char — 'a', with escapes \n \t \r \0 \\ \' \". It folds straight into an integer node; there is no separate char type.
• String — "...", decoded into the arena, emitted into __TEXT,__cstring with a trailing NUL, and deduplicated by content (first occurrence wins the l_strN symbol). A \0 inside a string is an error (\0 not allowed in string): __cstring is S_CSTRING_LITERALS and the linker merges literals at the first NUL, so "a\0b" and "a" would end up at one address. A string expression has type uptr.

2. Types #

Seven words in the core, and an eighth the core itself registers: i32 (M45). A ninth is type_alias (a second name for an existing one) or, since M24, type_new — a primitive the core has never heard of, registered by a module.

There is no i8/i16, no float, no bool — but each of those is now one line away, see "Types a module registers" below. A comparison yields i64 0 or 1; whether it is signed is decided by the operand types, and the rule is in "Comparisons" below.

Comparisons #

< <= > >= are unsigned when neither operand is a signed type and one of them is eight bytes wide, and signed otherwise. == and != never had a signedness to pick.

The mixed row is deliberately not C's rule. C makes i64 < u64 unsigned, which turns a negative left operand into a huge number; here the signed side wins, so a comparison that is correct today cannot change meaning. Two things follow for a program:

• a plain integer literal is i64, so u64 x; x >= 0x4000000000000000 is a signed comparison — correct for every literal below 2^63, which is every literal this language can write down. A constant with bit 63 set belongs in a u64 variable (u64 lim = one << 63;), because a cast of a literal folds to a literal and is i64 again;
• an address comparison needs no thought: uptr is eight bytes and unsigned, so cp < cend is unsigned wherever it is written.

Until contract version 6 every comparison was signed, which made u64 m = 2; m >= (one << 63) answer TRUE (machine.md § 3).

i32: the eighth word, and it is registered, not a keyword #

i32 is not in the TY_* ladder and is not a token tok_init creates. The core calls type_new("i32", 4, 4, TK_SINT) from core_types_init() before user_init(), so the word arrives through exactly the mechanism M24 gave a module — the alias table, word_add, type_of_token's last arm — and TY_MAX is still 7. Two consequences a program can see: i32 is reserved program-wide (a variable called i32 is name reserved by a syntax/type_alias registration), and a recreated compiler whose machine cannot sign-fill removes the word with type_disable(ty_i32), after which every position that names it answers i32: removed by this compiler (that is what examples/avr does).

The value of a type narrower than the word is defined by extension from its width — zero for u8/u16/u32, sign for i32:

u32 and i32 are the same four stored bytes; what differs is bits 63..32 of the value once it is read. (i32) 0xffffffff is -1 and (u32) of an i32 -1 is 4294967295.

The one observable difference from C, stated once: INT_MIN / -1. With i32 a = -2147483648; i32 b = -1;, a / b is a 64-bit sdiv on sign-extended operands and the answer is 2147483648 on every machine — no #DE trap on x86-64, where a 32-bit idiv would raise one — and INT_MIN again only after a store back into an i32. That is the price of "64-bit on the extended value", the model u32 has had since M0 (u32 a = 0xffffffff; i64 x = a + 1 is 4294967296), and it is what keeps fold() and the runtime in agreement.

uptr has no pointee type, so p + 1 is one byte further, never one element. There is no *p and no p->f: memory is read and written by explicit width (§ 5).

A cast is C-shaped, (u32) x, and unambiguous because a type keyword always follows the (. It narrows through a mask (u8/u16/u32) and is otherwise a no-op. (void) x is cast to void.

Types a module registers (M24) #

type_new(name, width, align, kind) (hooks.md § 3) appends ids past the ladder. The core appends the eighth, i32, from core_types_init() before user_init(); a module's first is the ninth. The rule that keeps the core honest is a number: an id below TY_MAX (7) is a core type and behaves exactly as it always has; an id at or above it was registered, and every core decision about it is delegated — to the registry's columns, and to nothing that tests the id itself. The core consumes three of the four columns and one reading of the fourth —

• width sizes a global, an array element and a frame slot (slot_new(type_width(ty))), so a 16-byte type gets 16 bytes of frame where an i64 gets 8;
• align is the alignment glob_place places it at;
• name is what --dump-ast prints (type=f64, not ?);
• kind — TK_INT, TK_FLOAT, TK_WIDE, TK_OPAQUE, TK_SINT — is what the machine dispatches on when it does not know the exact id. Since M45 the core reads it too, in exactly three places: type_signed (signed / % >>), fold_taught (what folds) and walk_narrow (what is extended after a call and before a return).

Everything else about such a type belongs to the module that registered it: its literals (syntax_lit), its arithmetic and its ABI (a derived machine table, machine.md) and its named hardware instructions (intrinsic). The core does not fold +, -, ~, ! or a cast over a literal of a TK_FLOAT, TK_WIDE or TK_OPAQUE type — it has no arithmetic for a representation it did not define, so the node reaches the module's machine untouched. It does fold a TK_INT or TK_SINT one, because for those two kinds the runtime is the core's own integer operators.

That is also what makes a signed narrow integer one line of a module: type_new("i16", 2, 2, TK_SINT) gets ldrsh, sxth, signed division, signed comparison and a narrowed call result from the core and from a machine that honours the kind, with no further line anywhere (lib/user_syntax_demo.mc does exactly this, and scripts/check-surface.sh asserts it).

type_new reserves the word for the whole program, exactly as type_alias does: a compiler that loads <float> has no identifier called f32. That is why <float> is not in lib/user_default.mc — nobody gets it by accident.

The name is valid in all seven type positions at once (global, local, parameter, extern, cast, array element, p_type()), because they all end in the same lookup.

A suffix on a type word the core owns #

type_new buys a new name. It cannot buy i64[], because the word that opens that type is i64 and word_add refuses the core type words — so a module that wants an element type spelled by the core and a container spelled by itself registers syntax_type(&f) (hooks.md § 3) instead. The handler is called right after the core read a type word, at every one of the six sites that read one — p_type(), a local, a cast, a parameter, an extern and a top-level declaration — receives the id the core read, may consume a suffix of its own ([], ?, *) and answers another type id, or 0 for "not mine".

What comes back is an ordinary registered type: the core reads its width, align, name and kind and nothing else, exactly as it does for a type_new the source spelled by name. lib/user_typearr.mc is the whole thing in twelve lines.

3. Expressions #

Precedence #

Higher binds tighter. This is the Pratt table --dump-rules prints; #infix and syntax_infix add rows to the same table, so a taught operator sits in one comparable order with the core's. A prefix operator has no table to grow — - ~ ! & are fixed — so a module adds one through syntax_expr instead: the handler reads the operator with p_next() and returns parse_unary() for its operand, which is what makes +x bind the way -x/!x already do (docs/reference/hooks.md § 4).

&& and || short-circuit, which is not an optimisation but a requirement: the compiler's own source is full of p != 0 && ld8(p) == 'x'.

Assignment is not in this table. x = e is a statement, not an operator, so there is no a = b = c and no if (x = 1). That is also why a syntax_infix handler can read an = of its own after the operator it owns.

// expect-exit: 42
i64 main() {
    i64 a = 2 + 4 * 10;                 // 42
    if (a != 42) return 1;
    if ((7 * 6 & 255) != 42) return 2;
    if ((1 << 5) + 10 != 42) return 3;
    u64 big = 0xFFFFFFFFFFFFFFFF;
    if (big / 2 != 0x7FFFFFFFFFFFFFFF) return 4;   // udiv: left operand is u64
    i64 neg = 0 - 8;
    if (neg / 2 != 0 - 4) return 5;                // sdiv: left operand is i64
    i64 z = 0;
    if (z && 1 / z) return 6;                      // short-circuit: no division happens
    return a;
}

Constant folding #

fold() runs over the whole unit before codegen, and folds constant against constant with the same signedness rule as the runtime. A #define is folded at definition time and is therefore a constant, never a textual macro. Folding x / 0 where both sides are constants is division by zero at compile time.

Folding stops at a kind the core's integer operators do not fit (§ 2): fold_unary, fold_binary and fold_cast each return early when an operand — or, for a cast, the destination — is TK_FLOAT, TK_WIDE or TK_OPAQUE. Without that, a float literal carried as its IEEE bit pattern would make 1.5 + 2.5 an integer add of two bit patterns and produce an infinity at compile time with no diagnostic. --dump-ast shows the tree before fold() runs, so the observable difference is in --dump-asm: a taught -1.5 leaves a fneg behind where a core -1 leaves one movz.

A TK_INT or TK_SINT type folds, core or registered, and fold_cast masks to the type's width and then fills the bytes above it by the kind — which is exactly what MTASK_CAST does at run time. So #define M (i32) 0x80000000 is the constant -2147483648 and not an instruction.

Division by zero at run time is the target's answer, not the language's #

A divisor that is only zero at run time, and INT64_MIN / -1, reach the hardware unguarded: AArch64's sdiv/udiv never trap and give 0, x and INT64_MIN; x86-64's idiv/div raise SIGFPE and the process dies; wasm will trap (docs/specs/M33.md § 8). The same source therefore behaves differently on the two shipped targets, deliberately — see ../core-language.md § "Division by zero, and INT64_MIN / -1".

4. Statements #

stmt := "if" "(" expr ")" stmt [ "else" stmt ]
      | "loop" block
      | "break" [ INT ] ";"
      | "continue" [ INT ] ";"
      | "return" [ expr ] ";"
      | type IDENT [ "[" INT "]" ] [ "=" expr ] ";"
      | IDENT "=" expr ";"
      | expr ";"
      | block
block := "{" stmt* "}"

• loop { } is the only loop. while and for come from the prelude as #rules (§ 8).
• break N; leaves N enclosing loops; break; is break 1;. N beyond the current depth is break out of range, and break 0; is break expects a positive level. No labels are needed because a level count says everything a label would.
• continue N; restarts the N-th enclosing loop, counted the way break N counts; continue; is continue 1; and restarts the innermost one. N beyond the current depth is continue out of range, continue 0; is continue expects a positive level, and a continue with no enclosing loop at all is continue outside loop. Inside a prelude for, restarting means the step is skipped — the step lives at the end of the generated body (§ 8).
• The assignment statement's left side must be a plain name (left side of assignment must be a name); assigning to an array name is assignment to array.

// expect-exit: 21
i64 main() {
    i64 i = 0;
    i64 s = 0;
    loop {
        loop {
            i = i + 1;
            if (i > 6) break 2;         // leaves BOTH loops at once
            s = s + i;
        }
    }
    return s;                            // 1+2+3+4+5+6 = 21
}

5. Memory #

There is no dereference syntax. Eight intrinsics do all of it:

&x gives the uptr of a local, a global, or a function (§ 7). An array name decays to uptr on its own. callp(p, a1, …, a7) calls the address p (§ 7).

Wrong arity on any of these is wrong arity in intrinsic.

// expect-exit: 0
// expect-stdout: 46368
#include <sys>

i64 fib(i64 n) {
    if (n < 2) return n;
    return fib(n - 1) + fib(n - 2);
}

i64 main() {
    u8 buf[24];
    st8(buf, 'x');
    if (ld8(buf) != 'x') return 1;
    putnum(fib(24));
    write(1, "\n", 1);
    return 0;
}

6. Declarations #

Locals #

type x = e; and type x[N];. A local lives on the frame at [sp, #k]. void is refused (local of type void).

• A local array's N * width must be ≤ 4095 (local array too large).
• The whole frame — locals, arrays and the spill area, rounded up to 16 — must be ≤ 4095, because sub sp, sp, #imm only carries 12 bits (frame too large).

Globals #

type x = <constant>; goes to __DATA,__data; type x[N]; with no initializer is a reservation in __DATA,__bss; type x[N] = { … } or type x[] = { … } is an initialised array in __data, aligned to 16.

• Elements are folded constants written at the type's width. Fewer elements than N zero-fills the rest; more is initializer with too many elements.
• For a uptr array, a string element writes eight zero bytes plus an UNSIGNED relocation against that string's l_strN symbol — that is how a table of pointers to literals is built.
• A non-constant initializer is global initializer must be constant.

// expect-exit: 42
uptr names[] = {"zero", "one", "two"};   // three relocated pointers in __data
u32  t[4]    = {1, 2, 3};                // the fourth element comes out zeroed
i64  sums[2] = {20 + 22, 7 * 6};         // folded in the element
u8   scratch[64];                        // __bss

i64 main() {
    if (ld8(ld64(names + 8)) != 'o') return 1;
    if (ld32(t + 12) != 0) return 2;
    st64(scratch, 1);
    if (ld64(scratch) != 1) return 3;
    return ld64(sums);
}

Functions #

type name(type a, …) { }, at most 12 parameters (at most 12 parameters): the first eight travel in registers and the rest on the stack (objects.md § 4). There are no varargs. A parameter of type void is refused. The unit is read in two top-level passes, so a function may be called before it is defined and mutual recursion needs no forward declaration.

A prototype is type name(params); with no body and no extern. The later definition must match the return type and the arity (declaration does not match prototype); a prototype nobody defines is prototype with no definition.

A call returns what it declares (M45) #

The result of a call has the type the callee's declaration gives it, and when that type is narrower than the word the compiler extends it — by the sign for i32 (and for any TK_SINT a module registered), by zero for u8/u16/u32. This is not an optimisation detail: AAPCS64, the SysV x86-64 ABI and both Windows ABIs all leave the bits above a 32/16/8-bit result unspecified, so a caller that reads all 64 of them is reading whatever the callee happened to leave. Declaring extern i32 open(...) is therefore how a C int is spelled, and i64 fd = open(...); if (fd < 0) then works.

The symmetric half holds too: an mc function declared u8 or i32 extends its own result before return, so a C caller of it gets what its ABI entitles it to. A function with no explicit return is untouched — that is what keeps the #opcode syscall wrappers, which rely on "the epilogue leaves x0 alone", working.

callp has no declaration to read, so the cast says it: a cast applied directly to a callp call declares what that call returns (§ 7). With no cast the result is i64, exactly as it always was.

// expect-exit: 42
u8  low(i64 x)  { return x; }        // 300 -> 44, on both sides of the call
i32 neg()       { return 0 - 1; }

i64 main() {
    if (low(300) != 44) return 1;
    if (neg() >= 0) return 2;        // -1, not 4294967295
    return 42;
}

extern #

extern type name(type a, …); declares an undefined symbol _name. The compiler does not check that the symbol exists. Who catches a typo depends on the output path:

• .o + ld — the link fails with Undefined symbols for architecture arm64.
• --exe — the binary is produced and signed, and dyld kills it at load time with Symbol not found, exit 134.

That difference is deliberate: --exe would have to read the SDK's .tbd files to do better, and there is no built-in list of known symbols. Use the .o path when you want the error at build time. #dylib (directives.md) says which library an extern comes from.

Declare the width C declares. A C function returning int is extern i32 f(...), not extern i64 f(...): the extra bits are unspecified and § 6 above says what the compiler does with the truthful declaration. <sys> (lib/sys.mc) is the one place in this repository that keeps i64 for a set of int-returning functions, and it does so on a measurement rather than a preference — libSystem's syscall wrappers hand back a full 64-bit -1 (M45 § Implementation notes) — and because the frozen seed compiles that file and cannot spell i32 at all. A program wanting the truthful declaration writes its own extern i32 open(uptr, i64, i64); and does not include <sys>, since two disagreeing declarations of one name are declaration does not match prototype.

c_int(v) — the same answer without the declaration. The compiler's own arena (src/arena.mc, part <mc/core_min>) exports one function for the case where the truthful declaration is not available:

i64 c_int(i64 v);      // the low 32 bits of v, sign-extended from bit 31

It is the value of a C int result read as a signed 32-bit quantity: c_int(0xffffffff) is -1, c_int(0x7fffffff) is 2147483647, and c_int(0x00000000ffffffff) and c_int(0xffffffffffffffff) are both -1 — which is the point. Bits 63..32 of a 32-bit return are unspecified on every ABI mc targets, so c_int is correct whether the callee sign-extended, zero- extended or left rubbish there, and it is pure arithmetic, so it needs no support from the machine.

src/ uses it instead of narrow declarations, and that is a constraint rather than a preference: src/*.mc and lib/sys.mc are compiled by the frozen C seed, which cannot spell i32 at all, and declaring those results u32 would make this compiler emit an extension the seed cannot emit — scripts/check-asm.sh compares the two over exactly those files. The call sites are c_int(open(...)), c_int(creat(...)) and c_int(waitpid(...)). <sys> (lib/sys.mc) keeps i64 for the same reason and does not need c_int, on the measurement recorded above.

A program that can be compiled by mc itself — anything outside the seed set — has no reason to use c_int: it writes extern i32 f(...) and the compiler does the extension at the call.

7. Function pointers #

The core has no function type. &name on a function or an extern is a uptr like any other (adrp/add, relocations PAGE21 + PAGEOFF12), and the indirect call is an intrinsic:

callp(p, a1, …, a11) puts p in x16 and the arguments where a direct call would put them — x0..x7, then the stack — and issues blr x16. Live depth registers are saved exactly as for a bl. Arity is 1 to 12 counting the pointer (callp expects 1 to 12 arguments).

A cast applied directly to a callp declares what the call returns. There is no callee to read a signature from and a cast is the only syntax in the language that names a type inside an expression, so (i32) callp(p, x) is the declaration and not a conversion: the compiler types the call i32, M45's call-side extension applies to it (§ 6), and the written cast then repeats an extension that is idempotent. With no cast the call is i64 and every bit the callee left in x0 comes back untouched.

The rule is what makes an indirect call to a function returning something other than a machine word work at all. A taught type is the case that cannot be written any other way: with <float> (../guide/96-a-new-primitive.md), (f64) callp(&dbl, 2.0) is what tells the machine the value comes back in d0 — without the cast the machine is told "integer", moves x0, and the arithmetic around it silently reads the pointer's own bits.

// expect-exit: 42
i64 wide(i64 x) { return 0x1234567800000000 + x; }

i64 main() {
    uptr p = &wide;
    return (i32) callp(p, 42);       // 42, not 1311768464867721258
}

// expect-exit: 42
i64 add2(i64 a) { return a + 2; }
uptr tbl[1];

i64 main() {
    st64(tbl, &add2);
    return callp(ld64(tbl), 40);
}

Known limit on the .o + ld path only: &name for an extern that lives in a dylib produces a correct object, but ld refuses it — an imported symbol only has an address through __got and the core does not emit GOT_LOAD_PAGE21. Through --exe it always works, because mc resolves the relocation itself and points it at the symbol's stub.

8. The prelude #

while, for, +=, -=, ++ and -- are not core syntax. They are four #tokens and six #rules in lib/prelude.mc, bundled as <prelude>, and they arrive only through an explicit include.

Consequences worth knowing:

• The body is always a block. while (c) x++; without braces does not match the pattern.
• for's step is ident $x = expr $step, because assignment is a statement in the core, so it is written i = i + 1 and not i++.
• for (; c; s) does not exist: the pattern requires a stmt $init and the core has no empty statement. Use while.
• while and for become reserved words from the include on: a rule's dispatch literal is registered in the lexer, so i64 while = 1; stops being a valid declaration.

// expect-exit: 0
// expect-stdout: 45
#include <sys>
#include <prelude>

i64 main() {
    i64 s = 0;
    for (i64 i = 0; i < 10; i = i + 1) {
        s += i;
    }
    putnum(s);
    write(1, "\n", 1);
    return 0;
}

9. Program shape and entry point #

A unit is a sequence of top-level declarations: globals, prototypes, externs, functions and directives. The entry point is i64 main(...); --exe refuses a unit without one (no _main: cannot generate an executable). main receives argc in the first parameter and argv — a uptr to an array of uptr — in the second, so argv[i] is ld64(argv + i * 8).

// expect-exit: 2
i64 main(i64 argc, uptr argv) {
    if (ld8(ld64(argv)) == 0) return 1;   // argv[0] is a non-empty string
    return argc + 1;                       // run with no arguments: 1 + 1
}

There is no runtime and no libc unless you include one: <sys> declares libSystem's open/creat/read/write/close/exit and adds strlen/puts/putnum written in the language; <sys_svc> is the same interface through svc #0x80 with no libSystem at all; <sys_linux_aarch64> / <sys_linux_x86_64> are the Linux syscall layer plus a _start. See bundle.md.

Memory is a global array plus a bump pointer — that is what src/arena.mc is. There is no malloc and no free in the language.

10. Limits #

Everything here is a real diagnostic; the full catalogue is diagnostics.md.

Every other table in the self-hosted compiler — nodes, symbols, strings, functions, the token table — has no ceiling: it is an arena block that doubles on demand, pre-sized from an estimate. mc limits reports the sizing; see cli.md and toml.md.

The C seed still has fixed MAX* constants, because it only ever has to compile src/mc.mc. make check-limits is the guard that warns before one of them fills up.

11. What the core deliberately does not have #

struct, while/for, switch, goto, labels, float, bool, i8/i16/i32, typed pointers, varargs, += and ++, multiple assignment, the ternary operator, operator overloading, modules, namespaces, generics, classes, and a standard library.

Most of them are reachable through the surface instead: while/for/+=/++ from the prelude, bool from type_alias, f32/f64 from <float> — a library, not a keyword, with its literals, its four ABIs and its instructions in lib/ and nothing in src/ — over M24's type_new/syntax_lit/intrinsic and a derived machine (hooks.md § 3, ../guide/96-a-new-primitive.md), classes and generics from syntax/syntax_expr (examples/lang teaches all of the last row). What is genuinely absent is struct — it would need a type $t hole in #rule and a layout model, which is more than #rule delivers; the answer in this codebase is #define offsets plus accessor functions, which is also what makes the compiler's own data flat and portable.

Edit this page