Lexical Structure

This page describes how the Braid lexer tokenizes source code. The lexer in braid-lang/src/compiler/lexer.c converts raw source text into tokens that the parser consumes.

Source Files

Braid source files use the .br extension. Script files use .bd. Compiled bytecode artifacts use .bx. Source files are UTF-8 text.

Comments

The lexer recognizes three comment styles:

// Line comment — everything until newline is ignored

/* Block comment — everything between /* and */ is ignored */

Hash comments (# line comment) are also supported at the lexer level.

Identifiers

Identifiers start with a letter (a-z, A-Z) or underscore (_), followed by zero or more letters, digits, or underscores:

let myVar = 1
let _private = 2
let camelCase123 = 3

Keywords

The following identifiers are reserved keywords (from braid.h: BraidTokenType):

fn     let    if     else   while  return
struct enum   match  model  import extern
impl   native true   false
diameter pole  observe evolve

Operators

Arithmetic:

+    Addition / string concatenation -    Subtraction / unary negation *    Multiplication /    Division %    Modulo

Comparison:

==   Equal !=   Not equal <    Less than <=   Less than or equal >    Greater than >=   Greater than or equal

Logical:

&&   AND ||   OR !    NOT (unary prefix)

Other:

=    Assignment ->   Return type arrow =>   Match case fat arrow @    Decorator prefix ...  Ellipsis (tensor slicing) .    Member/field access |    Pipe operator

Literals

Integer literals: Decimal digits only (lexer supports TOK_NUMBER for integers):

let a = 42
let b = 0
let c = 1000000

Float literals: Digits with a decimal point:

let pi = 3.14
let e = 2.71828
let zero = 0.0

String literals: Enclosed in double or single quotes:

let s1 = "hello"
let s2 = 'world'
let s3 = "string with spaces"

Boolean literals:

let t = true
let f = false

Nil literal: The null value:

let nothing = nil

Delimiters

{    Left brace — starts blocks, struct/diameter bodies}    Right brace — ends blocks (    Left paren — function params, grouping )    Right paren [    Left bracket — arrays, tensor literals, indexing ]    Right bracket ;    Semicolon — terminates statements ,    Comma — separates elements :    Colon — type annotations, struct field syntax

Operator Precedence (high to low)

1. () parentheses
2. - unary negation, ! logical NOT
3. * / % multiplicative
4. + - additive
5. < <= > >= == != comparison
6. && logical AND
7. || logical OR

Example: Full Tokenization

// Source:
let x: int = 10 + 20;

// Tokens:
TOK_LET   "let"
TOK_IDENTIFIER "x"
TOK_COLON ":"
TOK_IDENTIFIER "int"
TOK_ASSIGN "="
TOK_NUMBER "10"
TOK_PLUS "+"
TOK_NUMBER "20"
TOK_SEMICOLON ";"
TOK_EOF