Nelumbo

Built-in tokens and pattern holes

Level: nelumbo.lang. Available after import nelumbo.lang, or transitively through nelumbo.logic or any package.

When you declare a pattern with ::=, everything in angle brackets (<...>) is a hole — a placeholder that the parser fills in with a matching fragment of input. This page catalogues the kinds of hole that are built into Nelumbo and used throughout the standard library.

Integer ::= <NUMBER>                  @org.modelingvalue.nelumbo.integers.NInteger
Integer ::= <Integer> + <Integer>     #40
Set<E>  ::= { <(> <(> <E> <|> .. <)> <,> , <)*> }    @org.modelingvalue.nelumbo.collections.NSet

A hole is either a lexical-token hole (matches a single lexer token class), a type hole (matches an expression of some declared type), or a structural marker (builds repetition and grouping).

Lexical-token holes

These holes match a single token produced by the lexer, not an expression built from rules. They are Nelumbo's escape hatch to raw input.

<NUMBER>

Matches an unsigned integer token: one or more decimal digits.

Integer ::= <(> - <)?> <[> <NUMBER> <(> "#" <(> <(> <NUMBER> <|> <NAME> <)> <)+> <)?> <]>
            @org.modelingvalue.nelumbo.integers.NInteger

Examples that match <NUMBER> on its own: 0, 1, 42, 1000000. The leading sign and the optional #-prefixed base form are built at the pattern level, not by the lexer — the Integer literal above wraps <NUMBER> with an optional - and an optional #-radix tail.

Nelumbo's integer literal therefore admits base-N literals of the form N#digits, where N is the base (up to 36) and digits are digits in that base. This is how arbitrary-precision integers are printed for readability once they get large:

36#22r8fozas3n8w3
36#18nrvsuayughau0blk8aylvbyaqwiaqba77rdsgscn5hzwgbgaws8i8svp4xdmoo82plxiyogd5iaj1cspez8zfeio92a76t9n1frssxklr92wyyxm8r903o1ofgncikuggcwnf

Both of the above are base-36 integer literals — the values of fib(100) and fib(1000) respectively.

Rational literals are likewise built by composing two <NUMBER> tokens around a .:

Rational ::= <(> - <)?> <[> <NUMBER> . <NUMBER> <]>
             @org.modelingvalue.nelumbo.rationals.Rational

Examples that match: 0.0, -1.5, 3.14. There is no separate <DECIMAL> token — the decimal-point form is assembled by the rationals pattern, not by the lexer.

<STRING>

Matches a double-quoted string literal:

String ::= <STRING>   @org.modelingvalue.nelumbo.strings.NString

Examples that match: "", "foo", "Hello, World!".

<NAME>

Matches an identifier token — the kind of lexical fragment used for literal enumerations and parameters in pattern transformations:

Root ::= attr <Type> <NAME> <Type>  #100

From transformation.nl. The <NAME> hole captures a raw identifier that the transformation can use as the name of a new attribute.

<NAME> is primarily useful inside pattern transformations where you need the user's literal identifier to build new declarations on the fly. See ../guides/language-transformations.md.

Type holes — <T>

A type hole matches an expression of type T. The expression may itself be complex — it can be any pattern declared for type T, including the one currently being declared (allowing recursive patterns like <Integer> + <Integer>).

Integer ::= <Integer> + <Integer>    #40
Integer ::= fib(<Integer>)
Boolean ::= even(<Integer>)

A type hole may carry a precedence annotation:

T ::= <Boolean#5> ? <T> : <T>

From ternary.nl. The #5 restricts what can appear in the hole to expressions with precedence at least 5 — used to avoid ambiguity with surrounding operators. See precedence-and-associativity.md.

A type hole may also be marked with a visibility modifier:

Integer ::= <hidden Integer>  $$ <Integer>  #35
Integer ::= <visible Integer> $  <Integer>  #35

From hidden.nl. A <hidden T> hole accepts only an operand that the source leaves out (the parser fills in the hidden variable in scope); a <visible T> hole only a written operand. See visibility.md.

Variable holes — <Variable>

<Variable> matches a variable binding site, not a general expression. It is what lambdas use to introduce a bound variable (and quantifiers, which are lambdas), typed with the lambda's argument type:

Lambda2<A1,A2,R> ::= [<{Variable,A1}>,<{Variable,A2}>](<R#0>)   @nelumbo.logic.Lambda
Boolean          ::= E<Lambda<Boolean>>                         @nelumbo.logic.ExistentialQuantifier

From logic.nl (see lambdas). The quantifier E[x, y, z](body) expects binding sites, not pre-existing expressions, in the bracketed position.

Binding variables declared with <Variable> are scoped to the surrounding pattern — they do not leak outside.

Structural markers — repetition and grouping

Inside a pattern, special angle-bracketed operators build repeating and optional sub-structures. They are markers, not holes — they do not consume input themselves; they shape how the surrounding holes consume input.

Marker Role
<(> ... <)> Pure grouping — no repetition
<(> ... <)?> Optional group — zero or one occurrence
<(> ... <)*> Zero-or-more, separator-delimited
<(> ... <)+> One-or-more, separator-delimited
<,> Separator placeholder inside a repetition
<|> Alternation inside a group

Examples

Repetition  ::= { <(> <Integer> <,> , <)*> }     // {}, {5}, {3,5,7}
Option      ::= <(> super <)?> fast              // "fast" or "super fast"
Alternation ::= <(> A <|> B <|> C <)>            // "A", "B", or "C"

In a repetition, the literal that follows <,> is the actual separator token (, in the Repetition example above). The marker <,> says "here is the separator"; the real character is whatever follows.

Generic collections

The collections module uses repetition to define Set and List:

Set<E>  ::= { <(> <(> <E> <|> .. <)> <,> , <)*> }
List<E> ::= [ <(> <(> <E> <|> .. <)> <,> , <)*> ]

Both say "zero or more items, comma-separated, wrapped in brackets", where an item is an <E> or the incompleteness marker .. (an alternation nested in the repetition). {} and [] are empty literals for set and list respectively. {1,2,3} matches, as does [1], as does {}; {1,2,..} and {..} are open sets.

Precedence and native-binding annotations

Two trailing annotations can attach to a pattern declaration (not to individual holes, unless the hole is a type hole):

#N — precedence

Integer ::= <Integer> + <Integer>  #40

Declares that this pattern has precedence 40. See precedence-and-associativity.md.

@ClassName — native binding

Integer ::= <NUMBER>  @org.modelingvalue.nelumbo.integers.NInteger

Binds the pattern to a Java class that implements its semantics. See native-api.md (Phase 4).

See also