Adapted, not copied

Original notes for JavaScript360.org — project concept inspired by Eloquent JavaScript (4th ed.) by Marijn Haverbeke, built here with an original language design, syntax, and code. See the full attribution note.

Chapter 12 · Project: A Tiny Programming Language

Estimated time: ~45 minutes · Chapter type: Guided project

Every language you've ever used — including JavaScript itself — works the same way under the hood: source text gets turned into a structured representation (a syntax tree), and something walks that tree, executing it. In this project we build both halves for a deliberately tiny language we'll call Pebble, just expressive enough to define and call functions, do arithmetic, and branch on conditions.

What Pebble looks like

let square = fn(x) { x * x };
let max = fn(a, b) { if (a > b) { a } else { b } };
print(max(square(3), 20));

That's it — variable bindings, function definitions, function calls, arithmetic, and if. It's not going to replace JavaScript, but building it will teach you more about how any language works than years of just using one.

Step 1: parsing — turning text into tokens

The first stage, tokenizing, breaks the raw source text into meaningful chunks (numbers, identifiers, symbols), discarding whitespace:

function tokenize(source) {
  const tokenPattern = /\s*(=>|[-+*/%(){},;=<>!]|[A-Za-z_]\w*|\d+(\.\d+)?)/y;
  const tokens = [];
  let pos = 0;
  while (pos < source.length) {
    tokenPattern.lastIndex = pos;
    const match = tokenPattern.exec(source);
    if (!match || match[1] === undefined) {
      pos++; // skip stray whitespace the pattern didn't consume
      continue;
    }
    tokens.push(match[1]);
    pos = tokenPattern.lastIndex;
  }
  return tokens;
}

tokenize('let x = 2 + 3;');
// ["let", "x", "=", "2", "+", "3", ";"]

(This reuses the sticky-flag regex trick from Chapter 9 — y anchors each match to exactly lastIndex, which is what lets us walk through the string token by token.)

Step 2: parsing — turning tokens into a tree

Next, a parser consumes the token list and builds an abstract syntax tree (AST) — nested objects describing the program's structure. We use recursive descent: one function per grammar rule, each calling the others for its sub-parts. Here's the core of an expression parser handling numbers, identifiers, and binary operators:

function parseExpression(tokens) {
  let left = parsePrimary(tokens);
  while (['+', '-', '*', '/', '>', '<'].includes(tokens[0])) {
    const operator = tokens.shift();
    const right = parsePrimary(tokens);
    left = { type: 'BinaryOp', operator, left, right };
  }
  return left;
}

function parsePrimary(tokens) {
  const token = tokens.shift();
  if (/^\d+$/.test(token)) {
    return { type: 'Number', value: Number(token) };
  }
  if (token === '(') {
    const expr = parseExpression(tokens);
    tokens.shift(); // consume ")"
    return expr;
  }
  return { type: 'Identifier', name: token }; // a variable reference or function call target
}

parseExpression(['2', '+', '3']) produces:

{
  type: 'BinaryOp',
  operator: '+',
  left: { type: 'Number', value: 2 },
  right: { type: 'Number', value: 3 },
}

That nested-object shape is the abstract syntax tree — a direct, structural representation of "2 plus 3," with no more ambiguity about order of operations, precedence, or grouping.

Step 3: the evaluator

The second half is an evaluator — a function that walks the AST and actually computes a result, given an environment (a plain object mapping names to values, exactly like the scope concept from Chapter 3):

function evaluate(node, env) {
  switch (node.type) {
    case 'Number':
      return node.value;

    case 'Identifier':
      if (!(node.name in env)) throw new Error(`Undefined variable: ${node.name}`);
      return env[node.name];

    case 'BinaryOp': {
      const left = evaluate(node.left, env);
      const right = evaluate(node.right, env);
      switch (node.operator) {
        case '+': return left + right;
        case '-': return left - right;
        case '*': return left * right;
        case '/': return left / right;
        case '>': return left > right;
        case '<': return left < right;
      }
      break;
    }

    default:
      throw new Error(`Unknown node type: ${node.type}`);
  }
}

const ast = parseExpression(tokenize('2 + 3 * 4'));
evaluate(ast, {}); // 14 — note this simple version doesn't yet handle operator precedence!

That last comment is worth sitting with: the parser above is deliberately simplified and treats every operator with equal precedence, left to right, so 2 + 3 * 4 naively evaluates as (2 + 3) * 4 = 20 rather than the mathematically correct 2 + (3 * 4) = 14. Real parsers solve this with precedence climbing — parsing higher-precedence operators (*, /) in a nested function called from the lower-precedence one (+, -), so tighter-binding operators naturally end up deeper in the tree. It's a great next step once you've got the basic pipeline working — see "Extend it yourself" below.

Step 4: special forms — if and function definitions

Not everything in a language is a simple expression. if needs to evaluate only one of its branches (never both — that would defeat the purpose of conditional logic and could cause infinite loops or unwanted side effects), and function calls need to create a new environment for their parameters, layered on top of (but not replacing) the outer one — precisely the closure behavior from Chapter 3, now implemented explicitly instead of relying on JavaScript's own engine to do it:

function evaluateIf(node, env) {
  if (evaluate(node.condition, env)) {
    return evaluate(node.thenBranch, env);
  } else if (node.elseBranch) {
    return evaluate(node.elseBranch, env);
  }
}

function evaluateCall(node, env) {
  const fn = evaluate(node.callee, env);
  const args = node.args.map((arg) => evaluate(arg, env));

  // A fresh environment for the call, "inheriting" from where the function was defined —
  // this is what makes Pebble functions behave like closures, not global-only functions.
  const callEnv = Object.create(fn.closureEnv);
  fn.params.forEach((param, i) => { callEnv[param] = args[i]; });

  return evaluate(fn.body, callEnv);
}

Object.create(fn.closureEnv) is doing real work here — it makes callEnv's prototype the environment the function was defined in (Chapter 6's prototype chain, repurposed as a scope chain), so looking up a variable that isn't a parameter automatically falls back to the outer scope, exactly like real lexical scoping.

Step 5: cheating (on purpose)

A genuinely complete language needs primitives for things like print, arithmetic beyond what we've built, and more. Rather than implementing everything in Pebble itself, it's completely standard practice to expose a handful of host functions — real JavaScript functions — directly into the initial environment:

const globalEnv = {
  print: (value) => { console.log(value); return value; },
};

Every real language does this at some level (JavaScript itself calls down into lower-level engine code for things like array sorting) — there's no shame in "cheating" by leaning on the host language for primitives your toy language doesn't need to reimplement from scratch.

What this project actually taught you

• The two-stage pipeline — parse text into structure, evaluate structure into results — behind every interpreter and compiler you'll ever use
• Recursive descent parsing, a technique that shows up constantly (config file parsers, template engines, even parts of your build tooling)
• How lexical scoping and closures can be implemented explicitly, rather than taken for granted
• That "how does a programming language work?" has a genuinely learnable, buildable answer — it's not magic

Extend it yourself

• Fix the precedence bug: split parseExpression into parseTerm (handles *//) called from parseAddition (handles +/-), so multiplication correctly binds tighter than addition.
• Add support for let bindings as statements, and a sequence of statements separated by ;.
• Add a while loop special form, following the same pattern as if above.

 

Up nextCh. 13 — JavaScript and the Browser~25 min