fogserv.cloud fogserv.cloud

Runtime Internals — How Runtimes Work Under the Hood

Status: Active | Last Updated: 2026-08-29 Category: Runtime — Internals Prerequisites: Bun runtime or Node.js experience Tags: event-loop, v8, javascriptcore, jit, worker-threads, wasm Estimated Time: 5–7 hours (Self-paced, includes lab time)

Summary

What happens between bun file.ts and your code actually running? This article explains the engine, the event loop, JIT compilation, and how to make your code run faster. Useful for debugging performance issues, understanding async ordering, and knowing when Node's worker threads or Bun's WASM support help.

What You'll Learn (Core Competencies)

Table of Contents

  1. Architectural Overview & Core Schema
  2. Deep Dive & Implementation
  3. Anti-Patterns & Common Pitfalls
  4. Independent Challenge
  5. Consolidation & Key Invariants
  6. Next Steps

1. Architectural Overview & Core Schema

Foundational theory, system mechanics, and high-level design.

Engine Architecture

Both engines compile JavaScript to machine code at runtime. The difference is the strategies for when and how aggressively to optimise.

The Pipeline: Source to Machine Code

Source code (.js / .ts)
  ↓
  Parser → AST (Abstract Syntax Tree)
  ↓
  Bytecode (Ignition in V8, LLInt in JSC)
  ↓
  Execution — interpreter runs bytecode
  ↓
  Profiling — interpreter tracks hot functions, types, branches
  ↓
  Tier-up — JIT compiler generates optimised machine code
  ↓
  Deoptimise (if assumptions break)

The Interpreter

Every JavaScript engine starts with an interpreter that runs bytecode. The interpreter is fast to start, fast to interpret simple code, and doesn't assume anything about types. V8's interpreter is called Ignition; JSC's is LLInt (Low-Level Interpreter).

// The interpreter handles this directly
function add(a, b) {
    return a + b;
}
add(1, 2);  // run as bytecode

Why Profiling Matters

The interpreter doesn't just run code — it watches it. It tracks:

This profile data is what tells the JIT compiler where to focus optimisation effort.

V8 vs JavaScriptCore — Different Strategies

V8 (Node, Deno, Chrome):

AST → Ignition bytecode → (hot code) → Sparkplug baseline JIT → (still hot) → Maglev → TurboFan

JavaScriptCore (Bun, Safari, WebKit):

AST → LLInt bytecode → (hot code) → Baseline JIT → (still hot) → DFG → (still hot) → FTL

Both engines try to spend less time on cold code and more on hot code. JSC has one more tier than V8, so it can squeeze out more performance for code that runs a long time. V8's Maglev tier was added more recently to close the gap.

The Event Loop

Every async runtime has an event loop. The loop runs continuously, processing tasks and microtasks:

while (true) {
    runTimers()          // setTimeout, setInterval callbacks
    runPendingCallbacks() // I/O callbacks
    runIdle()             // internal housekeeping
    runPoll()             // I/O polling
    runCheck()            // setImmediate (Node) callbacks
    runClose()            // close events
    runMicrotasks()       // Promise.then, queueMicrotask (always runs after current task)
}

Key insight: microtasks (Promise resolution) run after the current task completes, before the next task. This means:

console.log("1");
Promise.resolve().then(() => console.log("3"));
setTimeout(() => console.log("4"), 0);
console.log("2");

// Output: 1, 2, 3, 4
// Microtasks (3) run before next macrotask (4)

The Call Stack and the Heap

The runtime has two main memory areas:

function c() {
    throw new Error("stack trace shows a -> b -> c");
}

function b() {
    c();
}

function a() {
    b();
}

a();
// Stack:
// [a] [b] [c]  ← top of stack

If the stack grows unbounded (e.g., infinite recursion), you get "Maximum call stack size exceeded."

Tasks vs Microtasks

Two queues, different priorities:

After every task, the runtime drains the entire microtask queue before moving on. This is why Promise.then always runs before setTimeout in the same tick.

Promise.resolve().then(() => {
    console.log("microtask 1");
    Promise.resolve().then(() => console.log("nested microtask"));
});
setTimeout(() => console.log("macrotask"), 0);

console.log("sync");
// Output: sync, microtask 1, nested microtask, macrotask

Why Microtasks Run So Often

Microtasks drain after every task and after every await. This is important for await:

async function f() {
    console.log("1");
    await something();
    console.log("2");  // runs as a microtask after the await
}

The "2" runs as soon as the microtask is dequeued, which can be much sooner than the next macrotask.

I/O and libuv

Node uses libuv to handle I/O across platforms. libuv has:

// These are offloaded to the libuv thread pool
import { readFile } from "fs/promises";
const data = await readFile("big.txt");

// These use the OS event loop directly
import { createServer } from "http";
const server = createServer((req, res) => {
    // runs in event loop, no thread pool involvement
});

Default thread pool size is 4. Set with UV_THREADPOOL_SIZE=8 node app.js.

2. Deep Dive & Implementation

Technical implementation, fully commented code blocks, and step-by-step logic.

JIT Compilation

Tier-Up: When Code Gets Faster

The interpreter profiles code as it runs. When a function is "hot" (called many times, or in a tight loop), the runtime sends it to a JIT compiler:

function sum(arr) {
    let total = 0;
    for (let i = 0; i < arr.length; i++) {
        total += arr[i];
    }
    return total;
}

// First 100 calls: interpreted (bytecode)
// Next: Sparkplug compiles to machine code
// After 10k+ calls: Maglev optimises with type feedback
// After 1M+ calls: TurboFan applies deep optimisation
sum(hugeArray);

Each tier trusts the types it has seen. If a function always receives integers, the JIT generates integer-only machine code — much faster than generic code.

When JIT Optimises and Deoptimises

The JIT makes optimistic assumptions. When those assumptions break, it deoptimises:

function double(x) {
    return x * 2;
}

double(5);       // JIT compiles with int multiplication
double(10);      // still int — runs compiled code
double("hello"); // type assumption broke — DEOPT
double(7);       // now runs interpreted or recompiled

Tips to stay in the optimised tier:

Hidden Classes and Inline Caches

V8 uses hidden classes (also called shapes or maps) to optimise object property access. When all objects of a class have the same shape, V8 generates fast property access code:

class Point {
    constructor(x, y) {
        this.x = x;  // shape: { x: any }
        this.y = y;  // shape: { x: any, y: any }
    }
}

const p1 = new Point(1, 2);
const p2 = new Point(3, 4);  // same shape — fast inline cache hit
const p3 = { y: 5, x: 6 }; // different shape — slower

// Bad: changing shape mid-life
function bad(p) {
    p.z = 10;  // adds property — different shape for next access
}

Optimization tips:

Inline Caching

The JIT caches the result of property lookups:

function getX(obj) {
    return obj.x;
}

getX({ x: 1 });  // cache miss
getX({ x: 2 });  // cache miss
getX({ x: 3 });  // cache miss

getX({ x: 4 });  // after 3 hits, V8 uses the inline cache
getX({ x: 5 });  // cache hit — fast

Once the cache has a hit, the same property access on the same shape runs at native speed.

Worker Threads and Concurrency

Node is single-threaded for JavaScript. The main thread runs your code, the libuv thread pool handles I/O, and the event loop dispatches callbacks. For CPU-bound work, you need worker threads (or child processes).

When You Need Workers

For I/O-bound work, the event loop + libuv already keeps your code responsive. Don't reach for workers prematurely.

Basic Worker Thread

main.js:

import { Worker } from "node:worker_threads";

const worker = new Worker(new URL("./worker.js", import.meta.url), {
    workerData: { input: 1_000_000 }
});

worker.on("message", (result) => {
    console.log("result:", result);
});

worker.on("error", (err) => {
    console.error("worker error:", err);
});

worker.on("exit", (code) => {
    console.log("worker exited with code", code);
});

worker.js:

import { parentPort, workerData } from "node:worker_threads";

// CPU-bound work
function fibonacci(n) {
    if (n < 2) return n;
    return fibonacci(n - 1) + fibonacci(n - 2);
}

const result = fibonacci(workerData.input);
parentPort.postMessage(result);

Shared Memory with SharedArrayBuffer

Workers can share memory directly (faster than message passing for large data):

// main.js
import { Worker } from "node:worker_threads";

const buffer = new SharedArrayBuffer(4 * 1024 * 1024);  // 4MB shared
const view = new Int32Array(buffer);

const worker = new Worker(new URL("./counter.js", import.meta.url), {
    workerData: { buffer }
});

worker.on("message", () => {
    console.log("counter value:", Atomics.load(view, 0));
});

// counter.js
import { workerData } from "node:worker_threads";

const view = new Int32Array(workerData.buffer);
let count = 0;

for (let i = 0; i < 1_000_000; i++) {
    // Atomic increment — safe across threads
    Atomics.add(view, 0, 1);
}

parentPort.postMessage("done");

Atomics ensures thread-safe operations on shared memory.

Worker Pool

For many small tasks, use a pool to avoid spawning overhead:

import { Worker } from "node:worker_threads";
import os from "node:os";

class WorkerPool {
    constructor(workerPath, size = os.cpus().length - 1) {
        this.workers = [];
        this.queue = [];

        for (let i = 0; i < size; i++) {
            this.workers.push(this.createWorker(workerPath));
        }
    }

    createWorker(path) {
        const worker = new Worker(path);
        worker.busy = false;
        worker.on("message", () => {
            worker.busy = false;
            this.processQueue();
        });
        return worker;
    }

    run(data) {
        return new Promise((resolve, reject) => {
            this.queue.push({ data, resolve, reject });
            this.processQueue();
        });
    }

    processQueue() {
        if (this.queue.length === 0) return;
        const worker = this.workers.find((w) => !w.busy);
        if (!worker) return;

        const { data, resolve, reject } = this.queue.shift();
        worker.busy = true;
        worker.once("message", resolve);
        worker.once("error", reject);
        worker.postMessage(data);
    }
}

Bun's Approach

Bun optimises the single-threaded case aggressively. For parallel work, Bun offers:

Bun's startup time is much faster than Node, so for short-lived parallel tasks, the difference can be significant.

WebAssembly

WebAssembly (WASM) is a portable, low-level binary format. Languages like C, C++, Rust, and Go compile to WASM modules that run in JavaScript runtimes.

Why WASM?

Compiling Rust to WASM

# Add target
rustup target add wasm32-unknown-unknown

# Compile
cargo build --target wasm32-unknown-unknown --release

// Load and use in JavaScript
import { readFile } from "fs/promises";
import path from "path";

const wasmBuffer = await readFile(path.join(import.meta.dirname, "target/wasm32-unknown-unknown/release/mylib.wasm"));
const wasmModule = await WebAssembly.instantiate(wasmBuffer, {
    env: {
        // import functions provided to WASM
        log: (ptr, len) => {
            const bytes = new Uint8Array(wasmModule.exports.memory.buffer, ptr, len);
            console.log(new TextDecoder().decode(bytes));
        }
    }
});

const result = wasmModule.exports.compute(42);
console.log("WASM result:", result);

Use Cases

WASM Limitations

For pure business logic, JavaScript is usually faster to develop and similar in performance. Reach for WASM when you have specific performance, security, or reuse needs.

Profiling Your Code

Node Built-in Profiler

# CPU profile (10 second sample)
node --prof app.js
# Generates isolate-*.log
node --prof-process isolate-*.log > profile.txt

# Heap profile (memory)
node --heap-prof app.js

Or use the inspector with Chrome DevTools:

node --inspect app.js
# Open chrome://inspect in Chrome
# Click "inspect" to see profiles, flame charts, snapshots

V8 Hidden Class Inspector

node --print-opt-code app.js     # dumps optimised code
node --trace-opt app.js          # shows optimisation decisions
node --trace-deopt app.js        # shows deoptimisations

The most common deopt cause: polymorphic call sites or unstable object shapes. Look for "deopted" in the trace.

Production Profiling

For production, use 0x (flame graph generator):

npx 0x app.js
# Generates a flame graph in ~/Documents/0x/<pid>.flamegraph.html

Each bar is a function. Wide bars = expensive functions. Tall stacks = deep call chains.

Memory Leak Detection

# Heap snapshot at startup
node --inspect app.js
# Take snapshots in DevTools, compare over time

# Programmatic snapshots
import v8 from "node:v8";
v8.writeHeapSnapshot("snap-" + Date.now() + ".heapsnapshot");

In DevTools, compare two snapshots. Objects that grow between snapshots are leaking.

Bun Profiling

Bun ships with built-in profiling:

bun --hot --inspect app.ts
# Same Chrome DevTools workflow as Node

Bun also has --cpu-prof and --heap-prof flags.

Guided Checkpoint

Run the following to verify your understanding of runtime internals — all commands should complete without errors and produce predictable output.

# 1. Confirm which engine your runtime uses
node -e "console.log(process.versions.v8)"    # V8 (Node)
# bun --version is not a V8 version check; Bun uses JavaScriptCore

# 2. Trigger and observe a deoptimisation
node -e "
function double(x) { return x * 2; }
for (let i = 0; i < 100000; i++) double(i);    // warm up — JIT compiles
double('string');                               // type assumption breaks
" 2>&1 | grep -i deopt || echo "No deopt trace (add --trace-deopt flag)"

# 3. Verify UV_THREADPOOL_SIZE effect
UV_THREADPOOL_SIZE=16 node -e "
  console.log('Thread pool size:', process.env.UV_THREADPOOL_SIZE);
"

# 4. Run the event loop phase-order test
node -e "
  console.log('sync');
  setTimeout(() => { console.log('timeout'); Promise.resolve().then(() => console.log('timeout-microtask')); }, 0);
  setImmediate(() => { console.log('immediate'); queueMicrotask(() => console.log('immediate-queueMicrotask')); });
  Promise.resolve().then(() => console.log('promise-then'));
  queueMicrotask(() => console.log('queueMicrotask'));
  console.log('sync-end');
"

3. Anti-Patterns & Common Pitfalls

Documented failure modes and how to detect/prevent them.

1. Microtask starvation

Microtasks drain after every task and after every await. If a promise chain keeps scheduling new microtasks (via .then() that returns a resolved promise), the microtask queue never fully drains, and macrotasks (I/O callbacks, timers) never get a chance to run. This can make the process appear unresponsive even when the CPU is idle.

Detection: Under moderate load, setTimeout callbacks are delayed by hundreds of milliseconds despite a lightly-loaded system. Adding a setTimeout(..., 0) in a test harness shows it never fires. Prevention: Use setImmediate to yield back to the event loop between batches of microtasks. Limit the depth of promise chains; prefer async/await with explicit await at boundaries to let the event loop breathe.

2. Creating objects with unstable shapes in hot paths

Adding properties to objects after construction, deleting properties, or initialising properties in different orders across different call sites causes V8 to track multiple hidden classes. This prevents inline cache hits and forces the JIT to use slower generic property lookup code.

Detection: node --trace-opt app.js | grep -v "not optimized" shows functions repeatedly deoptimising. Profile output shows KeyedStoreIC or LoadIC taking disproportionate time. Prevention: Always initialise all properties in the constructor. If an object needs dynamic keys, use a Map or WeakMap instead. Never delete obj.prop in production hot paths.

3. Polymorphic call sites across many types

A function called with many different argument types (e.g., a format function called with string, number, Date, and custom objects) creates a polymorphic or megamorphic call site. The JIT can no longer inline the call and must use a generic dispatch stub.

Detection: node --trace-opt app.js shows "NOT OPTIMISED" for functions with polymorphic arguments. Flame graphs show a wide stub dispatcher taking significant time. Prevention: Use function overloads or split the function into typed variants.

4. Deoptimising by mixing types in the same array

When an array contains both integers and floating-point numbers (or strings interleaved with numbers), the JIT can no longer assume a single numeric representation and falls back to a generic array element handler.

Detection: Hot loops over arrays suddenly become slower; node --trace-deopt shows deoptimisations on array-access instructions. Prevention: Normalise arrays to a single type before processing. If a data source is heterogeneous, coerce at ingestion time rather than inside the loop.

5. Ignoring SharedArrayBuffer synchronisation

Using SharedArrayBuffer without Atomics (or with incorrectly ordered reads/writes) leads to data races that are non-deterministic and hard to reproduce.

Detection: Intermittent corruption in shared worker state; results vary between runs with identical inputs. Prevention: Always pair SharedArrayBuffer with Atomics.load, Atomics.store, or Atomics.add. Do not use regular Int32Array indexing on shared buffers for concurrent reads.

6. Running node --inspect in production without a firewall

The Chrome DevTools inspector opens a TCP port (default 9229). If exposed to public networks, it allows remote code execution through the DevTools protocol.

Detection: Security scanners flag open port 9229; netstat -tlnp shows the inspector bound to 0.0.0.0. Prevention: Always bind node --inspect=127.0.0.1:9229 (localhost only), or disable the inspector in production entirely. Use a tunnel or VPN rather than exposing the port.

4. Independent Challenge

A problem to solve without step-by-step guidance; promotes synthesis.

Challenge: Build a runtime profiler that detects hidden-class instability

Problem: Write a Node script (hidden-class-tracker.js) that:

  1. Defines two object constructors (StablePoint and UnstablePoint) — StablePoint always initialises x then y; UnstablePoint sometimes initialises y first, sometimes adds z after construction
  2. Creates 10,000 instances of each and passes them through a hot loop (for) that accesses .x and .y
  3. Uses node --trace-opt output (redirected to a file) to count how many times each constructor's access function is reported as "optimised" vs "NOT OPTIMISED"
  4. Prints a comparative summary: which shape is faster and by approximately how many iterations before deoptimisation occurs

Constraints:

No solution is provided. The goal is to observe the effect of object shape stability on JIT optimisation through direct instrumentation of V8's trace output.

5. Consolidation & Key Invariants

Bullet-list summary of the must-remember rules.

6. Next Steps

Sequenced links to dependent lessons, deeper dives, or production guides.

Change Log

All meaningful modifications should be tracked here with a date and session context.

2026-08-29

Choose Theme

Your selection is saved locally.

Neural Cacophony
Aperture v2
Flux v1
Mosaic Chaos
Nexus v1
Nexus Zest
Prism v2
Synapse