A real GPT — hand-written from scratch, no libraries (attention, backprop, the optimizer, every line).
It starts as pure noise with no pretrained weights, no provider, no cloud, and teaches itself the voice of the
American founding live, while you watch — learning from nothing but the founding documents.
Independence as self-origination: an intelligence that owes its mind to no one.
SPEED (steps/tick) 2
TEMPERATURE 0.8
Press Declare Independence — or hit Verify the Math to prove the backprop is real before you trust a word of this.
PARAMETERS
—
STEP
0
TRAIN LOSS
—
HELD-OUT LOSS
—
TOKENS/SEC
—
LEARNING CURVE cross-entropy, lower = better
▦ ATTENTION — what one head learns to look at (last sequence)
IT WRITES, LIVE untrained
…
FROZEN TWIN — an identical net that never learns (pure noise, for contrast)
…
▸ TRAIN IT ON YOUR OWN TEXT
Paste any text — a poem, your essays, song lyrics, code. It throws away 1776, rebuilds its alphabet, and learns your voice from scratch.
How it's built: a char-level transformer (token + positional embeddings → 2 blocks of causal
multi-head self-attention + MLP, LayerNorm, residuals → LM head), with the forward pass, full backpropagation,
and an AdamW optimizer all written by hand in one file — no TensorFlow, no autograd library, no API. The
backprop is verified by a numerical gradient check (open the console: __gpt.gradCheck()). It trains
on the Declaration of Independence, the Constitution's preamble, and the Federalist Papers — so it learns a
style, not facts; a tiny model writing in 1776's cadence. The frozen twin proves the point: same architecture,
never trained, never coherent.
WEEK 2 · INDEPENDENCE ENGINES · independence is not a symbol, it's a mechanic
Most "AI builds" call someone else's model. This one is the model — a GPT written from scratch, every line of the attention and the training math by hand, running in this tab with no server and no pretrained weights.
It begins as noise. Press the button and it starts teaching itself to write — from nothing but the founding documents. Watch the loss fall and the gibberish resolve into the cadence of 1776, live.
Independence as self-origination: an intelligence with no ancestors, beholden to no provider. It declares independence from every model that came before it.