I made this interactive reference for transformer models, showing everything down to elementary math. I intentionally avoided matrix multiplication and used explicit sums and indices instead. Covers models from GPT-2 to Qwen 3.6, with MLA, MoE, RoPE, MTP, etc togglable. It's best viewed on desktop or tablet.
I made this interactive reference for transformer models, showing everything down to elementary math. I intentionally avoided matrix multiplication and used explicit sums and indices instead. Covers models from GPT-2 to Qwen 3.6, with MLA, MoE, RoPE, MTP, etc togglable. It's best viewed on desktop or tablet.