Recently came across a really well-done interactive MicroGPT walkthrough. It builds on Andrej Karpathy’s ~200-line Python GPT implementation and visually explains each step of how a language model works.
You can see tokenization, next-token prediction, softmax, loss, attention, and backpropagation play out step by step. If you’ve ever wanted to understand how GPT actually works under the hood, this is a great place to start.
growingswe.com/blog/microgpt
