Writing
Essays, notes, and rougher thinking.
This is the less formal side of the archive. Some entries are arguments. Some are field notes. All of them should feel like they came from the same mind.
01
2026-05-08
GPT-2 --> Llama 3, One Improvement At A Time
A thinking-out-loud walkthrough of moving a GPT-2-style block toward Llama 3 with RoPE, RMSNorm, and SwiGLU.
#llms#transformers#llama