Transformer

Plain English. The transformer is the neural-network architecture nearly every large language model is built on. Introduced in 2017, it reads a whole sequence at once rather than word by word, which is what let training scale onto thousands of GPUs in parallel. It is the substrate under GPT, Gemini, Claude and the open-weight models alike.

Why it moves money. The transformer's scalability is why the last decade of AI capital works at all: pour more data, parameters and compute into the same architecture and the model gets predictably better (see scaling laws). That predictability is what justifies multi-billion-dollar training runs and the entire GPU buildout. It also means the architecture itself is not the moat — everyone has it — so the value migrates to data, compute and distribution.

What to watch. Whether anything displaces it. Serious challengers so far (diffusion models for text, state-space models) have ended up either transformer-flavoured or niche. Its core cost driver is attention, which sets what context windows and the KV cache cost; a genuine successor would reset the hardware and training economics overnight.

From the signals. Etched bet its whole company on the transformer being permanent enough to etch into silicon. Qwen's latest open-weight frontier model ships yet another transformer variant.