Why do transformers use layer normalization instead of batch normalization?

Batch norm normalizes each feature across the batch, so its statistics depend on batch composition and on sequence padding, and it behaves differently at inference where it uses running averages. Layer norm normalizes across the features of a single token, independent of batch and sequence length, which makes it stable for variable length sequences, small batches, and autoregressive decoding.

They normalize over different axes. Batch norm takes a single feature and normalizes it across all the examples in the batch. Layer norm takes a single example, or a single token, and normalizes across its features. That difference decides everything else. With text you have variable length sequences and padding, so batch statistics get polluted by pad tokens and shift depending on what happened to be batched together. Batch norm also carries the train and inference mismatch, where you train on batch statistics and serve on running averages, and during autoregressive decoding you are often generating one token at a time with a batch of one, where batch statistics are meaningless. Layer norm has no batch dependence at all, so training and inference are identical and batch size does not matter. On placement, modern transformers put the norm before the sublayer rather than after, because pre norm keeps the residual path clean and lets you train deep stacks without a delicate warmup schedule. Most current models have also moved to RMSNorm, which drops the mean centering and just rescales, because it is cheaper and works about as well.