Context becomes part of the meaning.

BERT learned from text on both sides of a word and could then be adapted to tasks such as question answering and language inference.

Why it mattered: a reusable pre-trained model could support multiple language tasks, reducing the need to design a separate architecture for each.

The perspective: understanding and generating language involve different training choices. BERT is a useful chapter in that wider architectural story.