Pre-training
Pre-training is the first, most expensive stage of building a large model: it reads a vast collection of text (or images) and learns general patterns of language and knowledge. The result is a raw model that is later shaped for specific uses.
In one line, for a 12-year-old
Pre-training is the AI's giant reading marathon before it learns any manners.
An example
A lab may spend months and many millions of dollars of computing time pre-training one model.
Why it matters to people
Pre-training uses huge amounts of energy, water for cooling and data gathered from the web, which is why it is at the centre of debates about climate, copyright and privacy.