How does an LLM learn?

Learning by guessing-and-fixing

Imagine you’re learning to shoot a basketball. You shoot, you miss, you adjust. Shoot, miss, adjust. Shoot, swish.

That’s roughly how an LLM learns. Except instead of basketballs, it’s guessing words. And instead of one ball, it does trillions of guesses.

The “fill in the blank” game

Training works like this:

1. Take a real sentence: “The dog chased the ball.”
2. Hide the last word: “The dog chased the ___”
3. Ask the model to guess.
4. Compare its guess to the real answer.
5. Nudge its inner knobs so next time it does a little better.

Now do that trillions of times with text from books, websites, conversations, code. Slowly, the model gets eerily good at the game.

Two stages

Most modern LLMs are trained in two stages:

Stage 1: Pre-training (eats the library)

The model reads an enormous pile of text and just plays the fill-in-the-blank game. After this stage, it can finish sentences, but it’s a bit chaotic. It doesn’t know it should be helpful or polite.

Stage 2: Fine-tuning (good manners class)

Humans show the model what good answers look like, helpful, honest, kind. The model adjusts to talk that way. This is what turns “raw word predictor” into “helpful chatbot.”

What it costs

Training a big LLM uses:

• Lots of data: terabytes of text.
• Lots of computers: thousands of special chips running for weeks.
• Lots of money: sometimes millions of dollars.

That’s why only a handful of companies make the biggest LLMs. But once it’s trained, using it costs much, much less.