Large Language Models, Explained Briefly
The one picture to keep in your head: text goes in, and probabilities for the next word come out. Everything else in this article is a detail hanging off that sentence.
What's in this article
| # | Part | What it answers |
|---|---|---|
| 1 | The central idea | What an LLM actually does |
| 2 | From prediction to conversation | How a chatbot writes a reply |
| 3 | How the model learns | Parameters, pre-training, back propagation |
| 4 | Training vs. using the model | Why it doesn't learn while you chat |
| 5 | The scale of computation | How much math this really is |
| 6 | From text predictor to assistant | What RLHF adds |
| 7 | Hardware and architecture | Why GPUs and transformers mattered |
| 8 | Inside the transformer | Attention and feed-forward layers |
| 9 | Designed framework, emergent behaviour | Why nobody programmed the skills |
| 10 | At a glance | The whole thing in one table |
Part 1 The central idea
1.1 One job: predict the next word
Imagine you find a short movie script. A person is talking to an AI assistant, but the assistant's reply is missing.
Now imagine a machine that can read any text and predict the next word. You feed it the script. It predicts one word. You append that word to the script and ask again. You repeat until the reply is complete.
That is a modern chatbot. Nothing more exotic is going on.
1.2 Probabilities, not certainty
A large language model (LLM) is a very large mathematical function. It reads text and predicts what comes next but it is rarely sure about a single word. Instead it assigns a probability to every possible next word in its vocabulary.
💡 Keep this in mind The model never outputs a word. It outputs a distribution over all words. Something else picks from it.
Part 2 From prediction to conversation
2.1 The generation loop
To build a chatbot, the software first sets up text that looks like a conversation between a user and an assistant. Your message is inserted into it. Then the model writes the assistant's turn, one word at a time:
The model reads all the text so far and produces probabilities for the next word.
The software picks one word from those probabilities.
That word is appended to the text.
The whole thing runs again until the answer is finished.
2.2 Why the same question gets different answers
If the software always chose the single most likely word, replies would come out stiff and repetitive. So it sometimes picks a less likely word at random. This is called sampling.
Here's the subtle part: the model itself can be deterministic the same input produces the same list of probabilities every time. The randomness is added afterwards, at the picking step. That's why asking the same question twice can give you two different replies.
Part 3 How the model learns
3.1 Parameters: billions of dials
The model learns by reading an enormous amount of text, mostly collected from the internet. For GPT-3 alone, a person reading nonstop day and night would need more than 2,600 years to get through it. Newer models read far more.
Inside the model sit billions of numbers called parameters or weights. Think of them as dials on a giant machine. Turning a dial changes the probabilities the model assigns to the next word.
That's part of what "large" means in large language model: hundreds of billions of dials.
Nobody sets them by hand. They start random, so a fresh model produces pure nonsense. Training slowly turns them until predictions get good.
3.2 Pre-training and back propagation
Pre-training is the first big learning phase, and the task is deliberately simple: predict the next word.
A training example might be a few words or a few thousand. The model sees everything except the final word, guesses it, and the training system compares that guess to the real word.
Back propagation is how the model learns from being wrong:
The model makes a prediction say "dog" instead of "mat".
The size of the mistake is measured as a number called the loss.
Back propagation works backward through the network and works out how much each weight contributed to that error.
Every weight is nudged slightly so the correct word becomes a little more likely and the wrong ones a little less. That nudging is gradient descent.
Repeat across trillions of examples, and those tiny nudges accumulate into a model that predicts text remarkably well.
⚠️ A common misunderstanding Weights do not belong to individual words. There is no single weight for "mat" or "dog". All the weights work together on every prediction, so one update affects how the model handles many different words and sentences. That sharing is exactly why the model learns general patterns instead of memorising a lookup table.
3.3 How can it possibly get through all that text?
It really does train on almost all of it. A few tricks make that feasible:
Batches the model processes huge chunks of text together, averages the errors, and updates weights once per batch rather than once per word.
Every position at once in "The cat sat on the mat" it predicts "cat" from "The", "sat" from "The cat", and so on, all in a single pass. One sentence yields many training examples.
Thousands of GPUs working simultaneously.
One pass is usually enough the model typically reads its giant dataset only about once.
3.4 Generalization
The model doesn't only get better at predicting its training text. It also makes sensible predictions on text it has never seen before. That's called generalization, and it's the whole reason the thing is useful outside the lab.
Part 4 Training vs. using the model
It's tempting to assume the model is learning while you chat with it. It isn't.
Back propagation happens only during training. When you ask a question, the model is in inference mode: the weights are frozen. Your text runs forward through the network and words come out. Nothing gets updated.
It also almost certainly hasn't seen your exact question before. Training shaped its weights to capture general patterns grammar, facts, how explanations are structured, how code works. When you ask something new, it recombines those patterns into an answer it may never have seen word for word.
🎓 Think of a student A student practises thousands of maths problems with a teacher correcting them that's training. In the exam they get a new problem that's inference. They don't learn during the exam, and they never saw that exact problem, but they solve it with the methods they absorbed.
This also explains why LLMs get things wrong. If the learned patterns don't fit your question well, the model can produce an answer that sounds confident and is simply incorrect.
Part 5 The scale of computation
Inside the model, training is just simple arithmetic: adding and multiplying numbers. But it happens an absurd number of times, because the dataset is huge and every one of the billions of weights participates in every single prediction.
So here's a thought experiment. Imagine you're a superhuman calculator doing one billion calculations every second, never stopping. How long to complete all the math used to train the largest language models?
| Guess | Verdict |
|---|---|
| 1 year | not close |
| 10,000 years | still not close |
| Well over 100,000,000 years | that's the real answer |
Real training finishes in weeks or months only because thousands of specialised chips do this math simultaneously. Even then, it costs millions of dollars.
Part 6 From text predictor to assistant
Everything so far is pre-training. But being excellent at continuing random internet text is not the same as being a helpful assistant. Ask a purely pre-trained model "How do I bake a cake?" and it might reply with more questions because that's what a forum page looks like.
So chatbots get a second round of training: RLHF, or reinforcement learning with human feedback.
The model produces answers.
Human reviewers flag answers that are unhelpful or problematic, and indicate which answers are better.
That feedback is used to adjust the weights again.
The model becomes more likely to produce answers people actually prefer.
🎓 The student, again Pre-training is a student reading an entire library and absorbing enormous knowledge. RLHF is a teacher then showing that student how to answer a question clearly, helpfully, and politely.
For assistant-like behaviour, this stage matters roughly as much as pre-training does.
Part 7 Why GPUs and transformers mattered
7.1 GPUs: thousands of workers at once
A normal CPU is like one very capable worker handling tasks one after another. A GPU (graphics processing unit) is like thousands of simpler workers each handling a small task at the same moment. GPUs were built for video games, where millions of pixels must be computed at once. Training an LLM also needs millions of small calculations at once a near-perfect match.
7.2 The architecture has to allow parallel work too
Thousands of workers are useless if the job can only be done one step at a time. Before 2017, most language models read text one word at a time, in order. To process word 5 they had to finish words 1–4 first so most of the GPU sat idle.
In 2017, a team of Google researchers introduced the transformer. It doesn't have to read start to finish. It ingests all the words at the same time, which keeps every GPU worker busy and makes training on massive text practical.
📖 Picture it Older models read a book one word at a time, left to right. A transformer looks at the whole page at once.
7.3 Turning words into numbers
Computers can't do arithmetic on the word "cat". And training needs smooth numbers that can be nudged slightly up or down. So the first step inside a transformer is converting each word into a long list of numbers called a vector.
cat → [ 0.2, −1.3, 0.8, … ]
dog → [ 0.3, −1.1, 0.7, … ]
car → [ −0.9, 0.5, 2.1, … ]
(These numbers are invented for illustration real vectors have hundreds or thousands of dimensions.)
Those numbers are learned during training, and a vector can carry aspects of a word's meaning. Words with similar meanings end up with similar vectors: cat and dog land close together, while car sits far away.
Part 8 Inside the transformer
8.1 Attention: words talk to each other
Initially, a word's vector is identical in every sentence. But plenty of words change meaning with context. The signature operation inside a transformer attention lets the vectors for different words communicate. Each vector looks at the words around it and updates itself. And because the transformer sees all words at once, this happens in parallel.
Take the word bank:
"I deposited money in the bank." → a financial institution.
"We sat on the bank of the river." → the edge of a river.
In the second sentence, attention notices "river" nearby and adjusts the vector for "bank" so it leans toward the riverbank meaning. The financial sense fades out.
🗣️ Picture a group discussion Everyone listens to everyone else simultaneously, and each person updates their understanding based on what the others said.
8.2 Feed-forward networks: stored patterns
A transformer also contains feed-forward neural networks. Where attention mixes information between words, a feed-forward network operates on each word's vector independently, applying language patterns the model absorbed during training. They give the model room to store what it knows.
| Component | The question it asks |
|---|---|
| Attention | What do the other words tell me? |
| Feed-forward | What do I already know about this? |
8.3 The journey through a transformer
This doesn't happen once. The vectors pass through many layers, each adding more useful context like revising a draft over and over.
Encode turn each input word into a long vector of numbers.
Attend let vectors share context with each other, in parallel.
Transform apply feed-forward networks that layer in learned patterns.
Repeat pass through many layers, making every vector richer.
Predict use the final vector to produce a probability for every possible next word.
At the end, only the last vector in the sequence is used for the prediction. By then it has absorbed information from the entire input plus everything the model learned in training. A final function converts it into probabilities: "mat" 40%, "floor" 25%, and so on.
Then the loop closes: the transformer turns context into next-word probabilities, the chatbot picks a word, appends it, and runs the whole thing again.
Part 9 Designed framework, emergent behaviour
Researchers design the structure attention, feed-forward networks, the training procedure. But they don't write rules like "here's how to translate French" or "answer cooking questions this way".
Skills such as grammar, translation, summarising, and coding appear on their own once training tunes hundreds of billions of parameters across huge amounts of data. That's an emergent phenomenon.
🌱 Think of a farmer A farmer supplies soil, water, and sunlight, but doesn't design each leaf and branch. The tree grows its own shape. Researchers set up the conditions; the skills grow out of training.
❗ An LLM is not a database of answers Nobody wrote a list of questions and answers into it. It has weights shaped by training, and it constructs each answer fresh, one word at a time.
Why it's hard to explain
Researchers know precisely which mathematical operations run inside. But with billions of weights acting together, it's extraordinarily hard to say why the model chose one particular word over another. It's a little like the brain: we understand how neurons signal, yet we can't point to why you had one specific thought.
Simple goal, powerful result
The objective sounds almost too plain: predict the next word. But combine it with enormous models, enormous datasets, GPUs, transformers, and human-feedback training and that plain objective produces fluent, useful, occasionally surprising behaviour.
A simple prediction goal can create powerful behaviour once the model, the data, and the computation all become very large.
Part 10 The whole idea at a glance
| Idea | What it means |
|---|---|
| Prediction | An LLM assigns probabilities to possible next words. |
| Generation | A chatbot picks a word, appends it, and repeats. |
| Sampling | Picking with some randomness, so the same prompt can give different answers. |
| Parameters | Hundreds of billions of learned numbers that control the model. |
| Pre-training | Learning to predict the next word across trillions of examples. |
| Back propagation | Working out how to nudge every weight to reduce each error. |
| Generalization | Handling text the model has never seen before. |
| Inference | Using the trained model; weights frozen, nothing learned. |
| RLHF | Human feedback tunes the model toward answers people prefer. |
| GPUs | Chips that perform many calculations simultaneously. |
| Transformer | An architecture that reads all words in parallel. |
| Vectors | Lists of numbers that represent words and carry meaning. |
| Attention | Lets context reshape each word's vector. |
| Feed-forward | Stores and applies learned language patterns. |
| Emergence | Skills grow from training rather than being programmed which is why they're hard to explain. |
Based on 3Blue1Brown, Large Language Models explained briefly (YouTube, 7:57), with added explanations. Figures are presented as stated in the video; the vector values and probabilities shown are illustrative.