Inside
In Topic 4 I took a neural network apart and found arithmetic and a feedback loop. Which raises an obvious question: how does that same machinery end up writing code, holding a conversation, or painting a picture of a cat in a hat that has never existed?
That's the jump I want to make here. Everything in this post is built on the loop from last time, but pointed at a different job. Instead of sorting things into buckets, these models produce something new. I'll go through the three families that matter right now: generative models in general, the transformers behind LLMs, and diffusion models for images. No maths beyond what we've already covered, and I'll flag the bits I'm still fuzzy on.
Creating instead of judging
What generative AI is: machine learning aimed at making new content rather than labelling existing content.
The distinction is cleaner than the marketing makes it sound. Every model we've looked at so far takes something in and returns a verdict: is this spam, which digit is this, what's this house worth. A generative model runs the other way. It takes a prompt, or sometimes nothing but random noise, and produces text, an image, audio, code, video.
Broadly, the model trains on an enormous pile of examples and learns the patterns and structure inside them, not as rules it can recite but as weights. Then, given a starting point, it produces something new that fits those patterns. Somebody who has looked at ten thousand paintings can produce an original one in that style without copying any single painting, and it's roughly that, done with numbers.
Which is the part I keep circling back to. A model that can generate convincing content is useful and a genuine problem at the same time, and which one it is depends entirely on who's holding it. That's the security angle, and it's where I want to take this track next.
The four families, and what each is for
There are four main approaches. You don't need the maths to know which is which, and knowing the shape of each helps a lot when you're reading about a new model.
Two networks compete: one forges, one detects. The forger improves until the detector can't tell. Made early image generation work
Squeeze data down to a compact code, then rebuild it. Good for learning compressed representations
Generate one piece at a time, each informed by everything before it. This is what LLMs are
Start from noise and clean it up step by step. Behind most modern AI image tools
GANs are worth a moment because the idea is lovely. You train two networks against each other: a generator making fakes and a discriminator trying to spot them. Each one's improvement pressures the other, like a forger and an inspector learning in the same room. They produced the first genuinely startling AI images. They're also notoriously temperamental to train, and one classic failure is mode collapse, where the generator discovers one output that reliably fools the discriminator and just produces that forever. Technically winning, practically useless.
LLMs: predicting the next token, at scale
What an LLM is: a large language model is an autoregressive model trained on a vast amount of text, whose whole job is to predict what comes next.
That's worth sitting with, because it sounds far too simple to explain what these things do. The training task really is that plain: hide the next bit of text, make the model guess, measure how wrong it was, backpropagate, repeat across an enormous corpus. Everything else, the apparent reasoning, the ability to follow instructions, the tone, emerges from doing that at a scale that's hard to picture, with billions of parameters.
I'll be straight that I find the gap between "predict the next token" and what these models actually do genuinely unresolved. The mechanism is simple and well documented. Why competence emerges from it at scale is still argued over by people far deeper in this than me, and I'd rather say that than pretend it's settled.
What happens to your sentence
Here's the path text takes through a model, which demystifies a lot of the vocabulary.
Split text into small pieces
Turn each piece into numbers
Work out which words matter to which
Score every possible next token
Tokenisation is chopping text into pieces called tokens. Not always whole words: common words are usually one token, rarer ones get split into fragments. Models handle numbers, not letters, so this is the step that makes text processable at all. It also explains some of the odd failure modes, like models struggling to count letters in a word, since they never really see the letters.
Embeddings turn each token into a long list of numbers that carries its meaning. The useful property is that similar meanings end up near each other in that number space, so "king" and "queen" sit close together while "king" and "bicycle" don't. Meaning becomes geometry, which is what lets arithmetic operate on language at all.
Self-attention, the actual breakthrough
Attention is the mechanism that lets the model decide, for every word, which other words in the text actually matter for understanding it.
Take "the trophy wouldn't fit in the suitcase because it was too big". You know "it" is the trophy. Nothing in the word "it" tells you that, you got it from the relationship between the words. Self-attention is how the model does the same thing: for each word it scores the relevance of every other word, then builds that word's representation mainly from the ones that scored highly.
Why the transformer won: older architectures read text strictly in order, one word after another, which made them slow to train and forgetful over long passages. A transformer processes the whole sequence at once, and attention lets any word connect directly to any other word regardless of the distance between them. That parallelism is what made training on internet-scale text practical. The architecture didn't just improve results, it made the scale possible in the first place.
You'll also meet encoders and decoders. The short version: an encoder's job is understanding input, a decoder's job is producing output. Translation systems use both, chat-style models are mostly decoders.
Strip it back and the whole loop is five steps: tokenise, embed, attend, predict, repeat. Everything else is scale and training data.
Diffusion: sculpting an image out of noise
Text models build a sentence piece by piece. Image models mostly work in a way I found much less obvious, and much more fun once it landed.
What a diffusion model is: a model trained to remove noise from an image, used in reverse to create one from nothing but noise.
How the training works: take a real photo. Add a little random noise. Add a bit more. Keep going until the image is pure static with nothing recognisable left. Now train the network on the reverse: given a slightly noisy image, predict what the cleaner version looked like. Do that across millions of images and countless noise levels, and the model learns, in fine detail, what real images look like at every stage of degradation.
How generation works: here's the trick. If a model can reliably take one step from noisier to cleaner, then you can start with an image that is entirely noise, which contains no information at all, and ask it to clean that up. It has no photo to recover, so it invents a plausible one. Ask again on the result. And again. Each pass sharpens the picture, and after enough steps a coherent image emerges that was never there to begin with.
"a cat in a hat" becomes an embedding
Pure random noise, no image in it
Remove a little, steered by the text
An image matching the prompt appears
Where the prompt comes in: on its own the process would produce a plausible image of anything at all. To control it, your text goes through a text encoder (the same embedding idea from the LLM section) and that embedding is fed into every denoising step. So at each pass the model isn't just answering "what would make this less noisy", it's answering "what would make this less noisy and more like a cat in a hat". Thousands of tiny steered decisions, compounding into a picture.
The image-from-marble comparison is overused but it's accurate here, with one difference worth noting: the model isn't uncovering something already present. It genuinely invents the detail on the way down, which is why the same prompt with a different random starting noise gives you a completely different picture.
The vocabulary you'll keep meeting
A handful of terms come up constantly in this area. Here they are in plain words, so the papers and release notes stop being a wall.
| Term | What it actually means |
|---|---|
| Latent space | The compressed internal space where the model keeps learned features. Similar things sit near each other |
| Sampling | Drawing a new output from what the model learned. Different sample, different result |
| Mode collapse | A GAN failure: the model finds one output that works and produces variations of only that |
| Overfitting | The model memorised its training data rather than the general pattern, so new inputs go badly |
| FID score | Compares generated images against real ones. Lower is better. The usual benchmark for image models |
| BLEU score | Scores generated text against a reference translation. Rough, widely used, widely criticised |
On the scoring ones I'd add a caution I've come to hold fairly firmly: these metrics are proxies, not verdicts. FID and BLEU give you a number that's easy to compare and easy to over-trust, and plenty of models score well while producing output a person would call poor. Human evaluation is slower and still matters, which is a pattern that shows up everywhere in security testing too.
What keeps turning up
Two posts in and the thing I keep noticing is how much reuse there is. Embeddings turn up in language models and in image generation. The training loop from Topic 4 is underneath all of it. Attention was built for text and now runs inside image models. It's a smaller bag of tricks than the pace of announcements suggests, recombined and scaled.
What I'm still genuinely uncertain about is the emergence question I flagged earlier. "Predict the next token" explaining code, translation and apparent reasoning still feels like an unsatisfying answer even though I understand the mechanism, and I don't think I'm alone in that. If you've found an explanation that made it click properly, please send it my way.
Where I'm taking the track next is the part I actually care most about: the security side. Prompt injection, model extraction, data poisoning, what it means to pentest a system with one of these in the middle of it. That's the overlap between this track and my day-to-day, and it's where I want to spend real time.
Further reading
The original transformer paper, Attention Is All You Need (Vaswani et al., 2017), is the primary source for the architecture and is more readable than you'd expect. Jay Alammar's The Illustrated Transformer is the best visual walkthrough of attention I've come across, and Lilian Weng's write-ups on diffusion models are the clearest technical treatment if you want the maths behind the denoising.
FAQ
What is generative AI?
Generative AI is machine learning that creates new content rather than sorting or scoring existing content. A classifier looks at a picture and says cat. A generative model produces a picture of a cat that never existed. Same underlying networks, aimed at making instead of judging.
What is a transformer in AI?
A transformer is the network architecture behind modern language models. Its key trick is reading a whole sequence at once instead of word by word, which makes training far faster on modern hardware and lets the model relate any word to any other word regardless of distance.
What is self-attention?
Self-attention is how a model works out which other words matter when interpreting each word. For every word it scores the relevance of all the others and builds its understanding from the ones that count. That is how it resolves what it refers to in a long sentence.
How do diffusion models create images?
They are trained by adding noise to real images until nothing is left, learning to reverse each step. To generate, they start from pure random noise and remove a little at a time, guided by your text prompt, until a clear image that matches the description emerges.
What is tokenisation in an LLM?
Tokenisation is chopping text into small pieces called tokens, which may be whole words, word fragments or single characters. Models work with numbers, not letters, so each token is mapped to an ID and then to an embedding. It is the first step of any text going into an LLM.
Related reading
- Neural Networks & Backpropagation (Topic 4, the machinery all of this runs on)
- AI, ML & Deep Learning explained (Topic 1, where the terms fit together)
- Supervised Learning Algorithms (Topic 2, learning from labelled data)
- Browse the whole AI / LLMs track