LLM Architecture is the internal structure and flow of a Large Language Model that explains how it takes text as input, understands context, and generates the next words as output.
In simple flow:
Input Text → Tokens → Embeddings → Transformer → Attention → Context Understanding → Next-Token Prediction → Output Text
Example:
Input: “The cat is sitting on the…”
The LLM processes the words using tokens, embeddings, transformers, and attention, then predicts:
Output: “mat”
Section 1: how an LLM is created/trained.
Section 2: what happens when a user actually sends a prompt.
Section 1 — Training / Model Creation
These steps happen before you use ChatGPT-like models.
Section 1 in one line
Transformer Architecture
↓
Initialize Parameters
↓
Pre-training learns general patterns
↓
Optional Fine-tuning
↓
Instruction Tuning
↓
Trained LLM
Section 2 — Inference / Actual LLM Processing
Now suppose you type:
“Explain photosynthesis in simple words.”
This is where actual runtime processing starts.
- Input Prompt
Explain photosynthesis in simple words.
This is simply the text sent by the user.
Purpose: Tell the LLM what you want.
- Tokenization
The model cannot directly work with normal sentences. A tokenizer breaks text into smaller pieces.
For example, conceptually:
"Explain photosynthesis in simple words."
↓ Tokenization
["Explain", " photo", "synthesis", " in", " simple", " words", "."]
Real tokenizers may split words differently.
Purpose: Convert human text into units the model can process.
- Tokens
Each token is represented by a numerical ID.
For example:
Explain → 21435
photo → 9821
synthesis → 17420
in → 287
simple → 4382
words → 2456
So:
Text
↓
Tokenization
↓
Tokens
↓
[21435, 9821, 17420, 287, 4382, 2456]
Purpose: Give every text piece a machine-readable identity.
- Context Window
The context window defines how many tokens the LLM can consider together.
Imagine the model can process:
Previous conversation
+
System instructions
+
User prompt
+
Retrieved information
all within its available context.
Conceptually:
┌──────────── Context Window ───────────────┐
│ Previous conversation │
│ User: What is a plant? │
│ Assistant: ... │
│ User: Explain photosynthesis simply │
└──────────────────────────────────────────┘
Purpose: Determines how much information the model can use while answering.
A small correction to your diagram: context window isn't really a transformation step like tokenization or embeddings. It is better thought of as the limit/boundary around the tokens being processed.
- Embeddings
Token IDs such as:
21435
9821
17420
don't contain useful meaning by themselves.
The model converts them into vectors:
"plant"
↓
[0.21, -0.54, 0.88, 0.13, ...]
"sunlight"
↓
[0.17, -0.49, 0.91, 0.08, ...]
These vector representations allow the neural network to work with language mathematically.
Purpose: Convert token IDs into numerical representations that the Transformer can process.
So:
Text
↓
Tokenization
↓
Tokens / Token IDs
↓
Embeddings
Transformer Layers
Now the embeddings enter many Transformer layers.
Conceptually:
Embeddings
↓
Transformer Layer 1
↓
Transformer Layer 2
↓
Transformer Layer 3
↓
...
↓
Transformer Layer N
Every layer improves the model's representation of the text.
For our example:
Explain photosynthesis in simple words
The model gradually understands relationships involving:
photosynthesis
sunlight
plants
water
carbon dioxide
oxygen
Purpose: Perform the main language reasoning/processing.
- Attention
Attention happens inside Transformer layers.
This is extremely important.
For example:
“The plant uses sunlight to make its food.”
While processing the word “its”, attention can connect it strongly with:
plant
rather than:
sunlight
Simplified:
The plant uses sunlight to make its food
↑ ↑
└──────── Attention ──────┘
For your photosynthesis prompt, attention may establish relationships like:
photosynthesis
↓
sunlight
↓
plant
↓
water
↓
carbon dioxide
Purpose: Determine which other tokens are relevant to the token currently being processed.
So the correct relationship is:
Transformer Layer
│
├── Attention
├── Feed-Forward Network
├── Normalization
└── Residual Connections
Attention is not outside the Transformer. It is one of its core mechanisms.
- Parameters / Weights Applied
Remember the billions of weights learned during training?
They are now used throughout the Transformer calculations.
For example, during training the model may have learned strong patterns involving:
photosynthesis ↔ plants
photosynthesis ↔ sunlight
photosynthesis ↔ carbon dioxide
photosynthesis ↔ oxygen
Those learned relationships influence its processing.
Purpose: Apply previously learned knowledge/patterns.
But technically, your diagram simplifies this. Parameters aren't really a separate step after Attention.
A more accurate view is:
Learned Parameters
↓
Embeddings → Transformer Layers → Output
↑
Attention
The weights operate throughout the Transformer.
- Inference
Inference means:
Using the already-trained model to answer a new request.
Everything happening after you submit:
Explain photosynthesis in simple words.
is inference.
Therefore inference is actually the name of the whole runtime process, not just one small processing box.
Think:
Training
= Learning
Inference
= Using what was learned
Example:
TRAINING:
Read millions of science documents
INFERENCE:
User asks:
"What is photosynthesis?"
Model generates an answer
- Next-Token Prediction
Ultimately, an autoregressive LLM repeatedly predicts the next token.
Suppose it has started:
Photosynthesis is the process by which plants...
The model computes probabilities for possible next tokens:
use 45%
make 20%
convert 15%
absorb 10%
other 10%
One token is selected.
Then:
Photosynthesis is the process by which plants use
Now the model predicts again:
sunlight 70%
water 10%
energy 8%
food 5%
Select one:
sunlight
Then repeat.
Photosynthesis
↓
Photosynthesis is
↓
Photosynthesis is the
↓
Photosynthesis is the process
↓
Purpose: Generate the response token by token.
- Temperature
Temperature changes how adventurous/random token selection is.
Imagine:
Next-token probabilities:
sunlight 70%
energy 15%
water 8%
light 5%
other 2%
Low temperature
Model strongly favors high-probability choices.
sunlight
Usually gives:
more predictable
more focused
more consistent output
Higher temperature
Lower-probability tokens have a greater chance.
This can produce:
more variety
more creativity
sometimes more mistakes
Purpose: Control randomness during generation.
- Top-P
Top-P is another token-sampling control.
Suppose:
sunlight 50%
light 20%
energy 15%
water 8%
plants 4%
other 3%
With:
Top-P = 0.90
the sampler considers enough high-probability tokens to reach roughly 90% cumulative probability.
Conceptually:
sunlight 50%
+
light 20%
+
energy 15%
+
water 8%
-------------
93%
Lower-probability alternatives outside that selected probability mass can be excluded.
Purpose: Control how broad the candidate token pool is.
- Generated Response
After repeating next-token generation many times:
Token 1
↓
Token 2
↓
Token 3
↓
Token 4
↓
you finally see something like:
Photosynthesis is the process plants use to make food. They use sunlight, water, and carbon dioxide to produce glucose and release oxygen.
That is the generated response.
- Hallucination
Hallucination is not a processing step.
It is a possible failure mode of the generated response.
For example, imagine the LLM says:
“Plants perform photosynthesis mainly using oxygen.”
That would be incorrect.
The output may sound confident even though the information is wrong.
So your diagram correctly puts Hallucination separately as:
Generated Answer
│
└──── Possible Risk
↓
Hallucination
Purpose in the diagram: Show an important limitation/risk of LLM generation.
The actual flow to remember
For learning LLM processing, I recommend remembering it like this:
USER PROMPT
↓
Tokenization
↓
Tokens
↓
Embeddings
↓
┌─────────────────────────┐
│ TRANSFORMER LAYERS │
│ │
│ Attention │
│ + │
│ Learned Weights │
│ + │
│ Feed-Forward Network │
│ + │
│ Other Components │
└────────────┬────────────┘
↓
Output Scores
/ Logits
↓
Token Probabilities
↓
Temperature + Top-P
↓
Select Next Token
↓
Add Token to the Context
↓
Run Model Again
↓
REPEAT
↓
Generated Answer
│
└── Possible Hallucination
┌──────────────── CONTEXT WINDOW ────────────────┐
│ │
│ User Prompt + Previous Tokens + Generated Text │
│ │
└────────────────────────────────────────────────┘
One correction that makes your diagram much more technically accurate
Instead of remembering:
Inference
↓
Temperature
↓
Top-P
↓
Next-Token Prediction
remember:
Transformer Processing
↓
Next-Token Scores / Logits
↓
Probabilities
↓
Temperature / Top-P
↓
Select Next Token
↓
Append Token
↓
Repeat Transformer Processing
That repeating loop is the most important idea to understand about how an autoregressive LLM generates text.
Top comments (0)