“AI and LLM Foundations for Developers: From Machine Learning to Transformers, Tokens, Embeddings, and RAG

What are we trying to understand?

As a traditional developer, you're accustomed to this:

Input
  ↓
Your Code
  ↓
Business Rules
  ↓
Database/API
  ↓
Predictable Output

For example:

if age >= 18:
    return "Adult"
else:
    return "Minor"

Same input → same logic → same output.

An LLM application behaves differently:

User Input
    ↓
Prompt / Instructions
    ↓
Tokenization
    ↓
LLM
    ↓
Probabilistic Generation
    ↓
Validation
    ↓
Application

Understanding why that middle part behaves differently is the purpose of Week 1.

Machine Learning

Definition

Machine Learning (ML) is a way of creating software where the system learns patterns from data instead of you manually programming every decision rule.

Traditional programming:

Rules + Data
     ↓
 Program
     ↓
 Output

Machine learning:

Historical Data + Expected Outputs
              ↓
           Training
              ↓
            Model
              ↓
        New Data → Prediction

Example

Suppose MotoShare wants to predict whether a booking is potentially fraudulent.

Traditional approach:

if booking_amount > 100000:
    suspicious = True

But fraud can depend on:

booking amount
user history
vehicle
location
booking frequency
payment behavior
account age
device

ML learns patterns among these features.

Purpose

Learn ML because LLMs themselves are machine-learning models.

You don't need to become a classical ML expert first, but you should understand:

data
training
model
prediction/inference
features
evaluation

Deep Learning

Definition

Deep learning is a branch of machine learning that uses multi-layer neural networks to learn complicated patterns from large amounts of data.

Artificial Intelligence
        │
        └── Machine Learning
                │
                └── Deep Learning
                        │
                        └── Generative AI
                                │
                                └── LLMs

Why do we need it?

Traditional ML can work very well for structured problems such as:

Price prediction
Fraud detection
Spam detection
Customer churn

Deep learning is particularly powerful for unstructured/high-dimensional information:

Text
Images
Audio
Video
Language

Backend analogy

Think of ML as an application and a neural network as one possible implementation engine.

You don't need to understand every transistor inside your CPU to write Laravel/Python applications.

Likewise, you don't initially need the mathematics behind every neural-network operation to build LLM systems.

Neural Networks

Definition

A neural network is a collection of connected mathematical units that transforms inputs through learned parameters called weights.

Very simplified:

INPUT
  ↓
Layer
  ↓
Layer
  ↓
Layer
  ↓
OUTPUT

During training:

Input
 ↓
Prediction
 ↓
Compare with expected result
 ↓
Calculate error
 ↓
Adjust weights
 ↓
Repeat millions/billions of times

Backend analogy

Imagine an enormous configurable function:

output = model(input, billions_of_learned_parameters)

Except developers didn't manually write all those parameters.

They were learned during training.

Why learn this?

Because terms such as:

7B model
70B model
parameters
weights
training
fine-tuning

will otherwise be confusing later.

Generative AI

Definition

Generative AI produces new content based on patterns learned during training.

It can generate:

Text
Code
Images
Audio
Video
Structured data

Traditional ML vs Generative AI

Traditional ML might answer:

Input:
"This vehicle is excellent."

Output:
Positive

Generative AI could answer:

"Write a professional description
for this vehicle."

→

"Experience comfortable city travel
with this well-maintained..."

The second system generates content.

Purpose

Agentic AI is primarily built around generative models that can reason over instructions, produce responses and increasingly invoke tools.

Large Language Model (LLM)

Definition

An LLM is a large neural network trained on huge amounts of text to predict and generate sequences of tokens.

One crucial mental model:

An LLM is fundamentally predicting what token should come next.

Suppose the input is:

The capital of India is

Possible next tokens could have probabilities resembling:

Delhi       0.91
Mumbai      0.03
India       0.02
Kolkata     0.01

The generation process continues:

Input
 ↓
Predict next token
 ↓
Append token
 ↓
Predict next token
 ↓
Append token
 ↓

Why learn this?

This explains many things you'll encounter:

temperature
hallucination
tokens
context windows
prompt engineering
sampling

1. Transformer

This is one of the most important Week 1 concepts.

Definition

A Transformer is a neural-network architecture designed to process sequences while determining relationships between different parts of the input.

Modern LLMs are largely built using transformer architectures.

Why was it important?

Consider:

Ashwani deposited ₹10,000 into the bank.
He withdrew ₹2,000 the next day.
How much money remains?

The model needs relationships among:

Ashwani
He
₹10,000
₹2,000
bank
withdraw

Transformers help model those relationships.

Simplified architecture

chatgpt