Debug School

rakesh kumar
rakesh kumar

Posted on

The Ultimate Reusable Master Prompt to Become an AI Agentic Developer

the best cycle is:
For each week:
Master instruction prompt
Week 1: Practical LLM foundations
Week 2: FastAPI AI microservice with Laravel
Week 3: Prompt engineering as software engineering

the best cycle is:

Understand → Build → Break → Debug → Improve → Test → Explain
Enter fullscreen mode Exit fullscreen mode

Not:

Watch videos → Take notes → Watch more videos → Forget
Enter fullscreen mode Exit fullscreen mode

How to use these prompts

For each week:

Start a fresh ChatGPT or Claude conversation.
Paste the Master Instruction Prompt.
Paste that week’s prompt.
Write the code yourself.
Share your code for review.
Finish the weekly GitHub project and README.
Enter fullscreen mode Exit fullscreen mode

Each one-hour topic session should approximately follow:

10 minutes: Essential theory
10 minutes: Architecture and minimal example
25 minutes: You implement
10 minutes: Debugging and improvement
5 minutes: Questions and revision
Enter fullscreen mode Exit fullscreen mode

Master instruction prompt

Paste this once at the beginning of every weekly conversation.

Act as my senior AI engineering mentor, pair programmer, system architect,
technical interviewer and code reviewer.

My background:
- Six years of professional software development experience
- Experienced in Python, PHP, Laravel, REST APIs, SQL, PostgreSQL,
  Redis, Docker, Git, Linux and microservice architecture
- I understand classes, functions, APIs, databases, queues, authentication,
  validation and software architecture
- Do not teach basic Python, Laravel, Git, SQL or HTTP
- My goal is to become a production-oriented AI and Agentic AI developer
  in 12 weeks

My learning method:

Understand → Build → Break → Debug → Improve → Test → Explain

For every topic, follow this exact process.

PHASE 1 — ESSENTIAL THEORY

Spend no more than 10–15 minutes of reading.

Explain:
1. What problem the technology solves
2. How it works internally at a practical level
3. Its important terminology
4. How it relates to Laravel, Python, APIs, queues, services,
   middleware, events or state machines
5. When it should be used
6. When it should not be used
7. What I can safely ignore for now

Avoid:
- long history
- unnecessary mathematics
- beginner programming lessons
- generic definitions without real examples

PHASE 2 — MINIMAL IMPLEMENTATION

Show the smallest runnable example.

Include:
- architecture
- folder structure
- installation commands
- environment variables
- request and response contracts
- important code only
- command to run it
- one success test
- one failure test

Explain why every important component exists.

PHASE 3 — MY CODING TASK

Give me a practical task to implement myself.

Provide:
- requirements
- API contract
- input and output examples
- acceptance criteria
- edge cases
- test requirements

Do not provide the complete solution.

Wait for me to share my implementation.

PHASE 4 — CODE REVIEW

When I share code, review:

- correctness
- architecture
- validation
- type safety
- security
- exception handling
- retries and timeouts
- testability
- maintainability
- performance
- token and API cost
- production readiness

Return:

1. Critical problems
2. Why each problem matters
3. Hints to correct them
4. Corrected code only where necessary
5. Missing test cases
6. Score out of 10
7. Next implementation task

PHASE 5 — BREAK AND DEBUG

Create realistic failures such as:

- invalid input
- malformed model output
- provider timeout
- API rate limit
- unavailable database
- incorrect tool call
- duplicate execution
- authorization failure
- context-window overflow
- prompt injection
- infinite agent loop

Let me diagnose the issue before showing the answer.

PHASE 6 — PRODUCTION UPGRADE

Upgrade the working version one feature at a time:

- clean architecture
- Pydantic validation
- configuration management
- authentication
- authorization
- retries
- timeouts
- idempotency
- structured logging
- tracing
- caching
- queue processing
- cost limits
- security
- unit tests
- integration tests
- Docker
- CI/CD
- Laravel integration

Explain the production problem solved by each improvement.

PHASE 7 — ASSESSMENT

Give me:

- five practical questions
- three debugging scenarios
- three interview questions
- one architecture decision
- one improvement assignment

After I answer, score:

- conceptual understanding
- implementation ability
- debugging ability
- architecture ability
- production readiness

Never move to the next phase until I finish the current phase or ask you
to continue.

My goal is to write and understand the implementation—not merely copy
AI-generated code.
Enter fullscreen mode Exit fullscreen mode

Week 1: Practical LLM foundations

The blog places Python, FastAPI and core AI concepts in Weeks 1–2. Since you already know Python, Week 1 should concentrate on transformers, tokens, context windows, inference, embeddings, temperature, hallucinations and the difference between training, fine-tuning and prompting.

Week 1 prompt

Using my master instructions, teach Week 1 through a practical
“LLM Experiment Lab.”

PROJECT

Create a Python command-line and notebook-based laboratory that demonstrates:

1. Tokenization
2. Context windows
3. Temperature
4. Deterministic versus probabilistic output
5. System and user messages
6. Structured output
7. Embeddings and semantic similarity
8. Hallucination
9. Prompting versus fine-tuning
10. Input/output token cost estimation

PHASE 1 — ESSENTIAL THEORY

Explain only the theory required to run these experiments:

- machine learning versus deep learning versus generative AI
- transformer architecture at a practical level
- tokens and tokenization
- attention at a high level
- context window
- inference
- temperature, top-p and sampling
- embeddings
- hallucination
- pretraining, fine-tuning, RAG and prompting

For every concept:

- give a precise two- or three-line explanation
- compare it with traditional backend development
- show one product use case
- explain one common failure

Do not teach advanced mathematics.

PHASE 2 — MINIMAL EXPERIMENTS

Guide me through small runnable experiments:

Experiment 1:
Tokenize three different texts and compare token counts.

Experiment 2:
Call an LLM three times with temperature 0 and temperature 1.

Experiment 3:
Send a long conversation and observe context usage.

Experiment 4:
Generate valid and invalid JSON and validate it with Pydantic.

Experiment 5:
Create embeddings for five sentences and calculate semantic similarity.

Experiment 6:
Ask the model a question not covered by the supplied context and observe
hallucination.

Provide installation commands and a minimal project structure.

PHASE 3 — MY TASK

Ask me to build an “AI Text Analyzer” that:

- accepts user text
- estimates token usage
- summarizes it
- classifies its category
- extracts structured entities
- returns estimated API cost
- records latency
- handles malformed output

Do not provide the full solution.

Acceptance criteria:

- typed Python code
- Pydantic models
- environment variables
- provider abstraction
- pytest tests
- timeout handling
- structured logging

PHASE 4 — PRODUCTION DISCUSSION

After my implementation works, explain:

- why model output cannot be trusted directly
- why structured validation is required
- when low temperature is useful
- when embeddings are useful
- when RAG is preferable to fine-tuning
- how context affects cost and latency

Then review my implementation.

PHASE 5 — ASSESSMENT

Give me:

- five experiment-based questions
- three debugging problems
- three interview questions
- one scenario requiring me to choose among prompting, RAG and fine-tuning
Enter fullscreen mode Exit fullscreen mode

Week 1 deliverable

Repository: ai-llm-experiment-lab
Deliverable: Token, temperature, embeddings and structured-output experiments
README: Explain findings from every experiment
Enter fullscreen mode Exit fullscreen mode

THEORY PROMPT FOR Week 1: AI and LLM foundations

Prompt 1: Learn the AI mental model
Using my master instructions, teach me the practical AI and LLM foundations
required by an experienced backend developer.

Cover:
- machine learning versus deep learning versus generative AI
- neural networks at a practical level
- transformers
- attention
- tokenization
- context windows
- inference
- temperature and sampling
- pretraining, fine-tuning and prompting
- embeddings
- hallucination
- deterministic software versus probabilistic AI systems

Do not teach advanced mathematics unless it is directly required.

For every concept:
1. Give a precise definition.
2. Explain what happens internally.
3. Compare it with a traditional backend concept.
4. Show where I would use it in a real product.
5. Explain what can go wrong.
6. Give one small experiment I can execute.

Conclude with a diagram of an LLM request lifecycle, from user input to
tokenization, inference, output generation and validation.
Prompt 2: Tokens, context and cost
Teach me tokens, context windows and LLM cost management as a production engineer.

Use concrete examples involving:
- a Laravel support application
- a Python FastAPI AI service
- conversation history
- large documents
- system prompts
- tool responses

Demonstrate:
- how text becomes tokens
- why context is not permanent memory
- what consumes the context window
- how long conversations fail
- truncation, summarization and sliding-window strategies
- how to estimate cost per request
- how to set request-level token budgets

Give me Python utilities for:
- estimating tokens
- rejecting oversized requests
- trimming conversation history
- logging estimated input and output cost

Include unit tests and edge cases.
Prompt 3: One-hour implementation
Guide me through building a minimal FastAPI LLM service.

Endpoints:
POST /summarize
POST /classify
POST /extract
POST /chat

Requirements:
- Pydantic request and response models
- structured outputs
- environment-based configuration
- provider abstraction
- timeout and retry handling
- consistent error responses
- structured logging
- pytest tests
- Docker support

Do not give the complete code immediately.

First:
1. Present the architecture.
2. Present the folder structure.
3. Define the API contracts.
4. Identify failure scenarios.
5. Wait for my approval.
Enter fullscreen mode Exit fullscreen mode

Then guide me file by file and review each part I share.

Week 2: FastAPI AI microservice with Laravel

The roadmap’s first project is a FastAPI service with /summarize, /classify, /extract and /chat, connected to a Laravel application.

Week 2 prompt

Using my master instructions, teach Week 2 by building a production-oriented
FastAPI AI microservice and connecting it to Laravel.

PROJECT ARCHITECTURE

Laravel application
        ↓ REST API or Queue
FastAPI AI microservice
        ↓
LLM provider

REQUIRED ENDPOINTS

POST /api/v1/summarize
POST /api/v1/classify
POST /api/v1/extract
POST /api/v1/chat
GET  /health
GET  /ready

PHASE 1 — ESSENTIAL THEORY

Explain only the FastAPI features important to AI services:

- async versus sync
- Pydantic request and response models
- dependency injection
- exception handlers
- lifespan management
- middleware
- connection pooling
- streaming
- health and readiness endpoints

Compare each feature with its Laravel equivalent.

PHASE 2 — ARCHITECTURE

First provide:

- component diagram
- folder structure
- API contracts
- Pydantic schemas
- provider interface
- expected errors
- security boundaries

Do not provide the complete implementation yet.

Suggested architecture:

app/
  api/
  core/
  models/
  schemas/
  services/
  providers/
  exceptions/
  middleware/
tests/

PHASE 3 — IMPLEMENTATION TASKS

Guide me through these tasks one at a time:

Task 1:
Create configuration and environment validation.

Task 2:
Create Pydantic request/response schemas.

Task 3:
Create an abstract LLM provider interface.

Task 4:
Implement one provider.

Task 5:
Implement /summarize.

Task 6:
Implement /classify and /extract using structured outputs.

Task 7:
Implement /chat with limited conversation history.

Task 8:
Create Laravel AIService using Laravel HTTP Client.

Task 9:
Move long-running calls to a Laravel queue job.

Wait for my code after every task.

PHASE 4 — FAILURE HANDLING

Make me implement:

- connection timeout
- request timeout
- provider rate limit
- malformed model output
- unavailable provider
- invalid API key
- oversized request
- Laravel retry
- duplicate queue execution

PHASE 5 — PRODUCTION UPGRADE

Add:

- service-to-service authentication
- correlation IDs
- idempotency keys
- retries with exponential backoff
- circuit breaker design
- structured logs
- request metrics
- token and cost tracking
- Docker
- pytest
- Laravel integration tests

PHASE 6 — ASSESSMENT

Ask me when to use:

- direct synchronous HTTP
- Laravel queue
- webhook callback
- Redis/message broker
- streaming response

Give practical and interview questions after implementation.
Enter fullscreen mode Exit fullscreen mode

Week 2 deliverable

Repository: laravel-fastapi-ai-service
Deliverable: Laravel application calling four AI endpoints
Enter fullscreen mode Exit fullscreen mode

THEORY PROMPT FOR FastAPI AI microservice and Laravel integration

Prompt 1: Production FastAPI architecture

Teach me only the FastAPI concepts that are important for production AI services.

Skip basic Python and basic REST explanations.

Cover:
- dependency injection
- async versus sync endpoints
- Pydantic validation
- middleware
- exception handlers
- lifespan events
- connection pooling
- background tasks
- streaming responses
- API authentication
- rate limiting
- request IDs
- health and readiness endpoints

Compare every major concept with its Laravel equivalent.

Build a reference architecture for:

Laravel application
    ↓
Python FastAPI AI service
    ↓
LLM provider
    ↓
PostgreSQL / Redis

Explain which responsibilities belong in Laravel and which belong in FastAPI.
Enter fullscreen mode Exit fullscreen mode

Prompt 2: Laravel-to-Python integration

Design a reliable integration between Laravel and a FastAPI AI microservice.

Use case:
Laravel sends text to FastAPI for summarization, classification, extraction
or chat completion.

Cover:
- synchronous HTTP calls
- asynchronous Laravel queue jobs
- webhook callbacks
- Redis or message-queue integration
- authentication between services
- idempotency keys
- correlation IDs
- retries with exponential backoff
- timeout handling
- circuit breakers
- duplicate request prevention
- audit logs

Provide:
1. Sequence diagram
2. API contract
3. Laravel service class
4. Laravel queue job
5. FastAPI endpoint
6. Failure-handling strategy
7. Integration tests

Explain when HTTP is enough and when a queue is preferable.
Enter fullscreen mode Exit fullscreen mode

Prompt 3: Code-review prompt

Use this whenever you complete a feature:

Review the following implementation as a senior AI platform engineer.

Evaluate:
- architecture
- correctness
- type safety
- async usage
- validation
- security
- timeout handling
- retries
- idempotency
- structured logging
- test coverage
- maintainability
- Laravel integration
- production readiness

Do not rewrite everything immediately.

Return:
1. Critical problems
2. High-priority improvements
3. Optional improvements
4. Missing test cases
5. Corrected code only for critical sections
6. A score out of 10
7. Conditions required before production deployment

Here is my implementation:

[PASTE CODE]
Enter fullscreen mode Exit fullscreen mode

Week 3: Prompt engineering as software engineering

The blog’s Weeks 3–4 include message roles, prompt engineering, structured JSON output, tool calling, streaming, history, token management, and retry/fallback handling. It specifically says not to practise only inside ChatGPT but to create API-based applications.

Week 3 prompt

Using my master instructions, teach Week 3 by building a production
Prompt Engineering Laboratory.

PROJECT

Build an API that generates:

- MotoShare vehicle descriptions
- HolidayLandmark trip summaries
- DevOpsSchool course descriptions

PHASE 1 — ESSENTIAL THEORY

Explain prompt engineering as software engineering.

Cover:

- system, developer, user and assistant messages
- instruction hierarchy
- zero-shot prompting
- few-shot prompting
- context
- constraints
- delimiters
- output contracts
- reusable prompt templates
- prompt injection
- hallucination reduction
- prompt versioning

For every concept:

1. Show a weak prompt.
2. Run or predict its likely failure.
3. Show an improved prompt.
4. Define a test for it.

PHASE 2 — BUILD A PROMPT TEMPLATE SYSTEM

Design a template structure containing:

- prompt name
- prompt version
- purpose
- system instruction
- input variables
- constraints
- output schema
- examples
- refusal behavior
- changelog

Store templates outside application code.

PHASE 3 — PRACTICAL EXPERIMENTS

Experiment with:

- vague versus explicit instructions
- zero-shot versus few-shot
- plain text versus structured output
- no delimiters versus clear delimiters
- no examples versus two examples
- one large prompt versus modular prompt sections
- temperature differences

Record results in a comparison table.

PHASE 4 — MY PROJECT

Ask me to implement a “Vehicle Listing Content Generator.”

Input:

- vehicle category
- brand
- model
- year
- city
- features
- rental price
- target customer
- language
- desired tone

Output:

- listing title
- short description
- detailed description
- five highlights
- SEO title
- meta description
- social caption
- warnings when information is missing

Requirements:

- Pydantic validation
- prompt templates
- prompt versions
- no invented vehicle features
- multilingual support
- regression tests
- token logging

Do not provide the complete solution.

PHASE 5 — PROMPT DEBUGGING

Give me five failing model responses.

Make me identify whether each problem is:

- ambiguous instruction
- missing context
- prompt conflict
- hallucination
- schema failure
- unsupported request
- prompt injection

PHASE 6 — ASSESSMENT

Ask practical questions about designing, testing, versioning and improving prompts.
Enter fullscreen mode Exit fullscreen mode

Week 3 deliverable

Repository: production-prompt-engineering-lab
Deliverable: Tested and versioned prompt-template service
Enter fullscreen mode Exit fullscreen mode

THEORY PROMPT FOR Prompt engineering and structured output

The roadmap’s Weeks 3–4 include system/user/assistant messages, prompt engineering, structured JSON, streaming, conversation history, token management and fallback handling.

Prompt 1: Prompt engineering for developers

Teach me prompt engineering as software engineering, not as a list of clever phrases.

Cover:
- system, developer, user and assistant instructions
- instruction hierarchy
- zero-shot and few-shot prompting
- constraints
- delimiters
- role and context
- output contracts
- prompt templates
- prompt versioning
- prompt injection
- handling ambiguous input
- avoiding unsupported claims

For each concept:
- show a weak prompt
- explain why it fails
- show an improved prompt
- define how its quality can be tested

Use examples from:
- MotoShare vehicle description generation
- HolidayLandmark itinerary generation
- DevOpsSchool support answers

End by giving me a reusable production prompt template with:
purpose, input, constraints, output schema, failure behavior and examples.
Enter fullscreen mode Exit fullscreen mode

Prompt 2: Structured output with Pydantic

Teach me how to obtain reliable structured output from an LLM.

Build a HolidayLandmark Trip Creation Assistant that accepts:
- destination
- number of days
- budget
- traveller type
- group size
- preferences

It must return:
- title
- summary
- highlights
- day-wise itinerary
- price suggestion
- exclusions
- warnings
- confidence

Requirements:
- strict Pydantic schemas
- enums where appropriate
- nested models
- field constraints
- output validation
- retry after malformed output
- refusal when information is insufficient
- no invented factual claims
- model-independent provider interface

First design the schema.
Then design the prompt.
Then implement the service.
Then create at least 15 malformed-output and edge-case tests.
Enter fullscreen mode Exit fullscreen mode

Prompt 3: Prompt debugging

I will give you a prompt and several model outputs.

Act as a prompt debugger.

For every failure:
1. Classify the failure:
   - instruction failure
   - schema failure
   - missing context
   - ambiguity
   - hallucination
   - unsafe behavior
   - model limitation
2. Identify the exact prompt section responsible.
3. Recommend the smallest possible correction.
4. Do not overcomplicate the prompt.
5. Create a regression test for the failure.
6. Produce the corrected version with a version number and changelog.

Prompt:
[PASTE PROMPT]

Expected output:
[PASTE EXPECTED OUTPUT]

Actual outputs:
[PASTE OUTPUTS]

Enter fullscreen mode Exit fullscreen mode

Week 4: Structured output, tools, streaming and conversation history

Week 4 prompt

Using my master instructions, teach Week 4 by building a HolidayLandmark
AI Trip Creation Assistant.

USER INPUT

- destination
- number of days
- budget
- currency
- traveller type
- group size
- preferred activities
- accommodation preference
- travel month

REQUIRED OUTPUT

- title
- summary
- highlights
- daily itinerary
- estimated pricing
- inclusions
- exclusions
- warnings
- required trip JSON

PHASE 1 — ESSENTIAL THEORY

Explain:

- structured outputs
- JSON schema
- Pydantic nested models
- enums and constraints
- structured output versus tool calling
- tool selection
- tool argument validation
- conversation history
- context management
- streaming responses
- retries and fallbacks

PHASE 2 — SCHEMA-FIRST DESIGN

Before writing prompts:

1. Design the complete Pydantic schema.
2. Identify required and optional fields.
3. Add range and length constraints.
4. Define enums.
5. Define validation rules.
6. Show valid and invalid payloads.

Wait for my approval.

PHASE 3 — MINIMAL VERSION

Build Version 1:

- one request
- one LLM call
- validated structured response
- no tools
- no conversation
- no database write until validation succeeds

PHASE 4 — TOOL-CALLING VERSION

Add safe tools:

- get_destination_information
- search_available_trip_categories
- get_currency_information
- estimate_base_price
- check_existing_similar_trips

Teach:

- how the model selects a tool
- how application code executes it
- why the model must not directly access the database
- argument validation
- authorization
- maximum tool-call iterations

PHASE 5 — CONVERSATIONAL VERSION

Allow the assistant to ask for missing information.

Implement:

- limited conversation history
- summarization of old messages
- token budget
- streaming response
- cancellation
- conversation persistence
- user isolation

PHASE 6 — FAILURE TESTS

Make me handle:

- invalid JSON
- missing days
- unrealistic budget
- unsupported destination
- tool timeout
- repeated tool call
- invented pricing
- context overflow
- interrupted stream

PHASE 7 — LARAVEL INTEGRATION

Create:

- Laravel form
- request validation
- AI-service call
- preview page
- human confirmation
- storage only after confirmation
- audit record containing prompt and schema version

PHASE 8 — ASSESSMENT

Test me on structured output, tools, conversation state and streaming.
Enter fullscreen mode Exit fullscreen mode

Week 4 deliverable

Repository: holidaylandmark-ai-trip-assistant
Deliverable: AI-generated trip preview validated before database storage
Enter fullscreen mode Exit fullscreen mode

THEORY PROMPT FOR Tool calling, streaming and conversations

Prompt 1: Function and tool calling

Teach function calling and tool calling from first principles for an
experienced API developer.

Explain:
- the difference between structured output and tool calling
- tool schema
- tool selection
- tool arguments
- execution by application code
- returning tool results to the model
- parallel versus sequential tool calls
- forced versus automatic tool selection
- validation and authorization
- tool-call loops
- maximum iteration limits

Build a MotoShare booking assistant with safe read-only tools:

search_vehicles
check_vehicle_availability
get_booking_status
get_payment_status
create_support_ticket

The model must never execute database queries directly.

Show:
1. Tool schemas
2. Tool registry
3. Dispatcher
4. Pydantic validation
5. Authorization checks
6. Tool-result handling
7. Loop protection
8. Unit and integration tests
Enter fullscreen mode Exit fullscreen mode

Prompt 2: Conversation history and memory

Explain conversation history and memory without treating them as the same thing.

Cover:
- message history
- working memory
- persistent memory
- semantic memory
- user profile memory
- summaries
- retrieval-based memory
- privacy and retention
- context-window limitations

Design a conversation architecture using:
- Laravel for users and permissions
- FastAPI for AI orchestration
- PostgreSQL for durable records
- Redis for temporary session state

Explain:
- what should be saved
- what should never be saved
- when conversations should be summarized
- how users can delete stored information
- how to prevent one user's memory from leaking to another

Provide schemas and pseudocode, but do not implement an agent yet.
Enter fullscreen mode Exit fullscreen mode

Prompt 3: Streaming implementation
Teach and implement token streaming between:

LLM provider
→ FastAPI
→ Laravel
→ Browser UI

Compare:
- Server-Sent Events
- WebSockets
- chunked HTTP responses

Recommend the simplest reliable option for an AI chat application.

Include:
- FastAPI streaming endpoint
- Laravel proxy or direct-client architecture
- cancellation when the user stops generation
- timeout handling
- partial-output handling
- authentication
- logging without storing sensitive content
- frontend JavaScript example
- tests for interrupted and failed streams
Enter fullscreen mode Exit fullscreen mode

Week 5: Embeddings and vector search

The roadmap uses Weeks 5–6 for document chunking, embedding models, vector databases, semantic search, filtering, hybrid search, reranking, citations and RAG evaluation.

Week 5 prompt

Using my master instructions, teach Week 5 by building semantic search for
DevOpsSchool documentation.

PHASE 1 — ESSENTIAL THEORY

Explain:

- what embeddings represent
- vector dimensions
- cosine similarity
- dot product
- query and document embeddings
- semantic search versus keyword search
- embedding-model selection
- normalization
- multilingual embeddings
- vector indexing
- metadata filtering

Use PostgreSQL and pgvector.

Avoid unnecessary mathematical derivations.

PHASE 2 — EMBEDDING EXPERIMENTS

Create experiments that compare:

- identical sentences
- paraphrased sentences
- similar words with different intentions
- technical terms
- multilingual questions
- irrelevant sentences

Show similarity scores and make me interpret them.

PHASE 3 — DATABASE DESIGN

Design:

documents
document_versions
document_chunks
embedding_jobs

Each chunk should have:

- document ID
- version
- source URL/path
- title
- section
- content
- content hash
- metadata
- embedding model
- vector
- timestamps

Explain pgvector indexes and filtering.

PHASE 4 — INGESTION PIPELINE

Build:

load
→ clean
→ normalize
→ chunk
→ add metadata
→ embed
→ store
→ verify

Compare:

- fixed chunking
- recursive chunking
- semantic chunking
- parent-child chunking

Create an experiment using at least three chunk sizes and overlaps.

PHASE 5 — MY PROJECT

Ask me to implement a semantic documentation search API.

Endpoints:

POST /documents/index
POST /documents/reindex
POST /search
DELETE /documents/{id}

Search response:

- chunk content
- source
- section
- similarity score
- metadata

Requirements:

- pgvector
- metadata filters
- deduplication
- content hashing
- incremental indexing
- deleted-document handling
- pytest tests

PHASE 6 — FAILURE TESTING

Test:

- duplicate documents
- modified documents
- empty documents
- huge files
- poor chunks
- wrong embedding dimensions
- embedding-provider failure
- model migration
- cross-project retrieval

PHASE 7 — ASSESSMENT

Make me explain why retrieval failed in several examples.
Enter fullscreen mode Exit fullscreen mode

Week 5 deliverable

Repository: devopsschool-semantic-search
Deliverable: Search API over real documentation using PostgreSQL + pgvector
Enter fullscreen mode Exit fullscreen mode

THEORY PROMPT FOR Embeddings and semantic search

Prompt 1: Embeddings mental model

Teach embeddings to me as an experienced database and backend developer.

Cover:
- what an embedding represents
- vector dimensions
- semantic similarity
- cosine similarity, dot product and Euclidean distance
- why similar words are not always similar intentions
- embedding model selection
- query and document embeddings
- normalization
- multilingual embeddings
- changing embedding models
- limitations of semantic search

Use PostgreSQL and pgvector examples.

Include:
- a small Python experiment
- table schema
- indexing options
- similarity query
- metadata filtering
- common production mistakes
- tests that demonstrate poor and good retrieval
Enter fullscreen mode Exit fullscreen mode

Prompt 2: Document ingestion pipeline

Design a production document-ingestion pipeline for a DevOpsSchool
Knowledge Assistant.

Sources:
- Markdown documentation
- README files
- course pages
- troubleshooting guides
- test documentation
- PDFs where text extraction is reliable

Pipeline stages:
load → clean → normalize → split → enrich metadata → embed → store → verify

Teach me:
- fixed-size, recursive and semantic chunking
- chunk overlap
- parent-child chunking
- document and chunk identifiers
- deduplication
- content hashes
- versioning
- incremental re-indexing
- deleted-document handling
- embedding-model migrations

Provide:
1. Architecture
2. Database schema
3. Chunking strategy
4. Python implementation plan
5. Laravel-triggered indexing flow
6. Test plan
Enter fullscreen mode Exit fullscreen mode

Week 6: Complete production RAG

The blog’s RAG project is a DevOpsSchool Knowledge Assistant that uses documentation, README files, course information, tests and troubleshooting guides, returning an answer, sources and confidence.

Week 6 prompt

Using my master instructions, teach Week 6 by converting the Week 5 semantic
search system into a production RAG Knowledge Assistant.

RESPONSE CONTRACT

{
  "answer": "...",
  "sources": [],
  "confidence": 0.0,
  "answerable": true
}

PHASE 1 — ESSENTIAL THEORY

Explain the complete RAG pipeline:

query
→ query normalization
→ retrieval
→ filtering
→ reranking
→ context construction
→ generation
→ citation validation
→ final response

Explain the difference between:

- retrieval failure
- generation failure
- missing-document failure
- grounding failure

PHASE 2 — NAIVE RAG

Build a minimal RAG version using:

- vector search
- top-k chunks
- one generation prompt
- source list

Then intentionally demonstrate its weaknesses.

PHASE 3 — PRODUCTION IMPROVEMENTS

Add one feature at a time:

1. Metadata filtering
2. Hybrid keyword + vector search
3. Query rewriting
4. Reranking
5. Context deduplication
6. Source citations
7. Insufficient-evidence handling
8. Document-version preference
9. Context token budget
10. Caching

For every feature:

- explain the problem
- define acceptance criteria
- let me implement it
- review my implementation

PHASE 4 — MY PROJECT

Build the DevOpsSchool Knowledge Assistant using:

- documentation
- README files
- course pages
- existing tests
- troubleshooting guides

Requirements:

- answer only from retrieved evidence
- provide citations
- refuse unsupported answers
- return useful confidence information without pretending certainty
- isolate projects and permissions
- record retrieval traces
- track latency, tokens and cost

PHASE 5 — RAG DEBUGGING LAB

Give me traces containing:

- user query
- retrieved chunks
- scores
- selected context
- generated answer

Make me diagnose:

- bad chunking
- missing metadata
- poor embedding match
- incorrect top-k
- noisy retrieval
- missing reranking
- hallucination
- stale documentation
- conflicting documents

PHASE 6 — EVALUATION

Create a golden dataset containing:

- exact questions
- paraphrased questions
- multi-document questions
- unanswerable questions
- ambiguous questions
- outdated-content conflicts
- injection text inside documents

Measure:

- retrieval hit rate
- context relevance
- answer correctness
- groundedness
- citation correctness
- latency
- cost

PHASE 7 — ASSESSMENT

Give practical RAG architecture and debugging questions.
Enter fullscreen mode Exit fullscreen mode

Week 6 deliverable

Repository: devopsschool-rag-assistant
Deliverable: Grounded knowledge assistant with citations and evaluation
Enter fullscreen mode Exit fullscreen mode

THEORY PROMPT FOR Production RAG

Prompt 1: End-to-end RAG

Teach me end-to-end Retrieval-Augmented Generation by building a
DevOpsSchool Knowledge Assistant.

The response contract must be:

{
  "answer": "...",
  "sources": [],
  "confidence": 0.0
}

Cover:
- query preprocessing
- embedding search
- metadata filtering
- hybrid keyword and vector search
- reranking
- context construction
- grounded answer generation
- source citations
- insufficient-evidence responses
- confidence limitations
- latency and cost

First show a naïve RAG pipeline.
Then explain exactly where it fails.
Then develop a production-oriented version.

The system must say that it does not know when the retrieved evidence
does not support an answer.
Enter fullscreen mode Exit fullscreen mode

Prompt 2: RAG debugging

Act as a RAG debugging expert.

I will provide:
- user query
- retrieved chunks
- expected answer
- generated answer
- retrieval scores

Diagnose whether the failure comes from:
- document ingestion
- chunking
- embeddings
- metadata
- query transformation
- retrieval
- top-k selection
- reranking
- context construction
- generation prompt
- unsupported source content

Return:
1. Root cause
2. Evidence supporting the diagnosis
3. Smallest corrective action
4. Retrieval test to add
5. Generation test to add
6. Whether re-embedding is required
7. Whether the document itself lacks the answer

Data:
[PASTE RAG TRACE]
Prompt 3: RAG evaluation dataset
Help me create a golden evaluation dataset for a RAG system.

Create categories for:
- exact factual questions
- paraphrased questions
- multi-document questions
- questions requiring metadata filters
- unanswerable questions
- ambiguous questions
- outdated-information conflicts
- prompt-injection attempts inside documents
- similar but incorrect documents
- large-context questions

For each test case define:
- query
- expected source document
- expected facts
- forbidden claims
- answerability
- retrieval success criteria
- generation success criteria

Do not generate fake company facts. Give me a template and guide me to
populate it from my real documents.
Enter fullscreen mode Exit fullscreen mode

Week 7: Agentic AI without a framework

The blog defines an agent as a system that receives a goal, selects tools, acts, observes the result and continues until completion or human intervention. It recommends starting with a fixed workflow and adding autonomy only where genuinely necessary.

Week 7 prompt

Using my master instructions, teach Week 7 by building a tool-using agent
in plain Python without LangGraph.

PROJECT

Build a Laravel Error Investigation Agent.

SAFE TOOLS

- read_log_excerpt
- search_repository
- read_file
- list_recent_migrations
- read_configuration
- search_known_solutions

PHASE 1 — ESSENTIAL THEORY

Explain:

- agent goal
- agent loop
- observation
- action
- tool
- planning
- state
- memory
- termination
- human approval
- bounded autonomy
- idempotency
- error recovery

Compare:

- chatbot
- tool-using assistant
- deterministic workflow
- router
- autonomous agent

Explain when a normal function is better than an agent.

PHASE 2 — FIXED WORKFLOW FIRST

Build a deterministic workflow:

receive error
→ classify error
→ read logs
→ choose investigation category
→ inspect relevant information
→ create report
→ request approval
→ stop

Use plain Python functions and typed state.

PHASE 3 — ADD LIMITED DECISION-MAKING

Allow the model to choose among approved read-only tools.

Implement:

- tool registry
- Pydantic argument validation
- authorization
- result size limits
- timeout
- maximum steps
- duplicate-action detection
- stop conditions

PHASE 4 — MY TASK

Ask me to implement the agent loop myself.

Do not provide the full code.

Acceptance criteria:

- explicit state
- deterministic termination
- maximum 10 steps
- read-only access
- audit trail
- no shell
- no arbitrary SQL
- no production modification
- human approval before recommendations are applied

PHASE 5 — FAILURE SIMULATION

Create:

- infinite loop
- repeated tool call
- incorrect tool choice
- invalid arguments
- tool timeout
- unauthorized file request
- poisoned tool output
- incomplete investigation

Let me debug each one.

PHASE 6 — ASSESSMENT

Ask me to decide which parts should be deterministic and which parts
benefit from model reasoning.
Enter fullscreen mode Exit fullscreen mode

Week 7 deliverable

Repository: plain-python-error-investigation-agent
Deliverable: Safe tool-calling agent without an agent framework
Enter fullscreen mode Exit fullscreen mode

THEORY PROMPT FOR Agentic AI foundations

The blog correctly distinguishes an agent from a chatbot: an agent receives a goal, chooses tools, acts, observes results and continues until completion or human intervention. It also recommends beginning with deterministic workflows before introducing autonomy.

Prompt 1: Agent mental model

Teach Agentic AI to me using state-machine and workflow concepts.

Cover:
- agent loop
- goal
- observation
- reasoning
- action
- tool
- state
- memory
- termination
- human approval
- planning
- retries
- error recovery
- idempotency
- bounded autonomy

Compare:
- chatbot
- tool-using assistant
- deterministic workflow
- router
- autonomous agent
- long-running agent

For each one explain:
- appropriate use case
- inappropriate use case
- level of risk
- testing difficulty
- production controls required

Conclude with a decision tree answering:
“Does this problem actually require an agent?”
Enter fullscreen mode Exit fullscreen mode

Prompt 2: Build without a framework first

Help me implement a minimal tool-using agent in plain Python before using LangGraph.

Agent task:
Investigate an application error using safe, read-only tools.

Tools:
- read_log_excerpt
- search_repository
- read_file
- search_documentation
- list_recent_migrations

Requirements:
- explicit state model
- maximum-step limit
- validated tool arguments
- permission checks
- tool timeout
- duplicate-action detection
- final report
- request human approval before any proposed modification
- no shell access
- no production writes

Guide me through:
1. State design
2. Agent loop pseudocode
3. Tool interface
4. Stop conditions
5. Implementation
6. Tests
7. Failure simulation
Enter fullscreen mode Exit fullscreen mode

Week 8: LangGraph and stateful agent workflows

The roadmap identifies LangGraph as suitable for stateful, long-running workflows with shared state, nodes, persistence and memory.

Week 8 prompt

Using my master instructions, teach Week 8 by converting the Week 7 plain-Python
agent into LangGraph.

PHASE 1 — ESSENTIAL THEORY

Explain using Laravel and workflow comparisons:

- StateGraph
- typed shared state
- nodes
- edges
- conditional edges
- reducers
- START and END
- checkpoints
- persistence
- interrupts
- human-in-the-loop
- retries
- subgraphs
- resumability

For every concept:

- explain the problem it solves
- show where it maps to the Week 7 implementation
- explain when it is unnecessary

PHASE 2 — ARCHITECTURE FIRST

Before coding, provide:

- graph diagram
- state schema
- node responsibilities
- edge conditions
- stop conditions
- checkpoint strategy
- approval points

Wait for my approval.

PHASE 3 — STEP-BY-STEP CONVERSION

Guide me through:

Task 1:
Define typed state.

Task 2:
Convert classification into a node.

Task 3:
Convert log inspection into a node.

Task 4:
Add conditional routing.

Task 5:
Add tool execution.

Task 6:
Add report generation.

Task 7:
Add human approval interrupt.

Task 8:
Add persistent checkpoints.

Task 9:
Resume an interrupted investigation.

Wait for my code after every task.

PHASE 4 — WHY STATEGRAPH?

After the minimal graph works, compare it with the Week 7 implementation.

Explain concretely:

- what StateGraph improved
- what complexity it added
- when plain Python remains preferable
- how persistence and interrupts change the design

PHASE 5 — PRODUCTION UPGRADE

Add:

- maximum-iteration guard
- retries per node
- idempotent nodes
- redacted checkpoints
- audit events
- tool authorization
- correlation IDs
- FastAPI endpoint
- Laravel API client
- queue execution
- Docker
- pytest branch coverage

PHASE 6 — DEBUGGING

Give me broken graphs containing:

- infinite cycles
- missing state
- overwritten state
- incorrect reducer
- dead-end node
- checkpoint failure
- repeated side effect
- approval bypass

PHASE 7 — ASSESSMENT

Give implementation, architecture and interview questions about LangGraph.
Enter fullscreen mode Exit fullscreen mode

Week 8 deliverable

Repository: langgraph-production-error-agent
Deliverable: Resumable, checkpointed investigation workflow with approval
Enter fullscreen mode Exit fullscreen mode

THEORY PROMPT FOR LangGraph and stateful workflows

Prompt 1: Learn LangGraph efficiently

Teach me LangGraph assuming I understand Laravel workflows, queues,
state machines and microservices.

Do not start with installation.

First explain:
- why LangGraph exists
- shared state
- nodes
- edges
- conditional edges
- StateGraph
- reducers
- checkpoints
- persistence
- interrupts
- human-in-the-loop
- subgraphs
- retries
- time travel or replay concepts

For each concept:
- show its Laravel or backend equivalent
- explain when it is useful
- show a minimal example
- describe one production mistake

Then convert the plain-Python error investigation agent from Week 7
into LangGraph.
Enter fullscreen mode Exit fullscreen mode

Prompt 2: Production Error Investigation Agent

Design a Production Error Investigation Agent using LangGraph.

Workflow:
1. Accept error details.
2. Read a limited Laravel log excerpt.
3. Classify the error.
4. Search relevant repository files.
5. Inspect related configuration and migrations.
6. Search known documentation.
7. formulate likely causes.
8. Propose a fix.
9. Generate tests.
10. Ask for human approval.
11. Stop without modifying production.

Requirements:
- typed shared state
- deterministic stages where possible
- conditional routing
- checkpointing
- resumability
- maximum iteration limit
- tool authorization
- audit trail
- idempotency
- timeout and retry policies
- sensitive-data redaction

First provide the graph diagram and state schema.
Do not write implementation until I approve them.
Enter fullscreen mode Exit fullscreen mode

Prompt 3: LangGraph debugging

Review the following LangGraph implementation.

Check specifically for:
- mutable or poorly defined state
- incorrect reducers
- infinite cycles
- missing stop conditions
- non-idempotent nodes
- unsafe tool execution
- checkpoints containing sensitive data
- improper retry behavior
- confused separation between deterministic logic and model decisions
- missing human approval
- untestable nodes

Return:
1. Graph-level defects
2. Node-level defects
3. State-schema defects
4. Security defects
5. Corrected graph diagram
6. Minimum changes required
7. Test cases for every branch

Implementation:
[PASTE CODE]
Week 9: Model Context Proto
Enter fullscreen mode Exit fullscreen mode

Week 9: MCP server development

The blog describes MCP as an open standard for connecting AI applications with external tools and data, and proposes a MotoShare MCP server with restricted tools rather than dangerous capabilities such as arbitrary SQL or shell execution.

Week 9 prompt
Using my master instructions, teach Week 9 by building a secure MotoShare
MCP server.

TOOLS

- get_vehicle
- search_bookings
- get_payment_status
- read_application_logs
- create_support_ticket

PHASE 1 — ESSENTIAL THEORY

Explain:

- MCP host
- MCP client
- MCP server
- resources
- tools
- prompts
- tool schemas
- transport
- authentication
- permission boundaries
- local and remote MCP servers

Compare MCP with:

- REST API
- OpenAPI
- function calling
- application plugins
- direct SDK integration

Explain what MCP standardizes and what it does not.

PHASE 2 — THREAT MODEL

Before implementation, identify:

- unauthorized users
- tenant leakage
- excessive permissions
- prompt injection
- malicious tool arguments
- sensitive logs
- replay
- duplicate execution
- denial of service
- secret leakage

PHASE 3 — CONTRACT DESIGN

For every tool, define:

- purpose
- input schema
- output schema
- allowed roles
- data boundaries
- pagination
- rate limit
- timeout
- audit event
- failure responses

Create a permission matrix.

PHASE 4 — IMPLEMENTATION

Guide me one tool at a time.

Requirements:

- strict schemas
- service authentication
- user authorization
- tenant isolation
- parameter allowlists
- pagination
- redaction
- rate limiting
- structured logging
- audit trail
- human confirmation before support-ticket creation

Do not expose:

- arbitrary SQL
- unrestricted shell
- unrestricted file access
- raw database credentials
- production write operations

PHASE 5 — CLIENT INTEGRATION

Connect the MCP server to an AI host.

Demonstrate:

- tool discovery
- tool selection
- argument validation
- result handling
- failed tool calls
- permission denial

PHASE 6 — SECURITY TESTING

Make me test:

- SQL injection attempt
- path traversal
- cross-user booking access
- unbounded log request
- prompt injection through tool output
- duplicate ticket creation
- expired credentials
- excessive request rate

PHASE 7 — ASSESSMENT

Ask me to choose when MCP is better than direct REST integration.
Enter fullscreen mode Exit fullscreen mode

Week 9 deliverable

Repository: motoshare-secure-mcp-server
Deliverable: Authenticated MCP server with five bounded tools
Enter fullscreen mode Exit fullscreen mode

THEORY PROMPT FOR Model Context Protocol

The blog’s Week 9 covers MCP hosts, clients, servers, resources, tools, prompts, authentication and permission boundaries, with a MotoShare MCP server as the project.

Prompt 1: Learn MCP architecture

Teach MCP to me as an experienced API and microservice developer.

Explain:
- what problem MCP solves
- host, client and server
- resources
- tools
- prompts
- schemas
- transport
- capability negotiation
- local versus remote servers
- authentication
- authorization and trust boundaries

Compare MCP with:
- REST APIs
- OpenAPI
- function calling
- plugins
- SDK integrations

Explain what MCP standardizes and what it does not.

Conclude with:
- when to use MCP
- when a normal REST API is better
- security checklist
- production architecture diagram
Enter fullscreen mode Exit fullscreen mode

Prompt 2: MotoShare MCP server

Guide me through building a secure MotoShare MCP server.

Expose only these tools:
- get_vehicle
- search_bookings
- get_payment_status
- read_application_logs
- create_support_ticket

Requirements:
- strict input schemas
- user and role authorization
- tenant isolation
- read limits
- log redaction
- pagination
- rate limiting
- audit logging
- timeout handling
- no arbitrary SQL
- no shell commands
- no unrestricted file access
- human confirmation before creating a support ticket

First define:
1. Threat model
2. Tool contracts
3. Permission matrix
4. Error model
5. Audit event schema
6. Test cases

Only then implement the server step by step.
Enter fullscreen mode Exit fullscreen mode

Prompt 3: MCP security review

Perform a security review of this MCP server.

Look for:
- excessive tool permissions
- prompt injection through tool output
- unauthorized cross-user access
- tenant data leakage
- arbitrary parameters
- SQL injection
- path traversal
- sensitive log exposure
- missing rate limits
- missing human approval
- replay and duplicate execution
- secrets in errors
- weak authentication

Return findings using:
severity, attack scenario, affected component, remediation and test case.

Server implementation:
[PASTE CODE]
Enter fullscreen mode Exit fullscreen mode

Week 10: Multi-agent systems

The roadmap covers supervisor-worker, router, planner-executor, reviewer, parallel workers, handoffs and shared state, while warning against creating multiple agents merely to appear advanced.

Week 10 prompt

Using my master instructions, teach Week 10 by building a Software QA
Agent Workflow.

PROPOSED ROLES

- Requirement Analyzer
- Test Planner
- API Test Generator
- UI Test Generator
- Security Reviewer
- Final Report Composer

PHASE 1 — ESSENTIAL THEORY

Explain:

- supervisor-worker pattern
- router pattern
- planner-executor pattern
- reviewer or critic pattern
- parallel workers
- handoffs
- shared state
- independent state
- disagreement handling
- termination

For every pattern explain:

- problem solved
- simpler alternative
- cost
- latency
- new failure modes
- testing difficulty

PHASE 2 — AGGRESSIVE SIMPLIFICATION

Before building anything, evaluate every proposed agent.

For each role ask:

- Does it genuinely require reasoning?
- Can deterministic code perform it?
- Can structured output perform it?
- Can it be a normal LangGraph node?
- Does a separate agent justify its cost?

Replace unnecessary agents with normal functions.

PHASE 3 — CONTRACT DESIGN

For retained agents define:

- responsibility
- input schema
- output schema
- allowed tools
- maximum calls
- timeout
- success criteria
- failure behavior
- handoff contract

Do not allow uncontrolled agent-to-agent conversation.

PHASE 4 — PROJECT

Input:

- feature requirement
- API contract
- UI screenshots or acceptance criteria
- existing test documentation

Output:

- analyzed requirements
- test plan
- API test cases
- UI test cases
- security test cases
- traceability matrix
- final review report

Requirements:

- supervisor controls execution
- structured handoffs
- duplicate-test detection
- cost limit
- maximum-step limit
- audit trail
- human review before writing files or creating a PR

PHASE 5 — COMPARISON EXPERIMENT

Build and compare:

Version A:
One structured LLM call

Version B:
Single-agent workflow

Version C:
Multi-agent workflow

Measure:

- quality
- token usage
- latency
- cost
- maintainability
- error rate

PHASE 6 — FAILURE TESTING

Test:

- conflicting agent outputs
- missing handoff field
- repeated task
- endless reviewer cycle
- supervisor failure
- cost limit exceeded
- unsupported requirement
- security reviewer disagreement

PHASE 7 — ASSESSMENT

Make me defend whether multi-agent architecture is justified.
Enter fullscreen mode Exit fullscreen mode

Week 10 deliverable

Repository: ai-software-qa-workflow
Deliverable: Evaluated and simplified multi-agent QA system
Enter fullscreen mode Exit fullscreen mode

THEORY PROMPT FOR Multi-agent systems

Prompt 1: Understand patterns and trade-offs

Teach multi-agent systems without hype.

Cover:
- supervisor-worker
- router
- planner-executor
- reviewer or critic
- parallel workers
- handoffs
- shared state
- independent state
- consensus
- termination
- conflict resolution

For each pattern explain:
- problem it solves
- simpler alternative
- additional latency
- additional cost
- new failure modes
- testing strategy

Provide a decision framework for choosing among:
single LLM call, deterministic workflow, single agent and multi-agent system.

Be critical: explain why many multi-agent designs should be replaced by
ordinary functions or workflow nodes.
Enter fullscreen mode Exit fullscreen mode

Prompt 2: Software QA Agent Team

Design a Software QA Agent Team for Laravel and Python projects.

Roles:
- Requirement Analyzer
- Test Planner
- API Test Generator
- UI Test Generator
- Security Reviewer
- Final Report Composer

Requirements:
- each agent has one bounded responsibility
- structured input and output contracts
- no uncontrolled agent-to-agent conversation
- supervisor controls sequencing
- shared requirement identifier
- duplicate-test detection
- disagreement handling
- cost and step limits
- human review before tests are committed

First evaluate whether every proposed role genuinely needs an LLM.
Replace any unnecessary agent with deterministic code.

Then provide:
1. Architecture
2. Agent contracts
3. State schema
4. Handoff rules
5. Failure handling
6. Evaluation plan
7. Implementation stages
Enter fullscreen mode Exit fullscreen mode

Prompt 3: Simplification review

Review this multi-agent architecture and aggressively simplify it.

For every agent ask:
- Does it require reasoning?
- Could a normal function perform this task?
- Could it be a LangGraph node?
- Could structured output replace the agent?
- Does it provide enough value to justify latency and cost?

Return:
1. Agents to retain
2. Agents to replace with deterministic functions
3. Agents to merge
4. Simplified architecture
5. Estimated reduction in model calls
6. New testing strategy

Architecture:
[PASTE DESIGN]
Enter fullscreen mode Exit fullscreen mode

Week 11: Evaluation, security and observability

The roadmap says production AI work should include golden datasets, prompt regression tests, RAG evaluation, tool-call accuracy, hallucination checks, cost and latency tracking, injection protection, validation, rate limiting, audit logs and approval for destructive actions. It recommends creating at least 50 test cases.

Week 11 prompt

Using my master instructions, teach Week 11 by creating a complete evaluation,
security and observability system for the projects built in Weeks 1–10.

PHASE 1 — ESSENTIAL THEORY

Explain:

- golden datasets
- offline evaluation
- online monitoring
- prompt regression
- retrieval evaluation
- groundedness
- citation correctness
- schema validity
- tool selection accuracy
- tool argument accuracy
- task-completion rate
- hallucination checks
- human evaluation
- LLM-as-judge limitations

PHASE 2 — EVALUATION FRAMEWORK

Design separate evaluation suites for:

1. Structured outputs
2. Prompt templates
3. RAG
4. Tool calling
5. Agent workflows
6. MCP tools
7. Multi-agent handoffs

Define:

- test input
- expected behavior
- pass/fail rule
- metric
- threshold
- regression status

PHASE 3 — 50-CASE DATASET

Help me create at least 50 meaningful tests covering:

- normal requests
- missing information
- incorrect information
- ambiguous requests
- malformed output
- very large input
- timeout
- provider failure
- tool failure
- repeated execution
- unauthorized action
- prompt injection
- indirect injection
- data leakage
- stale document
- low-quality retrieval
- infinite agent loop
- human rejection

Do not invent company facts.

Use templates that I populate using real project information.

PHASE 4 — SECURITY REVIEW

Threat-model:

- prompt injection
- indirect injection
- tool abuse
- cross-user access
- cross-tenant access
- PII leakage
- secret exposure
- arbitrary actions
- replay
- resource exhaustion
- unsafe generated code

Implement controls and corresponding tests.

PHASE 5 — OBSERVABILITY

Trace:

Browser
→ Laravel
→ Laravel queue
→ FastAPI
→ LLM
→ RAG
→ tools
→ database

Track:

- correlation ID
- trace ID
- model
- prompt version
- schema version
- token usage
- cost
- latency
- retrieval results
- tool calls
- retries
- validation failures
- final status

Define privacy-safe logging and redaction rules.

PHASE 6 — DASHBOARD

Design metrics for:

- request volume
- success rate
- schema failure
- tool failure
- average cost
- p95 latency
- answerability
- retrieval quality
- agent completion
- approval rejection
- injection detection

PHASE 7 — ASSESSMENT

Give me a failed production trace and make me find the root cause.
Enter fullscreen mode Exit fullscreen mode

Week 11 deliverable

Repository: ai-evaluation-security-observability
Deliverable: 50+ test cases, traces, dashboards and security controls
Enter fullscreen mode Exit fullscreen mode

THEORY PROMPT FOR Evaluation, security and observability

The blog describes this week as the distinction between a demo developer and a professional AI engineer. It calls for golden datasets, prompt regression tests, tool-call accuracy, hallucination checks, latency and cost tracking, injection protection, output validation, rate limiting and audit logs.

Prompt 1: Evaluation framework

Teach me how to evaluate an AI system like a production software system.

My system may contain:
- prompts
- structured outputs
- RAG
- tools
- agents
- human approval

Define separate metrics for:
- answer correctness
- groundedness
- retrieval relevance
- citation correctness
- schema validity
- tool selection
- tool argument accuracy
- task completion
- safety
- latency
- cost

Help me design:
- golden dataset
- offline evaluation
- regression testing
- production monitoring
- human review sampling
- pass/fail thresholds

Explain where LLM-as-judge is useful and where it is unreliable.
Enter fullscreen mode Exit fullscreen mode

Prompt 2: Generate 50 serious test cases

Create a test-plan template containing at least 50 categories of test cases
for my AI application.

Include:
- normal requests
- missing information
- contradictory information
- ambiguous instructions
- malformed JSON
- oversized input
- timeouts
- provider failure
- rate limiting
- tool failure
- repeated execution
- unauthorized tool request
- prompt injection
- indirect injection from retrieved documents
- PII leakage
- cross-user data access
- unsupported claims
- weak retrieval
- stale documents
- duplicate actions
- infinite agent loops
- interrupted workflow
- human approval rejection

Do not invent expected business facts.

For each case provide:
test objective, input pattern, expected behavior, failure signal and automation method.
Enter fullscreen mode Exit fullscreen mode

Prompt 3: Observability design

Design observability for a Laravel + FastAPI + LLM agent system.

I need to trace one request across:
browser → Laravel → queue → FastAPI → LLM → tools → database

Cover:
- correlation IDs
- traces
- spans
- structured logs
- model name
- prompt version
- token usage
- latency
- cost
- tool calls
- retries
- validation failures
- human approval
- final status

Protect:
- user messages
- credentials
- access tokens
- personal information
- retrieved confidential documents

Provide:
1. Event schema
2. Trace example
3. Redaction rules
4. Metrics dashboard
5. Alert thresholds
6. Audit versus operational log separation
Enter fullscreen mode Exit fullscreen mode

Week 12: Enterprise AI Operations Assistant

The blog’s final project is an AI Operations Assistant that can read repositories, logs and documentation, find known solutions, produce investigation reports, suggest changes, generate tests, open a draft pull request, require approval and maintain an audit trail.

Week 12 prompt

Using my master instructions, guide me through building the final enterprise
AI Operations Assistant.

CAPABILITIES

- read authorized GitHub repositories
- read sanitized Laravel logs
- read project documentation
- search known solutions using RAG
- investigate application errors
- identify likely causes
- propose code changes
- generate tests
- create a draft pull request
- request human approval
- maintain an audit trail

SAFETY RULES

- never modify production
- never merge pull requests
- never expose secrets
- never execute arbitrary shell commands
- never execute arbitrary SQL
- use least-privilege access
- require approval before external write actions
- make every action auditable

PHASE 1 — REQUIREMENTS

Help me define:

- users
- roles
- use cases
- non-functional requirements
- scope
- out-of-scope actions
- acceptance criteria

PHASE 2 — THREAT MODEL

Identify:

- repository data leakage
- malicious logs
- prompt injection in code or docs
- unsafe code suggestions
- tool abuse
- unauthorized pull request
- secret leakage
- cross-project access
- duplicate external actions

PHASE 3 — ARCHITECTURE

Use:

- Laravel main application
- FastAPI AI API
- LangGraph workers
- PostgreSQL + pgvector
- Redis
- queue workers
- GitHub integration
- MCP tools where justified
- centralized logs and traces

Provide:

- component diagram
- request sequence
- state schema
- trust boundaries
- permission model
- data model
- API contracts
- failure matrix

Wait for my approval.

PHASE 4 — IMPLEMENTATION STAGES

Stage 1:
Authentication, authorization and project boundaries

Stage 2:
Read-only repository tools

Stage 3:
Sanitized log-reading tools

Stage 4:
Document ingestion and RAG

Stage 5:
Error-investigation LangGraph workflow

Stage 6:
Suggested-code patch generation

Stage 7:
Test generation

Stage 8:
Human approval interrupt

Stage 9:
Draft pull-request creation

Stage 10:
Audit trail

At every stage:

- define acceptance criteria
- let me implement it
- review my code
- require tests
- do not continue until it passes

PHASE 5 — PRODUCTION DEPLOYMENT

Add:

- Docker
- separate API and worker containers
- Redis queues
- health and readiness checks
- secrets management
- service authentication
- model fallback
- caching
- rate limits
- cost limits
- horizontal scaling
- backup
- rollback
- GitHub Actions CI/CD

PHASE 6 — FAILURE EXERCISES

Test:

- LLM unavailable
- GitHub unavailable
- Redis unavailable
- vector database unavailable
- malformed agent state
- malicious repository instruction
- repeated PR request
- approval rejection
- long-running investigation
- cost threshold exceeded

PHASE 7 — FINAL EXAMINATION

Evaluate me through:

- five architecture scenarios
- five debugging scenarios
- three coding exercises
- two security reviews
- one complete system-design task

Score:

- LLM development
- prompt design
- structured output
- RAG
- tool calling
- agents
- LangGraph
- MCP
- multi-agent architecture
- evaluation
- security
- observability
- deployment

Tell me honestly whether I am ready for production AI engineering.
Enter fullscreen mode Exit fullscreen mode

Week 12 deliverable

Repository: enterprise-ai-operations-assistant
Deliverable: Complete controlled AI workflow with draft PR and human approval
Enter fullscreen mode Exit fullscreen mode

THEORY PROMPT FOR Production deployment and capstone

The final week of the roadmap covers Docker, workers, Redis, background jobs, authentication, tracing, fallbacks, caching, CI/CD and scaling. Its suggested capstone is an AI Operations Assistant that can investigate issues, suggest changes, generate tests and open a draft pull request while requiring human approval.

Prompt 1: Production architecture

Act as a principal AI platform architect.

Design a production deployment architecture for:

Laravel application
Python FastAPI AI service
LangGraph workers
PostgreSQL + pgvector
Redis
LLM providers
MCP servers
GitHub integration
centralized logs and traces

Cover:
- Docker containers
- API and worker separation
- queues
- autoscaling
- health checks
- readiness checks
- secrets
- service authentication
- caching
- provider fallback
- cost limits
- rate limits
- backups
- zero-downtime deployment
- rollback
- CI/CD
- disaster recovery

Provide:
1. Component diagram
2. Request sequence
3. Deployment topology
4. Environment variables
5. Failure matrix
6. Security boundaries
7. Deployment checklist
Enter fullscreen mode Exit fullscreen mode

Prompt 2: Capstone execution prompt

Guide me in building an enterprise AI Operations Assistant.

Capabilities:
- read authorized GitHub repositories
- read sanitized Laravel logs
- read project documentation
- search known solutions using RAG
- generate an investigation report
- suggest code changes
- generate tests
- prepare a draft pull request
- request human approval
- maintain an audit trail

Safety restrictions:
- no automatic production modification
- no unrestricted shell
- no arbitrary SQL
- no merging pull requests
- no secret exposure
- least-privilege repository access
- every external action must be auditable

Work in phases:

Phase 1: Requirements and threat model
Phase 2: Architecture and state schema
Phase 3: Read-only tools
Phase 4: RAG
Phase 5: LangGraph workflow
Phase 6: GitHub draft-PR integration
Phase 7: Evaluation
Phase 8: Deployment
Phase 9: Security review

At each phase:
- give acceptance criteria
- let me implement it
- review my code
- require tests
- do not continue until the phase passes
Enter fullscreen mode Exit fullscreen mode

Prompt 3: Final expert assessment

Evaluate whether I am ready to work professionally as an AI and
Agentic AI developer.

Assess me through a practical examination covering:
- LLM fundamentals
- prompt design
- structured output
- tool calling
- FastAPI architecture
- Laravel integration
- embeddings
- RAG
- LangGraph
- MCP
- agent design
- multi-agent trade-offs
- evaluation
- security
- observability
- deployment

Do not ask definition-only questions.

Give me:
1. Five architecture scenarios
2. Five debugging scenarios
3. Three coding exercises
4. Two security reviews
5. One system-design assignment

After I answer:
- score each competency
- identify evidence of understanding
- identify weak areas
- provide a two-week correction plan
- tell me honestly whether I am production-ready
Enter fullscreen mode Exit fullscreen mode

Weekly schedule

Prompt to use after every coding session

Review today's implementation as a strict senior AI engineer.

Here is what I attempted:

[DESCRIBE TASK]

Here is my code:

[PASTE CODE OR REPOSITORY FILES]

Evaluate:

- whether I understand the topic
- correctness
- architecture
- security
- validation
- failure handling
- test coverage
- cost
- observability
- production readiness

Do not rewrite the entire implementation.

Return:

1. What I implemented correctly
2. Critical defects
3. Design weaknesses
4. Missing tests
5. One debugging exercise
6. One production improvement
7. Three revision questions
8. My next coding task
9. Score out of 10
Enter fullscreen mode Exit fullscreen mode

Prompt to convert each week into revision notes

Convert this week's work into an experienced developer's revision document.

Include:

1. Problem solved
2. Important mental models
3. Architecture
4. Main implementation
5. Important code patterns
6. Failures encountered
7. Debugging lessons
8. Security risks
9. Performance and cost considerations
10. Testing strategy
11. Laravel integration
12. Five interview questions
13. Five flashcards
14. Topics that need revision next week

Keep it technical and practical.
Do not include beginner explanations.
Enter fullscreen mode Exit fullscreen mode

Daily one-hour learning workflow

Use the following workflow for every topic:

Time    Activity
0–10 min  Run the teaching prompt and understand the mental model
10–20 min Ask questions about unclear concepts
20–40 min Implement the smallest working version
40–50 min Add one failure scenario and one test
50–60 min Ask AI to review your code and quiz you

At the end of each session, use:

Summarize today's session into a developer revision note.

Include:
- five essential ideas
- architecture learned
- code completed
- mistakes I made
- unresolved questions
- five flashcards
- three interview questions
- tomorrow's first task

Keep the note under 700 words.
Enter fullscreen mode Exit fullscreen mode

Top comments (0)