Generative AI — TechForge

100MChatGPT users · 60 days

$1T+Market by 2030

5+Output Modalities

2022Mass Adoption Begins

00

What is Generative AI?

Generative AI = AI that creates new content rather than just classifying or predicting. Traditional AI answers "Is this a cat?" Generative AI answers "Draw me a cat" or "Write a story about a cat." The model learns statistical patterns from vast data and uses them to synthesise novel, original output that matches those patterns.

Traditional AI: Classify(input) → label · GenAI: Generate(prompt) → new content

01

What Can GenAI Create?

Text: Articles, emails, summaries, stories, code, translations. Images: Illustrations, concept art, photorealistic scenes, product designs. Code: Functions, full programs, debugging, refactoring. Audio: Music composition, voice cloning, sound effects, podcasts. Video: Text-to-video clips, animation, VFX, video editing.

Input: prompt (text / image / audio) → Model → Output: new synthesised content

02

How It Works — the Big Idea

GenAI models are trained on massive datasets (trillions of tokens, billions of images). They learn the statistical structure of the data — what words/pixels tend to follow what. At inference, they sample from learned distributions to generate new outputs that look like plausible continuations of the training data. They don't "copy" — they extrapolate.

Train on massive data → learn P(next token | context) → sample to generate

03

Why Now? The Three Enablers

1. Transformers (2017): Attention architecture scales efficiently. 2. Scale: Scaling laws — more data + more compute = reliably better models. 3. Instruction tuning + RLHF: Made raw language models usable by non-experts. ChatGPT (Nov 2022) was the moment these three converged publicly.

Transformers + Scale + RLHF = ChatGPT moment · 100M users in 60 days

04

GenAI vs Discriminative AI

Discriminative (traditional ML): learns the boundary between classes — P(label | data). Used for classification, regression, detection. Generative: learns the full data distribution — P(data). Can generate new samples, do zero-shot tasks, understand context deeply. Not always better — for simple classification, discriminative models are faster and cheaper.

Discriminative: P(y|x) → best for prediction · Generative: P(x) → best for creation

The key insight: GenAI doesn't "copy." It learns statistical patterns and generates content that is structurally similar but distinct. Like a musician who has listened to 10 million songs — they write original music, shaped by but not copied from what they heard.

Large Language Models (LLMs)

Text · Code · Reasoning

Transformer-based models trained on trillions of text tokens. Predict the next token autoregressively. Scale dramatically with data and compute. The foundation of the current GenAI wave.

GPT-4 · Claude · Llama 3 · Gemini · Mistral · Phi

Diffusion Models

Images · Video · Audio

Learn to reverse a noise-adding process. Forward: add Gaussian noise over T steps. Reverse: train U-Net to denoise. Generate by starting from pure noise and iteratively denoising. Current SOTA for image and video generation.

DALL·E 3 · Stable Diffusion · Midjourney · Sora

GANs

Images · Fakes · Style

Generator vs Discriminator adversarial game. Generator creates fakes; Discriminator detects them. At equilibrium, generator produces realistic data. Fast inference — one forward pass. Training is notoriously unstable (mode collapse).

StyleGAN3 · CycleGAN · DCGAN · BigGAN

VAE

Latent Space · Compression

Encoder maps data to a distribution (μ, σ) in latent space. Decoder samples and reconstructs. KL divergence term makes latent space smooth and continuous — enables interpolation and controlled generation.

Used as latent space backbone in Stable Diffusion

Multimodal Models

Text + Image + Audio

Understand and generate across multiple modalities simultaneously. Vision encoders (CLIP, SigLIP) connect image and text spaces. Process images, audio, and text in a single model context window.

GPT-4V · Claude 3 · Gemini · LLaVA · Whisper

Model Comparison

Model Type	Output	Quality	Speed	Training Stability	Best For
LLM	Text / Code	⭐⭐⭐⭐⭐	Fast	Stable	Q&A, writing, coding, reasoning
Diffusion	Images / Video	⭐⭐⭐⭐⭐	Slow (50–1000 steps)	Very stable	Text-to-image, art, video
GAN	Images / Audio	⭐⭐⭐⭐	Very fast	Hard (mode collapse)	Real-time synthesis, style transfer
VAE	Images / Latents	⭐⭐⭐	Fast	Stable	Latent space, anomaly detection
Multimodal	Text + Image + ...	⭐⭐⭐⭐⭐	Medium	Stable	VQA, image captioning, OCR

00

What Makes an LLM?

A Large Language Model is a Transformer trained on massive text corpora to predict the next token. "Large" means billions to trillions of parameters. The surprising discovery (scaling laws): as model size and data grow, capabilities emerge that weren't explicitly trained for — reasoning, translation, code, arithmetic.

P(token_n | token_1...token_{n-1}) · autoregressive generation · one token at a time

01

Tokenisation — How LLMs See Text

LLMs don't process characters — they process tokens (subword chunks). "Tokenisation" → ["Token", "isation"]. GPT-4 uses ~100k token vocabulary. Context window = max tokens the model can "see" at once. GPT-4: 128k tokens ≈ 96k words ≈ a full novel. Tokens ≠ words: "ChatGPT" = 1 token, "supercalifragilistic" = 4–5 tokens.

"Hello world" → [15496, 995] (GPT tokenisation) · ~1 token ≈ 0.75 words

02

Pretraining vs Fine-tuning vs RLHF

Pretraining: Next-token prediction on internet-scale text (trillions of tokens). Learns language, facts, reasoning patterns. Produces a "base model" that completes text but isn't chat-friendly. SFT (Supervised Fine-Tuning): Fine-tune on human-written instruction-response pairs → model follows instructions. RLHF: Human rankers compare outputs → train reward model → PPO optimisation. Makes model helpful, harmless, honest.

Pretrain (next token) → SFT (instructions) → RLHF (human preferences) = ChatGPT

03

Scaling Laws — Why Bigger Works

Kaplan et al. (OpenAI, 2020) showed that LLM performance scales as a power law with model parameters, data size, and compute — predictably. Chinchilla law (DeepMind, 2022): optimal training uses ~20 tokens per parameter. GPT-3: 175B params, undertrained. Llama 3 8B: outperforms GPT-3 by being trained on far more data relative to size.

Loss ∝ N^(-0.076) · D^(-0.095) · C^(-0.050) · Chinchilla: optimal tokens = 20 × params

04

Temperature & Sampling

Temperature: controls randomness. T=0 → greedy (always pick highest prob token, deterministic). T=1 → sample from raw distribution. T>1 → more random, creative. Top-p (nucleus): sample from smallest set of tokens covering p% of probability mass. Top-k: sample from k most likely tokens. Production default: temp=0.7, top-p=0.9.

T=0: deterministic · T=1: balanced · T=2: creative/random · top_p=0.9: nucleus sampling

05

Key LLM Models — 2024/25 Landscape

GPT-4o (OpenAI): Multimodal, fast, frontier. Claude 3.5 Sonnet (Anthropic): Best for coding and reasoning. Gemini 1.5 Pro (Google): 1M token context. Llama 3 70B (Meta): Best open-source. Mistral Large: Efficient, European. Phi-3 (Microsoft): Small but punches above weight. Choose by: capability vs cost vs context length vs open/closed.

Frontier: GPT-4o, Claude 3.5 · Open: Llama 3, Mistral · Efficient: Phi-3, Gemma

Context

Context Window

Max tokens model can process at once. GPT-4: 128k. Gemini 1.5: 1M. Longer = more expensive. Most tasks fit in 8k.

Tokens

Tokenisation

Text is split into subword chunks. ~1 token ≈ 0.75 words. Cost and speed are measured per token, not per word.

Halluc.

Hallucination

LLMs generate plausible-sounding but false information. Fix: RAG (ground in real documents), self-consistency, citation. Never blindly trust LLM factual claims.

Embed.

Embeddings

Dense vector representations of text. Semantically similar text has similar vectors. Used in RAG, search, clustering. 768–1536 dimensions typical.

00

What is Prompt Engineering?

Prompt engineering is the art of designing inputs to LLMs to elicit the best possible output. The same model can give wildly different responses depending on how you phrase the request. Good prompts are: specific, contextual, structured, and often include examples. This is a high-leverage skill — small improvements compound across thousands of uses.

Same model + different prompt = dramatically different output quality

Bad vs Good Prompts

Bad Prompt

"Tell me about machine learning."

Too vague. No audience, no format, no depth specified. Model has no idea what kind of answer to give.

Good Prompt

"Explain machine learning to a software engineer who has strong Python skills but no ML background. Use 3 concrete real-world examples from e-commerce. Output a structured explanation with: (1) what it is, (2) how it works, (3) when to use it vs traditional programming."

Specifies: audience, background, examples domain, format, and scope. The model knows exactly what to produce.

Core Techniques

01

Zero-Shot Prompting

Ask the model to perform a task without any examples. Works well for simple tasks where the model has seen enough training data. Baseline starting point — try this first.

"Classify the sentiment of this review as positive, negative, or neutral: [review text]"

02

Few-Shot Prompting

Provide 2–5 input/output examples in the prompt before the actual task. Dramatically improves accuracy on formatting, style, and classification tasks. The model learns the pattern from your examples without any training.

Example 1: "I love this!" → Positive
Example 2: "Terrible service." → Negative
Now classify: "It was okay I guess." → [model completes]

03

Chain of Thought (CoT)

Instruct the model to reason step-by-step before giving a final answer. Significantly improves accuracy on maths, logic, and multi-step reasoning problems. Add "Let's think step by step" or "Reason through this carefully before answering."

"Solve this problem. Think step by step, showing each reasoning step before giving the final answer."

04

Role Prompting & System Prompts

Assign a persona or role to shape the model's behaviour and style. System prompt (in API) sets persistent context for the entire conversation. Role prompts narrow the model's focus and vocabulary to the domain.

System: "You are a senior Python engineer. Be concise, use type hints, explain tradeoffs."
User: "Review this function for production readiness."

05

Output Format Specification

Explicitly specify the format you want: JSON, Markdown table, numbered list, code block, specific length. LLMs are highly responsive to format instructions. For programmatic use, always request JSON with a defined schema.

"Return a JSON object with keys: summary (string), sentiment (positive|negative|neutral), confidence (0.0–1.0)"

06

Self-Consistency & Verification

For high-stakes tasks, generate multiple responses and take the majority answer. Or ask the model to verify its own output: "Check your answer for correctness." Combines chain-of-thought with ensemble thinking. Reduces hallucination on factual tasks.

"Answer this 5 times independently, then give the most consistent answer with your confidence level."

Production tip: Always version your prompts like code. Small prompt changes can cause large output shifts. Test prompts against a representative dataset of 50–100 inputs before deploying. Use prompt templates with variables, not hardcoded strings.

00

What is RAG?

Retrieval Augmented Generation = give an LLM access to external knowledge at inference time. Instead of relying solely on what the model memorised during training, RAG retrieves relevant documents from a knowledge base and injects them into the prompt. Fixes hallucination, adds up-to-date knowledge, and lets LLMs work with private/proprietary data without retraining.

RAG = LLM + external retrieval · no retraining needed · fixes hallucination · enables private data

01

Why RAG Instead of Fine-Tuning?

Fine-tuning is expensive (GPU hours), slow (days), needs labelled data, and the model can still hallucinate on facts it wasn't trained on. Knowledge goes stale. RAG is cheap (vector search), fast (milliseconds), requires no labels, always up-to-date, and you can inspect exactly which sources were retrieved. Use fine-tuning for style/format/behaviour; use RAG for knowledge.

RAG: cheap + fast + always fresh · Fine-tune: style + behaviour + custom tasks

RAG Pipeline

Documents

PDFs, wikis, docs

→

Chunk

Split into ~500 token chunks

→

Embed

text-embedding-3 or BGE

→

Store

Pinecone / Chroma / FAISS

→

Retrieve

Top-k similar chunks

→

Generate

LLM + context → answer

RAG Code Example

python — Basic RAG Pipeline with LangChain

from langchain.document_loaders import PyPDFLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.embeddings import OpenAIEmbeddings
from langchain.vectorstores import Chroma
from langchain.chains import RetrievalQA
from langchain.chat_models import ChatOpenAI

# 1. Load + chunk documents
loader = PyPDFLoader("company_docs.pdf")
splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=50)
chunks = splitter.split_documents(loader.load())

# 2. Embed + store in vector DB
embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
vectorstore = Chroma.from_documents(chunks, embeddings)

# 3. Build retrieval QA chain
qa_chain = RetrievalQA.from_chain_type(
    llm=ChatOpenAI(model="gpt-4o", temperature=0),
    retriever=vectorstore.as_retriever(search_kwargs={"k": 4}),
    return_source_documents=True
)

result = qa_chain("What is our refund policy?")
print(result["result"])
print("Sources:", [d.metadata["source"] for d in result["source_documents"]])

RAG best practices: Chunk size 300–600 tokens with 10–15% overlap. Use a reranker (Cohere rerank, BGE reranker) after retrieval to improve top-k relevance. Always show sources to users so they can verify. Hybrid search (vector + BM25 keyword) consistently beats pure vector search.

00

What is Fine-Tuning?

Fine-tuning = continue training a pretrained model on a smaller, task-specific dataset to adapt its behaviour, style, or knowledge. The model already knows language from pretraining — fine-tuning specialises it. Cheaper than training from scratch (days not months), but more expensive than prompting. Use when: prompting can't get the style right, you need consistent structured outputs, or you have proprietary task-specific data.

Pretrained model + your data → fine-tuned model · specialised behaviour + style

01

Full Fine-Tuning vs PEFT

Full fine-tuning: Update all model parameters. Best quality but needs the same GPU memory as training the model from scratch (GPT-3 = 8×A100 for weeks). PEFT (Parameter-Efficient Fine-Tuning): Freeze most parameters, only train a tiny fraction. LoRA is the standard approach — adds small rank-decomposition matrices to attention layers. Trains 0.1–1% of parameters, near-identical results, runs on a single consumer GPU.

Full FT: all params, expensive · LoRA: 0.1% params, 1 GPU, ~same quality

02

LoRA — Low-Rank Adaptation

Instead of updating weight matrix W directly, LoRA adds two small matrices A (d×r) and B (r×k) where r ≪ d. During training, only A and B are updated. At inference, the product AB is merged back into W — zero added latency. QLoRA (Quantized LoRA) uses 4-bit quantisation to further reduce memory — fine-tune 70B model on a single 48GB GPU.

W' = W + AB · r=8 or r=16 typical · QLoRA: 4-bit quantise base + LoRA adapters

03

When to Fine-Tune vs RAG vs Prompting

Prompting first: Always. Cheap, instant, reversible. Covers 80% of use cases. RAG: When the model needs access to private/fresh knowledge it doesn't have. Can't hallucinate about documents it can see. Fine-Tuning: When you need: consistent output format, domain-specific style, latency reduction (shorter prompts), or cost reduction (smaller model matching large model quality).

Prompt → RAG → Fine-tune · try in this order · each step costs more

04

Instruction Fine-Tuning Data Format

Standard format: {instruction, input (optional), output} triplets. Need 500–10,000 high-quality examples for most tasks (quality beats quantity). Use Alpaca format or ChatML format. Data generation tip: use GPT-4 to generate diverse instruction-output pairs from your domain, then manually review a sample.

{"instruction": "Summarize...", "input": "text...", "output": "summary..."} × 1000+

Fine-tuning myths: (1) "I need 1M examples" — No, 500–2000 high-quality examples beat 100k noisy ones. (2) "Fine-tuning adds new knowledge" — Mostly no. Use RAG for knowledge. Fine-tune for behaviour and style. (3) "Fine-tuned models don't hallucinate" — They still hallucinate, especially on topics not in fine-tuning data.

00

What is an AI Agent?

An AI agent is an LLM augmented with tools and a reasoning loop that lets it take multi-step actions to complete a goal. Instead of responding once, an agent: reasons → chooses a tool → executes it → observes output → reasons again → repeats until the goal is achieved. The LLM acts as the brain; tools are the hands.

Agent = LLM + Tools + Memory + Loop · Reason → Act → Observe → Repeat

01

ReAct — Reasoning + Acting

The dominant agentic pattern: model interleaves Thought (reasoning about what to do) and Action (calling a tool). Observation from tool → next Thought → next Action. Transparent — you can see the reasoning chain. Used in LangChain, AutoGPT, Claude Projects, GPT-4 plugins.

Thought: "I need to search for current price" → Action: search("AAPL stock price") → Observation: "$182.04" → Thought: "Now I can answer..."

02

Tool Types

Retrieval: web search, vector DB search, document lookup. Execution: Python code interpreter, shell commands, API calls. Memory: short-term (conversation history), long-term (vector store of past interactions). External services: email, calendar, Slack, databases, browser automation. The agent uses function calling (JSON schema) to invoke tools precisely.

Tools: [search, code_exec, email, calendar, db_query] · agent picks based on task

03

Multi-Agent Systems

Complex tasks split across specialised sub-agents: Orchestrator agent decomposes task and coordinates. Specialist agents handle subtasks (research agent, code agent, writing agent). Critic/reviewer agent checks outputs. Enables parallelism and specialisation. Frameworks: AutoGen, CrewAI, LangGraph, Semantic Kernel.

Orchestrator → [Research Agent | Code Agent | Writer Agent] → Critic → Final output

04

Challenges & Limitations

Reliability: Agents can get stuck in loops or make wrong tool choices. Cost: Multi-step reasoning = many LLM calls = high cost. Latency: Sequential tool calls add up. Safety: Agents with write access (email, code execution) need human checkpoints. Current best practice: always have a human-in-the-loop for consequential actions.

Production agents: max_iterations=10, human approval for irreversible actions, full audit log

Agentic stack in 2025: LLM backbone (Claude 3.5 or GPT-4o) + function calling + LangGraph or LangChain for orchestration + vector store for memory + code interpreter + web search. Start simple — one tool at a time — before building multi-agent systems.

OpenAI API

GPT-4o · DALL·E · Whisper

Access GPT-4o, embeddings, image generation, speech-to-text. Pay per token. Best for: prototyping, production apps, highest capability ceiling.

platform.openai.com · ~$5–15 per 1M tokens

Hugging Face

500k+ Open Models

Hub for open-source models: Llama, Mistral, Stable Diffusion, Whisper. Free Inference API for testing. Deploy your own. The GitHub of AI.

huggingface.co · transformers library · free tier

LangChain

LLM Framework · RAG · Agents

Framework for chaining LLM calls, building RAG pipelines, and creating agents. Large ecosystem. LangGraph for stateful multi-agent workflows.

pip install langchain · python.langchain.com

Vector DBs

Pinecone · Chroma · FAISS

Store and search dense embeddings for RAG. Pinecone: managed, production-ready. Chroma: local/open-source, great for dev. FAISS: Meta's library, ultra-fast, in-memory.

pinecone.io · chromadb.dev · facebook/faiss

Ollama

Local LLMs · Privacy

Run open-source LLMs (Llama 3, Mistral, Phi-3) locally on your Mac/PC. Zero cost, full privacy, no API calls. Perfect for dev/testing and sensitive data.

ollama.ai · ollama run llama3 · 8GB RAM min

Quickstart Code

python — OpenAI API with structured output

from openai import OpenAI
from pydantic import BaseModel

client = OpenAI()  # uses OPENAI_API_KEY env var

class Review(BaseModel):
    sentiment: str  # "positive" | "negative" | "neutral"
    score: int      # 1–10
    summary: str

response = client.beta.chat.completions.parse(
    model="gpt-4o-mini",
    messages=[
        {"role": "system", "content": "Analyse product reviews."},
        {"role": "user",   "content": "The battery life is incredible!"}
    ],
    response_format=Review,
)
review = response.choices[0].message.parsed
print(review.sentiment, review.score, review.summary)

python — Hugging Face local inference

from transformers import pipeline

# Text generation with Phi-3 (small, fast)
gen = pipeline("text-generation", model="microsoft/Phi-3-mini-4k-instruct",
               device_map="auto", torch_dtype="auto")

result = gen([{"role": "user", "content": "Explain RAG in 2 sentences."}],
             max_new_tokens=200, do_sample=False)
print(result[0]["generated_text"][-1]["content"])

The most common GenAI interview questions at AI startups, big tech, and ML engineering roles in 2024–25.

Q1

RAG vs Fine-tuning — when do you use each?

RAG: Use when the model needs external, private, or up-to-date knowledge. Cheaper, faster to update, no GPU needed. Always try RAG first. Fine-tuning: Use for consistent output format, domain-specific style or vocabulary, or reducing inference cost by shrinking context. Requires labelled data, GPU compute, and retraining when data changes. Best answer: start with prompting → add RAG → fine-tune only if both still fall short.

Knowledge → RAG · Style/Format → Fine-tune · Both failing → RAG + Fine-tune together

Q2

What is hallucination and how do you reduce it?

Hallucination = LLM generates confident, plausible-sounding but factually incorrect output. Causes: model interpolates between training patterns, no mechanism to know what it doesn't know. Mitigations: RAG (ground in real documents), temperature=0 for factual tasks, self-consistency (sample multiple times + vote), citation requirements ("only use information from the provided context"), structured output schemas that constrain response.

RAG + temp=0 + citation requirements + self-consistency = best hallucination defence

Q3

Explain how diffusion models generate images.

Forward process: mathematically add Gaussian noise to a training image over T steps until it's pure noise. Reverse process: train a U-Net neural network to predict the noise ε added at each step. At inference: start from pure Gaussian noise, iteratively apply the denoising network T times. Condition on text using CLIP embeddings + cross-attention — this is how "a sunset over mountains" guides the denoising. Classifier-Free Guidance (CFG) amplifies the conditioning signal.

x_T (noise) → denoise T times with U-Net → x_0 (image) · text conditions each step

Q4

What is RLHF and why is it important?

RLHF (Reinforcement Learning from Human Feedback): (1) collect human preference data — show 2 model outputs, ask which is better; (2) train a reward model that predicts human preference scores; (3) use PPO (RL algorithm) to fine-tune the LLM to maximise reward model score. Transforms a next-token predictor into a helpful assistant that follows instructions, refuses harmful requests, and produces human-preferred outputs. Used in GPT-4, Claude, Gemini. DPO is a simpler alternative that skips the RL step.

SFT base → collect preferences → train reward model → PPO optimise → aligned LLM

Q5

How does LoRA work?

LoRA freezes the original weight matrix W and adds two trainable rank-decomposition matrices: W' = W + BA where B is d×r and A is r×k, with r ≪ min(d,k). Only A and B are trained — typically 0.1–1% of total parameters. At inference, BA is merged into W with zero latency overhead. QLoRA also quantises W to 4-bit, reducing memory ~4× while maintaining quality. Enables fine-tuning 70B models on a single 48GB A100.

W' = W + BA · r=8–64 typical · QLoRA: 4-bit W + LoRA adapters · ~1 GPU

Q6

How would you build a production RAG system?

Ingestion: parse docs → chunk (500 tokens, 50 overlap) → embed (text-embedding-3-small) → store in Pinecone. Retrieval: embed query → top-8 chunks → rerank (Cohere) → top-4. Generation: inject chunks into system prompt → GPT-4o at temp=0 → require source citations. Evaluation: RAGAS metrics — context relevance, faithfulness, answer relevance. Monitoring: log queries + retrieved chunks + user feedback. Iterate on chunking and reranking based on failure cases.

Parse → Chunk → Embed → Store → Retrieve → Rerank → Generate → Cite → Eval → Iterate

Q7

What are embeddings and how are they used in GenAI?

Embeddings are dense vector representations of text (or images/audio) where semantically similar content has similar vectors. "King" − "Man" + "Woman" ≈ "Queen" (famous example). Used in: RAG (embed docs + queries, retrieve by cosine similarity), semantic search, clustering/classification without labels, recommendation systems, and as the shared representation in multimodal models (CLIP embeds both images and text into the same space).

cosine_similarity(embed("cat"), embed("dog")) > cosine_similarity(embed("cat"), embed("car"))

Everything you need about Generative AI on one page — model selection, prompting rules, key terms, and the decision framework.

Task → Model Selection

Task

Best Model

Open Alternative

Tip

Text / Reasoning

GPT-4o / Claude 3.5

Llama 3 70B

temp=0 for facts

Code generation

Claude 3.5 Sonnet

DeepSeek Coder

Always test output

Image generation

DALL·E 3 / Midjourney

Stable Diffusion XL

CFG scale 7–12

Text embeddings

text-embedding-3-small

BGE-large / E5

1536 dims standard

Speech → Text

Whisper large-v3

Whisper (open)

free + accurate

Long context

Gemini 1.5 Pro

Llama 3.1 128k

1M token context

Local / private

Ollama + Llama 3

Phi-3 (3.8B)

8GB RAM minimum

Prompting Quick Rules

Technique

When to Use

Key Tip

Zero-shot

Simple, well-defined tasks

Start here always

Few-shot

Formatting, classification

2–5 examples, diverse

Chain of Thought

Maths, reasoning, logic

"Think step by step"

Role / System

Consistent tone/expertise

Set in system prompt

Output format

Structured data (JSON)

Define schema explicitly

Self-consistency

High-stakes factual tasks

Sample 5×, majority vote

Key Terms at a Glance

Temperature

T=0: deterministic · T=1: balanced · T>1: creative

Use T=0 for factual tasks, T=0.7–1.0 for creative

Context Window

Max tokens = input + output combined

GPT-4o: 128k · Gemini 1.5: 1M · ~1 token ≈ 0.75 words

RAG Formula

Answer = LLM(query + retrieved_chunks)

Chunk 300–600 tokens · top-k=4–8 · always rerank

LoRA

W' = W + BA · rank r ≪ d

Trains 0.1% of params · merge at inference · zero latency

Diffusion CFG

ε̂ = (1+w)·ε_cond − w·ε_uncond

w=7.5 default · higher = more prompt adherence

Embedding Similarity

cos(A,B) = A·B / (|A|·|B|)

1.0 = identical · 0 = orthogonal · -1 = opposite

Decision Framework: What to Build?

Step 1

Can prompting alone solve it?

Always try a well-crafted prompt with few-shot examples first. Free, instant, requires no infrastructure. Covers ~70% of GenAI use cases.

If yes → ship it. If quality insufficient → Step 2.

Step 2

Does it need external knowledge?

If the model needs private data, up-to-date facts, or domain-specific documents it wasn't trained on → add RAG. Build in days, not weeks.

If yes → build RAG pipeline. If style/format still wrong → Step 3.

Step 3

Does it need consistent style or lower latency/cost?

Fine-tune with LoRA/QLoRA. 1000+ high-quality examples. Smaller fine-tuned model can match large model quality for your specific task at fraction of cost.

Fine-tune Llama 3 8B → matches GPT-4 for your task → 10× cheaper inference.

3 rules to never break: (1) Always evaluate with real users, not just eyeballing outputs. (2) Never use LLM outputs for high-stakes decisions without human review. (3) Never store sensitive data in prompts sent to third-party APIs — use local models or on-premise deployments for PII/PHI.