Computer Vision — TechForge

91%ImageNet Top-1 (ViT-G)

200+FPS (YOLOv8-nano)

1B+Masks (SAM training)

2012AlexNet revolution

00

What is Computer Vision?

Computer Vision (CV) is the field of AI that enables machines to interpret and understand visual data — images and video. The goal: go from raw pixels to meaning. A human glances at a scene and instantly knows "there's a cat on the sofa." CV systems replicate this by learning statistical patterns from millions of labelled images.

Pixels → Features (edges/textures/shapes) → Semantics (objects/scenes) → Actions

01

Images are Matrices of Numbers

A digital image is a 3D array of pixel values. Grayscale: [H × W × 1], each pixel 0–255. RGB: [H × W × 3], three channels for Red, Green, Blue. A 224×224 RGB image = 150,528 individual numbers. Before training, images are resized to a uniform dimension and normalised (scaled to 0–1 or standardised to mean=0, std=1 — critical for stable gradient flow).

224×224 RGB: shape [224, 224, 3] = 150,528 values · normalise: x/255 or (x-μ)/σ

02

The Five Core CV Tasks

Classification: one label per image ("cat"). Detection: locate + classify each object with a bounding box. Segmentation: classify every pixel (semantic) or separate each instance (instance). Keypoint: locate body joints, facial landmarks. Depth: estimate distance of each pixel from camera. Each task is progressively harder and provides richer spatial understanding.

Classify → Detect → Segment → Keypoints → Depth (increasing spatial detail)

03

The CV Revolution: AlexNet → ViT

Pre-2012: hand-crafted features (SIFT, HOG) + SVM. 2012: AlexNet trains a deep CNN on GPUs and wins ImageNet by 10 percentage points — the deep learning era begins. 2015: ResNet reaches human-level accuracy on ImageNet. 2020+: Vision Transformers (ViT) apply self-attention to image patches — beating CNNs at scale with enough data.

HOG+SVM (2011) → AlexNet (2012) → ResNet (2015) → EfficientNet (2019) → ViT (2020)

Face Recognition

Identify or verify identity from facial images. Used in phone unlock, security systems, border control.

Model: ArcFace / FaceNet · Metric: LFW accuracy

Autonomous Driving

Detect lanes, vehicles, pedestrians, traffic signs in real time. Fuse camera, LiDAR, radar. Tesla Autopilot, Waymo.

Model: YOLOv8 + BEV · Metric: mAP, NDS

Medical Imaging

Detect tumours in X-rays/CT/MRI, segment organs, classify skin lesions. Reaches radiologist-level on specific tasks.

Model: U-Net, ViT · Metric: Dice score, AUC

Industrial Inspection

Detect manufacturing defects, surface anomalies, component misalignments. Faster and more consistent than human inspection.

Model: Anomaly detection (PatchCore, FastFlow)

00

Why CNNs for Images?

A fully-connected network applied to a 224×224 image would need 150,528 input neurons — computationally explosive and prone to overfitting. CNNs exploit two key properties of images: Local connectivity (nearby pixels are related) and Translation invariance (a cat is a cat regardless of where it appears). Convolution filters share weights across the entire image — a dramatic reduction in parameters.

FC on 224×224: ~150k×128 = 19M params in first layer alone · CNN: 3×3 kernel = 9 params shared

01

The Convolution Operation

A filter (kernel) of size K×K slides across the input with stride S, computing a dot product at each position to produce a feature map. One kernel detects one type of pattern (e.g., vertical edges). Use 64 kernels → 64 feature maps, each detecting a different low-level pattern. Key formula: output size = (W − K + 2P) / S + 1.

out[i,j] = Σₘ Σₙ kernel[m,n] · input[i+m, j+n] · output size = (W-K+2P)/S + 1

02

Padding & Stride

Padding='same': add zeros around the border to preserve spatial dimensions (H×W unchanged). Padding='valid': no padding — output shrinks by K-1 pixels per side. Stride=2: skip every other position — halves spatial dimensions more efficiently than pooling, with learnable downsampling. Standard pattern: stride=2 at block boundaries instead of MaxPool.

same: output W = W · valid: output W = W-K+1 · stride=2: output W = ⌊W/2⌋

03

Pooling — Spatial Downsampling

MaxPool2d(2,2): take the maximum in each 2×2 window — halves H and W, retains the strongest feature activation. Provides mild translation invariance. AvgPool: takes mean — smoother. Global Average Pooling (GAP): collapses each feature map to a single number — replaces Flatten+Dense, dramatically reduces parameters and acts as a regulariser. GAP is the modern standard.

MaxPool(2,2): [B,C,H,W] → [B,C,H/2,W/2] · GAP: [B,C,H,W] → [B,C,1,1]

04

Feature Hierarchy — What CNNs Learn

CNNs learn a hierarchical decomposition of visual structure: Early layers detect edges, gradients, colour blobs (universal, dataset-agnostic — these features transfer!). Middle layers detect textures, patterns, object parts. Deep layers detect high-level semantics — faces, wheels, text. This hierarchy is why ImageNet pretrained features transfer so well to other vision tasks.

Layer 1: edges · Layer 3: textures · Layer 7: parts · Layer 12: objects · Last: semantics

05

Batch Normalisation in CNNs

Apply after Conv, before ReLU. Normalises across the batch per channel. Dramatically stabilises training, allows higher learning rates, reduces sensitivity to weight initialisation. In modern CNNs: [Conv → BN → ReLU → (Pool)] repeated N times → GAP → Linear. Replace BN with LayerNorm for very small batches or ViTs.

[Conv → BN → ReLU] × N → GlobalAvgPool → Dropout(0.2) → Linear(num_classes)

01

Image Classification

Assign one (or multiple) label(s) to an entire image. The ImageNet Large Scale Visual Recognition Challenge (ILSVRC) — 1.2M images, 1000 classes — drove a decade of CV progress. Top-1 accuracy jumped from 72% (2010, hand-crafted) to 91%+ (2022, ViT-G/14) through deep learning advances.

f(image) → [p_cat=0.92, p_dog=0.05, p_bird=0.02, ...] · argmax → predicted class

02

Data Augmentation — Essential for CV

Randomly transform training images to create diversity and prevent overfitting. Standard: random horizontal flip, random crop, colour jitter (brightness/contrast/saturation). Advanced: MixUp (blend two images + labels), CutMix (paste patch from one image to another), RandAugment (random policy search). Can be worth 2–5% accuracy, equivalent to 2× the training data.

transforms.Compose([RandomHFlip(p=0.5), RandomCrop(224), ColorJitter(0.4,0.4,0.4,0.1)])

03

Transfer Learning — The Production Pattern

Almost never train from scratch. Load ImageNet pretrained weights — the model has already learned universal visual features (edges → textures → objects). Strategy: (1) Freeze backbone, add new classification head, train head (lr=1e-3). (2) Unfreeze all, fine-tune at low lr (1e-5). Works with as few as 100 images per class. Use timm for the best pretrained models.

model = timm.create_model('efficientnet_b4', pretrained=True, num_classes=YOUR_N)

04

Multi-Label vs Multi-Class Classification

Multi-class: exactly one correct label per image (cat OR dog OR bird). Use Softmax + CrossEntropy. Multi-label: multiple labels can be true simultaneously (image has both a cat AND a sofa). Use Sigmoid + BinaryCrossEntropy per class. Instagram photo tagging, product attributes, chest X-ray pathologies are all multi-label tasks.

Multi-class: softmax, CE · Multi-label: sigmoid, BCE per class · chest X-ray: multi-label

Classification Model Comparison

Model	Top-1 (ImageNet)	Params	Speed	Best For
MobileNetV3-S	67.4%	2.5M	Very fast	Mobile / edge deployment
ResNet-50	80.9%	25M	Fast	General backbone, widely supported
EfficientNet-B4	83.8%	19M	Fast	Accuracy/efficiency sweet spot
ConvNeXt-B	85.8%	89M	Medium	Strong CNN baseline, no ViT complexity
ViT-B/16	85.5%	86M	Medium	Best with large data; Hugging Face friendly
ViT-G/14	90.9%	1.8B	Slow	Maximum accuracy, research

00

The Detection Task

For each object in an image: predict class label + bounding box [x_min, y_min, x_max, y_max] + confidence score. Multiple objects, different scales, partial occlusions. Primary metric: mAP (mean Average Precision) — averaged over classes and IoU thresholds. COCO benchmark: 80 classes, 330k images, the standard for detection research.

output: [(class, x1,y1,x2,y2, conf), ...] · mAP@[0.5:0.95] = COCO standard metric

01

IoU — Intersection over Union

The fundamental metric for bounding box quality. IoU = area of overlap / area of union between predicted and ground-truth boxes. IoU=1 is perfect; IoU=0 means no overlap. Used in: (1) defining true positives (IoU ≥ 0.5 threshold). (2) NMS — removing duplicate detections. (3) Training loss (DIoU, CIoU improve convergence).

IoU = |pred ∩ gt| / |pred ∪ gt| · TP: IoU≥0.5 · NMS threshold: keep if IoU<0.45

02

Two-Stage Detectors — R-CNN Family

Faster R-CNN: Region Proposal Network (RPN) generates candidate boxes → ROI Align extracts fixed features → classifier + box regressor. High accuracy (mAP ~55+ on COCO) but slower (~5fps). Best for accuracy-critical applications where speed is secondary (medical, satellite imagery). Torchvision includes production-ready Faster R-CNN.

Image → Backbone → FPN → RPN → ROI Align → Head → (class, box) · ~5fps

03

YOLO — The Real-Time Standard

Single-stage: divide image into a grid; predict bounding boxes + class probabilities for each cell in one forward pass. No region proposal step. YOLOv8 (Ultralytics, 2023): 37.3 mAP at 200fps (nano), 53.9 mAP at 87fps (large) on COCO. The default choice for production real-time detection. Tiny API, easy to fine-tune on custom data.

from ultralytics import YOLO · YOLO('yolov8n.pt') · train + inference in <10 lines

04

Feature Pyramid Network (FPN)

Multi-scale detection: small objects need high-resolution features; large objects need large receptive fields. FPN builds a pyramid of feature maps by combining high-level semantic features (top-down) with high-resolution spatial features (bottom-up). All modern detectors (YOLO, Faster R-CNN, DETR) use FPN or a variant. Critical for detecting objects at multiple scales.

P3 (small objects) + P4 + P5 (large objects) → predict at all scales → NMS to merge

05

DETR — Detection Transformer

Facebook AI (2020). Frames detection as a set-prediction problem using Transformers. No anchors, no NMS — outputs exactly N (100) box predictions directly. Hungarian matching assigns predictions to ground truths. Elegant but slow to train (500 epochs vs 12 for Faster R-CNN). RT-DETR (2023) achieves real-time speeds with Transformer quality.

CNN backbone → Transformer encoder/decoder → N object queries → (class, box) pairs

00

Three Types of Segmentation

Semantic: classify every pixel into a category. Two people = same "person" mask. Road, sky, building each get their own colour. Instance: separate mask per object instance. Two people = two distinct masks, even when overlapping. Panoptic: combines both — semantic for "stuff" (road, sky) and instance for "things" (people, cars). Metric: IoU per class (semantic), mAP (instance).

Semantic: pixel → class · Instance: pixel → (class, instance_id) · Panoptic: both

01

U-Net — The Segmentation Workhorse

Encoder-decoder with skip connections. Encoder: conv blocks + max pooling — extract features, shrink spatial dims. Decoder: transposed convolutions upsample back to original resolution. Skip connections concatenate encoder features to corresponding decoder layers — preserve fine spatial details lost during downsampling. Essential for medical imaging segmentation. Simple, fast, works with small datasets.

[Conv→Pool]×4 → bottleneck → [Upsample+Skip→Conv]×4 → 1×1 conv → mask

02

Mask R-CNN — Instance Segmentation

Extends Faster R-CNN by adding a parallel mask prediction branch. After ROI Align: (1) box regression + classification (from Faster R-CNN), (2) a small FCN predicts a binary mask for each ROI. Train with multi-task loss: classification + box regression + binary mask cross-entropy. Accurate but slower — ~5fps. The basis for most instance segmentation systems.

ROI Align → [class head | box head | mask FCN] · loss = L_cls + L_box + L_mask

03

SAM — Segment Anything Model

Meta AI (2023). Foundation model for segmentation. Trained on 1B+ mask annotations from SA-1B dataset. Accepts prompts: points, bounding boxes, or text — and segments the corresponding object in any image. Zero-shot generalisation across domains. SAM 2 (2024) extends to video. Paradigm shift — from training per-dataset models to universal promptable segmentation.

predictor.predict(point_coords=[[x,y]], point_labels=[1]) → masks, scores, logits

04

Evaluation: Dice Score & IoU

IoU (Jaccard): intersection / union per class. Average across classes = mIoU — standard for semantic segmentation. Dice score: 2×|A∩B| / (|A|+|B|). More sensitive to small regions. Standard for medical segmentation (lung nodules, tumours). Both range 0–1; higher is better. Dice = 2×IoU / (1+IoU) — mathematically related.

IoU = |A∩B|/|A∪B| · Dice = 2|A∩B|/(|A|+|B|) · mIoU = mean over all classes

1998

LeNet-5

First practical CNN for handwritten digit recognition (MNIST). 5 layers, ~60k parameters. Established the [Conv→Pool]×N pattern still used today.

Pioneer

2012

AlexNet

8 layers, 60M params, trained on 2 GPUs. Won ImageNet by 10.9% — the moment that launched the deep learning era. Introduced: ReLU activations, dropout, data augmentation, GPU training.

Revolution

2014

VGG-16 / VGG-19

Very deep stacks of 3×3 convolutions. Simple, uniform architecture. Still widely used as a feature extractor baseline. 138M parameters — large but interpretable.

Backbone

2015

ResNet-50 / 101 / 152

Residual (skip) connections: output = F(x) + x. Enables training networks 50–152 layers deep without degradation. Won ImageNet + COCO 2015. ResNet-50 remains the most widely used backbone in production CV.

Gold Standard

2018

MobileNetV2 / V3

Depthwise separable convolutions — 8–9× fewer operations than standard conv. Designed for mobile and edge devices. 2.5M params (MobileNetV3-Small). Used in on-device ML (TFLite, CoreML).

Edge / Mobile

2019

EfficientNet (B0–B7)

Compound scaling: simultaneously scale width, depth, and resolution by a fixed ratio. EfficientNet-B4 (19M params) matches ResNet-152 (60M) accuracy. The accuracy/efficiency Pareto frontier for CNNs.

Efficient

2020

Vision Transformer (ViT)

Split image into 16×16 patches, linearly embed each, treat as a sequence of tokens. Apply standard Transformer. With enough pretraining data (JFT-300M), beats CNNs at every scale. DeiT makes ViT data-efficient for smaller datasets.

Transformer

2022

ConvNeXt

Pure CNN modernised with Transformer design choices: large kernels (7×7), LayerNorm, fewer activations, inverted bottleneck. Matches or beats ViT on ImageNet without the complexity of self-attention. Best of both worlds.

Modern CNN

01

CLIP — Connecting Images and Language

OpenAI (2021). Train an image encoder and a text encoder jointly on 400M (image, caption) pairs using contrastive loss — matching image and caption embeddings. Result: a shared visual-language embedding space. Enables zero-shot classification: compare image embedding to text embeddings of class names ("a photo of a cat"). Foundation for DALL-E, Stable Diffusion text conditioning, and VLMs.

L = -log[exp(sim(Iᵢ,Tᵢ)/τ) / Σⱼ exp(sim(Iᵢ,Tⱼ)/τ)] · contrastive across batch

02

Stable Diffusion — Latent Diffusion

Run diffusion in a compressed latent space (4×64×64) rather than pixel space (3×512×512) — 48× more efficient. VAE encodes image to latent; VAE decoder reconstructs. U-Net denoises in latent space, conditioned on CLIP text embeddings via cross-attention. Classifier-Free Guidance (CFG) amplifies the text conditioning signal. Open-source (Stability AI, 2022).

z = VAE.encode(x) → denoise(z, text_emb, T steps) → x̂ = VAE.decode(z₀)

03

ControlNet — Structured Control

Add conditioning signals to Stable Diffusion beyond text: depth maps, edge maps (Canny), human pose skeletons (OpenPose), semantic segmentation, normal maps. Adds a trainable copy of SD's encoder, conditioned on the control image. Enables precise control: "generate a person in this exact pose, in a photorealistic style." The key enabler for professional creative workflows.

SD(text + canny_edges) → image that follows both prompt and edge structure

04

Video Generation — Sora & Beyond

OpenAI Sora (2024): extends latent diffusion to spacetime — treat video as a sequence of 3D patches. Generates up to 60s photorealistic video. Key insight: represents the world as a 3D physical simulation, not just frame interpolation. Runway Gen-3, Kling, Pika Labs offer accessible video generation APIs. 2025 is becoming the year of practical video AI.

3D patches + spacetime attention → coherent long video · ~30B params estimated

The CLIP → Diffusion stack: CLIP provides the image-text bridge. Diffusion provides generative quality. Every major image generation system (DALL-E 3, Midjourney, SD) uses both. Understanding CLIP is the prerequisite for understanding why text-to-image works at all.

python — YOLOv8 detection + custom fine-tuning

from ultralytics import YOLO

# ── Inference on image / video ──
model = YOLO('yolov8m.pt')          # n/s/m/l/x variants
results = model('street.jpg', conf=0.5, iou=0.45)
for r in results:
    boxes = r.boxes.xyxy.cpu().numpy()   # [x1,y1,x2,y2]
    cls   = r.boxes.cls.cpu().numpy()    # class IDs
    conf  = r.boxes.conf.cpu().numpy()   # confidences
    r.show()                             # display annotated image
    r.save('result.jpg')

# ── Fine-tune on custom data ──
# custom.yaml: path/train, path/val, nc (num classes), names
model.train(data='custom.yaml', epochs=100, imgsz=640,
            batch=16, device=0, workers=8,
            lr0=0.01, weight_decay=0.0005,
            augment=True, mosaic=1.0, mixup=0.1)
metrics = model.val()
print(f"mAP50: {metrics.box.map50:.3f}")

python — Transfer learning (EfficientNet via timm)

import timm, torch, torch.nn as nn
from torchvision import transforms, datasets
from torch.optim import AdamW
from torch.optim.lr_scheduler import CosineAnnealingLR

NUM_CLASSES = 10

# 1. Load pretrained model (no head)
model = timm.create_model('efficientnet_b4', pretrained=True, num_classes=NUM_CLASSES)

# 2. Transforms — use model's recommended config
cfg = timm.data.resolve_model_data_config(model)
train_tf = timm.data.create_transform(**cfg, is_training=True)
val_tf   = timm.data.create_transform(**cfg, is_training=False)

# 3. Dataset + DataLoader
train_ds = datasets.ImageFolder('data/train', transform=train_tf)
val_ds   = datasets.ImageFolder('data/val',   transform=val_tf)
train_dl = torch.utils.data.DataLoader(train_ds, batch_size=32, shuffle=True,  num_workers=4)
val_dl   = torch.utils.data.DataLoader(val_ds,   batch_size=32, shuffle=False, num_workers=4)

# 4. Train
device = 'cuda' if torch.cuda.is_available() else 'cpu'
model  = model.to(device)
opt    = AdamW(model.parameters(), lr=1e-4, weight_decay=1e-4)
sched  = CosineAnnealingLR(opt, T_max=20)
loss_fn = nn.CrossEntropyLoss(label_smoothing=0.1)

for epoch in range(20):
    model.train()
    for x, y in train_dl:
        x, y = x.to(device), y.to(device)
        opt.zero_grad()
        loss_fn(model(x), y).backward()
        nn.utils.clip_grad_norm_(model.parameters(), 1.0)
        opt.step()
    sched.step()

python — U-Net segmentation (medical imaging)

import segmentation_models_pytorch as smp

# U-Net with ResNet-34 encoder, pretrained on ImageNet
model = smp.Unet(
    encoder_name="resnet34", encoder_weights="imagenet",
    in_channels=3, classes=1  # binary segmentation
)

# Losses suited for segmentation
dice_loss = smp.losses.DiceLoss(mode='binary')
bce_loss  = smp.losses.SoftBCEWithLogitsLoss()
loss_fn   = lambda pred, gt: dice_loss(pred, gt) + bce_loss(pred, gt)

# Metrics
tp, fp, fn, tn = smp.metrics.get_stats(preds, masks, mode='binary', threshold=0.5)
iou  = smp.metrics.iou_score(tp, fp, fn, tn, reduction="micro")
dice = smp.metrics.f1_score(tp, fp, fn, tn, reduction="micro")

The most commonly asked Computer Vision interview questions at Google, Meta, Tesla, NVIDIA, and CV-focused ML roles.

Q1

Why do CNNs work so well for images?

Three structural properties: (1) Local connectivity: each neuron connects to a small local region, exploiting the fact that nearby pixels are correlated. (2) Weight sharing: the same filter is applied across all spatial positions — a massive parameter reduction (3×3 kernel = 9 shared weights vs millions for FC). (3) Translation invariance: a cat detector works regardless of where the cat appears — the convolution slides over the whole image. These inductive biases make CNNs extremely data-efficient for visual data.

FC: O(H×W×C) params · CNN: O(K×K×C_in×C_out) shared params — orders of magnitude fewer

Q2

What is IoU and how is it used in object detection?

IoU (Intersection over Union) = area of overlap between predicted and ground-truth box / area of their union. Used in three ways: (1) Defining TPs: a detection is a true positive if its class is correct AND IoU ≥ threshold (typically 0.5). (2) NMS: suppress boxes with IoU > NMS threshold (0.45) with a higher-scoring box. (3) Loss: DIoU, CIoU losses add penalty terms for centre distance and aspect ratio mismatch to improve convergence.

IoU = |A∩B| / |A∪B| · TP: class correct + IoU≥0.5 · NMS: remove if IoU>0.45

Q3

ResNet vs ViT — which do you choose?

ResNet / CNN: better with <100k images (stronger inductive bias), faster inference, lower compute, more deployment options (TFLite, ONNX, CoreML). ViT: better with large data (>1M images, or with strong pretraining like ImageNet-21k or CLIP). Scales better with compute. Better for fine-grained tasks requiring global context. In practice: start with EfficientNet-B4 as a strong baseline; upgrade to ViT-B/16 if you have ImageNet-21k pretrained weights and enough data.

<100k images: EfficientNet · >1M images: ViT · production inference: EfficientNet (faster)

Q4

Explain semantic vs instance vs panoptic segmentation.

Semantic: assign a class to every pixel, no distinction between instances. Two cars = same mask colour. metric: mIoU. Instance: separate mask per object instance — two cars = two distinct masks. Can handle overlapping objects. Metric: mAP. Model: Mask R-CNN. Panoptic: unified: semantic for amorphous "stuff" (road, sky, grass) + instance for countable "things" (cars, people). Metric: PQ (Panoptic Quality). Most complete scene understanding.

Semantic: pixel→class · Instance: pixel→(class,instance_id) · Panoptic: both unified

Q5

How does transfer learning work in CV, and when does it fail?

Pretrained ImageNet models learn universal visual features (edges, textures, shapes) that transfer to almost any visual task. Protocol: freeze backbone → train new head → unfreeze and fine-tune all at low lr. Fails when: (1) Target domain is very different from source (satellite imagery, medical microscopy — retrain with domain-specific pretrained weights like RadImageNet, BigEarthNet). (2) Target classes are extremely fine-grained (species identification — use specialist backbones). (3) Extreme class imbalance or label noise — domain-adaptive pretraining helps.

Freeze → train head (lr=1e-3) → unfreeze → fine-tune (lr=1e-5) · fails: domain mismatch

Q6

What is data augmentation and why is it critical in CV?

Augmentation creates training diversity by applying random transformations (flip, crop, colour jitter, rotation). Without it, CNNs memorise exact training images rather than generalising. In CV, augmentation is often the single highest-impact improvement available — equivalent to 2–5× more data. Advanced: MixUp, CutMix (blend images/labels) and RandAugment (automatic policy search) add 1–3% accuracy. Always apply augmentation to training only — never to validation or test sets.

train: RandomHFlip+Crop+ColorJitter+MixUp · val: Resize+CenterCrop only

Q7

How does CLIP enable zero-shot image classification?

CLIP trains image and text encoders to produce aligned embeddings — matching images and their captions are close in the shared embedding space. For zero-shot classification: (1) For each class, create a text prompt: "a photo of a [class name]." (2) Encode all prompts with the text encoder. (3) Encode the query image with the image encoder. (4) Compute cosine similarity between image embedding and all text embeddings. (5) The highest-similarity class is the prediction — no labelled training data needed for the target task.

score(class) = cos(CLIP_img(image), CLIP_txt("a photo of a {class}")) · argmax = prediction

Complete Computer Vision reference — task → model → metric mapping, architecture guide, and essential formulas.

Task → Model → Metric

Task

Recommended Model

Production Start

Metric

Image Classification

EfficientNet-B4

timm pretrained

Top-1 Accuracy

Large-scale Classification

ViT-B/16

ImageNet-21k weights

Top-1 Accuracy

Real-time Detection

YOLOv8m

ultralytics pretrained

mAP@0.5:0.95

Accurate Detection

Faster R-CNN

torchvision pretrained

mAP@0.5:0.95

Semantic Segmentation

U-Net / DeepLabV3+

smp library

mIoU

Instance Segmentation

Mask R-CNN / YOLO-Seg

torchvision / ultralytics

mAP

Universal Segmentation

SAM / SAM 2

Meta checkpoints

IoU

Mobile / Edge Inference

MobileNetV3 / YOLO-nano

TFLite / ONNX

Latency ms

Architecture Selection Guide

<100k

Small Dataset

EfficientNet or ResNet with heavy augmentation + transfer learning. Never train from scratch. Strong inductive bias of CNNs helps with limited data.

>1M

Large Dataset

ViT-B/16 or ViT-L/16 with ImageNet-21k initialisation. Transformer scales better with data — global attention captures long-range dependencies that CNNs miss.

Edge

Mobile / Real-time

MobileNetV3, EfficientNet-B0, YOLO-nano. Use TFLite or ONNX for deployment. Quantise to INT8 for 4× speedup with <1% accuracy drop.

Med

Medical Imaging

U-Net for segmentation. Use Dice + BCE loss. Pretrain on RadImageNet (not ImageNet). Heavy augmentation with domain-specific transforms (elastic deformation, staining normalisation).

Key Formulas

Conv Output Size

(W − K + 2P) / S + 1

W=input · K=kernel · P=padding · S=stride

IoU

|A ∩ B| / |A ∪ B|

0=no overlap · 1=perfect · TP threshold: 0.5

Dice Score

2|A ∩ B| / (|A| + |B|)

Standard for medical segmentation · Dice = 2IoU/(1+IoU)

Average Precision

AP = ∫ P(R) dR

Area under P-R curve · mAP = mean over classes

CLIP Contrastive Loss

-log[sim(Iᵢ,Tᵢ) / Σⱼ sim(Iᵢ,Tⱼ)]

Align image + text embedding space · cosine similarity

CFG (Diffusion)

ε̂ = (1+w)·ε_cond − w·ε_uncond

w = guidance scale · 7.5 default for SD

3 rules to never break in CV: (1) Augmentation on training only — never apply random transforms to val/test. (2) Always normalise inputs with ImageNet statistics (mean=[0.485,0.456,0.406], std=[0.229,0.224,0.225]) when using pretrained models. (3) Use mAP, not accuracy, for detection and segmentation evaluation — accuracy is meaningless for these tasks.