Machine Learning — TechForge

4Learning Types

12+Algorithms

80%Is Data Prep

∞Applications

00

What is Machine Learning?

Machine Learning is a subset of AI where systems learn patterns from data rather than following hand-coded rules. Give the machine labelled examples, let it figure out the rules itself. ML powers most AI products today.

Data + Algorithm → Trained Model → Predictions on new data

01

Why ML Beats Rule-Based Systems

Rule-based: You write every if-else manually. Breaks on edge cases. Never improves. Machine Learning: You provide examples, the model finds patterns. Gets better with more data. Scales automatically to millions of cases.

Rules: 500+ if-else blocks → fails on edge cases | ML: one model → generalises

02

The Four Types of Machine Learning

Supervised: Labelled data (X → y). Unsupervised: No labels — find hidden structure. Semi-Supervised: Mix of labelled and unlabelled. Reinforcement Learning: Agent learns from environment rewards. Most production ML is supervised.

Supervised | Unsupervised | Semi-Supervised | Reinforcement Learning

03

The Bias–Variance Tradeoff

Underfitting (high bias): Model too simple — misses real patterns. Poor on train and test. Overfitting (high variance): Memorises training data, fails on new data. Goal: the sweet spot that generalises.

Total Error = Bias² + Variance + Irreducible Noise

04

Features, Labels, and Weights

Features (X): Input variables — pixel values, age, income, word count. Labels (y): The target output to predict. Weights: Learned parameters mapping X → y. Training = finding weights that minimise prediction error.

ŷ = f(X; weights) · training: minimise loss(ŷ, y) over all examples

05

Real-World ML Applications

Spam detection · House price prediction · Credit card fraud · Netflix recommendations · Google Search ranking · Medical diagnosis · Autonomous driving · YouTube autoplay · Amazon "also bought".

If a product makes predictions at scale — it’s almost certainly ML

Regression

Continuous Output

Predict a number: house price, temperature, sales revenue

Classification

Discrete Output

Predict a category: spam/not-spam, cat/dog, fraud/legit

Clustering

Group Discovery

Find natural groups: customer segments, topic clusters

Anomaly

Outlier Detection

Find unusual points: fraud transactions, sensor faults

💡

Golden rule: Always start with the simplest model (logistic regression, decision tree). Only add complexity when simpler models fall short. A well-engineered feature often beats a more complex model.

00

Supervised Learning

You provide both inputs (X) and correct outputs (y) during training. The model supervises itself against known answers, adjusting until its predictions match. Every prediction task where you have labelled historical data is supervised.

Training: {(x₁,y₁), (x₂,y₂), ..., (xₙ,yₙ)} → Model → Predict ŷ on new x

01

Linear Regression

Fits a line through labelled data to predict a continuous value. Minimises Mean Squared Error. Highly interpretable — each weight is the feature’s marginal contribution. Foundation of all linear models.

ŷ = w₀ + w₁x₁ + w₂x₂ + ... · Loss = MSE = (1/n)Σ(yᵢ - ŷᵢ)²

02

Logistic Regression

Binary or multiclass classification using the sigmoid function. Despite the name, it’s a classifier. Outputs probability (0–1). Fast, interpretable baseline — always run this first before complex models.

σ(z) = 1/(1+e⁻ᵣ) · Loss = Binary Cross-Entropy · Use cases: fraud, churn, spam

03

Decision Trees

Splits data on feature thresholds — tree of if-else rules. Every split maximises Information Gain or minimises Gini Impurity. Interpretable but prone to overfitting. Solved by ensembles (Random Forest, XGBoost).

Split by feature threshold: max(Info Gain) or min(Gini) at each node

04

Random Forest

Ensemble of N decision trees, each trained on a random data and feature subset. Prediction = majority vote or mean. Dramatically reduces overfitting. Excellent out-of-the-box performance on tabular data.

ŷ = mode/mean(tree₁(x), tree₂(x), ..., treeₙ(x)) · n_estimators=100–500

05

Gradient Boosting — XGBoost / LightGBM

Builds trees sequentially, each correcting residual errors of the previous. The most powerful algorithm for tabular data. Wins Kaggle. Hyperparameter-sensitive — tune carefully with cross-validation.

F(x) = f₁(x) + α·f₂(residuals) + α·f₃(...) · lr=0.05–0.3, depth=3–7

06

Support Vector Machine (SVM)

Finds the hyperplane maximising the margin between classes. Kernel trick (RBF) maps data to higher dimensions for non-linear separation. Powerful for small, high-dimensional datasets. Slow on large data.

max margin between classes · Kernel: RBF best default · C controls margin-error tradeoff

07

K-Nearest Neighbors (KNN)

No training step (lazy learner). At predict time, find K nearest training points and use majority vote. O(n) per prediction — unusable at scale. K controls bias-variance. Always normalise features before using KNN.

predict(x) = majority_vote(k nearest points) · start K = √n, use odd K

08

Naive Bayes

Probabilistic classifier using Bayes’ theorem. Assumes features are independent. Fast, handles high-dimensional sparse data. Best choice for text classification baselines.

P(y|X) ∝ P(y) · ∏ P(xᵢ|y) · Use: spam, sentiment, news classification

python — Sklearn Pipeline (production pattern)

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import cross_val_score, train_test_split
from sklearn.metrics import classification_report

# Pipeline prevents data leakage
pipe = Pipeline([
    ('scaler', StandardScaler()),
    ('model',  RandomForestClassifier(n_estimators=200, random_state=42))
])
scores = cross_val_score(pipe, X_train, y_train, cv=5, scoring='f1')
print(f"CV F1: {scores.mean():.3f} +/- {scores.std():.3f}")
pipe.fit(X_train, y_train)
print(classification_report(y_test, pipe.predict(X_test)))

Algorithm selection: Tabular data → XGBoost/LightGBM. Need interpretability → LogReg or Decision Tree. Sparse text → Naive Bayes or linear SVM. Quick baseline → Logistic Regression always first.

00

Unsupervised Learning

No labels — the model discovers hidden structure in raw data on its own. Used when labelling data is expensive, impossible, or when you don’t know what categories exist.

Input X (no labels) → Discover structure → Clusters / Representations / Rules

01

K-Means Clustering

Assign each point to one of K clusters by minimising distance to cluster centroids. Iterate: assign → recompute centroids → repeat until stable. Fast and scalable. Assumes spherical clusters. Use Elbow Method or Silhouette Score to choose K.

centroidⱼ = mean(all points in cluster j) · repeat until no reassignments

02

DBSCAN

Density-based clustering — finds clusters of arbitrary shape, marks sparse-region points as outliers. No need to specify K. Parameters: ε (neighbourhood radius) and minPts (minimum neighbours). Great for spatial data and anomaly detection.

core point: ≥ minPts neighbours within ε · outlier: no core neighbour within ε

03

PCA — Principal Component Analysis

Linear dimensionality reduction. Finds orthogonal directions (principal components) of maximum variance. Projects high-dimensional data to fewer dimensions while preserving structure. Used for visualisation, noise reduction, preprocessing.

PCA: eigenvectors of XᵀX → top-k components → project X → X_reduced

04

t-SNE & UMAP

Non-linear dimensionality reduction for visualisation only. t-SNE preserves local cluster structure. UMAP is faster and preserves global structure better. Both used extensively to visualise high-dimensional embeddings from neural networks or LLMs.

t-SNE: minimise KL divergence between distributions · UMAP: graph-based, faster

05

Autoencoders

Neural network compressing data to a compact latent representation then reconstructing it. Learns meaningful representations without labels. Used for anomaly detection (high reconstruction error = anomaly), denoising, and as building blocks for VAEs and generative models.

input → [Encoder] → latent z (bottleneck) → [Decoder] → reconstruction

06

Association Rule Learning

Discovers co-occurrence rules in transactional data. "People who buy diapers also buy beer." Three metrics: Support (frequency), Confidence (rule accuracy), Lift (better than random). Used in recommendation systems and market basket analysis.

support(A→B) = P(A∩B) · confidence = P(B|A) · lift = confidence / P(B)

💡

Supervised vs Unsupervised: Supervised = you have labels, measure prediction error. Unsupervised = no labels, measure cluster quality. Semi-supervised: a few labels + many unlabelled — use both when labelling is expensive (medical imaging, speech).

Algorithm Quick Reference

Algorithm	Type	Task	Strengths	Weaknesses	Best For
Linear Regression	Supervised	Regression	Fast, interpretable	Linear assumption	Baseline, pricing
Logistic Regression	Supervised	Classification	Probabilistic, fast	Linear boundary	Baseline, fraud, churn
Decision Tree	Supervised	Both	Interpretable, no scaling	Overfits badly	Rule extraction
Random Forest	Supervised	Both	Robust, feature importance	Slow predict	General tabular
XGBoost / LGBM	Supervised	Both	Best tabular accuracy	Many hyperparams	Kaggle, production
SVM	Supervised	Both	High-dim, small data	Slow on large data	Text, bio
KNN	Supervised	Both	Simple, no training	O(n) predict	Small datasets
Naive Bayes	Supervised	Classification	Extremely fast, sparse	Independence assumption	Text classification
K-Means	Unsupervised	Clustering	Fast, scalable	Need K, spherical clusters	Customer segments
DBSCAN	Unsupervised	Clustering	Arbitrary shape, outliers	Sensitive to params	Geo, anomaly
PCA	Unsupervised	Dim Reduction	Fast, linear	Linear only	Preprocessing, viz
t-SNE / UMAP	Unsupervised	Visualisation	Non-linear, beautiful	Not for preprocessing	Embedding viz

Interactive Algorithm Selector

Do you have labelled data (known outputs)?

What is your output type?

Data type?

What do you want to discover?

When to Use Each

Tabular

Structured Data

Default order: LogReg → Random Forest → XGBoost. Never jump to neural networks for tables without trying these first.

Text

NLP Tasks

Short text: TF-IDF + Naive Bayes/SVM. Semantic or long text: Transformers (BERT, sentence-transformers).

Images

Visual Data

Always use CNNs or ViTs. Pre-train on ImageNet, fine-tune on your data. Never train from scratch without millions of images.

Time Series

Sequential Data

Classical: ARIMA, Holt-Winters. ML: LightGBM with lag features. DL: LSTMs, Transformers, N-BEATS.

1

Problem Definition

Define task type (classification/regression/clustering), success metric, constraints, and a non-ML baseline. Wrong problem framing is the #1 ML project failure mode.

Define: task · metric · baseline · constraints before touching data

2

Data Collection & EDA

Gather data. Explore distributions, correlations, missing values, class balance. 70% of real ML work happens here. Understand the data before modelling. Tools: Pandas, Matplotlib, Seaborn, Pandas Profiling.

df.describe() · df.isnull().sum() · sns.heatmap(df.corr()) · df.value_counts()

3

Data Preprocessing

Handle missing values. Encode categoricals (one-hot for nominal, ordinal for ordered). Scale features (StandardScaler for SVM, KNN, PCA). Remove or cap outliers. Split train/val/test (80/10/10).

X_train, X_test = train_test_split(X, y, test_size=0.2, stratify=y)

4

Feature Engineering

Create new features from existing ones. Date decomposition (year, month, weekday, hour). Polynomial features. Domain-specific combinations. This step often provides the biggest performance lift — more than switching models.

df['age_income'] = df['age'] * df['income'] · df['month'] = df['date'].dt.month

5

Model Selection & Training

Start with Logistic Regression baseline. Then try tree ensembles. Use 5-fold cross-validation. Tune with GridSearchCV or RandomizedSearchCV. Always use Pipelines to prevent data leakage.

GridSearchCV(model, param_grid, cv=5, scoring='f1', n_jobs=-1)

6

Evaluation & Error Analysis

Evaluate on held-out test set. Check confusion matrix. Where does it fail? Is error random or systematic? Examine individual wrong predictions. Real improvement comes from understanding failure patterns.

classification_report(y_test, y_pred) · inspect wrong predictions directly

7

Deployment & Monitoring

Serialise model (joblib). Build REST API (FastAPI). Monitor for data drift — real-world data changes over time and performance degrades silently. Set alerts on prediction distribution shifts. Retrain periodically.

joblib.dump(model, 'model.pkl') · monitor: input drift + output drift + performance

⚠️

Data leakage — the silent killer: Never fit your scaler or encoder on the full dataset before splitting. Fit only on training data. Always use Sklearn Pipelines — they enforce this automatically and prevent the #1 source of unrealistically high CV scores.

Classification Metrics

Acc

Accuracy

= (TP + TN) / Total

Overall correctness. Misleading on imbalanced datasets — 95/5 split gives 95% accuracy by predicting majority always.

✅ Use when: classes are balanced

Prec

Precision

= TP / (TP + FP)

Of all predicted positives, how many are actually correct? High precision = low false alarm rate.

✅ Use when: FP cost is high (spam filter)

Rec

Recall

= TP / (TP + FN)

Of all actual positives, how many did the model catch? High recall = low miss rate.

✅ Use when: FN cost is high (cancer screening)

F1

F1 Score

= 2·P·R / (P + R)

Harmonic mean of precision and recall. Balances both. Best single metric for imbalanced binary classification.

✅ Use when: classes are imbalanced

AUC

ROC-AUC

Area under ROC curve

Threshold-independent. 0.5 = random, 1.0 = perfect. Measures rank discrimination. Best for ranking tasks.

✅ Use when: threshold selection matters

MCC

Matthews CC

Balanced on all skew levels

Most informative metric for binary classification. Considers all four confusion matrix quadrants. -1 to +1 scale.

✅ Use when: severe class imbalance

Regression Metrics

MSE

Mean Squared Error

= (1/n) Σ(yᵢ − ŷᵢ)²

Penalises large errors heavily. Sensitive to outliers. Standard loss for training regression models.

RMSE

Root MSE

= √MSE

Same as MSE in original units — interpretable. "On average, predictions are off by ± X units."

MAE

Mean Absolute Error

= (1/n) Σ|yᵢ − ŷᵢ|

Average absolute deviation. Robust to outliers. Better when outlier errors shouldn’t dominate the metric.

R²

R-Squared

= 1 − SS_res/SS_tot

Proportion of variance explained. R²=1 perfect, R²=0 predicts mean, R²<0 worse than mean.

Metric selection: Imbalanced classes → F1 or AUC, never accuracy. Regression with outliers → MAE not RMSE. Ranking → AUC. Need single number for imbalanced binary → MCC. Stakeholders → RMSE (interpretable in units) or accuracy%.

python — Complete ML Workflow (Titanic)

import pandas as pd
from sklearn.model_selection import train_test_split, cross_val_score
from sklearn.preprocessing import StandardScaler
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import classification_report, roc_auc_score
from sklearn.pipeline import Pipeline
from sklearn.impute import SimpleImputer
import joblib

df = pd.read_csv('titanic.csv')
df['FamilySize'] = df['SibSp'] + df['Parch'] + 1
X = df[['Pclass', 'Age', 'Fare', 'FamilySize']]
y = df['Survived']

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, stratify=y, random_state=42)

pipe = Pipeline([
    ('imputer', SimpleImputer(strategy='median')),
    ('scaler',  StandardScaler()),
    ('model',   RandomForestClassifier(n_estimators=200, random_state=42))
])
scores = cross_val_score(pipe, X_train, y_train, cv=5, scoring='f1')
print(f"CV F1: {scores.mean():.3f} +/- {scores.std():.3f}")
pipe.fit(X_train, y_train)
y_pred = pipe.predict(X_test)
y_prob = pipe.predict_proba(X_test)[:, 1]
print(classification_report(y_test, y_pred))
print(f"AUC: {roc_auc_score(y_test, y_prob):.3f}")
joblib.dump(pipe, 'model.pkl')

python — XGBoost with RandomizedSearchCV

import xgboost as xgb
from sklearn.model_selection import RandomizedSearchCV

param_grid = {
    'max_depth':     [3, 5, 7],
    'learning_rate': [0.01, 0.05, 0.1],
    'n_estimators':  [200, 500],
    'subsample':     [0.6, 0.8, 1.0],
}
search = RandomizedSearchCV(
    xgb.XGBClassifier(eval_metric='logloss'), param_grid,
    n_iter=30, cv=5, scoring='roc_auc', n_jobs=-1
)
search.fit(X_train, y_train)
print("Best AUC:", search.best_score_)
print("Best params:", search.best_params_)

sklearn

Scikit-Learn

The ML standard library. Every classical algorithm: regression, classification, clustering, preprocessing, pipelines, evaluation. Start here for all tabular ML.

from sklearn.ensemble import RandomForestClassifier from sklearn.pipeline import Pipeline

XGB

XGBoost / LightGBM

Best-in-class for tabular data. Sequential gradient boosting. LightGBM is faster on large datasets. Both win Kaggle. Always try before neural networks on tables.

import xgboost as xgb import lightgbm as lgb

torch

PyTorch

The researcher’s deep learning framework. Dynamic graphs, Pythonic, flexible. Dominates academic research. Best for learning DL and custom architectures.

import torch import torch.nn as nn

TF

TensorFlow / Keras

Google’s production DL framework. Keras API makes neural networks accessible. Best for deployment (TF Serving), mobile (TF Lite), and TPU training at scale.

import tensorflow as tf from tensorflow import keras

🤗

Hugging Face

Pretrained transformer models for NLP and CV. 200k+ models. Access GPT, BERT, LLaMA, Stable Diffusion in 3 lines. Essential for modern NLP and GenAI.

from transformers import pipeline nlp = pipeline("text-classification")

Optuna

Hyperparameter Tuning

Bayesian optimisation — smarter than grid search. Integrates with XGBoost, PyTorch, sklearn. Reduces tuning time by 5–10x vs grid search.

import optuna study = optuna.create_study(direction="maximize")

🗺️

Learning order: NumPy + Pandas → Matplotlib/Seaborn → Scikit-Learn → XGBoost → PyTorch or TensorFlow → Hugging Face. Don’t jump to PyTorch before mastering sklearn pipelines.

💡

You don’t need deep proofs. You need intuition — understanding why each formula exists and what it does. Focus on: what problem does this solve? What happens when this value is too large or too small?

Core Math Topics

LinAlg

Linear Algebra ⭐

Vectors store data records. Matrices store datasets. Dot products compute predictions. Matrix multiply = batch predictions. Neural networks are matrix operations.

Stats

Statistics

Mean, variance, distributions. Understand your data before modelling. Feature selection, outlier detection, evaluation interpretation.

Prob

Probability

AI makes probabilistic predictions. Bayes theorem, conditional probability, distributions. Core to Naive Bayes, logistic regression, uncertainty estimation.

Calc

Calculus

Derivatives = direction of steepest change. Gradients = multivariable derivatives. Chain rule = backpropagation. How neural networks learn.

Optim

Optimisation

Loss functions measure error. Gradient descent minimises loss. Learning rate controls step size. The engine of all ML training.

Essential Formulas

GD

Gradient Descent

The fundamental learning algorithm. Iteratively moves weights in the direction of steepest descent of the loss. Too high α overshoots; too low α takes forever.

θ ← θ − α · ∇J(θ) · α = learning rate · ∇J(θ) = gradient of loss

B–V

Bias–Variance Decomposition

Every model’s error decomposes into these three terms. Irreducible noise is unavoidable. Your job is to minimise bias² + variance. Regularisation reduces variance at the cost of slight bias.

E[error] = Bias² + Variance + σ²(noise) · High bias → underfit · High var → overfit

L1/L2

Regularisation

L1 (Lasso): Adds |w| to loss → drives weights exactly to zero → automatic feature selection. L2 (Ridge): Adds w² to loss → shrinks weights but doesn’t zero them. ElasticNet: Mix of both.

Loss_L1 = Loss + λΣ|wᵢ| Loss_L2 = Loss + λΣwᵢ²

Soft

Softmax (Multi-class Output)

Converts raw logits into a probability distribution across classes. Output sums to 1. Used in the final layer of any multi-class classifier or language model next-token prediction.

softmax(xᵢ) = exp(xᵢ) / Σⱼ exp(xⱼ) · Σ softmax(x) = 1.0

BCE

Cross-Entropy Loss

Standard loss for classification. Penalises confident wrong predictions heavily (log scale). Binary cross-entropy for two classes; categorical for multiple.

BCE = −(y·log(ŷ) + (1−y)·log(1−ŷ)) · lower = better predictions

Math learning timeline: Basic intuition → 3–4 weeks with practice. Practical AI usage → 6–8 weeks. You do NOT need: complex derivations, formal proofs, advanced calculus. You DO need: intuition behind formulas and how they connect to model behaviour.

00

Why Data Prep Dominates ML Work

Real-world data is messy, incomplete, and inconsistent. 70–80% of ML project time is data preparation. No algorithm can rescue bad data. Garbage in, garbage out — always.

EDA → Clean → Engineer Features → Encode → Scale → Split → Model

01

EDA — Exploratory Data Analysis

Before modelling: understand distributions, correlations, missing rates, class balance, outliers. Never skip EDA — it determines your entire preprocessing strategy and often reveals the most important features.

df.describe() · df.isnull().sum() · sns.heatmap(df.corr()) · sns.pairplot(df)

02

Handling Missing Values

Drop: If >70% missing or completely random. Median/Mean impute: For numerical when missing at random. Mode impute: For categorical. KNN impute: Structured missingness. Indicator feature: Add binary column flagging "was missing" — the absence itself is often informative.

SimpleImputer(strategy='median') · KNNImputer() · never drop blindly

03

Encoding Categorical Variables

Label Encoding: Ordinal categories (Small=0, Medium=1, Large=2). One-Hot Encoding: Nominal categories (colour, city) — use drop='first' to avoid multicollinearity. Target Encoding: High cardinality — replace category with mean target value (apply only on training data to avoid leakage).

pd.get_dummies(df, drop_first=True) · OrdinalEncoder() · TargetEncoder()

04

Feature Scaling

StandardScaler: Zero mean, unit variance. Required for SVM, KNN, PCA, logistic regression. MinMaxScaler: Scale to [0,1]. For neural networks with bounded activations. Not needed: Tree-based models (RF, XGBoost) are scale-invariant.

StandardScaler: z = (x−μ)/σ · MinMaxScaler: x' = (x−min)/(max−min)

05

Feature Engineering — The Biggest Lever

Create new features from existing data. Date → year, month, day, weekday, hour, is_holiday. Text → word count, char count, sentiment. Combine → income/household_size = per_capita. Good features often beat more complex models.

df['per_capita'] = df['income'] / df['household_size'] · domain knowledge wins

06

Handling Imbalanced Classes

SMOTE: Generates synthetic minority examples. Apply to training data only. Class weights: class_weight='balanced' in sklearn. Threshold tuning: Adjust from default 0.5 using precision-recall curve. Never: apply SMOTE to the test set.

SMOTE() from imblearn · class_weight='balanced' · precision-recall curve for threshold

🔍

A

Algorithms

ML models: linear regression, decision trees, clustering, SVM, neural networks. Choose based on data type, size, and problem type.

B

Business Understanding

Translate business problems into data science tasks. Define objectives, success metrics, and constraints before touching data.

C

Cleaning Data

Handle missing values, outliers, duplicates, type errors. Crucial — 70% of DS time spent here. Garbage in, garbage out.

D

Data Visualization

Matplotlib, Seaborn, Plotly. Identify patterns, trends, outliers visually before any modelling attempt.

E

EDA

Exploratory Data Analysis. Understand data before modelling. Summary statistics, distributions, correlations, missing value analysis.

F

Feature Engineering

Create and select features to improve performance. Domain knowledge + creativity = better predictions than model complexity.

G

Gradient Descent

Optimisation: θ ← θ − α·∇L. Iteratively move weights in direction of steepest loss decrease. The learning engine.

H

Hypothesis Testing

Validate assumptions statistically. p-values, confidence intervals, t-tests, A/B testing. Essential for data-driven decisions.

I

Imbalanced Data

When one class dominates. Use SMOTE, class_weight='balanced', or AUC/F1 instead of accuracy. Never apply SMOTE to test data.

J

Jupyter Notebooks

Interactive coding environment combining code, output, and docs. The standard tool for ML experimentation.

K

K-Means Clustering

Unsupervised algorithm grouping similar data into K clusters by minimising distance to centroids. Fast and scalable.

L

Logistic Regression

Binary classification using sigmoid output. Fast, interpretable baseline. Always run this before complex models.

M

Model Evaluation

Accuracy, Precision, Recall, F1, AUC-ROC, RMSE, R². Choose metric based on problem type — F1 for imbalanced, RMSE for regression.

N

NumPy

Foundation of Python ML. Vectorised array operations, matrix multiply, linear algebra. All ML libraries build on NumPy.

O

Overfitting

Model memorises training data, fails on new data. High variance. Fix: regularisation, dropout, cross-validation, more data.

P

Pandas

Data manipulation and analysis. DataFrames, groupby, merge, pivot. The primary tool for structured data wrangling.

Q

Quantitative Skills

Linear algebra, probability, statistics, calculus. Need intuition — not proofs. Foundation for understanding ML deeply.

R

Regularisation

Penalise large weights to avoid overfitting. L1 (Lasso): sparse weights, feature selection. L2 (Ridge): shrinks weights evenly.

S

SQL

Query data from databases. SELECT, JOIN, GROUP BY, window functions. Essential for every data scientist — most data lives in SQL.

T

Time Series

Forecasting temporal data. Classical: ARIMA. ML: LightGBM with lag features. DL: LSTM, N-BEATS, Transformers.

U

Underfitting

Model too simple — misses real patterns. High bias. Fix: more features, increase model complexity, reduce regularisation.

V

Validation

K-fold cross-validation ensures the model works on unseen data. Always use CV — never evaluate on training data alone.

W

Web Scraping

Collect data from websites. BeautifulSoup, Scrapy, Selenium. Extract and parse HTML to build custom datasets.

X

XGBoost

Extreme Gradient Boosting. Best algorithm for tabular data. Wins Kaggle. Always try before neural networks on structured data.

Y

Yellowbrick

sklearn-compatible visualisations for model diagnosis. Residual plots, classification reports, feature ranking, learning curves.

Z

Z-Score

Standard score: z = (x − μ) / σ. Used for standardisation and outlier detection — |z| > 3 typically flags an outlier.

Phase 1FoundationWeeks 1–4

Week 1–2Python basics + NumPy arrays and vectorised operations

Week 3Pandas: DataFrames, groupby, merge, EDA workflow

Week 4Matplotlib + Seaborn: visualisation fundamentals

🏗 Project: Titanic survival prediction — EDA + feature engineering + logistic regression

Phase 2Core MLWeeks 5–10

SupervisedLogReg, Decision Trees, Random Forest, XGBoost — all from sklearn

UnsupervisedK-Means, DBSCAN, PCA, t-SNE

EvaluationAccuracy, F1, AUC, confusion matrix, cross-validation

PipelineSklearn Pipelines, GridSearchCV, leakage prevention

🏗 Project: Customer churn classifier — full pipeline from EDA to deployment

Phase 3Deep LearningWeeks 11–18

FoundationsPerceptrons, activations, backpropagation, gradient descent

FrameworkPyTorch or TensorFlow — pick one, master it

ArchitecturesCNNs for images, RNNs/LSTMs for sequences, Transformers

🏗 Project: MNIST digit classifier CNN · IMDB sentiment LSTM

Phase 4SpecialisationWeeks 19+

NLP TrackHugging Face, BERT, GPT fine-tuning, RAG pipelines

CV TrackYOLO, Faster R-CNN, segmentation, OpenCV

GenAI TrackLLMs, prompt engineering, agents, LangChain

MLOpsMLflow, FastAPI serving, Docker, monitoring, retraining

🏗 Project: Deploy a real product — chatbot, image classifier, or recommendation engine

💡

Key principle: Build projects at every phase — theory without code doesn’t stick. Enter Kaggle after Phase 2. Contribute to open source after Phase 3. Your GitHub portfolio matters more than certificates.

💡

Most frequently asked ML interview questions at Google, Amazon, Meta, and AI startups. Use STAR format for applied questions.

Q1

Supervised vs Unsupervised Learning

Supervised: Labelled data, predict output (regression/classification). Loss measured against known y. Unsupervised: No labels, discover structure (clustering, dim reduction). Evaluated with silhouette or visual inspection. Semi-supervised: Mix — few labels + many unlabelled to improve learning.

Supervised: y known → minimise loss(ŷ, y) | Unsupervised: y unknown → find structure in X

Q2

How do you handle missing data?

Analyse first: Is it MCAR, MAR, or MNAR? Options: Drop rows (if <5% missing), median/mean impute (numerical, MCAR), mode impute (categorical), KNN impute (structured missingness), add binary indicator column (missingness may be informative).

Never blindly drop — always check if missingness is random or a signal.

Q3

What is overfitting and how do you fix it?

High train accuracy, low test accuracy = high variance. Fixes: L1/L2 regularisation, dropout, cross-validation, more data, simpler model, early stopping, feature selection, data augmentation.

Signs: train=98%, test=72% | Fix order: more data → simpler model → regularise

Q4

Bagging vs Boosting

Bagging (Random Forest): independent models on bootstrap samples → average → reduces variance. Boosting (XGBoost, AdaBoost): sequential, each corrects previous errors → reduces bias. Both reduce total error.

Bagging: parallel, reduces variance | Boosting: sequential, reduces bias

Q5

When should you NOT use ML?

When simple rules solve it. When you have <1000 labelled examples (overfitting risk). When interpretability is legally required. When prediction errors are catastrophic and confidence can’t be verified. When data changes faster than retraining cadence.

Q6

Precision vs Recall — which matters when?

Precision: minimise false positives. Spam filter (don’t flag legit email). Fraud alerts (don’t block legit transactions). Recall: minimise false negatives. Cancer screening (don’t miss sick patients). F1: when both matter equally.

FP costly → Precision | FN costly → Recall | both → F1

Q7

L1 vs L2 Regularisation

L1 (Lasso): Penalty = λΣ|wᵢ|. Drives weights exactly to zero → feature selection. Use when many features are irrelevant. L2 (Ridge): Penalty = λΣwᵢ². Shrinks proportionally — none zero. Better when most features contribute.

L1 → sparse (selection) | L2 → small weights (multicollinearity) | choose with CV

Q8

Explain data leakage and how to prevent it.

Data leakage = future/test information accidentally influencing training. Example: fitting StandardScaler on full dataset before splitting. Causes unrealistically high CV scores that collapse in production. Prevention: always use Pipelines — fit preprocessing only on X_train.

Pipeline ensures scaler.fit() only on X_train — never on X_test

Q9

Dataset with 95% class A, 5% class B — how do you handle it?

Never use accuracy (95% baseline by always predicting A). Use F1 or AUC. Try: class_weight='balanced', SMOTE oversampling on train set only, threshold tuning (precision-recall curve), collect more minority class data.

metric: F1/AUC | fix: class_weight → SMOTE → threshold → data collection

Q10

How do you deploy an ML model?

Serialise with joblib. Build REST API (FastAPI). Containerise with Docker. Host on cloud (AWS SageMaker, GCP Vertex AI). Monitor input drift, output drift, performance decay. Retrain when metrics drop.

joblib.dump(model) → FastAPI → Docker → Cloud → Monitor → Retrain

Everything you need for an ML interview or project — on one page. Bookmark this tab.

Algorithm Selection

Data Type

Task

Start With

Best Model

Tabular

Classification

LogReg

XGBoost / LGBM

Tabular

Regression

Linear Regression

XGBoost / LGBM

Text

Classification

TF-IDF + NB

BERT / RoBERTa

Images

Classification

ResNet-18

EfficientNet / ViT

Time Series

Forecasting

ARIMA

LGBM + lag features

Unlabelled

Clustering

K-Means

DBSCAN + K-Means

Metric Selection

Situation

Use This Metric

Avoid

Balanced classes

Accuracy or F1

—

Imbalanced classes

F1 or AUC-ROC

Accuracy

FP is costly (spam)

Precision

Recall alone

FN is costly (cancer)

Recall

Precision alone

Ranking / scoring

AUC-ROC

Accuracy

Regression + outliers

MAE

RMSE / MSE

Regression, interpretable

RMSE + R²

—

Key Formulas

Gradient Descent

θ ← θ − α · ∇J(θ)

α = learning rate · ∇J = gradient of loss

Linear Regression

ŷ = w₀ + w₁x₁ + w₂x₂ + ...

Loss = MSE = (1/n)Σ(yᵢ − ŷᵢ)²

Logistic Regression

σ(z) = 1 / (1 + e⁻ᶻ)

Loss = Binary Cross-Entropy

Bias–Variance

Error = Bias² + Variance + Noise

High bias → underfit · High var → overfit

Precision & Recall

P = TP/(TP+FP) · R = TP/(TP+FN)

F1 = 2·P·R / (P + R)

L1 / L2 Regularisation

L1: Loss + λΣ|wᵢ| · L2: Loss + λΣwᵢ²

L1 → sparse (selection) · L2 → shrink

Softmax

softmax(xᵢ) = exp(xᵢ) / Σⱼ exp(xⱼ)

Multi-class output · sums to 1.0

K-Means Update

μⱼ = mean(all x where assign=j)

Repeat: assign → update → converge

The ML Pipeline in 60 Seconds

1

Define Problem

Task + metric + baseline

→

2

EDA

Understand data first

→

3

Clean + Engineer

Missing · encode · scale

→

4

Train + CV

Pipeline · 5-fold · tune

→

5

Evaluate

Test set · error analysis

→

6

Deploy + Monitor

FastAPI · drift alerts

3 rules to never break: (1) Never evaluate on training data. (2) Never fit scalers/encoders before the train/test split. (3) Never use accuracy on imbalanced classes.