Python Libraries — NumPy, Pandas, Matplotlib & More

01

pip & Virtual Environments

Working with External Packages

So far you used built-in libraries (math, datetime, json). Now learn how to install external packages created by the Python community. Popular examples: requests (API calls), numpy (numerical computing), pandas (data analysis), flask/django (web frameworks).

1️⃣ What is pip?

pip is Python's package manager. It installs external libraries from PyPI (Python Package Index).

Terminal — pip Commands

# Install a package
pip install requests

# After installing, use in Python:
import requests

# List all installed packages
pip list

# Show package information
pip show requests
# Displays: version, location, dependencies

# Upgrade a package
pip install --upgrade requests

# Uninstall a package
pip uninstall requests

2️⃣ What is a Virtual Environment?

A virtual environment isolates project dependencies. Without it, conflicts occur when different projects need different versions of the same package.

Problem without venv: Project A needs requests 2.25, Project B needs requests 3.0. Virtual environments solve this by giving each project its own isolated packages.

Terminal — Virtual Environment

# Create a virtual environment
python -m venv myenv
# This creates a folder: myenv/

# Activate Virtual Environment
# Windows:
myenv\Scripts\activate

# Mac/Linux:
source myenv/bin/activate

# Terminal will show: (myenv)
# Now packages install only in this environment

# Install package inside environment
pip install requests

# Deactivate environment
deactivate

02

datetime — Date and Time

What is datetime?

The datetime module supplies classes for manipulating dates and times. It's part of Python's standard library, so no installation is required.

Why use it: Handle timestamps, calculate time differences, format dates for display, schedule tasks, and work with time zones.

Python — datetime Examples

from datetime import datetime, date, timedelta

# Current date and time
now = datetime.now()
print(now)                  # 2025-01-15 14:30:45.123456
print(now.date())           # 2025-01-15
print(now.time())           # 14:30:45.123456

# Formatting dates
formatted = now.strftime("%Y-%m-%d %H:%M:%S")
print(formatted)            # 2025-01-15 14:30:45

# Common format codes
# %Y - 4-digit year
# %m - 2-digit month
# %d - 2-digit day
# %H - Hour (24h)
# %M - Minute
# %S - Second

# Creating specific dates
my_date = date(2025, 1, 15)
print(my_date)              # 2025-01-15

# Date arithmetic
tomorrow = my_date + timedelta(days=1)
print(tomorrow)             # 2025-01-16

# Parse string to date
date_str = "2025-01-15"
parsed = datetime.strptime(date_str, "%Y-%m-%d")

03

math — Mathematical Functions

What is math?

The math module provides access to mathematical functions defined by the C standard. It includes trigonometric, logarithmic, and exponential functions.

Why use it: Perform complex mathematical calculations, work with constants like π and e, and access functions not available in built-in Python.

Python — math Examples

import math

# Mathematical constants
print(math.pi)              # 3.141592653589793
print(math.e)               # 2.718281828459045

# Square root and powers
print(math.sqrt(16))        # 4.0
print(math.pow(2, 3))       # 8.0

# Rounding functions
print(math.ceil(4.2))       # 5 (round up)
print(math.floor(4.8))      # 4 (round down)

# Factorial
print(math.factorial(5))    # 120 (5*4*3*2*1)

# Trigonometric functions
print(math.sin(math.pi / 2))  # 1.0
print(math.cos(0))            # 1.0

# Logarithms
print(math.log(100, 10))    # 2.0 (log base 10)
print(math.log(math.e))     # 1.0 (natural log)

04

random — Random Number Generation

What is random?

The random module implements pseudo-random number generators for various distributions. It's used for generating random numbers, making random choices, and shuffling data.

Why use it: Create games, simulations, randomized testing, sampling data, and any scenario requiring unpredictability.

Python — random Examples

import random

# Random integer in range
print(random.randint(1, 10))   # Random int from 1 to 10

# Random float
print(random.random())        # Random float 0.0 to 1.0
print(random.uniform(1, 10))  # Random float 1.0 to 10.0

# Random choice from list
colors = ['red', 'green', 'blue', 'yellow']
print(random.choice(colors))

# Shuffle a list
cards = ['A', 'K', 'Q', 'J']
random.shuffle(cards)
print(cards)                  # Randomly reordered

# Random sample (unique choices)
lottery = random.sample(range(1, 50), 6)
print(lottery)                # 6 unique numbers

# Seeding for reproducibility
random.seed(42)
print(random.random())        # Same result every run

05

os & sys — System Interaction

What are os and sys?

The os module provides functions for interacting with the operating system — working with files, folders, and system paths. The sys module gives access to Python runtime information and command-line arguments.

Why use them: Automation scripts, backend systems, data processing pipelines, and any task requiring file system operations or system-level information.

os Module — Operating System Interaction

Common uses: working with files and folders, automation scripts, getting system paths.

Python — os Module Examples

import os

# Get current working directory
print(os.getcwd())

# List files in a folder
print(os.listdir())

# Create a new folder
os.mkdir("new_folder")

# Check if file exists
print(os.path.exists("data.txt"))

# Additional useful operations
os.makedirs("a/b/c")         # Create nested directories
os.rename("old.txt", "new.txt")  # Rename file
os.remove("file.txt")        # Delete file
os.rmdir("empty_folder")     # Delete empty directory
print(os.path.join("folder", "file.txt"))  # Build path

sys Module — System-Level Operations

Common uses: accessing Python runtime info, command-line arguments, system configuration.

Python — sys Module Examples

import sys

# Check Python version
print(sys.version)

# Command-line arguments
# Run: python script.py hello
print(sys.argv)  # ['script.py', 'hello']

# Exit the program with status code
sys.exit(0)  # 0 = success, non-zero = error

# Get Python path
print(sys.path)  # List of directories Python searches for modules

06

json — Data Exchange

What is JSON?

JSON (JavaScript Object Notation) is a lightweight data interchange format. Python's json module makes it easy to convert between Python objects and JSON format.

Why use it: APIs, web applications, configuration files, and data storage. JSON is the standard format for data exchange in modern applications.

Python — JSON Examples

import json

# Convert Python → JSON (string)
data = {"name": "Deepak", "age": 25}
json_data = json.dumps(data)
print(json_data)  # {"name": "Deepak", "age": 25}

# Convert JSON → Python
json_string = '{"name":"Deepak","age":25}'
data = json.loads(json_string)
print(data["name"])  # Deepak

# Save JSON to file
data = {"name": "Deepak", "age": 25}
with open("data.json", "w") as f:
    json.dump(data, f)

# Read JSON from file
with open("data.json", "r") as f:
    data = json.load(f)
print(data)

Key functions: dumps()/loads() work with strings, while dump()/load() work with files.

07

NumPy — Numerical Computing

What is NumPy?

NumPy (Numerical Python) is the foundation library for numerical computing in Python. It provides support for large, multi-dimensional arrays and matrices, along with a collection of mathematical functions.

Why use it: Faster operations than native Python lists (up to 50x), memory efficient, essential for data science, machine learning, and scientific computing.

1️⃣ Install & Import

Terminal + Python

# Install
pip install numpy

# Import (standard alias)
import numpy as np

2️⃣ Create NumPy Array

Python

import numpy as np
arr = np.array([1, 2, 3, 4])
print(arr)  # [1 2 3 4]

3️⃣ NumPy vs Python List

Key Difference: Python lists concatenate, NumPy arrays perform element-wise operations.

Python — List vs NumPy

# Python list — concatenates
a = [1, 2, 3]
b = [4, 5, 6]
print(a + b)  # [1, 2, 3, 4, 5, 6]

# NumPy array — element-wise addition
import numpy as np
a = np.array([1, 2, 3])
b = np.array([4, 5, 6])
print(a + b)  # [5 7 9]

# Vectorized operations (no loops needed!)
arr = np.array([1, 2, 3, 4])
print(arr + 10)   # [11 12 13 14]
print(arr * 2)    # [2 4 6 8]

4️⃣ Useful NumPy Functions

Python

arr = np.array([1, 2, 3, 4])

print(np.mean(arr))   # 2.5 (average)
print(np.sum(arr))    # 10 (total)
print(np.max(arr))    # 4 (maximum)
print(np.min(arr))    # 1 (minimum)
print(np.std(arr))    # Standard deviation

5️⃣ 2D Arrays (Matrices)

Python

# Create 2D array
matrix = np.array([
    [1, 2, 3],
    [4, 5, 6]
])
print(matrix)
# [[1 2 3]
#  [4 5 6]]

# Access element (row 0, column 1)
print(matrix[0, 1])  # 2

08

Pandas — Data Analysis

What is Pandas?

Pandas is a powerful data manipulation and analysis library. It provides DataFrame objects for working with structured data, similar to Excel spreadsheets or SQL tables.

Why use it: Easy data cleaning, transformation, analysis. Essential for data science, machine learning pipelines, business analytics, and financial analysis.

1️⃣ Install & Import

Terminal + Python

# Install
pip install pandas

# Import (standard alias)
import pandas as pd

2️⃣ What is a Series?

A Series is a one-dimensional labeled array (single column).

Python

import pandas as pd
data = [10, 20, 30, 40]
s = pd.Series(data)
print(s)

# Output:
# 0    10
# 1    20
# 2    30
# 3    40
# dtype: int64

3️⃣ What is a DataFrame?

A DataFrame is a table (like Excel) with rows and columns.

Python

import pandas as pd
data = {
    "Name": ["Amit", "Deepak", "Ravi"],
    "Marks": [78, 85, 90]
}
df = pd.DataFrame(data)
print(df)

# Output:
#     Name  Marks
# 0    Amit     78
# 1  Deepak     85
# 2    Ravi     90

4️⃣ Essential DataFrame Operations

Function	Description	Example
`df.head(n)`	First n rows	`df.head(10)`
`df.info()`	Column types, nulls	`df.info()`
`df.describe()`	Summary statistics	`df.describe()`
`df['col']`	Select column	`df['age']`
`df.dropna()`	Remove missing values	`df.dropna()`
`df.fillna(val)`	Fill missing values	`df.fillna(0)`
`df.groupby()`	Group and aggregate	`df.groupby('city')['salary'].mean()`
`df.to_csv()`	Save to CSV	`df.to_csv('out.csv', index=False)`

Real Example: Student Data Analysis

Python

import pandas as pd

data = {
    "Product": ["A", "B", "C"],
    "Sales": [100, 150, 200]
}
df = pd.DataFrame(data)

print("Total Sales:", df["Sales"].sum())
# Output: Total Sales: 450

09

Matplotlib — Data Visualization

What is Matplotlib?

Matplotlib is a comprehensive library for creating static, animated, and interactive visualizations in Python. It's the foundation for many other visualization libraries like Seaborn and Plotly.

Why use it: Create publication-quality plots, charts, and graphs. Essential for data exploration, presentations, and reports.

1️⃣ Common Chart Types

Function	Description	Example
`plt.plot(x, y)`	Line chart	`plt.plot(x, y, color='blue')`
`plt.scatter(x, y)`	Scatter plot	`plt.scatter(x, y, c='red')`
`plt.bar(x, height)`	Bar chart	`plt.bar(cats, vals)`
`plt.hist(x, bins)`	Histogram	`plt.hist(data, bins=30)`
`plt.pie(sizes)`	Pie chart	`plt.pie(sizes, labels=lbls)`

Complete Example: Multiple Plots

Python — Matplotlib Examples

import matplotlib.pyplot as plt
import numpy as np

# Line Plot
x = np.linspace(0, 10, 100)
y = np.sin(x)

plt.figure(figsize=(10, 6))
plt.plot(x, y, label='sin(x)', linewidth=2)
plt.title('Sine Wave')
plt.xlabel('X axis')
plt.ylabel('Y axis')
plt.legend()
plt.grid(True)
plt.show()

# Bar Chart
categories = ['A', 'B', 'C', 'D']
values = [23, 45, 56, 78]
plt.bar(categories, values, color='skyblue')
plt.title('Category Values')
plt.show()

10

Scikit-Learn — Machine Learning

What is Scikit-Learn?

Scikit-learn is the most popular machine learning library for Python. It provides simple and efficient tools for data mining, data analysis, and predictive modeling.

Why use it: Unified API for dozens of ML algorithms, preprocessing, model selection, and evaluation. Perfect for beginners and production use.

Essential ML Functions

Function	Description	Example
`train_test_split()`	Split data into train/test	`X_tr,X_te,y_tr,y_te = train_test_split(X,y,test_size=0.2)`
`StandardScaler()`	Standardize features	`scaler = StandardScaler(); X_sc = scaler.fit_transform(X)`
`LinearRegression()`	Linear regression model	`LinearRegression().fit(X_tr, y_tr)`
`RandomForestClassifier()`	Random forest model	`RandomForestClassifier(n_estimators=100)`
`accuracy_score()`	Classification accuracy	`accuracy_score(y_te, y_pred)`
`cross_val_score()`	Cross-validation	`cross_val_score(clf, X, y, cv=5)`

Complete Example: ML Pipeline

Python — sklearn pipeline

from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import accuracy_score

# 1. Split data
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

# 2. Train model
model = RandomForestClassifier(n_estimators=100)
model.fit(X_train, y_train)

# 3. Predict & evaluate
y_pred = model.predict(X_test)
print(f"Accuracy: {accuracy_score(y_test, y_pred):.3f}")

11

Requests — HTTP Library & API Calls

What is Requests?

Requests is an elegant and simple HTTP library for Python, built for human beings. It allows you to send HTTP/1.1 requests easily.

Why use it: Interact with APIs, web scraping, downloading files, automation. Much simpler than urllib.

1️⃣ Basic API Request

Python — GET Request

import requests

# Basic GET request
response = requests.get("https://api.github.com")
print(response.status_code)  # 200

# Status codes:
# 200 = Success
# 404 = Not Found
# 500 = Server Error

# Get API data as JSON (becomes Python dict)
data = response.json()
print(data)

# Access specific fields
print(data["current_user_url"])

2️⃣ Example: Random Joke API

Python — Joke API Example

import requests

# Fetch a random joke
url = "https://official-joke-api.appspot.com/random_joke"
response = requests.get(url)
data = response.json()

print(data["setup"])
print(data["punchline"])

# Output example:
# Why did the computer go to the doctor?
# Because it had a virus!

3️⃣ Sending Parameters to API

Python — API with Parameters

import requests

# Search GitHub repositories
params = {"q": "python"}
response = requests.get(
    "https://api.github.com/search/repositories",
    params=params
)
data = response.json()
print("Total repositories:", data["total_count"])

Quick Rules: API → program communication, requests.get() → send GET request, .json() → convert response to dictionary, status code 200 = success