Linux Basics
Filesystem, permissions, processes, systemd, networking.
Theory
Linux follows the Unix philosophy: everything is a file (devices, sockets, processes in /proc), tools do one thing well, and programs communicate via text streams through pipes. The kernel manages hardware; the shell (bash/zsh/fish) is just a program that interprets commands.
The filesystem hierarchy standard (FHS): /bin and /usr/bin hold executables, /etc holds config files (plain text, editable), /var holds variable data (logs in /var/log, databases in /var/lib), /tmp is cleared on reboot, /proc and /sys are virtual filesystems exposing kernel state.
File permissions use a 3×3 bit matrix: owner/group/other each have read(4)/write(2)/execute(1) bits. chmod 755 file sets rwxr-xr-x. chown user:group transfers ownership. Directories need execute bit to be entered (cd). setuid bit (chmod u+s) runs executable as owner — used by sudo, passwd.
Process management: ps aux lists all processes with PID, CPU, memory. kill -9 PID force-kills; kill -15 (SIGTERM) requests graceful shutdown. top/htop show real-time CPU and memory. systemctl start/stop/restart/status controls systemd units; journalctl -u service -f follows logs.
Networking: ss -tlnp shows listening sockets with PIDs (replaces netstat). ip addr shows interfaces; ip route shows routing table. curl -v tests HTTP endpoints. tcpdump -i eth0 port 80 captures packets. /etc/hosts for local DNS overrides; /etc/resolv.conf for nameservers.
Package management: apt (Debian/Ubuntu) uses apt install/remove/update/upgrade/autoremove. apt update refreshes package index; apt upgrade installs updates. dpkg -l lists installed packages. yum/dnf for RHEL/CentOS/Fedora. snap and flatpak for distribution-agnostic packages.
Cron schedules recurring jobs: crontab -e edits user crontab. Format: minute hour day month weekday command. */5 * * * * runs every 5 minutes. @reboot runs once at startup. Run scripts with absolute paths since cron has a minimal PATH. Redirect stderr: cmd >> /log 2>&1.
SSH: ssh-keygen -t ed25519 generates a key pair. Copy public key with ssh-copy-id user@host. ~/.ssh/config stores per-host settings (IdentityFile, Port, ProxyJump for bastion hops). ssh -L 5432:db:5432 user@bastion creates a local port forward to a database behind a firewall.
Architecture Diagram
Users / clients
|
Linux Basics
|
Core services
|
Data + observabilityExamples
# File operations
ls -lah /var/log # List with human-readable sizes
find / -name "*.conf" 2>/dev/null # Find all .conf files
grep -rn "error" /var/log/ --include="*.log" # Recursive search
tail -f /var/log/syslog # Follow log in real time
# Permissions
chmod 755 script.sh # rwxr-xr-x (owner full, others read+exec)
chmod 600 ~/.ssh/id_ed25519 # rw------- (private key must be 600)
chown www-data:www-data /var/www # Transfer ownership to web server user
find /uploads -type f -exec chmod 644 {} \; # Batch permission fix
# Process management
ps aux | grep python # Find Python processes
kill -15 1234 # Graceful shutdown (SIGTERM)
kill -9 1234 # Force kill (SIGKILL) — last resort
# Disk & memory
df -h # Disk usage by filesystem
du -sh /var/log/* # Size of each log directory
free -h # Memory usage (used/free/cache/buff)
vmstat 1 5 # CPU/memory stats every 1s, 5 times
# Network
ss -tlnp # Listening ports with process names
curl -I https://example.com # HTTP headers only
lsof -i :8000 # What process owns port 8000?
# SSH config (~/.ssh/config)
Host prod-bastion
HostName 52.1.2.3
User ubuntu
IdentityFile ~/.ssh/prod_key
Host db-private
HostName 10.0.1.50
User ubuntu
ProxyJump prod-bastion # Jump through bastion
# Systemd
systemctl status nginx
systemctl restart nginx
journalctl -u nginx -f # Follow nginx logs
journalctl --since "1 hour ago" -u nginx # Time filter
systemctl list-units --failed # Show failed units
Interview Questions
What does chmod 755 mean in octal notation?
7 (owner) = 4+2+1 = rwx. 5 (group) = 4+0+1 = r-x. 5 (other) = r-x. Common for executables and directories you want others to read/execute but not write. chmod 644 is typical for files (owner can write, others read-only).
How do you find what process is using port 8080?
ss -tlnp | grep :8080 (modern), or lsof -i :8080, or fuser 8080/tcp. ss is preferred over deprecated netstat. Output shows PID in the last column.
What is the difference between kill -9 and kill -15?
SIGTERM (15) asks the process to shut down gracefully — it can catch the signal, flush buffers, and clean up. SIGKILL (9) is sent by the kernel — cannot be caught or ignored, process is immediately terminated. Always try SIGTERM first; use SIGKILL only if the process is unresponsive.
How does sudo work, and what is /etc/sudoers?
sudo runs a command as another user (default: root). /etc/sudoers defines who can run what as whom. Entry format: user hostname=(runas) commands. NOPASSWD: skips password prompt. Use visudo to edit — it validates syntax to prevent lockouts. Sudoers.d/ directory allows per-file additions.
What is the difference between /etc/passwd and /etc/shadow?
/etc/passwd is world-readable and contains user info (username, UID, GID, home, shell) but not passwords (shows x). /etc/shadow is root-only and contains hashed passwords with aging info. This split prevents brute-force attacks from unprivileged users.
How would you debug a service that fails to start?
systemctl status service-name shows last exit code and recent logs. journalctl -u service-name --no-pager -n 50 shows more log context. Check the ExecStart command manually, verify the user the service runs as has permission to its files, and check for port conflicts with ss -tlnp.
What is a zombie process?
A process that has exited but its entry remains in the process table because the parent hasn't called wait() to collect the exit status. Zombies consume a PID but no memory or CPU. Fix: send SIGCHLD to the parent process or kill the parent (zombies are automatically reaped by init/systemd). Large zombie counts indicate a buggy parent.
FAANG: How would you investigate a Linux server that is suddenly slow?
Start with top/htop to identify CPU/memory saturation. Check load average — above core count means CPU-bound. vmstat 1 3 shows context switches and IO wait. iostat -xz 1 3 identifies disk bottlenecks. sar -n DEV 1 3 checks network saturation. netstat -s | grep -i retrans shows TCP retransmits (network issues). strace -p PID shows syscalls for a specific process. The RED method: Rate, Errors, Duration for the system's exposed service.
Best Practices
- Never run production services as root — create a dedicated service user with minimal permissions (
useradd -r -s /bin/false svcname). - Use
systemctl enable svcso services auto-restart on reboot — don't rely on manual restarts after patching. - Rotate logs with logrotate — configure daily rotation with
compressanddelaycompressto prevent disk-fill incidents. - Use SSH key authentication everywhere — disable password auth in sshd_config (
PasswordAuthentication no). - Lock down sudo — grant only the specific commands each service account needs, not blanket NOPASSWD ALL.
- Set file permissions on private keys to 600 (
chmod 600 ~/.ssh/id_ed25519) — SSH refuses to use world-readable keys.
Common Mistakes
- Setting SSH private key permissions to 644 or 755 — SSH silently refuses to use keys that are group/world-readable, giving a confusing "permission denied" error.
- Using
kill -9first — always try SIGTERM first to allow clean shutdown. SIGKILL can leave locked files, corrupt data, or zombie processes. - Editing /etc/sudoers with a regular editor instead of
visudo— a syntax error in sudoers locks all users out of sudo permanently. - Running
chmod -R 777 /var/wwwto "fix permissions" — makes every file world-writable, trivially exploitable if any web-accessible script has code execution. - Cron jobs with relative paths (
script.shinstead of/home/user/script.sh) — cron has a minimal PATH that doesn't include user directories, so jobs silently fail.
Cheat Sheet
Practical Exercises
Create a user webuser with no login shell. Create /var/www/app/ owned by webuser:webuser with 755 on dirs and 644 on files. Write a systemd unit that runs a Python HTTP server as webuser. Enable and verify it survives a reboot.
Start PostgreSQL on a remote VM with listen_addresses = 'localhost' (no external access). Create an SSH tunnel ssh -L 5432:localhost:5432 user@remote. Connect from your local machine with psql -h localhost -p 5432. Verify you cannot connect without the tunnel.
Start a web app. Fill disk by writing dd if=/dev/zero of=/tmp/bigfile bs=1M count=4000. Observe what breaks. Identify the failing service via journalctl -u app -f. Free disk by deleting bigfile. Verify recovery. Write a 5-line post-mortem.