NumPy Arrays & Vectorization
NumPy (Numerical Python) is the foundational core of the entire Python Data Science and Artificial Intelligence ecosystem (powering Pandas, Scikit-Learn, TensorFlow, and PyTorch).
Why are Standard Python Lists Slow for Numerical Data?
A standard Python list does not store raw numbers directly. It stores an array of pointers pointing to scattered PyObject instances located all over heap memory. Every single mathematical operation requires type-checking, pointer dereferencing, and memory lookups.
The NumPy Advantage: C-Contiguous Memory & SIMD Vectorization:
A NumPy ndarray (N-dimensional array) stores homogeneous data (e.g. all 64-bit floats) in a single continuous block of memory. This delivers three massive advantages:
- CPU Cache Locality: Sequential memory bytes are loaded directly into CPU L1/L2 caches in a single memory fetch.
- SIMD (Single Instruction, Multiple Data): Modern CPUs execute mathematical operations on 4 to 8 numbers simultaneously in hardware vector registers.
- Zero Type-Checking Overhead: Since all elements share the exact same data type (
dtype), calculations run at raw compiled C speeds (50x to 100x faster).
import sys
import time
# 1. Memory and Speed Comparison between Python List and NumPy Array:
python_list = list(range(1_000_000))
# Time list comprehension addition:
start = time.perf_counter()
list_result = [x + 2 for x in python_list]
list_time = (time.perf_counter() - start) * 1000
print(f"⏱️ Python List Loop (1M items): {list_time:.2f} ms")
print(f"📦 Python List Memory Overhead: {sys.getsizeof(python_list):,} bytes (~8.0 MB)")
# Note: In a real environment with 'import numpy as np':
# np_arr = np.arange(1_000_000)
# np_result = np_arr + 2 # Vectorized operation executes in ~1.2 ms (70x faster!)Vectorization is the practice of replacing explicit Python for loops with batch array expressions (e.g. arr * 2). Vectorized operations delegate loop execution entirely to compiled C and Fortran binaries under the hood.
NumPy arrays can represent 1D vectors, 2D matrices (spreadsheets/images), and 3D/4D tensors (video frames and neural network batch layers):
ndim: Number of dimensions (axes).shape: Tuple representing the size along each dimension (e.g.(rows, cols)).dtype: Data type descriptor (e.g.int32,float64,bool_).- Boolean Masking: Filtering array elements using conditional expressions (e.g.
arr[arr > 50]) without writing loops!
# Matrix Operations & Boolean Masking Simulation:
matrix_2d = [
[10, 25, 40],
[55, 70, 85],
[90, 15, 30]
]
print("--- 2D Matrix (3x3) ---")
for row in matrix_2d:
print(row)
# Simulated Slicing: Extract Row 1, Columns 1 to 2 -> [70, 85]
extracted_submatrix = [matrix_2d[1][1], matrix_2d[1][2]]
print("\nSlicing [Row 1, Cols 1..2]:", extracted_submatrix)
# Simulated Boolean Masking: Find all values > 50
flattened = [val for row in matrix_2d for val in row]
greater_than_50 = [val for val in flattened if val > 50]
print("Boolean Mask [values > 50]: ", greater_than_50)In NumPy, 2D slicing uses comma notation: arr[0:2, 1:3] (rows 0 to 1, columns 1 to 2). Unlike Python lists where slicing creates a new copy, NumPy slices are memory views that point to the original array without copying bytes!
Broadcasting describes how NumPy handles arithmetic operations between arrays of different shapes without creating unnecessary copies in memory.
The 2 Fundamental Rules of Broadcasting:
When operating on two arrays, NumPy compares their shape dimensions from right to left (trailing dimensions first). Two dimensions are compatible if:
- They are strictly equal in size, OR
- One of the dimensions is 1 (in which case NumPy stretches the dimension of size 1 to match the other array).
# Broadcasting Rules Demonstration:
shape_pairs = [
((3, 3), (1, 3), "Compatible: Dim 1 stretched to 3 -> Result: (3, 3) ✅"),
((4, 3), (3,), "Compatible: Rightmost dims match -> Result: (4, 3) ✅"),
((3, 4), (3, 5), "INCOMPATIBLE: 4 != 5 (Raises ValueError: operands could not be broadcast) ❌")
]
print("--- 📐 NumPy Broadcasting Dimension Compatibility Checks ---")
for shape_a, shape_b, status in shape_pairs:
print(f"Shape A: {str(shape_a):8} + Shape B: {str(shape_b):8} -> {status}")Broadcasting does not actually duplicate data in memory; it iterates over the same single row or column repeatedly using a stride offset of 0 bytes!
In NumPy, the asterisk operator a * b performs element-wise multiplication. To perform true linear algebra matrix dot-products, you must use the matrix multiplication operator a @ b (or np.dot(a, b)).
Simulate a NumPy vectorized operation: Write a function normalize_scores(scores) that subtracts the minimum and divides by (max - min) to scale values between 0.0 and 1.0.
def normalize_scores(scores):
min_val = min(scores)
max_val = max(scores)
rng = max_val - min_val
return [(s - min_val) / rng for s in scores]
raw_scores = [45, 80, 60, 100, 20]
print("Normalized Scores (0.0 to 1.0):")
print([round(s, 2) for s in normalize_scores(raw_scores)])Q What is the difference between np.reshape() and np.ravel() / np.flatten()?
np.reshape(new_shape) changes array dimensions without altering data. np.ravel() returns a flattened 1D array as a memory view (zero copy). np.flatten() returns a completely new copy of the flattened 1D array in memory.
Q What are Universal Functions (ufuncs) in NumPy?
Ufuncs are fast, element-wise compiled C functions that support broadcasting, type casting, and reduction (e.g. np.add, np.sin, np.exp, np.log).
Q What does axis=0 vs axis=1 mean in NumPy aggregations?
axis=0 performs operations vertically down columns (collapsing rows). axis=1 performs operations horizontally across rows (collapsing columns).