Performance & Profiling

10 / 10 Part of python advanced

🎓 Advanced · Topic 10

Performance & Profiling

Measure before you optimise. Learn to profile Python code, identify real bottlenecks, and apply targeted techniques — from algorithm choice to C extensions and memory management.

⏱ ~60 min
🔴 Advanced
🚀 Speed

Rule #1 — Measure First

Never optimise without data. Premature optimisation wastes time and often makes code harder to read without meaningful speedup. Use timeit for micro-benchmarks and cProfile for whole-program profiling.

timing.py

import timeit

# Compare list vs generator for summing squares
list_time = timeit.timeit(
    "sum([x**2 for x in range(10000)])", number=1000
)
gen_time = timeit.timeit(
    "sum(x**2 for x in range(10000))", number=1000
)
print(f"list: {list_time:.3f}s | gen: {gen_time:.3f}s")

Terminal — cProfile

# Profile an entire script
python -m cProfile -s cumtime myscript.py

# Or profile inline
import cProfile
cProfile.run("my_function()", sort="cumulative")

Key Optimisation Techniques

optimisations.py

# ✅ Use set for membership tests — O(1) vs O(n)
valid_ids = {1, 2, 3, 4, 5}
if user_id in valid_ids: ...   # fast

# ✅ Local variable lookup is faster than global
def process(items):
    _append = [].append    # cache method locally
    for item in items:
        _append(item * 2)

# ✅ str.join beats repeated string concatenation
parts   = ["a", "b", "c"]
result  = "".join(parts)     # ✅ O(n)
# bad = parts[0] + parts[1] + parts[2]  # ❌ O(n²)

# ✅ slots reduce memory per instance
class Point:
    __slots__ = ("x", "y")
    def __init__(self, x, y):
        self.x, self.y = x, y

numpy Vectorisation

For numerical work, numpy operations run in C — typically 50–500× faster than equivalent Python loops. The key: operate on entire arrays, not element-by-element.

numpy_speed.py

import numpy as np

N = 1_000_000
data = np.arange(N, dtype=np.float64)

# ❌ Python loop — slow
result = [x ** 2 for x in data]   # ~120ms

# ✅ numpy vectorised — fast
result = data ** 2                   # ~1ms — 120× faster

# Avoid copying — use in-place ops
data *= 2           # modifies array in place
np.sqrt(data, out=data)  # writes result into same buffer

Performance Toolbox

cProfile / snakeviz

Find which functions consume most time with call-graph visualisation

memory_profiler

Line-by-line memory usage — catch leaks and over-allocation

PyPy

Drop-in CPython replacement with JIT — 5–50× faster for pure Python

Cython / ctypes

Compile Python to C or call C libraries for near-native speed

numba

JIT-compile numeric functions with @jit — no C required

gc module

Control garbage collection, detect cycles, tune collection thresholds

🏆

Advanced Course Complete!

You've mastered 10 advanced Python topics — from decorators and metaclasses to async programming, design patterns, and performance tuning. You now have the tools to build production-grade Python systems.

✅ 10 Advanced Topics

💻 50+ Code Examples

🏗️ Production Ready

Menu

Performance & Profiling

Rule #1 — Measure First

Key Optimisation Techniques

numpy Vectorisation

Performance Toolbox

Advanced Course Complete!