Concurrency — Threads, Processes & Async

04 / 10 Part of python advanced

🎓 Advanced · Topic 04

Concurrency — Threads, Processes & Async

Speed up programs by running work simultaneously. Understand the GIL, when to use threads vs processes vs async, and write non-blocking I/O with asyncio.

⏱ ~70 min
🔴 Advanced
⚡ Performance

The GIL & Choosing the Right Model

Python's Global Interpreter Lock (GIL) prevents true parallel thread execution. Choose the right tool based on your bottleneck:

threading

Best for I/O-bound tasks — network calls, file reads, database queries. GIL is released during I/O.

multiprocessing

Best for CPU-bound tasks — image processing, number crunching. Each process has its own GIL.

asyncio

Best for high-concurrency I/O — thousands of simultaneous network connections, event-driven servers.

Threading

threading_example.py

from concurrent.futures import ThreadPoolExecutor
import urllib.request

def fetch(url):
    with urllib.request.urlopen(url) as r:
        return len(r.read())

urls = [
    "https://python.org",
    "https://pypi.org",
    "https://docs.python.org",
]

with ThreadPoolExecutor(max_workers=3) as pool:
    sizes = list(pool.map(fetch, urls))
print(sizes)   # all 3 fetched concurrently

Multiprocessing

multiprocessing_example.py

from concurrent.futures import ProcessPoolExecutor

def is_prime(n):
    if n < 2: return False
    return all(n % i != 0 for i in range(2, int(n**0.5) + 1))

numbers = range(10_000, 10_100)

if __name__ == "__main__":
    with ProcessPoolExecutor() as pool:
        primes = [n for n, p in zip(numbers, pool.map(is_prime, numbers)) if p]
    print(primes)

asyncio — async / await

Single-threaded concurrency driven by an event loop. await suspends the current coroutine while waiting for I/O, allowing other coroutines to run.

asyncio_example.py

import asyncio

async def fetch_data(name, delay):
    print(f"Start {name}")
    await asyncio.sleep(delay)    # non-blocking wait
    print(f"Done  {name}")
    return f"{name}: result"

async def main():
    # Run all coroutines concurrently
    results = await asyncio.gather(
        fetch_data("A", 1),
        fetch_data("B", 2),
        fetch_data("C", 1),
    )
    print(results)
    # Total time ≈ 2s, not 4s

asyncio.run(main())

💡 async all the way down

Once you use async/await, the calling code must also be async. Use asyncio.run() as the single entry point at the top level.

Menu

Concurrency — Threads, Processes & Async

The GIL & Choosing the Right Model

Threading

Multiprocessing

asyncio — async / await