# Asyncio in Python: async/await, TaskGroup, Threads, Multiprocessing and the GIL

URL: https://skillaria.top/en/guides/python/python-asyncio-async-await-threads-gil

Asyncio is Python's built-in library for asynchronous code: functions declared with async def (coroutines) pause at await, and while one waits for a network reply the event loop runs other tasks, so hundreds of requests run at the same time in one thread. For waiting on the network or disk, asyncio or threads (threading) fit; for heavy calculations use processes (multiprocessing), because the GIL stops threads of regular CPython from computing in parallel.

Asyncio is Python's built-in library for asynchronous code. Functions declared with `async def` can pause at `await`, and while one of them waits for a network reply, the event loop runs others. This is how hundreds of requests run at the same time in a single thread. But asyncio is only one of three tools: there are also threads (`threading`) and processes (`multiprocessing`). Let us look at how asyncio works in Python, what to choose for your task and what the GIL has to do with it.

## Waiting and computing are different tasks

Without this distinction nothing that follows will make sense. A program is slow for one of two reasons:

- **I/O-bound** — it waits for input/output: an API response, a database, reading from disk. The processor sits idle meanwhile.
- **CPU-bound** — it computes: processes images, loops over millions of records. The processor is busy.

For waiting, asyncio or threads are suitable: while one operation waits, another runs. For calculations you need real parallel computing on several cores — processes. Confusing them is the main mistake: threads will not speed up calculations in plain Python, and processes just to make three HTTP requests are like using a cannon to shoot a sparrow.

## What asyncio is: coroutines and the event loop

Asynchronous code is built on two keywords. `async def` declares a coroutine — a function that can pause. Calling it does not execute the body but creates a coroutine object. `await` inside it says: "here I will wait, and you go and do other tasks meanwhile". Everything is managed by the event loop, which is started with `asyncio.run()`.

```
import asyncio

async def fetch_rate(currency):
    await asyncio.sleep(1)          # simulates a request to an API
    return {"USD": 92.5, "EUR": 100.1, "CNY": 12.7}[currency]

async def main():
    rate = await fetch_rate("USD")
    print(rate)

asyncio.run(main())  # 92.5
```

`asyncio.sleep()` is an asynchronous pause: it hands control back to the event loop. An ordinary `time.sleep()` inside a coroutine would block the whole loop, and the other tasks would stand still.

### Sequentially or at the same time

If you write `await` in a loop, the requests go one after another — three currencies take three seconds. For them to run simultaneously, they must be started as tasks.

```
import asyncio, time

async def main():
    start = time.perf_counter()
    for cur in ("USD", "EUR", "CNY"):
        await fetch_rate(cur)                       # one after another
    print(f"{time.perf_counter() - start:.1f} s")   # 3.0 s

asyncio.run(main())
```

## TaskGroup: start together and wait for all

The modern approach (Python 3.11+) is a task group. Inside an `async with` block each task starts right away through `tg.create_task()`, and the block is exited when all of them have finished.

```
async def main():
    start = time.perf_counter()
    async with asyncio.TaskGroup() as tg:
        tasks = {cur: tg.create_task(fetch_rate(cur)) for cur in ("USD", "EUR", "CNY")}
    rates = {cur: t.result() for cur, t in tasks.items()}
    print(rates, f"{time.perf_counter() - start:.1f} s")   # ... 1.0 s

asyncio.run(main())
```

TaskGroup is strict: if one task fails, the others are cancelled, and all the errors are collected into an `ExceptionGroup`. They are caught with `except*`:

```
async def main():
    try:
        async with asyncio.TaskGroup() as tg:
            tg.create_task(fetch_rate("USD"))
            tg.create_task(fetch_rate("XXX"))   # KeyError
    except* KeyError as eg:
        print("unknown currency:", eg.exceptions)
```

### asyncio.gather and create_task

Before TaskGroup, the standard was `asyncio.gather()`: it runs coroutines at the same time and returns the results in the same order. There is still a lot of it in existing code.

```
async def main():
    usd, eur, cny = await asyncio.gather(
        fetch_rate("USD"), fetch_rate("EUR"), fetch_rate("CNY")
    )
    # with return_exceptions=True, errors are returned in the list instead of stopping everything
```

`asyncio.create_task()` starts a coroutine in the background without a group. You must keep a reference to the task, otherwise the garbage collector may remove it, and nobody will see its errors. That is why TaskGroup is more reliable for new projects.

### Cancelling tasks and timeouts

```
async def main():
    task = asyncio.create_task(fetch_rate("USD"))
    task.cancel()                     # CancelledError will be raised inside the task
    try:
        await task
    except asyncio.CancelledError:
        print("cancelled")

    try:
        async with asyncio.timeout(0.5):   # Python 3.11+
            await fetch_rate("EUR")
    except TimeoutError:
        print("took too long")
```

In older versions, `asyncio.wait_for(coro, timeout=0.5)` is used instead of `asyncio.timeout()`. There is also `asyncio.wait()`, which returns sets of finished and unfinished tasks and can wait for the first one (`return_when=asyncio.FIRST_COMPLETED`).

In Python 3.15, as the course shows, TaskGroup gained a `tg.cancel()` method: it cancels all tasks of the group at once. This covers the "first one wins" scenario: ask three mirror services, take the first answer and cancel the rest. Previously this required throwing a custom signal exception or cancelling tasks by hand.

## Synchronisation: Semaphore, Lock, Event, Queue

asyncio has its own primitives that work with `await`. The most useful is `Semaphore`: it limits the number of simultaneous operations, so that you do not overload someone else's API with a thousand requests.

```
sem = asyncio.Semaphore(10)

async def limited(url):
    async with sem:              # no more than 10 at a time
        await asyncio.sleep(0.1) # the request would be here
        return url
```

- `asyncio.Lock` — only one coroutine in a critical section.
- `asyncio.Event` — one coroutine waits on `await event.wait()`, another signals with `event.set()`.
- `asyncio.Queue` — a producer–consumer queue: `await q.put()`, `await q.get()`, `q.task_done()`.

## A trap: forgotten await

The most common mistake is calling a coroutine without `await`:

```
async def main():
    rate = fetch_rate("USD")    # forgot await
    print(rate * 2)             # TypeError
# RuntimeWarning: coroutine 'fetch_rate' was never awaited
```

Python did not execute the function, it only created a coroutine object. If you see "was never awaited" in the logs, look for the missing `await`. The second typical mistake is a blocking call (`time.sleep`, `requests.get`) inside a coroutine. For HTTP use asynchronous libraries (httpx, aiohttp), and move unavoidable synchronous code into a thread with `await asyncio.to_thread(func, arg)`.

## Multithreading in Python: the threading module

A thread is an independent line of execution inside one process; threads share the same memory. For waiting on the network they work very well and do not require rewriting the code as async.

```
import threading, time

def download(name):
    time.sleep(1)               # waiting for the network
    print(name, "done")

threads = [threading.Thread(target=download, args=(n,)) for n in ("a", "b", "c")]
for t in threads:
    t.start()
for t in threads:
    t.join()                    # everything in about 1 second
```

Shared memory is both an advantage and a source of errors: if two threads change the same variable, a `threading.Lock` is needed. It is usually simpler not to manage threads by hand, but to take a pool.

### ThreadPoolExecutor

`concurrent.futures.ThreadPoolExecutor` creates a pool of threads and hands tasks to them. `map` returns the results in the original order.

```
from concurrent.futures import ThreadPoolExecutor
import time

def fetch(url):
    time.sleep(1)
    return f"{url}: 200"

urls = [f"https://example.com/{i}" for i in range(5)]
with ThreadPoolExecutor(max_workers=5) as pool:
    for result in pool.map(fetch, urls):
        print(result)           # everything in about 1 second
```

## What the GIL is and why threads do not speed up calculations

The GIL (Global Interpreter Lock) is the global lock of the CPython interpreter. At any moment only one thread executes Python bytecode. While a thread waits for input/output, it releases the GIL, which is why threads are useful for network work. But on pure computations four threads run no faster than one: they hand the lock to each other in turn.

```
from concurrent.futures import ThreadPoolExecutor

def heavy(n):
    return sum(i * i for i in range(n))

with ThreadPoolExecutor(4) as pool:
    list(pool.map(heavy, [5_000_000] * 4))   # roughly the same as sequential
```

C-based libraries (NumPy and similar) often release the GIL during heavy operations, so threads do speed them up.

## multiprocessing: real parallelism

Each process has its own interpreter and its own GIL, so processes compute on different cores at the same time. The price is separate memory: data is passed between processes through serialisation (pickle), and starting a process is more expensive than starting a thread.

```
from concurrent.futures import ProcessPoolExecutor

def heavy(n):
    return sum(i * i for i in range(n))

if __name__ == "__main__":      # required on Windows and macOS
    with ProcessPoolExecutor() as pool:
        print(list(pool.map(heavy, [5_000_000] * 4)))
```

Without the `if __name__ == "__main__"` check, the child processes would start creating new processes when the module is imported. There is also a low-level `multiprocessing` module with `Process`, `Pool` and `Queue`, but `ProcessPoolExecutor` is simpler and has the same interface as a thread pool.

## What to choose: asyncio, threading or multiprocessing

| Task | Tool | Why |
| --- | --- | --- |

| Download 200 pages, many API requests | asyncio (httpx/aiohttp) or ThreadPoolExecutor | waiting on the network, the processor is free |

| Several requests inside an already asynchronous program | TaskGroup | the code is already async |

| A synchronous library with no async version | threads, `asyncio.to_thread` | no need to rewrite the code |

| A heavy calculation in plain Python | ProcessPoolExecutor | bypasses the GIL, uses all cores |

## Python without the GIL and subinterpreters: what is new

Since Python 3.13 there is a separate build without the GIL (free-threading, PEP 703) — it is marked with the letter t in its name, for example `python3.15t`. In 3.13 it was experimental; since 3.14 (PEP 779) it is officially supported. In it, threads compute in parallel on different cores. But by default you still get regular Python with the GIL: single-threaded code in the free-threaded build runs a little slower, and not all libraries are ready for it.

```
import sys
print(sys._is_gil_enabled())   # False in the free-threaded build
```

A second route to parallel computing appeared in 3.14 (PEP 734) — subinterpreters in the `concurrent.interpreters` module: several isolated interpreters in one process, each with its own GIL. They are most conveniently used through `InterpreterPoolExecutor`, which looks the same as a thread pool. In a measurement made for the course, four heavy tasks took 0.39 s against 1.03 s on one thread.

## Summary

Asyncio in Python is a tool for waiting tasks: coroutines defined with `async def`, pauses at `await`, simultaneous start through TaskGroup or `gather`. Threads solve the same problem without rewriting the code as async, while heavy calculations need processes, because the GIL stops threads of regular CPython from computing in parallel. Free-threading and subinterpreters are gradually changing this picture, but the basic rule "waiting — asyncio or threads, computing — processes" still holds. The course covers all of this in practice: the expense tracker fetches currency rates in parallel in a second instead of three.

## FAQ

**Can I call an async function from ordinary code?**

Yes, with asyncio.run(coro()) — it creates an event loop, runs the coroutine and closes the loop. Inside an already running loop you cannot call asyncio.run; there you use await.

**Does asyncio run in several threads?**

No, the event loop runs all the coroutines in one thread and switches between them only at await points. So there are fewer data races than with threads, but one blocking operation stops everything.

**Do I need to install asyncio with pip?**

No, asyncio is part of the Python 3 standard library; it is enough to write import asyncio. The asyncio package on PyPI is obsolete and should not be installed.

**How do I run a heavy function from asyncio without blocking the loop?**

Use loop.run_in_executor with a ProcessPoolExecutor for calculations, or asyncio.to_thread for blocking input/output; the event loop keeps serving the other tasks.

**How is a coroutine different from a task (Task)?**

A coroutine is an object that does nothing until someone awaits it. A task wraps a coroutine and immediately schedules it on the event loop, so it runs concurrently with other code.


---
Source: Skillaria.Top (https://skillaria.top)