Python
Topic 16 · Writing real programs · ⏱ about 4 h

Pythonic Python: Comprehensions, Generators, Decorators & Type Hints

Write Python the way experienced programmers do: comprehensions, iterators and generators, the most useful built-ins and standard-library tools, decorators, type hints and style.

Show:
In this topic
Beginner

Comprehensions

A comprehension builds a list, dictionary or set in one readable line: the result, then the loop, then an optional filter.

squares = [n * n for n in range(6)] # list evens = [n for n in range(20) if n % 2 == 0] # with a filter lengths = {w: len(w) for w in ["tea", "coffee"]} # dict initials = {name[0] for name in ["Sita", "Sam", "Leo"]} # set: {'S', 'L'}

Use them for simple transformations. If you need several lines of logic, a normal loop is clearer.

Try it

Try it: build a comprehension

Pick an expression, a source and a filter. See the comprehension, the loop it replaces, and the result.

Interactive: this part runs in your browser.

Beginner

Iterators and generators

Anything you can loop over is iterable. A generator produces values one at a time, only when asked, instead of building a whole list in memory. Write one with yield, or with a generator expression in round brackets:

def read_batches(rows, size): for i in range(0, len(rows), size): yield rows[i:i + size] # pause here and hand back one batch total = sum(x * x for x in range(10_000_000)) # no 10-million-item list is built

This is how data loaders feed huge datasets to a model in batches without loading everything at once.

Try it

Watch the code run: a generator pauses at yield

See how the generator runs only until the next yield, then waits until the loop asks for another value.

Interactive: this part runs in your browser.

Beginner

Built-ins and standard-library tools worth knowing

  • sum, min, max (with key=), any, all, sorted, reversed, zip, enumerate.
  • collections.Counter counts things; defaultdict(list) groups without setdefault.
  • itertools: product for grids of settings (great for hyperparameter searches), combinations, chain, islice.
  • functools.lru_cache remembers the results of slow function calls.
  • datetime, pathlib, json, random, statistics, time.
Beginner

Decorators

A decorator wraps a function to add behaviour, such as timing, caching, logging or registering a web route. @timer above a definition means f = timer(f):

import time, functools def timer(func): @functools.wraps(func) # keep the original name and docstring def wrapper(*args, **kwargs): start = time.perf_counter() result = func(*args, **kwargs) print(f"{func.__name__} took {time.perf_counter() - start:.3f}s") return result return wrapper

You will meet decorators everywhere: @app.route in Flask, @dataclass, @torch.no_grad(), @pytest.fixture.

Beginner

Type hints and style

Type hints say what a function expects and returns: def mean(xs: list[float]) -> float:. Python ignores them when running, but editors and tools such as mypy use them to catch mistakes, and they document your code for the next reader (often you, six months later).

Style: follow PEP 8 (four-space indents, snake_case names, CamelCase classes, lines under about 100 characters) and let a formatter such as Black or Ruff do it automatically. Choose names that say what a thing is: patient_ages, not x2.

Beginner

Code: Pythonic tools together

A comprehension, a generator, Counter, defaultdict, itertools.product, a decorator and type hints.

Python
import time, functools
from collections import Counter, defaultdict
from itertools import product

def timer(func):
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        start = time.perf_counter()
        result = func(*args, **kwargs)
        print(f"{func.__name__} took {time.perf_counter() - start:.4f}s")
        return result
    return wrapper

@timer
def sum_of_squares(n: int) -> int:
    return sum(x * x for x in range(n))        # a generator expression: no big list

print(sum_of_squares(1_000_000))

def batches(items: list, size: int):
    for i in range(0, len(items), size):
        yield items[i:i + size]

for b in batches(list(range(7)), 3):
    print("batch", b)

words = "to be or not to be".split()
print(Counter(words).most_common(2))

by_len = defaultdict(list)
for w in words:
    by_len[len(w)].append(w)
print(dict(by_len))

grid = [{"lr": lr, "depth": d} for lr, d in product([0.1, 0.01], [3, 5])]
print(len(grid), "settings:", grid[:2])
print(any(w.startswith("n") for w in words), all(len(w) <= 3 for w in words))
Open in playground ↗
Beginner

Practice: check yourself

Write the code, then press ✔ Check my answer: hidden tests run your code and tell you exactly what is right. ▶ Run just shows the output. Stuck? Use a hint; the solution is there if you need it.

Exercise 1. Comprehensions

In one line each, using comprehensions: squares = the squares of 1 to 10, evens = the even numbers in nums, and lengths = a dictionary of word → length for words.

Exercise 2. A generator

Write a generator function countdown(n) that yields n, n−1, …, 1. Then set result to list(countdown(5)).

Beginner

Summary

Key words

comprehension
a one-line loop that builds a list, set or dict
iterator
something you can loop over one item at a time
generator
a function that yields values one at a time, using little memory
decorator
a function that wraps another function to add behaviour
type hint
a note of the expected type: def f(x: int) -> str

Common mistakes

  • Comprehensions so long they no longer fit on a line: use a normal loop instead.
  • Expecting a generator to be reusable: once it is used up, it is empty.
  • Treating type hints as checks: Python does not enforce them at run time.
Check yourself

Quiz

1. What is [n * n for n in range(5) if n % 2 == 1]?

2. What is [n * n for n in range(5) if n % 2 == 1]?

3. Why use a generator for a huge dataset?

4. @timer above def f(): is the same as…

5. Does Python stop your program if a type hint is wrong?

Go deeper

📚 Further reading

Trusted, mostly free sources: official documentation, well-known textbooks and courses. When you want to confirm or go deeper on anything here, start with these.

Ready