Pythonic Python: Comprehensions, Generators, Decorators & Type Hints
Write Python the way experienced programmers do: comprehensions, iterators and generators, the most useful built-ins and standard-library tools, decorators, type hints and style.
Comprehensions
A comprehension builds a list, dictionary or set in one readable line: the result, then the loop, then an optional filter.
Use them for simple transformations. If you need several lines of logic, a normal loop is clearer.
Try it: build a comprehension
Pick an expression, a source and a filter. See the comprehension, the loop it replaces, and the result.
Interactive: this part runs in your browser.
Iterators and generators
Anything you can loop over is iterable. A generator produces values one at a time, only when asked, instead of building a whole list in memory. Write one with yield, or with a generator expression in round brackets:
This is how data loaders feed huge datasets to a model in batches without loading everything at once.
Watch the code run: a generator pauses at yield
See how the generator runs only until the next yield, then waits until the loop asks for another value.
Interactive: this part runs in your browser.
Built-ins and standard-library tools worth knowing
sum,min,max(withkey=),any,all,sorted,reversed,zip,enumerate.collections.Countercounts things;defaultdict(list)groups withoutsetdefault.itertools:productfor grids of settings (great for hyperparameter searches),combinations,chain,islice.functools.lru_cacheremembers the results of slow function calls.datetime,pathlib,json,random,statistics,time.
Decorators
A decorator wraps a function to add behaviour, such as timing, caching, logging or registering a web route. @timer above a definition means f = timer(f):
You will meet decorators everywhere: @app.route in Flask, @dataclass, @torch.no_grad(), @pytest.fixture.
Type hints and style
Type hints say what a function expects and returns: def mean(xs: list[float]) -> float:. Python ignores them when running, but editors and tools such as mypy use them to catch mistakes, and they document your code for the next reader (often you, six months later).
Style: follow PEP 8 (four-space indents, snake_case names, CamelCase classes, lines under about 100 characters) and let a formatter such as Black or Ruff do it automatically. Choose names that say what a thing is: patient_ages, not x2.
Code: Pythonic tools together
A comprehension, a generator, Counter, defaultdict, itertools.product, a decorator and type hints.
import time, functools
from collections import Counter, defaultdict
from itertools import product
def timer(func):
@functools.wraps(func)
def wrapper(*args, **kwargs):
start = time.perf_counter()
result = func(*args, **kwargs)
print(f"{func.__name__} took {time.perf_counter() - start:.4f}s")
return result
return wrapper
@timer
def sum_of_squares(n: int) -> int:
return sum(x * x for x in range(n)) # a generator expression: no big list
print(sum_of_squares(1_000_000))
def batches(items: list, size: int):
for i in range(0, len(items), size):
yield items[i:i + size]
for b in batches(list(range(7)), 3):
print("batch", b)
words = "to be or not to be".split()
print(Counter(words).most_common(2))
by_len = defaultdict(list)
for w in words:
by_len[len(w)].append(w)
print(dict(by_len))
grid = [{"lr": lr, "depth": d} for lr, d in product([0.1, 0.01], [3, 5])]
print(len(grid), "settings:", grid[:2])
print(any(w.startswith("n") for w in words), all(len(w) <= 3 for w in words))Practice: check yourself
Write the code, then press ✔ Check my answer: hidden tests run your code and tell you exactly what is right. ▶ Run just shows the output. Stuck? Use a hint; the solution is there if you need it.
In one line each, using comprehensions: squares = the squares of 1 to 10, evens = the even numbers in nums, and lengths = a dictionary of word → length for words.
Write a generator function countdown(n) that yields n, n−1, …, 1. Then set result to list(countdown(5)).
Summary
Key words
- comprehension
- a one-line loop that builds a list, set or dict
- iterator
- something you can loop over one item at a time
- generator
- a function that yields values one at a time, using little memory
- decorator
- a function that wraps another function to add behaviour
- type hint
- a note of the expected type:
def f(x: int) -> str
Common mistakes
- Comprehensions so long they no longer fit on a line: use a normal loop instead.
- Expecting a generator to be reusable: once it is used up, it is empty.
- Treating type hints as checks: Python does not enforce them at run time.
Quiz
1. What is [n * n for n in range(5) if n % 2 == 1]?
2. What is [n * n for n in range(5) if n % 2 == 1]?
3. Why use a generator for a huge dataset?
4. @timer above def f(): is the same as…
5. Does Python stop your program if a type hint is wrong?
📚 Further reading
Trusted, mostly free sources: official documentation, well-known textbooks and courses. When you want to confirm or go deeper on anything here, start with these.
- Python HOWTO: Functional Programming ↗ iterators, generators, itertools and functools
- The Python Tutorial: Classes ↗ classes, inheritance, iterators and generators
- PEP 8: Style Guide for Python Code ↗ the official Python style guide