Becoming an AI Researcher
Topic 3 · Part 1 · How research works · ⏱ about 4 h

Reading Papers Efficiently

Where to find papers, the three-pass reading method, critical reading checklists, note-taking that compounds, and reproducing a paper as the best training there is.

Show:
In this topic
Beginner

Where AI papers live

  • arXiv (cs.LG, cs.CL, cs.CV, cs.AI, stat.ML): preprints, often before or alongside publication. Not peer-reviewed: judge them on their content.
  • Conference proceedings: NeurIPS, ICML, ICLR (reviews public on OpenReview), ACL/EMNLP/NAACL (the ACL Anthology, all free), CVPR/ICCV/ECCV (CVF open access), AAAI, IJCAI, KDD, MICCAI, CHI.
  • Journals: JMLR, TMLR (open, rolling), TPAMI, Nature Machine Intelligence, and domain journals (e.g. medical informatics).
  • Search and discovery: Google Scholar ("cited by" and alerts), Semantic Scholar (TL;DRs, citation graphs, research feeds), Connected Papers / Litmaps (visual citation maps), and your library's databases.

Read the OpenReview discussions of papers in your area: seeing what reviewers criticise is one of the fastest ways to learn what good research looks like.

Beginner

Watch it: Reading a paper in three passes

A short narrated animation. Press ▶ Play, or step through it with ⏮ ⏭. Tick 🔊 narrate to hear each step.

Interactive: this part runs in your browser.

Beginner

The three-pass method

Srinivasan Keshav's classic advice: never read a paper start to finish on the first go.

  1. Pass 1 (5–10 minutes): title, abstract, introduction, section headings, figures, conclusion. Answer the "five Cs": Category (what type of paper), Context (which work it builds on), Correctness (do the assumptions look valid), Contributions, Clarity. Most papers stop here.
  2. Pass 2 (about an hour): read properly but skip proofs. Study figures and tables carefully (axes, error bars, baselines). Note references to read. You should be able to summarise the main argument and evidence to a colleague.
  3. Pass 3 (several hours): for papers central to your work. "Re-implement" the paper in your head (or in code): make the same assumptions and recreate the work, challenging every step. You will find the hidden assumptions, the weaknesses and your own next ideas.
Try it

Try it: a critical first pass

Read a short (fictional) abstract and results table. Mark every claim you would want checked before believing it.

Interactive: this part runs in your browser.

Beginner

A critical reading checklist

  • What exactly is claimed, and is it supported by the experiments shown?
  • Are the baselines strong, recent and tuned as carefully as the proposed method?
  • How many runs/seeds? Are there error bars or confidence intervals? Is the gain larger than the variation?
  • Are the datasets and splits standard and leak-free? Could test data have been seen in pretraining?
  • Do the ablations show that each component matters?
  • Is compute comparable between methods?
  • Are limitations and failure cases discussed? Is code available?

Being critical is not being dismissive: almost every paper has a useful idea. Your notes should capture both the idea and the doubt.

Beginner

Notes that compound

Use a reference manager (Zotero is free and excellent) from day one, with the browser connector, so every paper you read is saved with its citation. For each paper you read past pass 1, keep a short structured note:

Citation key: smith2024lowres One sentence: Adapters beat full fine-tuning for languages with < 10k examples. Problem / gap: … Method: … Evidence: 3 datasets, 3 seeds, no CIs Strengths: … Doubts: baselines not tuned; compute not matched Relevance to me: could test on Nepali; their code is public Follow-up refs: [x], [y]

After a few months these notes become your literature review, your related-work section and your PhD proposal. Read regularly: a few papers a week, every week, beats occasional binges.

Beginner

Reproduce a paper: the best training there is

Re-implementing a paper's main result teaches more than reading twenty. Choose a recent, well-scoped paper with a public dataset and, ideally, code you can compare against. Reproduce one key number, then vary something the authors did not (another dataset, fewer labels, another language). Many good first papers start exactly this way: "we reproduced X and found it fails when Y", or "X works, and here is why". Initiatives like the ML Reproducibility Challenge publish such reports.

Check yourself

Quiz

1. What do you do in the first pass of a paper?

2. arXiv preprints are…

3. A paper reports a 0.4-point gain from a single run with no error bars. You note…

4. Why use a reference manager from day one?

5. Why is reproducing a paper valuable for a new researcher?

Go deeper

📚 Further reading

Trusted, mostly free sources: official documentation, well-known textbooks and courses. When you want to confirm or go deeper on anything here, start with these.

Ready