Becoming an AI Researcher
Topic 15 · Part 3 · Writing & publishing · ⏱ about 4 h

Peer Review: Giving and Receiving It

How peer review works, how to write a useful review, the criteria reviewers apply, reading reviews of your own work, and writing a rebuttal that changes minds.

Show:
In this topic
Beginner

How review works at AI venues

At most AI conferences each paper gets 3–5 reviewers (often PhD students and researchers in the area), coordinated by an area chair who writes a meta-review and recommends a decision. Reviews are usually double-blind. Many venues have an author response (rebuttal) period of about a week; some (ICLR, TMLR) have open discussion on OpenReview. Acceptance rates at top conferences are often around 20–30%; workshops and journals vary. Randomness is real: the same paper can get different decisions from different reviewers, which is why resubmission after improvement is normal.

Beginner

Watch it: From submission to decision

A short narrated animation. Press ▶ Play, or step through it with ⏮ ⏭. Tick 🔊 narrate to hear each step.

Interactive: this part runs in your browser.

Beginner

Writing a useful review

  1. Summary of the paper in your own words (shows you understood it).
  2. Strengths: specific.
  3. Weaknesses: specific, prioritised, each with what would address it ("Results use one seed; reporting 3–5 seeds with CIs would show whether the 0.8-point gain is reliable").
  4. Questions for the authors whose answers could change your score.
  5. Recommendation with justification and your confidence.

Judge soundness (are claims supported?), significance (does it matter?), novelty (relative to prior work), clarity and reproducibility. Be kind and concrete: review the paper you were given, not the one you would have written. Reviewing is also the fastest way to improve your own writing: many students start by reviewing for workshops.

Try it

Try it: review a short paper

Read a one-page (fictional) workshop paper, score it on the standard criteria and pick its main weaknesses. Then compare with the area chair's meta-review.

Interactive: this part runs in your browser.

Beginner

Receiving reviews and writing a rebuttal

Read the reviews, then wait a day before replying. Sort every point into: misunderstanding (clarify, and fix the text that caused it), missing experiment or analysis (run it if feasible in the time; report the result even if mixed), disagreement (respond with evidence, politely), and out of scope (acknowledge, explain briefly).

  • Start with a short summary for the area chair: the main concerns and how you addressed them.
  • Answer each reviewer point by point, quoting the concern briefly. Lead with new evidence ("We ran 5 seeds: the gain is 1.9 ± 0.4, p = 0.003").
  • Be courteous even when a review is wrong; the area chair reads tone. Never argue with the reviewer's competence.
  • Promise only changes you will make, and make them in the camera-ready version.

If rejected: use the reviews to improve the paper and resubmit. Most good papers are rejected at least once.

Check yourself

Quiz

1. Who writes the meta-review and recommends the decision?

2. Which weakness statement is most useful?

3. A reviewer misunderstood your method. In the rebuttal you…

4. Your paper is rejected with useful reviews. Typical next step?

Go deeper

📚 Further reading

Trusted, mostly free sources: official documentation, well-known textbooks and courses. When you want to confirm or go deeper on anything here, start with these.

Ready