Becoming an AI Researcher
Topic 4 · Part 1 · How research works · ⏱ about 6 h

Literature Reviews & Finding Gaps

A systematic search strategy, snowballing through citations, the synthesis matrix, the kinds of research gaps, and checking that nobody has already done your idea.

Show:
In this topic
Beginner

A search strategy you can repeat

  1. Define the scope in one sentence: the problem, the setting and what you want to know ("parameter-efficient fine-tuning for low-resource language classification").
  2. List keywords and synonyms for each part (adapters, LoRA, prefix tuning; low-resource, under-resourced; classification, NLU).
  3. Search Google Scholar, Semantic Scholar and arXiv with combinations; sort by relevance and by recency.
  4. Snowball: from the best 5–10 papers, follow their references backwards (what they build on) and their citations forwards ("cited by": what built on them). Citation-map tools speed this up.
  5. Find surveys and recent workshop papers: they summarise the area and its open problems.
  6. Record what you searched and when; set alerts for new papers. Stop when new searches keep returning papers you already have.
Beginner

The synthesis matrix

A literature review is not a list of summaries ("Smith did X. Jones did Y."). It is a synthesis: what the field knows, how approaches compare, where they disagree, and what is missing. A synthesis matrix makes this concrete: papers as rows, the dimensions that matter as columns (task, data, languages, method family, evaluation, compute, findings). Patterns appear: every paper uses English; nobody reports variance; all evaluate on the same benchmark; two papers contradict each other.

Try it

Try it: find the gap in a synthesis matrix

Ten (fictional) papers on efficient fine-tuning. Sort and filter the matrix to find under-explored combinations, then choose the most promising gap.

Interactive: this part runs in your browser.

Beginner

Kinds of gaps

  • Setting gap: a method is untested in an important setting (low-resource languages, small data, a domain such as health records, edge devices).
  • Evaluation gap: results are measured with a weak metric, without variance, on contaminated or narrow benchmarks.
  • Explanation gap: something works and nobody knows why; or results conflict between papers.
  • Method gap: a known limitation (cost, instability, bias) with no good solution.
  • Data gap: no dataset exists to study an important question.

A gap is only worth filling if the answer would change what someone does or believes. "Nobody has tried method X on dataset Y" is a gap; it becomes a contribution if there is a reason to expect a different or important result.

Beginner

Has someone already done it?

Before investing months, search specifically for your exact idea with several phrasings, check the last 6–12 months of arXiv and the latest conference proceedings, and look at the "cited by" lists of the closest papers. Ask a supervisor or post a careful question to people in the area. If you find close work, that is not failure: read it carefully, and position your work as different in setting, evaluation or explanation, or build on it. Getting "scooped" happens to everyone; working on problems with several possible angles makes it less painful.

Code

Code: searching Semantic Scholar programmatically

The free Semantic Scholar API lets you collect candidate papers and their citation counts for your matrix.

Python · requests
import requests, csv
q = "parameter-efficient fine-tuning low-resource languages"
r = requests.get("https://api.semanticscholar.org/graph/v1/paper/search",
                 params={"query": q, "limit": 50, "fields": "title,year,venue,citationCount,externalIds,abstract"}).json()
with open("candidates.csv", "w", newline="") as f:
    w = csv.writer(f); w.writerow(["year", "venue", "citations", "title", "arxiv"])
    for p in sorted(r.get("data", []), key=lambda p: -(p.get("citationCount") or 0)):
        w.writerow([p.get("year"), p.get("venue"), p.get("citationCount"), p["title"], (p.get("externalIds") or {}).get("ArXiv")])
# then read abstracts (pass 1) and add the relevant ones to Zotero and your matrix
This code needs internet access (requests), which cannot run inside a web page. To run it, press Copy, then paste it into a new notebook in Google Colab ↗ (free; Runtime → Change runtime type → GPU for deep learning).
Check yourself

Quiz

1. Snowballing means…

2. A good literature review is…

3. When is "nobody has tried X on Y" a real contribution?

4. You find a 3-month-old preprint very close to your idea. Best response?

5. When can you stop searching?

Go deeper

📚 Further reading

Trusted, mostly free sources: official documentation, well-known textbooks and courses. When you want to confirm or go deeper on anything here, start with these.

Ready