How to Conduct a Delphi Study: A Step-by-Step Guide

The Delphi Method is one of the most powerful tools in applied research — but it's also one of the most commonly misunderstood. Researchers often start with the wrong question, recruit the wrong panelists, or design their rounds without a clear consensus criterion. The result is data that's hard to interpret and a methodology chapter that doesn't hold up to peer review.

This guide covers how to conduct a Delphi study from start to finish: from defining your research question to writing up results for publication. Each step includes common mistakes and what to do instead.

Before designing anything, verify that Delphi is actually the appropriate method for your question. Delphi is designed for situations where:

Delphi is not the right tool if a systematic literature review or meta-analysis could answer your question, or if you need to measure attitudes or experiences in a general population — that's a survey or qualitative study.

A vague research question produces a vague Delphi study. Before recruiting a single panelist, you need a research question specific enough to determine what items go into your questionnaire, what "consensus" means in your context, how many rounds you'll need, and what you'll do with the output.

Weak: "What are best practices in Delphi research?"

Strong: "Which criteria should be included in a minimum reporting standard for Delphi studies in nursing education, as rated by expert consensus?"

The more specific your question, the more defensible your panel selection, item list, and consensus criteria will be.

Panel selection is the part of Delphi methodology that reviewers scrutinize most closely — and where most researchers are too vague.

How many experts do you need?

There's no universal answer, but 15–30 panelists is a well-documented range for most academic Delphi studies. Too few (under 10) and your consensus data lacks credibility; too many (over 50) and managing the study becomes logistically difficult and attrition compounds across rounds.

What counts as expertise?

Define this in writing before you start recruiting, and report it explicitly in your methodology. Criteria might include number of publications in the field, years of clinical or professional experience, institutional affiliation, geographic representation, or disciplinary background.

How diverse does the panel need to be?

Delphi consensus is only meaningful if the panel represents the range of perspectives relevant to your question. A panel of 25 researchers from the same country and the same research tradition will produce narrower consensus than a geographically and methodologically diverse panel — and reviewers will notice if your panel demographics are thin.

Build in attrition buffer. Expect 10–25% drop-off between rounds. Recruit 20–30% more panelists than your target final panel size.

The first round questionnaire is the foundation of your entire study. Spend more time here than you think you need to.

Where do items come from?

The most methodologically defensible approach is to generate items from a systematic literature review or qualitative interviews with subject-matter experts before the Delphi begins. This separates the item-generation phase from the consensus-building phase, which strengthens validity.

Alternatively, you can use an open-ended first round where panelists generate items themselves — then synthesize those into a structured second-round questionnaire. This is more work but produces items grounded directly in expert knowledge.

How should items be phrased?

Each item should address a single, unambiguous concept. Avoid double-barreled items such as "this intervention is effective and cost-efficient" — panelists will rate the two parts differently, and you won't know which they were agreeing with.

What response format should you use?

A 9-point Likert scale (1 = strongly disagree, 9 = strongly agree) with clearly labeled anchors is the most common format for Delphi studies and maps directly to standard consensus metrics. 5-point scales work but give you less nuance; VAS scales are harder to analyze.

Pilot test. Have 2–3 colleagues outside the panel read through the questionnaire and flag anything that's confusing or ambiguous. Fix it before launch — you can't revise items mid-study without invalidating your data.

This is non-negotiable: your consensus threshold must be defined before you collect data, not after. Post-hoc consensus criteria are a red flag for reviewers and a common reason for manuscript rejection.

Common consensus thresholds:

Also define what happens to items that don't reach consensus. Are they dropped after a set number of rounds? Modified and re-rated? Reported as "no consensus reached"? Document all of this in your pre-study protocol, and report it in your methods section verbatim.

Provide structured feedback between rounds. This is what makes Delphi different from a repeated survey. After each round, every panelist receives their own rating for each item, the group's statistical distribution (median, IQR, percentage agreement), and anonymized qualitative comments from the group if collected.

This feedback is what allows panelists to reconsider their positions in light of peer perspectives, without social pressure or hierarchy distorting their re-rating.

Most Delphi studies run 2–3 rounds. A second round is almost always necessary; a third is often useful if several items are near but not at the consensus threshold. Beyond three rounds, response fatigue becomes a serious risk.

Track response rates per round — not just total completion, but which specific panelists responded in each round. You'll need this for your methods section.

Once consensus is reached, or the final round is complete, analyze your results systematically. For each item, report the median and IQR across all rounds, the percentage agreement per round, and the consensus status: reached, not reached, or borderline.

If you collected qualitative comments, summarize the key themes thematically — don't just list them. The synthesis of quantitative consensus and qualitative reasoning is what makes a Delphi study analytically rich.

For reporting standards, follow a recognized framework such as CREDES — it sets out exactly what your methods and results sections need to include for peer-reviewed publication.

Your methods section should allow a reader to fully replicate your study. That means reporting the rationale for Delphi, panel composition and recruitment, questionnaire development, consensus criteria, round structure, response rates per round, and the statistical approach used.

Getting each of these eight steps right is what separates a Delphi study that survives peer review from one that comes back with a request to revise and resubmit.