Research Design for Delphi Studies

A well-designed Delphi study starts with careful planning. Learn how to structure your research for maximum impact and validity.

Step 1: Define Your Research Question

Everything starts with a clear, focused research question. Your question should be:

• Specific: Narrow enough to be answerable but broad enough to be meaningful
• Relevant: Addresses a real gap in knowledge or practice
• Appropriate: Requires expert judgment rather than empirical data
• Feasible: Can be addressed within your time and resource constraints

Good Research Questions

• "What are the key competencies needed for data scientists in 2030?"
• "Which quality indicators should be prioritized in telehealth services?"
• "What are the most important factors influencing electric vehicle adoption?"

Poor Research Questions

• "What do experts think about technology?" (Too broad, vague)
• "How many people use smartphones?" (Empirical data, not expert opinion)
• "Is climate change real?" (Not appropriate for Delphi, requires scientific evidence)

Step 2: Choose Your Delphi Approach

Durvey.org supports two powerful Delphi approaches. Select the one that best fits your research timeline and methodology:

Classical Delphi

Multiple discrete rounds with controlled feedback between rounds. The moderator closes each round, analyzes results, and provides summary statistics before opening the next round. Best for rigorous academic research.

Duration: 6-12 weeks | Rounds: 2-3 structured rounds

Real-Time Delphi (Recommended)

Continuous feedback with live updates. Experts participate on their schedule while seeing real-time group statistics. As consensus emerges, participants can revise their responses. Ideal for modern, time-sensitive research with dynamic expert engagement.

Duration: 2-6 weeks | Participation: Continuous, flexible

Enhanced features: Real-Time mode enables rich interaction: experts comment with pro/contra sentiments, like other comments, explain their reasoning, and see consensus evolve naturally.

Step 3: Plan Your Panel Composition

Panel Size Guidelines

• 10-15 experts: Homogeneous panel (similar backgrounds/expertise)
• 15-30 experts: Heterogeneous panel (diverse perspectives and disciplines)
• 30-50+ experts: Large-scale studies or when stakeholder representation is critical

Expert Selection Criteria

• Relevant expertise and experience in the field
• Diversity of perspectives and backgrounds
• Availability and willingness to commit time
• Reputation and credibility in the domain
• Geographic or organizational diversity (if relevant)

Step 4: Design Your Questionnaire(s)

Your questionnaire design depends on your Delphi variant and round number:

Round 1: Exploration (Classical Delphi)

• Open-ended questions to gather diverse ideas
• Ask for examples, challenges, priorities, or predictions
• Keep it focused but allow creative responses
• Typically 3-7 broad questions

Example: "What do you consider the most critical challenges facing renewable energy adoption in the next decade?"

Round 2: Evaluation

• Structured items based on Round 1 analysis or literature
• Use rating scales (e.g., 1-5 Likert, 1-10 importance)
• Ask for likelihood, importance, feasibility, priority
• Typically 15-50 items

Example: "Rate the importance of each factor (1=Not Important, 5=Extremely Important)"

Round 3: Consensus Building

• Same questions as Round 2 with group statistics displayed
• Show mean, median, and their previous response
• Ask experts to reconsider or explain divergent views
• Optional: Request reasons for outlier positions

Example: "The group median was 4.2. You rated this item 2.0. Please reconsider or explain your reasoning."

Step 5: Define Consensus Criteria

Decide in advance how you will measure consensus. Common approaches:

• Percentage agreement: e.g., ≥70% of experts rate within a specific range
• Interquartile range: e.g., IQR ≤1 on a 5-point scale
• Standard deviation: e.g., SD ≤1.0
• Coefficient of variation: e.g., CV ≤0.5
• Median stability: Median doesn't change between rounds

Step 6: Plan Your Timeline

Typical Timeline (Classical Delphi)

• Weeks 1-2: Expert recruitment and confirmation
• Weeks 3-5: Round 1 distribution, completion, and analysis
• Weeks 6-8: Round 2 distribution, completion, and analysis
• Weeks 9-11: Round 3 distribution, completion, and analysis
• Weeks 12-16: Final analysis and reporting