Phase Two: Execution

Setting up your Delphi study inside Durvey.org ensures that everything is structured, traceable, and ready to produce reliable results. If you have never created a Delphi before, don't worry – every step is designed to guide you and prevent common mistakes.

What You'll Learn: Clear explanations about each execution step, why it matters, and how it will help you create a professional, high-quality Delphi survey.

Step 1: Create Your Project and Configure Survey Settings

Before you can start collecting expert opinions, you need to define the framework of your study. This foundational setup determines how your Delphi process will unfold.

Create Your First Project

Project Name and Description

Give your study a clear, descriptive name and provide a short explanation of its purpose.

Example: "Future of AI in Public Health 2035" – Exploring expert consensus on emerging technologies that will transform healthcare delivery.

Survey Mode Configuration

Real-Time Delphi

Participants see an updated group average as soon as more responses come in.

When to use:

• Need faster consensus-building
• Want dynamic, adaptive feedback
• Working with time-sensitive topics

Manual Rounds

Feedback is only shared after each round is closed by the moderator.

When to use:

• Want to reduce potential groupthink
• Need controlled feedback timing
• Conducting classical Delphi research

Important: This choice affects how opinions evolve over time. Real-Time Delphi can speed up consensus-building, while Manual Rounds can reduce groupthink by preventing early responses from overly influencing later ones.

Anonymous Participation Settings

Configure whether expert identities remain anonymous. Anonymity reduces bias from authority and encourages honest ratings without fear of judgment.

Full Anonymity

Experts cannot see who else is participating. Only aggregated group statistics are shown.

Visible Experts

Panel members can see who is participating, but individual responses remain anonymous.

Pre-Phase Configuration

Define how many participants you want to submit their first ratings before showing aggregated feedback. This prevents bias when the first person enters a rating (because otherwise there is no average yet).

Recommended Setting: Wait for at least 3-5 responses before showing group statistics to ensure meaningful averages and prevent anchoring bias from a single early response.

Automated Notifications

Enable reminders to inform early participants when they should return to complete a second evaluation once the group average is available.

Notification Types

• Round opening notifications: Alert experts when a new round begins
• Feedback available notifications: Inform early respondents when group data is ready
• Gentle reminders: Nudge non-respondents before deadlines
• Round closing alerts: Final reminder before round closes

Step 2: Design Your Delphi Questions and Rating Dimensions

The quality of your questions determines the clarity and usefulness of your results. If your statements are vague or your rating scales inconsistent, experts will struggle to provide meaningful input. That's why this step is essential.

Create, Duplicate, or Delete Pages

Organize your survey into logical sections or topics. This helps experts navigate complex studies and keeps related questions together.

Best Practices

• Group related theses into themed pages
• Use page duplications to maintain consistent formatting
• Avoid having too many theses on one page (max 10-15)
• Use clear page titles that describe the topic

State Your Theses or Questions

Each statement should be specific, relevant, and neutral in tone to avoid bias.

Good Example:

"Artificial intelligence will be the main diagnostic tool in public health by 2035."

Poor Example:

"Don't you think AI might somehow change healthcare?" (Too vague, leading, lacks timeframe)

Define Rating Dimensions

Decide how you want experts to rate each thesis. Common options include:

Likert Scales

1-5 or 1-7 agreement scales

Example: 1 = Strongly Disagree, 5 = Strongly Agree

Time Expectations

Year-based predictions

Example: "In which year do you expect this to happen?"

Probability Estimates

0-100% likelihood scales

Example: "What is the probability this will occur?"

Importance Ratings

1-10 priority scales

Example: "How important is this factor?"

Configure Comments and Additional Inputs

Enable rich expert interaction beyond numerical ratings. Durvey.org provides powerful commenting features that capture nuanced expert reasoning and facilitate engagement.

Pro/Contra Comments

Experts can add comments tagged with sentiment (pro or contra). This helps identify why experts support or oppose specific theses and provides context for ratings.

Like and Interact

Experts can "like" other comments to show agreement, creating a lightweight way to express support for particular viewpoints without writing duplicate comments.

Explanation Fields

Ask experts to justify their ratings. This qualitative data enriches your analysis and helps other participants understand different perspectives.

Research Benefit: These interactive features generate rich qualitative data alongside quantitative consensus metrics. Your final analysis will include both statistical agreement and deep expert insights.

Step 3: Define Metadata and Stakeholder Variables

Metadata enables you to analyze results by group and see patterns you would otherwise miss. For example, technology experts may rate a trend very differently than policymakers. Collecting this information enriches your analysis.

Identify Key Variables

Think about what categories are relevant. Examples:

Professional Background

• Researcher
• Practitioner
• Policymaker
• Industry expert

Experience Level

• Years in field
• Seniority level
• Publications
• Certifications

Geographic Context

• Region
• Country
• Urban/Rural
• Organization type

Explain Why You're Collecting Metadata

In your survey introduction, tell participants that this information will help compare perspectives across sectors and increase the quality of your findings.

Warning: If you skip this step, you lose the ability to see valuable differences between subgroups, which weakens your study's insights.

Step 4: Finalize and Test Your Survey

Before you invite experts to participate, you need to make sure everything works perfectly. Testing helps catch typos, technical issues, and confusing instructions. This reduces dropout rates and increases trust in your process.

Randomize Page Order (if needed)

Prevent order effects that could bias results by randomizing the sequence in which pages are presented to different participants.

Review Your Survey in Test Mode

Check the experience exactly as your participants will see it.

Testing Checklist

• All questions display correctly
• Rating scales work properly
• Navigation flows smoothly
• Instructions are clear
• Time to complete is reasonable (≤20 minutes)
• Mobile compatibility verified

Confirm Rating Scales and Instructions

Make sure all labels are clear and consistent across all questions.

Step 5: Create and Manage Participant Profiles

Your Delphi results are only as good as your expert panel. That's why it's critical to record clear participant profiles and manage them in a structured way.

Add Participants Manually

Add each expert manually, including:

• Name: Full name of the expert
• Email address: Valid email for invitations
• Professional background: Role, organization, expertise area
• Relevant metadata: Any stakeholder variables you defined

Bulk Import Expert Lists

If you already have a list of contacts in Excel, you can import them all at once to save time.

Tip: Ensure your spreadsheet includes all required fields (name, email, metadata) to avoid errors during import.

Step 6: Draft Professional Invitations, Reminders, and Thank-You Emails

A clear, motivating invitation increases the chance that experts will respond promptly and stay engaged throughout multiple rounds. Durvey.org provides comprehensive email management tools to streamline communication and maintain expert engagement.

Durvey's Email Management Features

• Custom templates: Create branded email templates with your institution's style
• Automated sending: Schedule invitations to go out at optimal times
• Smart reminders: Automated reminders sent only to non-respondents
• Personalization: Merge fields for expert names and custom details
• Tracking: See who opened emails and clicked links

Write an Invitation Email

Clearly explain:

• The purpose and relevance of your Delphi study
• Why the recipient was selected (acknowledge their expertise)
• How their input will make a difference (impact and value)
• What the expected time commitment is (be honest and specific)

Use or Customize Templates

Durvey.org provides ready-to-use templates you can adapt to your tone and audience.

Professional Tone

For academic or policy contexts requiring formal language

Conversational Tone

For industry or practitioner panels with collaborative culture

Compose Reminders and Thank-You Messages

Reminders

Schedule friendly reminders. This reduces dropouts and ensures high-quality longitudinal data.

Recommended: Send 3 days before deadline, then 1 day before

Thank-You Messages

At the end of each round, show appreciation and share a short update on next steps.

Step 7: Launch Your Delphi Study and Manage Invitations

Launching the survey officially starts data collection. You control when invitations go out and when rounds open and close.

Launch Your Delphi Study

Once you launch, participants can begin submitting their ratings and comments.

Ready to Launch? Double-check all settings, test the survey, and ensure your participant list is complete before hitting launch.

Send Targeted Invitations and Reminders

Send invitations and remind only those who haven't completed the round.

Smart Communication

• Send invitations in batches if you have a large panel
• Target reminders only to non-respondents
• Avoid over-messaging (max 2-3 reminders per round)
• Personalize messages when possible

Track Opened and Pending Invitations

See who has opened your email, started the survey, or not yet responded.

Tip: Regular communication shows professionalism and helps maintain engagement over several rounds.

Step 8: Monitor Participation Progress in Real Time

Ongoing monitoring ensures you can react early if engagement drops or certain stakeholder groups are underrepresented.

Review Completion Rates

See the percentage of invited experts who have finished each round.

Target Response Rates

• Excellent: ≥80% completion
• Good: 70-79% completion
• Acceptable: 60-69% completion
• Concerning: <60% completion (take action)

Check Participation by Segment

Analyze who is participating (e.g., by region, sector, experience) to ensure a balanced dataset.

Watch For: Systematic underrepresentation of certain stakeholder groups. If one segment has low participation, send targeted follow-ups or recruit additional experts from that group.

Step 9: Close the Delphi Study

Formally closing each round locks the data, allowing you to prepare feedback and start the next stage of analysis or iteration.

Decide When to Close the Round

You can close it based on numerous indicators:

• Time-based: Deadline has passed
• Response-based: Sufficient participation achieved
• Stability-based: No significant changes in recent responses
• Consensus-based: Target agreement levels reached

Notify Participants (Optional)

Send a closing message summarizing participation or thanking participants for their contributions.