Introduction to Dissent Analysis

Dissent analysis is a critical step in any Delphi study. It helps researchers and decision-makers understand where and why experts disagree, and whether this disagreement reflects legitimate diversity of opinion or systematic distortions such as bias or polarization.

By applying dissent analysis, you can:

• Detect hidden patterns of disagreement.
• Uncover potential biases in your data.
• Assess the reliability and clarity of expert forecasts.

Below, we'll explain the three most important dissent analysis methods step by step, using simple examples so you can understand even if you've never worked with them before.

1. Desirability Bias Analysis

What is Desirability Bias?

Desirability bias happens when people confuse what they want to happen with what they expect will happen. In other words, wishful thinking can lead participants to rate a scenario as more probable simply because they find it appealing.

This is a common issue in Delphi studies because scenarios often have a strong emotional component.

Why Use Desirability Bias Analysis?

If you don't check for desirability bias, your forecasts could be misleading. For example:

• You might assume there is consensus on a likely future when in fact, people are just expressing hope.
• Decision-makers could overestimate the chance of positive outcomes.

Detecting this bias makes your results more objective and transparent.

How to Detect Desirability Bias – Fully Automated with Durvey.org

One common cognitive bias in expert forecasting is desirability bias: the tendency to rate desired outcomes as more probable simply because we want them to happen. Durvey.org makes it easy to detect and document this bias. All data collection, statistical testing, and interpretation are automatically handled for you, so you can focus on insights – not on manual calculations.

Here's how the process works in Durvey.org:

1. Collect Two Ratings for Each Scenario

In your Delphi survey, you simply add two questions per scenario:

1. Desirability Rating: "How desirable is this outcome to you?" (e.g., scale from 1–10)
2. Expected Probability Rating: "How likely do you believe this outcome is?" (e.g., scale from 1–10)

Durvey.org automatically links these ratings for each respondent, so you don't need to match columns or check IDs manually.

2. Calculate the Pearson Correlation – Instantly

After data collection closes, Durvey.org automatically computes the Pearson correlation coefficient (r) between desirability and probability ratings for each scenario. This statistic quantifies how strongly preferences and expectations are related.

Interpretation made simple:

• If r > 0.6, this is considered strong evidence of desirability bias.
• The higher the correlation, the more likely that experts' forecasts are influenced by personal wishes.

You never have to set up formulas, run SPSS, or write code – Durvey.org shows you the result instantly in a clear dashboard.

Example Walkthrough

Let's see how this works in practice – all steps fully automated:

Scenario:

"By 2040, universal basic income will be implemented."

Step 1: Collect Ratings

• Desirability: 9/10 (very desirable)
• Probability: 8/10 (very probable)

Step 2: Compute Correlation

• Durvey.org calculates r = 0.75

Interpretation (auto-generated text):

"This strong positive correlation (r = 0.75) suggests that desirability bias may significantly influence probability estimates for this scenario."

Durvey.org even produces ready-made text you can paste directly into your report or publication.

Best Practices – All Integrated into Durvey.org

Durvey.org not only computes correlations but also ensures you follow best practices:

• Always report the correlation coefficient in your results. (Durvey.org includes this automatically in the downloadable report.)
• Discuss potential bias when r is high. (The system provides pre-written commentary you can adapt to your paper.)

Why This Matters

Detecting desirability bias is essential to ensure:

• Your Delphi forecasts are credible.
• Stakeholders understand when optimism or personal preferences might skew estimates.
• You can transparently communicate limitations of your results.

With Durvey.org, you don't need statistical expertise – the platform guides you step by step and delivers clear, publication-ready insights.

In short: Durvey.org automates every aspect of desirability bias detection – from data collection to final interpretation – saving you time and ensuring professional quality.

2. Outlier Analysis

What are Outliers?

Outliers are responses that are much higher or lower than all the others. They can be:

• Honest but unusual expert opinions.
• Data entry errors.
• Signs of misunderstanding the question.

Outliers can have a large impact on averages and distort your interpretations if not examined carefully.

Why Use Outlier Analysis?

Outlier analysis allows you to:

• Identify potentially problematic data points.
• Decide whether to include them in final calculations.
• Understand the diversity of perspectives.

How to Detect Outliers in Delphi Studies – Fully Automated with Durvey.org

In every Delphi study, you will occasionally encounter outliers – individual ratings that are extremely different from the rest. Identifying and documenting outliers is crucial to maintain the credibility and transparency of your results. Durvey.org takes all the manual work off your shoulders. The platform automatically computes outlier thresholds, flags unusual ratings, and generates clear explanations you can insert straight into your reports.

Below, you'll learn exactly how this process works – no statistics degree required.

Two Methods to Detect Outliers (and Why You Need Both)

Durvey.org uses two established statistical methods to detect outliers and validate results:

Method A: Interquartile Range (IQR) Method

The IQR Method is a robust way to detect outliers without assuming any specific distribution of ratings. Note: This can only be executed, if IQR is not equal 0.

How it works:

1. Calculate Q1 (25th percentile): 25% of all ratings fall below this point.
2. Calculate Q3 (75th percentile): 75% of all ratings fall below this point.
3. Compute the IQR: IQR = Q3 − Q1
4. Determine thresholds:
   • Upper threshold = Q3 + (1.5 × IQR)
   • Lower threshold = Q1 − (1.5 × IQR)
5. Any rating outside these thresholds is flagged as an outlier.

Durvey.org automatically computes all these values for you, flags outliers in the dataset, and displays them clearly in your dashboard.

Example (automated):

Suppose ratings for a scenario mostly range from 50% to 90%, but one expert gives 0%. Durvey.org will show you:

• Q1 = 60%
• Q3 = 80%
• IQR = 20%
• Lower threshold = 30%
• Upper threshold = 110%

Since 0% is below 30%, this rating is automatically marked as an outlier in your results.

Method B: Z-Score Method

The Z-Score Method identifies outliers based on how far each rating is from the mean, measured in standard deviations.

How it works:

1. Compute the mean and standard deviation of all ratings.
2. For each rating, calculate the z-score: z = (x − mean) / standard deviation
3. If the absolute z-score is greater than 2.58, the rating is considered an outlier at the 1% significance level.

Durvey.org automatically applies this method and highlights any rating with an extreme z-score, requiring no manual calculations.

Example (automated):

Suppose:

• Mean = 70%
• Standard deviation = 10%
• Rating = 40%

Durvey.org computes:

• z = (40 – 70) / 10 = –3.0

Because |–3.0| > 2.58, this rating is flagged as an extreme outlier.

Automated Cross-Validation and Reporting

Durvey.org applies both methods by default. The platform:

• Cross-validates flagged outliers.
• Summarizes which method identified each outlier.
• Generates ready-to-use text explaining the process, which you can insert directly into papers or reports.

Why Outlier Detection Matters

Proper outlier detection is essential because:

• Outliers can skew the mean and consensus measures.
• Transparency about unusual ratings builds trust and credibility with stakeholders.
• Documenting your methods helps meet scientific standards and peer-review requirements.

Best Practices – Fully Integrated in Durvey.org

Durvey.org automatically implements all recommended best practices:

• Use both IQR and Z-Score methods to cross-validate.
• Investigate flagged outliers before deciding to exclude or retain them.
• Document thresholds and decisions clearly in the downloadable reports.

You don't need to configure anything manually – everything happens automatically when you finalize your data collection.

3. Bipolarity Analysis

What is Bipolarity?

Bipolarity means the group splits into two opposite camps with little middle ground. For example:

• Half the experts strongly agree.
• Half strongly disagree.

This indicates polarization.

Why Check for Bipolarity?

• Bipolarity highlights controversial topics.
• Helps explain why consensus was hard to reach.
• Improves understanding of stakeholder divisions.

How to Measure Bipolarity in Delphi Studies – Fully Automated with Durvey.org

When running a Delphi study, it's important not only to understand central tendencies (like the average rating) but also how opinions are distributed. Sometimes, your panel splits into two opposing camps, creating what's called bipolarity or bimodality. Durvey.org automatically detects and reports this phenomenon, saving you time and ensuring your results are transparent and credible.

Below you'll learn how bipolarity detection works—and why it matters.

What Is Bipolarity?

Bipolarity occurs when your participants' ratings cluster into two distinct peaks instead of forming a single consensus.

Example:

• Half of the experts strongly agree (e.g., rating = 9/10).
• The other half strongly disagree (e.g., rating = 1/10).

Recognizing this split is crucial because it reveals:

• Deep disagreement among stakeholders.
• Potential polarization around the topic.
• The need for further discussion or clarification.

How Bipolarity Is Measured (Fully Automated)

The most widely used metric to detect bimodal distributions is the Bimodality Coefficient (BC). Durvey.org automatically computes BC for every scenario in your study.

Step 1: Calculate the Bimodality Coefficient

Durvey.org uses a formula that combines:

• Skewness (asymmetry of the distribution)
• Kurtosis (how "peaked" the distribution is)

All calculations happen in real time as soon as your data collection closes.

Step 2: Interpret the Bimodality Coefficient

Durvey.org makes interpretation simple:

• BC > 0.55 → Strong evidence of bimodal (bipolar) distribution
• BC ≤ 0.55 → Likely unimodal (one main consensus)

Example (automated):

Scenario: Experts rate desirability of Universal Basic Income:

• 10 participants rate 1/10 (strongly undesirable)
• 10 participants rate 9/10 (strongly desirable)

Durvey.org calculates:

• BC = 0.68

Interpretation (auto-generated):

"The high Bimodality Coefficient (BC = 0.68) indicates a clear bipolar distribution of opinions. This suggests significant polarization among participants."

Step 3: Visualize the Split

Durvey.org automatically generates histograms and distribution plots that make the two peaks easy to see.

These visuals can be exported as images for presentations, reports, and scientific publications.

Why Measuring Bipolarity Matters

Detecting and reporting bipolarity helps you:

• Reveal hidden divisions among experts.
• Avoid misleading conclusions based solely on averages.
• Provide transparent evidence for stakeholder discussions.

In many Delphi studies, stakeholders assume consensus when opinions are in fact polarized. Durvey.org ensures you see the real picture.

Conclusion: Why Dissent Analysis Matters

Dissent analysis makes Delphi studies more credible and insightful by:

• Revealing hidden biases.
• Detecting extreme opinions.
• Showing polarization and consensus levels.

This deeper understanding empowers better decisions and helps you communicate results with clarity and confidence.