Mini-Delphi Method

The mini-Delphi method is a variant of the Delphi method, a group decision-making technique that is used to gather and integrate the opinions of a group of experts - typically in the form of probabilistic forecasts. The mini-Delphi method is a simplified version of the Delphi method typically used with a smaller group of experts requiring less time and resources.

In the mini-Delphi method, a small group of experts is asked to provide their opinions on a specific issue or question through a series of rounds. Each round begins with the experts providing their initial opinions/forecasts, which are then anonymously compiled and shared with the group. The experts then have the opportunity to review and revise their opinions in light of the group's input. This process is repeated for a predetermined number of rounds - typically two or three.

How to use the mini-Delphi method

The mini-Delphi method, also referred to as ‘estimate-talk-estimate’,  is a particular form of nominal group collaboration typically used to survey panels of experts, so it’s no surprise the steps are similar between the two methods.

Unlike general nominal group collaboration, the mini-Delphi method is typically used to formulate a collective probabilistic forecast from a group of experts.

The typical process for the mini-Delphi is as follows:

1. Define the problem or question to be addressed by the group: As with a typical mini-Delphi, if we’re seeking a ‘forecast’ from experts, framing the question is an important, non-trivial step.
2. Select a group of experts or stakeholders who have relevant knowledge or experience: The more this process can be automated through surveys, the more opinions/forecasts can be collected.
3. Ask each member of the group to provide anonymous written responses: This would typically include the forecast, the proposed probability of the forecast, and the supporting argument/information.
4. Collect and compile the responses into a summary report: At the very least, this is a readout that summarizes the responses, but may also include the average of the probabilistic forecasts or groupings of similar responses.
5. Share the summary report with all members of the group and ask them to provide feedback on the responses: Participants should have the ability to comment on the individual responses of others as well as the summary of the report. Participants can revise their own responses async or in real-time.
6. Collect and re-compile the feedback into a final report, which should include a summary of the main findings and recommendations: This 'question -> answer -> synthesis' process can be repeated (often 2-3 times) if needed before compiling a final report

‍

Again, the mini-Delphi method is typically used for collecting and aggregating forecasts from groups of experts or to leverage the wisdom of crowds. For standard collaboration where we may just be requesting information, arguments, or feedback from groups, using nominal group collaboration is a similar method.

At this point, we might have a series of forecasts by individuals. Instead of just running a mini-Delphi over and over, we can retain these forecasts and start to build a ‘forecast market’. The team can continue to uncover new information and update their beliefs in real-time as these forecasts continue to shape decisions in the future.

Forecast, measure, revise: it is the surest path to seeing better.” - Philip Tetlock

Resources:

• A mini-Delphi approach: An improvement on single round techniques‍
• Philip Tetlock, Superforecasting: The Art and Science of Prediction‍
• The Delphi Technique: Making Sense of Consensus
• Indeed, Delphi Method: Definition, Stages, Pros, Cons, Examples‍
• Wikipedia, Delphi Method

‍

‍