Delphi Methods

What are Delphi methods?

The Delphi method is a forecasting process framework based on the results of several rounds of questionnaires sent to a panel of experts (i.e. people who have expertise in an area requiring decision making). Several rounds of questionnaires are sent out, and the anonymous responses are aggregated and shared with the group after each round.  It is a systematic interactive way of gaining opinions or forecasts from a panel of anonymous participants with relevant expertise to answer two or more sequential surveys to identify consensus or convergence.  The methods can be used to identify issues or outcomes of importance to stakeholder groups, allowing participants to reconsider their own views in the second survey round in light of those of other participants.  The process is intended to to lead to convergence towards identifying preliminary sets of core outcome domains. In healthcare, it is a good means to collate consensus on outcomes or care that is consistent with professional knowledge.  

The Delphi method is a type of consensus method which does not require face to face meetings. It can bring together and synthesize the knowledge of a group of geographically scattered participants that never meet.  A large number of individuals across diverse locations and areas of expertise can be included anonymously, thus avoiding domination of the consensus process by one or a few experts.

Summary of Steps

  1. Design questionnaire
  2. Invite the participants to take part
  3. Send out the first round of the questionnaire
  4. Analyse responses from round 1 questionnaire
  5. Prepare the second round questionnaire 
  6. Send out second round questionnaire
  7. Analyse responds from round 2 questionnaire
  8. Repeat steps 4-6 if you're doing subsequent rounds - typically you continue until consensus has been reached.
  9. Report findings

Recruitment

There is no consensus on the optimal number of participants for a Delphi survey.  Panel sizes are usually guided by practicality or question scope.  Participants are not sampled for representativeness but are viewed as 'experts' drawn from different stakeholder groups to allow comparison and consideration of different perspectives.  It is worth deciding who your stakeholders are  - do you have different groups of stakeholders? Then ensure you have invited as many as you can.  How they're chosen should be recorded -was it their willingness? their knowledge of a particular area? 

Participants are invited - this can be done by email, letter or through social media, such as through Facebook group pages. 

Questionnaire Design & Adminstration

A questionnaire is required for Round 1,which is then adapted for Round 2 and onwards.  The questionnaire can be conducted online - each participant is sent a link or they can be sent paper versions.  Similar to questionnaires used in surveys, the questionnaire should be pre-tested and piloted before being used. 

You can use a Likert scale or other rating methods to capture opinions on the quality indicators (this can help with the analysis by enabling you to calculate means, modes and medians).  Questions can be repeated in the following round(s).  Usually in Round 1, you would ask participants to rate and propose additional outcomes or issues and then combine these into questions or statements to test with the wider group in the next round.  The second round questionnaire also gives participants the opportunity to see how the rest of the group prioritised areas and to see if they want to change their opinion on the basis of the group consensus.

It is important to have short periods between rounds as this improves the quality of the quality.  Ideally 1-3 weeks between rounds.  Participants need to be kept anonymous so that they are not identifiable to each other (this could influence/bias their responses).  

Are Delphi Methods quantitative or qualitative methodologies? 

Although questionnaires are used, the Delphi method is essentially a qualitative method as it is used to gain insights to form a consensus.  It can also explore an area of future thinking that goes beyond the currently known or believed. 

Further Reading

Delphi Methods at www.healthknowledge.org.uk 

De Meyrick J (2003) "The Delphi method and health research", Health Education, Vol. 103 Issue: 1, pp.7-16, https://doi.org/10.1108/09654280310459112