The Delphi Technique

Posted on November 15, 2017

Tags: , , ,

 

What is it?

The Delphi technique (also referred to as Delphi procedure or process), is a method of congregating expert opinion through a series of iterative questionnaires, with a goal of coming to a group consensus. In fact, in 150 studies that used the Delphi technique, there was no universally agreed upon working definition of the technique. There are many variants used, some of which have departed widely from the original Delphi technique.

Since its development in the 1950’s by the RAND Corporation, several refinements and modifications have been made, such as specific strategies for different fields, including business, government, and healthcare.

There are four characteristic features of the Delphi technique that distinguish it from other group decision making processes. These are: anonymity, iteration with controlled feedback, statistical group response, and expert input.

When is it used?

The Delphi Technique can be an especially useful research methodology when there is no true or knowable answer, such as decision-making, policy, or long-range forecasting. A wide range of opinions can be included, which can be useful in cases where relying on a single expert would lead to bias.

The Delphi technique was recommended as the method of choice when:

How is it done?

  1. Survey Development
    • Define the research problem/questions and develop the first-round survey
    • Pilot the survey with a small group to ensure the responses will elicit appropriate answers to the research question
    • Round one is like a ‘brainstorming’ round, and allows participants to provide their own responses to the question. The responses are categorized by the researchers to provide the response options in future rounds. In the first round, participants may be asked to limit themselves to one response, or answer as many times as they would like depending on the research question and number of participants. Alternatively, pre-existing options could be provided for ranking or response, however, this approach could bias the responses or limit the available options
  2. Participant Recruitment
    • Research has indicated that participant subject matter knowledge (i.e. being an “expert”) may not have a substantial impact on study results, so it might be best to choose participants who have some understanding of the topic and an interest in the outcome of the study to limit attrition and encourage thoughtful responses to the surveys
    • Often, participants are selected via non-probability sampling techniques (either purposive sampling or criterion sampling), to save resources and ensure appropriate participants are selected
  3. Data Analysis
    • The first round of the Delphi technique involves participants providing answers to the research question, which will then be ranked in future rounds. In a classic Delphi, no items should be added or removed, and the wording used by participants should be kept for round two. However, this may be difficult or not feasible depending on the number and types of responses provided, and content analysis can be used to group similar themes prior to the second round. An informal literature review can also be used to identify further items depending on the research question
    • In subsequent rounds, participants are asked to rank/respond to the analysed options from round one. Between rounds the group’s responses are analysed, summarised, and communicated back to the participants, a process called controlled feedback. This is repeated until consensus is reached, or for a planned number of rounds depending on the research question
    • Subsequent rounds are analysed to identify convergence of participant responses, and to provide controlled feedback. Central tendencies (mean, median, and mode) and levels of dispersion (standard deviation and the inter-quartile range) are often used. These results are fed back to participants in the next round, although no consistent method for reporting exists
  4. Ending the Delphi Process
    • The process typically ends once acceptable level of consensus has been reached, however, there is no universally agreed cut-off. The level of agreement reached depends on sample numbers (i.e. high attrition or participant burnout), aim of the research (i.e. if complete consensus is required), and available resources
    • Two to four rounds will typically be conducted to ensure study goals are met but to avoid sample fatigue and unnecessarily use of resources

Pros and Cons of the Delphi Technique:

Pros:

Cons:

Similar/Alternative Methodologies:

Brainstorming and nominal group technique are similar techniques that allow incorporation of many individual perspectives.

 

References

Goodman CM. The Delphi technique: a critique. Journal of Advanced Nursing. 1987 Nov;12(6):729–734. doi:10.1111/j.1365-2648.1987.tb01376.x. Available from: http://onlinelibrary.wiley.com/doi/10.1111/j.1365-2648.1987.tb01376.x/full

Sackman H. (1974). Delphi critique; expert opinion, forecasting, and group process. Lexington, Mass: Lexington Books.

Hasson F, Keeney S, and McKenna H. Research guidelines for the Delphi survey technique. Journal of Advanced Nursing. 2000 Oct;32(4):1008–1015. doi:10.1046/j.1365-2648.2000.t01-1-01567.x. Available from: http://onlinelibrary.wiley.com/doi/10.1046/j.1365-2648.2000.t01-1-01567.x/full

Hsu C & Sandford BA. The Delphi Technique: Making Sense Of Consensus. Practical Assessment, Research & Evaluation. 2007 Aug;12(10). Available from: http://pareonline.net/getvn.asp?v=12&n=10

McKenna HP. The Delphi technique: a worthwhile research approach for nursing? Journal of Advanced Nursing. 1994 June;19(6):1221–1225. doi:10.1111/j.1365-2648.1994.tb01207.x. Available from: http://onlinelibrary.wiley.com/doi/10.1111/j.1365-2648.1994.tb01207.x/full

Linstone HA, Turoff M. (1975). The Delphi Method: Techniques and Applications. Addison-Wesley Pub. Co., Advanced Book Program.

 

Kristen Dufresne

Kristen Dufresne

Physical Therapy student at the University of Toronto

More Posts

Related Post

creative commons license
The Delphi Technique by Kristen Dufresne is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License. Unless otherwise stated, all images used within the blog are not available for reuse or republication as they are purchased for Students 4 Best Evidence from shutterstock.com.

Leave a Reply

Your email address will not be published. Required fields are marked *