Introduction
The necessity of an independent evaluation of the recommendations before their planned revision with the third-party opinion of competent specialists is obvious. This analysis allows to adapt the realization of recommendations in practice considering the equipment and specialists’ level.
Currently, there are several methods to assess the possibility, feasibility and effectiveness of decision-making in various clinical situations. The most suitable method is the Delphi method which allows to get a consistent general opinion based on individual assessments of experts after generalizing and processing the data. The advantages of the method are the formation of an independent opinion on the issues discussed by each member of the group followed by the output of a collegial opinion [1–3].
The method of the Delphi assessment has the main characteristics: phasing, anonymity of participants, absentee discussion, standardization of the survey for all participants, validity of third-party opinion, final interpretation based on the opinion of all experts, statistical aggregation of group responses and expert contribution [4].
The Delphi method is used in the field of technology and science forecasting; however, it has not yet used widely in the Russian scientific literature.
Purpose of the study
To analyze the efficacy, safety and accessibility of the clinical implementation of the methodological recommendation “The use of non-invasive ventilation” using the modified Delphi method.
Materials and methods
The second revision of the guidelines “The use of non-invasive ventilation” was published in 2019 [5]. Later during the interaction of the authors with the Center for Expertise and Quality Control of Medical Care of the Ministry of Health of Russia and placing the guidelines in the clinical recommendations rubricator, some additions were made to the text [6]; it was this version that underwent the Delphi assessment by the initiative of the Committee on Recommendations and Organization of Research of the All-Russian Public Organization “Federation of Anesthesiologists and Resuscitators” (FAR) and consisted of several stages:
The preparatory stage was the coordinators analysis of the Delphi evaluation of the methodological recommendation “The use of non-invasive ventilation” of 2020 [6] and the design of a questionnaire consisting of three sections: the evaluation of the methodological recommendation; the assessment of the quality criteria of medical care and the overall assessment of the methodological recommendation. The questions and criteria of the analysis were developed accordingly to the recommendations for the modified Delphi method use in clinical and pharmacokinetic studies [7]. A group of independent specialists in the field of respiratory support has been formed representing high educational institutions, leading federal and regional medical organizations.
To evaluate each statement of the methodological recommendation “The use of non-invasive ventilation” the designated specialists had to answer five questions:
- The thesis-recommendation contains specific understandable to anesthesiologists-resuscitators descriptions of what tactics, in what situation and which patients should be used?
- As an expert, do you understand how to evaluate the actions of an anesthesiologist-resuscitator?
- Can the thesis-recommendation be used in the structural units of the profile of anesthesiology-resuscitation?
- Is this thesis-recommendation useful for the providing the anesthesiology and resuscitation care?
- Will this thesis-recommendation be followed by anesthesiologists-resuscitators?
The quality criteria of the medical care suggested by the methodological recommendation were proposed for evaluation by experts on six issues:
- The quality criterion contains specific descriptions understandable to anesthesiologists-resuscitators of what tactics, in what situation and which patients should be used?
- As an expert, do you understand how to evaluate the actions of an anesthesiologist-resuscitator?
- Can this quality criterion of the medical care be introduced into the structural units of the profile of anesthesiology-resuscitation?
- Is this quality criterion useful for the providing of medical care in the field of anesthesiology-resuscitation?
- Will this quality criterion be followed by anesthesiologists-resuscitators?
- Is the quality criterion applicable in any medical organization that provides medical care to the adult population in the profile of “anesthesiology-resuscitation”?
The main stage. At this stage, a questionnaire was sent from an impersonal mailbox via the Internet. Respondents were asked to rate each question on a ten-point R. Likert scale from 1 to 10 depending on the expression by the respondents of their agreement (10 points — completely agree) or disagreement (1 point — absolutely disagree) with the theses proposed in the methodological recommendation. Based on the answers and additional information provided by the experts, a questionnaire of the second round was formed and also sent out.
Analytic stage
An assessment of the statements of the thesis-recommendations and quality criteria of medical care was done in the first round of discussions. Based on the results of the second round the final proposals were formed for the co-authors of the methodological recommendation. To select relevant answers the consistency of the scores of each answer with the final score was analyzed. Aggregation was performed using Microsoft Office Excel, 2016. The weighted average score was calculated based on the respondents' self-esteem using the formula:
(К1 × О1 + К2 × О2 + … + К9 × О9) : (К1 + К2 + … + К9) × 10 = … %,
where K is the self-esteem coefficient, O is the expert assessment.
If in assessing the statements and criteria for the quality of medical care the values of the median or mode of any position were less than 7, the weighted average score was less than 70 %; the values of the median or mode of the quality criteria of medical care are less than 7.5, the weighted average score is less than 75 %, then these statements (quality criteria) should be recommended for processing [1].
Results
The Delphi analysis involved 15 out of 19 specialists who were invited to participate in the survey. The main remarks and additions were in terminology, concretization of individual statements and stylistics. According to the results of the first round (Annex) of the discussion, only one recommendation did not receive 70 % of the weighted average rating (question 5 — Will this thesis-recommendation be followed by anesthesiologists-resuscitators? — 67.7 %) (Recommendation 19. In patients with high-flow oxygenation it is recommended to use the following tuning algorithm to increase its efficiency).
More discussion was found in the formulation of the thesis-recommendations and the specifics of the data presented. For example, when discussing the recommendation 1 when specifying the indications for non-invasive mechanical ventilation in acute respiratory failure, most experts considered it appropriate to replace the wording “with a moderate PEEP / CPAP” (positive end-expiratory pressure / constant positive airway pressure) with specific values of airway pressure.
Recommendations 1, 3, 8, 10 and 16 provide the information on the characteristics of respiratory support in immunosuppressed patients but do not specify the criteria by which this condition should be stratified, which is important in identifying this category of patients.
Recommendation 4 lacks the criteria for compensated acute respiratory failure (ARF), which should consider non-invasive mechanical ventilation instead of oxygen therapy to improve gas exchange, reduce respiratory performance and improve the prognosis. An important issue, according to experts, is the addition of information on respiratory support in patients with the new coronavirus infection COVID-19 (COronaVIrus Disease 2019) in the discussed guidelines and the revision of the indications for high-flow oxygen therapy with an initial oxygenation index of more than 150 mm Hg specified in recommendation 8.
Most experts supported supplementing the paragraph 5 of recommendation 12 with the Richmond Agitation-Sedation Scale (RASS) which suggests the refrain from non-invasive ventilation.
The experts' assessment of the quality criteria of medical care was focused on the possibility of fully implementing the statements of the methodological recommendation in real clinical practice and mainly affected the equipment of medical organizations and structural units in the field of anesthesiology and resuscitation. The shortage of blood gas and acid-base analyzers, devices for non-invasive mechanical ventilation, including high-flow oxygen therapy, in the departments makes therapeutic measures limitedly feasible. This fact is confirmed by low results (less than 75 %) of the weighted average assessment based on the results of the first round of the question “Is the quality criterion applicable in any medical organization providing medical care to the adult population in the profile of anesthesiology-resuscitation?”, related to the implementation of the quality criteria 1 (73.8 %), 2 (72.6 %), 6 (65.5 %) and 7 (73.9 %) and (Table 1) the quality criterion 7 “High-flow oxygen therapy initiated in hypoxemic acute respiratory failure in immunocompromised patients” is also not agreed upon in the discussion in the first round. This criterion received a weighted average score of 74.0 % on the fifth question on the understanding of the quality criterion by anesthesiologists-resuscitators due to the fact that there is no information in the recommendations about the criteria for immunocompromised patients.
Table 1. The quality care statements proposed to be changed
| Quality criterion | Formulation of the quality criterion of the methodological recommendation | Question* / Weighted average score (%) based on the first-round results | Recommendation /proposed reformulation resulting on the second-round results |
|---|---|---|---|
| 1 | Non-invasive mechanical ventilation in exacerbation of chronic obstructive pulmonary disease was initiated in moderate respiratory acidosis (7.35 > pH > 7.25) and compensated acute respiratory failure | 1/93.2 | Supplement the signs of compensated ARF |
| 2/97.0 | |||
| 3/89.9 | |||
| 4/94.3 | |||
| 5/84.4 | |||
| 6/73.8 | |||
| 2 | Non-invasive mechanical ventilation initiated in community-acquired pneumonia in patients with chronic obstructive pulmonary disease | 1/79.2 | Remove the criterion |
| 2/85.5 | |||
| 3/78.3 | |||
| 4/77.9 | |||
| 5/77.5 | |||
| 6/72.5 | |||
| 3 | Non-invasive mechanical ventilation started in cardiogenic pulmonary edema | 1/96.6 | |
| 2/94.5 | |||
| 3/92.2 | |||
| 4/96.8 | |||
| 5/90.8 | |||
| 6/84.5 | |||
| 4 | Non-invasive mechanical ventilation is initiated after tracheal extubation in patients with hypercapnia due to obesity or chronic obstructive pulmonary disease | 1/93.8 | Edit / Non-invasive mechanical ventilation started after tracheal extubation in patients at high risk of developing postoperative respiratory failure (hypercapnia due to obesity or chronic obstructive pulmonary disease) |
| 2/91.6 | |||
| 3/89.9 | |||
| 4/94.0 | |||
| 5/86.0 | |||
| 6/81.8 | |||
| 5 | Non-invasive mechanical ventilation initiated in hypoventilation syndrome in obesity | 1/93.0 | |
| 2/90.8 | |||
| 3/90.8 | |||
| 4/94.3 | |||
| 5/85.7 | |||
| 6/79.4 | |||
| 6 | High-flow oxygen therapy has been initiated in patients with hypoxemic acute respiratory failure due to community-acquired pneumonia | 1/88.8 | Remove the criterion |
| 2/86.4 | |||
| 3/77.7 | |||
| 4/88.3 | |||
| 5/81.4 | |||
| 6/65.5 | |||
| 7 | High-flow oxygen therapy initiated in hypoxemic acute respiratory failure in immunocompromised patients | 1/83.7 | Edit / High-flow oxygen therapy initiated in hypoxemic acute respiratory failure in immunocompromised patients (oncohematological, Pneumocystis pneumonia, after organ transplantation) |
| 2/83.5 | |||
| 3/83.0 | |||
| 4/87.3 | |||
| 5/74.0 | |||
| 6/73.5 | |||
| 8 | During non-invasive mechanical ventilation the vital functions (central nervous system, respiration and blood circulation) were monitored and the effectiveness of non-invasive mechanical ventilation was evaluated | 1/88.1 | It is necessary to specify the criteria for the effectiveness of NIV |
| 2/88.0 | |||
| 3/88.0 | |||
| 4/91.0 | |||
| 5/89.9 | |||
| 6/87.6 |
An analysis of the results of the overall estimation of the methodological recommendation showed that less than 70 % of positive answers were to two questions (Table 2):
- “The recommendations are unambiguous for understanding, contain specific descriptions of what tactics, in what situation and which patients should be used, according to the totality of the available evidence?” — 68.7 %;
- “Do you agree with the use of all the quality criteria of medical care specified in the clinical (methodological) recommendation?” — 43.6 %.
It is fundamental to note that the comments and suggestions formulated by the participants of the Delphi assessment can also be useful for the clinical recommendations of the FAR “Diagnosis and Intensive Therapy of Acute Respiratory Distress Syndrome” [8].
| Assessment criterion | % of positive answers |
|---|---|
| Is information on the diagnosis, treatment, rehabilitation, outpatient (dispensary) observation and prevention of the disease presented in clinical (methodological) recommendations to the extent that allows to ensure the medical care quality? | 86.7 |
| The information about patients to whom clinical (methodological) recommendations will be applied is presented in an amount that allows to ensure the medical care quality including the age and gender group of patients, is information about concomitant diseases and complications given? | 86.7 |
| Are the options for providing medical care for this disease described to an extent that allows you to ensure the quality of medical care? | 93.7 |
| The recommendations are unambiguous to understand, contain specific descriptions of what tactics, in what situation and which patients should be used, according to the totality of the available evidence? | 68.7 |
| Do you agree with the use of all the quality criteria of medical care specified in the clinical (methodological) recommendation? | 43.6 |
| Clinical (methodological) recommendations are accompanied by materials for their practical use (clinical scales, questionnaires, information for patients, etc.) by medical professionals in an amount that allows to ensure the medical care quality? | 81.2 |
| Do clinical (methodological) recommendations use international nonproprietary names or grouping (chemical) names of medicines and non-commercial names of medical devices (except in cases where these names are missing)? | 100 |
| Clinical (methodological) recommendations are developed with an indication of medical services provided for by the Nomenclature of Medical Services, approved by Order of the Ministry of Health of the Russian Federation dated October 13, 2017 No. 804n “On Approval of the Nomenclature of Medical Services”? | 100 |
| Are you ready to implement clinical (methodological) recommendations in the work of your medical institution? | 93.7 |
Conclusion
There is no doubt about the need for an independent Delphi assessment of methodological recommendations before a planned revision based on the third-party opinion of competent specialists. Consensus was reached on 20 of the 21 thesis recommendations, on four of the eight criteria for the quality of medical care and the re-writing of individual recommendations. The expertise made it possible to look at the implementation of the methodological recommendation from the perspective of practicing anesthesiologists-resuscitators including number in structural units with a low level of material and technical equipment.
Disclosure. E.M. Shifman is the President of the “Obstetrical Anesthesiologists Intensivists Association”, the Vice-President of the all-Russian public organization “Federation of anesthesiologists and reanimatologists”; I.B. Zabolotskikh is the First Vice-President of the all-Russian public organization “Federation of anesthesiologists and reanimatologists”. Other authors declare that they have no competing interests.
Author contribution. All authors according to the ICMJE criteria participated in the development of the concept of the article, obtaining and analyzing factual data, writing and editing the text of the article, checking and approving the text of the article.
Ethics approval. Not required.
Funding source. This study was not supported by any external sources of funding.
Data Availability Statement. The data that support the findings of this study are available from the corresponding author upon reasonable request.
