Multiple breath washout quality control in the clinical setting
Bettina S. Frauchiger1*, Julia Carlens1,2*, Andreas Herger3, Alexander Moeller 3, Philipp Latzin1, Kathryn A. Ramsey1#
1. Pediatric Respiratory Medicine, Department of Pediatrics, Inselspital, Bern University Hospital, University of Bern, Freiburgstrasse 8, 3010 Bern, Switzerland†
2. Clinic for Paediatric Pneumology, Allergology and Neonatology, Hannover Medical School, Carl-Neuberg-Strasse 1, 30625 Hannover, Germany
3. Division of Respiratory Medicine and Children’s Research Center, University Children’s Hospital Zurich, Zurich, Switzerland †
† Institutions where research was mainly conducted

* Co-first authors: Authors contributed equally

# Corresponding author:
Kathryn Ramsey
Inselspital, Bern University Hospital,
Freiburgstrasse 8, Bern 3010 Bern, Switzerland
Email: Kathryn.ramsey@extern.insel.ch
Funding: This project was funded by the Swiss National Science Foundation, Grant Nr. PZ00P3_168173 (Latzin) / 1; 32003B_182719 (Ramsey)
Keywords: Multiple breath washout, lung clearance index, quality control, cystic fibrosis, children
Running head: Quality control for multiple breath washout
Wordcount (without abstract): 3270 words
Abstract: 220 words
Author contributions: BF, JC, PL, AM, and KR were responsible for the conception and design of this study. BF, JC, and KR drafted the quality control guidelines and the structured implementation in clinical routine. Data acquisition was conducted by BF and AH. BF, JC, AM, PL, and KR were responsible for data interpretation. Statistical analysis was conducted by BF and KR. BF, JC, AM, PL, and KR drafted the manuscript and all authors revised and approved the manuscript for intellectual content before submission.

Abstract

Background: Multiple breath washout (MBW) is increasingly used in the clinical assessment of patients with cystic fibrosis (CF). Guidelines for MBW quality control (QC) were developed primarily for retrospective assessment and central overreading. We assessed whether real-time QC of MBW data during the measurement improves test acceptability in the clinical setting.
Methods: We implemented standardized real-time QC and reporting of MBW data at the time of the measurement in the clinical pediatric lung function laboratory in Bern, Switzerland in children with CF aged 4-18 years. We assessed MBW test acceptability before (31 tests; 89 trials) and after (32 tests; 97 trials) implementation of real-time QC and compared agreement between reviewers. Further, we assessed the implementation of real-time QC at a secondary center in Zurich, Switzerland.
Results: Before implementation of real-time QC in Bern, only 68% of clinical MBW tests were deemed acceptable following retrospective QC by an experienced reviewer. After implementation of real-time QC, MBW test acceptability improved to 84% in Bern. In Zurich, after implementation of real-time QC, test acceptability improved from 50% to 90%. Further, the agreement between MBW operators and an experienced reviewer for test acceptability was 97% in Bern and 100% in Zurich.
Conclusion: Real-time QC of MBW data at the time of measurement is feasible in the clinical setting and results in improved test acceptability.

1. Introduction

The lung clearance index (LCI) derived from the multiple breath washout technique (MBW) is sensitive to detect early lung disease in patients with cystic fibrosis (CF) 1-4. With the availability of commercial MBW devices, LCI is increasingly being used as an outcome in routine clinical surveillance 5-10. While MBW testing requires minimal cooperation from the subject, an acceptable test requires relaxed tidal breathing and a leak-free system11, which can be challenging in young children and individuals with respiratory disease12. Besides, prospective quality control (QC) of MBW measurements can be challenging in the busy clinical setting.
Quality control guidelines for MBW focus primarily on retrospective analysis and central overreading of MBW measurements by experienced users for research studies and clinical trials11,13-15. The 2013 European Respiratory Society (ERS) and American Thoracic Society (ATS) consensus statement for inert gas washout measurements proposed initial recommendations for testing procedure and technical acceptability criteria 11. Further to this, ATS published additional guidelines for the preschool age group 14. Jensen et al. proposed comprehensive guidelines for retrospective quality control of MBW measurements, which involved both qualitative and quantitative criteria for trial grading and acceptability13. These guidelines were further implemented in a standardized MBW training and quality control platform for central overreading in clinical trials15. However, for LCI to be used as a clinical outcome, prospective reporting of acceptability and test results is required for clinical decision making.
Therefore, we aimed to implement prospective, real-time quality control of MBW measurements in the clinical pediatric lung function laboratory in Bern, Switzerland. The first aim was to evaluate the acceptability of clinical MBW measurements in children with CF before and after the implementation of real-time quality control. The second aim was to assess the implementation of real-time quality control of MBW measurements in a pediatric lung function laboratory with less experience in MBW testing, Zurich, Switzerland. The third aim was to evaluate agreement in MBW test acceptability between the operator and a retrospective reviewer.

2. Methods

2.1 Development of MBW quality control criteria
The quality control criteria used in this study was based on the ATS/ERS consensus statement guidelines, ATS pre-school MBW technical statement, and the publications by Jensen et al and Saunders et al.11,13-15. We used these guidelines to create a simplified matrix for qualitative assessment of clinical Nitrogen (N2) MBW measurements that can be applied at the time of the measurement and did not require any further retrospective assessment.
Our quality control criteria are presented in Table 1 and details of how our criteria differ from the ERS/ATS consensus statement are provided in Supplemental Table E1. Detailed instructions on how to apply the guidelines are presented in the online supplemental. An A grade represents a perfect trial with relaxed, regular tidal breathing throughout the measurement, a B grade represents a good quality trial with only minimal deviations, and a C grade represents an acceptable trial with moderate deviations but no highly abnormal breaths during the pre-phase or start of washout. A, B and C grade trials are considered acceptable for outcome reporting. D grade represents trials with questionable quality due to variable breathing patterns, abnormal breaths, or evidence for hypo- or hyper-ventilation. D grade trials have no signs of leak and satisfy both the start and end of test criteria. Generally, D grade trials should be rejected and not used for reporting, however, sometimes the deviations in tidal breathing in a D trial do not significantly impact the primary outcomes. Therefore, we propose that D grade trials can be accepted if the primary outcomes (LCI and FRC) are within 10% of an acceptable trial (A, B or C grade). An F grade represents trials that need to be rejected due to not meeting the technical acceptability criteria for MBW: 1) Start of test criteria not met (last three breaths of pre-phase N2-concentration with normalized N2 concentration ≥ 77%); 2) End of test criteria not met (three consecutive tidal breaths with normalized N2 concentration < 2.5%; 3) No evidence of leaks (for detailed instruction see online supplemental).
The overall test occasion is classified as acceptable when (i) at least two trials are graded as acceptable (A, B or C), or (ii) one trial was graded as acceptable (A, B, or C) and one trial was graded as questionable (D) given that both FRC and LCI are within 10% of the A-C grade trial when there are only two trials or 10% of the median when there are three or more trials for this test occasion. A test occasion with only D trials should be rejected. We used the overall test repeatability criteria described in the consensus document (i.e. FRC variability within 25%)11. MBW outcomes from acceptable and repeatable test occasions are reported as the mean from all acceptable trials.
2.2 MBW data collection and study population
The N2MBW measurements were collected using the Exhalyzer D device (Ecomedics, Duernten, Switzerland) with Spiroware software (version 3.2.1) and were performed according to international guidelines 11 in both centers. We approached all pediatric patients with CF attending their regular three monthly outpatient clinic visits aged 4 to 18 years. Approval was obtained from the local ethics committee in Bern. Patients and caregivers gave informed consent.

2.3 Test acceptability before implementation of real-time MBW quality control in Bern

Before implementation of real-time MBW quality control criteria into clinical routine in our centre, MBW operators were trained in data collection and general test acceptability. However, due to time restrictions, they were not required to perform a detailed assessment of test quality during the measurement. They also did not routinely mark trial classification on the lung function reports. To assess the quality of these MBW measurements, 31 clinic visits from children with CF aged five to 18 years were evaluated. The visits were randomly selected by an independent person not involved in this study and only one visit per patient was included in the analysis. Retrospective quality control was performed by an experienced reviewer who was involved in the development of the criteria and was blinded to any test comments by the MBW operator. The reviewer graded each trial individually and then assessed overall test acceptability.

2.4 Test acceptability after implementation of real-time MBW quality control in Bern

To implement real-time quality control of MBW measurements, operators in our center received instruction on how to perform quality control. A printed copy of the quality control criteria matrix was provided and operators were given a presentation on how to use the matrix, grade individual trials, determine test acceptability and repeatability, and report outcomes (detailed information provided in online supplement). All MBW operators were required to perform real-time quality control of each MBW measurement. The operators reported a grade for each trial and provided a standardized comment regarding the acceptability of the test occasion in the clinical report (example provided in the online supplement).
To assess the quality of MBW measurements after implementation of real-time quality control in routine clinical testing, 32 clinic visits from the same population of children with CF were evaluated. The visits were randomly selected by an independent person not involved in this study and only one visit per child was assessed. The experienced reviewer performed retrospective quality control of these measurements while being blinded to the real-time quality control assessment of the operator.