Risk of Misleading Ventilator-Associated Pneumonia Rates with Use of Standard Clinical and Microbiological Criteria
- 1Department of Ambulatory Care and Prevention, Harvard Medical School and Harvard Pilgrim Health Care, Boston, Massachusetts
- 2Infection Control Unit, Channing Laboratory, Department of Medicine, Brigham and Women's Hospital, Boston, Massachusetts
- Reprints or correspondence: Dr. Michael Klompas, Dept. of Ambulatory Care and Prevention, Harvard Medical School and Harvard Pilgrim Health Care, 133 Brookline Ave., 6th Fl., Boston, MA 02215 (mklompas{at}partners.org).
Abstract
Ventilator-associated pneumonia (VAP) rates are advocated as a measure of hospitals' quality of care for critically ill patients. The standard definition used to measure VAP rates, however, is constructed of nonspecific clinical signs common to many common complications of critical care. We created a model in which we estimated the probability of patients with 6 different complications of critical care fulfilling diagnostic criteria for VAP. We then calculated how the apparent prevalence of VAP varies depending on the prevalence of these other conditions in an intensive care unit. Despite keeping the true, underlying prevalence of VAP fixed at 10%, the apparent rate of VAP varied between 6.0% and 31.6%, depending on the prevalence of other conditions. The addition of microbiological criteria to standard clinical criteria decreased the range of apparent VAP to 3.5%–15.5%. These wide margins of variability suggest that VAP rates are an unreliable measure of quality of care.
Ventilator-associated pneumonia (VAP) rates are increasingly being advocated by regulators and legislators as measures of hospital quality. New policies of the Joint Commission and the Centers for Medicare and Medicaid Services propose linking hospitals' accreditation and compensation to avoidance of complications of care, possibly including VAP. Linking accreditation and compensation to the incidence of VAP is problematic, however, because there is considerable uncertainty in rendering a VAP diagnosis.
There are no specific clinical signs or blood tests to aid in diagnosing VAP [1]. Instead, clinicians and infection control practitioners try to integrate multiple nonspecific signs—such as fever, increased pulmonary secretions, abnormal leukocyte counts, radiographic opacities, and pulmonary secretion culture results—to diagnose VAP. These clinical signs, however, are common to a host of other conditions frequently present in critically ill patients. Fever and leukocytosis can be caused by catheter-associated bloodstream infections, abscess, or thromboembolic disease. Changes in pulmonary secretions can reflect nonspecific underlying lung injury and interruption of normal pulmonary clearance mechanisms due to illness and intubation. New radiographic opacities are found with atelectasis, acute respiratory distress syndrome, pulmonary edema, hypersensitivity pneumonitis, or pulmonary hemorrhage. Even quantitative culture of lower respiratory tract specimens is not perfectly diagnostic, because the bronchoscopist can miss the area of active infection and fluid specimens can be contaminated by bacteria colonizing the endotracheal tube and oropharynx [2].
Because multiple alternative diagnoses can mimic the clinical signs of VAP, the apparent rate of VAP in an intensive care unit (ICU) might vary significantly, depending on the prevalence of other conditions diagnosed on the unit. To assess this possibility, we created a simple mathematical model in which the true underlying rate of VAP in an ICU is kept constant while the rates of other common conditions are varied. We report the potential variation in perceived VAP rates depending on the frequency of other conditions in an ICU population.
We began by estimating the probability of diagnosing VAP in patients with VAP and in patients with 5 other conditions that might mimic VAP (i.e., pulmonary edema, sepsis, acute respiratory distress syndrome, pulmonary embolism, and atelectasis). We used the infection surveillance diagnostic criteria published by the Centers for Disease Control and Prevention (CDC) to define VAP [3]. According to CDC criteria, VAP can be diagnosed in a patient receiving mechanical ventilation who has a new radiographic opacity, a change in pulmonary secretions along with evidence of impaired gas exchange, and either fever or an abnormal WBC count. A lower respiratory tract specimen with positive quantitative culture results is an optional additional CDC criterion for the diagnosis of VAP. We estimated the frequency with which fever, leukocytosis, change in pulmonary secretions, new radiographic opacities, and lower respiratory tract specimens with positive culture results occur in patients with VAP and with our 5 other target conditions. Estimates of the frequency of each sign in each disease were based on values reported in the medical literature, when available [2, 4–16], or our best estimates when literature values could not be located. A range of feasible values was associated with each clinical sign and disorder, to account for uncertainty of the true sign frequency as well as for interobserver variability. Frequency estimates were then combined to calculate the probability of a patient fulfilling CDC criteria. We generated these estimates for patients with and without positive cultures from lower respiratory tract specimens to mirror the flexibility permitted by CDC criteria. Frequencies for each condition and the associated probability of diagnosing VAP are shown in table 1.
Figure 1
Measured prevalence of ventilator-associated pneumonia (VAP) as a function of the prevalence of pulmonary edema and acute respiratory distress syndrome (ARDS). The figure shows how the apparent prevalence of VAP will vary as the prevalence of pulmonary edema and ARDS increases and decreases in a hypothetical intensive care unit. The true rate of VAP is 10% under all circumstances depicted in the figure.
Table 1
Prevalence of signs consistent with ventilator-associated pneumonia (VAP) and probability of diagnosing VAP in patients with various conditions common to critically ill patients.
We then calculated the apparent prevalence of VAP for the ICU as a whole, using literature reports to estimate the prevalence of each alternative condition in a theoretical ICU population [15, 17–22]. It should be noted that the true frequency of many of these disorders is, at best, an estimate, given the absence of a reliable reference standard to diagnose many of these conditions. We multiplied the prevalence of each condition by the probability of labeling the condition to be VAP and then summed the results to yield an overall prevalence of VAP for the population. In accordance with published clinical series of patients receiving mechanical ventilation, we presumed that individual patients can have >1 disorder concurrently [23, 24]. For example, under the assumption of a true underlying rate of 10% patients with VAP, an ICU population in which 20% of patients have pulmonary edema, 20% have sepsis, 5% have acute respiratory distress syndrome, 5% have pulmonary embolism, and 20% have atelectasis would appear to have a prevalence of VAP of 13.7% as determined by purely clinical criteria or 6.6% with inclusion of microbiological criteria (see table 2).
Table 2
Variation in the apparent prevalence of ventilator-associated pneumonia (VAP), depending on the frequency of other diseases in an intensive care unit population.
We then kept the true underlying incidence of VAP fixed at 10% but varied the prevalence of pulmonary edema in our ICU between 0% and 50% and varied the prevalence of acute respiratory distress syndrome between 0% and 10% while holding the prevalence of all other diseases constant. As we varied the rates of pulmonary edema and acute respiratory distress syndrome, we saw that the perceived rate of VAP in our theoretical ICU had a range of 7.6%–21.6%, despite keeping the “true” rate of underlying VAP fixed at 10%. These results are shown in figure 1. The inclusion of microbiological criteria would have reduced the variability to 4.3%–9.9%.
We did a similar analysis in which we varied the rates of all 5 potentially mimicking conditions over a range of realistic values, to estimate the range of perceived VAP prevalence in a more heterogeneous setting. As before, we kept the true underlying rate of VAP fixed at 10%. The final perceived rates for VAP depending on the frequency of other conditions in the ICU are shown in table 2. We estimate that the perceived prevalence of VAP could have a range of 6.0%–31.6%, depending on the prevalence of other diagnoses in an ICU population. The inclusion of microbiological criteria lowered the range of apparent VAP prevalence to 3.5%–15.5%.
We repeated this analysis while fixing the true underlying rate of VAP at 1%, given that some hospitals are now reporting rates of VAP much lower than those of historical data. We found that the estimated prevalence of VAP could have a range of 1.1%–27.1% with use of strictly clinical criteria or of 0.4%–11.5% with the incorporation of microbiological criteria. An Excel (Microsoft) spreadsheet is included [(online only) so that readers can calculate their own apparent VAP rates for any combination of 6 conditions and their clinical sign frequencies.
This demonstration that the apparent rate of VAP might vary by up to 5-fold in an ICU, depending on the frequency of other common disorders, makes VAP a questionable basis for assessing hospital quality. Clinical outcome measures selected to assess quality of care and to provide hospital benchmarks ought to be objective, reproducible, and meaningful. The lack of specificity in the diagnostic criteria for VAP means that other conditions common to ICUs can masquerade as VAP and thereby inflate the apparent prevalence of VAP. VAP rates are likely to fluctuate in accordance with changes in the distribution of other pulmonary conditions in a hospital's ICU population. In addition, hospitals with a high volume and a complex mix of patients receiving mechanical ventilation risk being penalized because of the breadth of their population receiving mechanical ventilation.
Compounding the lack of specificity inherent in the criteria used to diagnose VAP is the fact that many of the criteria are also highly subjective. For example, a change in pulmonary secretions is defined as an increase in the quantity of secretions, new-onset purulent sputum, increased suctioning requirements, or a change in the color, consistency, and odor of the sputum. Quantitative thresholds are not specified. Likewise, criteria for assessment of worsening gas exchange is left to the discretion of the observer.
Finally, the flexibility permitted to hospitals as to whether to incorporate microbiological criteria into their VAP definition magnifies the variability inherent in VAP surveillance. Hospitals that choose to include microbiological criteria in their VAP definition will have lower (albeit still highly variable) VAP rates than will those using purely clinical criteria. VAP rates reported to the CDC do distinguish VAP diagnosed purely on clinical grounds versus VAP with culture data included, but this subtle distinction is rarely made explicit in public reporting [23].
With the increasing pressure being placed on hospitals to report measures of quality, including VAP, there is a risk that hospital staff will subconsciously interpret the VAP definition to minimize their apparent VAP rates [24]. This could include variable interpretation of the signs described above or even just well-meaning incorporation of microbiological criteria into routine VAP surveillance. Some hospitals have already begun to report near-zero rates of VAP over extended intervals that contrast sharply with historical reports of VAP rates [23, 25, 26]. Although we hope that these dramatic improvements in VAP rates reflect meaningful changes in patient care, the variability inherent in the current VAP definition limits our ability to know whether these improvements reflect changes in measurement or bona fide decreases in histological pneumonias.
These limitations in the specificity, objectivity, and consequent reproducibility of the current VAP definition should preclude the use of VAP to compare hospitals or to make decisions about accreditation and compensation.
Acknowledgments
Potential conflicts of interest. All authors: no conflicts.
- Received October 10, 2007.
- Accepted December 24, 2007.
- © 2008 by the Infectious Diseases Society of America
References
This Article
- AbstractFree
- » Full Text (HTML)Free
- Full Text (PDF)Free
- Supplementary File
Classifications
Services
Citing Articles
Google Scholar
PubMed
Related Content
Share
- Email this article
-
CiteULike
Connotea
Delicious
Facebook
Google+
Mendeley
Twitter
Published on behalf of
IDSA Members: For your free access to this journal, log in via the IDSA members area.
The Journal
Impact Factor: 8.186
5-Yr impact factor: 7.898
For The Media
For Authors
- Services for authors
- Self-archiving Policy
- IDSA Conflict of Interest policy
- Instructions for Authors
- ICMJE Form
- Submit Now!
Open access options for authors visit Oxford Open
This journal enables compliance with the NIH Public Access Policy
Corporate Services
Alerting Services
Related IDSA Publications
