The Pneumonia Severity Index: A Decade after the Initial Derivation and Validation
- 1Division of General Internal Medicine, University of Lausanne, Lausanne, Switzerland
- 2Veterans Affairs Center for Health Equity Research and Promotion, Veterans Affairs Pittsburgh Healthcare System, University of Pittsburgh, Pittsburgh, Pennsylvania
- 3Division of General Internal Medicine, Department of Medicine, University of Pittsburgh, Pittsburgh, Pennsylvania
- Reprints or correspondence: Dr. Drahomir Aujesky, Service de Médecine Interne, BH 10-622, Centre Hospitalier Universitaire Vaudois, 1011 Lausanne, Switzerland (drahomir.aujesky{at}chuv.ch).
Abstract
The prognosis of community-acquired pneumonia ranges from rapid resolution of symptoms and full recovery of functional status to the development of severe medical complications and death. The pneumonia severity index is a rigorously studied prediction rule for prognosis that objectively stratifies patients into quintiles of risk for short-term mortality on the basis of 20 demographic and clinical variables routinely available at presentation. The pneumonia severity index was derived and validated with data on >50,000 patients with community-acquired pneumonia by use of well-accepted methodological standards and is the only pneumonia decision aid that has been empirically shown to safely increase the proportion of patients given treatment in the outpatient setting. Because of its prognostic accuracy, methodological rigor, and effectiveness and safety as a decision aid, the pneumonia severity index has become the reference standard for risk stratification of community-acquired pneumonia.
Understanding the prognosis of community-acquired pneumonia (CAP) is important from clinical, research, and quality-improvement perspectives [1]. From a clinical perspective, accurate prognostication allows physicians to inform patients about the expected outcomes of an acute illness. Furthermore, the ability to quantify the probability of serious adverse events (i.e., severe medical complications or death) can assist physicians in their initial management decisions, such as determining the most appropriate site of treatment (home vs. hospital), the intensity of hospital management (medical floor vs. intensive care unit), and the intensity of diagnostic testing and/or antibiotic therapy.
From a research perspective, both clinical trials and comparative effectiveness studies often compare the clinical outcomes of ⩾1 group of patients who differ with respect to the performance of a specified process of care (e.g., performance of blood cultures or initiation of antibiotic therapy within 4 h after presentation) or antibiotic treatment regimen. In such studies, it is imperative to assess whether observed differences in patient outcomes between treatment groups are confounded by inherent differences in illness severity, determined using well-validated severity measures. Objective risk stratification is also useful for comparing illness severity across study populations ascertained over different time periods and across diverse geographic areas, to ensure that study findings are comparable and generalizable to the types of patients cared for in a given clinical setting.
From a quality-improvement perspective, prediction rules for prognosis can be used to calculate severity-adjusted mortality rates in ongoing health care quality-assurance programs. Observation of severity-adjusted mortality rates that are statistically significantly higher than expected when aggregated at either the physician or the hospital level may identify deficiencies in quality of care. In the future, these severity-adjusted mortality rate estimates may be used to identify performance outliers and/or as part of pay-for-performance policies that remunerate providers, hospitals, and health care systems on the basis of such quality metrics.
Over the past decade, several pneumonia-specific prediction rules for prognosis have been published that define severity of illness on the basis of a predicted risk of short-term mortality [2–11]. We focus on the most rigorously studied prediction rule, the pneumonia severity index (PSI) [7], which categorizes patients into 5 risk classes, each with an incremental likelihood of mortality within 30 days (figure 1). Our goals are (1) to describe the derivation and validation of the PSI, (2) to review its primary clinical application, and (3) to compare the performance of the PSI with that of another commonly used disease-specific prediction rule for prognosis, on the basis of empirical studies.
Figure 1
Assignment to risk class based on the pneumonia severity index. BUN, blood urea nitrogen; yr, years.
Derivation and Validation of the PSI
The PSI was originally developed as part of the Pneumonia Patient Outcomes Research Team (PORT) project, with the goal of deriving a clinically applicable prediction rule for short-term mortality among patients with CAP [7]. The underlying hypothesis was that patients with CAP who are at low risk of mortality can be identified at presentation by use of readily available clinical information. The PSI was derived by analyzing data on 14,199 adult inpatients with a principal diagnosis of pneumonia based on the criteria of The International Classification of Diseases, Ninth Revision, Clinical Modification (or a secondary diagnosis of pneumonia with a primary diagnosis of sepsis or respiratory failure) in the 1989 MedisGroups Comparative Hospital Database. The database contained >250 demographic characteristics and baseline clinical variables for patients discharged from 78 hospitals in 23 US states [7]. In the derivation, we identified 20 prognostic variables that were independently associated with mortality and that were routinely available to physicians at the time of patient presentation: 3 demographic characteristics (age, sex, and nursing home residence), 5 coexisting illnesses (active neoplastic disease, congestive heart failure, cerebrovascular disease, renal disease, and liver disease), 5 findings of physical examination (pulse rate, respiratory rate, systolic blood pressure, temperature, and mental status), 6 laboratory measurements (blood urea nitrogen, glucose, hematocrit, and sodium levels; partial pressure of arterial oxygen; and arterial pH), and 1 radiographic finding (pleural effusion). The primary outcome was in-hospital mortality within 30 days after admission. The PSI was developed in 2 steps, to parallel more closely the clinical decision-making processes of physicians in the office or emergency department setting. In step 1, the PSI identified a subgroup of patients at very low risk of death (risk class I) solely on the basis of the presence or absence of 11 findings of medical history and physical examination (figure 1). In step 2, the risk of death was quantified for the remaining, non–risk class I patients by use of the same findings used in step 1 in addition to the laboratory and radiographic variables that compose the PSI. A total point score is calculated by summing the integer-based prognostic weights of each prognostic variable identified for a given patient. On the basis of the total point score, patients are classified into 4 additional risk classes (II–V), each with an increased probability of mortality (figure 1).
The PSI was then retrospectively validated using a 1991 Pennsylvanian MedisGroups database of 38,039 adults hospitalized with CAP in 193 hospitals in Pennsylvania [7]. The PSI was also validated in 2287 US and Canadian inpatients and outpatients, aged ⩾18 years, whose cases were managed at 5 medical centers participating in the Pneumonia PORT prospective cohort study [7]. No statistically significant differences in mortality were found across each of the 5 risk classes among the initial derivation and 2 validation cohorts (table 1). There was also a statistically significant relationship between higher risk class and other adverse medical outcomes in the PORT cohort. For example, among outpatients, there was a statistically significant increase in the risk of subsequent hospitalization with higher risk class; among inpatients, the rates of admission to the intensive care unit and the length of hospital stay also increased with higher risk class. Among the 1575 patients in the 3 lowest risk classes who participated in the PORT cohort study, there were only 7 deaths (0.4%), 4 of which were pneumonia related. Thus, the PSI is a validated prediction rule for prognosis that accurately identifies patients with CAP who are at low risk for mortality and other adverse outcomes.
Table 1
Thirty-day mortality rate by pneumonia severity index (PSI) risk class in the derivation and validation cohorts.
One of the major strengths of the PSI that distinguished it from previous prognostic models for CAP was that it met the vast majority of methodological standards for the derivation and validation of clinical prediction rules [12, 13]. It was derived using well-defined and relevant predictor variables and an unambiguous set of outcomes. The methods used to construct the rule avoided any biases in the assessment of the relevant predictor or outcome variables. The original work adequately described the underlying mathematical and/or statistical models and disclosed the rule's expected error rates. The generalizability and reproducibility of the rule were supported by its validation in thousands of patients across hundreds of clinical sites and diverse geographic areas. Although the PSI was used to make projections of its effectiveness and safety in guiding the initial site of treatment for patients with CAP in the PORT cohort study, the initial derivation and validation of the PSI fell short of assessing the effect of this prediction rule on actual patient care. Such applications awaited further clinical studies of the safety and effectiveness of the PSI as a decision aid.
Effectiveness and Safety of the PSI in Guiding Clinical Practice
Since the publication of the PSI in 1997, 5 studies have assessed the impact of the PSI on guiding the decision about the initial site of treatment for patients with CAP (table 2). In a prospective, quasi-experimental study conducted by Atlas et al. [14] in a single US hospital emergency department, physicians were provided with the PSI risk class and the risk class–specific mortality rates for 166 consecutive low-risk (risk classes, I–III) patients with CAP, coupled with a recommendation to provide treatment to these patients in the outpatient setting. Compared with a historical control group of 147 low-risk patients with CAP, the percentage of patients who initially received treatment as outpatients increased statistically significantly, from 42% during the historical control period to 57% during the intervention period, without an increase in the 30-day mortality rate. However, during the intervention period, more outpatients were subsequently admitted to the study hospital, compared with during the historical control period (9% vs. 0%). In a large, cluster-randomized effectiveness trial that enrolled 1072 low-risk patients with CAP from 19 Canadian hospital emergency departments, the implementation of a critical pathway recommending outpatient care for patients in PSI risk classes I–III statistically significantly increased the proportion of low-risk patients treated in the outpatient setting, from 51% at control sites to 69% at intervention sites [15]. Patient mortality, readmission, and quality of life were not statistically significantly different between treatment arms. In a randomized efficacy trial conducted at 2 Spanish hospitals [16], 224 patients with CAP in PSI classes II and III were randomly allocated to receive outpatient or inpatient care. Outpatients had a statistically significantly higher rate of satisfaction with care, compared with inpatients, with no statistically significant differences in readmission or mortality rates. In a large, cluster-randomized effectiveness trial that enrolled 1901 nonhypoxemic patients with CAP in PSI risk classes I–III from 32 US hospital emergency departments [17], a guideline recommending outpatient care for low-risk, nonhypoxemic patients was implemented using low-, moderate-, and high-intensity guideline-implementation strategies. Patients whose cases were managed using the moderate- and high-intensity strategies received treatment as outpatients statistically significantly more frequently than did patients whose cases were managed using the low-intensity strategy (37.5% and 61.0% vs. 61.9%, respectively), without compromising patient safety. Finally, in a prospective, controlled study involving 449 low-risk patients with CAP from 16 French hospital emergency departments [18], 50.9% of nonhypoxemic patients in PSI risk classes I–III received treatment as outpatients in emergency departments that used the PSI, compared with 29.3% of patients in emergency departments that did not use the PSI. After adjustment for pneumonia severity, mortality was lower in the emergency departments that used the PSI.
Table 2
Studies that assessed the effectiveness and safety of the pneumonia severity index (PSI) for the initial site of treatment.
Current methodological standards require that a clinical prediction rule must demonstrate a beneficial effect on patient care in a formal impact analysis before its use as a decision rule can be recommended [19]. Overall, these 5 studies that enrolled a total of 3949 low-risk patients with CAP at 60 study sites in 4 countries (the United States, Canada, France, and Spain) uniformly demonstrate the positive impact of the PSI on patient care. With its validation according to strict methodological criteria, its broad validation, and its positive effect on patient care shown in various impact analyses, the PSI has achieved the highest level of methodological rigor (level 5 evidence) for a clinical prediction rule (table 3) [19]. According to current methodological standards, level 5 prediction rules may be used in various settings with confidence that the application will augment the quality and/or efficiency of patient care [19].
Table 3
Methodological standards for development and evaluation of clinical prediction rules.
Despite the methodological strengths of the PSI and the empirical evidence supporting its effectiveness in guiding the choice of the initial site of treatment, those who use the PSI clinically must be aware of its limitations. First, use of the PSI as a decision aid has been restricted to immunocompetent adults with CAP, with exclusion of children, pregnant women, patients with immunosuppression (e.g., HIV-infected patients), and those with hospital-acquired pneumonia [7]. Second, to simplify application of the rule, the PSI was constructed with dichotomous predictor variables (abnormal vs. normal), which may oversimplify the manner in which physicians interpret the results of some of its predictor variables. For example, a physician would be unlikely to discharge a previously healthy 25-year-old patient with severe systolic hypotension (e.g., systolic blood pressure, <60 mm Hg), tachycardia (e.g., pulse, >150 beats/min), and no additional pertinent prognostic factors, despite assignment to PSI risk class II. Third, patients designated as low risk (PSI risk classes, I–III) may have important medical and psychosocial contraindications to outpatient care [7]. For example, administration of oral antibiotics in an outpatient setting to patients with intractable vomiting is not an option. Likewise, patients who are injection drug users, abuse alcohol, or have severe psychiatric conditions may require hospitalization to ensure compliance with treatment. Patients with severely impaired cognitive function who are unable to perform activities of daily living independently and those with little social support may also require inpatient care regardless of the severity of illness. Finally, the fact that the PSI consists of 20 predictor variables complicates its use in clinical practice. However, the use of pocket cards, electronic handheld devices, or Internet support systems greatly facilitates the application of the PSI in clinical practice.
Comparison of the PSI with the CURB-65 Prediction Rule
Among the various clinical prediction rules for prognosis of CAP, the CURB-65 score has recently emerged as a potential alternative to the PSI [8]. On the basis of prediction rules originally developed to identify patients with severe pneumonia [2, 10], Lim et al. [8] used data from 1068 patients with CAP to derive and internally validate the CURB-65 prediction rule. The CURB-65 prediction rule assigns less prognostic importance to comorbid illnesses and uses 5 variables (confusion, urea level >7 mmol/L, respiratory rate ⩾30 breaths/min, low systolic or diastolic blood pressure, and age ⩾65 years) to assign a score on a 6-point scale (0–5). Scores of 0–1 are considered to indicate low risk, with a 30-day mortality rate of 0%–2.1% [8].
The 5 largest studies that directly compared the performance of the PSI with that of CURB-65 as disease-specific prediction rules for prognosis in immunocompetent adults have shown that the PSI identifies a slightly greater proportion of patients as at low risk for short-term mortality and has a slightly higher discriminatory power for mortality than does the CURB-65 score (table 4) [20–24]. In contrast to the PSI, the effectiveness and safety of using the CURB-65 score to guide clinical practice have not been assessed in clinical studies. Thus, the advantages of the PSI over the CURB-65 prediction rule are its broader, external validation; superiority in identifying low-risk patients; and proven benefits in guiding patient care in large-scale impact trials [5–17].
Table 4
Studies comparing the prognostic accuracy of the pneumonia severity index (PSI) and the CURB-65 score.
Because of the CURB-65 score's ease of use, compared with the PSI, the updated 2004 British Thoracic Society pneumonia guidelines recommend that patients with a CURB-65 score of 0 or 1 may be suited to outpatient treatment, even in the absence of a formal impact study of the CURB-65 score as a decision aid [25]. The 2007 guidelines from the Infectious Diseases Society of America and American Thoracic Society recommend the use of either the PSI or the CURB-65 score for screening of patients with CAP who are potential candidates for outpatient care [26].
Conclusion
Over the past decade, the PSI has evolved from a prediction rule for prognosis to a decision aid to guide the choice of the initial site of treatment for patients with CAP. This evolution was facilitated by strict adherence to accepted methodological standards in its development; broad, external validation across heterogeneous patient populations; and consistent findings from large clinical trials demonstrating its effectiveness and safety in increasing the proportion of low-risk patients who receive treatment in the outpatient setting. Direct comparisons of the PSI with other disease-specific prediction rules for prognosis suggest that the PSI identifies a slightly greater proportion of low-risk patients and has a slightly greater discriminatory power for predicting mortality. Because of its methodological rigor, superior prognostic accuracy, and proven effectiveness as a decision aid, the PSI has become the reference standard for severity adjustment and risk stratification of patients with CAP.
Acknowledgments
Supplement sponsorship. This article was published as part of a supplement entitled “Workshop on Issues in the Design and Conduct of Clinical Trials of Antibacterial Drugs for the Treatment of Community-Acquired Pneumonia,” sponsored by the US Food and Drug Administration and the Infectious Diseases Society of America.
Potential conflicts of interest. D.A. and M.J.F.: no conflicts.
- © 2008 by the Infectious Diseases Society of America
References
This Article
- AbstractFree
- » Full Text (HTML)Free
- Full Text (PDF)Free
Classifications
Services
Citing Articles
Google Scholar
PubMed
Related Content
Share
- Email this article
-
CiteULike
Connotea
Delicious
Facebook
Google+
Mendeley
Twitter
Published on behalf of
IDSA Members: For your free access to this journal, log in via the IDSA members area.
The Journal
Impact Factor: 8.186
5-Yr impact factor: 7.898
For The Media
For Authors
- Services for authors
- Self-archiving Policy
- IDSA Conflict of Interest policy
- Instructions for Authors
- ICMJE Form
- Submit Now!
Open access options for authors visit Oxford Open
This journal enables compliance with the NIH Public Access Policy
Corporate Services
Alerting Services
Related IDSA Publications
