Background.Diagnosis of invasive fungal disease (IFD) is challenging, and it remains a significant cause of morbidity and mortality in immunocompromised patients. The (1→3)-β-d-glucan (BG) assay may be a useful adjunct, but its diagnostic performance is not well characterized.
Methods.We retrospectively assessed the diagnostic indices of the BG assay in patients at risk of IFD who had a compatible clinical syndrome for the diagnosis of IFD a week after initial BG testing and at the end of the hospitalization associated with the first BG value. Patients with IFD were classified according to current European Organization for Research and Treatment of Cancer-Mycoses Study Group criteria, independent of BG results.
Results.A total of 1308 BG assays were performed for 871 patients. One hundred twelve proven or probable IFD cases were diagnosed within 1 week after initial testing, and 116 cases were diagnosed by the end of hospitalization. Sensitivity of an initial BG level ⩾80 pg/mL for IFD at 1 week was 0.64 (95% confidence interval [CI], 0.55–0.73), specificity was 0.84 (95% CI, 0.81–0.86), the positive likelihood ratio was 3.93 (95% CI, 2.94–5.26), and the negative likelihood ratio was 0.43 (95% CI, 0.31–0.59). Albumin, intravenous immunoglobulin, and hemodialysis were associated with elevated BG levels in patients without IFD (odds ratio, 4.78; 95% CI, 2.59–8.80). After excluding patients with these factors, specificity and the positive likelihood ratio of an initial BG level ⩾80 pg/mL increased slightly. Empirical systemic antifungal treatment did not reduce overall BG sensitivity. Sensitivity was slightly lower among patients with hematologic malignancy or stem cell transplantation. Consideration of BG results would have increased the diagnostic certainty to probable in 54% of possible IFD cases.
Conclusions.BG level appears to be a fair diagnostic adjunct for IFD in patients with appropriate pretest probability and a suggestive clinical syndrome, especially when checked serially in patients not receiving factors associated with an elevated BG level in the absence of IFD.
Invasive fungal disease (IFD) remains a significant cause of morbidity and mortality in immunocompromised patients despite the availability of effective therapies [1-5]. Timely and accurate diagnosis is essential but challenging because of nonspecific clinical and radiographic findings, underlying patient debilitation that precludes potentially definitive diagnostic procedures, and low yield of cultures. Culture-independent serum antigen detection tests, such as the (1→3) β-d-glucan (BG) assay and galactomannan (GM) index may allow earlier diagnosis of IFD than is otherwise feasible with traditional methods [6-12].
Several studies have examined systematic BG screening in patients with hematologic malignancy, who are at high risk of IFD [7-10, 13-15], and a few studies have assessed posthoc BG characteristics in patients who previously received a diagnosis of IFD [16-18]. BG performance remains poorly defined as a diagnostic adjunct in patients at risk of IFD with compatible clinical syndromes, the manner in which the assay is commonly used in clinical practice. We therefore sought to evaluate the diagnostic performance and clinical usefulness of the BG assay in a large clinical cohort of patients at risk of IFD during the first 2 years of its use at our institution.
Patient selection and data collection. We reviewed the records of all inpatients at the Brigham and Women's Hospital-Dana-Farber Cancer Institute (Boston, MA) who had at least 1 BG assay result from October 2004 through September 2006. Demographic data, baseline diagnoses, immunosuppressive or systemic antifungal therapy, reasons for BG testing, and results of relevant laboratory, microbiology, radiology, and pathology studies were recorded. We collected information about factors potentially associated with BG level elevations in the absence of IFD, including hemodialysis with cellulose-containing membranes [19-21], administration of albumin and intravenous immunoglobulin [22-25], and parenteral antibiotics known to contain BG [26, 27].
Diagnostic certainty classification. Patients were categorized as having proven, probable, possible, or no IFD by 2 separate reviewers using 2008 European Organization for Research and Treatment of Cancer-Mycoses Study Group (EORTC-MSG) diagnostic criteria [28], independent of BG results. In addition, Pneumocystis jiroveci pneumonia (PCP) was considered to be probable IFD if underlying host factors, a compatible clinical syndrome, consistent radiology findings, and cysts in induced sputum or bronchoalveolar lavage specimens were present. We determined diagnostic certainty at 2 time points for each patient, according to available evidence 1 week after the first BG assay was performed and at the end of the first hospitalization associated with BG assay use.
BG testing. The Food and Drug Administration-approved Fungitell assay (Associates of Cape Cod) was used to determine serum BG levels. Beacon Diagnostics (Falmouth, MA) performed all testing without knowledge of clinical characteristics of tested patients. BG testing was performed at the discretion of clinical care teams.
Statistical methods. Proven and probable IFD were considered to be true-positive cases for analysis. Patients with possible or no IFD were considered to have true-negative cases. Because of the inherent uncertainty of the EORTC-MSG probable and possible classifications in capturing true-positive IFD, sensitivity analyses were performed to define IFD cases and non-IFD cases according to case groupings A (proven IFD vs no IFD), B (proven and probable IFD vs no IFD), and C (proven, probable, or possible IFD vs no IFD), as described elsewhere [29].
We calculated sensitivity, specificity, and likelihood ratios for several BG assay strata and plotted relevant receiver operating characteristic (ROC) curves. To minimize bias intrinsic to nonsystematic serial testing, we determined diagnostic performance of the first BG assay per patient for IFD diagnosis a week after testing, a clinically relevant time frame in which the diagnosis of IFD should either have become more firmly established by additional evaluations or made less likely by clinical improvement without antifungal therapy or determination of an alternative, non-IFD diagnosis.
These parameters were also calculated for the highest BG level during the first hospitalization for IFD diagnosis at discharge, to allow comparability to prior BG performance studies that reported per patient-episode analyses [6, 8-10, 13-15]. Ninety-five percent confidence intervals (95% CIs) for sensitivity and specificity were estimated using exact binomial distributions. Likelihood ratios and their 95% CIs were estimated using log-linear regression models. The area under each ROC curve (AUC) was estimated nonparametrically [30]. ROC AUC comparisons were performed using the method of Hanley and McNeil [31]. Youden index values were calculated for each initial and highest hospitalization BG value to determine the BG threshold with optimal overall diagnostic effectiveness [32].
Least squares linear regression was used to determine statistically significant predictors of BG positivity after controlling for IFD. To determine the impact of potential confounders of test performance, we performed sensitivity analyses by incrementally excluding patients who received hemodialysis, albumin, intravenous immunoglobulin, or parenteral antibiotics known to contain BG within a week before initial testing.
Analyses were repeated for subsets of patients with active hematological malignancy or hematopoietic stem cell transplantation (HSCT), for patients presenting with a pneumonic syndrome or febrile neutropenia, and for patients with common specific infections, including invasive aspergillosis, invasive candidiasis, and PCP.
All analyses were performed using Stata, version 10 (Stata). The hospital's Human Research Committee approved this study.
Patient characteristics. A total of 871 patients were tested over the study period. Baseline characteristics are presented in Table 1. The majority of our cohort was composed of patients at particularly high risk of IFD; 530 patients (60.8%) had at least 1 predisposing EORTC-MSG-defined host factor. Most patients were tested in the context of a pulmonary or febrile syndrome (Table 2).
A, Receiver operating characteristic curve (ROC) for performance of initial (1→3)-β-d-glucan (BG) assay for invasive fungal disease (IFD) at 1 week for the entire cohort. B, ROC curve for performance of the highest BG level during hospitalization for IFD at the end of hospitalization for the entire cohort.
IFD events. One week after initial BG testing, there were 69 proven IFD cases (7.9%), 43 probable IFD cases (4.9%), 97 possible IFD cases (11.1%), and 662 non-IFD cases (76.0%). At the end of hospitalization, there were 80 proven IFD cases (9.2%), 36 probable IFD cases (4.1%), 93 possible IFD cases (10.7%), and 662 non-IFD cases (76.0%). Table 3summarizes fungal species associated with proven and probable IFD at these time points.
European Organization for Research and Treatment of Cancer-Mycoses Study Group Proven or Probable Invasive Fungal Disease (IFD) Cases, by Fungal Species, Independent of (1→3)-β-d-Glucan (BG) Assay Results
Test use. A total of 1308 BG assays were performed. There was no difference in the number of assays performed for patients with or without IFD; patients in both groups had a median of 1 assay performed (interquartile range (IQR), 1–2 assays; range, 1–10 assay). The median initial BG value was <31 pg/mL (IQR, <31 to 46 pg/mL; range, <31 to >500 pg/mL) in patients without IFD and 157 pg/mL (IQR, 49 to >500 pg/mL; range <31 to >500 pg/mL) in patients with IFD at 1 week.
Overall performance of the BG assay. Full diagnostic indices for a range of cutoff strata for an initial BG assay for IFD at 1 week are presented in Table 4, and the corresponding ROC curve and AUC are shown in Figure 1A. Sensitivity of a highest BG level ⩾80 pg/mL for IFD at the end of hospitalization was 0.71 (95% CI, 0.62–0.79), specificity was 0.81 (95% CI, 0.78–0.84), the positive likelihood ratio was 3.71 (95% CI, 2.83–4.86), and the negative likelihood ratio was 0.36 (95% CI, 0.26–0.51). The corresponding ROC curve and AUC for the end of hospitalization analysis are presented in Figure 1B. There were no significant changes in these indices with use of methods A, B, or C to define true-positive cases and true-negative cases in either analysis set. There were no changes with the exclusion of the few asymptomatic patients who were tested.
Diagnostic Indices of Initial (1→3)-β-d-Glucan (BG) for Proven or Probable Invasive Fungal Disease (IFD) within 1 Week after Testing
Test performance excluding patients with factors that elevate BG in the absence of IFD. In patients who received albumin, intravenous immunoglobulin, or hemodialysis during the week before testing, the pooled OR of having an initial BG level ⩾80 pg/mL in the absence of IFD was 4.78 (95% CI, 2.59–8.80), compared with patients who were not exposed to these factors. Diagnostic indices were reevaluated after excluding 61 patients with these factors. There was no discernible effect on sensitivity of either an initial or highest BG level for IFD, although specificity and the corresponding positive likelihood ratio slightly increased for both analyses.
No parenteral antibiotics were associated with elevated BG levels in patients without IFD. There were no cases of significant surgical gauze exposure [33, 34] or Alcaligenes bacteremia [7], and there was only 1 case of Pseudomonas aeruginosa bacteremia [35]; all of these conditions have been reported to elevate BG level in the absence of IFD. The single patient with P. aeruginosa bacteremia had a BG level >500 pg/mL without IFD. Three patients with P. aeruginosa pneumonia did not have elevated serum BG levels.
Effect of empirical antifungal therapy on diagnostic performance. At the time of initial BG testing, 248 patients (28.5%) were receiving systemic antifungal therapy for a median duration of 3 days (IQR, 1–11 days; range, 1–90 days) prior to their initial BG test. Most were receiving antifungal therapy for empirical treatment of febrile neutropenia (61.3%), mucosal candidiasis (24.1%), and other clinical syndromes (Table 5). To analyze the potential effect of systemic antifungal exposure on the performance of the BG assay, we compared indices in patients who had not received antifungal treatment at time of initial testing with those of patients exposed for 1–7 days and those exposed for >7 days of antifungal treatment (Table 5) [36]. There were no appreciable differences in any of the test characteristics or the ROC AUC when the 3 groups were compared (P=.85). There was no obvious difference in the causative fungal species among these groups (Table 5).
Performance Characteristics of the (1→3)-β-D-Glucan (BG) Assay for Proven or Probable Invasive Fungal Disease (IFD) with or without Antifungal Exposure Before Testing
Performance of the BG assay in selected subsets of the cohort. Diagnostic parameters for BG level ⩾80 pg/mL in the subset of patients in our cohort with hematologic malignancy or HSCT who were not receiving intravenous immunoglobulin, albumin, or dialysis are presented in Table 6. BG assay sensitivity was somewhat lower and specificity was slightly higher in these patients than in the cohort as a whole.
Diagnostic Indices of an Initial (1→3)-β-D-Glucan (BG) Level ⩾80 pg/mL in Selected Patient Populations, Excluding Patients Who Received Intravenous Immunoglobulin, Albumin, or Hemodialysis
Diagnostic indices of a BG level ⩾80 pg/mL for the diagnosis of IFD in the most common presenting syndromes are presented in Table 6. Sensitivity of the BG assay for IFD in febrile neutropenia was lower than in other presenting syndromes, although specificity was quite high.
Performance of the BG assay for patients with invasive aspergillosis, invasive candidiasis, and PCP. Thirty-two patients had proven or probable invasive aspergillosis at 1 week and at the end of hospitalization. Initial and highest BG levels were ⩾80 pg/mL in 24 cases, for a sensitivity of 0.75 (95% CI, 0.57–0.89). Initial and highest GM indices were ⩾0.5 in 14 cases, for a sensitivity of 0.44 (95% CI, 0.26–0.62). Initial BG and GM levels were assessed simultaneously in 31 of these 32 patients; the GM index was checked 1 day after the initial BG testing in the remaining patient. Defining a clinically appropriate denominator for the calculation of specificity is problematic, because many syndromes overlap with invasive aspergillosis; however, including the remaining cases in which BG testing was performed for a pulmonary or sinus syndrome, specificity of an initial BG level ⩾80 pg/mL was 0.79 (95% CI, 0.74–0.83), the positive likelihood ratio was 3.52 (95% CI, 2.21–5.62), the negative likelihood ratio was 0.32 (95% CI, 0.16–0.64), and the ROC AUC was 0.81 (95% CI, 0.73–0.89) for invasive aspergillosis. Values were similar for a highest BG level ⩾80 pg/mL during hospitalization, with specificity of 0.76 (95% CI, 0.71–0.80), a positive likelihood ratio of 3.10 (95% CI, 1.96–4.91), a negative likelihood ratio of 0.33 (95% CI, 0.16–0.67), and an ROC AUC of 0.82 (95% CI, 0.75–0.89) for invasive aspergillosis. The GM index did not exceed 0.5 for any patient without invasive aspergillosis. Although GM index was considered in the assessment of diagnostic certainty, all 14 cases with a GM index ⩾0.5 had eventual independent mycological confirmation by culture or biopsy. Only 1 patient with invasive aspergillosis had a GM index ⩾0.5 without concurrent BG level elevation. Seven patients with invasive aspergillosis had both BG and GM results that failed to meet the usual threshold for positivity.
For 41 patients with proven or probable invasive candidiasis at 1 week, sensitivity of an initial BG level ⩾80 pg/mL was 0.63 (95% CI, 0.47–0.78). Sensitivity of a highest BG level ⩾80 pg/mL for invasive candidiasis at the end of hospitalization was 0.73 (95% CI, 0.58–0.85). Because of the difficulty of defining a suitable denominator composed of patients with a clinical syndrome suggestive of invasive candidiasis, we did not calculate specificity or other parameters for this subset.
Analyses were repeated for the entire cohort excluding patients with PCP, with no appreciable effect on BG diagnostic indices. Sensitivity of an initial or highest BG level ⩾80 pg/mL for probable PCP were both excellent at 0.92 (95% CI, 0.64–1.0), with a negative likelihood ratio of 0.10 (95% CI, 0.01–0.73). The ROC AUC for the initial BG assay was 0.89 (95% CI, 0.77–1.00) and for the highest BG assay was 0.89 (95% CI, 0.78–1.00).
Potential change in diagnostic certainty incorporating BG assay results. In 68 (54.0%) of 126 patients with possible IFD on the date of initial BG testing who were not receiving factors that elevate BG level in the absence of IFD, incorporating BG as mycological criteria would have increased the diagnostic certainty to probable IFD. Because of the absence of other mycological criteria, 37 of these cases remained classified as possible IFD at the end of hospitalization. As additional culture and biopsy data became available, 16 cases were reclassified as probable and 15 as proven IFD by the end of hospitalization.
Youden index calculations suggested 76 pg/mL as the BG cutoff value with maximal overall diagnostic effectiveness in our cohort. At this cutoff point, the sensitivity and specificity were 0.66 and 0.83, respectively, for the initial BG value and 0.72 and 0.80, respectively, for the highest BG value during hospitalization.
We assessed the performance of the BG assay in a large cohort of patients at risk of IFD who underwent syndrome-driven testing. To our knowledge, this represents the largest study of BG test characteristics to date and the only contemporary study of BG in which testing was prompted by clinical syndromes, as it is frequently used in clinical practice. An initial BG level ⩾80 pg/mL had sensitivity of 0.64, specificity of 0.84, a positive likelihood ratio of 3.93, and a negative likelihood ratio of 0.43 for confirmed IFD at 1 week. As expected, sensitivity improved when serial testing was performed and the highest BG level during hospitalization was considered, and specificity improved with exclusion of patients who received treatment associated with elevated BG level in the absence of IFD. Of interest, we found no decrease in sensitivity in the presence of systemic antifungal therapy, even in patients receiving >7 days of treatment, possibly because of the slow clearance of BG in some fungal infections [37] and the incidence of new breakthrough fungal infections with BG-containing organisms despite antifungal treatment. Consideration of BG results would have increased the diagnostic certainty to probable in 54% of 126 cases that were considered to be possible IFD at the time of BG testing.
BG testing has been assessed in the systematic surveillance of patients at high risk of infection [7-10, 13-15], but its characteristics are less established in the diagnostic setting. A few case-control studies have retrospectively assessed the posthoc performance of BG testing for patients who have already received a diagnosis of IFD, with reported sensitivity of 0.64–0.78, specificity of 0.71–0.98, positive predictive value of 0.87–0.89, and negative predictive value of 0.73–0.91 [16-18]. The variability of reported BG test characteristics in this context may be attributable to heterogeneity in derivation populations, case-control matching, BG assay used, antifungal prophylaxis rates, and delineation criteria of true-positive and true-negative results. In the only prior syndrome-driven study to date, Obayashi et al [6] reported initial and overall BG sensitivity of 0.76 and 0.90, respectively, for proven IFD in a heterogeneous group of febrile patients at variable risk of IFD, with specificity of 1.0, positive predictive value of 0.59, and negative predictive value of 0.97. BG assay sensitivity and specificity were both somewhat lower in our cohort, likely because of the inherent diagnostic uncertainty of patients in EORTC/MSG possible and probable IFD categories, who we included in our analysis, and by the inclusion of zygomycosis and cryptococcosis as IFD cases, even though these infections are not known to elevate the BG level [12].
The BG assay was somewhat less sensitive for IFD in patients with hematologic malignancy and HSCT, although specificity and the positive likelihood ratio were quite high. During screening, Hachem et al [15] similarly found BG to be less sensitive than expected in this population, with per-sample and per-patient sensitivity of only 0.58 and 0.67, respectively, for invasive aspergillosis and 0.47 and 0.63, respectively, for other mold infections. Limits of BG assay performance in this population at high risk of infection warrant further investigation.
Sensitivity of BG level ⩾80 pg/mL was substantially higher than sensitivity of GM index ⩾0.5 for invasive aspergillosis, and the agreement between these tests was only fair. Using BG level alone, we would have failed to detect 4 proven and 4 probable invasive aspergillosis cases, and using GM index alone, we would have missed 8 eventually proven and 10 probable cases of a total of 32 invasive aspergillosis cases. Our findings are comparable to the per-patient sensitivity values for invasive aspergillosis reported in prior screening and posthoc diagnostic studies [8, 14-17]. GM index sensitivity among patients who had BG testing performed was substantially lower than the 0.71 reported in a meta-analysis [38] and was at the lower end of the range of sensitivity values (0.38–0.88) for invasive aspergillosis reported to date in comparative assessments of BG level and GM index [8, 14-16]. Of 32 patients with invasive aspergillosis, 46.9% received antifungal therapy for 11 days (IQR, 4–60 days) before their initial BG level and GM value were determined, likely contributing to the low sensitivity of the GM index [36]. Despite their usefulness in clinical practice, the limitations of both serum antigen tests for the diagnosis of invasive aspergillosis highlight the need for development of better diagnostic modalities for this infection.
The sensitivity of the BG assay for invasive candidiasis was lower than that for invasive aspergillosis in our population. There is some evidence that the increase to peak serum BG level in invasive candidiasis may be slower than that in invasive aspergillosis [10], which may explain the low sensitivity of an initial BG assay for invasive candidiasis.
The BG assay appears to be an outstanding assay for excluding PCP in the appropriate clinical context, with its high sensitivity and low negative likelihood ratio. As described elsewhere [16, 37, 39, 40], PCP was associated with a robust elevation in BG level—the initial BG level was >500 pg/mL in 71.4% of PCP cases—that often preceded microbiologic diagnosis by several days. A few patients in our cohort had a clinical and radiographic syndrome suggestive of PCP, with highly elevated BG levels, and received empirical therapy for PCP with clinical improvement but without other microbiologic confirmation. The BG assay appears to be more sensitive than the current diagnostic standard for PCP in a fair number of cases, and the gold standard may need reevaluation [41].
We confirmed that intravenous immunoglobulin, albumin, or hemodialysis with cellulose membranes can elevate BG level in vivo in the absence of IFD. The BG assay must be used and interpreted cautiously, if at all, for patients with these factors.
Our study was limited by its retrospective design. We attempted to minimize the potential bias of nonsystematic testing by presenting data from the first BG assay per patient along with the per-patient analysis for the entire hospitalization. Also, all patients were enrolled from a single institution; performance estimates may have been affected by idiosyncrasies in the composition of our study population and institutional biases of when BG testing was performed.
The BG assay appears to be a fair diagnostic test for IFD in patients with appropriate pretest probability and a suggestive clinical syndrome, even in the presence of systemic antifungal therapy. Specificity and positive likelihood ratio were high in all groups, making a positive test result quite useful in guiding further management. Sensitivity was somewhat limited among patients with hematologic malignancy or HSCT, and a substantial number of invasive aspergillosis and invasive candidiasis cases would have been missed in the cohort with BG testing alone, even with the addition of GM testing for the diagnosis of invasive aspergillosis cases. Testing certainly does not obviate the need for vigilant clinical surveillance and prompt acquisition of further data to diagnose IFD if symptoms develop.
Potential conflicts of interest. All authors: no conflicts.
IDSA Members: For your free access to this journal, log in via the IDSA members area.
Open access options for authors visit Oxford Open
This journal enables compliance with the NIH Public Access Policy