Presented in part: 43rd Annual Meeting of the Infectious Diseases Society of America, San Francisco, California, 6–9 October 2005 [abstract 284].
Background. A double-sandwich enzyme-linked immunosorbent galactomannan assay has been approved for surveillance for invasive aspergillosis in immunocompromised patients. We undertook a meta-analysis to assess the accuracy of a galactomannan assay for diagnosing invasive aspergillosis.
Methods. Studies of the galactomannan assay that used the European Organization for Research and Treatment of Cancer or similar criteria as a reference standard and provided data to calculate sensitivity and specificity were included. Pooled sensitivity and specificity and summary measures of accuracy, Q* (the upper left-most point on the summary receiver-operating characteristic curve), mean D (a log odds ratio), and Youden index were calculated. Subgroup analyses were performed to explore heterogeneity.
Results. Twenty-seven studies from 1966 to 28 February 2005 were included. Overall, the galactomannan assay had a sensitivity of 0.71 (95% confidence interval [CI], 0.68–0.74) and specificity of 0.89 (95% CI, 0.88–0.90) for proven cases of invasive aspergillosis. The Youden index, mean D, and Q* were 0.54 (95% CI, 0.41–0.65), 2.74 (95% CI, 21.12–3.36), and 0.80 (95% CI, 0.74–0.86), respectively, indicating moderate accuracy. Subgroup analyses showed that the performance of the test differed by patient population and type of reference standard used. Significant heterogeneity was present.
Conclusions. The galactomannan assay has moderate accuracy for diagnosis of invasive aspergillosis in immunocompromised patients. The test is more useful in patients who have hematological malignancy or who have undergone hematopoietic cell transplantation than in solid-organ transplant recipients. Further studies with attention to the impact of antifungal therapy, rigorous assessment of false-positive test results, and assessment of the utility of the test under nonsurveillance conditions are needed.
Invasive aspergillosis (IA) occurs in 8%–15% of patients undergoing allogeneic stem cell transplantation and in 5%–15% of patients undergoing solid-organ transplantation [1]. Despite advances in therapy, IA is associated with considerable morbidity and mortality, ranging from 30% to 70% in transplant recipients [2]. Diagnosis of IA is challenging, because clinical and radiologic signs are very insensitive or nonspecific. Tissue biopsy as a means of making the diagnosis is invasive and is not always possible, especially among patients with thrombocytopenia. Early diagnosis leading to prompt institution of appropriate therapy may result in improved patient outcomes.
Much attention has been focused on developing a noninvasive test for diagnosing IA, particularly methods to detect galactomannan [1]. Galactomannan is a polysaccharide cell-wall component that is released by aspergillus during growth. Initial diagnostic assays including latex agglutination (Pastorex Aspergillus; Sanofi Diagnostics Pasteur) had poor sensitivity. More recently, a double-sandwich ELISA that incorporates the B 1–5 glactofuranose—specific EBA2 monoclonal antibody as both the detector and acceptor for galactomannan showed a high sensitivity with a threshold of 0.5 ng/mL and has been approved by the US Food and Drug Administration for use with serum samples (Platelia; Bio-Rad). However, results of test performance have been variable, and a systematic analysis of the overall accuracy of the test for surveillance of IA in high-risk populations has not been undertaken.
The goal of this meta-analysis was to characterize the clinical utility of the Platelia galactomannan assay for surveillance of IA in high-risk patient populations and to evaluate which variables affect its performance.
Literature search. Two investigators (C.D.P., N.S.) independently searched the published English- and Spanish-language literature using the Medline, PubMed, Cumulative Index to Nursing and Allied Health, and Cochrane Collaboration databases from inception to February 2005. Search terms included “Aspergillus,” “Aspergillosis,” “Platelia,” “galactomannan,” “diagnosis,” and combinations of these 5 terms. Abstracts of meetings of the Infectious Diseases Society of America, Focus on Fungal Infections, and InterScience Conference on Antimicrobial Agents and Chemotherapy were also reviewed. References from recent published reviews [1, 3] and from included studies were also perused. Authors of the original studies were contacted for additional information if data needed to calculate sensitivity and specificity were not provided.
Study selection criteria. We defined criteria for the inclusion of studies before reviewing specific reports. Studies of series of patients that used the European Organization of the Research and Treatment of Cancer/Mycoses Study Group (EORTC/MSG) criteria or similar criteria for diagnosis of Aspergillus infection [4] (table 1) and that provided data to calculate both sensitivity and specificity were included. Case reports and review articles were excluded. If a study appeared to meet selection criteria but had a patient population that appeared to be the same as or to overlap with the patient population of a similar study [5, 6], we chose to include the larger of the studies [6]. Studies that provided data only on serum samples, rather than patients, were excluded. Studies evaluating the performance of the galactomannan assay on specimens other than serum samples were excluded, as were all studies that assessed the performance of galactomannan assays other than the Platelia assay.
European Organization of the Research and Treatment of Cancer/Mycoses Study Group criteria for diagnosis of invasive aspergillosis.
Data extraction. We used a standard form to extract data on relevant characteristics. The Standards for Reporting of Diagnostic Accuracy statement was used to assess study quality [7]. However, studies were not excluded on the basis of quality.
We evaluated studies for all of the following characteristics: description of the study population, type of study (prospective or retrospective), the use of EORTC/MSG diagnostic criteria as the reference standard for case definition versus other similar criteria, the number of case patients and control subjects, the number and frequency of serum samples obtained, cutoff values for definition of a positive Platelia test result, the blinding of investigators to results, prevalence of IA, sensitivity, specificity, the use of antifungal therapy during testing, and the presence of bias that may have influenced results, particularly incorporation bias (i.e., using a positive galactomannan test result as part of the diagnostic criteria for IA).
Statistical analysis. We calculated pooled sensitivity and specificity with 95% CIs separately for proven IA and proven or probable IA. Cases of possible IA were considered to be negative for the purposes of analysis. Heterogeneity in the estimates of sensitivity and specificity was assessed using the Pearson χ2 test or Fisher's exact test.
To summarize the overall diagnostic ability of the tests, we computed the Youden index (sensitivity + specificity - 1) and D for each study. The Youden index is a summary measure of accuracy incorporating both sensitivity and specificity. We averaged these measures across studies and computed 95% CIs based on these averages.
D is a log OR that is the ratio of the odds that a person who has IA will have a positive test result to the odds that a person who does not have IA will have a positive test result. To account for heterogeneity, we also fit a summary receiver-operating characteristic (ROC) curve using the methodology proposed by Moses et al. [8] by linear regression of D on S. D = logit(TPR) - logit(FPR), and S = logit(TPR) + logit(FPR), where TPR is the true positive rate or sensitivity, and FPR is the false-positive rate or 1 - specificity. With use of these results, we then calculated Q*, where sensitivity = specificity on the summary ROC curve, corresponding to the upper left-most point on the summary ROC, and a 95% CI for Q* [9]. The summary measure Q* has been advocated over area under the ROC curve, because it is meaningful in the ROC region of greatest interest [9]. The higher the values of the Youden index, D, and Q*, the greater the accuracy of the test.
Finally, we computed positive predictive value and negative predictive value on the basis of the pooled estimates of the sensitivity and specificity. With use of a range of prevalences of 0.05–0.20, as reported in the literature and summarized by Wheat et al. [3], we examined how positive predictive value and negative predictive value changed for prevalences of 0.05, 0.10, 0.15, and 0.20.
We explored the reasons for heterogeneity by performing subgroup analyses. We examined whether characteristics of the patient population (e.g., presence of hematological malignancy, receipt of solid-organ transplant, or receipt of bone marrow transplant) affected the diagnostic accuracy of the Platelia assay. We also studied the performance of the test relative to the reference standard by stratifying studies according to those that used the EORTC/MSG criteria versus those that used similar (but not identical) criteria. Analyses were stratified by age to estimate the performance of the test for children and adults. For each analysis, we compared summary statistics across subgroups using 1-way analysis of variance. All statistical analyses were performed using S-PLUS software (MathSoft).
We identified 139 studies from our literature search. Of these, 29 reports encompassing 27 studies met our inclusion criteria. Figure 1 shows the literature search leading to selection of the final 27 articles. Tables 2 and 3 show detailed characteristics of these studies. If the study included both children and adults but was stratified by age, data are presented separately for children and adults.
Summary of study assessment and inclusion in a meta-analysis of studies involving diagnosis of invasive aspergillosis using a galactomannan assay. EORTC, European Organization of the Research and Treatment of Cancer; MSG, Mycoses Study Group.
Characteristics of studies included in the meta-analysis of diagnosis of invasive aspergillosis using a galactomannan assay.
Characteristics and results of studies included in the meta-analysis of diagnosis of invasive aspergillosis using a galactomannan assay.
Fifteen studies included only adults as the population under study, 2 provided data on children separately, 7 included both children and adults, and 5 did not provide data. Six studies were limited to bone marrow transplant recipients, 3 were limited to solid-organ transplant recipients, and 17 were limited to patients with hematological malignancy. One study did not specify the population under study but mentioned that the patients were immunocompromised.
Five studies reported the threshold for positivity as 0.5, whereas 13 studies used a threshold of 1.0, and 11 studies used a threshold of 1.5. Four studies reported whether the assessment was done in a blinded manner, whereas that information was not reported in 3 studies, and blinding was not done in the remainder. Antifungal therapy was mentioned in 4 studies, but only 1 study analyzed the effect of prior or current antifungal therapy on the performance of the test.
Nineteen studies used the EORTC/MSG criteria, whereas 10 studies used criteria that were similar to but not identical with EORTC/MSG criteria. Most studies showed some degree of incorporation bias, because Aspergillus galactomannan forms part of the reference standard for diagnosing IA in the EORTC/MSG criteria. However, this was present only for the probable cases, because a positive galactomannan test result is not part of the EORTC/MSG criteria for proven IA.
The test was used for surveillance of IA in all of the included studies. The frequency of sampling ranged from once per week to twice per week. Twenty-six studies reported data separately for proven cases and probable cases of IA, whereas 3 studies provided data only on combined proven and probable cases of IA.
Table 4 shows the pooled sensitivity and specificity of the galactomannan assay overall and in the prespecified subgroups. Overall, the sensitivity of the test was 0.71 (95% CI, 0.68–0.74), and the specificity was 0.89 (95% CI, 0.88–0.90) for proven cases. For proven and probable cases, the sensitivity was 0.61 (95% CI, 0.59–0.63), and the specificity was 0.93 (95% CI, 0.92–0.94).
Pooled sensitivity and specificity of the galactomannan assay for diagnosis of invasive aspergillosis (IA).
We found considerable heterogeneity among the studies (P < .0001). Summarizing across studies, for proven cases, the mean sensitivity was 0.73 (95% CI, 0.61–0.85), and the median sensitivity was 0.77 (interquartile range [IQR], 0.61–1.0). The mean specificity was 0.81 (95% CI, 0.76–0.87), and the median specificity was 0.81 (IQR, 0.72–0.93) for proven cases. For proven or probable cases, the mean sensitivity was 0.69 (95% CI, 0.59–0.79), and the median sensitivity was 0.69 (IQR, 0.52–0.87). The mean specificity was 0.89 (95% CI, 0.84–0.94), and the median specificity was 0.93 (IQR 0.81–0.97).
To summarize the overall diagnostic ability of the galactomannan test, we computed the Youden index, D, and Q*. The results are shown in table 5. The mean Youden index for proven cases was 0.54 (95% CI, 0.41–0.65), and the median Youden index was 0.55 (IQR, 0.31–0.74). For proven or probable cases, the mean Youden index was 0.57 (95% CI, 0.47–0.67), and the median Youden index was 0.60 (IQR, 0.48–.073). For proven cases, the mean D was 2.74 (95% CI, 2.12–3.36), and the median D was 2.59 (IQR, 1.55–3.79). For proven or probable cases, the mean D was 2.49 (95% CI, 1.81–3.17) and the median D was 2.76 (IQR 0.02–3.82). The values of the Youden index, D, and Q* are consistent with moderate accuracy of the test.
Summary measures of accuracy and and subgroup analyses for the galactomannan assay for diagnosis of invasive aspergillosis, compared with reference standard.
The Q* estimate was 0.80 (95% CI, 0.74–0.86) for proven cases and 0.85 (95% CI, 0.79–0.91) for proven or probable cases. A summary ROC plot, summarizing the performance of the galactomannan test for patients with proven cases and patients with proven or probable cases, is shown in figure 2.
Summary receiver-operating characteristic curve for the galactomannan assay (Platelia; Bio-Rad), compared with reference standard, for proven cases of invasive aspergillosis (A) and both proven or probable cases of invasive aspergillosis (B). Each dot represents a study.
The positive and negative predictive values across a range of prevalences are shown in table 6. As expected, the positive predictive value increased as prevalence increased for both proven cases and proven or probable cases. Negative predictive values were very similar for proven cases and proven or probable cases, whereas positive predictive value appeared to be somewhat higher for proven or probable cases than it was for proven cases alone.
Underlying disease. The sensitivity and specificity of the test for patients with hematological malignancy was 0.70 (95% CI, 0.62–0.77) and 0.92 (95% CI, 0.90–0.93), respectively. The mean Youden index, D, and Q* were 0.54, 3.13, and 0.83, respectively. For bone marrow transplant recipients, the sensitivity and specificity were 0.82 (95% CI, 0.70–0.90) and 0.86 (95% CI, 0.83–0.88), respectively. The mean Youden index, D, and Q* were 0.73, 3.02, and 0.82, respectively. For recipients of solid-organ transplants, the sensitivity and specificity of the test were 0.22 (95% CI, 0.03–0.60) and 0.84 (95% CI, 0.78–0.88), respectively. The limited numbers of studies precluded robust estimates of overall accuracy. The results of the test for heterogeneity were statistically significant. Overall, the performance of the test varied by type of transplantation, with a much poorer performance among solid-organ transplant recipients, compared with among bone marrow transplant recipients or patients with hematological malignancy.
Choice of reference standard. When stratifying by reference standard, we found that, when analyses were limited to studies using EORTC/MSG criteria, the sensitivity and specificity of the test were 0.64 (95% CI, 0.54–0.73) and 0.89 (95% CI, 0.88–0.90), respectively. Studies not using the EORTC/MSG criteria found that the sensitivity and specificity of the galactomannan assay were 0.79 (95% CI, 0.70–0.86) and 0.89 (95% CI, 0.87–0.90), respectively. The mean Youden index, D, and Q* values were 0.43, 2.30, and 0.75, respectively, in studies using the EORTC/MSG criteria. The mean Youden index, D, and Q* values were 0.70, 3.46, and 0.84, respectively, in studies not using EORTC/MSG criteria. Differences in the accuracy of the galactomannan assay when stratified by type of reference standard were statistically significant (P = .007 for Youden index and P = 0.049 for D).
Age. When analyses were restricted to studies including only adults, the sensitivity was 0.62 (95% CI, 0.52–0.72), and the specificity was 0.87 (95% CI, 0.85–0.88). The mean Youden, D, and Q* for studies of adults was 0.45, 2.30, and 0.83, respectively. For studies providing data on children, the sensitivity was 0.89 (95% CI, 0.51–1.00), and the specificity was 0.85 (95% CI, 0.85–0.89). The results of the test for heterogeneity were significant. The mean Youden, D, and Q* values were 0.54, 2.60, and 0.76, respectively. The number of studies providing data for children was far fewer (2 studies) than the number of studies providing data for adults.
Choice of cutoff values for defining positivity. Analyses stratified by 3 cutoff definitions for positivity (0.5, 1.0, and 1.5) showed that, overall, the accuracy of the galactomannan test improved with a higher threshold. However, the differences were statistically significant only for the D values (P = .016).
Proven or probable cases. The performance of the galactomannan test when probable cases were included with proven cases for the analyses is shown in tables 4 and 5. Overall, the results were not markedly different, comparing with those obtained for proven cases alone.
IA is a leading cause of death among immunocompromised patients, especially among those patients with hematological malignancy or those who undergo hematological or solid-organ transplantation [10–12]. Clinical and radiologic diagnosis of IA has limited sensitivity and specificity [13]. In patients with thrombocytopenia, a tissue diagnosis carries the risk of bleeding and is usually not advisable.
The use of a biological marker as an adjunct for screening for IA in high-risk patients is attractive, because it is noninvasive and may detect evidence of IA prior to the appearance of clinical signs and symptoms. Galactomannan is a polysaccharide cell-wall component that is released by Aspergillus species. Among the many tests that have been developed for detection of galactomannan, the double-sandwich ELISA (Platelia; Bio-Rad), which incorporates the B 1–5 galactofuranose—specific EBA2 monoclonal antibody as both the acceptor and the detector for galactomannan, has the most promise.
Although numerous studies have been performed to determine the sensitivity and specificity of the assay in various patient populations, the results are variable. The goal of our meta-analysis was to synthesize the existing literature on screening for IA in high-risk patients with use of the galactomannan assay and to calculate summary measures of accuracy.
Our main finding was that the assay was moderately useful for surveillance of IA in patients with hematological malignancy or hematological transplant recipients. However, the performance of the test dropped sharply for solid-organ transplant recipients, for whom it had poor sensitivity and specificity. Subgroup analyses showed that the sensitivity of the test was higher when a reference standard other than the EORTC/MSG criteria was used. However, the differences were small, probably reflecting the exclusion from our analysis of studies that used reference standards very different from the EORTC/MSG-proposed criteria.
We examined the effect of prevalence of IA on the performance of the assay. The pretest probability greatly influences the predictive values of any test, and not surprisingly, our analysis found that positive predictive value increased with increasing prevalence. The galactomannan assay should be used only when there is a high pretest probability of IA, such as in high-risk populations with neutropenia and malignancy or populations that have undergone transplantation.
Our analysis has limitations stemming from the heterogeneity in the design of the studies we analyzed. A recent review has elegantly summarized the challenges of evaluating the accuracy of the galactomannan assay [14]. We attempted to minimize heterogeneity arising from the choice of reference standard by requiring that all included studies use the EORTC/MSG criteria or criteria that were similar to the EORTC/MSG criteria for diagnosis of IA. The choice of cutoff value for defining a positive test result deserves mention. At the initial launch of the ELISA in Europe a decade ago, a cutoff serum ratio of 1.5 was recommended in the manufacturers' manual. Over the past several years, many studies have reported using a cutoff value of 1.0, and the US Food and Drug Administration—approved ELISA in the United States recommends a cutoff value of 0.5. We found that the accuracy of the test improved with a higher threshold. Incorporation bias is also a concern when the test under study is part of the reference standard [15]. The EORTC/MSG criteria include a positive galactomannan test result for defining probable but not proven cases; thus, incorporation bias would not be of concern for analysis of proven cases.
We also performed subgroup analyses to explore the reasons for the heterogeneity found in the main analyses. A major cause of variable test performance may be prior antifungal therapy, which may be expected to result in lower sensitivity and specificity of the galactomannan assay by decreasing fungal bioburden. Marr et al. [16] found that, for proven IA in hematopoietic cell transplant recipients, the sensitivity of the Platelia assay varied from 87.5 in those patients not receiving antifungal therapy to 20% in those patients receiving antifungal therapy. However, too few other studies included in our analysis reported whether patients were receiving or had received antifungal therapy at the time that blood samples were obtained for galactomannan testing, and we were unable to perform a subgroup analysis to explore this finding further. Other sources of false-positive galactomannan test results have recently been reported, particularly concurrent piperacillin-tazobactam treatment [17, 18].
It is important to acknowledge that, in all of the studies included in our analysis, the galactomannan test was used for surveillance for IA with repeated serum sampling in a high-risk patient population during a defined period of high risk. The usefulness of the galactomannan test for the diagnosis of IA in patients with signs and symptoms suggestive of IA needs study and cannot be inferred from results involving the study populations in our analysis. We also did not examine the performance of the galactomannan assay for samples other than serum samples. Additional studies are needed to examine test characteristics in other specimen types.
Finally, the most important outcome in studies of diagnostic tests is whether diagnosis impacts patient outcomes with regard to mortality and morbidity. That information was not provided in any of the included studies, and additional research is needed to determine the impact of early diagnosis (and presumably, earlier treatment) of IA on mortality. Additional research should focus on the impact of antimicrobial therapy on test performance, more-rigorous assessment of false-positive test results, and assessment of the usefulness of the test for improving patient outcomes in nonsurveillance conditions.
Potential conflicts of interest. All authors: no conflicts.
IDSA Members: For your free access to this journal, log in via the IDSA members area.
Open access options for authors visit Oxford Open
This journal enables compliance with the NIH Public Access Policy