Methods

Study site/settings. The multicenter study was conducted at 12 acute-care hospitals located in major urban centers in 6 different states. These hospitals were moderate in size (median, 488 beds; range, 225–953 beds), private or public tertiary- and general-care facilities that served all patients or specialty-care facilities that served only pediatric or adult patients (Appendix).

Study population. All patient admissions to the emergency department or hospital inpatient unit or their admission medical records were screened to identify patients with a rash or rashlike condition (e.g., rash, bump, eczema, varicella, herpes, skin infection, allergic reaction, and drug reaction). Each hospital screened admissions for a 12-month period within an overall study period from 5 December 2003 through 31 March 2005. Screening methods varied by study site (Appendix).

Screened patients with a rashlike condition were enrolled in the study if they met specific eligibility criteria for an AGVPR. An acute rash was defined as one that occurred ⩾7 days before admission. A generalized rash was defined as a rash with at least 20 lesions, at least 4 of which were on the head-neck area, with additional lesions distributed over 4 other body areas (abdomen-chest, back, left arm, right arm, left leg, or right leg). Fever was not required for enrollment.

The examining physician used a standard form to assess clinical presentation and smallpox risk. The physician recorded the presence of major and minor clinical smallpox symptoms and signs, the differential diagnosis, risk classification for smallpox, physician's specialty training, and patient admission demographic information, such as the date of birth, age in years, sex, race and ethnicity, county of residence, ZIP code, insurance (surrogate of income level), and admission and discharge diagnoses. Race and ethnicity were self-identified at most sites.

The subset of study participants who gave written consent to provide more-detailed information on their clinical illness, current medications, laboratory findings, and hospital course were interviewed; study staff abstracted their medical records. The study was approved by the institutional review boards at each study site and by the CDC institutional review board.

Smallpox risk classification algorithm. All eligible study participants were assessed to determine whether smallpox risk was accurately classified as low, moderate, or high. Training of study staff in use of algorithm is described in the Appendix. The risk classification scheme (figure 1) was published elsewhere [3]. A participant with major clinical features that were almost always present in ordinary-type smallpox during the pre-eradication era (major clinical criteria) was classified as being at high risk [1]. Major clinical criteria for smallpox in a patient with an AGVPR included (1) a febrile prodrome (a temperature ⩾38.3°C occurring 1–4 days before rash accompanied by prostration, headache, backache, chills, vomiting, or severe abdominal pain); (2) characteristic lesions, described as deep-seated, firm, hard, well-circumscribed vesicles or pustules; and (3) vesicles or pustules all at the same stage of development on any 1 body part (e.g., face, leg, or arm).

A patient classified as being at moderate risk for having smallpox had (1) a febrile prodrome, an AGVPR, and at least 1 other major criterion; or (2) a febrile prodrome, an AGVPR, and ⩾4 minor criteria. A patient classified as being at low risk had (1) an AGVPR with a febrile prodrome and <4 minor clinical criteria or (2) only an AGVPR [3]. Minor clinical criteria that distinguished ordinary-type smallpox from varicella during the pre-eradication era included (1) centrifugal rash distribution; (2) first lesions on the oral mucosa, face, or forearms; (3) a toxic or moribund appearance; (4) a slow rash evolution from macules to papules to pustules (1–2 days for each stage); and (5) lesions on palms and soles.

Risk classification was determined by the response of an examining physician to the question “Using the CDC's rash algorithm, how would you characterize this patient's risk of smallpox?” or by study staff review of the patient's medical record. The CDC independently assigned risk for smallpox to assess whether the physician or study staff risk assignment used only the major and minor criteria of the algorithm.

A selection scheme was used to choose the single, best classification of risk for patients who were examined and classified by multiple physicians. According to the algorithm, a patient classified as being at moderate or high risk for having smallpox should have a consultation with a dermatologist or infectious disease physician before final risk classification. Thus, for our analysis, the single, best classification of risk was provided by dermatologists or infectious disease physicians. In the absence of these consultations, selection of risk classification by an emergency medicine physician was preferred. If necessary, classification by physicians in other specialties was accepted. Last, classification by study staff was used.

Diagnosis of rash. Discharge diagnoses, obtained by study staff via medical record abstraction using codes from the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9/ICD-9CM) [5], were used to describe AGVPR etiology. Codes for the following specific conditions were included in the search: smallpox, chickenpox, herpes simplex and herpes zoster, other infectious diseases (viral, bacterial, rickettsial, fungal, and parasitic), dermatitis (allergic, contact, and medication induced), and other dermatological conditions.

Statistical analysis. The proportion of patients with an AGVPR was computed by dividing the number of patients with an AGVPR by the total number of patient admissions to the emergency department or inpatient unit during the study period. The 95% CI for this proportion was computed using SAS, version 8 (SAS Institute).

The number of nonparticipants was very small. Nonparametric statistical methods in StatXact, version 6.0 (Cytel), were used to compare participant and nonparticipant means and proportions and to compute the 95% CIs for proportions by use of an exact binomial distribution (e.g., proportion of patients classified as at low, moderate, and high risk for having smallpox).

Results

During a 12-month period, 26,747 patients (3.5% of all admissions) with a rash or rashlike condition were screened to identify patients with an AGVPR (table 1). Comparison of screened rashes or rashlike conditions with those found through discharge hospital systems using ICD-9/ICD-9CM codes and standard capture-recapture methods [5, 6] estimated that 40%–90% of all admitted patients with rashes or rashlike conditions had been identified through screening (Appendix).

Of the 26,747 screened patients, 51% were admitted to the emergency department or inpatient unit of hospitals caring for patients of all ages; the remainder were admitted to emergency departments and inpatient units of hospitals caring only for either pediatric patients (⩾18 years of age) (40%) or adult patients (8%) (table 1). Only 0.3% of screened patients met the AGVPR study eligibility criteria, representing 1.2 patients per 10,000 admissions.

Among eligible patients, 79 (89%) of 89 participated in the study. Six eligible patients from 1 study site refused to participate. Three eligible patients were excluded because they had incomplete information for classifying smallpox risk; 1 had a language barrier. Basic demographic information was available for 9 of 10 nonparticipants (table 2).

Study population. Study participant ages ranged from 0 to 85 years (mean, 21 years); 40 (51%) were children ⩾18 years of age (table 2). Sixty-two percent of the participants were male. Most participants were non-Hispanic black (35/72; 49%) or non-Hispanic white (27/72; 38%). Twenty-seven percent of the study participants had no health insurance; 30 of 43 insured participants had private insurance (table 2).

Compared with study participants, nonparticipants tended to be younger (mean age, 10 years; range, 0–39 years) and were significantly more likely to be Hispanic and recipients of public insurance (table 2).

Smallpox risk classification algorithm. No participants received diagnoses of smallpox or were classified as being at high risk for having smallpox (table 3). The CDC used the recorded presence of major and minor clinical criteria for smallpox to assign a low risk to 68 (86%) of 79 participants and a moderate risk to 11 (14%) (table 3). Physicians or study staff (henceforth referred to as “physicians”) classified 72 participants as being at low risk for having smallpox and classified 1 participant with lesions thought to be typical of classical smallpox as being at moderate risk; 6 participants were unclassified. There was 84% agreement in the risk classification of physicians and that assigned by the CDC for the 73 participants classified by both. All 11 participants assigned moderate risk by the CDC met 2 major clinical criteria: 9 had a febrile prodrome with classical lesions, and 2 had a febrile prodrome with lesions in the same stage of development.

Thirty-two (44%) of the 73 participants classified by a physician had a fever (temperature, ⩾38.3°C) that preceded the rash by 1–4 days (table 4). Nineteen (26%) had a recorded febrile prodrome associated with at least 1 of the following: prostration, headache, backache, chills, vomiting, or severe abdominal pain. Twenty-three (32%) had recorded lesions described as classic smallpox lesions; 12 (16%) had lesions in the same stage of development on any 1 body part. No patients had recorded ⩾4 minor smallpox clinical criteria (table 4).

Varicella-zoster virus infection was the most common discharge diagnosis among patients with physician-classified AGVPR (55/73; 75%). The frequencies of other diagnoses are shown in table 5. Fifty-four patients classified by physicians as being at low risk and 1 classified as being at moderate risk of having smallpox had a discharge diagnosis of varicella or disseminated herpes zoster. Of the 11 patients assigned moderate risk for smallpox by the CDC, 9 (82%) had a discharge diagnosis of varicella, and 1 each had a diagnosis of erythema multiforme and contact dermatitis. Overall, 50% of patients with diagnoses of varicella were adults, 73% were male, and 49% were of non-Hispanic black race and ethnicity.

Detailed clinical information was available for 41 of 73 physician-classified participants. Twelve participants classified as being at low risk for having smallpox and with diagnoses of varicella had direct contact with a person with varicella 3 weeks before their illness or an assumed contact because varicella transmission occurred in their community.

Health care providers. Sixteen of the 73 participants were classified by study staff. Of the remaining 57, 26 (46%) were classified by emergency medicine physicians, 11 (19%) by dermatologists, 6 (10%) by infectious disease specialists, 10 (18%) by another pediatric specialist, and 3 (5%) by an internal medicine specialist. Risk was classified by a single examiner for 48 (66%) of 73 participants, and 25 (34%) had multiple examiners. Among these 25 participants with multiple examiners, the multiple physician examiners for 24 (96%; 95% CI, 80%–100%) classified smallpox risk similarly. Dermatologists and infectious disease specialists classified risk similarly to other physicians.

Discussion

In this study of patients admitted to the emergency department or inpatient unit of urban acute-care hospitals for rashes that might be confused with smallpox but discharged with an alternate diagnosis, use of the algorithm Evaluating Patients for Smallpox: Acute, Generalized Vesicular or Pustular Rash Protocol did not result in misclassification of AGVPRs of other etiologies as high risk for smallpox. In the absence of smallpox, the algorithm functions as a solid assessment tool for smallpox risk, does not generate false-positive results, and limits orthopoxvirus testing to patients classified as being at high risk. Use of the algorithm by state and local public health agencies and health care providers in consultation with the CDC to evaluate suspected smallpox disease yielded similar results [3].

The presence of persistent fever with rash (present in 98%–100% of hospitalized patients with smallpox) [1] and of lesions reminiscent of classic smallpox are important influences for physicians classifying patients as being at moderate risk for having smallpox (figure 1). Most patients with an AGVPR in this study were afebrile within 1–4 days after rash onset and at the time of examination. Physicians who described lesions as deep seated, firm, and well circumscribed also considered the patient's clinical history and exposure to varicella, other infectious diseases, and medications in deciding whether the patient was at risk for having smallpox. This clinical judgment may explain the 16% discrepancy between physician risk classification and the risk assigned by the CDC, which adhered strictly to major and minor criteria.

A classic smallpox lesion typically is intraepidermal and is considered by dermatologists to be a tense, firm lesion that is not flaccid. The “deep-seated” designation is not generally used by dermatologists [7]. Participating physicians indicated that confusion may exist about the description of vesicular or pustular rashes as being deep seated. Physicians may report deep-seated skin lesions but may not believe they are deep enough to qualify as classic smallpox lesions. Education of health care and public health providers in the use of current dermatological terms may improve interpretation of vesicular or pustular lesions.

Admission to an emergency department or inpatient unit for any rash or rashlike condition is rare (3.5%). Severe rashes that might be confused with smallpox (AGVPRs) and require admission are exceedingly rare (1.2 per 10,000 admissions). Regardless, the algorithm proved specific enough to avoid false suspicion of high risk for smallpox. This algorithm was up to 100% specific in avoiding misclassification of patients without smallpox as being at high risk for having the classical smallpox disease found in the pre-eradication era. Theoretical concern remains that, in the absence of laboratory confirmation, this algorithm could result in misclassification of true but atypical smallpox cases as moderate or low risk and, therefore, could have a lower specificity than estimated by the present study. However, modification of the algorithm to attempt to encompass atypical smallpox presentations would likely introduce false alarms due to misclassification of AGVPRs as atypical smallpox. This algorithm's specificity for historically recognized, ordinary-type smallpox disease ensures that 1 case classified as high risk should raise serious suspicion of smallpox [1].

Severe varicella-zoster virus infection should always be considered in patients classified as being at low and moderate risk for having smallpox when varicella-zoster virus is circulating in the community. In the present study, patients classified as being at low or moderate risk for having smallpox were highly likely to receive a varicella diagnosis. Severe varicella was also the disease most commonly confused with smallpox in both developed and developing countries during the pre-eradication and immediate posteradication eras [1] and, in recent years, was suspected to be possible smallpox disease by state public health agencies and health care providers who consulted the CDC [3].

The 6 patients evaluated for suspected smallpox disease solely by the CDC were disproportionately racial and ethnic minorities. The substantial proportion of AGVPRs identified among African American males in this study may reflect the populations served by study hospitals.

This study has several limitations. We did not sample patients suspected by their physicians to have smallpox. Patients were instead actively ascertained on the basis of defined rash criteria. Conclusions regarding algorithm performance are, therefore, derived from a patient population without circulating variola virus, using selection criteria that likely would have raised suspicion of smallpox during the pre-eradication era.

Second, differential case ascertainment at the various hospitals may have led to incomplete case finding in study sites and a disproportionate number of patients with AGVPRs identified from 1 city. Standard capture-recapture methods identified completeness of rashlike conditions ascertainment ranging from 40% to 90% (Appendix) [6].

Additionally, recent varicella epidemiology in the United States supports the average proportion of admissions with an AGVPR identified in this study. In 2000, the estimated varicella incidence in 3 active surveillance sites ranged from 6 to 25 cases per 10,000 population [8]. During 2002, the annual utilization of ambulatory services for varicella was 8.8 varicella-related visits per 10,000 visits [9], and, in 2001, the rate of varicella-related hospitalizations was 0.13 per 10,000 [10]. In the present study, varicella was responsible for 0.9 per 10,000 admissions to the emergency department and inpatient units, less common than the varicella-related ambulatory visits but more common than varicella-related hospitalizations.

During the pre-event phase of preparation for response to smallpox release, benefits and costs must be weighed in deciding which strategies are most efficient for detecting the first case(s). In the pre-event phase, we are most interested in classic smallpox illness at approximately the third day of presentation. Attempts at earlier detection of nonspecific disease will unnecessarily use public health and health care resources. Thus, we excluded patients that may have had a generalized rash of <20 lesions with specific body parts having <4 lesions. We may have thereby missed other rashes that might have been confused with smallpox and thus altered the specificity estimation.

Since June 2003, the Council of State and Territorial Epidemiologists (CSTE) has recommended that state, city, and territorial public health agencies collect and investigate information on rashes suspected of being caused by smallpox [13]. Recent studies suggest that primary care providers and emergency physicians need more education about smallpox diagnosis and management [11, 12]. These providers may also benefit from knowledge about the CDC algorithm. A patient at high risk for having smallpox, according to the algorithm, meets the smallpox clinical case definition for both the CSTE and the CDC and is considered to be a probable smallpox case [13]. The algorithm can be an important and highly specific tool for classifying smallpox risk and for detecting a probable case of smallpox. Poster and interactive, Web-based formats are available at the CDC Web site [14].