Validation of a Screening Instrument to Identify Persons for Exclusion from Smallpox Vaccination

Since the inception of the smallpox vaccination programs in the United States in January 2003, more than 39,000 civilian health care workers and public health workers and >623,000 military personnel have been vaccinated to provide adequate response capability in the event of an intentional smallpox attack [1, 2]. The pre-event use of smallpox vaccine is contraindicated if the potential vaccinee meets any of the following criteria: (1) history or presence of atopic dermatitis (also known as eczema); (2) history of Darier disease; (3) other acute, chronic, or exfoliative skin conditions with active disease; (4) conditions associated with immunosuppression; (5) pregnancy or breastfeeding; (6) age <18 years; or (7) vaccine-component allergy [3]. Vaccinia virus may spread from person to person, and smallpox vaccination is also contraindicated when there is a household contact who has atopic dermatitis, Darier disease, or active skin disruption, who is immunosuppressed, or who is pregnant. Accurate identification of contraindications is critical, but recall of skin disease that occurred during early childhood and knowledge of medical conditions in close contacts may be inaccurate [4].

In 2002, an expert workgroup from the Centers for Disease Control and Prevention (Atlanta, GA) developed a screening instrument to help identify persons who should not receive smallpox vaccine [5]. A series of questions in this screening survey were developed to distinguish persons with atopic dermatitis from those with other types of skin disease. We performed a validation study to evaluate the overall sensitivity of the smallpox screening questionnaire relative to medical record review for a sample population of hospital workers.

Subjects and methods. In January 2003, we asked department managers to distribute a self-administered survey to ∼1000 nonphysician employees of St. Joseph's Hospital, a 500-bed tertiary care facility in Marshfield, Wisconsin. Managers distributed the generic survey packets to staff in radiology, respiratory therapy, pharmacy, laboratory, housekeeping, and all acute care nursing units. We did not have access to hospital personnel files and were therefore unable to address survey packets to specific individuals or to monitor study participation in detail.

Employees were eligible to participate if the Marshfield Clinic (Marshfield, WI) was their primary provider of health care during the preceding 2 years and if they had had at least 1 outpatient visit since 1999. The survey included detailed questions about atopic conditions (such as asthma, hay fever, and food allergies) in addition to questions about atopic dermatitis and past history of recurrent, itchy rash. An algorithm based on these questions was developed to identify individuals with a dermatologic contraindication (figure 1). Questions addressed the presence of relevant conditions in both respondents and their close contacts (i.e., anyone who lived in the same household or who had intimate contact with the respondent).

View larger version:

• In this window
• In a new window

• Download as PowerPoint Slide

Figure 1

Screening algorithm used to identify persons with a dermatologic contraindication to smallpox vaccination in a study involving 174 employees of a 500-bed tertiary care facility in Marshfield, Wisconsin.

We reviewed medical records for all consenting participants and their dependent children to validate survey responses. Medical record data were abstracted from 1993 onward; relevant data from earlier years were recorded if noted in the participant's past medical history. The abstracted information included receipt of an organ or bone marrow transplant, recent chemotherapy for cancer, recent radiation therapy, allergy to vaccine components, and diagnoses of atopic dermatitis, eczema, Darier disease, malignant neoplasm, or immunodeficiency syndrome. Computerized diagnosis codes were not used to identify vaccine contraindications. Identifying links were destroyed after medical record review was completed.

The primary outcome was the sensitivity of the screening instrument, defined as the proportion of persons with a contraindication noted in the medical record who reported a contraindication on the screening survey. A 95% CI for this proportion was calculated. Similar sensitivity calculations were performed, based on the specific questions regarding dermatologic contraindications.

The Marshfield Clinic Institutional Review Board reviewed and approved the study. All participants provided written informed consent.

Results. We distributed 1000 surveys to hospital employees, and 215 (22%) were completed and returned. Forty-one individuals did not meet the eligibility criteria and were excluded from all analyses. The mean age of the 174 eligible respondents was 44.5 years (range, 22–71 years), and 93% were women. Sixty-eight percent were employed as nurses, 23% were clinical support staff, and 9% were clerical or administrative staff. There were 158 dependent children associated with 81 respondents.

Eighty-nine (51%) of 174 respondents reported at least 1 contraindication for smallpox vaccination, including 65 (37%) who reported a diagnosis of either atopic dermatitis or eczema for themselves or their close contacts. Five additional participants did not report these specific diagnoses, but they met exclusion criteria, based on their answers to other rash algorithm questions. With reference to the medical record as the gold standard, the sensitivity of the survey was 60% (30 of 50) for respondents alone and 68% (52 of 77) for respondents and their dependent children (table 1). Contraindications identified in the medical record (since 1993) but not reported by respondents included 19 atopic dermatitis or eczema diagnoses, 6 autoimmune disease diagnoses, 1 immunodeficiency syndrome diagnosis, and 1 allergy to tetracycline.

View larger version:

• In this window
• In a new window

• Download as PowerPoint Slide

Table 1

Sensitivity of responses to a survey, which was administered to employees at a 500-bed tertiary care facility in Marshfield, Wisconsin, for detecting smallpox vaccine contraindications in respondents and their dependent children.

The survey identified 22 of 33 persons with a dermatalogic contraindication to smallpox vaccination documented in their own medical record (contact contraindications were excluded). All 22 gave a positive response to the question about a personal history of atopic dermatitis or eczema. No additional contraindications were detected on the basis of questions pertaining to rash location, food allergies, asthma, or hay fever.

A secondary sensitivity analysis was performed for the presence of a dermatologic contraindication in respondents, with a composite gold standard as the reference. In this analysis, a true contraindication was defined as either documentation of eczema or atopic dermatitis in the respondent's medical record or self-report of these specific diagnoses in the survey. In this analysis, the sensitivity of the survey was 77%: 37 contraindications were detected by the survey among 48 detected by either the survey or the medical record review. Fifteen respondents reported a diagnosis of eczema or atopic dermatitis that was not found in the respondents' medical record review.

Discussion. The optimal strategy to screen potential recipients of smallpox vaccine has not been determined. The ideal screening strategy would be rapid, convenient for participants and vaccine clinic staff, and highly sensitive for detection of contraindications. In this study, the sensitivity was 60%–77%, which was somewhat lower than expected. Hospital staff in general might be expected to have a greater familiarity with medical terminology, leading to recall of diagnoses that is better than that for the general public. However, recall of eczema or atopic dermatitis was comparable to that of the general population in central Wisconsin [4]. We were required to destroy personal identifiers as soon as chart abstraction was complete, which precluded any follow-up with hospital staff to learn more about the reasons for lack of recall. Questions pertaining to hay fever, asthma, and food allergies did not contribute to the screening process, although it is possible that they may be useful in populations with a higher prevalence of these conditions.

The risk of eczema vaccinatum was low in the 1960s, and it appeared to be even lower during the bioterrorism preparedness vaccination campaign in 2002 and 2003. The incidence of eczema vaccinatum was ∼1 per 25,000 first-time vaccinees during the era of routine childhood smallpox vaccination in the United States [6]. In contrast, no cases of eczema vaccinatum have occurred among >660,000 civilian or military vaccinees and their close contacts since 2002. On the basis of historical rates, ∼25 cases of eczema vaccinatum would have been expected to occur. Several factors have likely contributed to the reduced occurrence of eczema vaccinatum. First, all recent vaccinations have occurred in adults, and many of the civilian vaccinees had received smallpox vaccination during childhood. It is well documented that the risk of eczema vaccinatum is markedly lower among revaccinees than it is among persons receiving smallpox vaccine for the first time [7]. Second, the rate of secondary vaccinia transmission to susceptible household contacts may be lower now than it was in the earlier era of smallpox vaccination. Finally, heightened awareness of contraindications and adverse events may have contributed to a higher rate of self-deferral in the current era, especially in the volunteer civilian population.

Limitations of this study include the low participation rate and the inability to compare responders with nonresponders. Anecdotal feedback from hospital staff suggested that the low participation rate was partially due to the erroneous belief that participants might receive smallpox vaccination or be placed on a list of potential vaccinees; others expressed concern about privacy and access to medical records. Privacy concerns precluded access to data that would have allowed a direct comparison of responders and nonresponders in terms of age, sex, job category, and years of employment. However, the age and sex distribution of study participants was similar to that of the entire hospital employee population, which is 85% female with a mean age of 41.5 years. The prevalence of atopic dermatitis among respondents was 19%, which is higher than the expected prevalence among adults in this region [4]. This suggests that individuals with dermatalogic disease were overrepresented among participants.

In summary, the results of this study suggest that screening questions recommended for detection of smallpox vaccine contraindications in the civilian population have a sensitivity of 60%–77%. The performance of screening procedures may vary in different populations, and further assessment of smallpox vaccine screening procedures will be needed if the vaccine becomes more widely available in the future.

• Received June 25, 2004.
• Accepted October 14, 2004.

• © 2005 by the Infectious Diseases Society of America