Skip Navigation

Acceptable Rates of Treatment Failure in Osteomyelitis Involving the Diabetic Foot: A Survey of Infectious Diseases Consultant

  1. Eli N. Perencevich1,2,
  2. Keith S. Kaye3,
  3. Larry J. Strausbaugh4,
  4. David N. Fisman5,
  5. Anthony D. Harris1,2, and
  6. Infectious Diseases Society of America Emerging Infections Network
  1. 1VA Maryland Healthcare System, University of Maryland School of Medicine, Baltimore
  2. 2Department of Epidemiology and Preventive Medicine, University of Maryland School of Medicine, Baltimore
  3. 3Department of Medicine, Duke University Medical Center, Durham, North Carolina
  4. 4Infectious Diseases Section, Medical Service, Veterans Affairs Medical Center, Portland, Oregon
  5. 5Hamilton Social and Public Health Services Department and Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ontario, Canada
  1. Reprints or correspondence: Dr. Eli N. Perencevich, Dept. of Epidemiology and Preventive Medicine, University of Maryland School of Medicine, BT-111, VA5D-150, 50 N. Greene St., Baltimore MD 21201 (eperence{at}epi.umaryland.edu).

Abstract

Shortening the duration of antibiotic therapy is an attractive strategy for delaying the emergence of antimicrobial resistance. The paucity of data about optimal treatment durations hinders adoption of this approach. This study used contingent valuation analysis to identify failure rates for treatment of diabetic foot osteomyelitis acceptable to infectious diseases consultants (IDCs). The Infectious Diseases Society of America's Emerging Infections Network (EIN) provided members with the case scenario and 1 of 10 failure rates; members were asked, assuming delivery of standard therapy, if they would accept or reject the given failure rate. The relationship between specific failure rates and the willingness of IDCs to accept them was analyzed. The median acceptable failure rate for EIN members was 18.1%; 75% of IDCs found a failure rate of 7.8% to be acceptable, and 25% found a rate of 28.4% to be acceptable. The methodology used in this study may prove useful in delineating acceptable treatment failure thresholds, an initial step in shortening durations of antimicrobial therapy.

Infections caused by bacteria resistant to commonly used antibiotics have emerged as an important medical problem. Several studies have associated these infections with increased morbidity, mortality, and costs [16]. The growing awareness of this problem has led to the development of several types of antibiotic management programs that are designed to optimize prescribing in an effort to curb antimicrobial resistance [711]. These strategies include restricted use of antibiotic formularies in hospitals and targeted education to reduce the prescription of antibiotics for nonbacterial infectious conditions (e.g., upper respiratory tract infections).

Levy [12] has proposed another strategy for retarding the emergence of resistance: decreasing the duration of antimicrobial therapy, which also limits exposure of bacteria to these agents. Unfortunately, very few clinical studies have attempted to determine optimal treatment durations. It seems likely, given the lack of such data, that many physicians prescribe antibiotics for inordinate lengths of time. A recent review emphasized this point, stating that duration of therapy for diabetic foot infections had not been adequately studied [13]. Fear of treatment failure likely motivates some physicians to overtreat their patients. Physicians are, as a group, risk-averse, and they probably prescribe antibiotics for unnecessarily long periods of time [14,15]. The risk-averse nature of antibiotic prescription is suggested by the tendency of physicians to preferentially prescribe newer, rather than older, antibiotic agents, even if resistance to the older antibiotic occurs in <1% of bacterial isolates [16]. However, little work has been published to date regarding physicians risk-preference for avoiding antimicrobial therapeutic failure.

The aim of this study was to evaluate, among a cohort of infectious diseases consultants (IDCs), the threshold of acceptable antimicrobial therapeutic failure rates for osteomyelitis of the foot in a patient with diabetes. Physician preferences were elicited through the use of a realistic clinical vignette, and a methodology was derived using contingent valuation. The results demonstrate the usefulness of this methodology in delineating acceptable treatment failure thresholds, a first step in optimizing durations of antimicrobial therapy.

Materials and Methods

Pilot study. A clinical case description was developed using sources in the literature. It included history, findings of a physical examination, test results, and treatment plan for a 66-year-old patient with diabetes mellitus, hypertension, and peripheral vascular disease who developed osteomyelitis that required surgical debridement and a course of antibiotics. A case of osteomyelitis in a patient with diabetes mellitus was chosen because this scenario is a common medical problem that is likely to be seen by IDCs.

The clinical vignette was distributed electronically to 5 physicians who were established experts in infectious disease and asked for their opinion of the case and an estimate of a treatment failure rate that would be tolerable or acceptable, assuming treatment that met the standard of care. Treatment failure was defined as “recurrent soft tissue infection or lack of wound healing requiring additional antibiotic therapy, and/or hospital evaluation/admission, and/or further surgery of the affected limb.”

After receipt of comments and estimated failure rates from the expert physicians, the clinical description was modified on the basis of their suggestions and sent to a larger group of 55 IDCs, which included those affiliated with large academic medical centers and community hospitals. They were asked to specify the treatment failure rate that they would find to be acceptable given the treatment plan as outlined, assuming that standard-of-care therapy was provided. Treatment failure was defined as above. Approximately one-half of the IDCs were given an open-ended question and asked to determine the acceptable failure rate without any prompting. The remaining IDCs were provided with 5 ranges of failure rates in a multiple-choice format and were asked to select an appropriate failure rate. The pilot results obtained from the 55 IDCs were used to determine the contingent valuation choices that were used in the full study.

Study subjects. The Infectious Diseases Society of America (IDSA) Emerging Infections Network (EIN) is a provider-based sentinel network that was established using a Cooperative Agreement Program Award from the US Centers for Disease Control and Prevention (CDC) in 1995. It comprises IDCs who belong to either the IDSA or the Pediatric Infectious Diseases Society, who regularly engage in clinical activity, and who volunteer to participate in network activities. Although its members reside in 25 countries, 95% of EIN members practice in the United States.

Questionnaire. In December 2002, the EIN distributed questionnaires titled “Failure Rates for Treatment of Osteomyelitis of the Foot in Diabetic Patients” via facsimile and e-mail to 679 members who were practicing adult IDCs in the United States. This group included members from 49 states and the District of Columbia. Pediatric IDCs were not included because they were not likely to have seen a case of adult diabetic osteomyelitis in clinical practice. EIN members who did not respond to the first survey received a second survey via facsimile, containing the same single choice of acceptable failure rate and a reminder notice, 2 weeks after the first was distributed. In addition, a third survey was distributed via facsimile in January 2003 in case some respondents were unavailable during the holiday season. This survey also gave the EIN member the same single choice of acceptable failure rate.

The survey included a 1-page introduction to the topic and a 1-page questionnaire (figure 1). Treatment failure was defined as it was in the pilot study, but instead of open-ended or multiple-choice questions, IDCs were given a specific failure rate and asked if this failure rate was acceptable (i.e., they were asked to respond “yes” or “no”), assuming standard-of-care therapy. The specific failure rates that were given to the IDCs were determined by using the results from the pilot study. The upper range of failure rates selected was 50%, because 2 people who responded to the open-ended pilot questionnaire reported a 40% failure rate as being acceptable, and 2 who responded to the multiple-choice questionnaire selected acceptable failure rates of >20%. For the lower range, 0.1% was chosen, because 3 people who responded to the open-ended questionnaire reported 1% and 2 who responded to the multiple-choice questionnaire picked <1% as an acceptable failure rate. A range of 10 choices of possible acceptable failure rates was used, with the expectation that 300 individuals would respond to the questionnaire.

Figure 1

Questionnaire regarding acceptable rates of treatment failure, distributed by the Emerging Infections Network

Thirty IDCs, on average, were expected to respond to each question. The median failure rate in the open-ended pilot study was 17.5%, so the 10 possible acceptable failure rates that were distributed to the IDCs were 0.1%, 1%, 10%, 15%, 17.5%, 20%, 25%, 30%, 40%, and 50%. The EIN randomly distributed these 10 failure rates among members practicing in the 9 regions of the United States delineated by the CDC. Each EIN member received only 1 potential acceptable failure rate (e.g., 40%) to which to respond “yes” (if the failure rate was acceptable) or “no” (if the failure rate was not acceptable). In addition to being asked whether they found the proposed failure rate to be acceptable, subjects were asked to provide the number of years that they had been in clinical practice.

Elicitation of preferences. The approach used for elicitation of preferences in this study was closely related to the contingent valuation methodology used in economic studies of willingness to pay or willingness to accept payment [17]. This methodology was initially used to generate monetary values for abstract entities that are difficult to quantify, such as wildlife preservation and water quality. It has been used extensively to determine cost preferences in health care settings for both workers and patients [1822]. Modifications of this technique have also been used to determine medical preferences or health utilities in the general population [23]. Elicitation of willingness to pay or willingness to accept payment using contingent valuation is subject to framing effects and anchoring biases (i.e., sensitivity of subject responses to opening bids, ranges of bids provided, and phrasing of questions) when open-ended questions are used. In order to minimize such biases, we used a “take it or leave it” approach, in which study participants were presented with only a single bid, which they could accept or reject. IDCs were asked: “Given the standard treatment scenario described above, do you feel that the treatment failure rate of XX% would be acceptable (i.e., is the rate listed below a reasonable estimate of the percentage of patients that you would expect to fail appropriate therapy)?” The subjects were thus asked to answer “yes” or “no” to the above question, indicating that the failure rate listed was either acceptable or not acceptable.

Statistical analysis. For each of the 10 failure rates randomly distributed to the IDCs, the proportion of respondents finding the number acceptable was calculated. A logistic regression analysis was performed to evaluate the relationship between willingness to accept a specific failure rate and the failure rate offered to the subject in the questionnaire. Median and upper and lower quartile failure rates were estimated for the entire group of respondents. Univariable and multivariable logistic regression was performed to identify respondent characteristics that might influence willingness to accept failure. Statistical analyses were performed using Stata, version 7.0 (Stata), and Excel 2000 (Microsoft).

Results

Pilot study. A total of 21 IDCs completed the open-ended questionnaire. The mean acceptable failure rate (± SD) reported for this portion of the pilot study was 17.1% ± 11.8% (median, 17.5%; range, 1%–40%). Thirty-four IDCs completed the multiple-choice portion of the pilot study. They reported a median acceptable failure rate of >5% but <10%, with 85% of responses falling in the 2%–20% range. These pilot study results were used, as described above, to determine the 10 possible acceptable failure rates that were distributed to the EIN members in the questionnaire study.

Questionnaire study results. Overall, 356 (52%) of the 679 adult IDCs who were members of the EIN responded to the survey. This response rate was similar to that of other EIN surveys [24, 25]. Three subjects were excluded from the analysis because their surveys were incomplete, leaving data on 353 subjects for use in the complete analysis. Subjects were distributed evenly across the entire United States (table 1). In addition, 4 subjects were from Canada, and 1 was from a US territory. The mean length of time (± SD) that the respondents had been in clinical practice was 16.1 ± 8.6 years (range, 1–50 years).

Table 1

Geographic distribution of 356 members of the Emerging Infections Network (EIN) who responded to the EIN questionnaire regarding acceptable rates of treatment failure.

There were ⩾29 respondents for each proposed therapeutic failure rate, with an average of 35.3 respondents for each rate and a maximum of 46 (table 2). Overall, the number of IDCs who found a particular failure rate to be acceptable decreased as the listed failure rate increased. Eighty-nine percent of IDCs found a failure rate of 0.1% for the clinical vignette to be acceptable, whereas none found a failure rate of 50% to be acceptable (table 2). In the univariable logistic regression model with “given failure rate” as the predictor for the binary outcome “yes” (the failure rate was acceptable) or “no” (the failure rate was not acceptable), the odds that an IDC would find a given failure rate to be acceptable decreased by 0.90 for each 1% increase in a given failure rate (OR, 0.90; 95% CI, 0.88–0.92; P < .001). In other words, if the failure rate in the clinical vignette was increased from 1% to 2%, the odds of IDCs accepting this increase would decrease by a factor of 0.9; or if the failure rate increased from 5% to 10%, the odds of this increase being acceptable would decrease by a factor of 0.95 (i.e., 0.6). A fitted logistic curve of the expected proportion of IDCs willing to accept a failure rate across a range of given potential failure rates is presented in figure 2. The projected median failure rate was 18.1%. Thus, presented with a failure rate of 18.1%, one-half of the study subjects would have been expected to agree that that failure rate was acceptable. Seventy-five percent of subjects would have found a failure rate of 7.8% to be acceptable, and 25% of subjects would have found a failure rate as high as 28.4% to be acceptable.

Table 2

Acceptable rates of treatment failure in osteomyelitis involving the diabetic foot, according to 353 infectious diseases consultants (IDCs) who responded to a questionnaire distributed by the Emerging Infections Network.

Figure 2

Relationship between rates of treatment failure and the proportion of infectious disease consultants (IDCs) willing to accept each failure rate. Diamonds represent the response of between 29 and 46 IDCs to the question of whether they would accept that particular rate of treatment failure. The line is a fitted curve that uses all of the data describing the expected proportion of IDCs willing to accept each failure rate.

A multivariable logistic regression model was created to adjust for potential confounding of acceptable failure rates by years in practice (a marker for IDC experience). No change in the OR for acceptable failure rate was identified in the adjusted model, and number of years in practice was not a significant independent predictor of accepting a given failure rate (OR, 1.01; 95% CI, 0.98–1.05, P = .41).

Discussion

The emergence of antibiotic resistance in bacterial pathogens is directly related to antibiotic use, with the inevitable selection of drug-resistant bacterial pathogens [12]. A large proportion of antibiotic prescriptions are considered to be inappropriate or unnecessary [2631]. For many infections, prolonged courses of antibiotics may foster the development of resistance not only to the prescribed antibiotic, but also to many other classes of antibiotics. Several authors have suggested that shortening the treatment duration for certain diseases will decrease the emergence of antimicrobial resistance [12].

A case scenario of osteomyelitis of the foot in a patient with diabetes mellitus was selected because this is a common medical problem that is likely to be seen by IDCs. In addition, this condition is associated with a high treatment failure rate and may require treatment with several courses of antibiotics [13, 32]. Therefore, although failure of therapy may have serious consequences for the patient, therapeutic failure would probably not be life threatening, and so some degree of tolerance for failure might be expected. Also, high levels of antibiotic exposure in these patients may lead to colonization or infection with multiple highly drug-resistant pathogenic bacteria, which again may make shortening the duration of antibiotic therapy desirable, despite associated risks.

Current treatment guidelines suggest short-course therapy for uncomplicated cystitis (for which 3 days of therapy are suggested), vaginal trichomoniasis, cervical Chlamydia trachomatis infections outside of pregnancy, and urogenital gonococcal infections (for each of which single dose therapy is suggested) [33, 34]. Very few studies have been published on whether a shorter duration of antibiotic treatment is effective for treating infections other than these urogenital infections. One reason that treatment duration has not been shortened for several types of infections is that appropriate levels of treatment failure associated with these diseases and attitudes toward treatment failure rates have not been studied. One would expect physicians to be more comfortable with the higher treatment failure rates associated with shortened duration of antimicrobial therapy in cases in which that therapy is being used to treat certain relatively benign infections, such as bronchitis or sinusitis, compared with more complicated and serious infections, such as prosthetic valve endocarditis. Until acceptable failure rates for commonly treated infections are determined, it will be difficult to select optimal treatment durations for antimicrobial therapies.

Our study determined, for a representative case of diabetic osteomyelitis, the rate of treatment failure considered acceptable by a group of clinical IDCs. The questionnaire methodology involved an approach similar to economic contingent valuation, which has been used to determine willingness-to-pay preferences for diverse health products, including needlestick avoidance devices, in vitro fertilization, autologous blood donation, and asthma therapies [1822]. A similar methodology has also been used to determine societal preferences for health states [23].

We chose a “take it or leave it” method, in which respondents were asked to accept or reject a single failure rate, rather than an open-ended question format because a closed-ended, single-bid approach to valuation may be less prone to framing and anchoring biases than an open-ended approach in contingent valuation studies [17, 35]. In the present study, an example of an anchoring bias would be a change in willingness to accept failure based on an initial higher, rather than lower, presented failure rate, whereas an example of a framing bias might be differential willingness to accept a given outcome if the scenario was framed in terms of failure, rather than success, of therapy (e.g., a 70% chance of failure as opposed to a 30% chance of success). Furthermore, response to open-ended questions requires more time from respondents and decreases study participation [36]. Finally, “take it or leave it” questions may better approximate real-life decision making [36].

The median acceptable failure rate for treatment of osteomyelitis of the foot in a patient with diabetes, as determined in this analysis, was 18.1%, which was very similar to the median acceptable rate determined by the open-ended portion of the pilot study (17.5%). Additionally, this methodology was able to determine that 75% of respondents would have found a failure rate of 7.8% to be acceptable and that 25% of respondents would have found a failure rate of 28.4% to be acceptable. This additional calculation of acceptable ranges would not have been possible with the open-ended or multiple-choice questionnaire formats. The survey case (figure 1) involved treatment of a patient with diabetes and osteomyelitis with surgical removal of the first metatarsal bone and administration of antibiotic therapy. A study involving a retrospective cohort that included 50 patients similar to the patient described in the survey, each with deep tissue infection or suspected osteomyelitis (Wagner stage 3), found that the infections of 30% of those patients who were treated conservatively (i.e., without surgery in the first 5 days after admission) failed to respond to therapy [37]. In this study, treatment failure was defined as subsequent amputation, incomplete healing, or the appearance of a new contiguous lesion. Other authors have estimated failure rates of 10%–20%, assuming optimal management of mild-to-moderate infections [13, 38]. Therefore, given the above studies, our methodology has returned failure rates similar to those found in the literature, which suggests that acceptable failure rates could be influenced by the IDC's knowledge of the literature, in addition to their clinical experience.

In a clinical trial assessing 2 competing antimicrobial regimens for limb-threatening foot infections in patients with diabetes, Grayson et al. [39] reported that abbreviated courses of antibiotic therapy (mean duration, 12.5–16.5 days) for those who had osteomyelitis were associated with treatment failure rates of 27%–35%. Because these rates exceeded the median acceptable failure rate in our study, it is likely that the cohort of IDCs in our study would consider the failure rates associated with antibiotic treatment durations of <2 weeks to be unacceptably high.

Some issues not considered in this study are the acceptable rates of treatment failure from the perspective of the patients and the impact of treatment failure on their quality of life. No appropriate treatment strategy can be deployed without understanding these values. Some have advocated a shared decision-making model for determining antibiotic prescriptions that incorporates individual physician and patient understanding and preferences [40, 41]. Understanding the failure rates that are considered acceptable by a large group of physicians could improve communication with patients and facilitate informed decision making from the patient's perspective. Use of these failure rates could also help avoid “framing” manipulations (e.g., instances in which physicians report relative risk instead of absolute risk to patients, a practice that may bias patient decision making) [42]. Thus, understanding physician preferences is the first critical step in developing effective shared decision-making models.

We believe that reducing antimicrobial treatment durations will ultimately slow the emergence of antimicrobial resistance. However, because antimicrobial resistance is a complex process, it is possible that reducing durations of therapy in certain instances will not make a large impact in reducing resistance. If this is the case, then the benefits of optimization of the duration of treatment could still include reduced medical costs and reduced antibiotic-associated adverse events, such as those seen in short-course therapy for acute bacterial cystitis [43].

The acceptable failure rate according to IDCs for treatment of diabetic osteomyelitis is high. Having an estimate of this failure rate allows a frame of reference for evaluating outcomes of clinical trials or retrospective cohort studies. This failure rate, when combined with patient preferences regarding treatment of diabetic osteomyelitis, will facilitate improved shared decision making during the physician-patient consultation.

A benefit of the methodology we used is that it provides plausible estimates of the rates of treatment failure considered acceptable among expert physicians. These acceptable failure rates, as determined by patients and physicians, can be used to implement interventions to reduce excess duration of antibiotic treatment. The reduction of excess duration of antibiotic treatment is an important step toward the ultimate goal of reducing exposure to antibiotics and thereby slowing the emergence of antibiotic-resistant organisms.

Footnotes

  • Financial support: US Centers for Disease Control and Prevention (cooperative agreement U50/CCU112346); Veterans Affairs Health Services Research and Development Service Research Career Development Award (RCD-02026-1 to E.N.P.); City of Hamilton Public Health Research, Education, and Development Program (to D.N.F.); and National Institutes of Health (K23 AI01752-01A1 to A.D.H.).

  • The contents are solely the responsibility of the authors and do not necessarily represent the official views of the Centers for Disease Control and Prevention.

  • Received June 11, 2003.
  • Accepted September 14, 2003.

References

| Table of Contents