Validity of the AHRQ Patient Safety Indicator “Central Venous Catheter-Related Bloodstream Infections”

Background

“Central venous catheter-related bloodstream infections” (CR-BSIs) is one of the patient safety indicators (PSI 7) developed by the Agency for Healthcare Research and Quality (AHRQ) to screen for potential safety events. We sought to investigate the validity of this PSI using the medical record as the gold standard.

Study Design

We conducted a retrospective cross-sectional study of all hospitalization records that met the criteria for PSI 7 within Veterans Health Administration (VA) hospitals from fiscal years 2003 to 2007. Trained abstractors used a standardized abstraction tool to review electronic medical records for the presence of a CR-BSI and the clinical circumstances surrounding the event. We determined the validity of this PSI by calculating its positive predictive value (PPV), and analyzed both true and false positive cases.

Results

Of 112 reviewed cases, 42 were true events of CR-BSIs, yielding a PPV of 38% (95% CI 29% to 47%). Seventy cases were false positives; these were attributed to correct ICD-9-CM codes but had diagnoses that fell outside the scope of the indicator (n = 28, 40%), coding inaccuracies (n = 21, 30%); and present on admission (POA) diagnoses (n = 21; 30%). Among the 42 patients with CR-BSIs, catheters were left in place for an average of 11 days, and 20% (n = 8) were placed in the femoral position.

Conclusions

PSI 7 has relatively poor predictive ability for identifying true events. Coding-related issues were the main reason for the low PPV. Implementing POA codes and using more specific ICD-9-CM codes would improve its validity. As it currently stands, PSI 7 should not be used as a pay-for-performance measure, but should be limited to use in internal quality improvement efforts.

Abbreviations and Acronyms: AHRQ, Agency for Healthcare Research and Quality, CR-BSI, central venous catheter-related bloodstream infection, IRR, inter-rater reliability, PICC, peripherally inserted central venous catheter, POA, present on admission, PPV, positive predictive value, PSI, patient safety indicator, VA, Veterans Health Administration