The ACCP Opposes Newly Endorsed ICU Measures
The ACCP, along with the American Thoracic Society (ATS) and the American Association of Critical Care Nurses (AACN), appealed two measures on ICU outcomes recently endorsed by the National Quality Forum (NQF).
The NQF has proposed the use of ICU length of stay and in-hospital mortality measures using the Mortality Prediction Model-III (MPM-III) as the risk adjustment tool to create standardized mortality ratios (SMRs) and length of stay ratios by which to compare the performance of ICUs to one another. On October 20, the Presidents of the ACCP, ATS, and AACN wrote a letter to the steering committee on patient outcome measures, which outlined multiple concerns related to the possible implementation of these performance measures.
The ACCP first commented on the ICU measures in May 2010, responding to the NQF National Voluntary Consensus Standards for Patient Outcomes, First Report for Phases 1 and 2: A Consensus Report (see below). Despite these comments, the measures were endorsed by the NQF.
Although the ACCP, ATS, and AACN believe the use of risk-adjusted hospital mortality is potentially a valid outcome measure in ICU patients, the current proposal is unacceptable because the inclusion of interhospital transfers has been shown to alter SMRs, making them no longer an accurate reflection of the quality of care provided in an ICU. The group is concerned the SMRs would incentivize a "gaming" of mortality data by some hospitals, shifting patients at risk of death from one hospital to the next, or to costly long term acute facilities, in order to appear to be of higher quality. In addition, the letter expressed concern that length of stay is not a patient-centered measure, having more to do with resource allocation than ICU quality, and it can also be "gamed" by discharging patients from ICUs "sicker and quicker" than was previously the case.
On November 3, the NQF Consensus Standards Approval Committee (CSAC) met in Washington, DC, to review the concerns expressed by the ACCP, ATS, and AACN in their October 20 letter. R. Adams Dudley, MD, the developer of the measures, responded to the concerns expressed in the letter. He stated that the MPM-III model has been extensively validated in a large number of peer-reviewed publications and represents a lesser data collection burden than similar prediction models, such as acute physiology and chronic health evaluation (APACHE) or simplified acute physiology score (SAPS). While Dr Dudley acknowledged that length of stay did not reflect directly on ICU quality, he believes this measure would still be useful because he does not think an ICU will want to be known as having a short length of stay while at the same time having a high actual to predicted mortality ratio.
Responding on behalf of the ACCP, ATS, and AACN, Robert Hyzy, MD, FCCP, Medical Director of the Critical Care Medicine Unit at the University of Michigan Hospital and member of the ACCP Quality Improvement Committee, reiterated that the ACCP, ATS, and AACN believe in the validity of using SMRs but are requesting the measure adopted is a more genuine representation of ICU quality. As the inclusion of interhospital transfers has been shown in the work of several investigators, including Dr Dudley, to exert a distorting effect on SMRs, the proposed measure is at variance with espoused principle of "scientific acceptability" by NQF. Dr Hyzy recommended the exclusion of interhospital transfers in the generation of SMRs and suggested 30-day mortality would be a more fair way to evaluate and compare ICU quality. He went on to say that he was pleased Dr Dudley agreed length of stay did not reflect ICU quality, emphasizing that the adoption of this measure would potentially pose an unnecessary risk to patient safety. He suggested the ongoing measurement of hospital-acquired conditions known to lengthen ICU length of stay, such as the rate of catheter-associated bloodstream infection or ventilator-associated pneumonia, were better reflections of ICU quality; as a result, ICU length of stay should not be adopted as a quality measure by NQF.
Dr Dudley stated the magnitude of change in SMRs created by excluding interhospital transfers in his research study did not result in any alteration in the ranking of ICUs: those with high actual to predicted mortality ratios and those with low actual to predicted mortality ratios were the same if transferred patients were excluded and 30-day mortality, rather than hospital mortality, was used to make the comparison. In addition, ICU mortality and length of stay have been publically reported in the state of California for 4 years. When Dr Hyzy pointed out that Dr Dudley’s study in California had occurred prior to the onset of public reporting and that the distortion in SMRs wrought by the inclusion of transfers has probably become significantly higher, Dr Dudley responded that no one in California was complaining about data methodology. Dr Hyzy urged CSAC, at this time of an increasing intensivist shortage and an aging population with a burgeoning need for critical care services, not to pass the measures as proposed, which misaligns incentives for patients to get the best care in the right location for as long as is required. The NQF CSAC voted to pass the measures.
The ACCP QIC remains committed to representing ACCP members effectively to agencies generating quality measures, including NQF. However, the repeated episodes of nonresponsiveness to our concerns led to the withdrawal of the ACCP as a dues-paying member of NQF in 2009. This week’s events are just the latest in a series of episodes where the ACCP has found its well-reasoned and evidence-based opinions unheeded by NQF. The ACCP fears for the long-term credibility of this agency as a neutral arbiter of quality improvement measures.
Search the Performance Measures database to read ACCP comments to other NQF endorsed measures.
On May 31, the ACCP responded to the NQF National Voluntary Consensus Standards for Patient Outcomes, First Report for Phases 1 and 2: A Consensus Report, with the following:
- ICU Length of Stay
For all patients admitted to the ICU, total duration of time spent in the ICU until time of discharge; both observed and risk-adjusted length of stay (LOS) reported with the predicted LOS measured using an adjustment model based on the Mortality Probability Model III
ACCP: Disapprove with comments. On behalf of the ACCP, the ACCP Quality Improvement Committee (QIC) appreciates the opportunity to comment on this measure. The QIC feels that this measure does not measure quality. The QIC also noted that while this measure can be risk-adjusted for patient factors, it cannot be risk-adjusted for other factors, such as availability of step-down units, long-term ventilator facilities, nurse staffing, and bed availability. - Intensive Care: In-Hospital Mortality Rate
For all patients admitted to the ICU, the percentage of patients whose hospital outcome is death; both observed and risk-adjusted mortality rates are reported with predicted rates based on the Mortality Probability Admission Model III
ACCP: Disapprove with comments. On behalf of the ACCP, the ACCP QIC appreciates the opportunity to comment on this measure. The QIC applauds the principle and understands the need for measuring mortality in terms of quality care. However, the QIC notes that there are too many variables that cannot be accounted for in this measure. The QIC noted that there is not any narrowly defined expected outcomes in this area. The QIC fears that this measure may be gamed for more favorable results.
Despite these comments, the measures were endorsed by the NQF.
