Health Care Quality Assessment

Quality in health care, including in hospitals, can be described as doing the right thing, at the right time, in the right way - and having the best possible results. In practice however, even in the best hospitals, some patients will experience complications either after a surgical operation or as a result of other in-hospital patient care. Patient Safety Indicators provide information on how well hospitals care for patients with a wide range of health problems. Specifically, these indicators show how well a hospital is providing safe patient care by examining the number of medical errors or “adverse events” that occur during surgeries, medical procedures, and child birth.

PSIs were selected and determined, as measures of quality of patient care during hospitalization, by the Agency for Healthcare Research and Quality (AHRQ) after years of research and analysis. AHRQ developed the PSIs to help hospitals identify potentially preventable adverse events or serious medical errors. When an adverse event is identified, hospitals can put corrective systems in place to prevent the error from recurring. The Centers for Medicare and Medicaid Services (CMS) lists some of these errors as “never events”.

In 2009, the New Jersey legislature enacted the Patient Safety Act (S2471), requiring that the Department include hospital-specific data on patient-safety performance and serious medical errors in the annual New Jersey Hospital Performance Report. Evidence shows that most of the adverse events classified under each PSI are potentially preventable. This section of the report focuses on the 12 PSIs mandated for public reporting.

PSIs differ from the way the recommended care measures are calculated. Unlike the recommended care measures, a lower rate in PSIs indicates better performance by a hospital. With PSIs, lower rates mean fewer medical errors or adverse events. In addition, the numbers on the PSI tables are not scores or simple percentages, as used with the recommended care measures; they are either rates or actual volume of medical errors.

The data comes from the New Jersey hospital discharge database also known as the Uniform Bill (UB) data. Hospitals submit these data to the State. The data used for this analysis are from 2015.

The PSIs tables show the occurrence of medical errors or adverse events in each of the 71 licensed hospitals in New Jersey. Each PSI measure shows the extent to which patients experience a particular problem during their hospital stay. A rate is expressed as the number of complications or medical errors per 1,000 eligible patients.

For example, suppose a hospital had 1,000 obstetric patients who had vaginal deliveries without the assistance of an instrument. Suppose 43 out of these 1,000 patients experienced trauma during delivery. Then, the rate of occurrence of trauma at this hospital for that type of hospital admission (obstetric patients who had a vaginal delivery without an instrument) would be 43 per every 1,000 patients or 4.3% (4.3 out of 100 patients).

For PSIs, lower numbers mean fewer medical errors/adverse events. This is different from the recommended care measures, where higher numbers mean better performance.

The PSIs rates in this report are calculated by applying the AHRQ PSIs Software (Version 5.0) to the 2015 hospital discharge (UB) data. The software is known for its strength in performing “risk-adjustment”. Risk-adjusted rates are calculated by comparing the number of complications (adverse events) expected in a particular hospital, and how many patients actually experienced the adverse events.

Hospitals that treat sicker or older patients may be unfairly compared to other hospitals with healthier patients. It is very important to make adjustments for differences in patient characteristics such age, sex, comorbidities, severity of illness, etc. so that hospitals may be compared fairly.

For example, if a patient has a pre-existing chronic illness before entering the hospital, this condition may increase the likelihood or risk of that patient acquiring a complication and perhaps not surviving the procedure or treatment. Advanced age is another example of a characteristic that may increase the risk of experiencing complications.

Since 2008, hospitals have been reporting data on Present on Admission (POA) for each patient on their UB forms. Patients may have other illnesses and conditions (comorbidities) upon admission in addition to the health problem for which they were admitted.

It is often difficult to separate these pre-existing conditions from new health problems acquired during hospitalization. The POA indicator identifies these pre-existing conditions and those that occur during the hospital stay. This way, patients with the POA can be excluded from the rate calculation, when appropriate, so that performance comparison remains fair and balanced.

See the technical report for additional details such as the total number of adverse event, the total number of eligible discharges, observed and expected adverse event rates and the 95% confidence intervals for the risk-adjusted rates (when applicable).

The footnote labels, "better than statewide average” and “worse than statewide average”, shown at the top of the table describe the interpretation of the PSI rates in a meaningful way. These labels help identify hospitals that have better than average, average, or worse than average performances compared to the statewide performance, which is shown on the top row of the table and labeled “Statewide Rate.”

When a hospital’s rate is marked by a single asterisk, it means the hospital’s performance is better than the statewide average, meaning fewer adverse events than the statewide rate.

When a hospital’s rate is marked by double asterisks, it means the hospital’s performance is worse than the statewide average, meaning more adverse events than the statewide rate.  

When a hospital’s rate is not marked by an asterisk, it means the hospital’s performance is the same as or similar to the statewide rate.

Hospital rates are determined after adjusting for the risk factors of their patients. A hospital’s rate is ‘worse than average’ if its 95% confidence interval falls completely above the statewide rate. By comparison, a hospital’s rate is ‘better than average’ if its 95% confidence interval falls completely below the statewide rate.

Some rates that appear very large are not marked as ‘worse than average’ while others that appear very small are not marked as ‘better than average’. The reason for such cases may be, that rates calculated from small numbers of events tend to have wider confidence intervals that make the statewide rate fall within the interval, giving the appearance of good performance by that hospital compared to a hospital whose rate is based on higher volume (large number) of events.

Information on confidence intervals is available in the technical report.

Remember: Lower rates are better and mean the hospital has fewer adverse events than the statewide average rate.

Performance on a single PSI measure cannot reliably indicate actual quality differences among hospitals. Examining the results of all the 12 PSIs together will produce a more complete picture of overall quality of patient safety care.

Even then, PSIs are not intended as definitive quality measures and cannot provide a complete picture of quality performance in a hospital.  However, evidence has shown that these patient safety measures do show differences in hospital performance. Specifically, they measure differences in the hospitals’ ability to reduce severe and potentially preventable complications and adverse events.