U.S. News Best Hospitals Rankings May Be Misleading, According to Comparion

BALTIMORE--(BUSINESS WIRE)--Comparion Medical Analytics, a company specializing in comparing the cost and quality of medical care, released a new research study today which suggests patients and purchasers may be misled by hospital rankings released in U.S. News & World Report’s 2012-2013 national honor roll of best academic medical centers.

The U.S. News study evaluates major academic medical centers across 16 specialties, 12 of which are assessed on survival rates and patient safety indicators and the other four based on the hospital’s reputation among doctors who practice in that specialty. According to Dr. Thane Forthman, president and CEO of Comparion, the study has a number of potential shortcomings which may limit the usefulness of the rankings. These include:

• The small number of quality indicators used to assess 75% of the specialties;
• Use of subjective, reputation-based data to rank the other 25% of specialties;
• Assumption that comparing academic medical centers to less than 10% of U.S. hospitals nationally provides reliable information regarding their quality of care.

Comparion’s research findings, displayed in Table 1 below, were based on the company’s patent-pending CareChex^® composite quality scoring methodology which evaluates hospital care using a comprehensive set of process of care measures, outcome measures, and patient satisfaction scores. “Our study indicates that when a complete set of objective quality measures are used, the U.S. News rankings have no correlation to our CareChex quality rankings. In fact, the correlation is slightly negative.” says Dr. Forthman.

Table 1. Comparison of U.S. News and Comparion Hospital Rankings
U.S. News Rank	Hospital/Medical Center	Comparion   CareChex Rank	Hospital/Medical Center
1	Massachusetts General	1	Indiana University
2	Johns Hopkins	2	Mayo Clinic
3	Mayo Clinic	3	Duke University
4	Cleveland Clinic	4	University of Michigan
5	UCLA	5	Northwestern
6	Barnes-Jewish	6	Brigham and Women's
7	New York-Presbyterian	7	UCSF
8	Duke University	8	Massachusetts General
9	Brigham and Women's	9	UCLA
10	University of Pittsburgh	10	University of Pittsburgh
11	NYU Langone	11	New York-Presbyterian
12	Northwestern	12	Barnes-Jewish
13	UCSF	13	NYU Langone
14	Mount Sinai	14	Johns Hopkins
15	University of Pennsylvania	15	University of Pennsylvania
16	Indiana University	16	Cleveland Clinic
17	University of Michigan	17	Mount Sinai
(Spearman’s Correlation Coefficient = -.142; Confidence Interval = 95%; p < .01)

Of the hospitals ranked by U.S. News, only the University of Pittsburgh Medical Center and the Hospital of the University of Pennsylvania remained unchanged in their rankings (10^th and 15^th respectively). Both Indiana University Health and University of Michigan’s Hospitals and Health Centers experienced a radical improvement in their rankings, moving from the 16^th and 17^th respectively to 1^st and 4^th based on their CareChex scores. The largest drop in rankings was experienced by Johns Hopkins Hospital and the Cleveland Clinic which moved from 2^nd and 4^th respectively to 14^th and 16^th.

While these findings are significant in themselves, when these hospitals were compared to the national norm of all general, acute, non-federal U.S. hospitals, interesting observations were noted. Specifically, when each hospital’s CareChex composite quality score was evaluated, the distribution of quality performance across academic medical centers varied greatly--with many falling below the national average. Table 2 below identifies how each hospital’s quality performance compared to the nation.

Table 2. Distribution of Comparion’s National Composite Quality Scores by Hospital

Top 5%

Indiana University Health (99.6^th percentile)

Mayo Clinic (97.7^th percentile)

Duke University Medical Center (96.7^th percentile)

University of Michigan’s Hospitals and Health Centers (95.5^th percentile)

Top 10%

Northwestern Memorial Hospital (94.9^th percentile)

Brigham and Women’s Hospital (94^th percentile)

Top 15%

UCSF Medical Center (85.5^th percentile)

Average (~ 50th percentile)

Massachusetts General Hospital

UCLA Medical Center

University of Pittsburgh Medical Center

New York-Presbyterian Hospital

Below Average (25th to 49.9th percentile)

Barnes-Jewish Hospital

NYU Langone Medical Center

The Johns Hopkins Hospital

Well Below Average (< 25th percentile)

Hospital of the University of Pennsylvania

The Cleveland Clinic

Mount Sinai Medical Center

Dr. Forthman noted, “In order to provide reliable comparisons among hospitals, the study removed unusually difficult cases (i.e., outliers) and adjusted for differences in patient risk factors (i.e., clinical and demographic characteristics). Otherwise, differences in quality performance could be attributed to differences in the complexity of cases treated. Additionally, all performance indicators were tested for statistical significance so a hospital’s relative rank and percentile score could be accurately determined.”

The important conclusion of Comparion’s research is patients and purchasers should not assume that academic medical centers ranked by U.S. News necessarily provide the best quality of care. There are many hospitals not affiliated with universities that provide equal or better care, and many U.S. News best hospitals actually perform poorly on comprehensive measures of quality. This conclusion is supported by other research conducted by Comparion which evaluated the quality performance of 118 university hospitals (Fierce Healthcare, November 2010; Becker’s Hospital Review, November 2010).

About Comparion Medical Analytics

Comparion Medical Analytics is one of the nation’s largest privately held healthcare information services companies with offices located in Greenville, SC, Baltimore, MD, and Phoenix, AZ. Over 1,000 healthcare professionals across leading academic medical centers, health systems, Fortune 500 companies, and payers rely on their software solutions to assess the cost and quality of medical services delivered across the continuum of care.

For more information regarding their products and services, visit ComparionAnalytics.com or contact [e-mail].