Competing for Customers, Nursing Homes May Report Inflated Quality Scores to CMS

CMS’s 5-Star Rating System for nursing facilities is designed to help providers, patients, and their families make the best decisions possible. Those decisions become more difficult—and the system itself less meaningful—when many facilities receive top ratings.

"So many nursing homes have a five-star rating…but it's difficult to see through that and determine what kind of service they're actually providing,” said Xu Han, an assistant professor in the Department of Information Technology & Operations Management at Florida Atlantic University (FAU)’s College of Business. “In reality, many of them are not really providing five-star services."

Han and his colleagues collaborated with researchers from the University of Connecticut (UConn) to study that phenomenon. What they found was that some facilities’ scores may be artificially inflated.

CMS’s system averages scores on in 3 areas to arrive at a single rating out of 5: health inspections, staffing practices, and quality measures. The latter 2 are self-reported, however, and potentially easy to exaggerate. Staffing ratings are based on 2 components: registered nurse hours per resident day and total staffing hours per resident day. Quality measures are based on 8 long-stay and 3 short-stay metrics submitted through the federally-mandated process that applies to all Medicare or Medicaid-eligible nursing homes.

Facilities have a clear incentive to achieve high ratings: The team says it found a significant correlation between star-rating and profits. Financial motives don’t necessarily lead to improved service or honest reporting, though.

Han said the research team was able to empirically demonstrate rating inflation through their work. The study focused on facilities in California, exploring data from the first 5 years of the program following its 2008 introduction. The number of resident complaints, for example, was consistent between nursing facilities that performed similarly on their health inspections. Number of complaints varied significantly, however, between facilities that received the same overall rating.

The authors say this suggests inflation in the self-reported measures. They also found little direct correlation between self-reported results and health inspection results, which also raised eyebrows.

The base of CMS’s rating, according to the program standards, is the health inspection. That cannot be fudged, and it is measured comparatively, so only the top 10% of facilities per state can receive 5 stars on that. An additional star is either added or subtracted to that based on the 2 self-reported measures.

The weight placed on the measures that the facilities can’t control may further motivate them to inflate the things that they can. The researchers used a prediction model to find that least 6% and as many as 8.5% of the facilities studied had inflated their self-reported measures to boost their scores.

To Han, the work suggests further research is needed into how technology could be used to mitigate the inaccuracies created by self-reporting, and how CMS can design a new audit for the program. The authors suggest that their work could serve as the basis for that audit.

“Winning at All Costs: Analysis of Inflation in Nursing Homes’ Rating System,” was published recently in the journal Production and Operations Management.