This page describes the methodology used to calculate the skilled nursing facility (“SNF”) ratings used to create the CareWatch Rating Much of the methodology was inspired or created in relation to Griffin, John M. and Priest, Alex, Overbilling and Killing? An Examination of the Skilled Nursing Industry (October 15, 2024). Available at SSRN: https://ssrn.com/abstract=4987755 or http://dx.doi.org/10.2139/ssrn.4987755

​​

SNF Rating Categories 

​Rating categories between 1 to 5 were assigned based on their percent ranking (or percentile) on metrics analyzed. Specifically, SNFs in the top 10% for a particular rating category were assigned a “5” and SNFs in the lowest 20% for a given metric were assigned a “1”. SNFs in the 20th to 90th percentiles were assigned evenly among “2”, “3”, and “4”.  In the Externally Validated Health Outcomes and Ethical Reimbursement Rating facilities receive points based on their performance in each metric which are then summed to get a total point value for each category similar to the CMS Five-Star Quality Rating System. Missing values due to low population size are currently treated as a 3-star rating (average). The individual category ratings are then weighted under the following setup and summed up to create a composite rating:

• CMS Health Inspection Rating (15%)

• CMS Staffing Rating (15%)

• Externally Validated Health Outcomes (35%)

• Ethical Reimbursement Rating (35%)

The lowest 20% of composite ratings are assigned a 1-star CareWatch rating, while the highest 10% of composite ratings are assigned a 5-star CareWatch rating. The middle 70% is distributed into three equal size categories for 2-star, 3-star, and 4-star facilities.

​​​

CMS Health Inspection Rating (15%)

The CMS Health Inspection Rating is the initial starting point for the the Five-Star Quality Rating System contributes 15% to the CareWatch SNF rating system. The health inspection rating uses state health inspections to identify deficiencies at the SNF. Each deficiency is given a weight based on the scope and severity of the problem. CareWatch uses the raw rating published by CMS for the purposed of calculation.

 

CMS Staffing Rating (15%)

The CMS Staffing Rating is one component of the Five-Star Quality Rating System that contributes to 15% to the CareWatch SNF rating system. The staffing rating aims to reward facilities that achieve continuity with staff, and have a higher number of nursing hours per resident day. Multiple studies, like The Relationship Between Nurse Staffing Levels and the Quality of Nursing Home Care (Kramer and Fish, 2001), have shown that there is a visible relationship between nursing home quality of care and nurse staffing ratios. Additionally, recent studies like Association between staff turnover and nursing home quality - evidence from payroll-based journal data (Zheng, Williams, Shulman, and White, 2022) have shown that high staff turnover is associated with worse resident outcomes. CareWatch uses the raw rating published by CMS for the purposes of calculation.

​​

Externally Validated Health Outcomes (35%)

Within the CMS Five-Star Quality Rating System is a star rating focused purely on health outcomes of patients at the SNF. Some of the metrics include urinary tract infections (UTIs), major injury falls, and pressure ulcers. Since the CMS metrics are based on data reported on MDS assessments filed by the SNF, it is possible for a facility to manipulate their rating to appear to provide better care, by not reporting conditions that patients experience. The Externally Validated Health Outcomes rating aims to address the flaws in the CMS Quality Rating and is comprised of 3 categories of 6 subcomponents:​​

• Abusive Reviews and Review Sentiment - The percent of online reviews that allege abuse of a patient and he average sentiment score of online reviews ranging from -1 (very negative) to 1 (very positive)

• Misreported UTIs and Pressure Ulcers- The percent of all patients that were not reported as having a UTI at the SNF, but were reported with a UTI within 2 days of discharge at an inpatient facility and the percent of all patients that were not reported as having a pressure ulcer at the SNF, but were reported with a UTI within 2 days of discharge at an inpatient facility

• Post-discharge Mortality and Rehospitalization - The percent of all patients who die within 90 days of discharge from the SNF the percent of all patients that are readmitted to a hospital within 30 days of discharge from the SNF

​

Abusive Reviews and Review Sentiment
The abusive reviews and review sentiment are two subcomponents of the Externally Validated Health Outcomes. Each component is based on data collected from online reviews from websites like Caring.com, an online senior care website that provides information and hosts nursing home reviews. Labeling of whether a reviews alleged abuse was completed manually on 100 of the over 60,000 reviews that were collected and linked to a specific nursing facility. Each unlabeled reviews was then fed through a Support Vector Machine algorithm to predict the probability that a given review indicated abuse occurred. For the sake of the metric, only reviews that had a probability of alleged abuse above 90% were considered abusive reviews. The final metric can be understood as the percent of online reviews collected that allege abuse.

​

Online reviews were also used to calculated the average review sentiment for each facility. Sentiment in this case can best be understood as a scale where one side indicates negative sentiment (i.e. "My father hated his stay at the SNF") represented by a negative value, while the other side indicated a positive sentiment (i.e. "My mother loved her stay at the SNF") represented by a positive value. The values for each review range from -1 to 1. All reviews at a given facility are averaged out to calculate the review sentiment score for the facility. 

​

Misreported UTIs and Pressure Ulcers 
Pressure ulcers and UTIs are components of the CMS Five-Star Quality Rating System. As previously discussed, these data that is used to calculate these metrics is self-reported by the SNF. Previous analysis by Integra Med Analytics has shown that underreporting of these key quality measures occurs at significant rates. As a modification to previous analysis, this metric calculates a lower bound for the number of patients that had a UTI or pressure ulcer that was not reported by the SNF. The numerator for the metric is the total number of patients who, within 2 days of discharge from the SNF, were admitted to an inpatient facility with a diagnosis of a UTI or pressure ulcer respectively that was not reported at the SNF. The denominator for each metric is the total number of Medicare patients that went to the SNF. This results in a true minimum value for the percentage of patients misreported for UTIs and pressure ulcers. Additional situations of misreporting that are not included are patients that leave the SNF but do not go to an inpatient facility, and any instance of a misreported condition during the SNF stay. 

​

Post-discharge Mortality and Rehospitalization
The final components of the Externally Validated Health Outcome rating are post-discharge mortality and rehospitalization rates. The post-discharge mortality metric reports the percentage of patients that died within 90 days of discharge from the SNF and the rehospitalization metric reports the percentage of patients that are rehospitalized within 30 days of leaving the SNF. 

​

Ethical Reimbursement Rating (35%)

Overbilling and Killing? An Examination of the Skilled Nursing Industry (Griffin and Priest, 2024) shows that systems (affiliated entities) that code conditions that have high medicare reimbursement, have higher rates of adverse health outcomes in their patients. In a recent settlement, The Grand Health Care System admitted to keeping patients for extended lengths of time and falsifying the quantity of therapy patients provided. This is just one recent instance of a situation where a SNF system put profit over patients. The Ethical Reimbursement Rating is a star rating comprised of 3 categories of 8 metrics that are connected to high reimbursement conditions within the Patient Driven Payment Model:​​

• Length of Stay - The average number of medicare reimbursed days patients spend at a facility before discharge

• Patient Driven Payment Model Nursing Components

  • Depression - The percent of patients that were coded as depressed under the PHQ-9 on the MDS

  • Special Care High - The percent of all patients that were coded for a condition that qualified them for special care high

  • Low Function Score - The percent of all patients that were coded with a total function score between 0 and 5 for the sections within the nursing component of the Patient Driven Payment Model

• Patient Driven Payment Model SLP Components​

  • Acute Neurologic Conditions - The percent of all patients whose primary diagnosis is an acute neurologic condition

  • SLP Comorbidities - The percent of all patients diagnosed with at least one condition considered an SLP comorbidity

  • Cognitive Impairment - The percent of patients coded as cognitively impaired, primarily from the BIMS assessment

  • Swallowing Disorders and Mechanically Altered Diets - The percent of all patients coded for both a swallowing disorder and a mechanically altered diet

​

Length of Stay

Length of stay is one of the 8 metrics used within the Ethical Reimbursement Rating. While some patients will inevitably need to stay for extended lengths of time, the intention of a SNF stay is to receive short term care following an inpatient (hospital) stay. This means that most patients should leave the facility within 30 days of admission, something reflected by the average length of stay of around 28 days nationally. This metric includes the average length of stay of all Medicare Part A patients within the SNF.​

​

Patient Driven Payment Model Nursing Components
The depression, special care high, and low function score metrics are all subcomponents of the PDPM Nursing component. Each condition results in the SNF receiving additional reimbursement every day during the patients stay. Tacking on conditions that the patient does not have results in higher rates of each of the three conditions, leading to a lower Ethical Reimbursement Rating. 

​

The depression component is based on section D of the MDS, otherwise known as the PHQ-9. During this assessment, the resident will be asked questions related to their current mood status. In situations where a patient's PHQ-9 is filled out inaccurately, they may receive medication (antidepressants) that they do not need. It is important that the SNF properly completes this section regardless of the financial benefit they could gain from an inaccurate assessment. The depression metric is the percent of all Medicare Part A patients that are coded as depressed based on their PHQ-9. 

​

Special care high is another component within the nursing component of the Patient Driven Payment Model that includes many conditions that are deemed to require additional effort/care from the facility. These conditions are very serious and should result in specialized treatment for the patient. Alternatively, when the patient doesn't have one of these intensive conditions, two possible scenarios can occur; the facility will receive additional reimbursement for services not provided or the facility will provide services that the patient does not need. The special care high metric is the percent of all Medicare Part A patients that are coded for special care high based on conditions or services documented on their MDS assessment. 

​

Function score is the final component within the nursing component of the Patient Driven Payment Model. A patient's function score is calculated based on the information provided in Section GG: Functional Abilities and Goals within the MDS assessment. A lower function score results in higher reimbursement under PDPM. If a patient is given a lower function score than they have, it could result in a decrease in therapy received. The lowest function score category is designed to be reserved for patients with mobility that is extremely limited or worse. The low function score metric is the percent of all Medicare Part A patients that are coded for the lowest function score grouping based on data contained in Section GG on their MDS assessment. 

​​

Patient Driven Payment Model Speech Language Pathology Components

The acute neurologic condition, SLP comorbidities, cognitive impairment, and swallowing disorder and mechanically altered diet metrics are all subcomponents of the PDPM SLP component. Each condition results in the SNF receiving additional reimbursement every day during the patients stay. Tacking on conditions that the patient does not have results in higher rates of each of the three conditions, leading to a lower Ethical Reimbursement Rating. 

​​

The acute neurologic condition component is based on item I0020B on the MDS, otherwise known as the primary diagnosis. The primary diagnosis is the primary reason that the patient is having a Medicare Part A SNF stay. Under the PDPM billing system each primary diagnosis maps to one the 10 PDPM clinical categories (additional detail available). Acute neurologic conditions are one of these 10 clinical categories and result in a significant increase in reimbursement for the SLP component. An improperly coded primary diagnosis can lead to the patient receiving different treatment plans than is necessary to recover and have a safe discharge from the facility. The acute neurologic condition metric is the percent of all Medicare Part A patients that are diagnosed with an acute neurologic condition in their primary diagnosis. 

​

Comorbidities are additional conditions that the patient has beyond their primary condition. Under PDPM, CMS designates 12 conditions and services as SLP-related comorbidities including ALS, Dysphagia, and Traumatic Brain Injuries to name a few. When a patient is coded for at least one of these conditions, the facility receives additional reimbursement for every day during their stay. The SLP comorbidity metric is the percent of all Medicare Part A patients that are diagnosed with at least one SLP comorbidity.

 ​

Cognitive impairment status is defined as as receiving a score of 12 or less on the Brief Interview for Mental Status (BIMS). When a patient qualifies as cognitively impaired, the facility receives additional reimbursement. Additionally, cognitive impairment status opens the door for bad facilities to take advantage of patients by switching their coverage from Medicare Advantage to Medicare Part A resulting in lost benefits. The cognitive impairment metric is the percent of all Medicare Part A patients that qualify as cognitively impaired on the BIMS assessment.

​

The final SLP related metric swallowing disorders and mechanically altered diets. Under the SLP component, patients can be coded both, one or neither of the two dietary related conditions. Coding for one condition increases reimbursement slightly, but a patient coded for both conditions increases reimbursement significantly. This can lead to facilities tacking on the second dietary related condition when only one is present to increase reimbursement. As such, the swallowing disorders and mechanically altered diet metric is the percent of all Medicare Part A patients that are coded as having both a mechanically altered diet and a swallowing disorder.