Methodology
II. Applicability of Data Sources
III. Risk Adjustment
IV. Benefits of Using the MedPAR Data Sets
V. Limitations of Using the MedPAR Data Sets
VI. Description of Data Quality Checks
VII. Measures UCHC Uses
B. Condition and Procedure Measures
C. Quality Measures for Recommended Care of Specific Conditions
VIII.Calculations of "IQI and PSI" results.
IX. Calculation of "Conditions and Procedures" Results
X. Interpretation of the "IQI and PSI" Results
XI. Interpretation of Conditions and Procedure Analysis Results
XII.Interpretation of Quality Measures for Recommended Care of Specific Conditions (DHHS)
XIII.Cautions of Interpretation of results
UCompareHealthCare LLC uses the Medpar (Medicare Provider Analysis and Review) data sets to calculate our measures relating to hospital quality, conditions and procedures. These data is obtained from the federal government and specifically Center for Medicare and Medicaid Services (CMS). Our data spans multiple years and currently contains many millions of inpatient discharge records. We report one year of data at a time. The MedPAR data file contains records on all Medicare beneficiaries who use hospital inpatient services. The records are stripped of most data elements that permit identification of beneficiaries.
Data used for the "Quality Measures for Recommended Care of Specific Conditions" is obtained from the Department of Health and Human Services (DHHS).
Data for the services offered and demographic information for each institution are obtained from the federal government's Provider of Service (POS) data file, from recent Medicare cost reports, and by contacting or viewing the institutions Web sites, as well as any public information available.
II. Applicability of Data Sources
Hospital administrative discharge data make possible the evaluation of the medical care delivered in our nation's hospitals. These data sets, which are collected as a routine step in the delivery of hospital services, provide information on diagnoses, procedures, age, gender, admission source, and discharge status. Although quality assessments based on administrative discharge data cannot be definitive, they can be useful in the process of identifying potential quality problems as well as showing quality efforts in a quantifiable format. These findings can be used to further investigate the quality of care one might receive at hospitals throughout the United States. It is important to remember that the MedPAR data set typically accounts for approximately 50% of an institutions volume.
Data for the Quality Measures for Recommended Care of Specific Conditions is provided in a calculated format from the federal government. Thus UCHC performs no additional calculations of this data.
Demographic information is derived directly from federal government records. We have derived service information by inference upon review and analysis of the POS and Medicare cost reports.
Risk adjustment is required for meaningful and fair comparisons within the healthcare system. Risk adjustment is a process of accounting for differences in patient populations based on many parameters such as age, sex, severity of illness, risk of mortality, and other attributes. Risk adjustment is needed because diseases and medical conditions are rarely randomly spread across people and populations. UCHC risk adjustment has employed the 3M APR™ DRG Risk Adjustment Software. This software was created by and is owned and licensed by the 3M Company. All copyrights in and to the 3M APR™ DRG Software and to the 3M APR™ DRG Classification System (including the selection, coordination, and arrangement of all codes) are owned by 3M. All rights reserved. The use of this product allows us to provide our users with meaningful comparisons among hospitals across a wide range of resource and outcome measures.
IV. Benefits of Using the MedPAR Data Sets
Our MedPAR database includes data on most U.S. hospitals, with the exception of VA (Veterans Affairs) and military hospitals. These data are readily available for transformation to a usable form for comparative purposes. The federal government requires hospitals to submit uniform, complete, and accurate information about patient discharges. This requirement, backed by substantial penalties for failure to provide this information, results in a cost effective, reliable, and usable source of administrative discharge data for the purpose of assessing quality.
V. Limitations of Using the MedPAR Data Sets
MedPAR data sets constitute primarily an older section of the population, which tends to consume healthcare services more often, as a general rule, than the population as a whole. For this reason, the data contain minimal information on obstetrical and childhood diseases. Other considerations in using these data include coding differences among hospitals, ambiguity about when a condition occurs, and limitations in ICD-9-CM coding. The ICD-9-CM is recommended for use in all clinical settings, but it is required for reporting diagnoses and diseases for all programs under the U.S. Public Health Service or the Centers for Medicare & Medicaid Services (formerly known as the Health Care Financing Administration).
We ask that this information be kept in mind as you review the results of your comparisons. As no single source of data is conclusive, we encourage further investigation with healthcare professionals of your choosing. We have taken many steps to insure the validity and reliability of our data and calculations. We want to make sure you realize that no single method is conclusive in all regards. Consequently, it is inevitable that some data may be incorrect, outdated, or missing.
VI. Description of Data Quality Checks
UCHC uses several data cleaning (quality control) techniques to ensure the integrity of our databases. These techniques include data verification, audits, and quality screening to remove possibly erroneous data. When data are transferred from one source to another, we assess the completeness of transfer by checking raw totals versus transferred totals in the new medium. In the first phase of cleaning, we search for inaccurate and inconsistent fields and remove all records that have these characteristics. We then apply more stringent criteria, including removing all records that show negative reimbursement amounts, negative length of stays, or negative total charges. Whenever any inconsistency is discovered, the record is identified and eliminated from any calculations.
We have adopted many of the Inpatient Quality Indicators (IQIs) and Patient Safety Indicators (PSIs) that have been developed under the guidance of the federal Agency for Healthcare Research & Quality (AHRQ). These indicators have withstood extensive review and are thoroughly documented. We have applied those measure that we feel provide the best picture of quality based on the data sets we are using (MedPAR) as well as the recommendations put forward in AHRQ reference manuals. We process our data for these measures in the format outlined by AHRQ. The data are later reviewed, as described in the Calculations of IQI and PSI Results section below.
B. Condition and Procedure Measures
These measures are provided in a graphic format for comparison purposes. Please note that these are not measures which have been evaluated by the AHRQ. We provide these measures for comparative purposes only. We have applied stratified risk adjustment (3M APR-DRG™) with the indirect standardization method in order to better reflect case mix. We have included 19 conditions or procedures for your review. These conditions and procedures are common and thus constitute reasonable volumes at most institutions.
C. Quality Measures for Recommended Care of Specific Conditions
The U.S. Department of Health and Human Services (DHHS) records data on how often hospitals provide particular recommended care for patients with specific conditions. We have included some of these measures of frequency of recommended care for adults being treated for heart attack, heart failure, or pneumonia, or having surgery. UCHC obtains this information from DHHS.
Please note that quality information is not available in this category for children's, psychiatric, rehabilitation, or long-term care hospitals because they generally do not treat adult patients for heart attack, heart failure, or pneumonia, or perform surgery on adults.
VIII. Calculations of "IQI and PSI" results.
IQIs (Inpatient Quality Indicators) are calculated as defined by the AHRQ (Version 3.0 (February 20, 2006). PSIs (Patient Safety Indicators) are calculated as defined by the AHRQ utilizing (Version 3.0a (May 1, 2006). Risk adjustment of IQIs is performed with 3M APR-DRG™ software and risk adjustment of the PSIs is performed as described by AHRQ.
For those measures plotted on the modified Levy-Jennings chart, we establish a range of "typical" performance. This range represents the middle 50% of the populations "risk adjusted value ("RP")" for the specific measure. All volume measures are displayed as the "TP" value. Results that appear in the top 25% of the graph is performance considered "better than typical". Results in the lower 25% of the graph represents performance considered "less than typical". We establish these regions in the following manner.
Data for each measure is cleaned, processed, risk adjusted, and calculated in the prescribed format, then imported into our central processing center for further analysis. Each population of results for each measure are graphed and presented for visual inspection and review.
Evaluation of the total population of IQI results is carried out in three initial phases. Review of all results, review of results that are characterized by those institution that had greater than 6 results and finally by those that had greater than 20 results for the measure, if data is available. PSI "risk adjusted values" are plotted with no consideration given to the number of events. We evaluate 57 different statistical parameters on the total population of each group of results (IQIs and PSIs). These analyses allow us to establish the shape i.e. (skewness and kurtosis) of the population of results.
We display to our users only those institutions that have more that 6 results for each calculated IQI measure and all results for each PSI measure. From this population, we eliminate from future calculations those institutions that have in-hospital mortality rates or post procedural mortality rates of "0%" (TP=0) for the specific IQI measure. If risk adjustment methodology has increased an institution's "0" mortality rate we do not report the risk adjusted increased value. We report the "0%" value. These institutions will be plotted automatically at the top of the graph. Those that have PSIs measures of "0 / 1000 discharges" are also plotted at the top of each range, although they are not eliminated from the calculation of (Median, Range, ect.). In the case of PSI (2) Death in Low Mortality DRGs, PSI(5) Foreign Body Left During Procedure and PSI (16) Transfusion Reaction, we report the observed values. The rationale for this approach is that these events are considered "never events" (events that should not occur) and thus those institutions that have no events are plotted at the top of the "better than typical" range. We have plotted these measures in this manner as a result of feedback from "AHRQ Quality Indicators Support dated November 27, 2006."
We calculate standard scores using the median (as opposed to the mean). Standard scores or values for a given variable are derived values used in statistics to make individual values on a given variable more easily interpretable. Standard scores or values remove the relativity of the particular "measurement units and scale" used for a given variable by putting the values on a Ôstandard and directly comparable scale" which has "equal units" and will indicate how many such units the original value was above or below a set fixed comparison point (in our case, the median).
Usually the "fixed comparison point" is the mean of the variable in question and the "equal units" are the number of standard deviation units above and below the mean. UCHC has chosen to use the median as the fixed comparison point. When the underlying distributions of a set of variables are extremely skewed, such as those in this database, a z-score and z-scale based on the mean can be and most often is highly misleading. For skewed distributions, the median is a far better comparison point. The median is the Ôtrue center" of the distribution and is not biased by the skewing or other aspects of the underlying distribution. No matter what the nature of the underlying distribution, the median represents the point where half the values are above and below this point. We then establish a point +25% of the median value and -25% of the median value. This band is our range of typical performance, which is the middle 50% of the observed results.
We then calculate a standard score based on the interquartile range (IQR) for the specific measure from all data on institutions that have more than 6 of the specific procedure of interest for IQIs and all institutions for PSIs. The IQR is calculated by taking Q3 (75th percentile) and subtracting Q1 (25th percentile) and this difference equals the "IQR". Q3 also equals the +25% point from the Median (i.e. the 75th percentile) and Q1 equals the -25% point from the Median (i.e. the 25th percentile).
Next we calculate the percentile score for each result for each institution. A percentile score for a given Z-score (whether it is median or mean based) is the percentage of cases, in this case the percentage of hospitals in the particular sample, that are at or below that particular Z-value. The percentile score allows the Z-value to be exactly located in the distribution of values in the sample.
A percentile score is another type of ordinal-based standard score that is most appropriate for skewed distributions, as the percentage of cases in each deviation unit along the range of the distribution differ markedly from the normal distribution. The Z-value in question would be misinterpreted if it was converted to the standard normal distribution percentages for the z-scale. The exact percentile score in combination with the exact Z-score, therefore, gives the best and most accurate description and standardization of a particular value in a sample and allows it to be compared to other values in that sample and between samples.
We display a confidence interval. This confidence interval allows the reader to interpret the difference between two values and determine if the difference is statistically significant. The significant difference between two percentiles in the same distribution is conceptually the same as the standard error of measurement or the standard error of sample of a percentile point. The median of the distribution in question is the fiftieth (50th) percentile of the distribution. It is a specific percentile point no different from any other percentile point in the distribution. The standard error of sampling of the median is an estimate of how much a given percentile in the distribution could vary due to sampling error. The standard error of sampling of the median therefore is the sampling error unit needed to construct a confidence interval around any percentile in the distribution and compute the number of percentile points the value in question needs to differ from the next percentile point, above or below it, to be significantly different and not just a difference due to sampling error at the .05 level (95% confidence). This value is presented at the bottom of each graph for each institution.
IX. Calculation of "Conditions and Procedures" Results.
UCHC uses a stratified risk adjustment process with indirect rate standardization for calculation of "Length of Stay", "Mortality", "Total Charges", "National Average LOS" and "National Total Charges" based on specific APR-DRG subclasses. All volume measures are presented as raw volume.
Our process involves the evaluation of each patient record for the "subclass" of either the "risk of mortality" (rom) or the "severity of illness" (soi) grouped by specific APR DRG category within that institution. For all mortality measures we use risk of mortality as the designation of the subclass. For other measures except volume we use the severity of illness for subclass identification.
UCHC performs calculations necessary to derive the following. The "observed" value which is the value based on straight forward weighted calculations of the inter hospital data set. The "expected" value for the institution which includes national average (reference) values applied to the calculation of length of stay and total charges. The "risk adjusted" value for length of stay and total charges for the specific institution. These calculations require the determination of the national averages for each component; they are included for comparison as well.
Mortality rates are calculated in a similar format with the exception of using the risk of mortality subclass versus the severity of illness subclass for stratification.
UCHC presents the following data on our Conditions and Procedures Analysis reports.
| Volume: | Total number of observed cases for the institution. |
| Average Length of Stay: | Average time that a patient stayed in this hospital for the designated condition or procedure. |
| Average Total Charges: | Average total charges the hospital reported for the patients hospital stay for the designated procedure or condition. |
| Average Mortality: | Average percent of patients that die who have had the designated procedure or condition. |
| National Average Length of Stay: | Average time that a patient stays in the hospital within the nation for this condition or procedure. |
| National Average Total Charges: | Average total charge within the nation for the designated condition or procedure. |
| Test of Significance: | We provide a number which will let the reader know if the difference between the risk adjusted value reported for the hospital is statistically different from the national value at a confidence level of .05 |
The test of significance is important because it will allow the reader to determine if the risk adjusted value is statistically different from the National Average at a level of 95% confidence (.05 level). We have applied the Fisher z-test as our test of significance vs the student t-test used more for comparing two sample values.
X. Interpretation of the "IQI and PSI" Results
Results are plotted on a modified Levy-Jennings plot for the Inpatient Quality Indicators (IQIs) and Patient Safety Indicators (PSIs). This will allow an easy approach to reviewing the comparative values for each of the four hospitals that you have selected. If one of the institutions you have selected has less than (6) six patients for any IQI, the data for that hospital will not be displayed. We will provide the appropriate designation that this data is not available and why. We only report data on those institutions that have 6 or more events with the exception of all volume indicators and PSIs in which case we display all statistics regardless of the number of events for the specific hospital. In the case of volume indicators we provide the "raw" volume for the institution. If data are not available or the institution had less than (6) six events for any IQI calculation, we will provide appropriate designation of that fact. In those cases, we will identify the specific institutions result by placing "No Data Available" in the field. No results will be plotted and there will be no display of the associated statistics for that specific institution.
PSIs are plotted in the same format as the IQIs. We include all results and do not discriminate by number (N) between institutional events for PSI measures.
The Levy-Jennings plot will allow the user to very quickly observe the performance of selected institutions in a comparative format for each quality measure of interest. The top 25% of the chart represents performance that is "better than typical" with performance that falls in the middle 50% considered the "typical" range. The bottom 25% or "worse than typical" range represents performance that is less typical or worse than the other ranges.
XI. Interpretation of Conditions and Procedure Analysis Results
The results presented for each of the conditions and procedures will be helpful in terms of providing information on the volume of procedures, mortality rates, average length of stay and total costs with national comparative results for the designated condition or procedure. Review of this information provides insight into questions such as how long one might be in the hospital with this kind of procedure or condition. The number of procedures performed in comparison to other hospitals and an appreciation of the total cost for the designated condition or procedures. We have presented this information in a graphic format for ease of interpretation. At the bottom of each hospital graphed result we have included the Fisher z-test value and have designated with a "y" or "n" if the hospital result is statistically different from the national average value.
XII. Interpretation of Quality Measures for Recommended Care of Specific Conditions (DHHS)
Data which we collect for these quality measures is obtained from the Department of Health and Human Services. Hospitals are not required to provide this information.
The Centers for Medicare and Medicaid Services (CMS) along with other collaborators publish information relating to treatment of various conditions and the degree to which each hospital in the US conforms to such treatments. Research has shown that these treatments provide the best results for most adults with these conditions. Experts have agreed that these measures provide a view of how well hospitals provide these specific types of care.
All results are presented in graphic format for ease of interpretation. The national and state averages are also displayed for you review. The percentage represents the percent of patients that have received the designated care.
XIII. Cautions of Interpretation of results
UCHC has taken great care to make sure that we have presented the information contained in the most comprehensive and straight forward means possible. You as the user of this information must keep in mind that no single source of quality information is conclusive. We ask that you keep this in mind when you are reviewing the results. We have clearly presented this in several different areas of this site.
Please make sure that you use additional information and most importantly discuss all of your medical concerns and questions with qualified healthcare professional and most specifically a physician of your own choosing.
