Day 2: Session II Presenter: Jeffrey Geppert, Battelle AHRQ QI User Meeting September 26-27, 2005 Methods for Creating Aggregate Performance Indices AHRQ QI User Meeting September 27, 2005 Jeffrey Geppert Battelle Health and Life Sciences Overview
Project objectives Why composite measures? Who might use composite measures? Alternative approaches Desirable features of a composite Proposed approach for the AHRQ QI Questions & Answers Project Objectives Composite measures for the AHRQ QI included in the National Healthcare Quality Report and Disparities Report Separate composites for overall quality and/or quality within certain domains (e.g., cardiac care, surgery, avoidable
hospitalizations, diabetes, adverse events) A methodology that can be used at the national, state and provider/area level Project Objectives Feedback Does the proposed approach meet user needs for a composite? What analytic uses should the composite address? What are the important policy issues? How should the composite be incorporated into the AHRQ QI software? Goals of
National Healthcare Reports National Level Provide assessment of quality and disparities Provide baselines to track progress Identify information gaps Emphasize interdependence of quality and disparities Promote awareness and change State / Local / Provider Level
Provide tools for self-assessment Provide national benchmarks Promote awareness and change Unique challenges to quality reporting by states States release comparative quality information in a political environment Either must adopt defensible scientific methodology or make conservative assumptions Examples of reporting decisions: Small numbers issues Interpretive issues (better/worse, higher/lower) Purchasers demanding outcomes and cost information from states
Why Composites? Summarize quality across multiple measures Improve ability to detect quality differences Identify important domains and drivers of quality Prioritize action Make current decisions about future (unknown) healthcare needs Avoids cognitive short-cuts
Why Not Composites? Mask important differences and relationships among components (e.g. mortality and re-admissions) Not actionable Difficult to identify which parts of the healthcare system contribute most to quality Detract from the impact and credibility of reports Independence of components
Interpretation of components Who Might Use Them? Consumers To select a hospital either before or after a health event Providers To identify the domains and drivers of quality Purchasers To select hospitals in order to improve the health of employees Policymakers To set policy in order to improve the health of a population Examples Americas Best Hospitals (U.S. News & World
Report) Leapfrog Safe Practices Score (27 procedures to reduce preventable medical mistakes) NCQA, Americas Best Health Plans QA Tools (RAND) Veteran Health Administration (Chronic Disease Care Index, Prevention Index, Palliative Care Index) Joint Commission (heart attack, heart failure, pneumonia, pregnancy) National Health Service (UK) Performance Ratings CMS Hospital Quality Incentive Demonstration Project Examples Composite
Components Utility Measures Goals Alternative Approaches Approach Goal Utility Opportunity Appropriate care
Volume of opportunities Burden Minimize excess death/costs Measures with most excess Expected quality Better than reference Lowest ratio Variation
Better than reference Outliers Latent quality Reduce variation Measures with greatest variation Desirable Features Valid - Based on valid measures Reliable Improve ability to detect differences Minimum Bias Based on unbiased measures Actionable Interpretable metric Benchmarks or standards Transparent Predictive Should guide the decision-maker
on likely future quality based on current information. Representative Should reflect expected outcomes for population Proposed Approach A modeling-based approach Latent quality observed correlation in individual measures is induced by variability in latent quality Individual measures with highest degree of variation have larger contribution to
composite Theoretical interpretation Consistent with goal of reducing overall variation in quality Proposed Approach Component Component Component Component Latent Quality
Advantages Avoids contradictory results with individual measures or the creation of composites that may mislead Construction of the composite increases the power of quality appraisals Allows for both measure-specific estimates and composites Allows for validation with out-of-sample prediction Advantages (Continued) Hierarchical for small numbers, the best estimate is the pooled average rate at similar hospitals Allows for incorporation of provider characteristics to explain betweenprovider variability (e.g., volume,
technology, teaching status) Gives policymakers information on the important drivers of quality Overview of AHRQ QIs Prevention Quality Indicators Inpatient Quality Indicators Patient Safety Indicators Ambulatory care sensitive
conditions Mortality following procedures Mortality for medical conditions Utilization of procedures Volume of procedures Post-operative complications Iatrogenic conditions Examples IQI Surgical Mortality 0.14
0.012 POSTOP RESPIRATORY FAILURE 0.010 POSTOPERATIVE PE OR DVT 0.008 POSTOPERATIVE SEPSIS 0.006 0.004 0.002 0.000 High
Medium Late nt Quality Low Examples PSI Technical Adverse Events 0.160 0.140 0.120 0.100 DECUBITUS ULCER 0.080 FAILURE TO RESCUE INFECTION DUE TO MEDICAL CARE
0.060 0.040 0.020 0.000 High Medium Late nt Quality Low Examples PSI Technical Difficulty 0.005 0.004
0.004 0.003 IATROGENIC PNEUMOTHORAX 0.003 POSTOP HEMORRHAGE OR HEMATOMA 0.002 ACCIDENTAL PUNCTURE/LACERATION 0.002 0.001 0.001 0.000 High Medium Latent Quality
Low Hierarchical Models Also referred to as smoothed rates or reliability-adjusted rates Endorsed by NQF for outcome measures Methods to separate the within and between provider level variation (random vs. systematic) Total variation = Within provider + Between provider (Between = Total Within) Reliability (w) = Between / Total Signal ratio = signal / (signal+noise) Hierarchical Models Smoothed rate is the (theoretical) best predictor of future quality
Provides a framework for validation and forecasting Smoothed rate (single provider, single indicator) = Hospital-type rate * (1 w) + Hospital-specific rate * w Multivariate versions Other Years (auto-regression, forecasting) Other Measures (composites) Non-persistent innovations (contemporaneous, nonsystematic shocks) Outcomes and Process Source: Landrum et. al. Analytic Methods for Constructing Cross-Sectional Profiles of Health Care Providers (2000) Hierarchical Models Quality Patient
Provider Provider Type Policy Policy and Prediction The best predictor of future performance is often historical performance + structure The greater the reliability of the measure for a particular provider, the more weight on historical performance
The less the reliability of the measure for a particular provider, the more weight on structure Volume often improves the ability to predict performance for low-volume providers Other provider characteristics (e.g. availability of technology) do as well Area characteristics (e.g., SES) do as well Socio-Economic Status The Public Health Disparities Geo-coding Project Harvard School of Public Health (PI: Nancy Krieger) Evaluated alternative indices of SES (e.g. Townsend and Carstairs) Occupational class, income, poverty, wealth, education level, crowding
Gradations in mortality, disease incidence, LBW, injuries, TB, STD Percent of persons living below the U.S. poverty line Most attuned to capturing economic depravation Meaningful across regions and over time Easily understood and readily interpretable Socio-Economic Status PQI #1 Diabetes Short-term Complication 0.0016 0.0014 0.0012 0.0010 0.0008
0.0006 0.0004 0.0002 0.0000 1 2 3 4 5 6
% Below U.S. Pove rty De cile 7 8 9 10 Limitations Measures and methods difficult Restrictive assumptions on correlation Correlations may vary by provider type
Requires a large, centralized data source Expansions Flexibility in weighting the components Empirical domains driven entirely by empirical relationships in the data A priori domains determined by clinical or other considerations Combination empirical when the relationships are strong and the measures precise, otherwise a priori Welfare-driven Composites Meta-composites Composites
Measures Welfare-Driven Composites Making current decisions about future needs maximize expected outcomes, minimize expected costs Policymaker focus for a population A provider focus for their patients A employer focus for their employees A consumer focus based on individual characteristics
Acknowledgments Funded by AHRQ Support for Quality Indicators II (Contract No. 290-04-0020) Mamatha Pancholi, AHRQ Project Officer Marybeth Farquhar, AHRQ QI Senior Advisor Mark Gritz and Jeffrey Geppert, Project Directors, Battelle Health and Life Sciences Data used for analyses: Nationwide Inpatient Sample (NIS), 1995-2000. Healthcare Cost and Utilization Project (HCUP), Agency for Healthcare Research and Quality State Inpatient Databases (SID), 1997-2002 (36 states). Healthcare Cost and Utilization Project (HCUP), Agency for Healthcare Research and Quality Acknowledgements We gratefully acknowledge the data organizations in participating states that contributed data to
HCUP and that we used in this study: the Arizona Department of Health Services; California Office of Statewide Health Planning & Development; Colorado Health & Hospital Association; Connecticut Chime, Inc.; Florida Agency for Health Care Administration; Georgia: An Association of Hospitals & Health Systems; Hawaii Health Information Corporation; Illinois Health Care Cost Containment Council; Iowa Hospital Association; Kansas Hospital Association; Kentucky Department for Public Health; Maine Health Data Organization; Maryland Health Services Cost Review; Massachusetts Division of Health Care Finance and Policy; Michigan Health & Hospital Association; Minnesota Hospital Association; Missouri Hospital Industry Data Institute; Nebraska Hospital Association; Nevada Department of Human Resources; New Jersey Department of Health & Senior Services; New York State Department of Health; North Carolina Department of Health and Human Services; Ohio Hospital Association; Oregon Association of Hospitals & Health Systems; Pennsylvania Health Care Cost Containment Council; Rhode Island Department of Health; South Carolina State Budget & Control Board; South Dakota Association of Healthcare Organizations; Tennessee Hospital Association; Texas Health Care Information Council; Utah Department of Health; Vermont Association of Hospitals and Health Systems; Virginia Health Information; Washington State Department of Health; West Virginia Health Care Authority; Wisconsin Department of Health & Family Services. Questions & Answers Questions And Answers
Todd Poland - Top Rail Solutions Inc. Andy Sutherland - Wells Fargo Rail Corp. SAMPLE DDCT NOTIFICATION E-MAIL TO MARK OWNER: ACACSO Spring Meeting May 2018. ACACSO Spring Meeting May 2018. SAMPLE AAR RULE 1 DDCT INCIDENT:
The South Cardiff and Vale Crisis Resolution And Home Treatment Team Jayne Bell Team Leader 029 20 906222 Where we are Based in the Hamadryad Centre, co-located with the South West CMHT Approximately geographically central to our sector (traffic allowing!)
France obtient Nice et la Savoie (Armistice de Cherasco, 1796) Paix de Tolentino Les Républiques sœurs République Cispadana (Modène et Reggio Emilia) République Cisalpine (ajout de la Lombardie) Traité Campoformio (1797) République Ligure République Helvétique et la République romaine La...
Introduction. In this paper, we investigate the implications of integrating DNA in the DBMS storage hierarchy. We present OligoArchive, an architecture for using a DNA-based storage system as the archival tier of a relational DBMS.
Podemos llegar a afirmar que la experiencia de ser un hijo amado de Dios, es el rasgo primordial que configura al discípulo de Jesús. No puede haber discípulo que no viva la relación filial con Dios, su padre. Y, naturalmente,...
, 933 F.2d 1576, 1584 (11th Cir. 1991) (compensatory education even when student is aging out) Board of Educ. Of Oak Park v. Illinois State Bd. of Educ., 79, F.3d 654,,660 (7th Cir. 1996) (compensatory education is a benefit that...
Ready to download the document? Go ahead and hit continue!