Accountability Agreements in Ontario: How can we make them more

Accountability Agreements
in Ontario:
How can we make them more
effective instruments for change?
U of T HSPRN Seminar
November 19, 2008
Dr. Ben Chan, MD MPH MPA
CEO, Ontario Health Quality Council
1
OHQC Mandate – A Dual Role
An independent body, created by the
Government of Ontario to:
• Report directly to Ontarians on the state of
our publicly funded health system; and
• Support continuous quality improvement
2
Reporting on Attributes of Quality &
High Performing Health System
•
•
•
•
•
•
•
Effective
Efficient
•
Equitable
•
Accessible
Safe
Patient-centred
Focused on
population health
Integrated
Appropriately
resourced
3
4
Accountability
“By “accountable” I mean making sure that the
government and our health partners clearly agree
on what outcomes we need to achieve together.
Accountability means being answerable for our
actions, not just our good intentions. We need
clearer performance targets, greater transparency,
and better lines of communication. And let me be
clear: accountability isn’t a burden we place on
others, it’s a responsibility we all accept and share ―
and I include this government and my ministry.”
― George Smitherman, Ontario Minister of Health
and Long-Term Care in 2004
5
Timelines
Year
Key Milestones
2003
Work through JPPC to develop a multi-year funding and accountability
framework for Ontario hospitals
2004
Commitment to the Future of Medicare Act introduces accountability agmts
2005
First accountability agreements are negotiated between Ministry and hospitals
2006
Local Health System Integration Act establishes LHINs
2007
LHINs responsible for service accountability agreements with local providers
2007
Ministry-LHIN accountability agmts enter into force for 3 years
2008
LHIN-hospital 2 year agreements
2009
LHIN agreements to be introduced in CHCs; community mental health and
addiction services, community service agencies, CCACs
2010
LHIN agreements to be introduced in long-term care facilities
6
Indicator Development Process
• Indicator development “reference groups”
developed
• Identify indicator candidates from previous
reporting activities, health system scorecard,
etc.
• Apply decision tree with defined criteria
7
Selection Criteria
Primary Criteria
Secondary Criteria
• Direct measure (or
potential measure) of
Ministry strategic goal or
priority
• Construct validity
• Evidence basis
• Within hospital control
• Responsiveness to change
• Availability and
timeliness of data
• Data quality and
reliability
• Acceptability and
familiarity
8
Classification of
Hospital Indicators
• Performance indicators
• have targets and consequences if hospitals miss their targets
• Monitoring indicators
• No targets or consequences; might graduate to performance
indicators in future
• Developmental indicators
• No targets or consequences
• Data quality or methodological limitations; require further
development before useable as performance indicators
• Explanatory indicators
• provide operational information and may provide context for the
interpretation of the performance or monitoring indicators
• Not considered candidates for performance indicators
9
Classification of Indicators
• Four domains:
–
–
–
–
–
Financial
Organizational
Patient access & outcomes
System integration
Patient experience (identified by no indicators
yet)
10
Ministry-LHIN Agreements
Agreement Performance Indicators
Agreement Pilot Indicators (2007-08)
Access
• 90th %ile wait times for surgeries
(cancer, CABG, cataract, hip & knee
repl), MRI, CT
Quality
• Readmission rates for AMI
Integration
• Rate of ED visits that could be
managed elsewhere
• Hospitalization rate for ambulatory
care sensitive conditions
• Median wait time for LTC placement
• %a of alternate level of care days (no
target for 07/08)
•
•
•
•
•
•
•
•
11
Change in hospital productivity
% of chronic/complex continuing care
patients with new stage 2 or greater
ulcers
Perception of change in quality of care
In-hospital cancer deaths as a %age of
all cancer deaths
Psych readmission rates in hospitals
Time to first post-acute home care visit
Readmission rates of CCAC clients
referred by hospitals
% of individuals with multiple psych
hospitalizations in the past fiscal year
Hospital-LHIN Agreement –
Performance Indicators
FINANCIAL
• Total margin
• Current ratio
ORGANIZATIONAL
• Percentage of full-time nurses
12
Hospital-LHIN Agreement –
Performance Indicators
ACCESS & OUTCOMES
• Readmissions to own facility for specified CMGs
(AMI, stroke, COPD, CHF, pneumonia, GI,
diabetes)
• new stage 2+ skin ulcers (complex ctg care)
• Volume indicators
13
The Study
• How are accountability agreements working
in Ontario?
• Are they effective tools for promoting better
quality, system management?
• Do they reinforce other system activities
aimed at performance improvement?
14
Who We Talked To
•
•
•
•
•
•
•
Key Ministry officials
LHIN representatives
Researchers & institutes (ICES, CIHI, OHRI)
Cancer Care Ontario
Quality Improvement groups, experts
Associations (OHA, OACCAC)
Jt Policy & Planning Cmte staff
15
What is the Overall Picture of
Quality within Accountability
Agreements?
16
Missing Attributes of Quality
• Population health
• Equity
• Appropriately resourced
17
Quality – or Utilization??
• Heavy emphasis on throughput as proxy for access
• Hospital-LHIN performance indicators on volume:
–
–
–
–
–
–
–
Total (inpatient and day surgery) weighted cases
Mental health inpatient days
Elderly Capital Assistance Program rehab inpt days
Complex Ctg Care weighted pt days
Ambulatory care visits (outpatient and ED)
Emergency visits
Other volumes
18
Measuring Whole System Quality
vs Narrow Slices?
• Current accountability agreements measure
only a small component of a particular
attribute of quality
– Example: Complex Ctg Care – safety: pressure
ulcers
• Other safety issues? Med errors? Missed dx?
• Danger: divert attention from other NB
areas not being monitored
19
Ideas for “Big Dots”?
• Safety
• A measure of global hospital adverse event rates
– Trigger tools?
• Integration
– Improved measure of ALC days using objective criteria
– Measures of continuity and co-ordination between primary
care and hospitals
• Access
– Global measures of access for all surgeries
– Wait times for a broad basket outpatient and communitybased services
20
Data Quality Concerns
• lack of data
• incomplete data
• lack of standardized definitions and data
inconsistencies across sites
– e.g. ALC bed days – physician discretion in
coding
• over-reliance on administrative data
21
Data Quality
• Ideas for improvement
– More systematic assessments of data quality
– More investment in standardized tools
• e.g. for ALC days
– include data quality indicator in future
accountability agreements?
22
Data Collection Burden
•
•
•
•
too many indicators?
not clear how information will be used
lack of dedicated resources for data collection
too much emphasis on reporting, not enough on
quality improvement
• multiple, uncoordinated reporting requirements for:
–
–
–
–
Accreditation, accountability agreements, CCO,
wait times strategy, emerg department reporting,
trauma hospital reporting, radiation therapy,
LHIN growth funding
23
Targets & Corridors
• Targets for performance indicators
– negotiated between each hospital and LHIN
– Accounts for hospital’s past performance and the
hospital’s capacity to manage risk
• Corridors:
– set for each performance indicator
– typically ± 2.5 and 3 standard deviations from the target
• Example: corridor for 30-day readmission rates for specified
case mix groups is the target plus three times the standard
deviation of that number
• Performance met if target missed but within corridor
24
The Goal … in Industry
25
Approaches to Quality
• Six sigma
– Relentlessly shrink your variation in processes
– keep your defects to 3.4 per million opportunities
• Ontario’s accountability agreements
– Huge variation is tolerated
– Performance at the lowest 1% is a pass
26
What If You Miss the Target?
• For financial indicators
– Intense scrutiny at multiple levels
– Precedents of trustees appointed for
management
• Wait times strategy
– Financial penalties
• For quality indicators
– “Discussion with the Board”
27
Indicator Cascade
• What is strategy for moving system-wide
indicators of performance?
• What needs to happen at meso, micro
levels of system to get macro level change?
• Is there a clear “line of sight” or “chain of
accountability” between leaders’ goals and
front-line staff goals?
• Does measurement system support this top
to bottom strategy?
28
Hypothetical Indicator Cascade
29
Accountability Agreements:
Are They Aligned Today?
30
Challenges for Future Design
• Map accountabilities in one sector which
affect another
• Address shared accountabilities
– Primary care, specialty clinic, cmty services
impact chronic dis mgt & hospitalizations
• How to handle accountabilities to primary
care for downstream impacts
– E.g. primary care access => ED visits
31
Alignment with QI Campaigns
• Existing campaigns:
– Wait Times Strategy
• ↓ waits for cancer, caaract, hip & knee surgery, CT, MRI
– Safer Healthcare Now
• Medication reconciliation, infections (surgical site, central
line, ventilator associated pneumonia, improve AMI care,
rapid response teams (↓ preventable codes / deaths)
– FLO Collaborative
• Improve patient flow from hospital to cmty, LTC, home care
– Quality Improvement & Innovation Partnership
• Improve diabetes, colorectal screening, access in FHTs
32
Alignment with QI Campaigns
• Indicators, targets to support wait times
strategy
• FLO Collaborative
– Little reflection in accountability agreements
• ALC in partial use in some LHINs; data quality concerns
• Safer Healthcare Now
– No accountability indicators to reflect campaign
• AMI readmits relate to post-discharge, not care in hosp
33
Opportunities to Reinforce
QI Initiatives
• Track similar indicators in initiative & AAs?
• Set targets for improvement in AAs, to reflect
targets in QI projects?
• Set accountabilities for participation or
active engagement?
34
Public Reporting Landscape
35
Public Reporting Activities
• Not well aligned with accountability
agreements
– (with exception of wait times strategy)
– Information on actual performance results in AAs
not available nor not easily accessible to public
– Some inconsistent definitions
• e.g. hospital readmissions
– Public reporting on many issues not covered in
AAs
36
Public Reporting - Ideas
• More centralized reporting
• More transparency of accountability
agreement indicator data to public
• Better coordination of reporting efforts
37
Key Points
• Problems with alignment of indicators at
different levels, QI, public reporting, AA’s
• Stronger mechanisms of accountability for
financial vs quality indicators
• Indicators capture “slices”, not big picture
– Driven by availability of data
38
Aim for Strategic Alignment of
Indicators
•
•
•
•
•
Public reporting
Corporate dashboards of organizations
Accountability agreements
Accreditation
Major QI initiatives
– Safer Healthcare Now, FLO Collaborative, QIIP
Family Health Team/CHC Initiative
39
Contact Us
• Email: [email protected]
• Website:www.ohqc.ca
– Download 2008 Report and Summary
– Accountability Agreements white paper
– Free quality improvement tools & resources
40
`