0:05
In this video, we will be reviewing how to
use data and measures to evaluate success of an improvement effort.
My name is Julia Kim and I'm a faculty member at
the Johns Hopkins University School of Medicine in
Pediatrics and the Armstrong Institute for Patient Safety and Quality.
I serve as the Associate Vice Chair of Ambulatory Quality and Safety and the
Co-Director of the Armstrong Institute Patient Safety and Quality Leadership Academy.
I have worked on both outpatient and inpatient quality improvement projects.
When discussing data and measures,
it's always good to go back to definitions.
What is quality improvement?
The Health Resources and Services Administration defines
quality improvement as systematic and continuous actions that lead
to measurable improvement in health care services and
measurable improvement in the health status of targeted patient groups.
So, we can see that measurement is the foundation and the basis for quality improvement.
There are several frameworks for quality improvement.
One, is the Lean Sigma framework,
where you define a problem,
measure, analyze, improve and then control and sustain your improvement.
There's also the Institute for Healthcare Improvement,
with the three big questions addressing the aim,
measures, and ideas for change and
then the plan to study act cycles for continuous improvement.
There's also the translating evidence into practice or TRIP model,
where one summarizes the evidence,
identifies local barriers and implementation,
measure performance and ensures that all patients receive interventions.
In each of these frameworks and other frameworks as well,
you can see that measurement plays an important role,
an important step in your approach to quality improvement.
In defining your problem and in defining your aim for your quality improvement project,
it is always good to consider the Institute of Medicine Aims for Improvement,
as outlined in the 2001 Crossing the Quality Chasm report,
that outlined the aims for Healthcare Improvement to be safe,
effective, patient-centered, timely, efficient, and equitable.
Similarly, it's also helpful to think about
your aim and couch it as a smart aim and for it to be specific,
measurable, achievable, realistic, and relevant, and timely.
3:09
So when we get to the measurement aspect and we ask ourselves a question,
how will we know that a change is an improvement?
It boils down to measurement.
Your key measure should be related to your aim.
Also, measurement over time is an essential aspect to quality improvement and you want to
ensure that your measure is feasible to
collect and that's able to show improvement quickly.
One tip is to select
multiple measures and to
choose different types of measures to accurately evaluate your changes.
Three types of measures include outcomes measures,
which are your clinical outcomes or the outcomes that reflect changes in the system.
There's also process measures,
which reflect the steps that need to be taken to achieve the outcome measures,
and these are often identified as some of
the key drivers to achieving your clinical or systems-based outcomes.
There are also balancing measures or
those measures that are not directly related to your quality improvement project,
but will reflect whether or not there are
unintended consequences of your interventions or ideas for change.
And ideally, you'd use a balanced set of measures for all improvement efforts.
4:53
When you think about selecting your measure,
you want to be able to define your measure to be able to plot it on a graph.
You want to be able to plot the measures on the graph to
show the effect of the changes you are testing.
You want to be able to select an appropriate metric and think about,
is it best to define your measure as a ratio or a percentage, and ideally,
you would be explicit with the numerator and denominator of your measure,
or is it best to track your measure as a rate versus account?
And if it's an event that you're trying to track that may not be as common,
you can also consider tracking the time between events.
5:51
Another tip would be to collect data as efficiently as possible.
And if resources are not available to collect
your measures on a daily basis for all patients,
then you can use sampling strategies.
For example, to collect data on 25 patients that present
during a week or to collect data for a week out of the month.
Also, you want to be able to plot the data often enough to
be able to provide your teams with timely feedback using the graphs.
This can often help to motivate the teams to continue with the improvement project.
And in thinking about data collection and how to present the data,
you can think about whether this is going to be monthly data,
weekly data or daily data and often
by collating the data monthly and presenting it in monthly graphs.
This can minimize the variability of day to day results.
Another tip would be to try to integrate the data collection into the daily workflow of
the staff to minimize
the burdens of collecting data for your quality improvement project.
And if resources are available,
think about incorporating automated data reports from an electronic health records system
or to include the information technologists on your quality improvement team.
One of the key aspects of measurement is to measure data over time,
and there are a couple of ways that the data can be presented.
And one of the most common and effective ways to
demonstrate that data over time is through a run chart or a time series graph.
And in this example of the run chart,
we can see time along
the horizontal X axis and we can see that your measurement, your metric,
would be along the Y vertical axis,
potentially as a percentage,
as in this example and you would collect baseline data.
And in this example,
there are 10 data points and you want to have
at least 10 data points in your baseline data.
And then, you would look at the baseline median and extend that line going forward.
And then, you would then track your interventions or ideas for change.
And you can see an intervention one demonstrated,
that it started right after Time point 10.
And then, you would continue to plot your time points and then
you would demonstrate when a second intervention potentially was implemented.
And you would also note your goal line,
and in this case, the goal was 70%.
So, there's a lot of information that can be conveyed through this run chart.
9:30
When interpreting the results of your run chart,
you want to look for several different patterns in the data.
And so, you can look for shifts in the data where there are
at least six points above or below the center median line.
You can try to look for astronomical data,
or points that are so far off compared to the other data points,
or you can look for trends in the data,
where there are at least five consecutive points that either run upwards or downwards.
So, these are some examples of signals for an effective change process taking place.
So another way to demonstrate measurement of
data over time is through the use of control charts.
Similarly, we can see the key metric plotted over time.
The difference with control charts are that they
include upper control limits and lower control limits.
And these control limits indicate three sigma from the mean.
And they help us to differentiate between
random common cause variation of a data versus special cause variations.
So, there are also special cause variation
tests that we can use to try to determine whether
or not we are impacting and having effective change.
And so, some key differences between the run chart and the control chart
are that the run chart plots the median while the control chart plots the mean.
The run chart does not include control limits while the control chart includes +/-
3 sigma upper control and
lower control limits which can help us
distinguish between common cause versus special cause variation.
The run chart requires less data points,
at least 10 baseline data points,
control chart requires more.
And we have signals in both run charts.
We can identify shifts, trends,
runs or astronomical points on the run chart or through the control charts,
we can also identify shifts,
trends, cycles or outliers.
And here are some references that you can refer to find out
some more information about how to use
data and measurement in your quality improvement process.
Matt AustinAssistant Professor
