Post Image

Did we change the system yet: Using control charts

Motivation

Have you ever asked yourself “How do we know our Kanban system really improved”,  or “How can we see which impact experiments have on our system”?

Control charts (XmR charts) offer a relatively simple way to analyze your system’s behavior and distinguish noise (normal variation within the natural boundaries of that complex system) from signals (significant changes, indicating the system is “out of control” in a good or bad way). They can help you working at the right points in your Kanban system.

There’s more good news: Generating these charts is easy and does only require basic mathematical skills. Advanced interpretation requires some skills but visual clues are quite clean and helpful from the very beginning. Of course this does not necessarily mean that tool vendors have  included it in their reporting modules 😉

Introduction

Have a look at the sample below. It shows an XmR chart of lead times of user stories in consecutive order over a period of 9 months from a software development team working with Kanban.

Xmr charts consist of two parts. The upper X area plots the time series of the values in question (e.g. lead time or throughput, failure load). [Don’t forget to sort the items by their finish date.] It contains a central line (average), as well as the Upper and Lower Natural Process Limits (NPLs).

Upper / lower Natural Process limits = Average X ± (2,66 * average mR)

The lower moving range (mR) chart shows the absolute value of the difference of each value with its predecessor, i.e. the variation of the X values. This chart also contains the average of these mR values as central line as well as an Upper Control Limit (UCL).

Upper Control Limit = 3,27 * average of mR

If the values stay within these limits it means the process is “in control”. Thus it is behaving within its natural process limits. It is important to understand that this is the voice of the process, not how we or customers expect it to behave!

All data points outside the limit as well as certain patterns (e.g. 5 consecutive values at one side of the central line) are signals that are worth exploring! The charts help you focus on the right data instead of wasting efforts with “noise”.

Working with the data

To get back to the sample data: In this case, improvements were initiated around week 15/16 in order to make lead times predictable and improve flow. From looking at the charts , would you say these improvements had a significant effect? We certainly see that from around week 20, the values are generally lower and stay well within the Natural Process Limits.

So we want to analyze this further! For that, we recompute central lines and process / control limits with a new base, only including values of the weeks 20 to 41 (see the little gap in the charts at week 20). The X and mR values stay the same. See the resulting chart below.  Rebased values are shown as thick lines, the original ones as thin lines:

Average X (lead time) has dropped from 9 to 7 days. The Upper Natural Process Limit is down from 28 to 22 days. Average moving range has also improved from 7 to 6 days, the Upper Control limit is down from 24 to 19 days, with 2 values slightly out of this range. This supports our hypothesis that the system changed — average lead times decreased, with an increased predictability.

If we zoom into weeks 20-41 we can see that the lead times produced by the process operate within the reduced natural process limits after the rebase.


In the right area of the chart we can see that variations are increasing (limits in mR chart broken) and also the X values (lead times) moving towards the upper natural process limit. This is an indicator that the system is about to get out of the established control limits and points to further analysis.

Health warning

Always have your brain in operating mode “on”. This is generally a good idea when you work with data. Be conscious about the source and quality of the data. Work with hypotheses and use the data analysis as an additional source of information, not a substitute.

Reading more

Still interested? Then I recommend reading the book “Understanding variation – the key to managing chaos” by Donald J. Wheeler [1]. It is a surprisingly quick and easy read given the topic;)  and you might have a few aha! moments.

If you’d like to have a look at the data and simple calculations, see the Google spreadsheet embedded below. For reading convenience, a number of rows has been hidden.

For a summary of the formulas you can also refer to “Shewart Individuals Control Chart” at Wikipedia.

Feel free to post questions or comments below. Thanks for your feedback!

References

[1] Wheeler, Donald J, (1993) Understanding Variation The Key to Managing Chaos. Knoxville, Tennessee, SPC Press, Inc.

Leave a Reply

Your email address will not be published. Required fields are marked *

For a better user experience, please switch your device to portrait mode.