The KPI.org Blog

Gail Stout Perry Gail Stout Perry

Gail is co-author of The Institute Way with over 20 years of strategic planning and performance management consulting experience with corporate, nonprofit, and government organizations.

PS: Our Balanced Scorecard Saved The U.S. Army $26 Million

By Gail Stout Perry

Sep 30, 2013 9235 Views 0 Comments FacebookTwitterLinkedInGoogle Plus

I was working with an Army command at Ft. Sam Houston this week and had invited a special guest - Scott Hencshel - to address the group regarding the organizational challenges of implementing a balanced scorecard system within Army.  (Scott’s command is also stationed at Ft. Sam Houston -  Army Medical Department Center & School (AMEDDC&S), an Institute “Award for Excellence” winner.)  

As Scott was wrapping up, someone asked a final question, “What was the biggest benefit that AMEDDC&S realized after implementing its strategic balanced scorecard?”  Scott talked about alignment, focus, and data-driven decision making.  Then as he was making his way to the door he turned back and said, “Oh yeah, we immediately saved the Army $26 million.” 

Say what?!?!

AMEDDC&S is where the U.S. Army educates and trains all of its medical personnel – over 27,000 soldiers. One of the strategic measures on AMEDD’s balanced scorecard is “attrition rates.”  Before the scorecard was implemented, it was commonly believed that discipline issues were the primary reason for soldiers not completing their training programs – because resolution of these discipline issues were what consumed everyone’s time.  Once the scorecard was implemented, attrition was measured more thoroughly and two discoveries were made:

  1. Attrition was MUCH higher than originally thought.  The traditional calculation was flawed and attrition was actually over 34%.  That means 1/3 of those entering the medical training programs would “drop-out” thereby wasting the Army’s investment in their training.
  2. Academic performance, not discipline, was discovered to be the primary reason for attrition.

So as the scorecard team delved further, they looked for root causes of poor academic performance resulting in attrition incidents.  They discovered that a major cause was a lack of communication between the Brigade leadership and the AMEDDC&S faculty.  Students in the medical training program were being assigned Brigade duties that prevented them from having proper opportunities to study and prepare for classes and exams.   A prime example was students falling asleep during final exams due to having served Brigade guard duty the night before. 

Once the communications issues were corrected, overall attrition rapidly dropped from 34% to below 20%...thereby saving the U.S. Army $26 million.

PS:  Did I mention that I have the best job in the world?!?  It is extremely rewarding to hear about results like this.

For more examples of break-through performance, we invite you to read “The Institute Way: Simply Strategic Planning & Management with the Balanced Scorecard. 

Stacey Barr Stacey Barr

Stacey Barr is a specialist in organizational performance measurement and creator of PuMP, the refreshingly practical, step-by-step performance measurement methodology designed to overcome people’s biggest struggles with KPIs and measures.

3 Essential Signals to Look for in Your KPIs

By: Stacey Barr

Sep 3, 2013 1435 Views 0 Comments FacebookTwitterLinkedInGoogle Plus

You won’t find true signals of changes in performance by looking at month-to-month comparisons, or trendlines, or moving averages. The signals you really need to know about, the only signals that you ought to respond to, are revealed through one particular graph only.

The typical analysis methods we use for our performance measures are based on assumptions that don’t make much sense.

Month-to-month comparisons assume that there is no routine variation over time and wrongly interpret any difference as a signal.

Trendlines assume that all change is linear and gradual, and that if Excel can calculate a trend line then there must be a trend.

Moving averages assume that seasonal patterns exist, and also that change is smooth and gradual.

The only analysis technique I have ever come across that clearly filters the noise and highlights the signals in our performance measures is the XmR chart.

The XmR chart filters the noisy routine variation in our measures by showing us how much of this routine variation there is, by way of the Natural Process Limits. And coupled with the Central Line, these Natural Process Limits give us a meaningful baseline to quickly assess when performance has changed; when there is something else going on that’s not part of the routine variation.

There are three very specific signals to look for.

SIGNAL 1: Outlier or special cause

When a measure value falls outside the Natural Process Limits, it means that more than just the routine variation is at play. It’s a signal that something else has happened.

If Employee Attendance plummeted below the bottom Natural Process Limit, a likely cause could be a flu epidemic or local natural disaster that kept many more people away from work in that period.

Even though this is a signal that something out of the norm has happened, because it’s just a one-time event, we don’t react to it. We find out what caused it, but we don’t run around madly trying to fix it. That would be a massive waste of time and money because we’d essentially by trying to fix something that won’t happen again, or that is completely outside our control.

SIGNAL 2: Long run

To be convinced that a change in the level of performance has happened, we need to see seven (yes, seven) points in a row on the same side of the Central Line. The probability that a pattern like that is part of routine variation is close to zero (0.78%, to be precise). Seven points, not three or five or one.

If a measure of Invoice Accuracy showed a long run above the Central Line, it might be evidence that an initiative to simplify the pricing strategy had successfully reduced the errors in invoices.

When we see a long run signal in our measure, we certainly need to find the cause for it. Sometimes it will be a signal of improvement, and we want to confirm what caused the improvement. Other times it will be a signal that performance has deteriorated and the cause of that is very important to identify!

SIGNAL 3: Short run

You’re no doubt thinking to yourself ‘I can’t wait for seven months before I can know if I should take action!’ You can either measure more frequently to pick up signals sooner (as long as it makes sense to), or plan for bigger signals.

A bigger signal appears as a short run, of three out of four consecutive measure values closer to a Natural Process Limit than they are to the Central Line. The probability of this pattern happening also has a very close to zero probability.

A short run above the Central Line for On-time Deliveries for a trucking company would likely be due to an initiative that had a substantially large impact. It could be something like doubling the fleet size. But it also could be a new competitor in the market that poached a large percentage of their customers.

Again, with a signal like the short run, it’s really important to find the cause before responding.

XmR charts take only a little effort to create, but their usefulness is so powerful they are absolutely worth trying.

Stacey Barr is a specialist in organisational performance measurement and creator of PuMP, the refreshingly practical, step-by-step performance measurement methodology designed to overcome people’s biggest struggles with KPIs and measures. Learn about the bad habits that cause these struggles, and how to stop them, by taking Stacey’s free online course “The 10 Secrets to KPI Success” at www.staceybarr.com/the10secretstokpisuccess.