X

The KPI.org Blog

Stacey Barr Stacey Barr

Stacey Barr is a specialist in organizational performance measurement and creator of PuMP, the refreshingly practical, step-by-step performance measurement methodology designed to overcome people’s biggest struggles with KPIs and measures.

Creating Authentic Urgency for Better KPIs

By: Stacey Barr

Oct 10, 2017 949 Views 0 Comments FacebookTwitterLinkedInGoogle Plus

Complacency is a big reason why useless KPIs stay. A sense of urgency is what’s needed to spark the change to better KPIs. But it must be authentic, and speak to the head and the heart.

It’s not true that we want better KPIs just to improve the bottom line. But it can be quite hard to get managers and leaders to take better KPIs seriously, and adopt a KPI approach that truly works. So, linking to financial performance might just be the angle you need to take to put your KPI case forward.

Putting a number to the hidden costs of poor KPIs, and the lost opportunities of better KPIs, can create what John Kotter would call creating a sense of urgency.

But, we can’t be all head and no heart. It can’t be just about the cost of our current poor KPIs. It’s about how we feel about those poor KPIs too.

A manager of freight business, Marty, asked me to help him sort through his KPI mess. He told me he had too many KPIs and no information. We did a stocktake of what he actually had.

There were over 300 KPIs, many of which were different versions of the same measures. There were over 50 performance reports produced. And we estimated – very conservatively, mind you – that this was costing Marty over 1200 hours of effort every month. If the average hourly wage for a business analyst was $70, this is just over $1,000,000 a year.

More than $1,000,000 a year for information that’s neither useful nor usable is appalling.

Marty joked with me that he’d print a hard copy of every one of those 50 reports, and push them into the next management meeting, piled up in a wheelbarrow.

I wish he had done that. It would have been just the perfect way to bring the problem to life in both the heads and hearts of his management team. And much easier to create the sense of urgency that could spark a much-needed change. As Kotter says:

“Mindless emotion is not the point. Generally, the challenge is to fold a rational case directed toward the mind into an experience that is very much aimed at the heart.”

Have you ever estimated the cost you’re paying for your current KPIs?

If you are committed to the idea of better performance measurement, and you know that what your organization is doing now is wasteful and useless, there is something you can do.

Take a sample of areas of your organization. Maybe a sample of reporting analysts, or a sample of business units. And talk to them to get some rough but realistic numbers to estimate costs like these:

  • The cost of collecting data that’s not needed.
  • The cost of reporting information that’s not used.
  • The cost of making the wrong decisions with bad KPIs.
  • The cost of the time spent debating and reworking KPIs.
  • The cost of the missed opportunity to improve performance.

And why not try Marty’s wheelbarrow idea? March right in there with physical evidence of the problem and let people react to it. Let them feel it; the awkwardness, the shock, the realization. The undeniable need for change.

Forget about writing a detailed business case for a better KPI approach. Forget about educating people on what a good approach is. Forget about trying to retrofit tiny little ideas from a good approach to make micro improvements in your existing one. Forget about coercing, cajoling and compelling. These are a waste of time.

If you want to create an authentic sense of urgency, you need to be fact-based and visceral. And probably a bit daring and provocative. You need to be a bit of a story-teller. You need to feel the urgency, yourself. But not be beaten down by it: you must see a vision of life on the other side of it. And see it clearly enough that you can show it to others.

An authentic sense of urgency can unseat complacency toward KPIs, but it must be both fact-based and visceral.
[tweet this]


Stacey Barr is a specialist in organisational performance measurement and creator of PuMP, the refreshingly practical, step-by-step performance measurement methodology designed to overcome people’s biggest struggles with KPIs and measures. Learn about the bad habits that cause these struggles, and how to stop them, by taking Stacey’s free online course “The 10 Secrets to KPI Success” at www.staceybarr.com/the10secretstokpisuccess.

Stacey Barr Stacey Barr

Stacey Barr is a specialist in organizational performance measurement and creator of PuMP, the refreshingly practical, step-by-step performance measurement methodology designed to overcome people’s biggest struggles with KPIs and measures.

Just Because Some KPIs Are Bad Doesn’t Make KPIs Wrong

By: Stacey Barr

Oct 3, 2017 762 Views 0 Comments FacebookTwitterLinkedInGoogle Plus

KPIs have a bad reputation, from bad pasts experiences people have with them. But we can’t let their reputation become the truth about them.

Stephen works in the public sector in the UK, and was struggling with how to change the culture in his organisation, which is awash with unhelpful KPIs. No doubt there is a lot of cynicism and avoidance of measurement there.

And not nearly enough mastery.

Anders Ericsson’s Deliberate Practice research inspired Malcolm Gladwell’s proposition that 10,000 hours of practice brings mastery. But it isn’t quite that simple:

  • Firstly, the amount of practice can lead to mastery in a field where the rules and approach are clear and stable. Like carpentry, but not entrepreneurship.
  • Secondly, the quality of practice matters much more than the quantity. It must be regular and deliberate, not sporadic and distracted.

KPIs are more like carpentry than we might think.

There is a clear approach, and even some rules, that guide a logical and practical series of steps to create good KPIs. And the quality with which those steps are practiced does deepen the know-how and skill to create even better KPIs.

When I teach people the PuMP Measure Design technique (step 3 of PuMP’s 8 steps), we practice using a case study. The participants work in smaller groups of 5 or 6 people. Do you think they’d each end up with different measures for the same case study goals?

Well, over several years of using the same case study, with many hundreds of people, the amount of variation in the KPIs they design is incredibly small. A logical and practical series of steps creates good KPIs. And then the more that people practice Measure Design in their own work context (and we observe this on the 3rd immersion day of the PuMP training, as well as in private facilitation sessions), they get even faster at creating even better KPIs.

We can’t let the reputation of KPIs become the truth about them.

We need to lift the conversation above the current KPIs, and focus on the approach that produced them, and how well we implement it.

We’re too easily tempted to ask which KPIs to cull, where to find better KPIs, what to do to get people to own the KPIs. But there are some questions that are more important to ask:

1.    Do we have a KPI methodology? A REAL one?

2.    Are we following that methodology? All of it?

3.    Are we giving ourselves the time to learn it, practice it, and master it? Before judging it?

4.    Are we engaging more and more people in exploring and learning it? And rolling it out on the waves of that engagement?

Great KPIs only come when we master the method for creating them. But too often we don’t bother thinking about the method. We don’t have the discipline to use it. We underestimate what it takes to implement it properly. We push and rush to get KPIs out of the way, as quickly as possible.

Will you change that?

The reputation of KPIs is not the truth about them. Bad KPIs come from bad approaches and poor practice.
[tweet this]

Stacey Barr is a specialist in organisational performance measurement and creator of PuMP, the refreshingly practical, step-by-step performance measurement methodology designed to overcome people’s biggest struggles with KPIs and measures. Learn about the bad habits that cause these struggles, and how to stop them, by taking Stacey’s free online course “The 10 Secrets to KPI Success” at www.staceybarr.com/the10secretstokpisuccess.
Stacey Barr Stacey Barr

Stacey Barr is a specialist in organizational performance measurement and creator of PuMP, the refreshingly practical, step-by-step performance measurement methodology designed to overcome people’s biggest struggles with KPIs and measures.

How Do You Measure The Impact Of An Initiative?

By: Stacey Barr

Sep 5, 2017 867 Views 0 Comments FacebookTwitterLinkedInGoogle Plus

Let’s get right to the point: if you have to ask how to measure the impact of an initiative, it means you got things back to front.

Asking “how do you measure the impact of an initiative” means you don’t already know what the measure is. Obviously. But it can also mean you don’t even know what the impact should be.

And that means you chose the initiative before you fully understood what you needed it for. Stay on this path, and you’ll end up wasting a lot of time, money, and angst.

What was the goal that needed the initiative?

Sure, you might have had a goal the initiative related to. But it was likely a vague goal. If it had been a clear goal, it would have been very easy to also find a good measure for. And in that case, the impact of any initiative you chose for the goal would be measured by the measure of the goal.

Say, for example, your goal was to “Reduce the amount of time employees spend doing rework”. You might even go a step further and define rework as time spent changing or re-doing something that wasn’t done right the first time. Then it’s super easy to see this could be meaningfully measured by something like the percentage of employee work hours that are spent on rework.

And, continuing the example, the impact of any initiative to reduce rework would be measured by the percentage of employee work hours that are spent on rework. The impact would be the difference in this measure before and after the initiative was implemented.

Measures come before initiatives.

If you don’t know how to measure the impact of your initiative, you’ve chosen the initiative way too soon.
You’ve set the initiative out of logical order. But it’s common to do such things, and you’ll see it happening in many planning processes. It goes something like this:

1.    Set a goal [usually weasely and immeasurable]

2.    Pick an initiative to achieve the goal [without first thinking about the cause or leverage]

3.    Set a target [usually something meaningless like ‘10% improvement’]

4.    Choose a measure [as an afterthought, many weeks later, sometimes even after the initiative has been implemented]

There is a more logical order to approach planning, so that initiatives fit in sensibly and make the right impact (more detail here):

1.    Set a goal and make it easy to understand and measurable

2.    Design a meaningful measure as evidence of how much the goal is happening

3.    Set a sensible target for the measure to express how much improvement will mean the goal is achieved

4.    Analyze the business process that most impacts on the measure and find out the biggest cause that stops it performing at the target level

5.    Set an initiative that will mitigate this cause

6.    Implement the initiative in a way that isolates its impact.

Setting initiatives isn’t about jumping through planning process hoops. It isn’t about justifying our pet projects. It’s about spending resources as wisely as possible to reach specific goals.

We might need to practice non-attachment.

Following the second process, above, for aligning initiatives to measures and goals can often reveal that some initiatives are irrelevant or lame and will waste lots of time and money. And even if it stings our egos a little bit, it’s better to know that now, before all that time and money and angst goes to waste. And makes no difference that matters.

If you don’t know how to measure an initiative, you chose the initiative too soon.
[tweet this]

Stacey Barr is a specialist in organisational performance measurement and creator of PuMP, the refreshingly practical, step-by-step performance measurement methodology designed to overcome people’s biggest struggles with KPIs and measures. Learn about the bad habits that cause these struggles, and how to stop them, by taking Stacey’s free online course “The 10 Secrets to KPI Success” at www.staceybarr.com/the10secretstokpisuccess.
Stacey Barr Stacey Barr

Stacey Barr is a specialist in organizational performance measurement and creator of PuMP, the refreshingly practical, step-by-step performance measurement methodology designed to overcome people’s biggest struggles with KPIs and measures.

How to Make Sure Your Targets Match Your Measures

By: Stacey Barr

May 24, 2016 480 Views 0 Comments FacebookTwitterLinkedInGoogle Plus

Most performance measures or KPIs will have targets. After all, we have them to focus us on improving performance. But that focus can fracture if we don’t take care in how we express our measures and our targets, so they speak the same language.

Nerida (not her real name) emailed me this question:

"We have a measure “Number of staff with excess recreation leave balance” with a target of “90% of staff with no excess leave accrued”. Can the measure and target be like this or should they speak literally the same language?"

This is a very common situation. I see it in at least half of the strategic plans I look over. The targets are expressed in a language that doesn’t match the measure.

In Nerida’s case, the measure is focused on staff with excess recreation leave. But the target is focused on staff without excess leave. What’s more, the measure is a count but the target is a percentage. This won’t do.

It’s unnecessarily messy.

When people read the measure and the target, they’ll baulk at the inconsistent focus. This is distracting and confusing, and it erodes some of the credibility of measurement as a tool to inform decision making.

Clean up the mess by matching the units and the direction.

Nerida’s measure and target can easily be made simpler, clearer, and more credible by aligning two important features of measures and targets. The first feature is the units of measurement. Both her measure and target should be either a count, or a percentage. The second feature is the direction of improvement. Both her measure and target should focus on either staff with excess leave, or staff without excess leave.

So measure and target could be expressed more consistently this way:

Measure Name: Staff Without Excess Recreation Leave

Measure Description: Number of staff with no excess recreation leave balance, as a percentage of all staff

Target: 90%

A good formula to use, to express a measure and target.

Notice how the improved measure and target combination is written, above. There is a deliberate measure name, just a few words that capture the gist of the measure. Then there is a measure description, which fleshes out the quantitative method for computing the measure. And then the target is simply expressed as a number, in units that match the measure’s quantitative method.

The only thing missing in Nerida’s example, which I’d suggest you add, is a timeframe or date by which the target is to be achieved by. Or even perhaps, setting a target trajectory.

Stacey Barr is a specialist in organisational performance measurement and creator of PuMP, the refreshingly practical, step-by-step performance measurement methodology designed to overcome people’s biggest struggles with KPIs and measures. Learn about the bad habits that cause these struggles, and how to stop them, by taking Stacey’s free online course “The 10 Secrets to KPI Success” at www.staceybarr.com/the10secretstokpisuccess.
Stacey Barr Stacey Barr

Stacey Barr is a specialist in organizational performance measurement and creator of PuMP, the refreshingly practical, step-by-step performance measurement methodology designed to overcome people’s biggest struggles with KPIs and measures.

The 3 Signs To Say Goodbye to a KPI

By: Stacey Barr

Nov 25, 2014 408 Views 0 Comments FacebookTwitterLinkedInGoogle Plus

Hoarding is a disorder where people have incredible difficulty parting with possessions, or an incredible compulsion to acquire more possessions. They, and everyone close to them, drown in clutter. Are you a hoarder… of KPIs?

If you have too many measures or KPIs, and even a mild fear of letting them go in case you need them one day, then you might also be drowning in clutter: a clutter of data.

The problem with having too many KPIs is that you can’t really function with them all. It’s impossible to monitor them all, and too hard to know which are the priorities. You’re paralysed. You can’t make decisions.

Having too many KPIs is as bad as having none. It’s time to clean house. Here are three signs you can use to decide which KPIs to say goodbye to:

Sign #1: The KPI has reached its target and is staying there.

We measure specific results with KPIs because we want to improve those results. Now, while continuous improvement is certainly the philosophy that underpins good performance measurement, not everything should be improved forever.

If you have a KPI that has reached its target, and is now at a level of performance that isn’t a priority to keep improving (because there are other higher priorities to improve), it’s time to let it go. Otherwise, you’ll be wasting resources on gold plating that’s not needed.

Sign #2: Strategy has changed and the KPI no longer aligns.

If you do performance measurement well, then your KPIs will have a direct alignment to your strategic and operational goals. Some of those strategic or operational goals will change, as your organisation and its environment changes. So some of your KPIs will naturally change too.

Don’t be afraid to say goodbye to a KPI if it was designed for a goal that is no longer important. If you don’t, you’ll waste precious attention on results that don’t matter enough anymore.

Sign #3: The KPI turns out not to be as useful as expected.

We can master performance measurement but I don’t think we can perfect it. Consequently, we might design a KPI that sounded like a brilliant idea at the time, but later when we start using it, we realise it doesn’t have the power we hoped for.

Let that KPI go, so that it doesn’t mislead people. And if you need to, replace it with one that’s more useful.

What does saying “Goodbye, KPI” really mean?

I’m a believer in keeping historic data because it’s cheap enough to do it these days, and it can be a treasure trove for analysis in the future. Data doesn’t really take up space, like a KPI does in a performance report.

So, archive your KPIs, by retaining documentation of their definitions. You can do this in your corporate performance measure dictionary, marking the KPI as ‘archived’. Then take them out of your performance reports, and stop extracting the data to compute them.

Stacey Barr is a specialist in organisational performance measurement and creator of PuMP, the refreshingly practical, step-by-step performance measurement methodology designed to overcome people’s biggest struggles with KPIs and measures. Learn about the bad habits that cause these struggles, and how to stop them, by taking Stacey’s free online course “The 10 Secrets to KPI Success” at www.staceybarr.com/the10secretstokpisuccess.
Stacey Barr Stacey Barr

Stacey Barr is a specialist in organizational performance measurement and creator of PuMP, the refreshingly practical, step-by-step performance measurement methodology designed to overcome people’s biggest struggles with KPIs and measures.

6 Reasons to Stop Measuring It

By: Stacey Barr

Jul 22, 2014 512 Views 0 Comments FacebookTwitterLinkedInGoogle Plus

If you’re problem isn’t having too few performance measures, then most certainly it’s having too many. We don’t like to let go of our performance measures because we might just need them someday, and it took a lot of effort to get them in the first place. These aren’t good reasons for keeping them, and doing so will cause us more harm than good.

We do best when we have the fewest possible performance measures that help us make the progress we want toward our most valued goals. Too many measures make us confused and sluggish, overwhelmed and mediocre. Less is more.

You won’t get the fewest greatest performance measures until you declutter and create space to more clearly see what you truly need. You must be brave and ditch those desultory measures.

Use the following six reasons as something of a checklist to test if you should kill off a performance measure. I’d suggest that if even just one of these reasons applies, it’s really worth getting rid of that measure.

Reason #1: It’s not really a true performance measure.

Many ditch-worthy measures are not really measures at all. Examples of non-measures include milestones, activities and vague phrases:

  • ‘Research, discussion, and position papers supported and/or written’
  • ‘85% staff have been briefed on the marketing plan’
  • ‘Increased market acceptance’

Get rid of any ‘measure’ that does not meet the definition of what a true performance measure is.

Reason #2: It doesn’t strongly link to anything strategic or mission-critical.

We measure to improve. While absolutely everything could potentially be improved, it’s ludicrous to do it. So we set priorities: goals for the things that are the most worthwhile to improve.

Performance measures should align with those things that are the most worthwhile to improve. If you’re measuring it just because it’s easy, because it’s always been measured, because it might be important one day, or because it’s ‘interesting’, then you can safely stop measuring it.

Reason #3: It’s making people behave the wrong way.

If people believe that performance measures are noticed, they will choose the easiest way to make sure that measure reflects positively back on them. The easiest way to do this might be:

  • report only the good data (leave out deliveries that are more than 1 hour late from the on-time delivery measure)
  • fabricate data that makes the measure look good (getting account managers to fill in the customer survey on behalf of customers, rather than giving the survey to the customers themselves)
  • create their own definition of the measure, based on what they know will look good (reporting on the time it takes to get the first luggage item from the plane to the baggage carousel, rather than the median time for all luggage)
  • sacrifice performance in another area to focus more on the one in the spotlight (leave out safety checks on freight vehicles in order to shorten delivery times)

If you can’t mitigate the unwanted behaviors, you’re safer to kill off the measure and work with the team to establish better reasons to measureHelp them choose goals and measures they care about and can influence through process improvements.

Reason #4: It can’t give you unbiased and accurate enough evidence.

Some measures might be vitally important, but the data just might not be available to make the measure also trustworthy.

One manager wanted to measure the cost of the end-to-end supply chain in his organisation. The ideal data for this required that costs could be allocated to specific tasks through the supply chain. But their accounting system simply wasn’t set up to do this.

So while Total Supply Chain Costs might be a very valuable measure, finding a proxy for it (the manager considered using Labor Costs) would be too misleading. He’d be better off ditching the Labor Costs measure and putting the effort into properly defining the Total Supply Chain Costs measure and setting up activity-based accounting to produce the data they need.

Reason #5: It’s measuring a result beyond your circle of influence.

Particularly for government and not-for-profit organisations, it’s easy to end up with measures you can’t do much to affect. That’s because the outcomes of these organisations tend to be about changing other people’s beliefs, attitudes and behaviours.

Yes, it’s important for a health organisation to know if more people are getting healthier. But this outcome is affected by so many other organisations and factors that it’s near impossible to isolate the effect of the one health organisation.

It’s more useful (and moralizing) for the health organisation to measure something more directly resulting from their policies, messages and programs. For example, they could measure the reach of their messages, the reactions people have to those messages, and the changes in health of those people participating in their programs. These things are within their circle of influence.

Reason #6: There’s no ownership of it by the people who should use it.

You might think it’s the Best Performance Measure In The World, but if the people who should use it say ‘meh’, then think again.

I facilitated a group of people in a government transport authority to come up with some performance measures for the impact of projects on the condition of roads. Some members of the team were engineers, naturally. And one of them came up with a doozy of a performance measure that no-one else understood or wanted to use.

No ownership means the measure won’t be used. And having and using a good measure is better than having and ignoring a great one. Save the great measure in your measures dictionary so you don’t forget it, and so it’s at the ready when it’s time has come. But don’t waste effort implementing it.

Stacey Barr is a specialist in organisational performance measurement and creator of PuMP, the refreshingly practical, step-by-step performance measurement methodology designed to overcome people’s biggest struggles with KPIs and measures. Learn about the bad habits that cause these struggles, and how to stop them, by taking Stacey’s free online course “The 10 Secrets to KPI Success” at www.staceybarr.com/the10secretstokpisuccess.
Stacey Barr Stacey Barr

Stacey Barr is a specialist in organizational performance measurement and creator of PuMP, the refreshingly practical, step-by-step performance measurement methodology designed to overcome people’s biggest struggles with KPIs and measures.

3 Essential Signals to Look for in Your KPIs

By: Stacey Barr

Sep 3, 2013 402 Views 0 Comments FacebookTwitterLinkedInGoogle Plus

You won’t find true signals of changes in performance by looking at month-to-month comparisons, or trendlines, or moving averages. The signals you really need to know about, the only signals that you ought to respond to, are revealed through one particular graph only.

The typical analysis methods we use for our performance measures are based on assumptions that don’t make much sense.

Month-to-month comparisons assume that there is no routine variation over time and wrongly interpret any difference as a signal.

Trendlines assume that all change is linear and gradual, and that if Excel can calculate a trend line then there must be a trend.

Moving averages assume that seasonal patterns exist, and also that change is smooth and gradual.

The only analysis technique I have ever come across that clearly filters the noise and highlights the signals in our performance measures is the XmR chart.

The XmR chart filters the noisy routine variation in our measures by showing us how much of this routine variation there is, by way of the Natural Process Limits. And coupled with the Central Line, these Natural Process Limits give us a meaningful baseline to quickly assess when performance has changed; when there is something else going on that’s not part of the routine variation.

There are three very specific signals to look for.

SIGNAL 1: Outlier or special cause

When a measure value falls outside the Natural Process Limits, it means that more than just the routine variation is at play. It’s a signal that something else has happened.

If Employee Attendance plummeted below the bottom Natural Process Limit, a likely cause could be a flu epidemic or local natural disaster that kept many more people away from work in that period.

Even though this is a signal that something out of the norm has happened, because it’s just a one-time event, we don’t react to it. We find out what caused it, but we don’t run around madly trying to fix it. That would be a massive waste of time and money because we’d essentially by trying to fix something that won’t happen again, or that is completely outside our control.

SIGNAL 2: Long run

To be convinced that a change in the level of performance has happened, we need to see seven (yes, seven) points in a row on the same side of the Central Line. The probability that a pattern like that is part of routine variation is close to zero (0.78%, to be precise). Seven points, not three or five or one.

If a measure of Invoice Accuracy showed a long run above the Central Line, it might be evidence that an initiative to simplify the pricing strategy had successfully reduced the errors in invoices.

When we see a long run signal in our measure, we certainly need to find the cause for it. Sometimes it will be a signal of improvement, and we want to confirm what caused the improvement. Other times it will be a signal that performance has deteriorated and the cause of that is very important to identify!

SIGNAL 3: Short run

You’re no doubt thinking to yourself ‘I can’t wait for seven months before I can know if I should take action!’ You can either measure more frequently to pick up signals sooner (as long as it makes sense to), or plan for bigger signals.

A bigger signal appears as a short run, of three out of four consecutive measure values closer to a Natural Process Limit than they are to the Central Line. The probability of this pattern happening also has a very close to zero probability.

A short run above the Central Line for On-time Deliveries for a trucking company would likely be due to an initiative that had a substantially large impact. It could be something like doubling the fleet size. But it also could be a new competitor in the market that poached a large percentage of their customers.

Again, with a signal like the short run, it’s really important to find the cause before responding.

XmR charts take only a little effort to create, but their usefulness is so powerful they are absolutely worth trying.

Stacey Barr is a specialist in organisational performance measurement and creator of PuMP, the refreshingly practical, step-by-step performance measurement methodology designed to overcome people’s biggest struggles with KPIs and measures. Learn about the bad habits that cause these struggles, and how to stop them, by taking Stacey’s free online course “The 10 Secrets to KPI Success” at www.staceybarr.com/the10secretstokpisuccess.