X

The KPI.org Blog

How to Measure Strategic Impact in a ‘Right Now’ World

By: David Wilsey

May 19, 2022 284 Views 0 Comments FacebookTwitterLinkedInGoogle Plus

The focus of most strategy and measurement efforts used to be long term. We encouraged clients to step back from the day-to-day whirlwind of daily operational activities and firefighting and think about desired long-term objectives. The word “impact” implies this more reflective thinking.

But the strategic environment has changed quickly for many of our clients, causing them to rethink this attitude to better react to what is being called the “right now” world. Whether it is post-COVID changes, political turmoil, inflation, or the tragedy in Ukraine, some clients are being forced to adapt to threats like never before. And some technology clients have long insisted that strategic agility was an absolute requirement for their success.

So how should a strategy or measurement professional adapt to this new environment? As usual, the most important factors are either related to culture or process.

Culturally, organizations must think of strategy and measurement in terms of general principles rather than absolutes. Strategy is not an event. Strategic thinking is a skill that can be applied to any endeavor over just about any time frame. As many OKR experts such as Felipe Castro and Dan Montgomery will point out, if our old static 5-year strategic plan is no longer useful due to a rapidly changing world, the principles of connecting dots and articulating desired strategic outcomes need to be applied in a more iterative manner that that can be used to validate (or not) shorter-term strategic hypotheses. If top-down culture and long feedback cycles are not effective anymore, use those articulated desired outcomes to create shorter-term alignment and faster performance cadences. Measurement and reporting need not be relatively static exercises done by the special few at the organization level. They are skills that should be taught to all managers and supervisors so that they can effectively do their daily job.



Of course, most improvement happens at the process level. BSI has a new KPI development process, called the Measure-Perform-Review-Adapt (MPRA) model. While we will be announcing more about that model at a later date, the key point for this blog is that the new model places the emphasis on a regular cycle of review and adaptation. We start by articulating and communicating strategic intent before measures are considered, selected, and defined. Then in the Perform-Review-Adapt cycle the organization has a chance to react quickly to changes in the strategic environment or reforecast targets for the next quarter. For many of our clients that were in the habit of setting their KPI targets and then forgetting them, this review cycle is the missing discipline needed to keep their teams on track and to get things done. It borrows the key principle from the agile world that assumes that we cannot possibly know everything about what we want at the beginning of the process and so need discipline around learning and adaptation.

To learn more about our general approach to KPI development, check out our KPI Professional certification program.

Keep Your Unit of Measure Intuitive

By: David Wilsey

Apr 5, 2022 224 Views 0 Comments FacebookTwitterLinkedInGoogle Plus

Anyone trying to eat healthily or count calories can relate to the confusing state of nutritional measures. The app I (sometimes) use tracks bread in grams, almonds in ounces, soup in grams, and sometimes ambiguously lists “serving” as the unit of measure. So, if I eat a handful of almonds, 2 slices of bread and a can of soup, did I eat too much? Of course, I can do a quick search and find that 2 slices of bread amount to about 56 grams, 12-13 almonds count as a half of an ounce, and the can of soup is 242 grams. But for something I don’t want to be spending my time on, more required research means it’s not worth the trouble.

Units of measurement (which is what’s being counted, such as dollars, number of items, degrees Fahrenheit, seconds, or unit produced, etc.) should be intuitive where possible. Most of us can visualize 12 almonds but not a half ounce.

This principle applies to KPI design in the workplace. Sometimes I will see a client get too clever without an obvious benefit. One client thought it was too pedestrian to simply measure sales dollars, pipeline size, win rate, average deal size, and other common business development measures, so they created a convoluted index that no one really understood. The sales team would meet to discuss the pipeline sales index performance and you could see the confusion and frustration on many of the faces. No one knew where to focus their improvement efforts.

This sometimes happens due to the best of intentions. Web traffic measures don’t always tell you what you want to know and so the web design world has moved towards more sophisticated measurements of “engagement”. While an engagement score (which typically measure how much your website visitors are interacting with your website and online brand) might be more meaningful to an educated audience, there is a risk that the users of the data no longer understand what is being measured.

The key is that the farther you get from the most intuitive unit of measure, the more you must educate people about the measure. For example, the baseball world has evolved away from simply counting home runs and batting average to the more sophisticated ‘sabermetrics’ like Wins Above Replacement (WAR) and Weighted Runs Created (wRC+). For diehards of either baseball or statistical analysis, this movement has been a breath of fresh air that has revolutionized the game. To casual fans, however, these statistics are confusing and off-putting. Home runs and batting average might not be as meaningful, but at least all of us know what we are counting.

If you lead an organization, you must ask yourself if most of your employees will find the more convoluted measures useful enough to justify the additional educational effort needed. If it is a strong measure, it will be an easy decision. The simpler measure might not be good enough – the more complicated solution might really be transformational. But in some cases, you might decide that you simply want your employees to understand that they are trying to hit it over your metaphorical fence and leave it at that. Remember that measurement is not something that everyone wants to spend their time on. Make sure that it is worth the trouble.

If you would like to learn more about our general approach to KPI development, please consider our KPI Professional certification program.

Sources

Image Source: Wikipedia

How to Cheat at Wordle Using Indirect Measurement

By: David Wilsey

Mar 9, 2022 568 Views 0 Comments FacebookTwitterLinkedInGoogle Plus

You have likely heard of the word-guessing game Wordle. If you are on social media, you might even be tired of seeing the related green and yellow grids on your feed each day. You might not know, however, that a few folks have discovered a sneaky way to cheat.

Apparently, the Wordle word-of-the-day has been showing up on the list of top searches on certain online dictionary sites around 9:00 am US east coast time lately. So, if you are so inclined, you could scan that list for 5-letter words before playing and have a good idea what to guess.

This example demonstrates how an organization might use indirect measurement (or a “proxy” measure) as a KPI. An indirect measure can be used in cases where it is not practical to directly measure an intended result or goal and is based on a hypothesis around correlation or contribution. Metaphorically speaking, let’s say the word-of-the-day is something an organization wants to accomplish but is finding it difficult to measure directly. You could create a KPI around the number of searches for any particular 5-letter word on the dictionary sites and then use that data to inform your word-guessing decision. The hypothesis is based on an expected correlation between the dictionary searches and the game.

It also demonstrates the imprecise nature of indirect measures. The dictionary search measure informs my word-guessing decision, but I can’t assume that there isn’t some other reason why a 5-letter word might show up on the list. Direct measurement is almost always preferred. If we want more sales, the direct sales dollars tell us what we want to know precisely. Once I shift to something that is indirect, like web traffic on my shopping cart website, my measurement becomes less reliable and requires more work to connect to the intended result.

But in the management world, many times a direct measurement is not obvious. A non-profit that works in advocacy might not be able to identify tangible intended results of their work. Teams that succeed through collaboration and innovation sometimes end up with softer, more indirect, measures of success. Or in the example above, maybe I want to measure the web traffic to better understand what is contributing to sales results. The point is that sometimes indirect measurement is useful, as long as you understand where the weaknesses are.

If you would like to learn more about indirect measurement or our general approach to KPI development, please consider our KPI Professional certification program.

Source: https://www.washingtonpost.com/video-games/2022/02/19/wordle-cheat/

Image source: Wikipedia

David Wilsey David Wilsey

David Wilsey is the Chief Operating Officer with the Balanced Scorecard Institute and co-author of The Institute Way: Simplify Strategic Planning and Management with the Balanced Scorecard.

KPI Development: When to Use Ratios

By: David Wilsey

Nov 30, 2020 724 Views 0 Comments FacebookTwitterLinkedInGoogle Plus

Let’s say you are managing a large organization and your HR director comes to you with bad news. The total number of employees leaving the company has skyrocketed. They show you the chart to the right to indicate that the raw number of employees leaving last year was below 20 and this year it is 50. Should you panic?

The answer is that it depends. What if I told you that this year the company acquired a competitor and that the total number of employees had expanded from just over 500 to over 2200 (see the chart below), and so the turnover percentage (# leaving / total # of employees) had actually decreased slightly (see Turnover ratio). Would that make you panic? Probably not.

 

The point is that raw counts can be misleading if you are not careful about the comparisons you are making relative to the underlying population. You might have seen debates in the news about how political leaders or writers have highlighted raw COVID 19 counts to support an argument when a per capita ratio suggests a different conclusion. It’s not that raw counts aren’t sometimes useful, but it is important to be aware of what you are comparing and how the overall population might impact interpretation. We recommend that any measures that could be misinterpreted be expressed as a ratio. Common examples of ratios include:

  • Percent completion
  • Coverage, fraction of the total possible
  • Error or defect rate
  • Per capita
  • Efficiency = Output / Input
  • Productivity = Output / Cost (or Output / work hour)

Beginners in the KPI space often make the mistake of developing raw count measures where ratios would be more meaningful.

One type of ratio you do NOT see on this list is percent increase. Many of our clients mistakenly think that if they create a ratio comparing the current period with the previous period that they have solved this problem. While a percent increase is technically a ratio, creating a ratio using the 2017 raw count and the 2016 raw count is still just as misleading, as more than 150% increase in turnover from year to year is still unnecessarily alarming. Of course, you could choose to track a percent increase of the ratio, but that would be addressing a different question, one that I will address in an upcoming blog.

To learn more about ratios and other KPI development issues, please see our KPI Professional Certification program.

KPI Development: When to Use Ratios

By: David Wilsey

Nov 30, 2020 1310 Views 0 Comments FacebookTwitterLinkedInGoogle Plus

Let’s say you are managing a large organization and your HR director comes to you with bad news. The total number of employees leaving the company has skyrocketed. They show you the chart to the right to indicate that the raw number of employees leaving last year was below 20 and this year it is 50. Should you panic?

The answer is that it depends. What if I told you that this year the company acquired a competitor and that the total number of employees had expanded from just over 500 to over 2200 (see the chart below), and so the turnover percentage (# leaving / total # of employees) had actually decreased slightly (see Turnover ratio). Would that make you panic? Probably not.

The point is that raw counts can be misleading if you are not careful about the comparisons you are making relative to the underlying population. You might have seen debates in the news about how political leaders or writers have highlighted raw COVID 19 counts to support an argument when a per capita ratio suggests a different conclusion. It’s not that raw counts aren’t sometimes useful, but it is important to be aware of what you are comparing and how the overall population might impact interpretation. We recommend that any measures that could be misinterpreted be expressed as a ratio. Common examples of ratios include:

  • Percent completion
  • Coverage, fraction of the total possible
  • Error or defect rate
  • Per capita
  • Efficiency = Output / Input
  • Productivity = Output / Cost (or Output / work hour)

Beginners in the KPI space often make the mistake of developing raw count measures where ratios would be more meaningful.

One type of ratio you do NOT see on this list is percent increase. Many of our clients mistakenly think that if they create a ratio comparing the current period with the previous period that they have solved this problem. While a percent increase is technically a ratio, creating a ratio using the 2017 raw count and the 2016 raw count is still just as misleading, as more than 150% increase in turnover from year to year is still unnecessarily alarming. Of course, you could choose to track a percent increase of the ratio, but that would be addressing a different question, one that I will address in an upcoming blog.

To learn more about ratios and other KPI development issues, please see our KPI Professional Certification program.

David Wilsey David Wilsey

David Wilsey is the Chief Operating Officer with the Balanced Scorecard Institute and co-author of The Institute Way: Simplify Strategic Planning and Management with the Balanced Scorecard.

4 Ways Your KPIs Remind Me of Our New Puppy

By: David Wilsey

Aug 31, 2020 969 Views 0 Comments FacebookTwitterLinkedInGoogle Plus
We welcomed a new puppy named Romeo to our home recently, having finally given in to the kids after about five years of pleading. While we are generally happy with the decision, late last night I stood outside in the rain watching the dog refuse to relieve himself and realized that there are several similarities between a new puppy and the challenges that my KPI clients face.

 

They Lack Discipline

I keep telling myself that it’s not that puppies want to create a big mess, it’s that they just don’t know better. Many organizations develop KPIs that are designed by folks that are new to the KPI world also lack discipline. Without a disciplined development process, organizations tend to brainstorm KPIs that monitor activities or operational elements that aren’t very meaningful. Even when decent measures are selected, implementation requires discipline. I can’t tell you the number of training participants who have told me that they’ve selected measures but haven’t gotten around to tracking anything yet.

They Bite When They Play

My kids love playing with the puppy but complain about his playful nipping. Similarly, measures can play rough at first too as you begin to learn things you didn’t know about your performance. I’ll never forget the KPI champion I worked with that stopped supporting the program soon after he realized that their new performance measure was making him look bad. Just like it takes puppies a few months to learn not to bite, every organization must learn how to manage the fear that is associated with measuring performance. Bad news should be welcomed as it provides focus for the organization.

They Are More Work at the Beginning

With puppies, things can get easier over time provided you invest the time and effort to establish good habits. Performance measures are the same way. Some clients get discouraged by the mountain of work required to develop, define, and implement good measures. It is hard to convince some it does get easier once the measures are in place and you establish an effective reporting routine.

They’re Often Excited About the Wrong Things

Romeo hasn’t met a roll of toilet paper yet that he doesn’t want to devour. KPIs can also cause clients to fixate on inappropriate things if no one has clarified what is important. One recent training delegate told me that they still track and report on network uptime on a regular basis simply because the data was handy. Network uptime was of no great strategic importance for their organization and so the entire effort was a waste of time. KPIs are supposed to be “KEY”, meaning they indicate improvement of an important performance result. Strategic measures tell you if the organization is moving forward or changing to meet the needs of a new world. Operational measures tell you if you if you are delivering quality products and services effectively and efficiently. Most of the clients that tell us they need help with measurement actually need help articulating what they are trying to accomplish.

To learn more about how to overcome these and other measurement challenges, please see our KPI Professional Certification program.

David Wilsey David Wilsey

David Wilsey is the Chief Operating Officer with the Balanced Scorecard Institute and co-author of The Institute Way: Simplify Strategic Planning and Management with the Balanced Scorecard.

The “Formula” for KPI Success

By: David Wilsey

Jan 31, 2020 1077 Views 0 Comments FacebookTwitterLinkedInGoogle Plus

I was facilitating a client performance measure development session recently, and the team was wrestling with measure definition. When I reviewed the customer perspective team’s work I found:

Measure Name: % of Customers Satisfied
Description:
% of Customers Satisfied
Formula:
% of Customers Satisfied

The team was mystified by what the formula should be. The following definitions and examples helped them get back on track. Measure name is a word or short phrase that names the specific measure and is understood by users. The description details what the measurement is about, including its intent, why it matters and what it includes and excludes. Imagine describing the measure to someone on an elevator. A formula is the mathematical equation or rule used to calculate the measurement, and expressed in symbols, essentially a set of instructions for creating the desired metric. To demonstrate, below are a few common measures from various walks of life, with an example of a description and formula.

Name: Customer Satisfaction Survey Score
Description:
Periodic survey of a sampling of customers, who are asked to quantify an “I am satisfied”-type response on a five-point Likert scale.
Formula:
# indicating Agree or Strongly Agree / total # of responses

Name: Fuel Efficiency
Description:
the distance traveled per unit of fuel used. Used in the US, UK, and much of Asia. The higher the value, the more efficient the car is at using fuel.
Formula:
# of miles driven / # of gallons of fuel used (or # of km driven / # of liters used)

Name: Fuel Consumption
Description:
amount of fuel used per unit distance driven, often expressed as liters used per 100 kilometers driven. Used in much of Europe. The lower the value, the more economical a vehicle because it takes less fuel to drive a certain distance.
Formula:
(# liters of fuel used / kilometers driven) *100

Name: Batting Average (baseball)
Description:
Batting average is used to compare the consistency of batters in baseball. It is calculated as the number of hits divided by the official number of at-bats and is expressed as a decimal to three places of accuracy.
Formula:
Total # of hits / total # of at bats, not including walks, sacrifices, or hit by pitches

Name: Steps Description: Tracking the number of steps walked each day is an effective way to ensure that you are getting enough physical activity each day. Countless research has indicated that measurable benchmarks in fitness can help improve health outcomes.
Formula:
Total # of steps walked per day

Name: Graduation Rate
Description: Graduation rates is the percentage of students who graduate from a school within a prescribed time period (which varies between school levels but is typically 150% of the published time for to complete a program). Graduation rates are considered important because it shows how committed schools are to helping students learn all that is required to meet the graduation standard within a reasonable amount of time.
Formula: # of students graduating within a prescribed time period / total # of students in the graduating class

Name: Employee Turnover Rate
Description:
Employee turnover is a measurement of how many employees are leaving a company. High turnover can be costly, as replacing employees is expensive. High voluntary turnover of top performers can also be an indicator of poor management or morale issues.
Formula: # of employees (voluntarily) exiting the job / average actual # of employees (during the same period)

For strategy-minded people more interested in discussing lofty strategy options than grinding through data collection and validity details, data definition can be a challenge. But I’ve found that if you can master these three basic data definition steps, you’ll be a long way towards consensus about exactly what you are counting and why.

For more information about KPI / performance measure development, please visit KPI.org/KPIP.


9 Things We Wish We Had Known About KPIs

Recorded October 30, 2018 - Replay Available

Oct 30, 2018 4010 Views 0 Comments FacebookTwitterLinkedInGoogle Plus

Recorded October 30, 2018 - Replay Available Below

Isaac Newton famously said: "If I have seen further it is by standing on the shoulders of Giants” which means it is easier to discover truth when you can build upon previous discoveries.

BSI's COO David Wilsey and Corporater's former Chief Strategy Officer & VP, Gail S. Perry will share the 9 things they wish they had understood about KPIs from the very beginning, instead of learning them thru mistakes, hard experience, and untangling KPI messes at numerous clients. They have a combined 50 years of performance management experience and they have the battle scars to show for it. 

While neither Gail nor David claim to be giant or genius, the 9 lessons they share will jump start your understanding of KPIs and help you leapfrog over similar missteps. 





Sign up for our newsletter to get the latest information about our next webinar! 

 

Gail Stout Perry Gail Stout Perry

Gail is co-author of The Institute Way. With a career spanning over 30 years of strategic planning and performance management consulting with corporate, nonprofit, and government organizations, she enjoys speaking, training, and writing, sharing her experience with others. She currently is the Chief Strategy Officer and VP Americas for Corporater.

The Four Things I Wish I’d Known - Part 3

By: Gail S. Perry

Oct 15, 2018 5241 Views 0 Comments FacebookTwitterLinkedInGoogle Plus

KPIs Are Essential (But Know Your Audience)

I wanted to sink into the conference room floor. I was so embarrassed and was convinced that I must have just asked the stupidest question in the world. To this day, I cringe at the memory of standing there, in front of the entire leadership team at a prestigious, world-renowned, non-profit organization, while the entire team stared blankly at me. I was well into the second decade of my consulting career and was accustomed to taking on major projects. This time, I’d been asked to design a dashboard of metrics for this organization.  I’d gathered the heads of all functions and departments to explain the purpose of the project, their roles as stakeholders, and then, poised to write responses on the whiteboard, I asked the question: “Can each of you tell me what three to six key metrics you use (or would like to us) to manage your part of the organization?” My thinking was that this would give us a quick and rough outline of what metrics mattered most to the people who ran the organization – these same people who were now staring at me. Finally, one gentleman spoke up and said, “I believe this is what we hired you to do – don’t you know what metrics we should use?”  

Fast forward another decade and I have a lot more KPI experience under my belt and I’ve worked with dozens of major organizations on performance management as well as implemented KPIs for my own use in managing an organization. And in hindsight, I now realize that the managers who were staring at me should have known their key processes and value drivers and been able to articulate what they were trying to accomplish and how to measure it.

I have since learned that there are two kinds of managers/leaders. Those who operate at a tactical level and those who see the full picture. The tactical managers keep very busy managing what Covey calls the whirlwind of daily operations. Some focus only on the day-to-day actions that are required of them.  Some are great at people skills. Some are more entrepreneurial and implement innovations, initiatives, and projects they feel are needed as they sense and respond to risks and signals at the tactical level. But after all these years, I see how these sorts of managers consistently fail to achieve meaningful long-term results. They perform well on daily operations, but few can achieve sustainable improvements in those operations.  And that is exactly what happened to every one of the managers in that conference room. They did their daily jobs well, but they couldn’t produce long-term results for the organization.  Within five years, all were replaced.

The other type of managers/leaders see the full picture of key processes and value drivers and they leverage KPIs to monitor and manage performance. They know KPIs (metrics) will enable them to better manage overall performance as well as to assess the impact of any innovations, initiatives, and projects. 
  
I’ve since learned that I didn’t ask a stupid question. I simply was asking it of the wrong sort of manager/leaders. I’ve asked that same question of the other type of managers and they rattle off metrics faster than I can write them down.  

I have learned to assess the audience first and be sensitive to the fact that not everyone knows about KPIs or how they enable managers with insights and power for improving performance. Some individuals may need some basic education about the topic, they may have a long change management journey to buy into the value and use of KPIs, and they most likely will need coaching help to figure out their key processes and value drivers, as well as how to determine appropriate KPIs to use.  

It’s not rocket science. To some of us, it is simply common sense. But not everyone is wired this way.  We are all born with different natural tendencies so I’ve tried to learn to keep that in mind. And I no longer sink into the floor when someone stares blankly at me. I simply start asking more questions until we find common ground and then work forward from there.  

Read Part 2 of The Four Things I Wish I’d Known here. Read Part 4 here
123


Get our tips straight to your inbox, and 
become more strategy focused