The KPI.org Blog

How to Measure Strategic Impact in a ‘Right Now’ World

By: David Wilsey

May 19, 2022 284 Views 0 Comments FacebookTwitterLinkedInGoogle Plus

The focus of most strategy and measurement efforts used to be long term. We encouraged clients to step back from the day-to-day whirlwind of daily operational activities and firefighting and think about desired long-term objectives. The word “impact” implies this more reflective thinking.

But the strategic environment has changed quickly for many of our clients, causing them to rethink this attitude to better react to what is being called the “right now” world. Whether it is post-COVID changes, political turmoil, inflation, or the tragedy in Ukraine, some clients are being forced to adapt to threats like never before. And some technology clients have long insisted that strategic agility was an absolute requirement for their success.

So how should a strategy or measurement professional adapt to this new environment? As usual, the most important factors are either related to culture or process.

Culturally, organizations must think of strategy and measurement in terms of general principles rather than absolutes. Strategy is not an event. Strategic thinking is a skill that can be applied to any endeavor over just about any time frame. As many OKR experts such as Felipe Castro and Dan Montgomery will point out, if our old static 5-year strategic plan is no longer useful due to a rapidly changing world, the principles of connecting dots and articulating desired strategic outcomes need to be applied in a more iterative manner that that can be used to validate (or not) shorter-term strategic hypotheses. If top-down culture and long feedback cycles are not effective anymore, use those articulated desired outcomes to create shorter-term alignment and faster performance cadences. Measurement and reporting need not be relatively static exercises done by the special few at the organization level. They are skills that should be taught to all managers and supervisors so that they can effectively do their daily job.

Of course, most improvement happens at the process level. BSI has a new KPI development process, called the Measure-Perform-Review-Adapt (MPRA) model. While we will be announcing more about that model at a later date, the key point for this blog is that the new model places the emphasis on a regular cycle of review and adaptation. We start by articulating and communicating strategic intent before measures are considered, selected, and defined. Then in the Perform-Review-Adapt cycle the organization has a chance to react quickly to changes in the strategic environment or reforecast targets for the next quarter. For many of our clients that were in the habit of setting their KPI targets and then forgetting them, this review cycle is the missing discipline needed to keep their teams on track and to get things done. It borrows the key principle from the agile world that assumes that we cannot possibly know everything about what we want at the beginning of the process and so need discipline around learning and adaptation.

To learn more about our general approach to KPI development, check out our KPI Professional certification program.

Keep Your Unit of Measure Intuitive

By: David Wilsey

Apr 5, 2022 224 Views 0 Comments FacebookTwitterLinkedInGoogle Plus

Anyone trying to eat healthily or count calories can relate to the confusing state of nutritional measures. The app I (sometimes) use tracks bread in grams, almonds in ounces, soup in grams, and sometimes ambiguously lists “serving” as the unit of measure. So, if I eat a handful of almonds, 2 slices of bread and a can of soup, did I eat too much? Of course, I can do a quick search and find that 2 slices of bread amount to about 56 grams, 12-13 almonds count as a half of an ounce, and the can of soup is 242 grams. But for something I don’t want to be spending my time on, more required research means it’s not worth the trouble.

Units of measurement (which is what’s being counted, such as dollars, number of items, degrees Fahrenheit, seconds, or unit produced, etc.) should be intuitive where possible. Most of us can visualize 12 almonds but not a half ounce.

This principle applies to KPI design in the workplace. Sometimes I will see a client get too clever without an obvious benefit. One client thought it was too pedestrian to simply measure sales dollars, pipeline size, win rate, average deal size, and other common business development measures, so they created a convoluted index that no one really understood. The sales team would meet to discuss the pipeline sales index performance and you could see the confusion and frustration on many of the faces. No one knew where to focus their improvement efforts.

This sometimes happens due to the best of intentions. Web traffic measures don’t always tell you what you want to know and so the web design world has moved towards more sophisticated measurements of “engagement”. While an engagement score (which typically measure how much your website visitors are interacting with your website and online brand) might be more meaningful to an educated audience, there is a risk that the users of the data no longer understand what is being measured.

The key is that the farther you get from the most intuitive unit of measure, the more you must educate people about the measure. For example, the baseball world has evolved away from simply counting home runs and batting average to the more sophisticated ‘sabermetrics’ like Wins Above Replacement (WAR) and Weighted Runs Created (wRC+). For diehards of either baseball or statistical analysis, this movement has been a breath of fresh air that has revolutionized the game. To casual fans, however, these statistics are confusing and off-putting. Home runs and batting average might not be as meaningful, but at least all of us know what we are counting.

If you lead an organization, you must ask yourself if most of your employees will find the more convoluted measures useful enough to justify the additional educational effort needed. If it is a strong measure, it will be an easy decision. The simpler measure might not be good enough – the more complicated solution might really be transformational. But in some cases, you might decide that you simply want your employees to understand that they are trying to hit it over your metaphorical fence and leave it at that. Remember that measurement is not something that everyone wants to spend their time on. Make sure that it is worth the trouble.

If you would like to learn more about our general approach to KPI development, please consider our KPI Professional certification program.


Image Source: Wikipedia

How to Cheat at Wordle Using Indirect Measurement

By: David Wilsey

Mar 9, 2022 568 Views 0 Comments FacebookTwitterLinkedInGoogle Plus

You have likely heard of the word-guessing game Wordle. If you are on social media, you might even be tired of seeing the related green and yellow grids on your feed each day. You might not know, however, that a few folks have discovered a sneaky way to cheat.

Apparently, the Wordle word-of-the-day has been showing up on the list of top searches on certain online dictionary sites around 9:00 am US east coast time lately. So, if you are so inclined, you could scan that list for 5-letter words before playing and have a good idea what to guess.

This example demonstrates how an organization might use indirect measurement (or a “proxy” measure) as a KPI. An indirect measure can be used in cases where it is not practical to directly measure an intended result or goal and is based on a hypothesis around correlation or contribution. Metaphorically speaking, let’s say the word-of-the-day is something an organization wants to accomplish but is finding it difficult to measure directly. You could create a KPI around the number of searches for any particular 5-letter word on the dictionary sites and then use that data to inform your word-guessing decision. The hypothesis is based on an expected correlation between the dictionary searches and the game.

It also demonstrates the imprecise nature of indirect measures. The dictionary search measure informs my word-guessing decision, but I can’t assume that there isn’t some other reason why a 5-letter word might show up on the list. Direct measurement is almost always preferred. If we want more sales, the direct sales dollars tell us what we want to know precisely. Once I shift to something that is indirect, like web traffic on my shopping cart website, my measurement becomes less reliable and requires more work to connect to the intended result.

But in the management world, many times a direct measurement is not obvious. A non-profit that works in advocacy might not be able to identify tangible intended results of their work. Teams that succeed through collaboration and innovation sometimes end up with softer, more indirect, measures of success. Or in the example above, maybe I want to measure the web traffic to better understand what is contributing to sales results. The point is that sometimes indirect measurement is useful, as long as you understand where the weaknesses are.

If you would like to learn more about indirect measurement or our general approach to KPI development, please consider our KPI Professional certification program.

Source: https://www.washingtonpost.com/video-games/2022/02/19/wordle-cheat/

Image source: Wikipedia

KPI Development: When to Use Ratios

By: David Wilsey

Nov 30, 2020 1311 Views 0 Comments FacebookTwitterLinkedInGoogle Plus

Let’s say you are managing a large organization and your HR director comes to you with bad news. The total number of employees leaving the company has skyrocketed. They show you the chart to the right to indicate that the raw number of employees leaving last year was below 20 and this year it is 50. Should you panic?

The answer is that it depends. What if I told you that this year the company acquired a competitor and that the total number of employees had expanded from just over 500 to over 2200 (see the chart below), and so the turnover percentage (# leaving / total # of employees) had actually decreased slightly (see Turnover ratio). Would that make you panic? Probably not.

The point is that raw counts can be misleading if you are not careful about the comparisons you are making relative to the underlying population. You might have seen debates in the news about how political leaders or writers have highlighted raw COVID 19 counts to support an argument when a per capita ratio suggests a different conclusion. It’s not that raw counts aren’t sometimes useful, but it is important to be aware of what you are comparing and how the overall population might impact interpretation. We recommend that any measures that could be misinterpreted be expressed as a ratio. Common examples of ratios include:

  • Percent completion
  • Coverage, fraction of the total possible
  • Error or defect rate
  • Per capita
  • Efficiency = Output / Input
  • Productivity = Output / Cost (or Output / work hour)

Beginners in the KPI space often make the mistake of developing raw count measures where ratios would be more meaningful.

One type of ratio you do NOT see on this list is percent increase. Many of our clients mistakenly think that if they create a ratio comparing the current period with the previous period that they have solved this problem. While a percent increase is technically a ratio, creating a ratio using the 2017 raw count and the 2016 raw count is still just as misleading, as more than 150% increase in turnover from year to year is still unnecessarily alarming. Of course, you could choose to track a percent increase of the ratio, but that would be addressing a different question, one that I will address in an upcoming blog.

To learn more about ratios and other KPI development issues, please see our KPI Professional Certification program.

9 Things We Wish We Had Known About KPIs

Recorded October 30, 2018 - Replay Available

Oct 30, 2018 4010 Views 0 Comments FacebookTwitterLinkedInGoogle Plus

Recorded October 30, 2018 - Replay Available Below

Isaac Newton famously said: "If I have seen further it is by standing on the shoulders of Giants” which means it is easier to discover truth when you can build upon previous discoveries.

BSI's COO David Wilsey and Corporater's former Chief Strategy Officer & VP, Gail S. Perry will share the 9 things they wish they had understood about KPIs from the very beginning, instead of learning them thru mistakes, hard experience, and untangling KPI messes at numerous clients. They have a combined 50 years of performance management experience and they have the battle scars to show for it. 

While neither Gail nor David claim to be giant or genius, the 9 lessons they share will jump start your understanding of KPIs and help you leapfrog over similar missteps. 

Sign up for our newsletter to get the latest information about our next webinar! 


Get our tips straight to your inbox, and 
become more strategy focused