measurement systems
Oh, Now I See!
“I was seldom able to see an opportunity until it had ceased to be one.” Mark Twain.
“Oh, now I see!” It’s a phrase we use so often to convey that we understand, or get it. We use sight as a metaphor for understanding all the time. The word lens is used to mean a channel through which something can be seen or understood. “Mary sees the world through rose colored lenses,” (an optimist, or naive). We are creatures of pattern recognition and our conjuring process requires imagery to put things in place, or to make sense of what we experience or think. We will typically apply what we know (our storehouse of imagery) to what we see and work hard to make sense of things.
Lenses matter and the choice of lenses have interesting effects on what follows. We believe that lenses allow us to see better, and that is true, but for a very limited and specific range of stuff. The lens is helpful in that it blocks out an infinite number of things we could see or consider so that we get clarity and detail on what the lens puts into focus for us. I’ve looked through telescopes and microscopes, sunglasses and readers, wide angle lenses and telephoto ,,,, all bringing into focus different stuff and making me oblivious to everything else around me. If driving fast, I do no longer see what was in front of me seconds before. Nor should I, be looking anywhere but where it’s critical when driving. Texting while driving is illegal in some states, thank goodness.
In our enterprises, we make choices about lenses all the time. We don’t call them lenses, even though they affect what we see and subsequently interpret. If our lenses are the wrong ones, then we’ll just have to deal with interpreting what we see and worry about what we don’t see. If our lenses cover too much to absorb at once, because we’re driving the business so fast, then we’ll just to trust our luck that we did not miss an important turn-off or on-ramp. If our business roadways are all smooth and devoid of danger or speed traps, then it doesn’t matter so much. If the scenery doesn’t change or we’re not trying to take our enterprise anywhere new, then all is good.
Among our most important lenses are what we measure, how well we measure, how often we measure, and what we do with what we measure. They are important if we do actually do something of value with what we measure in time to make a difference. It does us little good to find out that we missed a turn-off two weeks or a month ago, unless we’re pretty good at u-turns and restarts, except when opportunities don’t wait around for us to u-turn. Also, who decides what to measure, or who interprets what we measure, or who decides what to do with what we measure, or who reports what we measure, or doesn’t report what we measure is likely to matter a whole bunch too.
When we go for our annual physical, all the same measurement stuff applies, and we surely hope the doctor and the lab get it right. I lost a cousin to cancer this weekend because a doctor and a lab got the measurements wrong years ago when they could have done something in time to save Bob’s life.
So, when was the last time you checked the lenses you use at work, home, or play?
“There are three classes of people; those who see, those who see when they are shown, and those who do not see.” Leonardo da Vinci.
It’s Your Call
A blown call costs a pitcher a perfect game. This week, it really happened and everybody felt terrible, apologies ensued and the guilty umpire felt genuine remorse and accepted full responsibility for the failed measurement. A poor measurement did not change the perfection of the real performance, a better gage, instant replay validated that, but rather the record of what happened. Those that missed this story and are evaluating the statistics of pitching performance will only have the historic data to evaluate, data that is a false witness of events. Imagine the effects of all the poor measurements in one year of major sports events. Do they change important outcomes? Do they steer rewards or punishments? How about all the stuff that goes on with gamblers in or out of Las Vegas?
Bad measurement in sports evokes big emotions, outrage, indignation and a score of aftereffects that include bragging rights. Does bad measurement in our enterprises conjure similar reactions? What are the chances that we are making decisions as a result of poor measurement, the wrong lens, an obstacle in the way, poor technology, get the picture? If so, the issue is ubiquitous. In over two decades of helping organizations with performance gaps, poor measurements have always been at play, sometimes with disastrous consequences.
The issue is not a simple one. For example:
• Do we use the data that we have and try to conjure meaning from it? Or do we start with what we want to know and then measure accordingly?
• Are we sure that the movement in the data is representative of what is actually happening within the process?
• Do different individuals or functions measure differently? Would they come up with the same value when measuring the same process?
• Does the data just not make any sense?
• How about our “calls” on what we evaluate? Do two managers reach the same conclusion about someone’s performance? If not, who is right? What are the consequences to the individual?
• Do we introduce our own bias into the measurement and evaluation?
• Do we have folks who are easier graders and those that are more demanding? Do they evoke similar or different performance?
• How much of our decision process rely on a subjective call (an opinion) versus an objective measurement (an actual number)? Do we know how often our calls are wrong?
• Do compliance requirements change how we measure performance?
• What happens when lab results are wrong? What if wrong results bring really bad news or they mask the bad news and bring good news?
• Are we ever surprised by events that would have been very visible had we measured differently?
• Does a part of the organization hide or hoard data?
• Do our customers measure our deliverables and call about problems that we should have prevented them from experiencing? What did our data say?
• Do we have our vision checked from time to time? Why is that?
• Do we ever catch how some advertisers deceive with clever use of statistics? How about in our enterprises?
• Is it safer for ourselves to call someone “safe” rather than “out” when we’re not sure, just in case? Consequences are often more severe in one direction versus the other.
• Have we ever spent a lot of money and resources on a decision made with poor data?
So, how’s our data today?
