Decision Systems Yes .. but a Data Skeptic

Research has suggested that we overestimate our accuracy by about 20% when left to our own devices when dealing with a complex decision where we do not have deep experience or insight.1  Thankfully, access and capacity to increasing volumes of data is helping to overcome this decision bias.

In a recent survey by Cap Gemini is was determined that “participants estimate that, for processes where Big Data analytics has been applied, on average, they have seen a 26% improvement in performance over the past three years, and they expect it will improve by 41% over the next three.

This is good news for the data industry…but I am a data skeptic.  What does that mean? It means that I don’t trust data, or more specifically I do not trust data alone.

Often, data is positioned as the antidote to an overconfidence bias, but the truth is that data itself can be one of the drivers of overconfidence as Jim Harris points out in his post “Big Data and the Treadmill of Overconfidence”

In an interesting twist, a McKinsey report highlights the fact that for leaders who need to get things done, “Holding a somewhat exaggerated level of self-confidence isn’t a dangerous bias; it often helps to stimulate higher performance.” 2

In other words, overconfidence could be just the thing we need when making decisions!

decision

In short, we can’t rely on data alone to drive a good decision

To make good decisions we need a lot more than good data and good analytics, and we certainly can’t rely exclusively on our own over confident intuition.  We need good Decision Systems, and by Decision Systems, I mean more than the currently popular logic based Decision Models, If A, then B that are algorithm driven and real time.

I mean holistic, systemic decision models that include people, and process, biases and other slippery constraints, and which are the underpinnings of the majority of real decisions made today, in the past and probably for the foreseeable future.  Decision Systems that take into account psychology and organizational behavior

In our next post, we will tackle the key components of a good repeatable, self learning decision system, and how data analytics provides a measured improvement for our clients.

What has been successful with Decision Systems you have used? Share your thoughts @firststepsinc


 

References

  1. Lichtenstein, Sarah; Fischhoff, Baruch; Phillips, Lawrence D. (1982). “Calibration of probabilities: The state of the art to 1980”. In Kahneman, Daniel; Slovic, Paul; Tversky, Amos. Judgment Under Uncertainty: Heuristics and Biases. Cambridge University Press. pp. 306–334. ISBN 978-0-521-28414-1. ↩︎
  2. http://www.mckinsey.com/business-functions/strategy-and-corporate-finance/our-insights/the-benefits-and-limits-of-decision-models# ↩︎

Strategy and Insights on Decisions from the UP24

I love to track personal activity data. I track my weight, lean mass, fat mass, heart rate, calorie consumption, calorie burn, running pace, running distance, cadence, cycling speed, cycling distance, cycling cadence, power outage, elevation gain, cumulative fatigue, and on and on…

I track all this (and other stuff I will not bore you with) partly because it serves a practical purpose, and partly because I am someone who loves the intellectual challenge of building functional workflow for capturing and using that data to drive decisions and continuous improvement.

This Christmas the hot item under the tree for me was the UP24 by Jawbone. This will be my second Jawbone UP, and I have about a year of experience using the UP, along with other devices, to track activity.

Based on this experience, I have stumbled upon some insight from my personal analysis and workflow that I think is equally relevant for data driven business.

img

80% of time is spent on data capture

If you have spent any amount of time with a personal tracking device, you will quickly realize that you spend…well…most of your time focused on data capture. The majority of my time spent interacting with my Jawbone App is to enter food information, or adjust sleep data, or shift the app into different measuring modes.

I understand that some people will have a fitness band which they put on and never touch, but I would argue that to get real value out of a fitness band you need to be quite disciplined about ensuring that you are entering the relevant data consistently.

I believe that the same applies to business. If you are serious about making data driven decisions, you need to invest some time and money into data capture. It is not good enough to just purchase some hardware and turn in on.

A great example is Google Analytics. Many businesses who build a website will install Google Analytics but very few will take the time to customize Google Analytics to capture custom data that is relevant to their KPI’s or goals. Using custom data capture in Google Analytics or a service like StatHat is real data capture.

Strategy Takeaway: Take time to identify your investment of time and money in data capture. If you are “collecting data” but not investing any resources to do so, you are probably not capturing the data that matters.

Not all the data is relevant

There are many bits of captured or estimated data from my UP24 that are simply not relevant, or more accurately, are not relevant to Me at a given moment in time. My UP24 app will inform me of the the estimated amount of sodium and fiber in my diet, which is good information and likely is relevant for some who are trying to keep to a low sodium high fiber diet, but is not critical for me at the moment.

Or, my UP24 will keep me abreast of exactly how many steps I aquired throughout the day in a handy “Timeline”. Again, interesting but not really important for me. I know I am active while walking to work, inactive when I sit in front of my computer and active again when I commute home.

Studying this information, just because it is available, is not the best use of time. I personally care about my aggregate consumption and activity per day, and my aggregate sleep levels. If I can keep those personal KPI’s in line for six months consistently, then maybe I can move on to more nuanced refinement, like nutritional balance.

Too many businesses we work with are getting drawn into highly nuanced data insights, when they are not measuring and managing the basics.

Strategy Takeaway: Master the data capture and management of your macro measurements first, then move on to more detailed data measures for refinement. The detailed analytics are not relevant until you have mastered the macro analytics.

Data is personal

Hardly a day goes by when someone does not ask me about my UP24 band. As popular as these fitness trackers are, they still seem to fascinate and people want to understand why and how you are using the device. Inevitably I am drawn into explaining all the great data I am tracking with the device and how I am using this data to improve my lifestyle and maintain some balance.

Despite my best efforts to make it interesting, the poor individual who asks this question is usually left looking pretty bored after 5 minutes. They are probably interested in the abilities of the band, but are not interested in my data. They are day dreaming about how the device would be useful to them, and the data they would like to capture and utilize.

The data that we capture and assess is utimately useful to address personal needs and pain points. Each one of us is looking for some comparison in data sets that is relevant to us. Our contribution to a successful quarter, our performance vis-a-vis peers, our success in driving growth or improving throughput.

Strategy Takeaway: Individuals will collect, manage and assess data to the extend that it is relevant to them personally. Design your data collection and assessment to be personal and it will drive more engagement in the collection and processing of data for decision making.

We can only measure part of what matters

Despite the many cool things that my new UP24 band can measure, it still cannot measure when my daughter is sick, or I injure my back, or I meet up with a friend I have not seen in 15 years. Life is significantly more dynamic than any fitness device can account for.

Whether I should stay up a bit later, or have that extra beer, or sit on the couch all day should be driven by more than the difference between calories consumed and expended on a given day.

I find the UP24 is incredibily helpful to make me aware of the basic trends in my personal behavior, but this simply provides a starting point in a personal decision.

Strategy Takeaway: Allow your data assessment to start a decision process, but allow experience, context and many other inmeasurable insights to refine and finish the decision.

Stella IPad App Review – Mobile System Dynamics

As someone who actively uses system dynamics to tackle complex issues for clients using the pricier iThink software from isee systems, I was quite intrigued when isee released a system dynamics app for the iPad

I should note right off that the app as of this review is $40, so pricey by iPad app standards, but for anyone who owns the full desktop software this will seem like a steal. In fact, I believe one of isee’s objectives with the app is to democratize the exploration and use of systems dynamics as a tool for addressing complex problems, and I applaud them for this.

When you first open the app and create a new model, you are provided a clean canvas to work with on which you can tap to add stocks, flows and converters. As you will see in the image below, I have added a stock for “Visitors” and simple flows of “New Visitors” and “Abandoning Visitors”.

Read more…

Patient Decisions

Making Decisions vs Finding Answers

Much has been written about the importance of being patient when making decisions. I was reminded of this recently after reading a popular circulating article by Jennifer Roberts on how much can be learned by taking the time to contemplate and observe.

With the current emergence of big data and real time information placing a highlight on real time decision making it is more important than ever to get the balance right between how much time is spent getting context and collecting information before using information to make decisions.

Read more…

Shiny and New Decisions

Nothing holds more potential for both benefit and loss than the shiny new tool or application. Whether it is software, hardware or that great new circular saw you are craving at Home Depot, there is moment in time when you need to decide if the Shiny and New thing in front of you is a toy or a real asset.

While we might be inclined to think this is a rather straight forward ROI assessment of benefit vs cost, the history of poorly spent dollars on large scale projects as well as failed corporate aquisitions by very smart people suggests that this is a non-trivial issue. Procurement decisions are not just about dollars and cents, but influenced by existing technology or skill sets , the theatre of internal politics and a variety of other variables.

I would propose that one of the challenges in procurement is that a proper procurement decision is a complex decision, but is often treated as a rather uncomplex decision based on a few variables that are relevent within a narrow context.

Read more…

Modeling with a Limited Data Set

Problem:

How do we model when we lack data for assumptions?

We have entered the age of big data. It is impossible to search predictive analytics and not get bombarded with news on Big Data. I think big data as proposed is exciting, and is ushering in a whole new category of analytics, but I find the importance of earnest predictive modeling may get lost in the tidal wave of exploratory approaches to big data.

Big Data will eventually become table stakes for all predictive models, but even then companies will find their biggest data may still be missing some of the data needed to complete predictive models.

One of the questions that will continue to nag business managers is what role data plays in making decisions and building predictive models? I would like to propose some considerations when tackling this question.

Read more…

The Problem with Me – Personal Decision Bias

The problem with decision analysis is ME.

Despite the mountain of data suggesting that one option is highly desirable I have a strong tendency to choose the option that most appeals to me. At the heart of the question of how we use data to drive and inform good decisions is the question of why is it that we as people ignored data when making decisions?

Here are four reasons why people (including myself) ignore data

  • People have a personal bias
  • People do not understand the data
  • People know data is only part of the decison
  • Finally, People simply do not care


What you ask me? Why would anyone choose to pursue something that they know is a bad decision simply because they don’t care? Why would they not care? Let me try to put this in the context of everyday decisions that we all typically make.

Read more…

An End to Aimless Images – Visuals that Matter

I am admittedly, a relative newcomer to the world of blogging. By trade I am a modeler and an analyst who has entered into the world of blogging largely out of a desire to connect with and converse with those who are interested in and could benefit from thoughtful models and data analysis.

As I started to blog it seemed as if the images used in blog posts were a critical component in what would attract people to a particular post. Women with thoughtful, quizzical looks. Airbrushed, high density images that acted as a sort of irresistible eye candy.

I registered with stock photography sites and set up my accounts on Flickr and Instagram.

With each blog post I would spend 50% of the time writing the content and 50% of the time looking for the right visual image.

Research tells us that people like pictures. Pictures of people, and more specifically pictures of women. If I get more women on my site, into my blog posts, I will generate more visits more click through’s and more social activity.

The one thing I will not necessarily generate is more business.

Read more…

You Know Your Price is Right When

The pursuit of margin will more often than not come back to revenue. Components of revenue are heavily dependent upon decisions around price. Getting your price right is one of the most important decisions you will make. How do you know if your price is right?

While a common answer to this question might be “it depends”, I would suggest that a workable definition for your ideal price is one that maximizes your long term profitability. This simple definition accounts for both the maximization of margin, as well as the range of pricing strategy available.

Read more…

One Action, Many Decision Makers

Watching the Bruin's recent domination of the Pitsburgh Penguins I have been impressed by the fact that they are operating as a single unit where each team members decision represents the collective direction of the team. Alternatively, when I watch the Penguin's game play it appears more like a group of individuals making independent decisions which are being passed of as group play.

One of the key organizational challenges in strategy is making sure that decisions are made as a group rather than a series of independent decisions becoming a group decision.

Read more…